A profile name describes an envelope
Multi-Instance GPU does not divide a device into arbitrary percentages. Each supported GPU exposes a defined set of GPU instance profiles, and each profile carries an envelope of memory, streaming multiprocessors, cache, copy engines, and sometimes media engines. The legal profiles and the number of instances that fit vary by GPU model. Capacity planning therefore begins with the exact product and supported profile table, not a generic claim such as one seventh of a GPU.
That envelope has several independent ceilings. A model can fit the profile's device memory yet miss its latency target because compute is constrained. An input pipeline can be limited by copy engines. A video workload can select a small compute profile that lacks the required media engine. Record peak allocated memory, steady and burst compute demand, transfer behavior, required engines, and latency or throughput objective before selecting a profile.
Measure the deployed workload shape
Measure the artifact and serving configuration that will actually ship. Include weights after quantization, runtime workspaces, graph captures, framework overhead, KV cache at the intended context length and concurrency, and temporary buffers. Warm the service, exercise the production prompt and output-length distribution, and collect peak memory rather than an idle snapshot. For training, include optimizer state, gradients, activation strategy, batch shape, and checkpoint behavior.
Use a decision rule with headroom: reject any profile if observed peak device memory exceeds 80 percent of its reported capacity during the acceptance workload, or if any required engine is absent. The 80 percent threshold is an illustrative operating policy, not a vendor requirement. Choose a different margin from observed variance and failure cost. Memory fit is only the first gate; p95 latency, successful requests per second, and error rate must also pass at the intended concurrency.
Map demand to legal layouts
Next enumerate layouts the hardware actually supports. NVIDIA's profile tables state the fraction of memory and SMs, hardware units, copy engines, and maximum instance count for each product. Do not infer that two profiles can coexist merely because their memory numbers add up. Placement geometry and profile combinations matter, and management tooling may require a specific sequence or reset to establish a layout.
Build a small compatibility matrix for procurement and scheduling. Its rows are exact GPU products and driver-qualified configurations. Its columns are candidate workload classes. Each cell contains the smallest passing profile, instances per physical GPU under a legal layout, and the benchmark receipt. Mark unsupported or untested combinations explicitly. This prevents a scheduler label such as mig-1g from being mistaken for an interchangeable capacity unit across different GPU models.
| Workload observation | Profile gate | Validation |
|---|---|---|
| Peak memory | Peak stays below policy threshold | Long soak with worst accepted context and concurrency |
| Compute demand | Latency and throughput SLO pass | Load test at target arrival pattern |
| Transfers | Copy path has acceptable headroom | Measure host-to-device and device-to-host phases |
| Media use | Required engine exists in profile | Exercise decode or encode path |
| Layout | Combination is documented as supported | Create, destroy, and recreate on target hardware |
Worked example: capacity after fit testing
Assume an illustrative inference service has a measured peak allocation of 7.2 GB under its accepted traffic shape. A candidate profile reports 10 GB. The planning ratio is 7.2 divided by 10, or 72 percent, so it passes the illustrative 80 percent memory rule. During a sustained test it delivers 18 successful requests per second at the p95 target. Demand is 90 successful requests per second, so the arithmetic minimum is 90 divided by 18, or 5 instances.
Now apply resilience and utilization limits. If policy caps planned utilization at 70 percent, required instances are ceiling of 5 divided by 0.70, which is 8. Assume the exact target GPU's documented legal layout permits seven of the tested profile per physical GPU. Steady demand then requires ceiling of 8 divided by 7, or 2 GPUs. This count does not add a separate spare-GPU failure allowance; add that explicitly if the service must survive loss of a physical GPU. These numbers are illustrative, not performance claims. Actual throughput depends on model, software, input shape, profile, GPU, clocks, and contention, so only the measured receipt is reusable.
Validate on the supported stack
Pin the GPU model, MIG profile, firmware and driver branch, container digest, CUDA runtime, framework, serving engine, model digest, tokenizer digest, and benchmark dataset. Create the intended MIG layout, confirm device enumeration, and run correctness before performance. Then test cold load, warm steady state, burst concurrency, cancellation, out-of-memory behavior, and instance restart. A result on a full GPU does not qualify a MIG profile, and a result on one GPU product does not qualify another.
Measure isolation-sensitive behavior too. Run neighboring instances at their accepted peak and compare the candidate workload with its solo baseline. NVIDIA documents dedicated GPU compute and memory resources for MIG instances, including isolated memory paths and cache resources, but host CPU, storage, PCIe paths, power, and network can still influence service performance. MIG partitioning alone is not a complete tenant-security boundary, so access control, process and container isolation, and data-handling controls still need separate validation. Define a measurable neighbor rule, such as p95 latency degradation no greater than 5 percent in the co-load test. That percentage is an illustrative policy and must be chosen from the service objective.
Count layout friction and stranded capacity
The smallest profile that passes is not automatically the cheapest layout. Mixed demand may strand slices because the remaining geometry cannot form another required profile. Forecast demand by workload class and calculate feasible layouts for each time window. Compare useful accepted throughput per physical GPU, not merely the number of instances. Include reconfiguration time and the operational impact of draining workloads when changing layouts.
MIG also does not make every workload divisible. Large models may require a full GPU or multi-GPU parallelism, while irregular jobs can make static partitions inefficient. Keep an escape pool for profiles that do not fit the dominant layout, and revisit the matrix when model versions or context policies change. Include demand percentiles, not only daily averages, because simultaneous peaks can invalidate an otherwise efficient mix. Reserve maintenance capacity for draining a physical GPU without evicting every service tier at once. Compare the chosen layout with whole-GPU placement using successful work per device and operational complexity. ClusterBid is most useful here when the request specifies the exact qualified GPU, profile layout, and measured throughput rather than a generic GPU count. The limitation is structural: a profile is a fixed resource envelope. Capacity becomes predictable only after the workload shape is bounded and tested inside it.