Three power numbers should not be substituted
A rack power review needs three separate quantities. Nameplate demand is the equipment rating or design maximum used for compatibility and worst-case planning. Observed demand is what meters recorded under a defined workload and time window. Planned concurrency is the set of machines and supporting devices expected to run together. Treating any one as a substitute for the other creates false precision.
Nameplate totals are useful for identifying an obviously impossible layout, but they do not predict a particular training run. A seven-day average can hide a five-minute overlap that matters to a facility limit. A GPU utilization graph is also not a rack power meter: host CPUs, memory, fans, network switches, storage, and power conversion remain in the path. Keep the three columns visible in every review.
Start at the facility-confirmed boundary
Ask the facility operator to state the usable limit at the exact boundary being evaluated: outlet, branch circuit, rack PDU, rack, row, or reserved allocation. Request the applicable redundancy state, metering source, averaging interval, alarm threshold, and whether the limit changes during maintenance or a feed failure. These are requests for confirmed operating constraints, not instructions for changing electrical equipment.
NVIDIA's DGX H100 data-center design illustrates why topology matters. Its documented patterns distinguish circuit capacity, peak server demand, redundancy, and the load carried after a source is lost. That vendor example should not be copied into a different site. The facility's one-line design, protective devices, local rules, cooling envelope, and operating policy control the actual answer. A server owner should never infer an allowable load merely from voltage and current labels.
Build a metered rack envelope
Collect time-aligned readings from the facility or intelligent rack PDU at the highest available trustworthy resolution. Include per-feed or per-phase values when the operator exposes them, plus server state, job ID, GPU activity, and major non-GPU loads. Mark missing samples and clock offsets. Use a window that includes startup, data loading, compilation, synchronized training, checkpoint writes, and cooldown rather than selecting only the steady middle.
For each workload class, report the maximum rolling demand at several operational windows, such as one minute and fifteen minutes, plus a high percentile over repeated runs. A peak from a single unverified sample is a diagnostic clue, not a capacity fact. Compare node telemetry with rack metering to explain timing, but use the facility-designated meter for the boundary decision. NVIDIA documents GPU power controls and telemetry, yet GPU readings alone still exclude the rest of the rack.
Work an illustrative concurrency case
Consider a purely illustrative rack for which the facility confirms a 30 kW normal operating ceiling at the rack boundary and asks the tenant to preserve 3 kW of operational margin. Actual limits vary and must come from the facility. Network and storage devices draw an observed 2.5 kW during tests. Four identical GPU servers each show a repeatable 5.8 kW one-minute rack-meter increment during the accepted workload.
The planned four-server overlap is 2.5 kW plus 4 multiplied by 5.8 kW, which equals 25.7 kW. The planning ceiling after margin is 30 minus 3, or 27 kW, leaving 1.3 kW. Five servers would produce an illustrative estimate of 31.5 kW and fail the rule. This arithmetic supports a request for approval; it does not prove electrical safety or cooling capacity.
| Illustrative input | Value | Role in decision |
|---|---|---|
| Facility-confirmed ceiling | 30 kW | Boundary limit |
| Reserved operating margin | 3 kW | Facility or team policy |
| Shared rack load | 2.5 kW | Measured non-server demand |
| Per-server increment | 5.8 kW | Measured accepted workload |
| Four-server estimate | 25.7 kW | Below 27 kW planning ceiling |
Test overlap rather than adding unrelated peaks
Summing the independent maximum of every device can overstate demand because those maxima may occur at different times. Taking only a rack average can understate it because coordinated jobs create simultaneous phases. Construct a concurrency trace by aligning the workloads as the scheduler will actually launch them. If start times are deliberately staggered, verify that the scheduler enforces the stagger after retries and node replacements.
A measurable admission rule can be simple: approve a workload shape only when its observed rolling maximum plus the agreed uncertainty allowance remains below every applicable facility-confirmed limit, including any rack-total, feed, circuit, and phase limits, in both normal and declared degraded-feed states. Do not infer degraded-state compliance by dividing an aggregate rack value evenly among feeds. Require at least three representative repetitions and no unexplained meter gaps around the peak. The repetition count and allowance are local policy choices, not universal electrical standards.
Diagnose a headroom surprise in order
When measured demand exceeds the model, first confirm timestamps and meter units. Second, identify which servers, switches, and storage devices were energized. Third, compare job phases and retry events. Fourth, look for a changed GPU power limit, firmware policy, fan response, CPU workload, or data-path activity. Fifth, ask the facility to validate that the meter and boundary match the agreed limit. Stop increasing concurrency while the difference remains unexplained.
Do not respond by changing breakers, feeds, cords, rack PDUs, or power caps outside normal approved procedures. NVIDIA notes that multiple sources can impose GPU power limits and that the most conservative policy is selected. An apparent cap therefore deserves configuration review, but changing it may alter validated system behavior. Escalate hardware and electrical changes to the equipment vendor and facility operator.
Turn the accepted envelope into scheduling policy
Record allowed workload combinations, rack placement, required stagger, meter boundary, validation date, and the facility approver. Express concurrency in workload classes rather than a raw GPU count. Two jobs using the same number of GPUs may have different host, network, and storage demand. ClusterBid can help match capacity to these measured requirements once the accepted envelope is explicit.
The main limitation is change. New firmware, power limits, model shapes, ambient conditions, or adjacent equipment can invalidate an old trace. Re-run the acceptance case after material changes and review meter trends periodically. Headroom is not unused inventory to be silently consumed. It is an operating constraint with an owner, evidence, and a date.