GPU Rack Integration: Power Distribution, Cooling, and Physical Infrastructure in 2026

GPU racks consume power at levels unprecedented in data centre history. A single rack of 8x B200 SXM GPUs in an HGX baseboard draws approximately 16-18 kW for the GPUs alone, plus 3-5 kW for CPUs, networking, and storage -- totalling 20-23 kW per rack. For comparison, a standard data centre rack from 2020 averaged 5-8 kW. AI GPU racks have tripled the power density standard.

At scale, a 1,024-GPU B200 cluster requires approximately 2.5-3 MW of critical IT load, plus 1-2 MW for cooling infrastructure. Total facility power requirement: 4-5 MW. This is the equivalent power consumption of approximately 3,000-4,000 homes. Data centres that were designed for pre-AI workloads simply cannot support this density without major upgrades.

This post covers the power distribution, cooling, and rack integration engineering required to deploy GPU clusters, with specific guidance for the dominant GPU generations at mid-2026.

GPU rack power distribution starts at the facility level. The standard approach for GPU clusters uses 415V three-phase power distribution (rather than the traditional 208V), which reduces current and allows higher power delivery per rack. A 415V feed delivers approximately 60-70 kW per rack with standard busway ratings, sufficient for 2-3 GPU nodes per rack.

Within the rack, power distribution options include: busway (overhead power distribution with tap-off boxes, most flexible for reconfiguration), rack PDU (power distribution unit with C13/C19 outlets, 30-60 kW capacity), and dedicated branch circuits (hard-wired to the rack, highest reliability). Most GPU deployments use a combination: busway for primary distribution and rack PDU for per-node power metering.

The GPU node power supply is typically 3.2-4.0 kW (for an 8-GPU HGX with dual CPUs). Each node requires two or three C19 power feeds. The rack PDU must support 3-phase power with load balancing across phases. Power monitoring at the PDU level is essential for capacity planning and power capping.

GPU cooling requirements have escalated with each generation. The table below shows thermal design power (TDP) and cooling requirements for current GPU generations. The practical implication: air cooling is no longer viable for GPU racks above approximately 15 kW/rack without specialised high-density cooling zones.

Direct liquid cooling (DLC) is the standard for GPU deployments at mid-2026. Approximately 75% of new GPU installations use DLC, compared to 30% in 2024. DLC removes 85-95% of GPU heat through liquid, leaving only 5-15% to be handled by air cooling for other components. The typical DLC loop uses 25-35C water inlet temperature with a 10-15C temperature rise, rejecting heat through facility cooling towers or chillers.

DLC for GPU clusters uses cold plates mounted directly on the GPU and CPU packages, with a liquid coolant (typically a water-glycol mixture or dielectric fluid) circulating through the cold plates to remove heat. The cooled liquid is returned to a coolant distribution unit (CDU) that transfers heat to the facility water loop.

The DLC implementation requires: GPU cold plates (custom-designed for each GPU form factor, SXM cold plates differ from PCIe), flexible coolant hoses with quick-disconnect fittings for GPU servicing, leak detection (optical or capacitive sensors at every connection point), and CDU with capacity matching the rack power (30-100 kW per CDU for a rack or row).

The most common DLC deployment model at mid-2026 uses row-level CDUs serving 4-8 racks each. This balances capital cost with cooling efficiency. The CDU provides temperature control (typically 25-32C liquid supply temperature) and pressure monitoring. Power usage effectiveness (PUE) for DLC GPU deployments averages 1.10-1.20, compared to 1.30-1.45 for air-cooled high-density deployments.

GPU rack layout must accommodate: GPU nodes (4-8 nodes per 42U rack depending on node depth and power distribution), InfiniBand or Ethernet switches (typically 2-4 per rack, 1-2U each), storage nodes (optional, if local NVMe storage is needed), PDU equipment (vertical or horizontal, 2-6U), and CDU or coolant distribution hardware (if rack-level DLC is used).

Cable management is a significant challenge. Each 8-GPU node has: 8x power cables (C19 or C13), 8x InfiniBand cables (one per GPU, 400-800 Gbps each), 2x management network cables, 2x coolant hoses (if DLC), and 2x NVLink cables for inter-baseboard connectivity (in larger clusters). For a 16-node rack, this is over 350 cables. Structured cable management with horizontal and vertical cable managers is essential for serviceability.

The recommended rack layout for a B200 GPU rack: positions 1-4 (18U): 4x GPU nodes (4U each), positions 5-6 (3U): 2x InfiniBand leaf switches, position 7 (2U): management switch, positions 8-10 (6U): PDU equipment, and the remainder: cable management and blanking panels for airflow.

Not all data centres can support GPU clusters. The minimum requirements for a GPU-capable data centre at mid-2026 are: power capacity of 20+ kW per rack (preferably 30-50 kW with headroom for B200/B300 upgrades), liquid cooling compatible (cooling water loops available at each rack location, not just perimeter cooling), 800 Gbps network backbone (multiple fibre paths between GPU rows and the wider network), physical security with multi-factor access control, environmental monitoring per rack (temperature, humidity, leak detection), and sufficient floor loading (2,000-3,000 kg per rack for fully populated GPU nodes).

Due to these requirements, most GPU deployments are concentrated in purpose-built AI data centres. The largest AI data centre campus in mid-2026 (operated by a hyperscaler in Northern Virginia) has 200+ MW of critical IT capacity, designed exclusively for GPU clusters with DLC infrastructure throughout. The lead time for new GPU-capable data centre space is 12-24 months, a significant constraint on GPU cluster expansion.

For organisations that cannot justify a full data centre buildout, colocation providers (Equinix, Digital Realty, CyrusOne) offer GPU-ready cages with pre-installed DLC infrastructure. At mid-2026, approximately 40% of Equinix's new colocation deployments include DLC-ready configurations, up from 10% in 2024.

Power and cooling represent a significant and growing share of GPU total cost of ownership. For an H100 cluster at $0.08/kWh power cost (US average), power and cooling add approximately $0.30-0.40/GPU-hour. For a B200 cluster at the same power cost, the power and cooling add $0.50-0.70/GPU-hour due to higher TDP.

At 5-8 cents per kWh (typical for US), power is 12-18% of total GPU cost. In European markets (15-25 cents/kWh), power is 25-40% of total GPU cost. This geographic power cost differential is a significant driver of GPU cluster location decisions. Many AI companies are locating training clusters in regions with low power costs (US Midwest, Nordics, Quebec) and keeping inference capacity closer to end users.

The 5-year total cost for power and cooling for a 1,024-GPU H100 cluster at $0.08/kWh: approximately $8-12M. For B200: approximately $14-20M. These costs are substantial enough that power efficiency (PUE, GPU TDP, cooling system efficiency) is a primary selection criterion for GPU infrastructure procurement, alongside GPU pricing itself.