THE HEAT PROBLEM AT 700W+
NVIDIA H100 SXM draws 700W per GPU, B200 draws 1,000W, and the upcoming B300 is projected at 1,500-1,800W per GPU. At these power densities, a single rack of 72 B300 GPUs would consume 108-130 kW, exceeding the typical 40-50 kW per rack air-cooling limit by 2-3x. The thermal management challenge has become the primary constraint on AI cluster density, not compute performance.
As of mid-2026, approximately 35% of new AI data center deployments use direct-to-chip liquid cooling, 8% use immersion cooling, and the remaining 57% rely on air cooling with raised floor and hot-aisle containment. The choice determines not just operating temperature but also PUE, hardware reliability, and geographic placement flexibility.
DIRECT-TO-CHIP LIQUID COOLING
Direct-to-chip liquid cooling uses cold plates mounted directly on GPU packages, circulating dielectric fluid or water-glycol mixture through a closed loop. A typical DTC system for an H100 SXM uses cold plates rated for 1,200-1,500W per GPU at 35-45 C inlet temperature, with flow rates of 1-2 L/min per cold plate.
For B200 and B300, cold plate capacity must scale to 1,800-2,200W. Current DTC systems from CoolIT, Boyd, and Asetek handle this with larger cold plates, higher flow rates, and lower inlet temperatures. The CDU cost per GPU is approximately $400-800 for H100 and $600-1,200 for B200.
IMMERSION COOLING: FULLY SUBMERGED GPUS
Immersion cooling submerges entire GPU servers in dielectric fluid. Two-phase immersion handles 2,000-3,000W per GPU with PUE of 1.02-1.04 versus 1.15-1.25 for DTC and 1.3-1.5 for air cooling. The CAPEX premium is $1,500-3,000 per GPU for single-phase and $2,500-5,000 for two-phase.
At $0.10/kWh power cost, immersion saves $400-800 per GPU per year versus air cooling from PUE improvement alone. The breakeven is 3-6 years at current power prices, shortening to 1-2 years in high-power-cost regions like Europe.
COOLING COST MODELING FOR AI CLUSTERS
A 1,024-GPU H100 cluster draws approximately 1.2 MW. Air cooling at PUE 1.4 adds 0.48 MW, totaling 1.68 MW. At $0.10/kWh, annual power is $1.47M. DTC cooling at PUE 1.2 saves $210K/year. Immersion at PUE 1.03 saves $324K/year.
For most AI teams planning GPU deployments through 2028, direct-to-chip liquid cooling is the recommended baseline, with immersion reserved for extreme density requirements or high-power-cost regions.