The debate about AI infrastructure is dominated by two scarcities: GPUs and electricity. Both matter. But inside the building, a third constraint now decides how much compute a site can actually run, and it is the oldest problem in electronics – getting the heat away from the processor and out of the building.
Every watt a processor consumes becomes a watt of heat that must be removed, continuously, for the life of the facility. Compute can be bought. Heat rejection has to be engineered.
Let’s start with the scale of the change. A decade ago, a rack drawing 10kW was considered dense. Today’s leading AI accelerators – ie, GPUs – dissipate between roughly 70W and more than 1kW per device according to their public datasheets, and rack-scale systems have moved accordingly. A single rack with 72 accelerators is now rated at about 120kW.
The International Energy Agency estimates that datacentres consumed 415TWh of electricity in 2024 and projects, in its base case, roughly 945TWh by 2030. Nearly every watt of that ends up as heat that someone has to move out of a building.
Air has run out of headroom
Air served the industry well for decades, but it is a poor coolant, and physics does not negotiate. Per unit volume, water carries roughly 3,500x more heat than air for the same temperature rise. At 120kW per rack, cooling with air would mean moving implausible volumes of it. Fan power climbs, noise climbs, and the geometry of fins and heat sinks simply runs out of surface. Air cooling does not fail gradually. Designs cross a threshold beyond which no practical fan or heat sink can hold semiconductor temperatures within their limits, and the newest rack-scale systems sit far beyond that threshold.
That is why the industry’s centre of gravity has shifted to liquid – cold plates mounted directly on the devices that dissipate the most power, coolant distribution units at the rack or row level, and a facility loop that carries heat outside. The question for operators is no longer whether liquid cooling will arrive in the AI hall, but how much of the heat it can capture and at what temperature.
Most facilities will not flip overnight. Existing halls built for air are being retrofitted with rear-door heat exchangers and partial liquid loops, and each step helps – but each also confirms the same physics. Retrofitting liquid into a live facility is slow, capital-intensive work, which is why the new generation of AI factories is designed cooling-first, and why older sites face a hard ceiling on the compute they can ever host, whatever their power supply.
Warm water changes the economics
The counterintuitive answer is to run the coolant warmer, not colder. Published reference designs for the next generation of AI factory platforms are now 100% liquid-cooled, with coolant supplied at temperatures up to 45°C. At that level, heat can be rejected to the outside air through dry coolers for much of the year in many climates, without mechanical chillers – and, in the published design figures, with on-site water consumption reduced to near zero where conditions allow.
Individual operators will make their own choices, but the direction of travel is clear. Coolant temperature has become an efficiency variable in its own right, and the cooling loop is now part of the product architecture rather than the building services.
Cooling also decides geography. A facility in a cool, dry climate can reject heat cheaply for much of the year. One in a hot, humid region, with ambient temperatures of 30°C to 35°C and no free-cooling hours to harvest, pays for every degree of approach temperature in energy and capital. Grid capacity, water availability and heat rejection now sit in the same site-selection spreadsheet, and increasingly they are the deciding rows.
Design for heat first
The cost of getting this wrong is not only energy. A system starved of cooling protects itself by throttling, and a throttled accelerator is capacity that has been bought, installed and powered but cannot be used. Cooling quality shows up directly in useful output per watt – the measure by which AI factories are increasingly judged.
Three habits separate the facilities that scale from the ones that stall.
First, treat cooling as a design input at the silicon, server and rack level – not as a facilities afterthought to be solved once compute is chosen.
Second, measure what matters – device junction temperatures and coolant temperature rise, not just the air temperature of the room. By the time the room is warm, the silicon has been throttling for some time.
Third, design for end-of-life rather than day one. Fouling, pump wear and the next, denser generation of racks all consume margin, and a cooling design with no margin is a capacity limit waiting to be discovered.
The industry spent decades treating cooling as plumbing – necessary, unglamorous, and somebody else’s problem. In the AI factory it is none of those things. It is the constraint that determines how much intelligence a building can produce. The organisations that design for heat first will deploy faster than those still designing around it.
Ajay Sekar is a thermal engineer at Nvidia
