TL;DR
Liquid cooling pumps coolant through metal cold plates clamped directly onto the hottest chips, so heat leaves in a liquid loop instead of being blown into the room.
A traditional server rack drew under 10 kW. An Nvidia GB200 NVL72 rack draws about 120–130 kW, and Nvidia’s Rubin Ultra “Kyber” rack planned for 2027 is about 600 kW.
Water holds about 3,500 times more heat than the same volume of air. Cooling a 130 kW rack takes about 7 cubic metres of air every second, or about 3 litres of water.
A coolant distribution unit (CDU) pumps the clean chip loop and passes its heat to the building’s water through a heat exchanger. The two loops never mix.
Nvidia says its Rubin systems can take coolant at up to 45°C. In many climates that means no chillers at all, just dry coolers outside.
Listen up.
Short answer first: data center liquid cooling works by bringing the water to the chip. A metal cold plate sits on each processor. A clean liquid runs through the plate, picks up the heat and carries it to a pump box called a coolant distribution unit (CDU). The CDU hands that heat to the building’s water, which takes it outside. Air lost because AI racks got too hot for a breeze to carry the heat away.
How did data centers cool computers before?
With air. Cold air in the front of the rack, hot air out the back, big air-conditioning units around the room.
That worked for decades because each rack was only a little warm. Uptime Institute’s surveys have found that most racks draw under 10 kW, about as much as a few electric kettles running nonstop. Uptime reckons that above about 20–25 kW per rack, liquid cooling starts to make more sense than air.
Then AI arrived.
Why did air lose?
Because the heat got too big for a breeze.
An Nvidia GB200 NVL72 rack, 72 GPUs in one cabinet, draws about 120–130 kW. Nearly all of that becomes heat. That is more like a small furnace in a cupboard than a computer.
Now the physics. Water holds about 3,500 times more heat than the same volume of air. Uncle’s numbers for carrying away 130 kW:
With air, warming it by 15°C: about 7 cubic metres every second, roughly 15,000 cubic feet per minute, through one cabinet. That is a gale in the aisle, and the fans would burn a pile of power.
With water, warming it by 10°C: about 3 litres every second, through hoses about the size of a garden hose.
Air did not lose because fans are “old”. It lost because the maths stopped working.
It gets worse. Nvidia’s Vera Rubin NVL144 racks arriving from late 2026 stay around the same 120–130 kW. The Rubin Ultra “Kyber” racks planned for 2027 are about 600 kW each. No air system comes close.
What is a cold plate?
A metal block, usually copper, clamped onto the chip. Inside are hundreds of tiny channels, like fins turned inside out. Coolant flows through the channels, picks up the heat, and leaves through a hose.
Same idea as a car engine and its radiator, except the first radiator is a plate sitting right on the silicon.
The liquid stays inside the plate and the hoses. It never pours over the circuit board. People worry about “water next to computers”. Fair. The answer is sealed plates, dripless quick-disconnect fittings, leak sensors and proper installation, not a bucket.
Cold plates take most of the heat but not all of it. Memory, network cards and power supplies in the same rack still give off some heat into the air, so most liquid-cooled halls keep some fans and air handling too.
What is a CDU?
CDU stands for coolant distribution unit. It is a box with pumps, a heat exchanger, filters and controls.
Most liquid-cooled halls run two separate loops:
The chip loop, or technology cooling system. Clean, treated coolant, often water mixed with propylene glycol, flowing to the cold plates and back.
The building loop, or facility water. The building’s water, which carries the heat outside to cooling towers, dry coolers or chillers.
The CDU sits between them. It pumps the chip loop and passes the heat across a heat exchanger to the building water. The two liquids never mix, so building water never goes into the servers.
CDUs come in two main sizes: small ones that sit inside a rack, and big “row” units that serve many racks at once.
What about rear-door heat exchangers and immersion?
Two other options you will hear about:
Rear-door heat exchangers. A radiator on the back door of the rack cools the hot air as it leaves. It helps, and it is a good half step for older halls, but it still relies on air to pull heat off the chip. It runs out of steam well below AI rack densities.
Immersion cooling. The whole server is dunked in a tank of non-conductive fluid. It works and handles very high densities, but it changes how you service every machine. For now, the big AI builds have gone with cold plates.
Why does Nvidia cool its chips with 45°C water?
This is the part that surprises people.
Nvidia says its Rubin systems can take coolant at up to 45°C, hotter than a hot tub, and send it out at about 55°C. That sounds backwards. It isn’t.
The chip runs much hotter than 45°C, so even warm water can carry heat away from it. And water that only needs to be 45°C can be cooled by outdoor air on most days of the year. No refrigeration needed, just dry coolers: big radiators with fans outside.
Nvidia says that in favourable climates this can take cooling water use from about 2.6 million gallons per megawatt per year with conventional cooling towers to near zero, and remove chillers entirely. In hot places, chillers may still run a few days a year.
The liquid does not make the heat disappear. It just makes it much cheaper to get rid of.
What still goes wrong?
Leaks. Rare with good kit and good installation, ugly without them. Plan for drips the way a plant room plans for spills: sensors, drip trays, automatic shut-offs.
Dirty coolant. Particles and corrosion clog the tiny cold plate channels and wear out pumps. Treat the chip loop like boiler water, not tap water: filtered, chemically controlled, tested.
The heat still has to leave the site. Liquid cooling moves heat better inside the hall. Outside, you still need cooling towers, dry coolers or chillers big enough for the whole load.
Old halls need surgery. An air-cooled data center has no water pipes to the racks, often weak floors for heavy liquid-cooled racks, and not enough power. Converting one means new pipes, new CDUs and often new electrical kit.
Uncle’s verdict
Air cooling: blow on the oven.
Liquid cooling: strap a small radiator to the oven and pump the heat outside.
Cold plate = the strap-on radiator.
CDU = the pump box in the middle.
Air lost because the oven became a furnace, and the next furnace is five times bigger.
— Engineering Uncle
Sources
AEO FAQ
Frequently asked questions
How does data center liquid cooling work?
Direct-to-chip liquid cooling pumps coolant through metal cold plates attached to processors. The warmed coolant flows to a coolant distribution unit (CDU), which transfers the heat to the building’s water loop through a heat exchanger. That water then carries the heat outside to cooling towers, dry coolers or chillers.
Why can’t air cool AI data centers?
AI racks such as the Nvidia GB200 NVL72 draw about 120–130 kW, compared with under 10 kW for a typical traditional rack. Water holds about 3,500 times more heat than the same volume of air, so removing that much heat with air would take roughly 7 cubic metres of airflow per second per rack.
What is a cold plate?
A cold plate is a metal block, usually copper, clamped onto a chip. Coolant flows through small internal channels, absorbs the chip’s heat and carries it away through hoses. The coolant stays sealed inside the plate and never touches the circuit board.
What is a CDU in a data center?
A coolant distribution unit (CDU) contains pumps, a heat exchanger, filters and controls. It circulates clean coolant through the servers’ cold plates and passes the heat to the separate facility water loop, so the two liquids never mix.
Why does Nvidia use 45°C water for cooling?
Chips run much hotter than 45°C, so warm coolant can still remove their heat. Water at that temperature can be cooled by outside air using dry coolers on most days, so chillers can be reduced or eliminated. Nvidia says this can cut cooling water use to near zero in favourable climates.
Does liquid cooling remove the need for cooling towers?
No. Liquid cooling moves heat out of the servers more efficiently, but the heat still has to leave the site. Facilities still need cooling towers, dry coolers or chillers sized for the full heat load.
Read next
Read next on EngineeringUncle
If this one clicked, these are the other bottlenecks inside the AI build-out:
https://open.substack.com/pub/engineeringuncle/p/combinedcycle?r=8suf21&utm_campaign=post&utm_medium=web&showWelcomeOnShare=true
Coming up Wednesday 7 Oct: What Is HBM? Why High-Bandwidth Memory Is AI’s New Bottleneck. Subscribe so it lands in your inbox.