For decades, data centers cooled servers the same way: fans pushing chilled air through racks, with hot air extracted and cooled again. AI is changing that. Modern AI accelerators consume and dissipate so much power that air struggles to remove the heat, pushing operators toward liquid cooling.

Why AI racks are so hot

Almost all the electricity a chip uses becomes heat. Traditional server racks drew in the range of a few to around 10–15 kilowatts. Racks packed with the latest AI accelerators can draw many tens of kilowatts, and the industry is designing for well over 100 kilowatts per rack. Moving that much heat with air alone requires enormous airflow, which is noisy, energy-hungry and eventually impractical.

Liquids carry heat far more effectively than air — water can absorb vastly more heat per unit volume — so bringing liquid close to the chips is the natural answer.

The main approaches

Direct-to-chip (cold plate) cooling

Metal cold plates sit directly on the hottest components — GPUs and CPUs. A coolant flows through channels in the plates, absorbs heat and carries it to a coolant distribution unit, where it is transferred to the facility’s water loop. Fans still cool remaining components. This is currently the most common approach for high-density AI systems.

Immersion cooling

Entire servers are submerged in a dielectric fluid that does not conduct electricity:

  • Single-phase immersion: the fluid stays liquid and is pumped through heat exchangers.
  • Two-phase immersion: the fluid boils on hot components, and the vapor condenses on a cooled surface, carrying heat away efficiently.

Immersion can handle very high densities but requires specialized tanks, servers and maintenance procedures.

Rear-door heat exchangers

Liquid-cooled radiators mounted on the back of racks cool exhaust air. They are a transitional option for upgrading existing air-cooled facilities.

Benefits

  • Higher density: more compute in less space.
  • Energy efficiency: less fan power and more efficient heat removal can improve power usage effectiveness (PUE), the ratio of total facility energy to IT energy.
  • Heat reuse: warmer return water can be used for district heating in some locations.
  • Quieter operation.

Trade-offs and challenges

  • Retrofitting air-cooled facilities is costly; many new AI data centers are designed for liquid from the start.
  • Standards and skills: maintenance teams need new procedures, and leaks must be prevented and detected.
  • Water use: closed-loop liquid systems can use little water, but some facility designs reject heat with evaporative cooling towers that consume water. The overall water footprint depends on design and climate.
  • Supply chain: cold plates, pumps and distribution units are in high demand.

Why it matters for AI

Cooling has become a strategic constraint on how fast AI capacity can be built. Chip roadmaps now assume liquid cooling, which makes thermal engineering as important to AI progress as model architecture.

What to watch next

  • Rack power keeps rising. Each new generation of AI systems pushes power density higher, making liquid cooling the default for new AI halls.
  • Standardization. Common designs for cold plates, connectors and coolant distribution will reduce costs and speed up deployment.
  • Heat reuse projects. Data centers supplying heat to buildings and district heating networks.
  • Water reporting. More detailed disclosure of water consumption and cooling methods by operators.
  • Retrofits vs. new builds. How quickly older facilities can be upgraded will influence where AI capacity grows.

Key takeaways

  • AI racks draw far more power than traditional servers, and nearly all of it becomes heat.
  • Liquids remove heat much more efficiently than air.
  • Direct-to-chip cold plates are the most common approach for dense AI systems; immersion handles even higher densities.
  • Liquid cooling can improve energy efficiency and enable heat reuse.
  • Water impact depends on how the facility ultimately rejects heat, not just on using liquid inside racks.

Frequently asked questions

Is liquid cooling dangerous for electronics?

Systems are designed to prevent and detect leaks. Immersion cooling uses non-conductive fluids, so components can be submerged safely.

Does liquid cooling use more water?

Not necessarily. Closed loops circulate the same coolant. Water consumption depends on the facility’s final heat rejection method, such as evaporative towers versus dry coolers.

What is PUE?

Power usage effectiveness: total facility energy divided by the energy used by IT equipment. A value closer to 1.0 means less overhead for cooling and power distribution.