Data Center and Critical Infrastructure

How liquid cooling helps keep AI flowing

24 June 2024
Source: PUNRH/CC BY-SA 4.0

Editor's note: This article is part of our Engineering Behind AI, presented by Boyd, theme week.

Artificial Intelligence (AI) data centers are vital for processing large amounts of data and supporting AI algorithms. Equipped with high-performance servers, these facilities handle a variety of tasks including natural language processing, image recognition and big data analytics. As the demand for AI services increases, the need for efficient data center operations becomes essential.

Historically, AI data centers have used air cooling systems to manage heat from servers. Indeed, this remains the de facto technique and every good thermal management program begins with passive cooling, through materials like baffles and thermally conductive materials,

However, as computing demands have grown, these systems sometimes need supplementary cooling systems. To address these issues, there has been a shift toward liquid cooling solutions, which offer better heat dissipation by applying coolant directly to heat-generating components or immersing them in a conductive liquid. This change aims to improve server performance through better temperature control and reduce both energy use and environmental impact.

The need for advanced cooling

AI data centers host an array of high-performance computing systems that are central to processing large datasets and executing complex algorithms. These systems, equipped with powerful processors and GPUs, generate significant amounts of heat due to electrical resistance and computational tasks. The heat output is further amplified by the density of the hardware packed into these centers, creating a microenvironment where temperatures can escalate quickly. If not adequately managed, this heat can lead to hardware malfunctions, reduced efficiency, and shorter equipment lifespans, which in turn compromise the reliability and performance of the data center.

Basics of liquid cooling technology

Liquid cooling is a technique that uses a liquid coolant to absorb and dissipate heat from computer components directly. Unlike air, which has limited capacity for heat absorption, liquids can carry away more heat more efficiently. The process involves circulating a coolant — commonly water or a specially formulated fluid — through a closed-loop system that directly contacts the heat-generating components. As the coolant passes over these components, it absorbs the heat and carries it away to a cooling radiator where it is expelled from the system.

Two types of liquid cooling systems are most common in data centers.

Direct-to-chip (D2C)

This system involves attaching cooling blocks directly to the processors or chips that generate the most heat. A coolant is then circulated through these blocks, directly absorbing the heat from the chips before being cycled back to a heat exchanger. D2C cooling is particularly effective for targeting the intense heat output of specific components like CPUs and GPUs.


In immersion cooling, entire servers or components are submerged in a non-conductive liquid that has excellent thermal conductive properties. This method is more radical than D2C as it involves direct contact between the coolant and all components of the hardware, not just the hottest parts. Immersion cooling is highly efficient in dispersing heat and also reduces the need for other forms of cooling infrastructure, such as air conditioning units or extensive ductwork.

Benefits of liquid cooling

Liquid cooling systems enhance overall cooling efficiency, allowing data center operators to push hardware closer to its performance limits without the risk of overheating. Maintaining lower temperatures reduces thermal stress on components, which can increase reliability and extend their lifespan. Furthermore, processors can run at higher speeds for prolonged periods, boosting computational throughput and efficiency.

By employing liquid cooling, data centers can achieve reductions in energy consumption. Liquid cooling systems require less energy than air conditioning units and large-scale fans used in traditional air cooling setups. The improved efficiency translates into lower electricity bills and operational costs, making liquid cooling a cost-effective solution for when passive cooling might not be enough.

The use of liquid cooling allows for a more compact design of data centers. Since liquid cooling systems take up less space than traditional air handlers and ductwork, data centers can optimize their layout by reducing the aisle space needed for air circulation. This space efficiency not only maximizes the use of real estate but also allows for the accommodation of more servers within the same footprint, potentially expanding computing capacity.

Liquid cooling in big tech data centers

Several leading technology firms and data centers have embraced liquid cooling to tackle the heat management challenges associated with AI computations. For instance, Google has been experimenting with liquid cooling for their AI processors, particularly for their tensor processing units which are pivotal in managing large-scale AI operations. Similarly, Microsoft has deployed immersion cooling technologies in some of its Azure data centers to efficiently manage the heat produced by increasingly dense server configurations.

Google reported a substantial reduction in cooling energy consumption, attributing to better sustainability metrics and operational savings. Microsoft noted enhancements in computational throughput and reliability, thanks to the stable operating temperatures provided by their immersion cooling systems.

Navigating the complexities

The implementation of liquid cooling systems presents several technical challenges, including the need for precise system design and the integration of suitable plumbing infrastructure to manage the flow and disposal of coolants. Additionally, regular maintenance is necessary to prevent issues such as blockages or degradation of cooling components, which could compromise system efficiency.

While the long-term benefits of liquid cooling are clear, the initial setup costs can be substantial. Upfront expenses include specialized equipment, installation labor, and potential modifications to existing structures to accommodate new cooling hardware. However, the return on investment can be favorable, as operational savings from reduced energy consumption and lower maintenance costs over time often offset the initial expenses.

One of the inherent risks of liquid cooling is the potential for coolant leaks, which can lead to hardware damage and operational disruptions. This risk necessitates the installation of leak detection systems and regular inspections. Moreover, the need for specialized infrastructure, such as reinforced flooring to support liquid cooling tanks in the case of immersion systems, adds complexity and cost to the deployment.

Innovating for tomorrow's AI data centers

The future of cooling technologies in AI data centers appears promising, with continuous advancements in both the materials used for coolants and the systems that deploy them. Innovations are likely to focus on increasing the efficiency and environmental friendliness of coolants, as well as enhancing the integration of liquid cooling into more conventional data center designs. As AI continues to drive demand for more powerful and efficient data processing, cooling technology will evolve to meet these needs, likely making liquid cooling a standard for future data centers. The ongoing shift towards sustainable practices may also accelerate the adoption of these advanced cooling solutions, underscoring the role of technology in achieving broader environmental goals.

Author byline

Jody Dascalu is a freelance writer in the technology and engineering niche. She studied in Canada and earned a Bachelor of Engineering. As an avid reader, she enjoys researching upcoming technologies and is an expert on a variety of topics.

Powered by CR4, the Engineering Community

Discussion – 0 comments

By posting a comment you confirm that you have read and accept our Posting Rules and Terms of Use.
Engineering Newsletter Signup
Get the GlobalSpec
Stay up to date on:
Features the top stories, latest news, charts, insights and more on the end-to-end electronics value chain.
Weekly Newsletter
Get news, research, and analysis
on the Electronics industry in your
inbox every week - for FREE
Sign up for our FREE eNewsletter