In this discussion, engineers consider the latest cooling strategies for AI data centers, focusing on advanced systems for water management.
HVAC and plumbing insights
- Hybrid air/liquid cooling strategies are becoming the standard for artificial intelligence (AI)-driven data centers as computing workloads push rack densities beyond what traditional air cooling can support.
- Engineers are increasingly prioritizing flexibility, free cooling and advanced water management systems to improve efficiency and adapt to rapidly evolving data center load demands.
Respondents:
- Brook Gummere, PE, FPE, ATD, Colorado BES Market Sector Leader, HDR, Denver
- Bill Kosik, PE, CEM, LEED AP, Mission Critical Sector Leader, HED, Chicago
- Daniel Noto, PE, LEED AP, Southeast Market Leader, Fitzemeyer & Tocci Associates Inc., Alpharetta, Georgia
- Ken Urbanek, PE, LEED AP, ASHRAE HBDP, ATD, Client Executive and Senior Principal, IMEG, Denver
What unique cooling systems have you specified into data center projects? Describe a difficult climate in which you designed a heating, ventilation and air conditioning (HVAC) system for a data center.
Brook Gummere: We are increasingly implementing innovative hybrid cooling strategies to keep pace with the rapid thermal demands of modern graphics processing units (GPUs) and high-density computers. Our designs often combine multiple free cooling technologies โ such as evaporative cooling, dry coolers and plate and frame heat exchangers โ with refrigeration systems used only as trim cooling to meet peak design conditions. One of our most challenging projects is a data center located in a region where outdoor temperatures can drop to -50ยฐC, which introduces several unique considerations to the design.
How are high-density AI and GPU-driven workloads changing cooling strategies, such as liquid cooling, direct-to-chip systems or hybrid air/liquid approaches?
Ken Urbanek: They are changing it significantly. Most projects have moved to direct liquid cooling for direct-to-chip cold plates to handle the high amount of heat rejects. The need for direct liquid cooling should be evaluated above 50 kilowatts (kW) per rack and is, for the most part, required above 60 to 70 kW per rack. As rack densities exceed 200 kW per rack, we are seeing computing manufacturers evaluate moving more load to liquid cooling, with an industry goal of getting liquid cooling to 100% of the computing load.
Bill Kosik: The growth of AI data centers has required new thinking in power and cooling infrastructure design, moving it closer to the design for high-performance computing facilities. There is a basic assumption that AI data centers will continue to grow in number and in power density. There is also an underlying premise that in the near term, different data center types such as enterprise and colocation facilities will adopt liquid cooling to increase energy efficiency and reduce the size of the data halls. However, it is likely that design, start-up and operations of liquid cooling in these facilities will not be widespread until there are clearer design standards and guidelines.
Brook Gummere: GPU-driven workloads are accelerating adoption of liquid cooling as a primary strategy in modern data centers. While the percentage of the data center load that can use direct-to-chip cooling is increasing, there is still a profile of the load that requires an air-cooling solution. However, as a portion of the infrastructure still requires air-based cooling, we are experiencing hybrid air/liquid systems as the prevailing approach today.
Describe a project in which the building used free cooling.
Ken Urbanek: With the warmer temperatures found in direct liquid cooling systems there is often an opportunity for free cooling in all but the warmest environments. When deployed in milder climates, those with annual extremes lower than 95ยฐF, there is potential for nearly year-round free cooling. Although water use for evaporation needs to be closely reviewed for the given location, in some areas where water is more plentiful, we still see it used to reduce energy consumption. Evaporative fluid coolers can extend the compressorless free-cooling window. We are seeing this as an option in some northern latitude climates where water is more plentiful, enabling year-round compressorless cooling.
What best practices should be followed to ensure an efficient HVAC system is designed for this kind of building?
Daniel Noto: Understanding the future growth needs of the client ensures that a design will require minimal changes in the future. If you are designing HVAC systems for current needs, youโre already 10 years behind schedule.
Brook Gummere: Designing a system that can be quickly adapted to a new load profile is essential for data centers of the future. Establishing the right balance between initial capital cost and long-term flexibility is critical and these discussions need to happen early in the project and design stage. Careful attention to equipment and distribution sizing early on will enable efficient operations between all load profiles.
Ken Urbanek: When it comes to low power usage effectiveness projects, the ability to push technical cooling loop temperatures warmer combined with high-efficiency fluid coolers can be one avenue to achieve a higher degree of efficiency. Adding evaporative precooling can improve that operational efficiency even more.
What type of specialty piping, plumbing or other systems have you specified recently? Describe the project.
Ken Urbanek: On the cooling side, water routed to the cold plates is typically carried through stainless steel piping with sanitary tri-clamp fittings or, in some cases, fusion-welded polypropylene piping systems.
Brook Gummere: Maintaining high water quality within the technology cooling systems is critical. To protect these components, stainless steel piping and fittings along with advanced water filtration systems and welding practices are specified to help maintain the quality of water needed at the chip. Proper water chemistry management is equally important, as the โradiatorโ on the chip can easily become clogged with bacterial growth or water impurities.
What are some of the challenges or issues when designing for water use in such facilities?
Bill Kosik: Based on my project work and development of research papers, there is a fundamental misunderstanding of water use in data centers. There are legacy (and some new) facilities that use open cooling towers, which rely on evaporation of water to cool the facility. A data center that uses water-cooled chillers and open cooling towers will be one of the highest consumers of water. But this is not universally true. The amount of water consumption is based on data center location, cooling tower performance characteristics and the operational strategy. In more humid climates, more water is used; less is used in dry climates. What needs to be understood is that newer hyperscale data centers use air-cooled equipment where there is zero water use, except for potable water and irrigation.
Brook Gummere: Managing water use in high-density data centers presents several challenges, particularly in regions where evaporative cooling is a viable strategy. One of the most significant considerations is navigating a wide variety of local codes and water use regulations. Some jurisdictions prohibit the reuse of blowdown water or the collection of rainwater, while others mandate it, requiring designers to tailor solutions to each locationโs specific rules. Additionally, on-site treatment of blowdown can be highly effective in certain climates yet far more difficult or cost prohibitive in others.
Ken Urbanek: Traditionally, the idea of having chilled water near computing systems was to be avoided, but with in-row coolers, rear-door coolers and now direct liquid cooling it is a common occurrence. Given the deployment of water-based systems, we are not typically seeing drip trays but rather extensive use of leak detection rope. We run continuous segments of leak detection rope below all wet systems. Additionally, at the rack level we provide separate leak detection zones on the terminal branch and flex hoses that are tied to quick-close flood prevention valves that interlock with the data center infrastructure management system to initiate rack shutdown. In some instances, we will even shunt trip rap power and shutdown valves upon water detection.
