Can data centers become energy producers?

Recovering "waste" heat from servers in data centers opens the door to innovation and efficiency.

By Bill Kosik, PE, CEM, LEED AP, BEMP; exp U.S. Services, Chicago January 24, 2018

Learning objectives

  • Provide an overview of heat-recovery systems for commercial and institutional buildings, and offer discussion on the issues pertaining to design parameters, efficiency, and opportunities to recycle energy
  • Discuss heat recovery from data centers, specifically demonstrating how data centers with water-cooled computer equipment will create highly effective energy-recycling processes.

The amount and quality of recovered energy originating from HVAC systems depends on the building use and internal processes, from fume hood exhaust air in laboratories, process exhaust in industrial facilities, and toilet and other exhaust air in office buildings. The amount of heat recovered, and the first cost of the recovery equipment are the primary components when analyzing the total cost of ownership; it is important that this analysis be completed in the planning phases of the project. At a high level, the primary goal of an energy-recovery system is to transfer the recovered energy to a second system. This second system requires energy to achieve the design objective. By using the recovered energy, less purchased energy is needed.

There are many advantages in using heat recovery: 

  • Reduces energy costs: All waste heat that can be applied to another process that offsets/replaces the energy needed for the other process will reduce the overall energy use, consequently reducing utility costs. 
  • Reduces cost of capital equipment: If the overall energy use is reduced, primary equipment, such as boilers, can potentially be downsized. This will be driven by the calculated peak load and the acceptable safety margin.
  • Reduces impact on the environment: Equipment that uses fossil fuels to operate can potentially be operated at a reduced capacity assuming the heat-recovery system can meet the balance of the load. This will result in lower greenhouse gas (GHG) emissions.

Research on heat recovery

The U.S. Department of Energy (DOE) published a paper on the impacts of using heat recovery in all kinds of applications, primarily in the manufacturing sector. The DOE classified the different processes into different temperature groups: low, <600°F; medium, 600 to 1,200°F; high, >1,200°F. The water temperatures in a data center will fall into the "low" category.

While it might seem that applications that fall into the "low" category will be less beneficial as compared with the other categories, the DOE reports that applications in the 77⁰ to 300⁰F temperature range represent nearly 80% of the total estimated waste heat. (Temperatures of cooling water exiting water-cooled computers will generally have water temperatures in that will range from 100⁰ to 150⁰F). Based on this, it is possible that HVAC industry will see greater opportunity and develop products and approaches that will make heat recovery in the data center more viable. Also, it is encouraging to know that the average simple payback for 19 sample heat-recovery projects is 1.5 years. 

Types of systems

There are many different methods of designing heat-recovery systems for commercial and institutional buildings. These methods cover a large variety of building types and HVAC system configurations. But as a starting point, the following examples demonstrate the core principles of energy-recovery systems: 

  • Runaround coil loop: This moves heat from the exhaust airstream to the supply airstream using two heat exchangers (finned coils) and a heat-transfer fluid, usually water or glycol. A pump is required to move the fluid through the system. A runaround coil loop is best used in situations when the airstreams are not adjacent to each other. If the temperature of the fluid heated by the exhaust air is greater than the outdoor air, energy can be recovered. However, to make this system financially feasible, the warm-water temperature must be considerably greater than the supply-air temperature. And long runs of piping add to material, installation, and energy costs (increased pump power). Generally, the economics of this system are more favorable for facilities located in colder climates that have considerable amounts of constant exhaust air.
  • Packaged energy-recovery systems: In commercial and institutional buildings, packaged energy-recovery systems are the simplest and most effective heat-reclaim solution. Many manufacturers offer products that are designed specifically to work with typical air conditioning and ventilation loads for office environments, schools, libraries, etc. One common way of implementing this type of energy-recovery system is to use air handling units with integrated heat wheels, thermosyphons, air-to-air heat exchangers, or evaporatively cooled air-to-air heat exchangers. Each one of these configurations will have different heat-transfer and efficiency characteristics and must be analyzed based on the actual loads, climate, space available, maintenance characteristics, etc.
  • Total energy wheel: Most energy-recovery devices transfer heat (sensible) energy only. An enthalpy wheel, or total energy wheel, exchanges both heat (sensible) energy and moisture (latent) energy between the supply and return airstreams. This type of wheel can be coated with a desiccant (moisture-absorbing) material and is rotated between the incoming fresh air and the return air. Heat and moisture in the return air are transferred to the wheel. During cold, dry outdoor conditions, the outside air passes over the rotating wheel, providing pre-conditioning, as the heat and moisture are transferred to the air stream. In the cooling mode, the outside air is precooled and the moisture content is lowered. These processes reduce the amount of energy required by gas-fired heating equipment and compressorized cooling equipment. 

Energy recovery in data centers

Certain types of data centers (with specific types of computer equipment) produce large amounts of usable waste heat. Supercomputing centers are a prime example having extremely high power densities (greater than 1000 W/sq ft in some cases) and liquid (water mainly) cooled computers. Heat recovery systems for these facilities prove to be very effective in reducing energy use. Because supercomputing facilities are generally purpose-built, and undergo a major evolution in technology every few years, there is room for the development of innovative solutions driven by the distinct characteristics of these data centers:

  • Much higher supply- and return-air temperatures when compared with an office building (and even other data centers). This creates greater opportunity for recovering both heating and cooling energy.
  • When the computers in the data center are water-cooled, heating energy can be recovered due to the high internal temperature in the computers.

Air-based energy recovery: Data centers that do not use water-cooled computers can still benefit from recovering energy. As long as the outside dry-bulb temperature is less than the return-air temperature, cooling energy is recoverable from the outdoor air. For example, if the design supply-air temperature for the data hall is 75⁰F and the return-air temperature is 95⁰F, when the outside air is less than 95°F there will be a reduction in cooling energy. And knowing there are many domestic and international locations that have very few hours per year that exceed 95⁰F, it becomes clear that this type of energy-recovery approach can be very efficient.

Even more efficient, at 75⁰F outdoor air, very little (if any) mechanical cooling energy is needed. Consider that in Seattle, there are 8,516 hours below 75⁰F (97%); in Chicago, 7,614 hours (87%); in Miami, 3,041 hours (35%). As with many other energy efficiency schemes, the climate plays a very important role in the overall effectiveness of the approach.

Water-based heat recovery: Many of the energy-recovery techniques discussed so far (both air- and water-based), are applicable to air handling systems using exhaust/return- and supply-air arrangements. However, depending on the facility type, it also is possible to recover energy from other types of processes that use water to cool the internal components of equipment, then transfer the heat to another process. Historically instead of recovering the heat in these processes, heat has typically been rejected to the outside, losing recoverable energy.

In data centers, water-cooled computers enable hydronic heat-recovery options. There are several options for cooling computers with water (some of these solutions also work with refrigerant in lieu of water): 

  • IT cabinets with integral chilled-water fan coil units (side cars).
  • Rear-door chilled-water heat exchangers, with or without fans, mounted on the back of the IT cabinet.
  • An IT cabinet and servers that have built-in thermal planes. When the servers are installed into the cabinet, the planes contact one-another enabling heat transfer from the server to the circulating cooling water.
  • Internal heat sinks mounted directly to the server’s central processing unit (CPU), dual in-line memory modules (DIMM), and the graphics processing unit (GPU). The heat sinks are cooled directly by the water, without additional intermediate heat exchangers. Since the cooling water contacts the sources of heat directly (via heat sinks) the process is very efficient.

As an example, the water that cools the computers will be heated up to around 150⁰F. This temperature water can be used for a wide variety of things-increasing the feedwater temperature for boilers, heating a swimming pool, supplementing a district heating system, and even providing heat to temper an indoor arboretum.

Hydronic energy recovery

When planning a heat-recovery system using water-cooled computers, it is essential to determine the expected temperatures in the energy-recovery water loop. The temperature is a function of how "hard" the computers are working. Generally, as the workload increases, the temperature of the CPUs, GPUs, and DIMMs begin to increase, resulting in greater heat output. Ultimately, the water temperature in the energy-recovery loop is determined by several variables, and because the amount of energy recovered from computers is not constant, care must be taken in estimating the amount of energy that can be recovered. 

External factors impacting the design approach

Recovering heat from computers will end up as a purely academic exercise unless the recovered heat can actually be used elsewhere. Viability must be assessed using long-term savings and short-term costs. The total cost of ownership (TCO), which includes any necessary modifications to existing hydronic systems, must show promise in terms of financial and commercial viability for the project to succeed.

ASHRAE Thermal Guidelines for Data Processing Environments has a section on water-cooled computers and contains a table on water-temperature classes, with each class having a facility cooling-water temperature range from minimum to maximum inlet temperatures. It is necessary to have this information when discussing data center temperatures with the technology planners and thermal engineers. (Note that the temperature ranges for water-cooled computers are higher than the inlet temperatures for air-cooled computers.) When the heat is removed directly from the motherboard components, the heat transfer is more effective and creates a more controlled heat-removal process.

Figure 1 can assist in determining the inlet and outlet water temperatures of the heat-recovery loops. The data shows how the inlet temperature and system power (which correlates to the workload intensity) affect the discharge temperature. In this example, the discharge temperatures range from 100⁰ to 130⁰F; if this computer were water-cooled, these values would change, but the concept still applies. 

Energy, cost-reduction effectiveness

A foundational concept in data center HVAC design is to "collect" the heat, contain it so it doesn’t mix with cold supply air, and get it out of the data center via heat-rejection equipment. This heat collection/rejection system will have three to four separate points of heat exchange. At the end of this chain, condensing units, dry coolers, or cooling towers reject the heat to the atmosphere. There is a considerable amount of energy expended moving heat from the computers to the outdoor environment.

Using the "heat-transfer chain" as a baseline in the process of determining feasibility and financial viability of a heat-recovery scheme, the following items need to be considered: 

  • What is the monetary value of the recovered heat, in energy savings but also re-selling the recovered heat?
  • Will the energy consumption of the heat-rejection equipment be reduced?
  • How many hours per year can the reclaimed heat be used?
  • Is there more than one use available for the reclaimed heat in the event of a change? What is the load profile for these uses?

The answers to these questions will feed into the overall feasibility study to determine the TCO of the heat recovery concept. 

Operational profile and available heat

Use of servers and other computer equipment in data centers will vary based on several factors. Some will peak at certain times of the day, week, or month depending on the type of operation they support. However, data centers that support high-performance computing (HPC) or supercomputers have a very different load profile and will often have near-constant use (electrical load). In the science community, these computers are widely sought after because of the massive computational power and data throughput needed for complex, data-driven projects. Engineers, scientists, and researchers from private industry often wait their turn until the machines are free. So, other than scheduled downtime and when jobs have completed, the machines run at near full capacity.

This type of operational profile is ideal for recovering heat for use in other applications, such as heating and snow melting. And because both are weather-dependent, the building automation system (BAS) can modulate the central heating plant equipment (boilers, pumps) as to not under- or over-compensate when making up for any heating shortfall that the heat-recovery system may encounter. Because heating a building is a critical operation, and considering the problems that will occur if heating is lost, it is recommended the heat-recovery system be designed as a supplement, rather than the primary means of heating the building. 

Recovering energy from water-cooled servers

To demonstrate an application of recovering and using waste heat from the data center, the following analysis focuses on two hydronic-based building systems: a hydronic heating system and a water-based snow-melting system. These are common hydronic building systems found in many buildings, especially in colder climates. Both of these systems operate in a temperature range that is similar to the water loop temperatures serving the computer equipment; this is another aspect of why these are well suited to using recovered heat.

  • For the building heating system, adding a secondary source of heat into the water loop (via a heat exchanger) will augment the system capacity and reduce the amount of fossil fuel (natural gas, propane, oil) consumed by the boiler. Typical water temperature ranges from 100⁰ to 180⁰F.
  • Hydronic snow-melting systems work by circulating warm water through plastic tubing embedded in the pavement; the water temperature varies based on factors such as the rate of snowfall, wind speed, and outside temperature. These systems operate at lower water temperatures, typically less than 130°F. 

Climate analysis

The climate is one of the most important components to understand the savings potential of using a heat-recovery system. Buildings located in colder climates will show the greatest benefit by using a building heat-recovery system. Simply put, there are many more hours per year that a building will require heating. Minneapolis is used in this analysis because of its cold, dry climate (ASHRAE Climate Zone 6A). It also is a good proxy for other cold-climate cities in Zone 6A. In Minneapolis, the dry-bulb temperature is less than 52⁰F during 60% of the year and less than 32⁰F during 35% of the year. Mitigating the building envelope heat losses from conduction and infiltration is a significant portion of the heating-energy consumption. Therefore, heat-recovery systems make economic sense in climates like Minneapolis, in general terms.

When performing a building performance simulation that includes a heat source with a nonlinear output, (such as a heat-recovery system from computer equipment), the model becomes numerically intensive especially when analyzing the interdependencies between the computers’ heat output, the outdoor conditions, the building’s internal heat gains, and the physics of the building envelope. However, using a simplified analysis allows for a preliminary look at the amount of heat that can be recovered from the computers to offset losses through the building envelope and to provide sufficiently warm water for a snow-melt system. This analytic method is used as a starting point when first examining the viability of implementing a heat-recovery system. After this initial analysis, more detailed work is required (see Figure 2).

Modeling and simulation approach

An energy model was developed to study the practicality of using recovered heat (in the form of warm water) for both snow-melting and building heating; calculations were developed at a high level to simulate the operation of the systems. The simulation accounts for changes in the use of the computers, resulting in variations in the heat output.

One of the first steps in the simulation is to develop a heat-loss profile of the building and the heat required to make warm water for the snow-melt system. Visualizing the data assists in developing a quick overall understanding of the magnitude of the heat loss and how it changes over 1 year.

The next step is to simulate the potential heat output from the computer equipment in the data center. Because the heat-recovery system is water-based and tied into the cooling loop for the computers, a few considerations need to be taken:

  • Approximately 50% to 75% of the computer’s heat output should be transferred to the cooling loop in the data center. In this example, the power demand of the computer equipment is 500 kW; 375 kW will be dissipated to the cooling loop while the remaining 25% (125 kW) is rejected to the air in the data center.
  • Heat exchangers in the system are used to decouple the water loops; assume that each heat exchanger will be 90% efficient.
  • Electrical use for supercomputers will vary based on the specific attributes of the machine and the types of research being conducted. While it is difficult to develop a precise usage model, most supercomputers currently run at high-usage rates. (One of the general provisions to receive federal funding for a computational research facility is to demonstrate that there is a stated need to perform research on the computer; this ensures that the machine will be used to its fullest capability). A usage model applied to this analysis was constructed based on operational data from several supercomputers (see Figure 3).

Displaying the building’s heating-requirement data with the data showing the availability of recovered heat generated by the computers (Figure 4) on one chart shows the operational interdependency between the building heating system and the data center’s heat-recovery system.

Although the actual outdoor temperatures (the primary driver of the building heating requirements) will certainly vary from the weather data used in the energy modeling, this analysis is a good demonstration of when and how often the recovered heat can successfully supplement the heating and snow-melt systems.

Based on the simulation and subsequent analysis, the heat-recovery system from the water-cooled computers will be able to supplement heating the building and the snow-melt system.

As computers continue to get more powerful and, in some cases, consume greater amounts of energy, it is important to consider to investigate heat-recovery systems when water-cooled computers are used. A tremendous amount of electricity powers computer equipment in data centers. The technology industry stands to benefit financially if heat recovery systems are used. By doing this, an opportunity opens up to drive new innovation and demonstrate how reducing energy benefits both the bottom line and the environment.

Bill Kosik is a senior mechanical engineer with exp U.S. Services. He is a member of the Consulting-Specifying Engineer editorial advisory board.