Getting to the bottom—and top—of PUE
The parts are greater than the whole
The keys in achieving the greatest energy efficiency are to optimize the cooling systems and also to understand the dynamics of the data center as a whole. For example, the ASHRAE environmental classes (Figure 4) were developed to address the operation of the data center at elevated temperatures, as a means to reduce energy consumption. However it is essential to understand the impact that higher temperatures have on the servers themselves.
Becauseenergy savings in the air conditioning systems is also a fundamental idea behind the development of the ASHRAE, one may assume that as the supply air gets warmer, less compressor power is needed and more hours of economization are available. This premise is generally true, but not universally. In hotter climates, increasing the supply air temperatures generally results in significant reductions in energy use. In colder climates the savings are less dramatic simply because there are more hours annually when the outdoor air can be used in an economization strategy. In these cases, increasing the supply air temperature will not accomplish much because the data center temperature may be greater than the highest annual temperature in that climate.
System and subsystems
Each of the cooling systems consists of multiple energy-consuming devices: compressors, fans, pumps, and humidification equipment. Using the specifics of the actual project is vital in forming an itemization of the various components' annual energy use. Nevertheless, assumptions based on ASHRAE minimum energy performance targets can be applied to the individual components.
Compressorized cooling equipment—This equipment will range from unitary direct expansion equipment to water-cooled chillers. The basis to effective energy optimization for compressorized cooling equipment is the ability to unload the compressors (or decrease speed of variable speed compressors) at an even pace that is in lockstep with the actual cooling load. This avoids over- or under-provisioning of cooling capacity and the corresponding energy use. Also, the equipment must be able to take advantage of cooler outdoor temperatures and lower condenser temperatures.
Supply fans—The power requirement of a supply fan is determined by the air volume, fan/motor efficiency, and static pressure drop of the components that make up the air handling system. The best energy efficiency will come when the difference between the supply and return air is maximized and the static pressure drop is made as small as possible.
Scavenger fans—Used in the indirect air systems, these fans induce outdoor air across the heat exchanger. Because the indirect air systems vary depending on the manufacturer, it is essential to understand how these fans will operate, including the airflow rate, motor power, and operational profile (e.g., fan speed based on outdoor temperature). Scavenger fans can vary speed based on the amount of outdoor air needed to effectively transfer heat from the return air.
Return/exhaust fans—Used primarily for direct air systems as a means of removing the outdoor air from the building to avoid overpressurization. Ultimately, depending on the building design, these fans will range from powerful centrifugal or vane-axial fans to low-powered propeller fan relief hoods. These fans should vary speed based on air volume, and in climates that can use outdoor air for economization a large percentage of the year, the fan system should be carefully designed because they will be running near 100% most of the year.
Pumps—Used in water-based systems only. Similar strategies to fans—keep head pressure as low as possible and vary pump motor speed based on flow requirement.
Humidification/evaporative cooling systems—Using an adiabatic process to humidify or cool the air is necessary to achieve maximum energy savings. In some climates it is not necessary to add moisture to the air based on the ASHRAE temperature and humidity classes, so designing a humidification system may not be necessary.
Water-cooled IT cabinets—Think of these as miniature data centers—the cooling and air movement are built-in. These cabinets rely on fans to move air across a coil mounted in the cabinet and pumps that distribute water to multiple cabinets. The energy used from the fans in the IT cabinet and pumps are not trivial and need to be included in the overall energy use calculation.
Water-cooled computers (component level cooling)—Theoretically the lowest cooling energy consumer, the primary components are pumps and heat rejection (cooling towers, etc.). These are primarily used in high-performance computing applications where individual server cabinets are rated at 80 kW (or more). The goal is to avoid using vapor compression cooling and rely on cooling tower water only given the allowable high cooling water temperature. Parts of the computer, network, and storage systems are not able to be water cooled, so this air conditioning load must be accounted for and cooled by some other means and included in the energy analysis.
Itemization of energy consumers
Energy use simulation is a powerful tool that can be used to provide data to make decisions. Using energy simulation and analysis techniques gives the engineer insight into how the individual components behave based on IT load, supply air temperatures, and outdoor conditions. Applying data visualization techniques using line graphs allows for a detailed scrutiny of the energy usage of the components over the course of a year. This is necessary because the cooling systems perform very differently in cold weather than they do in hot weather. This approach also is used for evaluation of cooling system energy when analyzing different locations (climates). This type of analysis will expose cooling systems that might work well in certain climates and not in others, so worldwide prototypical solutions can be varied by location.
Indirect cooling with direct expansion assist—Before a detailed evaluation of the energy simulation results is performed, it is often helpful to do a visual investigation of the annual energy use line graphs to make initial observations (Figure 5). This figure illustrates the difference between an indirect air cooling system (system 1) and an indirect evaporative cooling system (system 2), both with direct expansion (DX) cooling assist. These systems are designed to use DX cooling when the supply air temperature setpoints can no longer be maintained, augmenting the cooling capability of the system.
In this specific example (excerpted from analyses based on actual project documentation), the two systems perform quite well when compared to other standard data center cooling solutions. The primary differences show up in the fan power and the effectiveness of the heat transfer mechanisms. In system 1, the scavenger fan energy is much less than for system 2, but the supply fan energy is higher than in system 2. Also, the heat transfer effectiveness of system 2 outperforms system 1. This is evident when inspecting the line representing the data points for the cooling energy; system 1 has higher peak power occurring more often during the colder months of the year.
Notice that in both systems, the humidification energy is negligible. In simple terms, good energy performance (in a temperate climate) is exemplified by little or no compressor energy expended during the fall, winter, and spring months. During the summer months (June, July, and August in the northern hemisphere), compressor energy is depicted by a smooth curve that follows the curve represented by outdoor temperatures. The cooling energy expended in a data center has a strong correlation to outdoor air temperatures and should follow these conditions as closely as possible to avoid over-provisioning of cooling capability causing inefficiencies and unneeded consumption of energy. Figure 6 shows the same parameters as Figure 5, but Sao Paulo, Brazil, weather data is used. The overall energy usage is greater than Chicago, but there is a more uniform energy use across the year, where Chicago has higher spikes of power use in the summer months. Also, Sao Paulo is in the southern hemisphere—the seasons are the opposite of the northern hemisphere.
Water-cooled chillers with water economizer—The first thing that becomes evident when reviewing the line graph of the energy use data for water-cooled chiller systems is the number of components used (Figure 7). Each of these uses energy, and the overall efficiency of the system is still good. This is why this type of system has been the gold standard for many years in large data centers and commercial buildings. Because the water economizer is based on the moisture contained in the outdoor air, in more humid climates less time is available annually to use the economizer.
In Figure 7, systems 3 and 4 show the annual energy use of a water-cooled chiller with water economization and AHU, and the same cooling system using water-cooled computers in place of AHUs. The line graphs for system 3 have a telltale sign indicating room for energy efficiency improvement: the compressor power in the summer months shows little fluctuation; that is, there is little economization taking place during this time period.
In system 4, very little fan energy is needed except for cooling areas outside of the data center area. This becomes a primary driver of why system 4 is more efficient than system 3. Another important contributing factor is that in system 4, the water temperature is 20 F warmer than the water in system 3. This has a twofold effect: first, the compressor energy is lower because of the increased evaporator temperature, and second, there is an increase in the number of hours in which the water economizer can be used based on a higher acceptable wet-bulb temperature.
Designing cooling systems for data centers is a process that has many variables and requires many decisions, including how the IT equipment will interface with the cooling. It is vital that the engineering team use a methodical design process that includes detailed energy simulation and analysis techniques which will help make decisions throughout the life the project. Also, taking into consideration that the IT equipment will consume more than 75% of the data center annual energy, there needs to be a careful assessment and optimization of the design and operational parameters to create long-lasting energy-savings synergies.
William Kosik is principal data center energy technologist with HP Critical Facilities Services, Chicago. Kosik is one of the main technical contributors shaping HP Critical Facilities Services’ energy and sustainability expertise. He has worked on energy analysis and strategy projects in more than 25 countries, and consults on client assignments worldwide. A member of the Consulting-Specifying Engineer editorial advisory board, he has written more than 25 articles and spoken at more than 45 industry conferences.