Data centers achieve a new level of high-tech: Energy efficiency

Respondents

• Bill Kosik, PE, CEM, BEMP, senior energy engineer, DNV GL Technical Services, Oak Brook, Ill.
• John Peterson, PE, PMP, CEM, LEED AP BD+C, mission critical leader, DLR Group, Washington, D.C.
• Brian Rener, PE, LEED AP, principal, mission critical leader, SmithGroup, Chicago
• Mike Starr, PE, electrical project engineer, Affiliated Engineers Inc., Madison, Wis.
• Tarek G. Tousson, PE, principal electrical engineer/project manager, Stanley Consultants, Austin, Tex.
• Saahil Tumber, PE, HBDP, LEED AP, technical authority, ESD, Chicago
• John Gregory Williams​, PE, CEng, MEng, MIMechE, vice president, Harris, Oakland, Calif.

---

CSE: What level of performance are you being asked to achieve, such as Uptime Institute tier guidelines, WELL Building Standards, U.S. Green Building Council LEED certification, net zero energy, Passive House or other guidelines? Describe a project and its goals, identifying the geographic location of the building.

Peterson: Throughout the Eastern United States we’ve seen requests that buildings be designed to LEED certification standards without actually going through the certification process. Uptime certification, both designer certification and building certification has waned in the U.S. but is gathering increased traction overseas. There are some data centers in South America and India requiring Uptime Institute Tier III or IV certification. Whether an owner is targeting a certain PUE or certification, we keep focused on designing the top energy performers in the world that still maintain superior reliability.

Tumber: From a sustainability perspective, most of our projects are being designed to achieve LEED certification, with LEED Gold being the preference. Few hyperscale providers are pursuing certifications via the LEED Volume Program. The program streamlines the process for large users and allows them to certify facilities throughout their portfolio.

I recently worked on a 68 megawatts data cente,r which is on track to achieve LEED Gold certification. Notable features include heat recovery system to serve the administration building and advanced water treatment technologies to maximize cycles of concentration for the evaporative cooling system (5 COC or better).

Williams: USGBC LEED and Certification of Energy Efficiency for Data Centers are among the most popular data center certification programs in terms of energy efficiency. Currently, less than 5% of U.S. data centers are LEED certified but this number is expected to increase rapidly. Some mention Tier Certification from Uptime Institute — this only evaluates data center infrastructure in terms of systems’ availability for business requirements (service resilience). The Department of Energy’s Energy Star program is another certification program that mostly focuses on energy efficient servers and storage systems.

CSE: What unusual systems or features are being requested to make such projects more energy efficient?

Starr: One we run into every so often is the request for 415 volts distribution, due to the single-phase being 240 volts (within range of a 250 volts computer power supply). We see this commonly with hyperscale customers. Following this approach, with the right equipment selections, there is savings both in energy and avoiding purchase/space of floor standing PDUs. Consider specifying UPSs with economy features, reviewing transformer rise ratings impacting HVAC design; and, choosing medium-voltage distribution when it makes sense for lower cable losses. The most efficient design is a right-sized one. Be responsible with the future operating budget by using frame sizes and technologies that scale, for example power module-based UPS with module management to cycle modules on/offline modules depending on the demand load.

Peterson: Cooling technology is rapidly evolving and new advances are being made every few years. What has also helped is advocating to change the conditions toward what the silicon-based machines can tolerate, not their carbon-based managers. As the temperatures continue to rise the possibilities to reuse the energy becomes greater. For power, the desire to add microgrid techniques for energy savings at the campus or facility level has been increasingly requested, while reducing power demand and peak shaving has been approached more with software-defined power at the rack and row level.

CSE: What types of sustainable features or concerns might you encounter for these buildings that you wouldn’t on other projects?

Peterson: Using direct air cooling has greater concerns because particulates can build quickly with airflows gusting through a data center much faster than a typical office environment. Recent studies have shown how particulates of direct air cooling increased the failure rates of components, so we need to ensure that direct air filtering is designed for the IT equipment and not just acceptable to the occupants.

CSE: What types of renewable or alternative energy systems have you recently specified to provide power? This may include photovoltaics, wind turbines, etc. Describe the challenges and solutions.

Tumber: I have worked on data center projects which used fuel cells as the primary source of power. The solid oxide fuel cells convert natural gas to electricity using an electrochemical process. The projects had unique requirements and fuel cells were selected after an exhaustive review of commercially available energy systems.

One of the challenges was getting approval from AHJ and the utility provider. They wanted the design team to comply with codes and standards applicable to combustion equipment such as boilers and generators. Fuel cells use combustion-free technology and we had to explain the technology to AHJ and utility provider to get them on board.

Peterson: Our energy teams have been designing renewable power systems and focus on the life cycle costs of buildings and systems to determine the best return on investment for our client’s capital assets. We do this by designing on-site and large-scale renewable energy systems. With our deep base of expertise and close partnerships we have helped to develop and promote energy and storage systems of all sizes for our clients, including multimegawatt utility-scale photovoltaic arrays. One of the challenges has been to ensure that the facility can operate with dual sources and change over from one to another seamlessly, which has become increasingly easy with more intelligent interfaces and experienced contractors.

CSE: How has the demand for energy recovery technology influenced the design for these kinds of projects? Describe a mixed-use building in which the heat from the data center was used in other portions of the building.

Kosik: Note: Heat recovery or HR, is sometimes referred to as energy recovery. There are technical differences between HR and ER, but for consistency I will use the term HR in this discussion. Also HR is also used to recover “coolth,” not just heat. When considering HR for a building project, there are many factors that need to be considered. Topping the list are recovery efficiency, financial feasibility and reducing greenhouse gas emissions created on at the site and at the source energy generation. Using these data points as a means of developing a strategy are important, especially during the initial planning process. At a high level, the primary goal of an HR system is to transfer recovered energy from one system to a second system; transferring energy from the first system to the second system reduces overall energy use. By using the recovered energy, less purchased, source energy is needed.

• Recovery efficiency can be expressed as a percentage of the actual heat recovered (after losses attributable to the heat exchanger) to the amount of heat taken from the primary system.
• Financial feasibility can be determined using different analysis techniques. A simple payback method using annual energy cost savings and upfront equipment expenditures will indicate the number of years it takes until the HR system “pays for itself.”

Because HR systems have a high degree of interdependency with other HVAC equipment, systems and operations, making design decisions early in the project design will minimize potential difficulties in coordinating the design and operation of the HVAC system.

One type of HR approach directs waste heat to other systems where it is used to reduce energy use. When HR is applied to data centers, the goal is to take the immense amount of heat generated from the servers, storage equipment and networking gear and transfer it to adjacent, nondata center buildings. Data center HR is currently being used in applications such as heating adjacent buildings, snow melt systems, boiler preheat, domestic water preheat, mechanical room heating and others. The temperature of the air exhausted from the computer equipment is proportionate to the computing speed. When computers are operating close to the maximum processing speed, the more “work” the computers are doing, the higher the exhaust temperature will be. The challenge arises when the energy consumption (and heat output) of the computers fluctuates, resulting in a variable amount of energy that is available to the second system.

Another interesting aspect of using HR in data centers arises when direct-cooled computers or water-cooled IT cabinets are used. In these types of systems, the heat from the computers is transferred directly to the heat rejection water loop with minimal efficiency loss. This design allows for a much cleaner method of HR because the heat transfer medium that is being directed to the second system is water, requiring much less space and reducing motor horsepower. Due to the elevated temperatures in data centers, outdoor air can be used to cool down the air return from the computers. This is achieved with air handling units with heat exchangers that exchange energy from the outside air to the return air, without outside infiltration. This is another form of heat recovery that is used in data centers.

Peterson: A deployment of water-cooled servers running at higher temperatures let us capture energy, indirectly, to support heating hot water for the facility. It was successful enough that the servers could support the entire building heating needs and the occasional sidewalk snow melting in the winter. The mechanical cooling system operated only to stabilize the supply water during peak, continuous processing.

CSE: What value-add items are you adding these kinds of facilities to make the buildings perform at a higher and more efficient level?

Peterson: From our experience of recapturing “stranded” capacity in existing buildings we now include the ability to better scale with more flexibility in new designs. This has led toward ensuring that the way we are maximizing the design possibilities can be achieved in the final, real data center. Each design tends to be unique to the location, the client openness for new innovations and how close we can get to matching power and cooling performance.

CSE: How have energy recovery products evolved to better assist in designing these projects?

Peterson: Unique solutions, such as water-to-water heat pumps and air-to-air enthalpy wheels, have evolved and the efficiency and performance for energy recovery from data centers becomes more compelling every year. As densities increase and systems change toward water cooling closer to the chip the ability to recover and reuse waste heat will catch on quickly. Along with this, operational staff have become more involved in vetting approaches to implement energy recovery products, bringing their experience to weigh in on possible domino benefits or outage-causing pitfalls.

Do you have experience and expertise with the topics mentioned in this content? You should consider contributing to our WTWH Media editorial team and getting the recognition you and your company deserve. Click here to start this process.