What ASHRAE 90.4 does for data center energy efficiency

It is clear that ASHRAE raised the bar significantly in terms of increasing the minimum efficiency requirements for heating, ventilation and air conditioning systems and electrical systems in ASHRAE Standard 90.4: Energy Standard for Data Centers.

Fortunately, as strategies have matured for reducing energy use in data centers, a number of organizations have developed best practices and recommendations that took a holistic view of the data center and recognized the interdependencies between the building systems and the information technology equipment. These were watershed moments because, for the first time, an industrywide movement started to gain momentum.

Data center energy efficiency was once considered a niche practice; now it is developing into a more mature, mainstream discussion with diverse professions working within the data center realm.

Data center industry comes together

While it might be tempting to look back on the early years as a time of indecisiveness and conflicting priorities, I think it was very good that we went through this period. Many ideas, processes and designs were launched, resulting in recommendations still used. While developing strategies, different segments of the data center industry began to converge on similar approaches to reduce data center energy consumption. Ultimately this resulted in practical, technically achievable and defendable energy efficiency solutions that would become part of the foundation for the development of future standards and codes.

In 2008, as a part of the ASHRAE Datacom Series, Technical Committee 9.9 published a book, “Best Practices for Datacom Facility Energy Efficiency.” TC 9.9 focuses on all aspects of data centers, technology spaces and electronic equipment/systems. As described by ASHRAE, “The intent of this publication, as part of the ongoing ASHRAE Datacom Series, is to provide the reader with detailed information on the design of datacom facilities that will aid in minimizing the life cycle cost to the client and to maximize energy efficiency in a facility.”

Since then, ASHRAE has published several books related to data center energy use reduction, including volumes on data center thermal conditions and particulate contaminants, advanced server cooling and integration of information technology and power/cooling systems. While some of these topics are not directly referenced in Standard 90.4, many of these strategies can improve energy efficiency.

The members of ASHRAE TC 9.9 and scores of industry experts volunteered their time and know-how to shape the primary topics in ASHRAE 90.4. Tapping into this expertise, especially for data centers, created a framework and boundaries that would later become the 90.4 standard.

ASHRAE 90.1 updates

During the first decade of the 21^st century, a growth in computer use, both by corporations and individual users, created an unprecedented demand for data centers. This resulted in a “double learning curve”:

• The proliferation of very large data centers (and computer technology in general) stemmed from rapid advancements in the design and production of processors, memory and digital storage.
• Power and cooling system design was out-paced by rapid advances in computer technology. Also, industrywide consensus-driven energy-efficiency guidelines for data centers were not consistently used.

During this time period, ASHRAE issued five editions of ASHRAE Standard 90.1: Energy Efficiency for Buildings Except Low-Rise Residential Buildings (1999, 2001, 2004, 2007 and 2010). There is only a small amount of data center-specific language in these versions, which posed design challenges to meet energy compliance requirements for new, state-of-the-art data centers. Although all of the same provisions in ASHRAE 90.1 that apply to other building types also apply to data centers, the exceptions on outdoor air economizer and the defined HVAC system type that applies to the data center (direct expansion) in some cases did not depict the design or operations intent as developed by the owner and design engineer.

Standard 90.1 contained minimal information on how to deal with a data center that is significantly larger than a telecom closet. This was a part of the imperative that the conversation, vis-a-vis national standards, needed to expand. The conversation continued with industry involvement, particularly from end users, some of whom already had mature internal energy-efficiency programs for their data centers. (It is important to note that these end users provided invaluable hands-on, operational data that played a key role in the ultimate development of the ASHRAE 90.4 standard).

ASHRAE Standard 90.1 is widely used nationally (and outside the U.S.) as a tool to ensure energy compliance for commercial buildings. Because ASHRAE 90.1 is code-ready, many jurisdictions include it in their building codes. Given its long history and widespread use, Standard 90.1 seemed the logical choice to build on the existing language used for server rooms and telecom closets.

It wasn’t until the ASHRAE 90.1-2007 edition was released that ASHRAE commenced its process of soliciting proposed changes from the public that would appear in the 2010 standard. As part of their maintenance process, ASHRAE received dozens of responses proposing revisions for the upcoming 2010 standard, including a response from ASHRAE Technical Committee 9.9 (Mission Critical Facilities, Data Centers, Technology Spaces and Electronic Equipment).

TC 9.9 proposal recommended several changes to the 2007 standard, resulting in enhanced technical requirements and clearer, more consistent language on data center energy efficiency. In the end, the recommendations proposed by TC 9.9 were published in the release of the 2013 edition of ASHRAE 90.1, addressing many of the design and modeling concerns from past editions. This was a pivotal time for the advancement toward the implementation of a dedicated standard for data center energy efficiency.

What’s in ASHRAE 90.4?

In September 2016, ASHRAE published the long-awaited Standard 90.4-2016. In the foreword, there are several important statements that set expectations as to the goals and philosophy of the standard. Many of the elements that embody the technical segment of the data center industry include innovation applying engineering and analysis. The new ASHRAE 90.4 standard was developed “… in a manner that does not stifle innovation in the data center industry, while simultaneously offering criteria to help ensure energy savings.”

Also, given the different methods across the industry to comply with ASHRAE 90.1, the ASHRAE Standing Standards Project Committee 90.4, the new standard “… was developed to be code-intended, similar to Standard 90.1.”

The heart of ASHRAE Standard 90.4 is defining the path to energy efficiency compliance, specific to data centers, while the compliance requirements for “nondata center” components will continue to be contained in Standard 90.1. According to ASHRAE “… the chairs of the committees responsible for both Standard 90.1 and Standard 90.4 will meet with members of the ASHRAE Standards Committee to address any conflicts in scope between the two standards.”

Therefore, ASHRAE 90.4 refers to ASHRAE 90.1 for compliance requirements for nondata center elements:

• Building envelope.
• Lighting.
• Mechanical cooling equipment efficiencies.
• Service water heating.
• Other equipment criteria.

Arguably, the most important aspect in ASHRAE 90.4 is that it is performance-based design standard where the energy efficiency of the proposed mechanical and electrical systems are judged against a standard built loosely around the power usage effectiveness metric. PUE was developed by The Green Grid, which defines its “… mission is to drive accountable, effective, resource-efficient, end-to-end ICT ecosystems.”

However, ASHRAE specifically points out that these metrics “… [do] not directly compare to a data center’s annual measured design power usage effectiveness.” The metrics used in the 90.4 standard are mechanical load component and electrical load component, both compliance methodologies new to ASHRAE.

ASHRAE’s mechanical load component

Annualized MLC, is the sum of all power required for cooling, fans, pumps and heat rejection equipment, divided by the power for the data center ITE design. The design MLC was introduced in ASHRAE Standard 90.4-2016, but was removed from the 2019 edition.

It was replaced by the annualized MLC calculation, which requires a more detailed analysis, resulting in a higher degree of accuracy. Additionally, the equation to determine the annualized MLC was revised from ASHRAE 90.4-2016. Before the revision, the annual energy use calculation required that the MLC is less than or equal to the value at 50% load.

To demonstrate impact of part-load conditions on cooling equipment and overall system design, the calculation for the annualized MLC now requires the energy use of the HVAC system to be calculated at 25%, 50%, 75% and 100% of the ITE load. Once the annualized MLC is calculated, it is compared to the maximum annualized MLC values in ASHRAE 90.4.

Based on advancements in HVAC equipment and system design (and feedback from the industry), ASHRAE revised the annualized MLC requirements published in Standard 90.4-2016. This raised the bar for the energy-efficiency requirements for the mechanical systems. Table 6.5 lists the maximum values for annualized MLC for ITE systems that are ≤300 kilowatts and >300 kilowatts. Both of these associates a maximum annualized MLC value with a climate zone as defined by ASHRAE Standard 169: Climactic Data for Building Design Standards.

As in any building design, the energy use of a data center is heavily dependent on the climate zone, especially when using economization and elevated supply air temperatures.

Another interesting feature of ASHRAE 90.4 is the recognition that energy consumption is greatly impacted by the reliability requirements of the mechanical systems, starting with systems that have no redundancy all the way up to systems that have multiple redundant units where maintenance can be performed without losing and cooling in the facility. The standard provides a procedure to determine energy efficiency for these types of HVAC systems to ensure the reliability level is incorporated into the compliance process.

ASHRAE’s electrical load component

In a commercial building such as an office or school, the electrical system losses are minimal compared to the overall building energy consumption. The primary losses are from the distribution transformer which, in general, account for 1% to 2% of the overall energy use.

In a data center, losses from the incoming transformers, uninterruptible power supply equipment and electrical distribution to the ITE systems. Depending on the complexity and reliability goals, historically these losses were 6% to 10% at 100% load.

Knowing that most electrical systems will typically operate below 50%, the efficiency drops even further. Depending on the type of data center, current designs are applying innovative solutions such as offline UPS and simply minimizing the amount of power run through the UPS. Most likely there will be an overall gain in electrical system efficiency even for a “typical” data center.

The compliance path for electrical system is accomplished by using the design ELC. This equation determines the losses in the proposed electrical system and comprises the three primary segments of the electrical power delivery:

• Incoming electrical service segment: Commences with the load side of the incoming electrical service continuing through any intermediate transformers, switchboards and feeders; the segment terminates at the UPS equipment.
• UPS segment: This segment starts at the UPS input and ends at the output. ASHRAE 90.4 includes detailed language on the specific items that must be included in the calculation.
• ITE distribution segment: Calculating the losses of this segment begins at the output of the UPS and ends at the connection between the IT equipment and the receptacles and power strips. The IT equipment and power supplies are not included in this analysis.

The design ELC is calculated using the worst-case parts of each segment of the power chain to demonstrate a minimum level of electrically efficient design.

Similar to the mechanical system reliability, the electrical systems “consume” more energy at part-load conditions. But they are very different: Energy consumed by mechanical systems results in some form of useful work (cooling air, moving air, pumping water). The energy “consumed” by the electrical system feeding the information technology equipment is the difference in energy entering an electrical system and the energy leaving the electrical system.

Different types of UPS will have a significantly different loss profile, especially when comparing lower kilovolt-amperes rated to equipment to the large, enterprise scale systems. Generally, the profile remains fairly flat until the system hits 50% capacity. The losses incurred by the electrical system manifest in two ways:

• The energy that is lost is not available for powering the computers, so either more energy must be delivered to the facility or the capacity of the IT systems must be limited to the available power.
• The lost energy is dissipated as heat and, depending on the type and location of the equipment, cooling and/or ventilation is required. The greater the losses in the electrical system, the greater the amount of energy used by the HVAC equipment.

For data center electrical systems, determining energy use performance at part load is vital when developing strategies to minimize energy consumption. Although UPS equipment manufacturers are striving to develop more efficient equipment, ultimately the capacity, type and operating characteristics will heavily influence the efficiency.

To address this, ASHRAE’s ELC compliance path for electrical systems consists of different scenarios that allow the user to model different concepts, including part load performance, capacity of ITE systems and reliability schemes.

The attribute categories required to determine the maximum ELC for the base electrical system are defined as follows:

UPS redundancy configuration:

• Single-feed UPS (N, N+1, etc.) or no UPS.
• Active dual-feed UPS (2N, 2N+1, etc.).

Calculation percentage (of total ITE load):

• Single-feed UPS: 100%, 50%.
• Active dual-feed UPS: 50%, 25%.

Segments of ELC and overall ELC (loss/efficiency values for the following items listed in the standard):

• Incoming electrical service segment.
• UPS segment.
• ITE distribution segment.
• Electrical loss/efficiency total.

Maximum ELC:

• Single-feed UPS: 100%, 50%.
• Active dual-feed UPS: 50%, 25%.

While it is not possible to define performance requirements for all types of electrical equipment and systems, the system types defined in the standard will most likely apply to a majority of data centers.

Compliance with ASHRAE 90.4

To demonstrate compliance, ASHRAE requires the following submittals:

• Criteria used to develop the basis of design, including calculations.
• An electrical single-line diagram.
• ELC calculations including as-designed values and the values used to demonstrate compliance.
• Floor plans identifying areas served by all distribution.

MLC and ELC are of primary importance in applying ASHRAE 90.4 to data center projects. In addition to these, the standard includes exceptions, alternate configurations, criteria for new and renovated buildings and many other elements required to comply with the standard.

Also, Standard 90.1 is an integral part of ASHRAE 90.4, so it is essential that the ASHRAE 90.1 requirements are part of the overall process. A long time in the making, Standard 90.4 is now in its second edition. For people in the data center industry, this standard will be vital to minimizing data center energy consumption.

Similar to how energy efficiency goals percolated in the industry, ASHRAE sums it up well: “It is the hope of the committee knowledgeable people within the data center industry will continue to offer their talents to sustain and improve the quality and accuracy of the standard moving forward.”

Why is ASHRAE 90.4 so important?

I am a firm believer that to succeed in the future, we need to learn about the past. So, when the editor of Consulting Specifying Engineer asked me to develop an article on the ASHRAE Standard 90.4: Energy Standard for Data Centers, I welcomed the opportunity.

I suppose my enthusiasm was easy to spot, given that this will be the fourth article I have written on Standard 90.4 since 2017, each new article building on its predecessor and providing a snapshot of data center energy efficiency efforts.

Why am I so interested in data center energy consumption? There was a time when approaches to data center energy efficiency were very new and at best were not understood and at worst were not taken seriously. But these types of attitudes were a direct result of several conflicting guidelines and recommendations developed by leaders in the data center industry.

This situation was further aggravated by approaches to data center energy efficiency that had a silo approach, such as isolating electrical systems, information technology equipment and heating, ventilation and air conditioning efficiency. Experts in their respective fields wanted to do the right thing by working on what they are best at.

Do you have experience and expertise with the topics mentioned in this content? You should consider contributing to our CFE Media editorial team and getting the recognition you and your company deserve. Click here to start this process.