Data center design: Electrical and power systems

In the information age, data centers are one of the most critical components of a facility. If the data center isn’t reliable, business can’t be done. This is a review of the key electrical and power issues in data centers.

By Consulting-Specifying Engineer January 27, 2014

 

Respondents

  • Kevin V. Dickens, PE, LEED BD+C, Mission critical design principal, Jacobs Engineering, St. Louis
  • Terrence J. Gillick, President, Primary Integration Solutions Inc., Charlotte, N.C.
  • Bill Kosik, PE, CEM, BEMP, LEED AP BD+C , Principal data center energy technologist, HP Technology Services, Chicago
  • Keith Lane, PE, RCDD, NTS, RTPM, LC, LEED AP BD+C, President/CEO, Lane Coburn & Associates LLC, Bothell, Wash.
  • David E. Wesemann, PE, LEED AP, ATD, President, Spectrum Engineers Inc., Salt Lake City

CSE: What standby or backup power solutions have you specified recently in a data center or mission critical facility? Why was it successful?

Lane: The standard backup solutions include standby generators and UPS systems. IT is critical to ensure that the standby generators are not overloaded under all failure conditions. Additionally, the generator must perform properly under all failure modes harmonic distortions. The engineer must specify a generator with the appropriate level of subtransient reactance. Also, it is a good idea to specify a generator that is “mission critical” rated for a load factor of 85%.

Wesemann: Time-tested, tried, and proven systems: standby diesel generators with closed-transition bypass isolation switches and static double-conversion on-line UPS with batteries. Paralleling gear is necessary when capacity or redundancy requires it, but I avoid generator paralleling gear unless necessary for capacity. It has become a single point of failure on some projects.

Gillick: In large mission critical facilities, a current trend among engineers is to specify medium-voltage generators because it is possible to obtain utility service at 15,000 V or higher—a significant benefit over the conventional 480- or 600-V systems. This has required larger but fewer feeders—therefore, less copper—with an associated cost savings. This is a successful, robust power strategy, but we have had to manage the increased risk of arc flash events and other medium-voltage safety practices and equipment to satisfy medium-voltage electrical commissioning and testing—often at full load.

CSE: What types of electrical/power flexibility are current data center clients requesting?

Gillick: We are seeing many different electrical/power flexibility requirements depending on the type of data center. In a typical Tier III or Tier IV data center, we are being asked to commission typical load densities between 75 and 250 W/sq ft. However, owners are being asked to provide power at an even higher density for some equipment rack installations, and these can range from a density of 6 kW per rack to as high as 35 kW per rack. Anticipating electrical and mechanical distribution requirements, owners are requesting design of electrical distribution systems with enough flexibility to deliver various load densities out to the raised floor.

Lane: Flexibility and modularity are the key features clients require: Flexibility to modify the design for future phases and to ensure the infrastructure is in place to provide for changes. Modularity to ensure incremental components can be added as density and/or redundancy increases are required.

Wesemann: They are requesting the ability to make changes and additions quickly and easily, some of which can be avoided by standardizing on a typical size/capacity of rack circuits and power distribution plug strips; the ability to start with minimal capacity requirement then grow modularly and easily without taking the system down, and an A and B bus system feeding dual power cord loads is very common today.

CSE: What experience have you had with The Green Grid report “Recommendations for Measuring and Reporting Overall Data Center Efficiency: Version 2 – Measuring Power Usage Effectiveness (PUE) at Data Centers”?

Kosik: I use this quite often when modeling energy use. Even though it was written around measuring power/energy in existing facilities, it can be used as guidance in the development of energy use and PUE of a not-yet-constructed data center.

Gillick: We have commissioned a 100,000-sq-ft data center design based on The Green Grid guidelines, and we have used these for measuring and reporting data center efficiency. One of the challenges this industry faces today is agreeing on the elements that constitute PUE—not all owners “do the math” in the same way—so we welcome these recommendations if they move the industry toward greater consensus and standardization.

Wesemann: It is the most common unit of measuring data centers in design discussions today. Most engineers and data center owners/users are familiar with the term, and PUE is one of the criteria that is targeted in the initial planning and design of a data center. It seems as though there is an unofficial competition out there to achieve the lowest PUE.

CSE: How have sustainability and PUE requirements affected how you approach electrical/power systems in data centers?

Gillick: As owners seek greater energy efficiency and lower PUE in their data centers and the new systems become increasingly efficient, we have observed that they are beginning to review and, in some cases, shorten their end-of-life projections for their existing electrical power and UPS systems We have also noted increasing deployment of air- and water-side economizers on mechanical systems, which has increased efficiency and lowered PUE. From a design review perspective, we try to ensure that the owner is weighing both the benefits and costs associated with trying to achieve greater energy efficiency and lower operating costs.

Lane: The use of more efficient UPS systems, modular implementation, transformerless UPS systems, more efficient transformers, more efficient lighting and lighting controls, warmer cold isle temperature, and hot isle containment. Building with modular components also ensures that electrical components are loaded at levels that run more efficiently. For example, some 2N data centers that are lightly loaded could have loading percentages during normal operation that are less than 20%. UPS systems and transformers can run far less efficiently when loaded to 20% or less.

Kosik: There is a much stronger focus on leveraging reliability through the use of modular electrical system design to make systems much more efficient, especially at low loads. 

Wesemann: This has become a major consideration that enters into the selection of most equipment, in particular the UPS systems and transformers. We are also seeing more metering being requested to track the actual energy usage to verify that it meets design expectations, and also tout the sustainability of a data center as it has become a marketing point for many data centers. Some clients are careful not to let energy efficiency override reliability. For example, the “offline” UPS systems that are offered to save energy are rarely implemented because data center operators don’t want the risk of the transfer to the inverter/rectifier when power fails. However, owners who have accepted this risk reap significant cost savings in reduced energy bills.