Data center power strategies

11/17/2013


415/240 Vac distribution

Figure 4: In a 415/240 Vac power distribution system configuration (top), the primary goal is to gain efficiency by eliminating the transformer losses associated with the PDU and by allowing the IT loads to operate more efficiently at a higher voltage. InA power distribution strategy that is becoming more widely used in the data center is 415/240 Vac. This strategy eliminates the PDU and distributes power at the higher voltage form the UPS straight to the server cabinet. The primary goal is to gain efficiency by eliminating the transformer losses associated with the PDU and by allowing the IT loads to operate more efficiently at a higher voltage (see Figure 4).

In North America, the standard power distribution system is set up in a “wye” configuration with a phase-to-phase voltage of 208 V and a phase-to-neutral voltage of 120 V. In Europe the standard power distribution system is set up in the same “wye” configuration but with a higher voltage distribution. The phase-to-phase voltage is 415 V and the phase-to-neutral voltage is 240 V.

In an effort to standardize between North America and Europe, IT power supplies were developed to accommodate a range of voltages of 100 to 240 V. The concept behind this power strategy is to push the IT power supply to the high side of its voltage range (240 V) and use an established European voltage.

Advantages:

  • Energy efficiency (5% to 7% reduction in losses)
  • Reduced load on the cooling systems
  • Increased reliability
  • Smaller feeder and branch circuit conductor sizes to deliver the same amount of power
  • Gain white space in the data center (two cabinets per PDU eliminated)
  • Reduced maintenance costs (PDU and mechanical systems)
  • Power distribution equipment is readily available.

Challenges:

  • Higher levels of available fault current
  • Potential for arc flash requires higher levels of personal protective equipment (PPE) to work on equipment
  • Full neutral conductor required throughout the system
  • Harmonic influences on the rest of the system.

The main challenge with a 415/240 Vac distribution system is the high levels of available fault current. Removing the PDU from the system also removes the transformer impedance that limits the available fault current downstream in the data center.

Therefore, it is recommended that a short circuit analysis be performed early in the design to determine the available interrupting current (AIC) rating of all electrical equipment and to ensure the equipment is capable of withstanding the higher interrupting current. One option to consider when designing a 415/240 Vac system is breaking up the distribution system into smaller, more modular pieces. By using smaller high-impedance substation transformers, the engineer can reduce the overall fault current on the entire system. Another option to consider is the use of current limiting devices. Since current limiting devices tend to have quick reaction time, it is also recommended that a coordination study be performed to verify that reliability of the system has not been affected.

480/277 Vac distribution

The 480/277 Vac power distribution strategy is similar to the 415/240 Vac in that it eliminates the PDU and distributes power at a higher voltage straight to the server cabinet. The primary goal, advantages, and challenges of the 480/277 Vac power distribution strategy are exactly the same as the 415/230 Vac power distribution strategy (see Figure 5).

Figure 5: The primary goal, advantages, and challenges of the 480/277 Vac power distribution configuration (shown here) are exactly the same as the 415/230 Vac power distribution strategy. Courtesy: Jacobs

A major disadvantage of the 480/277 Vac power distribution strategy is that 277 V exceeds the 240 V rating of most IT equipment power supplies. Implementation of this strategy requires the purchase of custom-made servers with power supplies designed to operate at 277 V. For this reason, the 480/277 Vac power distribution strategy is not as prevalent as the 415/240 Vac power distribution strategy. Currently it is only used in very large facilities where the energy savings outweigh the cost of custom servers due to the high volume of servers that are purchased.

600 Vac distribution

The 600 Vac power distribution strategy is based on using the standard Canadian voltage of 575/347 Vac. Power is stepped down to 600 Vac at the substation transformer and distributed to the UPS system. Power then is distributed from the UPS system at 600 Vac to a PDU located near the data center. At the PDU the voltage is stepped down to either 415/240 V or 208/120 V and distributed to the IT equipment (see Figure 4).

Advantages:

  • Reduction in copper cost (smaller equipment buses and smaller feeders to deliver the same amount of power)
  • Use the full rating of 600 V electrical equipment
  • Lower available fault current (PDU transformer impedance).

Challenges:

  • No gain in efficiency (PDU transformer losses)
  • No gain of white space in data center
  • No reduction in maintenance costs.

Although the 600 Vac distribution strategy does not eliminate PDU transformer losses or reduce maintenance costs, it can lower initial capital expenditure costs. A 600 Vac system takes advantage of the reduced current at higher voltages resulting in smaller or less conductors. Using smaller or fewer conductors will decrease that amount of copper and reduces cost. Higher voltage also allows for larger substations. Depending on the size of the data center, using larger substations may result in a reduction in the total number of substations required.  

380 Vdc power

Figure 8: This building information modeling (BIM) rendering is of a high-density data center floor. The data center was configured with isolated redundant UPS systems and used 415/240 V distribution to the cabinets. Courtesy: JacobsContrary to common belief, dc power is very common in the world today. The telecom and transportation industries have been using dc power for years. Alternative and renewable energy generation sources such as solar power, wind power, and fuel cells are dc-based power sources. Most electronic devices in residential homes and in offices internally operate on dc power. And, most importantly, energy storage devices such as batteries and UPS systems operate on dc power.

When you look at a typical traditional data center distribution system, the power gets rectified from ac to dc, inverted from dc to ac, transformed from 480 Vac to 208 Vac, rectified again from ac to dc, and then transformed down to 12 Vdc before powering the IT equipment. Every time the power is converted, losses occur in the form of heat resulting in a decrease in energy efficiency.

The 380 Vdc power distribution strategy distributes dc power from the UPS (dc rectifier) straight to the IT power supply. The primary goal is to gain efficiency by eliminating the inverter losses in the UPS, the rectifier losses in the IT power supply, and the transformer losses associated with the PDU (see Figure 4).

Advantages:

  • Energy efficiency (8% to 10% reduction in losses)
  • Reduced load on the cooling systems
  • Increased reliability
  • Smaller physical footprint
  • Integrates with alternate energy sources
  • Reduced maintenance costs.

Challenges:

  • Limited knowledge and difficult to find electricians with experience on dc systems
  • dc current does not have a zero crossing, difficult to extinguish the arc
  • Have to account for voltage drop on the positive and negative feeders
  • dc arc flash hazards (NFPA 70E provides guidelines for dc arc flash protection).

In addition to the limited number of electricians with dc power experience, the main challenge with dc power in the past has been the lack of standards. This, however, is starting to change. Both the European Telecommunications Standards Institute (ETSI) and the EMerge Alliance have standardized on 380 Vdc and produced guidelines for dc power distribution.  

Figure 6: This shows the legacy 208/120 Vac power distribution system in a single line diagram. Courtesy: JacobsUnless the data center is completely powered by an alternate source of power, such as fuel cells, it is most likely being provided ac power from the utility. In a dc power system the UPS is used to rectify the power from ac to dc. Because the distribution to the data center is dc, any bypass of the UPS system will also need a rectifier. Consequently, dc systems are more cost effective in a fully redundant (Tier IV) system where a second UPS (dc rectifier) is used as the bypass. Additional things to be aware of when designing a dc power distribution system include using proper protection devices rated for use in dc systems and following the specific requirements for a dc grounding system (refer to IEEE Standard 1100-2005 – IEEE Recommended Practice for Powering and Grounding Electronic Equipment).

In an effort to increase efficiency and reduce cost, different power strategies for distributing power to the data center are starting to be used. Whether you are planning to update an existing data center, expand an existing data center, or build a new data center, designing the power distribution system is a critical part of the plan and one that must be evaluated to determine which system is the correct system for the application.

Theoretical case study

Figure 7: The 380 Vdc power distribution system single line diagram shows the redundant UPS (dc rectifier) as the bypass and did not include a separate bypass on each of the UPS (dc rectifier) systems. Courtesy: JacobsThe two power strategies for distributing power to the data center that seem to be gaining the most popularity include the 415/240 V higher ac architecture and the 380 Vdc architecture. A theoretical case study was performed by Jacobs-KlingStubbins to compare the capital expenditure (CAPEX) and operating expenditure (OPEX) of these two power distribution strategies against the typical 208/120 V data center. The case study was based on a theoretical simplified data center with 2 MW IT load, 2 N redundancy (Tier IV), six 750 kVA UPS modules, and 30 5 kW cabinets per row.

The 415/240 Vac system had a 12% CAPEX savings and a 20% OPEX savings when compared to the legacy 208/120 V data center. The 380 Vdc system had a 14% CAPEX savings and a 28% OPEX savings when compared to the legacy 208/120 V data center. It should be noted that unlike the legacy and the 415 Vac systems, the 380 Vdc used the redundant UPS (dc rectifier) as the bypass and did not include a separate bypass on each of the UPS (dc rectifier) systems.


Kenneth Kutsmeda is an engineering design principal at Jacobs (KlingStubbins) in Philadelphia. For more than 18 years, he has been responsible for engineering, designing, and commissioning power distribution systems for mission critical facilities. His project experience includes data centers, specialized research and development buildings, and large-scale technology facilities containing medium-voltage distribution.


<< First < Previous 1 2 Next > Last >>

LEONID , MD, United States, 12/31/13 12:28 PM:

In my opinion, 120 volt power branch circuits should be eliminated in all industrial installations and large IT centers. Three phase 380/220 VAC and/or single phase 480/240 VAC circuits should be used instead.
Consulting-Specifying Engineer's Product of the Year (POY) contest is the premier award for new products in the HVAC, fire, electrical, and...
Consulting-Specifying Engineer magazine is dedicated to encouraging and recognizing the most talented young individuals...
The MEP Giants program lists the top mechanical, electrical, plumbing, and fire protection engineering firms in the United States.
Water use efficiency: Diminishing water quality, escalating costs; Lowering building energy use; Power for fire pumps
Building envelope and integration; Manufacturing industrial Q&A; NFPA 99; Testing fire systems
Labs and research facilities: Q&A with the experts; Water heating systems; Smart building integration; 40 Under 40 winners
Maintaining low data center PUE; Using eco mode in UPS systems; Commissioning electrical and power systems; Exploring dc power distribution alternatives
Protecting standby generators for mission critical facilities; Selecting energy-efficient transformers; Integrating power monitoring systems; Mitigating harmonics in electrical systems
Commissioning electrical systems in mission critical facilities; Anticipating the Smart Grid; Mitigating arc flash hazards in medium-voltage switchgear; Comparing generator sizing software
As brand protection manager for Eaton’s Electrical Sector, Tom Grace oversees counterfeit awareness...
Amara Rozgus is chief editor and content manager of Consulting-Specifier Engineer magazine.
IEEE power industry experts bring their combined experience in the electrical power industry...
Michael Heinsdorf, P.E., LEED AP, CDT is an Engineering Specification Writer at ARCOM MasterSpec.