Direct Current Do’s and Don’ts

Electrical engineers and researchers debate the pros and cons of direct current (DC) vs. alternating current (AC) for mission-critical data centers.

CSE: Why even consider DC power for a data center?

SULLIVAN : Efficiency is the driving force. Performing the power conversion from AC to DC once for the entire data center, as opposed to at each server, enables a more efficient power conversion.

TSCHUDI : We’ve shown that 10% to 20% savings are possible when compared to typical systems. But even more importantly, the first cost should be lower for a DC system, because there are fewer components, which means fewer potential points of failure and more reliability. Furthermore, eliminating the AC feed solves any power quality issues. Since the AC to DC conversion inside the server goes away, the power supply can be made smaller, and there will be less heat inside the server.

GODRICH : When the relatively high cost of operating mission-critical buildings is analyzed, the surprising conclusion is that as much as 30% to 50% of the overall expenses are energy costs.

For example, a test done by Intel shows that in order to supply the power and environment required by a 1U AC-input 300-watt server, 973 watts of power are required. Having determined that the CPUs are ultimately using voltage-regulated modules, power under 2 volts DC supplied through a DC/DC multi-output switcher and/or DC/DC converters, the most efficient solution might be supplying DC power directly to the server racks. At the same time, it would be a huge mistake to think that the server or rack level is the end of the story. The most important thing is to include the DC approach into an end-to-end electro-mechanical approach.

LOVORN : I have to disagree about the cost. While some nominal operational cost savings have been reported through the use of DC, the calculations that I have reviewed do not take into consideration the actual losses of modern uninterruptible power supply (UPS) systems, which have much higher efficiencies than that used in the analyses. In addition, the calculations do not consider the substantial losses characteristic of DC distribution systems. By the time one adjusts the numbers for operating costs and then takes into consideration the cost of the massive conductors required to distribute the DC power around a computer room of any size, I do not believe the cost savings gained in efficiency will ever justify the added distribution costs.

CSE: How exactly does a DC power scheme work?

TSCHUDI : Power typically enters the data center at 480 volts AC. In the DC system, it’s converted to 380 volts and then sent directly into the server. The server’s first stage conversion is bypassed, allowing the DC to directly arrive at a point in the server that normally sees 380 volts DC. A flywheel UPS is connected to the 380-volt DC distribution to provide the capability to ride through disturbances or disruptions in the utility power.

LOVORN : There is a single conversation from the incoming power into the data center to DC, at either 12, 18, 24 or 48 volts. The DC voltage is then distributed directly to the server racks, and individual DC voltage converters change the distribution voltage to the many DC voltage levels utilized in a server. When AC power is used in the servers, there is a multi-tap transformer in the server power supply that transforms incoming power into fully rectified AC, which is filtered and converted into the various DC levels for the server.

SULLIVAN : AC-to-DC conversion is done in place of the present static UPS systems. The DC power is then distributed to computer equipment cabinets and subsequently to individual servers, which eliminates the AC-to-DC power supplies in the servers. At the same time, there is still DC-to-DC conversion within the servers to go from distribution DC voltages to operating DC voltages.

GODRICH : There are various configurations available for different environments and applications. For example, Google, several weeks ago, began urging manufacturers to go to 12-volt DC schemes at the server level. Some manufacturers are very comfortable with this approach, but the end-to-end concept remains for the consulting engineers.

The -48-volt DC approach is still one of the most popular, due to the fact that it is very safe and there is a vast amount of knowledge about these distributions from the telecom world, including a wide spectrum of standards. Availability of these systems is a problem—most were designed and operated for one failure in 20 years. The problem with this approach is the size of the system, which is limited to, say, 500 kW in tele-data applications due to distribution constraints. Even so, for small to medium data centers, this approach can be applied successfully.

Implementation of a higher voltage distribution, say in the range of 550 volts DC, might solve a lot of concerns. Using such concepts, and DC/DC converters at the row level, it is possible to create 3- to 5-MW power blocks to supply data center modules in a reliable and safe way.

Another range that server manufacturers are very comfortable with is 340 to 380 volts DC. Tests have been implemented successfully in the United States and Europe. ETSI even implemented standards for 300-volt DC direct to the rack distribution several years ago, although the solution didn’t evolve successfully. We believe that such direct DC feed to racks—not only to these servers, but spreading the range as much as possible under the NEC voltage limitation to take advantage of distribution costs—is a viable solution for relatively safe data center environments, but not for facilities with relatively high staff dynamics.

For a financial data center environment with up to 15,000 changes per month, plant safety concerns might be raised, but for a super-computer environment where once implemented it becomes a “machine-room” with minimum staff dynamics, this solution might be optimal.

CSE: What are the drawbacks of DC when compared with AC power?

GODRICH : DC cannot be implemented easily in existing AC data centers. The advantages are mostly for new facilities. That said, small to medium applications have been successfully implemented with proven economic advantages.

But due to the fact that the DC servers are not the most popular on the market, the price of their power supply options is more expensive at this point.

In addition, not all manufacturers have a wide range of DC equipment. While the 48-volt to 250-volt DC range offers a lot of options, in the range of 250- to 1,000-volt DC, there are only about four manufacturers in the market. There’s also a lot left to do in the area of protection relaying—to optimize the existing AC ranges to DC operation.

LOVORN : The primary drawback is the overwhelming cost of converting the existing distribution system, where all of the equipment is designed and installed on a conventional AC distribution system, as conversion to a DC system would require removing and replacing nearly all of the AC distribution, UPS equipment, PDUs, panel boards, server power supplies and everything else associated with power to the servers. Even in a new installation, the relative installed costs may not justify the extra expense for small operational cost savings.

SULLIVAN : The biggest concern is the size of the conductors needed to distribute the DC power. If standard DC voltages are utilized, buss bars, rather than wires in conduits, would be required to handle the high currents.

CSE: Are there existing applications that can attest to the efficiency of DC power for data centers?

SULLIVAN : A pilot demonstration project by Lawrence Berkeley National Labs (LBNL) this year, found a 10% to 12% increase in efficiency. Although at least 20% efficiencies were expected, they were not able to achieve that level with the equipment installed.

GODRICH : We actually designed, in the last year, the first multi-MW data center for high density, 300 to 600 watts per sq. ft. Implementation of such an approach might improve the end-to-end efficiency numbers to more than 25%.

Taking into account that a lot of elements are eliminated in the serial distribution, these configurations have about a 17% lower probability of failure in a five-year span than comparable AC configurations.

Companies like Rackable Systems, in Milpitas, Calif., have installed a number of applications across the U.S. with great success. Tests performed by LBNL, EPRI and Sun also look very promising. Most server manufacturers have equipment and are considering expanding their DC equipment range.

TSCHUDI : There are numerous systems that operate on high voltage DC current. In fact, the military uses DC in many applications and entire ships operate on DC. Also, transportation systems operate on DC and elevators in New York use DC power. Adding to the list, the telecommunications industry has a long history of operating on 48-volt DC in phone switching centers—which has proven to be extremely reliable.

CSE: What will it take for DC power to become a more accepted and widespread choice for data centers?

LOVORN : In my opinion, DC power distribution cannot overcome Ohm’s law. Therefore, I do not believe that it will ever be more widespread than some demonstration sites that show that it is not economically viable.

GODRICH : There has to be a chain reaction with a positive techno-economical impact for the client, with the first step being an understanding and requirement to design for higher efficiency end-to-end. Server manufacturers generally will price their equipment based on the market requirement such that the rack level will be more attractive. It will take time for DC power to approach the mainstream in data centers, but for some applications, it will get there sooner than expected.

TSCHUDI : It’s at the chicken-and-egg stage right now. At this time, pilot installations in operating data centers are being planned to help spark demand and allow the server manufacturers to start the certification process. Most likely a few early adopters will spark the interest of many data center operators. The server manufacturers are ready and watching the situation, so it will be relatively simple for them to move if demand is created.

SULLIVAN : As mentioned, the first step will be getting a “critical mass” of computer equipment manufacturers to produce products that will function on DC power. Until then, a data center that converts would have to operate two power systems. Next, users must be educated about the benefits of installing equipment using DC power. Finally, engineers and service personnel will have to be educated and trained to design and service DC power systems and DC operated computer equipment.

Participants

Kfir Godrich

Director of Technology Development

EYP Mission Critical Facilities, New York

Ken Lovorn , P.E.

President and Chief Engineer

Lovorn Engineering

Pittsburgh

Robert Sullivan

Senior Consultant

The Uptime Institute

Morgan Hill, Calif.

William Tschudi

Principal Investigator

Lawrence Berkeley National Laboratory

Berkeley, Calif.

Tech Support for DC Power

How can an end user be assured of finding a DC power expert to assist with maintenance and repairs? “All the major equipment manufacturers have been supporting DC power systems for years,” says Kfir Godrich, director of technology development with EYP MCF, New York.

“In reality, there is little to differentiate AC and DC from an installation point of view,” suggests William Tschudi, principal investigator with Lawrence Berkeley National Labs. “For starters, all of the equipment for high-voltage DC is UL-rated, which means that it has undergone a rigorous safety evaluation.”

But Ken Lovorn, P.E., of Lovorn Engineering, Pittsburgh, cautions that finding maintenance and repair personnel familiar with DC distribution systems of this scale might prove difficult. “While some of these people from the early telephone central stations are still around and would have the expertise directly applicable to the DC data center systems, most of the newer technicians in the telephone industry are used to seeing servers that are similar to those in a data processing center and, thus, would be no better off than the technicians that are now servicing the data center power distribution systems,” he says.

Do you have experience and expertise with the topics mentioned in this content? You should consider contributing to our CFE Media editorial team and getting the recognition you and your company deserve. Click here to start this process.