Investing in Power: Emergency Backup at an Investment Firm
When you’re responsible for more than 10 million shareholders and 172 institutional clients, you can’t afford to blink. With more than 3,500 employees and six computerized service centers in Massachusetts, London, and Tokyo, Putnam Investments has invested in emergency backup power systems for all its facilities to assure continuity of service to its customers.
Putnam is one of the largest money-management companies in the world, offering mutual funds retirement plans, college savings plans, insurance products and institutional portfolios.
As the company grew in the late 1980s and early 1990s, Putnam learned about the differences in backup power systems firsthand. Chronically malfunctioning switchgear installed in 1986 at the company’s first data center led management to reconsider its choice of equipment when planning the backup system for its headquarters in Boston’s Post Office Square. Installed in 1990, this new system included a single 800-kW generator and three open-transition automatic transfer switches manufactured.
These equipment changes were the result of a scoping process in which Putnam executives and S.B. Sager Associates, the system designer, investigated not only the equipment but also the capabilities, reputation and performance history of various system suppliers.
“[The systems we finally specified] were from one of three firms we looked at,” says Barry Mosher, Putnam’s project engineering manager. “We knew what we wanted the equipment to do, and we compared the failure histories, service records and so on.”
Putnam’s concerns included more than inconvenienced customers and reduced employee productivity. The company was concerned about the outright loss of electronic data as well as its ability to conduct routine daily tasks. Large numbers of checks and statements have to be processed on time. Other documents and publications have to be printed and mailed on schedule. Putnam sells all its retail funds through financial advisors, so there are large numbers of inbound calls and e-mail messages from shareholders, financial advisors, and banks in addition to outbound communications from Putnam’s marketing and support staff.
To keep up with its rapid growth, Putnam built new facilities in three Boston suburbs, including Franklin and Andover, between 1994 and 2000. Designed for each building and installed during construction, a 3,000-kW backup power system includes three generators and closed-transition paralleling switchgear. The backup power system at Putnam’s headquarters was upgraded to 1,500-kW and supplemented with a web-based monitoring system that captures data (even from the London and Tokyo facilities) for analysis later.
The power control systems in all four Putnam facilities have redundant programmable logic controllers (PLCs), which coordinate the systems and can switch a facility off the grid to generator power whenever there is a problem. The controls constantly monitor the utility feed.
In the event of an unforeseen power loss at any of the three suburban facilities, closed-transition switchgear will automatically start the generators, parallel their outputs and pick up loads in order of priority. While this entire process takes only about 20 seconds, that is still long enough for computers to crash. With so much at stake, Putnam also invested in uninterruptible power supply (UPS) systems to prevent even the slightest power blink.
Because data centers need absolutely pure power, Putnam is taking no chances. The suburban power lines that feed these centers are strung between poles and are subject to damage from storms and motor vehicle accidents. To smooth out any irregularities or anomalies in the power coming into its three suburban facilities, Putnam runs the utility feeds through its UPS systems. This conditions the utility power and eliminates the damaging effects of harmonic distortion and brownouts.
As with the company’s disciplined scoping process, Putnam’s care is evident in its precautionary pre-storm transfer protocols. Although the reliability of the utility feed to Putnam’s Franklin facility has since improved, from 1994 to 1999 it failed an average of 14 times per year, with outages averaging 2-3 hours each. So today, whenever a thunderstorm, blizzard, or ice storm is predicted for the suburbs (4-5 times a year), maintenance personnel use the equipment to parallel the utility feed and the emergency generators and to transfer to emergency power before the storm hits. Each system has enough fuel to run for 10-12 days if necessary. In Boston, however, most power lines are underground, utility power is more reliable, and outages are rare, so such precautions are unnecessary.
Because the closed-transition system design transfers each facility’s loads seamlessly from utility feed to backup generators, Putnam can test the systems on a regular basis without the slightest impact on operations. In addition to the pre-storm transfers, Putnam exercises each of its suburban systems twice a month, during a workday, for four hours. Every quarter they test them for eight hours. Neither customers nor employees ever notice these tests.
Finally, one last major point in the equipment vendor’s favor was that it supports its customers with 7x24 field service. Mosher had heard about this, but in 1991 when one of his transfer switches developed a problem, the vendor’s response made him a true believer.
“I had to shut the building down that weekend,” he recalls. “If I couldn’t fix the switch, we’d be out of business. I called the vendor’s toll-free field service number and was told the engineer in charge would be paged. He called back a few minutes later from his daughter’s wedding! He understood the jam I was in and said he would be there in about half an hour. He was still in his tuxedo when he showed up. He quickly fixed the switch and checked the other switches. I was so impressed! And I never even got a bill.”
For more information about switchgear and automatic transfer switch equipment from Russelectric, click here .