Case study: Financial call center

As more industries deem their daily operations to be mission critical, upgrading existing electrical distribution infrastructure will take on a vital role.

By John Yoon, PE, LEED AP ID+C, McGuire Engineers Inc., Chicago November 18, 2013

For today’s customer service-oriented industries, securing data isn’t the only mission critical objective—maintaining the ability to interact with the customer is now mission critical as well.

At one national financial institution’s 150,000-sq-ft call center located in a western suburb of Chicago, Ill., supporting five floors full of responders is just as crucial to its operations as maintaining its computers. So, when severe power quality issues stemming from the local electrical utility led to a series of brownouts and blackouts that were deemed unacceptable by the financial institution, an electrical distribution system upgrade was in order.

McGuire Engineers created a basis of design document for the call center to determine what it would take to keep this line of business in operation if the utility would fail. This required a broad understanding the facility’s load profile, much of which was derived from historical data from existing half-hour interval demand meters installed by the utility, to determine the amount of uninterruptible power supply (UPS) and generator capacity that would be needed to support that load.

Beyond gathering the hard data, the engineering team also considered the fact that while ensuring power reliability on a typical day is crucial to the call center, ensuring it during a natural disaster may not be. This is because the function of the facility requires responders to be present for operations, and in the event of a significant natural disaster, responders likely wouldn’t be able to access the facility either.

This led to the abandonment of a design that specified 2 MW generators with paralleling switchgear. Instead, a second, redundant medium-voltage (MV) substation utility feed and an automatic throw-over (ATO) switch were incorporated into the design, allowing the facility to automatically switch from one utility feed to another as needed. Functioning similarly to an automatic transfer switch (ATS) but on the utility side, the ATO turned out to be both more economical and appropriate for the call center’s required level of reliability.

Once the backup power source was in place, it was determined that five (one for each floor) 160 kVA UPS systems would be needed to create an uninterrupted power supply when transitioning from one utility feed to the next. This would help the call center maintain the desired customer experience, creating no opportunities for a loss of data or dropped calls during a power source transfer.

Because the established call center was already fully functional and has long hours of operation (early morning to late evening), the biggest challenge was conducting the upgrade without disrupting the line of business. While in an ideal world McGuire Engineers would have right-sized the UPS, segregating the most critical equipment only for backup, this would have required a rewiring and rerouting of the facility, which would have necessitated a complete system shut-down. Instead, to minimize the level of interruption to the facility, each floor in its entirety, including the lighting, was put on the floor’s local UPS.

Because it wasn’t feasible to provide a N+1 or 2N redundant UPS design, each UPS became a single point of failure. Therefore, the engineers specified an external maintenance bypass cabinet (MBC) for each UPS, which typically gets value-engineered out of projects. However, at the call center, it allows maintenance personnel to take the UPS off line for routine maintenance or even at the end of its life, for replacement, while still maintaining power to the critical load. Note the external MBC was specified as opposed to a dual-input UPS because the latter often provides a false sense of security, as a catastrophic failure within the dual-input UPS still acts as a single point of failure.

Each floor’s UPS and external MBC are housed in dedicated rooms away from call center responders and can be accessed only by maintenance personnel. Finding these rooms was a challenge, as it’s common for a UPS system to weigh 200 lbs/sq ft or more, while most areas within a building are structurally designed to hold only up to 100 lbs/sq ft. Typically, the places that are most able to accommodate such heavy weights are where the building has structural steel, right at the column lines. The design team worked with a structural engineer to identify key locations for UPS system so as to ensure safety and not disrupt the function of the facility. While this seclusion is crucial to maintaining the UPS systems, it also presents a challenge for the facilities operator: How will a UPS failure be recognized?

Two levels of remote monitoring were established. The first, a remote point monitoring system with simple dry contact alarms, sends an e-mail to the facility manager letting him know that something is wrong with one of the UPS systems, without any further details. Once the facility manager gets the initial alert, he can log onto the second level of monitoring, a network interface card that hosts a Web page for each UPS on the building’s intranet. This will allow the facility manager to drill down through all the parameters on the UPS to figure out exactly what’s wrong. Building personnel can therefore access all UPS systems remotely and after hours, without having to be in the building itself.

John Yoon is a senior electrical engineer with Chicago-based McGuire Engineers and has been designing and implementing electrical distribution infrastructure solutions for higher education, hospitality, and Fortune 500 corporate clients for 19 years.