Web Hosts Call for Warp Speed

Internet hotels and Web-hosting centers are springing up in unusual places and are being built under nearly impossible schedules. The need for these facilities has been spurred by the explosion of Internet-service providers and e-commerce companies that require new facilities to maintain a competitive advantage.

By Peter Oliveri, P.E., Principal, Einhorn Yaffee Prescott Mission Critical Facilities Group, White Plains, N.Y. June 1, 2001

Internet hotels and Web-hosting centers are springing up in unusual places and are being built under nearly impossible schedules. The need for these facilities has been spurred by the explosion of Internet-service providers and e-commerce companies that require new facilities to maintain a competitive advantage. Competition in this market has been fierce, with reliability and speed being the key selling points. Speed to market has also become a requirement, with design and construction schedules measured in weeks rather than months.

A global telecommunications firm constructed a new 100,000-square-foot Web-hosting center in New York. The project was typical of demands made on designers and constructors to finish projects in record time. Located on six upper floors of a high-rise building, the $65-million center was completed in two phases.

Phase one was completed in 10 weeks and involved the complete fit-out of 20,000 square feet of Web-hosting space on two floors and the addition of all new supporting infrastructure. The remaining four floors—with a total 30,000 square feet of Web-hosting space—were completed over a nine-month period.

There were several challenges in meeting the first phase of the schedule:

  • Locating essential equipment.

  • Marshalling manpower resources.

  • Keeping the rest of the building operational.

  • Finding a way to shoehorn new infrastructure in the basement.

  • Bringing backbone power cabling to the upper floors.

The project’s electrical infrastructure included: 13,200 amperes (A) of utility service capacity; 8 megawatts of diesel generators with paralleling switchgear; three primary 1,500-kilovolt-amperes (kVa) uninterruptible power-supply (UPS) systems and a 2,000-kVa N+1 distributed-redundant “catcher” UPS system. In addition, there are four automatic-transfer switchgears, two double-ended substations, which are the focal point of power distribution to the center’s critical backbone, and a 2N redundant chiller plant. The new electrical and mechanical infrastructure occupies approximately one-half of the center’s total square footage.

The infrastructure was designed to provide extremely high uptime service with an availability of 99.999 percent. The project was designed to deliver zero tolerance to the loss of operations. Because the entire infrastructure was new, the installation of utility-service disconnect switches and various levels of redundant automatic-transfer and double-ended switchgears was necessary.

Dual path architecture had to be inserted into a functioning building to avoid interrupting the existing building services. The electrical and mechanical infrastructure was developed to achieve high tolerance to faults and to allow for concurrent maintenance. The new infrastructure was designed to handle 90 watts per square foot of Web-hosting and data-processing load and 90 watts per square foot of the heating, ventilating and air-conditioning (HVAC) load. The entire infrastructure was phased and installed in sections to allow for the testing, commissioning, acceptance and transfer of load onto an active system without disrupting the on-line clients.

Locating Equipment

Finding key equipment for the first phase of the work proved to be a challenge. While much of the equipment had been ordered on the first day, because of the long lead times that would be required for delivery of desired equipment, designers made compromises. Normally, designers would have used more reliable wet-cell batteries for the UPS system but the lead time of 52 weeks was unacceptable. So they turned to dry seal-type batteries that could be delivered within the tight schedule.

Two key components were a diesel generator and a three-module parallel UPS system. The initial 2,000-kW generator was flown in from Singapore. Prior to on-site delivery, the generator was converted from European voltage and frequency ratings to standard U.S. 480/277V, 60-Hz ratings. The generator was on-site within 30 days. The UPS system was manufactured, factory tested and on-site within four weeks.

During the first phase, paralleling switchgear and other essential components were not available. Therefore, one diesel generator was connected directly to the automatic-transfer switchgear to provide standby power in the event that utility power was unavailable while the remainder of the project was completed. In addition, a double-ended switchgear was connected to allow for a secondary point to distribute power to the initial UPS system, computer-room air-conditioning (CRAC) units and the automatic- static-transfer switch/power distribution units (ASTS/PDU) necessary to operate this initial phase of the Web-hosting space.

Marshalling Resources

To meet the tight phase-one schedule, more than 150 electricians were needed around the clock. However, there was a severe shortage of electricians in the labor pool, and a commitment for this manpower was successfully negotiated with New York City’s largest electrical contractor. With a strong game plan from the construction manager in place, a commitment from the equipment manufacturers for on-time deliveries and the fullest cooperation from the building manager, the project was heading in the right direction. Without the commitment of all interested parties, the project could not have been completed on time.

Space Constraints

Bringing new feeders into an already congested basement, as well as bringing the backbone up to the six upper floors, presented a major challenge. The base building infrastructure, which was situated in the basement, consisted of four 5,000-A utility-service disconnect switches. The new Web-hosting infrastructure required an additional two 5,000-A and two 1,600-A utility-service disconnect switches. Phase one required a single 5,000-A disconnect switch to feed the double-ended switchgear, which in turn fed the UPS system and associated critical distribution panels.

Due to space constraints in the basement, several existing utility kilowatt-per-hour (kWh) demand meters needed to be relocated. This required a power shutdown from the utility company to accommodate the installation. However, the building manager only allowed a six-hour window to perform this work. Therefore, the existing telephone switch portion of the building was placed on standby-generator power and several lengthy utility power shutdowns were conducted in order to relocate the existing kWh meters. In addition, the 5,000-A feeder tap that was needed to feed the new service switch was enclosed in concrete to meet local codes.

A decision made early in the project to bring the backbone power feeders up through two elevator shafts resolved the issue of riser-space access. The building is serviced by six high-rise and six low-rise elevators, one of each was taken out of commission to serve as a shaft for these feeders. The new utility feeders were routed in a pancake-like fashion through the basement up to the automatic-transfer switchgear level, rerouted over to the decommissioned elevator shafts and run up to their respective floors.

Maintaining Existing Services

Much of the rest of the building serves as an existing long-distance telephone-switch hub. It was critical throughout the project to protect these systems from disruption. Since the maintainability of the telephone-switch portion of the building was crucial to the functioning of the entire country’s telephone system, it was decided at the early part of the project to completely divorce the Web-hosting infrastructure from the telephone backbone. In the Web-hosting center, independent utility-service switches, generators, UPS systems, distribution panels and associated cooling equipment were installed to serve the hectic, fast-track world of Web hosting.

Redundancy Level

While redundancy has increased in many Web-hosting facilities, the dual-path redundancy designed into this facility was a bit unusual. The client was concerned with having the capability of maintaining and testing all equipment installed to operate this center. The dual-path philosophy was used throughout, from the utility-service switches to the redundantly fed dual outlets in each of the customer racks.

Each one of the two automatic-transfer switchgears can handle the entire critical UPS load without disrupting any one of the on-line clients. Each of the three primary UPS systems feed the preferred side of the ASTS/PDU, which in turn feed the customers through remote panels. The alternate side of the ASTS/PDU is powered by the N+1 distributed-redundant catcher UPS system. In the event of a primary UPS failure, all the affected ASTS/PDUs will transfer to the N+1 redundant UPS system, which will then carry the critical load. This allows the redundant UPS system to act as a backup, in the event any one of the primary UPS systems fail. This N+1 system also allows for redundancy during annual maintenance.

The cooling-system CRAC units that serve the Web-hosting center are redundantly fed with dual power feeds from independent distribution panels. All the power feeds are terminated within automatic-transfer switches located within the CRAC units. In the event of a feeder failure, the transfer switch would automatically transfer to its alternate source and feed the operating CRAC units. This design approach assures the center of 24×7 operation without the fear of losing their customers load due to overheating, which would result if the cooling system is lost for a short period of time at 90 watts per square feet of heat load.

Speed has become a given in the Web world: connection and data acquisition speed, as well as speed to market with new facilities. Flexibility, adaptability and creativity have become important tools for getting up to speed.

From Pure Power, Summer 2001.