Analyzing data centers: automation and controls

Data is the lifeblood of any business or organization—which makes a data center a facility’s beating heart. Here, engineers with experience on data center projects show how to succeed on such facilities, and how to keep your finger on the pulse of data center trends in regards to automation and controls.

By Consulting-Specifying Engineer April 27, 2017


Respondents

  • Robert C. Eichelman, PE, LEED AP, ATD, DCEP, Technical Director, EYP Architecture and Engineering, Albany, N.Y.
  • Karl Fenstermaker, PE, Principal Engineer, Southland Engineering, Portland, Ore.
  • Bill Kosik, PE, CEM, LEED AP, BEMP, Senior Mechanical Engineer, exp , Chicago
  • Kenneth Kutsmeda, PE, LEED AP, Engineering Manager—Mission Critical, Jacobs, Philadelphia
  • Keith Lane, PE, RCDD, NTS, LC, LEED AP BD&C, President, Lane Coburn & Associates LLC, Bothell, Wash.
  • Brian Rener, PE, LEED AP, Senior Electrical Engineer, SmithGroupJJR, Chicago
  • Mark Suski, SET, CFPS, Associate Director, JENSEN HUGHES, Lincolnshire, Ill.
  • Saahil Tumber, PE, HBDP, LEED AP, Senior Associate, Environmental Systems Design, Chicago
  • John Yoon, PE, LEED AP, Lead Electrical Engineer, McGuire Engineers Inc., Chicago

CSE: How have you worked with the building owner or facility manager to implement the Internet of Things (IoT) into their facility management? Have you helped catalog every device in a data center, such as lights, fire alarms, electrical outlets, and other products?

Yoon: While a certain quantity of IoT devices is expected in any project, the extent that you describe is not something that we’ve ever been asked to perform. It seems to be something of an amalgam of data center infrastructure management on steroids and BIM. While it’s an interesting idea, what is the value proposition there? What would it do to increase availability for critical systems or push down operational expenses or PUE? Does the increased information provided by IoT devices justify the associated increase in device-management and security requirements? As an engineer, it’s exciting but currently hard to justify.

CSE: What are some of the challenges of incorporating the IoT into facility design for existing facilities or new construction projects?

Rener: Owners have security concerns with newer IoT devices even in non-data center environments. However, in data centers, the possibility of gaining entry through back-door facility IoT systems is a major concern.

Yoon: Security, security, security. Last year’s massive Dyn Distributed Denial of Service attack only served to confirm the deeply held fears that many in the IT industry have about IoT devices. This was not an isolated occurrence. Among others, Verizon’s 2017 Data Breach Digest detailed an attack on an unnamed university where a botnet used seemingly harmless IoT devices, such as vending machines and parking lot lights. That university’s post-mortem firewall analysis discovered that more than 5,000 discrete hosts on the segment of their campus network dedicated to IoT contributed to that attack. With these types of potential vulnerabilities, there is still significant resistance to connecting IoT devices to mission critical networks. While IoT has incredible potential, the primary question is how to address security concerns as IoT gets pushed into ever-greater quantities and types of devices. While high-value devices, such as smartphones, get regular firmware updates, will the same happen with the multitude of mundane commodity-level IoT devices that are coming to market (light fixtures, vending machines, etc.)?

CSE: When working on monitoring and control systems in data centers, what factors do you consider?

Fenstermaker: The complexity of systems employed, the quantity and location of dedicated facility operations staff, and overall project budget are three important considerations taken into account when we are developing recommendations for data center control systems, in addition to the required reliability and resiliency of the data center.

Eichelman: When designing monitoring and control systems, the specific operational needs and constraints of the facility need to be clearly understood. We ask questions including:

  • What is the clear purpose of these systems?
  • How do they need to be designed to best support the overall mission requirements?
  • What levels of concurrent maintainability and fault tolerance are appropriate?
  • What parameters need to be monitored to support the mission and, perhaps just as important, which do not?
  • How is that information to be best displayed, communicated, and analyzed to support the continued reliable operation of the facility?
  • What conditions are best controlled automatically, and which are best left to be controlled manually?
  • Should control and monitoring be converged into a single DCIM platform?

Data centers are routinely provided with very robust monitoring and control systems. Critical points throughout the electrical and mechanical plant are continuously monitored, giving operators the ability to quickly understand system status, rapidly recognize and respond to pre-alarms and alarm conditions, and analyze the demand and energy consumption to ensure equipment and systems are operating efficiently and within design limits.

Rener: On the monitoring side, we carefully review with the owner the levels of power monitoring outlined in The Green Grid standard to determine PUE. We believe it’s important to capture losses in all facility systems, including downstream PDUs, and directly measuring IT-load power use. The real benefit to monitoring is trending and logging the data for diagnostic purposes. We’ve implemented a program where we receive the data from the facilities management group and review the performance of the data center to help optimize the energy use, especially at low- and part-load conditions.

Tumber: It is important to have well-defined project requirements, as they will influence the design of the building management system (BMS). The BMS should be fully integrated and incorporate energy management, equipment monitoring and control, alarming, reporting, data collection, trending, and self-diagnostic capabilities. It should be resilient so that failure of any single component or network connection does not impact its operation or the critical loads. Attributes such as scalability, expandability, responsiveness, and flexibility are also important. It is essential that the BMS be strategically engineered similar to other systems in the data center, and the design should not be delegated to the controls contractor.

CSE: What types of system integration and/or interoperability issues have you overcome in data centers, and how did you do so?

Yoon: While specifying the parts and pieces is relatively straightforward, the primary challenge is managing the exchange of data between equipment from different manufacturers. While each manufacturer will claim a certain amount of functionality when interacting with their own equipment, it’s always a challenge getting different manufacturers to play nice with each other if you want anything more elaborate than simple status and supervision control. We’ve encountered significant coordination issues with systems where multiple manufacturers are providing hardware and software like networked power quality and revenue-metering systems. The engineering drawings and specifications may clearly state the required look, feel, and functionality of the final product, but the integration process will often run amok if there isn’t a dedicated systems integration contractor.

CSE: How has the convergence of automation and controls affected the design of a data center or data closet within a building?

Eichelman: With the increasing convergence of IT systems with the physical plant, DCIM systems are becoming more common and necessary. These systems allow the IT assets, as well as the physical plant that supports the IT environment, to be monitored, managed, and controlled from a single integrated system overlay. DCIM systems provide the ability to optimize the operation of the IT equipment in conjunction with the electrical and mechanical system to ensure the continued reliability of the facility while minimizing energy consumption.