Further Grounding Points
Oftentimes, the practical approaches to building grounding systems are more complicated than the principles that explain them.
As discussed in Part I, the principles of single- and multi-point grounding can be applied to grounding systems that are effective for various building types. However, a building may have multiple grounding subsystems. They all must work independently of each other, while being tied to the same voltage reference point. The task for the electrical engineer becomes joining these various systems together while keeping in mind the prevailing grounding principles of safety and equipment operation.
In a mission critical building, a hybrid approach—using the fundamentals of both systems—is normally effective. The grounding systems most frequently seen in data center construction are: 1) the NEC equipment safety ground, 2) a lightning protection system and 3) a signal reference grounding (SRG) system.
Environment and Structure
When designing a grounding system, the building’s environment, construction and functions must be taken into account. For instance, a data center in a concrete high-rise in Florida would be treated different from a three-story steel frame office building in Pennsylvania.
Weather, surrounding soil and structural materials are also very important in determining the grounding system.
Soil composition and electrical storms are two major environmental factors that influence the ability to equalize a building’s potential to earth. The building’s structural members and/or grounding electrodes are in contact with the soil. In this regard there are three important electrical considerations: 1) the impedance of the grounding electrode, 2) impedance at the contact point between an electrode and soil and 3) the impedance of the soil itself. Of these, the soil’s impedance has the greatest impact. Gravel type soil has a much higher resistivity than a sandy, tightly packed soil (Refer to IEEE Std. 142, Table 10). High moisture content and temperature will decrease a soil’s impedance, making it more efficient at dissipating grounding currents. However, a high fault current can evaporate the moisture and increase the resistivity of the soil around electrodes.
In many cases, a steel frame building can rely on steel columns set in concrete footings as grounding electrodes. This can be an effective method, since concrete has a lower resistivity than most soils, allowing currents to travel from the building steel to the footing, then radiate to surrounding soil.
Usually, steel frame buildings have multiple columns and footings creating multiple current paths, thus decreasing the building’s overall impedance. This method is economical and satisfactory for most commercial buildings, but depends on secure connections between anchor bolts, footings and structural members. It also requires close coordination between the electrical and structural engineers; introduces the possibility of circulating currents on the grounding system; and relies on the impedance of the surrounding soil to be less than the impedance of the building to properly dissipate energy.
Concrete structures have similar advantages to steel frame buildings. They can use rebar in concrete members to equalize the building’s potential to ground. Steel rebar normally provides less resistance than driven ground rods (less than 1 ohm) if implemented correctly. A continuous concrete foundation with imbedded rebar creates a counterpoise ground ring around the structure to equalize potential. When using rebar in this fashion, the engineer must be cautious of the integrity of connections between numerous rebar members and verify the compliance of the size of the rebar with NEC Article 250. This can be difficult to enforce and monitor during construction, and difficult to maintain over time due to corrosion. Rebar sized for structural reinforcement is often undersized for grounding; it must be sized according to current carrying capacity.
The main objective of a lightning protection system is to protect the building, not electronic equipment housed within. This is achieved by diverting lightning strikes to earth. Factors affecting the design of the system include: type of construction (steel vs. concrete), height, surrounding buildings and frequency of lightning strikes.
When steel members are employed as down conductors, the engineer must be aware of other grounding systems bonded to building steel. Many times in data centers, signal reference grids (SRGs) are bonded to building steel. This is an example of multi-point grounding that could have adverse results. Lightning energy, which is a high frequency voltage surge, could travel along the building steel and then to the SRG. This could cause data corruption or even damage to electronic equipment bonded to the SRG.
Lightning protection systems for concrete structures often consist of air terminals, copper down conductors and ground rods. Unlike a steel frame structure, concrete structures may not manifest a Faraday cage effect (if rebar is not interconnected). Therefore, sensitive electronic equipment located near the data center perimeter could be at risk of lightning side flashes. Lightning energy could “flash over” from down conductors to equipment if a voltage differential is present. Consideration should be given to the location of the IT equipment relative to the down leads and their routing.
Most often lightning down conductors are bonded to a counterpoise ground grid installed to comply with NFPA 780. The counterpoise grid acts as an accessible connection to outdoor equipment such as light poles, fences, outdoor switchgear, etc. The counterpoise grid should not be used as a substitution to the building ground system but as another grounding subsystem tied to the main ground bar (MGB).
NFPA 70 – Article 250
NFPA 70 requires an effective ground path from the device back to its source. The objective is to facilitate the operation of protective devices, thus isolating the faulted system. The grounding path can be via conduit, ground wires, equipment frames or cable trays. Typically, this type of grounding system is for low frequency currents and does not have the low impedance properties to be effective at high frequencies. The intention of this equipment grounding system is personnel safety, not proper operation of IT equipment.
NFPA 70 also requires a grounding point at each separately derived source and at the building service, providing a voltage reference and over-voltage control for the power distribution system. The incoming service to a building is grounded by taking the grounded conductor (neutral wire) and bonding it to the equipment grounding conductor (EG). The neutral/ground bond is tied to a reference ground (building steel, grounds in soil, ground ring, etc.). For a single ground rod, the maximum impedance required by NFPA 70 is 25 ohms. There are no impedance requirements if more than two rods are used. This grounding system is designed for life safety, the operation of overcurrent protective devices under a fault condition and the creation of a common ground reference for the electrical distribution system. The theoretical assumption is that the reference ground will be at a lower potential than the building and utilization equipment. Improperly referenced grounds can lead to circulating currents, which develop between the reference ground and the building ground when a potential difference exists. This can lead to the misoperation of IT equipment.
Although isolated ground systems (IG) are not necessary in most commercial buildings, they are still present in some existing and new facilities. Most isolated ground systems currently employed are meant to provide a “low noise” ground to IT equipment to minimizing data corruption. The goal was to reduce common mode noise (voltage noise inflicted on the ground conductor by the phase conductor) that developed on the load circuit and to provide a reference point for proper equipment operation. IG systems were implemented because electronic equipment utilizes a 1V to ground reference for data transmission.
However, the IG system was usually not very effective, since most installations are installed where inadvertent equipment ground connections are created. Also, isolated ground systems were prone to catastrophic equipment failures, as voltage developed on IT quipment frames referenced to the IG ground are different from that of the grounded objects referenced to the equipment ground within the building.
To remedy the problem, most current IT equipment uses transmission referenced at 5V to ground. However, the data in modern data centers is transmitted at higher rates, and thus prone to errors when referenced to a low frequency ground. A low impedance and low noise grounding system is essential for proper operation of IT equipment. This scenario argues in favor of multi-point grounding and SRGs, which keep electronic equipment referenced to the SRG at an equal potential.
Design of an IT space must take into account circulating currents and EMI/RFI (electromagnetic and radio frequency interference). Electrical interference is generated from such sources as electric motors, ballasts, loading switching and lightning. In data center environments, EMI commonly travels via metallic conduits or radiates from fluorescent lighting ballasts.
The nature of electronic equipment leaves it susceptible to EMI, as voltage surges sweep along signal grounds. Since most digital circuits operate at a low voltage to ground (5 volts) and at high speeds, interference on the ground systems can have negative impacts. EMI propagation on the grounding system will impact the voltage reference of electronic equipment. Voltage surges that infiltrate equipment can deteriorate semiconductors until they are permanently damaged.
High frequency grounding using SRGs is essential for the modern data center. Much of today’s IT equipment has built-in EMI filters and greater voltage tolerances to help mitigate the effect of EMI. However, a strong EMI surge, such as lightning energy, can be too great for filters. In this case, the low impedance pathways in SRGs, particularly at high frequencies, are very effective at diverting the surge to ground.
Impact on Data Centers
An improperly designed electrically complex facility can fall victim to circulating currents: for example, solidly grounded paralleled generators—especially at low voltage. There are normally two options for grounding paralleled generators: grounding each generator locally or bringing the neutral and ground from each generator to the paralleling switchgear and creating a neutral/ground bond there. Either will develop circulating currents on the systems depending on the type of load served. Consideration should be given to other components such as paralleled UPS systems, main-tie-tie-main switchgear, etc.
Although relatively simple in design, power distribution units (PDUs) can impact operation of IT equipment if improperly grounded. Since PDUs have transformers to provide electrical isolation, voltage transformation and reduce common mode noise, the Xo (neutral terminal) of the transformer must be grounded per NFPA 70. Since PDUs are typically located on a raised floor, the natural tendency is to tie the PDU to the nearest building steel to meet code. However, this leaves the PDU susceptible to surges from electrical storms. The PDU must be grounded to the SRG, which serves as the grounding grid for IT equipment (high frequency) and power equipment (low frequency). The SRG and ground bars in all distributed power and telecom rooms should have a “home run” back to the MGB at the building service entrance. The telecom ground and all power grounds should be connected at this location.
While mission-critical facilities can be made of multiple grounding systems, each with its own unique requirements and functions, the overriding concept of single-point grounding ties them all back to the MGB. The use of “home runs” to the MGB and subsequent ties to building steel (and other code-required grounding connections at the service) minimize the potential differences among separated grounding sub-systems. Meanwhile, the benefits of multi point grounding are realized in high frequency grounding practices that protect sensitive electronic equipment. Grounding calls for a holistic approach, accounting for building structure, equipment, and personnel within the building.