Diary of a Commissioning Agent
Friday, April 1, approximately 6:00 p.m.: I arrive at the site, but find no one. I'm here to commission a new generator integrated with the existing electrical system at a health-care facility. On initial inspection, I find two load banks in the outside alley behind the central plant. These will accommodate load bank testing. They include one large 2,000-kW adjustable and one 1,200-kW fixed load bank. The cables that will connect the load banks to the existing paralleling switchgear run along the ground, up the side of the central plant wall to the roof and into the genrator switchgear room.
6:35 p.m.: I'm finally able to contact the construction project manager. He advises that all workers were sent home due to a forecasted thunderstorm. Tonight's cable hookup, scheduled for 7:00 p.m., has thus been delayed until tomorrow morning at seven—and may be further delayed because a surgery has been slated to start at that time.
Saturday, April 2, 6:45 a.m.: I return to the job site to find the west end of the alley under one to two feet of water from last night's rain. Fortunately, load banks are at the east end, away from the flooding.
7:00 a.m.: Surgery has begun, and the owner's representative has decided that we cannot risk connecting the temporary cables while surgery is in progress. The head nurse will let us know when the surgery is complete.
10:35 a.m.: In the generator room, I find representatives of the generator supplier and electrical contractor inspecting some unterminated control wires at the new standby generator. They advise that the switchgear representative will terminate the wires when he arrives.
Outside, cables for connecting the load banks to the switchgear are lying in water. They are being removed and dried off individually in preparation for terminating at the switchgear. Inside, the surgery is expected to be finished by 11:00. Our hope is to start connecting cables at noon. We anticipate it will take two to three hours to connect all the cables, so testing will not start before 3:00 p.m. We are now five hours behind schedule.
11:00 a.m.: We've been informed that another surgery will be performed, and we can't connect the temporary cables until that surgery is complete. The engineering team gets together to discuss the various upcoming tests. We decide to perform the crank cycle test first, as it can be performed without the need for connected load banks.
12:05 p.m.: We are getting ready to do the crank cycle test. The switchgear rep and generator rep are trying to decide who will do what to initiate the test. NFPA 110, Emergency and Standby Power Systems , requires that the generator manufacturer must prevent the generator from starting. Then, the generator starter must crank for 15 seconds, rest 15 seconds, crank 15 seconds, rest 15 seconds and crank for a final 15 seconds. We decide to put the switchgear master control in manual position, and allow the generator vendor to cycle the generator through its 75 seconds of battery testing.
The generator control panel has the built-in means to do the crank cycle test, so the generator rep initiates the test at approximately 12:15. Batteries pass the test with no problem. The batteries registered 26 volts when we were not cranking; this would typically drop to 22 volts during the crank cycle test.
12:30 p.m.: The generator rep is providing a demonstration of the adjustable load bank. This unit has inductors as well as resistors, so load can be simulated with a 0.8 power factor as needed. Software will allow load to be added in 15-kW increments via a hardwired communications cable that has been routed from the load bank controller to a separate remote PC in the generator room. A separate control cable has been run from the fixed load bank to a remote switch at the generator room, allowing the entire load bank to be remote-controlled.
The latest word on surgery is that a cardiac catheterization procedure is being performed, and the operation won't be finished until about 3:30 p.m. At this point, temporary cable hookup has been delayed for more than 18 hours.
3:20 p.m.: Walking into the alley where the load banks are, we find a wind tunnel effect with up to 40-mph gusts. Hopefully, it won't affect the load testing. But it's a clear and sunny day. Everything should proceed as planned as long as we can get the cabling connected and the test started. The last cath lab procedure is complete, and the contractor is now terminating the temporary cables.
4:00 p.m.: All temporary cables have been terminated, and the contractor performs a final inspection to make sure that everything is properly connected. The generator and switchgear reps are also going through the final checkout before starting the new and two existing gensets.
4:12 p.m.: Generators are started, run briefly and then shut down.
4:19 p.m.: The team initiates a test to parallel all three generators, which come online and synch to the bus within six seconds. The three generators' characteristics are monitored, unloaded, for roughly 20 minutes prior to testing the actual load.
The next step is to introduce load to the three generators. This will enable us to calibrate them to share load equally. This will take the next 1.5 hours.
Unfortunately, the day tank serving the new generator runs out of fuel. The automatic signal was not hooked up to the existing main fuel tank transfer pump.
5:55 p.m.: The generators are carrying load. Real power (kW) is balanced, but there is a problem balancing reactive load (kVAR). The day tank for Generator 3—the new one—is still not working properly, so it must be filled manually to keep the load balancing operation going.
6:35 p.m.: We determine that the new contact that calls for the day tank's remote pump to start has not been connected. We decide to jumper the start circuit for Day Tank 2 to the start contacts for Day Tank 3. Thus, the day tank will automatically fill when Generator 3 is running.
7:00 p.m.: We restart all generators, but are experiencing too wide of a frequency swing. Instead of seeing the frequency jump to 61%%MDASSML%%62 Hz and then drop back to 60 Hz, frequency is dropping well below 60 Hz before locking into 60 Hz.
8:02 p.m.: The kW and kVARs are balanced on the three gensets, plus or minus a few percent. We decide to go ahead with a 100% load test of the new generator.
8:20 p.m.: The rep applies 100% load to the new generator. The voltage drops to 400 volts, and the frequency meter does not register at all. The test is aborted.
8:50 p.m.: We determine the V/Hz ratio on the voltage regulator is improperly adjusted. Adjustments are made and the 100% load test resumes.
9:05 p.m.: The test appears to be successful. Generator 3 accepted 100% load in one step, and we observed proper voltage and frequency. Unfortunately, the required readings—cranking time, time required to come up to operating speed, voltage and frequency overshoot, and time required to achieve steady-state condition—were not recorded. We need to abort the test and start over.
9:25 p.m.: After reviewing the required test readings, we reinitiate the 100% load test. The generator picks up full load with a reasonable frequency overshoot; voltage overshoot and the other proper readings are recorded. The generator runs for two hours with proper readings taken every 15 minutes as required by NFPA 110. We decide to defer the system load test and functional test until the following morning at 8:00.
Sunday, April 3, 7:15 a.m.: The team gets together to discuss the system load test. People are assigned to record various readings and times as required by the standard. NFPA 110 also requires the facility's main switch to be open so as to initiate a full system test. But this is a functioning hospital. Opening the facility's main switch would place patients at unacceptable risk. Rather than shed the entire facility, we shed load to low priority transfer switch "ATS-27."
7:50 a.m.: The owner's electrician interrupts power to ATS-27. After all generators start, come up to speed and sync to the bus, a load of 2,100 kW is added to meet normal operating conditions. Everything goes well, and we are now into the two-hour system test. The team meets to discuss the functional test that will occur immediately after the final load test.
8:50 a.m.: We are about halfway through the load test. We modified the functional test procedure and delivered updated documentation to the manufacturer's lead technician.
9:10 a.m.: About half an hour remains for the system load test. Everything is proceeding well, but we note that the DC volts on Generator 2's battery charger are pegged out and need to be checked after the test is complete.
10:12 a.m.: The normal breaker-to-transfer switch has been re-closed. The time delay on retransfer to normal for ATS 27 has been set at 30 minutes, so we decide to readjust that relay manually to speed up retransfer to normal power. The switch transitions to normal power at this point, and all of the generator breakers open at the paralleling switchgear as planned. At 10:17, generators hit their five-minute cool down and shut down.
We initiate the functional test shortly after completing the final load test. The primary purpose is to demonstrate proper load-add and load-shed features. Per specifications, load is added and dropped incrementally to demonstrate load add, load shed, generator shed and generator add. We experience a few minor glitches along the way, but as by 11:45, most of the required operations are demonstrated. Overall, the new generator, and upgraded existing systems, have passed the functional and load tests at the required levels.
Commissioning a new generator system is a complicated process. And the complexity is ramped up considerably when a new genset is being added to an existing system.
But perhaps the greatest complication of all is when the system is for an occupied health-care facility. Patient safety dictates that tests be planned well in advance, and that the life and death situations of a hospital take precedence over system testing. No amount of preplanning is excessive, and test personnel should leave generous amounts of time to complete the process. As the above case—a true story taken directly from job records—shows, "Murphy's Law" issues, such as inclement weather or unexpected unterminated wires,increase the complexity.
Moreover, such endeavors are not for the inexperienced. It is highly recommended that the on-site switchgear and generator reps' credentials be evaluated and approved prior to testing. In this case, properly experienced individuals ran the generator expansion. Their experience enabled them to deal with issues as they occurred, thus accomplishing the goals of testing the new generator and upgraded system.