Microprocessor controls remain among the most significant innovations the industry has seen with respect to UPS system, transfer switches, circuit breaker trip units, etc. With their high speeds and digitized signal processing, microprocessor controls respond much more precisely and much faster to abnormal conditions. They also eliminate the “drift” of alarm settings that plagued earlier UPS systems.
One might argue that, without them, the automatic static transfer switch (ASTS, or STS), which transfers critical loads between sources so quickly they don’t register any disruption, would not exist.
On the other hand, microprocessor controls have greatly increased the number of settings, menu selections and configurations that team members and owners have to address throughout the design, procurement, testing, commissioning and operational phases of a critical facility project. An erroneous setting can cause an unnecessary circuit breaker trip, relay activation or similar occurrence that may, in turn, affect the critical load. But it may be necessary to rethink the word “acceptance,” at least as it applies to how critical power system components “decide” that incoming power is okay.
The affected site in this example has rotary UPS equipment in a 2N redundant configuration, with static transfer switches set up so that half receive their preferred source from the “A” side and half from the “B” side of the UPS output bus. The UPS systems incorporate motor-generator sets fed either directly from the input or via a rectifier/inverter array with stored energy, in the form of batteries or flywheels, available to the inverter input. Normally, they operate via the direct input.
In the event of a loss or disturbance of the direct input, the units switch to their rectifier/inverter paths and draw upon their stored energy if the need arises. They may also switch to their rectifier/inverter paths in response to an external signal for output synchronizing. This occurs, for example, when one side of the system operates via generator while the other remains on utility power.
The commissioning of this system revealed routine problems and issues that the manufacturers and contractors addressed. The owner accepted the system and operated it without difficulty for a few months. But then came word of anomalous STS behavior. The STS were transferring back and forth repeatedly for several minutes at a time, and then operating normally for several hours. Episodes of false transfers sometimes occurred a dozen times per day, and not at all on others.
Investigation linked the false STS transfers to UPS system transitions between direct and rectifier/inverter paths. The UPS outputs during these transitions were monitored to identify what characteristic of their output deviated from the STS source acceptance criteria. The waveforms showed no deviation from disturbance criteria as determined by the Computer and Business Equipment Manufacturers Association and Information Technology Industry Council (known as the CBEMA/ITIC curve). The question had to be asked, “What constitutes ‘source acceptance?’”
Finally, it came down to another “ah-ha” moment. The rotary UPS systems activate an internal clock when they operate via their rectifier/inverter paths. When switching to their direct paths, they speed up or slow down to sync up with the direct path prior to switching.
This introduces a delta f, or “frequency slew,” like that used in paralleling gear to sync up diesel engine-generator sets. The rotary UPS rate of frequency change, or “slew rate,” in this installation was very slow — less than 0.1 Hz per cycle. At this rate, a given cycle would be no more than 28 microseconds longer or shorter than the previous cycle.
It turned out, however, that the STS manufacturer shipped the switches for this project with maximum frequency sensitivity due to problems they encountered on a previous project. The manufacturer also hard-coded the slew rate sensitivity of these STS into the firmware, rather than providing a customer-accessible adjustment. After the installation of new firmware chips with a higher frequency slew tolerance, the false transfers stopped and the owner has since enjoyed reliable operation.
The STS false transfers never caused a critical load outage, but the owner's information technology managers understandably questioned the integrity of the system, demanded answers and prepared to activate contingency plans. As a result, the facility managers endured a lengthy and difficult period of heightened scrutiny and spent considerable sums to address the problem.
Also, swapping out the firmware chips in the STS posed human error risk in the form of bypassing each one, accessing their control cards, verifying them with the new chips and returning them to automatic operation. Meanwhile, the underlying issue — what do we mean by “source acceptance?” — still surfaces from time to time in commissioning projects. Correct voltage and frequency, absence of spikes, sags, surges, transients, etc. are givens, but other parameters can prove just as important.
Focusing on the language of commissioning may seem pedantic or unnecessary — a luxury that cannot be afforded. Experience, however, says just the opposite.
Improving language will pay dividends in the form of faster problem solving, better coordination and more thorough system validation. Meanwhile, these examples reveal that the language has to be understood across the silos of design, manufacturing, contracting and operations, among white collars and blue collars, both in the field and in the office, between mechanical and electrical, between management and professional, and so on. That’s a tall order, but it’s worth the effort. In mission-critical spaces, communication is often mission critical. 
David M. DiQuinzio, PE, directs Commissioning & Sustainability Services Group of RTKL, one of the world's largest architecture and engineering firms. He has more than 20 years of engineering experience in commercial, educational, institutional, data processing and telecommunications facilities.
Communication Key to Ensure Uptime in Mission-Critical Data Centers
Facility Managers May Want to Revisit Concept of Factory Testing
Faster Problem Solving, Better Coordination Benefits of Communication
