The issue arose recently during a full-load burn-in test of a diesel engine-generator set after it had been installed at a hospital site that was being commissioned. The unit had made it past the halfway mark when the team heard a loud bang and saw a bright flash emanate from its output section. The output circuit breaker tripped and the unit shut down.
Upon removing the covers, the team discovered missing output cable lugs and arc flash residue on the line side of the output circuit breaker. Worse, an inspection of the generator itself showed residue and metallic particles in the vicinity of the air gap. Clearly, fast-moving air, driven by the radiator fan, had drawn plasma into the air gap. Subsequent testing found the generator a total loss.
The initial suspicion was that nuts, bolts or washers left inside the output section vibrated loose, fell in between phase conductors and shorted, as that scenario had been seen before. When this occurs, however, telltale residue patterns are often left behind. In this case, there were none.
Instead, one of the team members had another of those “ah-ha” moments upon inspecting the unit. The generator manufacturer had provided a circuit breaker bus extender for two busswork terminals per phase instead of one. The idea was to avoid too much current passing through one set of terminals at full load. As delivered, however, the extender’s terminals didn’t have any attached cables. Instead, the manufacturer had landed all of them on the breaker’s main busswork terminals.
At full load, they carried too much current, overheated and the lugs failed under thermal runaway. Upon inspection, one other generator at the site had its output cables terminated the same way — yet it survived the full-load burn-in for the specified duration — while the others had equal numbers of cables land at the extender terminals and the main terminals for each phase as intended.
This finding begged the obvious question: “Weren’t these generators factory tested?” Yes, they were. The owner had another project further along and had purchased generators for both sites in one order. A review of the factory acceptance test record showed that the earlier project required the generators’ output cables to connect at the bottom, while the project being commissioned required the cables to connect from above.
The manufacturer, it turned out, had modified the generators headed for the hospital site for top-feed after completing the factory tests. The modification included removing the existing terminations, rearranging the busswork and terminating the output cables on the modified busses. It appeared that during this process, the output cables ended up entirely on the main output circuit breaker busswork terminals on the two units found with this condition.
The hospital had sufficient redundancy to provide a viable workaround for the failed generator. The owner and the manufacturer arranged for a replacement to arrive at the site a few weeks later. The installation, however, required heavy rigging and construction work that proved risky and disruptive in a live, critical environment.
The team’s assumption was that, when anyone in our industry refers to a “factory test” or “factory witness test,” they mean “a test of the equipment exactly as it will leave the factory” and that nothing else, besides preparing the gear for shipping will occur afterward. The team counted on this understanding even though no one from the team was involved in the factory testing process.
Communication Key to Ensure Uptime in Mission-Critical Data Centers
Facility Managers May Want to Revisit Concept of Factory Testing
Faster Problem Solving, Better Coordination Benefits of Communication
