Ramble On

Wednesday, February 6, 2013

That Super Power Outage


So, I’ve been thinking about that power outage in the Super Dome the other night.  As the television showed the impressive logistics of striking Beyonce’s stage, we had just a glimpse of the complexity or the equipment and electrical power involved.  It’s not a surprise that there would have been strains on the systems there that night - not at all.

I’ve worked in critical buildings before, government data centers, where there was a requirement for steady power on a 24/7/365 basis.  These buildings generally have a primary and back-up source of electrical power; they often have a third back-up so that one of the others can be taken down for maintenance while adequate back-up is still available.

I don’t know the specifics of the arrangement at the Super Dome, but I am sure that power infrastructure along these lines is provided there.

Still, with nearly $4-million in commercial revenues per minute on the line, that building experienced a power crisis – for more than 30 minutes!  Somewhere this week there is a facility manager who’s feeling more pain than just being the butt of a bunch of bad jokes.

My hunch is that the problem happened during the switchover between power sources – either because of a need to go to the back-up system, or because of the extra requirement the half-time show required.  

In those 24/7/365 operations I mentioned, the best of them would have gone through a series of rehearsals for this process starting weeks in advance.  There would have been a script with detailed information and assigned individuals for each action, from switching circuits to standing by the back-up in case it was needed.   
The procedure would have been documented step by step, and the team would have practiced over and over.  The back-up plan would also have been documented and practiced, not only to reduce the risk of failure but to ensure that any potential outage could be corrected in a few seconds – not minutes, and certainly not a half hour later!

I guess that eventually we’ll find out what happened at the Super Dome last Sunday night.  But until we do, this event was a reminder that while Beyonce and her team were in the spotlight, the value of the facilities team can’t be underestimated.  You can’t take the performance of good building engineering for granted when the stakes are high – that’s when the professionals need to be ready to step up into the breach.  

No comments: