News is slowly starting to trickle out about a massive explosion at a Houston data center run by The Planet. You may be familiar with this operation. Ironically, they were just awarded
for their management practices:
The Planet privately held dedicated hosting company’s Vice President of Facilities, Jeff Lowenberg, has garnered SearchDataCenter.com’s first-ever ”Data Center Manager of the Year” award.
The award recognizes excellence in data center project management, according to senior site editor Matt Stansberry. Mr. Lowenberg was recognized for his work that will save the company over $1 million on energy costs in 2008 alone, based on the implementation of new ”green” data center efficiencies.
It becomes even more strange when you note that Lowenberg has been slashing power consumption in an “efficiency” operation:
Initial results demonstrate that while critical server loads increased by 5 percent, power used for cooling decreased by 31 percent. Overall, the company experienced power reductions of up to 13.5 percent through a broad range of improvements. The new green initiatives were conducted across its six world-class data centers.
That was May 29, 2008. And then this (also documented here) happened:
May 31 – 10:46pm
On Saturday, May 31st at 4:55pm CDT in our H1 data center, electrical gear shorted, creating an explosion and fire that knocked down three walls surrounding our electrical equipment room. Thankfully, no one was injured. In addition, no customer servers were damaged or lost.
We have just been allowed into the building to physically inspect the damage. Early indications are that the short was in a high-volume wire conduit. We were not allowed to activate our backup generator plan based on instructions from the fire department.
I do not have many details on this but there are some blogs and forums discussing how the fire department prevented the company from running redundant lines. While it may be possible to somehow allocate blame to local regulations or even the fire department inspectors for risk that was run by management, the bottom line is that the risk was accepted by The Planet management and a massive catastrophic failure has happened on their watch.