Rackspace Explains December 18, 2009 Outage

Yesterday afternoon, we reported that Rackspace had suffered a somewhat large-scale outage, leaving many sites down for the better part of 35 minutes (from 4:37 PM Eastern to 5:12 PM Eastern). In that report, we did not have much (if any) information with regard to the outage, but Rackspace has now explained the reason for outage in a blog post.

In essence, the problem arose from a router at the company’s Dallas data center which was responsible for handling approximately 20% of web traffic at that particular location. Specifically, “[t]he problems stemmed from a configuration and testing procedure made at our new Chicago data center, creating a routing loop between the Chicago and Dallas data centers.”

The incident apparently occurred during final preparations for the company’s monthly maintenance window, in which plans were set to test the network integration between the Chicago and Dallas data centers.

Rackspace states that their network is “operating under normal conditions,” and does not expect any residual outages to occur in the near future.

So much for the 100% network uptime guarantee.