Archive

Archive for January 18th, 2008

Fall down seven times, get up eight

January 18th, 2008 Phil Comments

Every system we ever use will fail. How we respond to the failure is a test of character and preparedness. This morning all of 37 signals products were out of action because of a problem with a load balancer at their hosting service. Basecamp, Backpack, TadaList, Campfire, Highrise . . . all down.Their reaction – communicate and fix.

37signals System Status

37signals System Status via kwout

While the problem persisted the urls of the broken applications were pointed to a system status page. Once a fix was in place the product blog was updated with an apology and an explanation.

What happened this morning? - (37signals)

What happened this morning? – (37signals) via kwout

Given that Murphy’s Law was operating they handled the situation well; communicated clearly, found the problem, fixed it, and did so promptly. It underlines the need to plan for failure and recover from disasters.

Basecamp help: General

Basecamp help: General via kwout

It is possible to export XML backups from Basecamp but I’m sure that many this morning felt a sinking feeling for a few moments, while some of those with XML files would have been wondering what to do with them.

37signals Product Blog: Exporting data from Basecamp

37signals Product Blog: Exporting data from Basecamp via kwout

Web applications are tremendously convenient, with integration and sharing features that we’d never have dreamed of a few years ago. But still we need to plan for the worst.

Murphy's Laws - End of the World Production, LLC.

Murphy’s Laws – End of the World Production, LLC. via kwout

[Slashdot] [Digg] [Reddit] [del.icio.us] [Facebook] [Technorati] [Google] [StumbleUpon]