Comments on: Anatomy of an outage http://storagemojo.com/2008/05/14/anatomy-of-an-outage/ Data storage info & analysis Thu, 20 Nov 2008 21:45:50 +0000 http://wordpress.org/?v=2.6.3 By: e2eiod http://storagemojo.com/2008/05/14/anatomy-of-an-outage/#comment-195686 e2eiod Sat, 17 May 2008 05:42:45 +0000 http://storagemojo.com/?p=710#comment-195686 Well done! I applaud your skills, tenacity and success but I second ValB's and Ryan's suggestions. Well done!
I applaud your skills, tenacity and success but I second ValB’s and Ryan’s suggestions.

]]> By: ValB http://storagemojo.com/2008/05/14/anatomy-of-an-outage/#comment-195664 ValB Thu, 15 May 2008 16:46:29 +0000 http://storagemojo.com/?p=710#comment-195664 Hi Robin, Sorry to hear about your troubles here, but it's certainly a sobering & educational reminder of the realities of web "commerce" in this day and age! At the risk of trivializing your technical problem space, let me share a simplistic best-practice from my early days of selling NetApp filers to Web 1.0 companies: - Run your site from a locked-down storage-based read-only snapshot Most blog pages are read-mostly, write-sometimes (post, comments, etc...) workloads which may still be well-suited to an update process which basically appends HTML changes to a "safe" edit area which can be subsequently be re-snapshotted and re-served back into the "wild" Best of all, you can build this yourself with ZFS if you'd like. Or perhaps borrow Ryan's sage advice above and outsource the whole thing. A halfway compromise of course would be to merely outsource the storage piece to a proven web storage supplier instead of building your own! Sorry, couldn't resist! :-D Hi Robin,

Sorry to hear about your troubles here, but it’s certainly a sobering & educational reminder of the realities of web “commerce” in this day and age!

At the risk of trivializing your technical problem space, let me share a simplistic best-practice from my early days of selling NetApp filers to Web 1.0 companies:
- Run your site from a locked-down storage-based read-only snapshot

Most blog pages are read-mostly, write-sometimes (post, comments, etc…) workloads which may still be well-suited to an update process which basically appends HTML changes to a “safe” edit area which can be subsequently be re-snapshotted and re-served back into the “wild”

Best of all, you can build this yourself with ZFS if you’d like. Or perhaps borrow Ryan’s sage advice above and outsource the whole thing. A halfway compromise of course would be to merely outsource the storage piece to a proven web storage supplier instead of building your own!

Sorry, couldn’t resist! :-D

]]> By: Brian http://storagemojo.com/2008/05/14/anatomy-of-an-outage/#comment-195662 Brian Thu, 15 May 2008 15:42:49 +0000 http://storagemojo.com/?p=710#comment-195662 Tripwire (http://www.tripwire.com/products/enterprise/ost/) or its ilk are your friend for things like this. (don't know if your host allows/provides it, I use it on internal Linux servers). It will take a "snapshot" of everything on your system, then alert you to anything that has changed. I don't believe that it will prevent attacks, but is certainly useful for finding out what files have been changed, failed logins, etc. It helps you catch problems quicker, and ensure that you have cleaned up everything. Tripwire (http://www.tripwire.com/products/enterprise/ost/) or its ilk are your friend for things like this. (don’t know if your host allows/provides it, I use it on internal Linux servers).

It will take a “snapshot” of everything on your system, then alert you to anything that has changed. I don’t believe that it will prevent attacks, but is certainly useful for finding out what files have been changed, failed logins, etc. It helps you catch problems quicker, and ensure that you have cleaned up everything.

]]> By: Ryan Malayter http://storagemojo.com/2008/05/14/anatomy-of-an-outage/#comment-195659 Ryan Malayter Thu, 15 May 2008 13:18:40 +0000 http://storagemojo.com/?p=710#comment-195659 Running a web *application* clearly isn't your core business. Why wouldn't you outsoursce to say Blogspot (Google)? Cost is minimal (actualy probably far lower given how much time and effort you've just wasted), and the provider worries about the securtiy/patching/uptime. What's the new B-School mantra? Something like "if its an aspect of your business that faces competitive pressure, you must keep it yourself. Outsource everything else with a positive ROI". Robin, your *content* is your core business, not running the website itself, right? Running a web *application* clearly isn’t your core business. Why wouldn’t you outsoursce to say Blogspot (Google)? Cost is minimal (actualy probably far lower given how much time and effort you’ve just wasted), and the provider worries about the securtiy/patching/uptime.

What’s the new B-School mantra? Something like “if its an aspect of your business that faces competitive pressure, you must keep it yourself. Outsource everything else with a positive ROI”. Robin, your *content* is your core business, not running the website itself, right?

]]> By: John http://storagemojo.com/2008/05/14/anatomy-of-an-outage/#comment-195658 John Thu, 15 May 2008 12:51:52 +0000 http://storagemojo.com/?p=710#comment-195658 I am glad that you shared all this because it has made me so much more aware for my sites. I am glad that you shared all this because it has made me so much more aware for my sites.

]]>