Main | August 2004 »

July 27, 2004

I'm back - ugh

Well this has been fun.

My webserver had long been hosted in a customer's computer room, where it had plenty of physical security, good bandwidth, controlled environment, clean power, and two years of uptime. What's not to like?

Two months ago the huge UPS that powers the whole room failed, and the crash took out a hard drive in my webserver. This webserver is an old but rugged Compaq Proliant with RAID5 for the main content, and a spare standalone drive for a nightly online backup - the backup drive is what failed (and it failed hard; disassembly showed oxide scraped off the platters). Bummer, but no big deal, it was just a backup drive.

I replaced it and backed up (and verified) the whole system and then started an OS upgrade to replace the aging Red Hat 6.2, and a few hours later had a shiny new operating system on a RAID 5 partition, all ready to accept the saved data.

Or not.

The data on the backup drive was bogus, and none of my recovery tools would even pretend to find the partition or the filesystem. I decided that the RAID controller probably ate it, taking down nearly my whole website; so much for having a backup.

Thankfully, I had nearly all my static content checked into a Perforce SCM depot on my home network, so the better part of unixwiz.net lives, but everything else - tools, weblog, email, other sites, etc. - are gone.

So I got myself a new rackmount box - a Compaq Proliant 1850R - and have parked it at a data center, and am slowly rebuilding everything.

And setting up an offsite rsync backup.

Posted by steve at 06:36 PM | Comments (5)