So, you’re really cool. You’ve set up a huge RAID1 array on your server. You sleep at night knowing that the chances of losing two disks before you can rebuild the array is pretty small. And then on a nice cool Monday morning your systems administrator tells you that a drive in the array has failed. You order a replacement and have it sent overnight. On Tuesday night the systems administrator calls you with the worst possible news; the second drive is failing and the array has not been rebuilt. You frantically try everything you can; upgrade the kernel, check for bad ram, fsck the hard drives. All provide no solution. Your data, it appears, is lost forever.
You assess the situation. You, personally, have lost 3 years of email. A client of yours has lost a database application with about 80GB of data. Other clients have lost various amounts of email as well. Sound like something that doesn’t happen? Think again. It happened to me this week. I’ve learned my lesson: never trust RAID. It doesn’t matter how many drives you have, never trust the array. So what are my options?
- Drink heavily to dull the pain.
- Contact a hard drive recover operation and find out what your options are.
It looks like it’s going to cost me $100 to get the drive evaluated. If they can recover any of the files off the drive they will send me a list of files and a solid quote (which will be between $500 and $2400). I should know by the end of next week as to what the outcome of the evaluation is. I can’t afford $2400, but I’d think long and hard about the $500. I didn’t even know that such places exist, but if it works it will be a total life saver.
93% of companies that lost their data center for 10 days or more due to a disaster filed for bankruptcy within one year of the disaster. 50% of businesses that found themselves without data management for this same time period filed for bankruptcy immediately.
Needless to say I’ve spent a lot of time over the last couple of days setting up rsync scripts to back up email, databases and web folders. I first rsync database dumps, mail and web folders into the user’s home directory. I then rsync my own directories and other important directories down to my 300GB firewire hard drive.
If I end up doing the disk recovery service I’ll post a review to the site and let everyone know how it goes.
If you go the way of drive recovery, I’ll really look forward to hearing your review of the process. My company has a paternership with drivesavers.com, but we’ve never gotten to use them yet because whenever we give the customer the estimate they always opt to rebuild all their data from scratch or decide they can live without it.
Joe OnTrack.com is an awesome company for this. That said, look into a backup aplication. Rsync/tar are nice but for a client you’ll want to restore quickly, ie, minimize downtime. Look at Bacula.