Public Service Announcement 2: Backups!

As you might have guessed from the previous post’s title, I have more than one piece of wisdom to impart. I have a question for you: “Are you backing up your data?”

To cut the the chase, I recently had my 400GB Western Digital My Book external hard drive fail. I don’t know exactly what happened to it, but it got a number of bad sectors, meaning there were lots of files that I couldn’t read. It should be noted that I’m actually pretty good about redundancy and backups, but this was my working drive for video editing projects. In general, files on this drive are usually captured video from a tape, some in-progress work, and renders to be written to DVD. Most of these files are replaceable (by recapturing from tape, re-rendering, etc.) but not all of them. Basically, I didn’t back up this drive simply because I didn’t have the redundant storage space to do so.

Since the drive didn’t fail altogether, I was able to try a number of things to recover some of my data. The file allocation tables were OK, but some of the files were inaccessible due to “CRC errors” when trying the copy them. The first step in my recovery was to use Robocopy to move all of the complete non-corrupted files to another drive. Robocopy is a command line copy tool, but what makes it great is its ability to skip files that fail. After running Robocopy, I had a new folder on another drive with all of my safe files and the corrupted files were left on the failing drive.

Now that I had a drive full of corrupted files, I took a look to see which files were important to me. Some had been backed up to DVDs, other projects had been finished, and other files were available on other drives. I decided there was only one file worth trying to salvage: a 600 MB wave file holding an audio commentary that a few friends and I had recorded for one of our home videos. If I weren’t stupid, I would have backed up that file somewhere else, and to make things worse, I’d just tossed the marked-up copy of the “script” a couple of days before. At any rate, I wanted the data out of this file, even if it was incomplete. I tried a number of utilities such as Unstoppable Copier, Copycat, etc. but the ones I had the best luck with were called CopyItAnyway and JFileRecovery. As the names suggest, these utilities grab whatever data can be read and ignore those sectors that are bad. After all was said and done, I’d recovered all but a couple of minutes of audio from that file. The bad sections were just silent. I’d rather not have lost the file, but I can work with what’s left.

Here’s the advice section. If you have files you don’t want to use, you need to back them up! In my opinion, this backup needs to be automatic and scheduled, because most of us are lazy and won’t do the backups regularly if it’s a manual process. You can back up to another computer, an offsite server, CDs/DVDs or an external hard drive. Anything is better than nothing, but in my opinion, your first priority is to guard against hardware failure and your second priority is offsite backups to guard against a problem with your home (like a flood, house fire, etc.)

My backups are done to both a dedicated file server in my house (just an old computer I had lying around) and to an offsite FTP server. Here’s my process, for reference:

  • Every night, my laptop, my desktop and Dusty’s laptop back up their important files (documents, mail, photos, etc.) to the file server.
  • Every monday night, the file server backs up my photos and audio projects to my offsite FTP server
  • Weekly, I have processes to back up my Remember The Milk tasks, Google Calendar events, and other online services to the file server.
  • Weekly, I back up my web host (on which this site is hosted) to the file server.

Additionally, when I get around to it (see warning above), I burn my photos to DVD and keep a copy in my desk at work. The photos are already duplicated in three different places, but they’re the most important data I have.

If you need some software for automating your backups, check out SyncBack (for Windows). I purchased the full version for $30, but the freeware version is pretty good too.

Many many times people have brought me computers and asked for help in recovering their data. Sometimes it works, and sometimes it doesn’t. Your best bet is to never find yourself in that situation. Do one of two things: do your backups today, or decide that you won’t miss any of your data. If you need help, don’t hesitate to ask!

Edit: here’s a good post on Lifehacker for backing up your Google apps data. I use most of these services so it’s a helpful article.

9 Comments

  1. Ugh. I’ve been thinking about this a LOT lately. I keep thinking I need to buy an external drive… I lost a bunch of pics (somehow) when I moved, and it makes me sad. 🙁

  2. Don’t wait! Buy the first external hard drive you can afford and we’ll set up a backup schedule. If not, at least burn some CD’s or DVD’s of your photos and keep them in your desk at work.

  3. Wow, all that work. Here’s how I do my backups.

    1) Set laptop on desk
    2) Plug in 500GB WD MyBook
    3) Let Time Machine do its thing

    I still need an offsite solution, but I’m covered if my MacBook’s hard drive implodes.

  4. Hey Andy – thanks for dropping in. I’ve heard a lot of good things about Time Machine. Does that take a lot of space? I’m on Windows, so I can’t use exactly Time Machine, but I skipped other options like Norton Ghost, etc. because I didn’t care to backup all my system files, program files, etc. I do keep Powerquest Drive Image files for the initial configuration on all my PC’s. If a computer gets corrupted, I just blast the drive and put the image back on. It’s a lot easier than finding all the drivers, etc.

    If you lose both your laptop and your external drive, chances are you’ve got other things to worry about!

  5. … It’s on my list of things to do in two weeks. I don’t need *much* … but.. some. I do need to burn a CD of my last round of concert pics, though… thanks for the reminder. Especially since I’ve got another round coming tonight….

  6. Time Machine is about as space-efficient as you can get without doing block-level (as opposed to file-level) differences.

    Then you first plug in a drive and tell TM to use it, it’ll sync your entire hard drive. In my case it was about 988,000 files. After that, it only backs up what’s changed since the last backup. And instead of having to search for any file modified since the last backup, Leopard keeps a via the fseventsd (filesystem events daemon – kind of like a FileSystemWatcher in .NET); fseventsd is also how Spotlight (system-wide search) keeps track of what files need to be re-indexed.

    It’s very low-profile. If I’m at my desk, the only way I even know TM is on the job is if I hear the external drive spin up and start writing, or see the icon in the menu bar spinning. My system doesn’t get bogged down by it at all.

    Each backup is represented by a directory on your TM drive. BUT, each one is filled with hard links back to the last “real” version of the file. So each directory is actually a handful of real files, and then pointers to old copies which haven’t changed.

    I have a 120GB drive in my MacBook, with about 55GB free space (so after losses to formatting, figure about 55GB of files). My 500GB MyBook formatted at 465GB, and presently has 406GB free.

    So, a full backup with daily incrementals (since Feb. 2) of my 55GB system is taking about 60GB. Not too bad.

    It doesn’t play as nicely with VMWare disk images and large video files – one little change means you’re backing up the whole thing. Yes, it works – it’s just very space-intensive.

    If I ever need to restore my internal hard drive from scratch, the Leopard Setup Assistant will let me point it at a Time Machine drive and bingo – I’m back in business.

    The same day I bought my MyBook, my father picked one up (CompUSA going out of business sale), along with a copy of Arconis TrueImage. He has yet to get a usable backup. Every time it’s run, his system has BSOD’d.

  7. One thing I forgot to mention. The most important thing about Time Machine is that It Just Works. I didn’t have to mess around with any setup, no drive image configuration, nothing.

    I plugged in my drive, Leopard said “I see you plugged in a new drive. Do you want to use it for Time Machine?” And I was finished.

    Simplicity – ease of use. That’s what gets people to make backups. When I used my PC, I never made backups because it was such a hassle. Even backing up my photos with Picasa was a pain, despite the fact that the feature was built in.

  8. Andy- very good point. Automatic and easy backups are the kinds that people will do.

    I’ve actually heard some good things about Windows Home Server. Apparently it will fairly seamlessly back up every computer in your home.

Leave a Reply

Your email address will not be published. Required fields are marked *