Kevin Kelly recently posted about digital continuity on The Technium. He references David Pogue’s experience of having problems reading DVDs which were only 4 years old.
No problem, I thought: I’ve got all of the original iMovie projects backed up on DVD, in clear cases, neatly arrayed in a drawer next to my desk. (My hard drive wasn’t big enough to hold those 50 videos a year.)
Guess what? On the Mac I use for video editing, most of the DVD’s were unreadable. They’re less than four years old!
Tried them on another machine. About half of them were readable.
This is an issue which effects all digital artists, and visualists in particular, as digital video tends to take up large amounts of space. Traditionally I’ve used optical media to backup and archive my work, but recently I’ve changed my habits and now I’m keeping everything on hard drives, consolidating smaller drives on to bigger ones as often as architecture changes or Moore’s law makes it affordable. 750GB (around 11c/GB) seems to be the current dollars-per-gig sweet spot, (especially when you keep in mind the increased manufacturer ripoff factor for a “1TB” drive). 750GB will fit around 170 DVDs worth of data, which obviates even the financial argument for burning DVDs – even if it only takes 4 minutes to burn each DVD, that’s over 11 hours of burning time.
My current stack of drives ready for Moveage
Update: This post has already generated some fantastic feedback, so we decided to expand it slightly, and promoted it from an Aside to Full Article status.
OrT Says:
I used to work for a company that, among other things, specialized in data recovery just up to the clean room.
Their suggestions:
– Tape is still the most safe long-term storage
– CD’s/DVD’s should be re-written every 2 years
– For live usage: two storage servers with redundancy, spread over two seperate buildings (can be cheaper than it sounds) – one at your location and one a few houses/buildings down or on another remote locationFor re-writing a CD/DVD archive, there are burning robots that can automate the process. It will, ofcourse, cost you.
Oh – and for CD’s/DVD’s you should use every once in a while: burn them twice and never touch the second copy to avoid chemical reactions and decay.
This is of course a good point. Tape is a very stable storage medium if stored correctly. I keep all of my DV tapes, so I have every moment of video footage I’ve ever shot, filling various cardboard boxes a secure, climate-controlled bunker. However the data on them is very time consuming to format-shift, so the likelihood that I’ll put them on to a more modern, stable, high-capacity tape format is unlikely, and I’m not really holding out hope for my DV-based cameras to still be in daily use 10 years from now, so retrieving that data can become problematic.
previewlounge says:
i have lost more data on disk drives (three 80 gig drives and a 350 gig drive over about 7 years) than i ever did lose from DVD’s
in fact it was the DVD’s that saved the day! 🙂
therefore i suggest backing up to hard drive and also toasting your best animations onto DVD’s as data disks.
This brings us to the crux of the matter, and the most important rule of thumb with data storage: If it doesn’t exist in at least two places, then you don’t really want to keep it.
The point of this post isn’t that hard drives are a superior media for long-term storage, but that long-term storage in general is risky, so if you can keep your data moving, it’s much safer. For most people backing up is a chore, so if the process of moving the data is faster and simpler, it’s much more likely to happen regularly enough to keep everything safe.
The good thing about hard drives is that it’s a reasonably trivial task to duplicate them, and 2 years is about the right amount of time for Moore’s law to let you consolidate your previous generation of drives to the newest, cheap drive. For example, the last generation of drives I need to back up are:
320GB x2
160GB x3
80GB x4
40GB x3
Which happens to add up to about 2x 750GB drives. If this data was on DVDs, it would be at absolute minimum 340DVDs, but probably closer to 400 as there’s generally quite a bit of blank space when I’m backing up to DVD. Even giving just 5 minutes per disk burnt, that’s 33 hours worth of DVD burning just to keep them up to date. Realistically, that’s never going to happen, but copying large amounts of data to new hard drives is a relatively hands-off process, and I’m not expecting it to take me more than an afternoon.