Darslicing

I want to back up my photos and videos on DVD. This sounds like a pretty straightforward task to me, but apparently it’s not, at least if you’re on Linux, like I am. I have a collection of digital photos and low-resolution videos, dating back to 2001. The total file size is about 40 gigabytes, organized in folders per year, like this:

• photos.2001
• videos.2001
• photos.2002
• videos.2002
• etc.

The folders vary greatly in size. Many of them exceed 4.3 gigabytes which is what you can store on a standard DVD. (Otherwise I would have just copied those folders to individual DVDs, forgetting about the wasted storage space left on many of the DVDs.)

I’ve done backups on DVD before, the layman’s way: Drag over files to k3b, KDE’s burning program, until there’s no space left, burn the DVD, remember the last file stored on DVD, then start over with the next file to be burnt on a new DVD, and so forth. But that’s tedious and error-prone. What I wanted was to simply drag all photo and video folders into some application, and then let the application slice them so that they would fit on regular DVDs. I was sure that other people had had the same problem before, so there had to be an application that does just that. But there isn’t, at least to my knowledge, although I’ve asked many people and googled many forums. The best advice I found was „do this manually“ (d’uh) or „use a hard disk“ (double d’uh). I just want to back up my photos and videos on DVD, come on. And I believe in read-only media when it comes to backing up historical stuff that will never change, like photos and videos.

I haven’t found a solution that slices my files so that they fit on DVD, and saves them on DVD as files, uncompressed. There’s not much point in compressing JPEG or MPEG files, anyway, and this would give me the additional advantage of being able to view them directly from DVD (for example on my mother’s TV set which can display JPEG slide shows and play MPEG movies).

I said I haven’t found a solution, but that’s not entirely true. I’ve found a Python script that’s pretty short and fairly easy to read, but it’s the opposite of fault-tolerant, and it’s also pretty buggy. One tiny user mistake is enough to stop the process, rendering the last inserted DVD a case for the trash can. Using that script comes close to doing things manually. If I could write Python I’d probably hack that script and make it available publicly. But I can’t.

So I reverted to the next best solution, which is just slicing my files into 4.3 gigabyte pieces, with no way of looking at the individual files straight from DVD. Yes, I could use tar for this, but tar stands for Tape Archiver, and was originally meant for backing up files to tape. What tar does is create one big file, „tarring“ all the individual files into that monolith file, while optionally compressing them. That’s great for „full“ backups, for example snapshot backups of home directories or log files that you only need to restore fully (in case of disaster, or if you’re transferring files to another disk or computer). But you cannot retrieve individual files or folders from a tar archive file. Also, there’s no built-in slicing in tar, so tar’s archive files might become bigger than 4.3 gigabytes, thus not fitting on DVD.

Enter dar. Dar stands for Disk Archiver, which sounds a bit more modern to me than Tape Archiver. Dar basically does the same as tar, that is, „tarring“ files into one big file, optionally compressing them, but it has two advantages over tar:

1. Dar can be configured to store that one big file in slices, that is, in a set of files not exceeding a particular file size each.
2. You can retrieve individual files from a dar archive easily.

So what I did was create sliced dar files, which I decided to call darslices, each 4.3 gigabytes in size, and burn those files onto DVD using k3b. Dar seems to be less performant than tar, so it took a whole working day to create a dar archive of my 40 gigabytes. I ended up with 16 darslices, which I then burned onto DVD.

Now I said it before that I’m originally a Windows guy, and while I’m running lots of self-coded shell scripts to do repeated tasks, I still prefer to use a graphical user interface (GUI) when it comes to performing tasks that I do only rarely, like once per year. DarGUI comes to the rescue. As you can imagine, it’s a graphical front-end to dar. DarGUI is no more than that, but it’s decently coded, and, unlike so many open source applications, it has great documentation. So here’s a mini-tutorial on getting started with DarGUI, and thus with dar.

First of all, make sure dar is installed. As the name implies, DarGUI is just a graphical front-end, it doesn’t include dar. Then start DarGUI. In KDE, you’ll likely find it in the K menu under System, Backup.

To create a new archive, follow these steps:

1. Select File, New (or click the Create a new archive icon).
   
   This opens a new window with six tabs.
   In the Archive tab, fill in a name for the archive file (or files, see later on). If you do regular backups (you do, don’t you?) the Add timestamp to name checkbox comes handy.
   Saves archive in can be an arbitrary location for storing the dar files (or dar slices).
   The Base directory field is the directory which contains all the files and directories that you want to archive. The directories that you enter on the Directories tab must be in this directory! I got this wrong the first time, although that’s exactly how it’s documented. Getting it wrong (by setting the Base directory to some other path) later on resulted in this error:
   
   You can leave the value of Batch file value as is. Check Differential backup if you’re not doing a full or initial backup, but rather follow-up backups that just back up the files that have changed since the last full backup.
2. In the Directories tab, add the directories you want to back up. For me, this was just the fotovideo folder on my massive storage device. 🙂
   
3. I didn’t use the Files tab at all, because I was interested in backing up complete directories, rather than individual files.
   
4. Likewise, I didn’t use the Compression tab, because compressing photo or video files normally doesn’t make much sense.
   
5. In the Options tab, I checked the Use slices checkbox, setting the value to 4500 Mb (megabytes), since this is what fits on a 4.3 gigabyte DVD.
   
   The Test run only is handy to just try out if your setup will work. It simulates dar processing, but without actually writing files.
6. I wanted to run the process immediately, so I didn’t fill in anything in the Scheduling tab.
   

That’s it. Now it’s time to press OK to start the process. This will open a console window which looks like this:

As said, for my 40 gigabytes of photos and videos, this ran all day. For the initial test run, I stopped it by pressing CTRL+C, then restarted the process after unchecking the Test run only checkbox in the Options tab.

Once the darslices are created and the process has ended, you can browse through the directories and files by selecting File, Open in the main window, or by pressing the Open an archive icon. Select the dar file, or the first dar file of your darslices. The interface will look like this:

Note that it can take some time until the file(s) are loaded and ready to be browsed, if you have a huge dar file or file set. While loading, DarGUI will display BUSY in the status bar.

To restore a file, apply a right click on it and select Restore selected. Then select a location to store your file to. You can do likewise for directories, or even for all directories or files in the dar archive.

Having talked about how great dar and DarGUI are, here’s a caveat that might make my whole effort look somewhat pointless. Dar relies on its darslices to be present on one physical medium. That is, you cannot retrieve individual files from a dar archive that’s stored on a multitude of media, like a set of DVDs. In other words, if I’d plan to restore my photos.2001 folder from my 16 darslices, I’d first have to copy all my 16 DVDs back onto disk. That’s not what I was aiming for in the first place, so I’ll continue looking for alternative solutions. (If you can point out a working solution I’d be more than grateful.) Still I think dar (and DarGUI) can be used for other scenarios, so I don’t regret having spent time on learning how to handle these programs.

Before I forget, I’d like to say thank you to Malcolm Poole, the author of DarGUI, for promptly replying to some questions that I had about DarGUI, and for pointing out to me that what I was looking for was actually documented. (In case you don’t know, I lead the MySQL Documentation Team, so being politely told that there’s an RTFM option always gives me a special feeling, if you know what I mean.)