November 5, 2007 07:32am CST

Live FileVault and Sparse Bundle Backups in Leopard

Posted By ScottW

Mac OS 10.5 (Leopard) introduces the use of Sparse Bundles for FileVault. These are mountable directories full of banded data in 8M chunks. When you add, modify and remove files in FileVault, one or more of the band files will change, depending on where your data is stored.

This opens up so many possibilities for making backups in a secure and quick fashion.

For example, it is possible to backup your FileVault directory while you are logged into it. No, I'm not talking about the actual data and storing it insecurely somewhere else, or even on another sparse image or sparse bundle file, I'm talking about copying the FileVault sparse bundle itself. Using a handy included command line utility Rsync, I copied my 16GB mounted FileVault directory to another drive which took in the neighborhood of 15 minutes or so. I double clicked on it to mount it, and it asked for my password and to my amusement, it mounted without logging out of FileVault or doing anything other than just running the backup.

If that wasn't handy enough already, I grabbed about 20 files I had on my desktop (so much for using Stacks) and tossed them into a new 'untitled folder.' I unmounted the copied FileVault directory, ran Rsync against comparing my live FileVault sparse bundle with the copied FileVault sparse bundle. This time, instead of 15 minutes to perform the backup, it took around 30 seconds, updating only the changed bands in typical Rsync fashion. I then crossed my fingers, swiveled around twice in my chair, and re-mounted the copied sparse bundle. It worked. My changes I just made to my live FileVault appeared on the duplicate.

Feeling young and spry I decided to continue my fantasy adventure. While the copied FileVault volume was mounted, I made changes to my live FileVault desktop folder and then did another Rsync. About 30 seconds go by and nothing happened. I had hoped my changes would show up on the mounted file, but they did not. I really didn't expect it to work, but then again, I didn't expect copying a live (not logged out) version of FileVault would work. Note though, that I unmounted the copied FileVault sparse bundle and remounted it, and when I went to view the Desktop folder where I made my changes, the changes for a brief second showed up, then went back to the previous state. So, while there was some weird "corruption" going on, the sparse bundle still mounted. I then, unmounted the sparse bundle, ran another 30 second Rsync between my live FileVault and my copied FileVault, then remounted the copied FileVault and everything was back in working order and my change showed up exactly as they where on my live FileVault.

What Can I Do With This You Ask?

You can setup an automated task to run Rsync every 15 or 60 minutes, or every 24 hours to have a working mirror of your FileVault directory stored on another hard drive. Your backups are 100% secure, because you're not backing up the data files inside FileVault, you're backing up the encrypted files that make up your FileVault.

Rsync was made to keep files in sync over a network connection. Whether you are running Apple File Sharing, Samba, or even an SSH tunnel, you can backup your FileVault directory securely to remote system, be it on your local network in your home or office, or over the Internet. For those looking to backup over the Internet, getting the first backup will be the painful process (when dealing with slower upload speeds on Broadband and DSL services) but from that point on, backups should be fast and painless, with only the amount of data needed to be transferred based on what changed.

But, don't let your imagination stop there. Let's forget about FileVault and let's just say that we'd like to backup our iPhoto Library to an offsite location that we don't own and we want our iPhoto Library to remain encrypted from the eyes of others. All we would need to do is to create a Sparse Bundle with Disk Utility, use Rsync (or your favorite File Sync Utility) to sync your iPhoto Library with your local sparse bundled file. Then use Rsync to mirror the sparse bundle to a remote system. While this is a two step process, it does indeed work.

Words of Caution

Always test your backups. Never assume they will always work and be available to you. For your own sanity, log out and make a copy of your FileVault sparse bundle the old fashion way, prior to giving this a try. It will make me feel better about providing you the commands to work the magic discussed earlier and hopefully give you a little more freedom to play around. If you already have a solid backup of your FileVault directory, good for you, you may begin.

How To Guide

1) Open Terminal

2) If your User directory is called "scott", and your backup drive is called "SnackFood", then you will want to make use the following command to make the first copy of your FileVault sparse bundle.

Code:

rsync -avE /Users/.scott/scott.sparsebundle /Volumes/SnackFood/

Sit back and watch the files copy. The "v" option above in the "-avE" tells rsync to provide a verbose output, and let you know what the heck it is doing. The "E" option, we will only use on the first copy, this copies over special files that will make your copied directory show up as a double-clickable disk image, vs just a worthless folder.

3) Once that is complete, go and mount the copied directory and if all went well, you should see an exact duplicate of your FileVault directory. Word of caution, SpotLight is hungry for data and once you mount it, it will start to chewing on it for fuel. It is best to add it to your Spotlight Private list as soon as you mount it, else you will be cussing at Spotlight when you attempt to eject your copied disk.

If Spotlight beats you, you can do this to eject it if dragging it to the Trash can doesn't work.

Code:

hdiutil detach /Volumes/scott -force

4) Make some changes to your FileVault file system, duplicate a few files or create new folders and move a few things around, then Rsync again, only using this command.

Code:

rsync -av /Users/.scott/scott.sparsebundle /Volumes/SnackFood/ --delete

Caution: Syntax is VERY important with rsync, adding or not adding a tailing slash can mean something you did not intend to do. So, please follow my example. In addition the --delete will remote any files in the sparse bundle that have been removed and makes sure your copy is an exact sync of your original sparse bundle. In my tests, I didn't use the --delete function, and it would be safe to try it without it your first time to make sure your syntax is correct. However, you will want to use it for regular backups should you choose to ue this method.

Dealing with spaces in file paths on the command line. If your hard drive is named "Snack Food", then in the above examples, you would have to put a backward slash in front of the space, this is called "escaping" the text.

Code:

rsync -avE /Users/.scott/scott.sparsebundle /Volumes/Snack\ Food/

TAGS: Leopard,Rsync,FileVault,Sparse Bundle,How To
BOOKMARK: del.icio.us, Reddit

Article Comments

plsuh - Posted November 21, 2007 08:32am CST

THIS IS A REALLY, REALLY BAD IDEA, DON'T DO IT. IT WILL ALMOST CERTAINLY LEAD TO A CORRUPTED BACKUP OF YOUR FILEVAULT SPARSEBUNDLE.

When a disk image is mounted read/write, Mac OS X can and will rearrange files on it whenever it needs to. This often happens when an application requests space to write to a file. Such rearrangements will lead to writes to both the file space itself and the disk catalog. Since stuff is always being written to ~/Library/Caches and ~/Library/Preferences, changes are always occurring to the sparsebundle when it is mounted as a FileVault home directory.

The author only tried mounting the image. He did not run hdiutil verify username.sparesebundle on it to check that the backup of the sparsebundle was truly intact. Even if it was intact once, there is no guarantee and in fact a strong likelihood that subsequent backups will be corrupted. There is a reason why Apple does not do portable home directory syncs of the sparsebundle while the FileVault sparsebundle home directory is in use. Setting up a sync to the file server using Sync Services would have been trivial, but the disk image corruption and sparsebundle band locking issues would have been much too difficult.

--Paul

ScottW - Posted January 12, 2008 08:00am CST

I have been running this method, Rsyncing every 4 hours, every day since prior to publishing this article to present. To address the previous posters issues, hdiutil verify xyz.sparsebundle does not work, because it only verifies sparse disk images, not sparse bundles. Sparse bundles are a different monster. This method is proven to work and works flawlessly from my experience. Running diskutil verifyVolume on the sparse bundles show everything is in order. Just because Apple doesn't do this with Time Machine, doesn't mean that it's not a safe way to run a backup.

masonk - Posted March 7, 2008 10:11am CST

Thanks for this tip, Scott. I might be missing something, but it seems that rsync has no -E option. It's not in the man page, or on their website. And so I was not surprised when mason@big-spring~$rsync -avE /Users/.mason/mason.sparsebundle /Volumes/data/backups/ rsync: -avE: unknown option rsync error: syntax or usage error (code 1) at main.c(875) What's up with that?

grusaren - Posted March 11, 2008 10:04am CDT

I have started to backup this way to see how it comes out, and it has worked perfectly for me. As a test I ran diskutil verifyVolume to see what happened at it returned an error: Incorrect number of thread records Don't know really what it means but it has not affected my ability to restore files. Question no 2: Why do you not use the -E flag on the second backup? Is it slower? I might be stupid here but I can't see the real difference. Otherwise, great post!

ScottW - Posted March 24, 2008 08:58am CDT

masonk: Type "which rysnc" in terminal to see if you have more than one version installed. grusaren: The -E flag copies special files associated with the Mac. Once the this has been done the first time, you don't need to do it again because no new resource files will be created for that bundle.