I hadn't considered FSEvents. I remember hearing about them before Leopard came out. From a quick look through the docs, it looks like it's really meant for real-time monitoring of folders for changes, though. I'll take a closer look later.
I just ran a little test to see what the maximum speed I could expect would be when going through every file. I ran this:
Code:
find ./Backups.backupdb -print >> ~/Desktop/finddump
I let it run for 10 seconds to see how many files it could return. After those 10 seconds, the text file had 47,000 entries in it. Not bad at all. There are roughly 470,000 entries (files + folders) per snapshot, so that makes rough math very easy.
So, 100 seconds per snapshot, 50 snapshots (increasing by one per week). That's 80 minutes just to dump the path names of all those items, without any special processing. Since my processing will most likely take longer than the directory traversal itself, I guess I need to lower my standards down to "fast enough to complete overnight".
My program currently goes through an abysmal 1200 entries in 10 seconds, almost 40x slower. Of course, it's doing a fair amount of processing in there (creating a SQLite database for future use, and of course recording inodes), and it would get a lot faster after it finished the first snapshot (which would take about 65 minutes, at that rate). I'll have to test how [NSFileManager enumeratorAtPath:] performs when I don't do any special processing, so I can better compare it to find. If it turns out to be a bottleneck, I'll rewrite my program. If it's only a bit slower, I'll accept it and look for something else to optimize.
Thanks a lot for the ideas, guys.