Update: Sadly, it appears I’m wrong. See the update at the bottom of this post and the comments. It was fun while it lasted. And my congrats to the Apple Time Machine UI team still stand. Brilliant. The original post follows.

Lest The Suspense Kill You: The Answer Is Yes
First of all, congrats to the Leopard Time Machine team for putting a beautiful and intuitive face on recovering data from snapshot copies. It is no accident that Scott Forstall, who introduced Time Machine at Apple’s World Wide Developers Conference, has the title “VP Platform Experience”. The Time Machine interface is brilliant.

ZFS On Leopard
I love competitive analysis, and this is a great example of the process. As I’ve noted before (see ZFS On Leopard: How Cool Is That?, Apple Mojo In High-End Storage and Bring Me The Head Of WinFS ) there is documented interest inside Apple in ZFS (for more info on ZFS see ZFS: Threat or Menace?), the coolest file system cum storage manager out there.

ZFS is open source, so it fits within Apple’s miniscule storage budget, and it has a lot of great features that would kick the Mac’s storage capabilities way beyond the aging NTFS used in Windows Vista.

Some of those great features are reflected in the Time Machine announcement.

  • Time Machine “makes a complete backup of all the files on your system” including system files
  • Infinite snapshots – the basis for Time Machine’s “time-based browser”
  • Totally automatic – you select the drive to backup to, select the items you don’t want backed up – and it goes

Teasing Out The Truth
Now, I’ve been in Apple’s position of wanting to talk about a future product without blabbing about how the magic works. I’ve gone through the announcement video and all the Apple webpages that reference Time Machine. They’ve been carefully scrubbed of any reference to the underlying technology. That is the first clue: the lack of chest-beating about this great new technology says it is not Apple-developed. Even with the DTrace announcement, where they buried mention of DTrace, they did claim some invention.

Clue #2: TM Is Creating Disk Clones
Looking at the list of features above is telling as well. The first feature could be implemented using the Darwin command-line utility ditto. While nowhere does Apple mention creating a bootable copy, if you copy all the system files, including the invisible ones, preserve symbolic links and resource forks – and they do stress the “complete backup of all files” – we can safely assume the backup copy is bootable. So either they are using a Unix/Darwin utility or they are copying the entire filesystem.

Clue #3: TM Is More Than Cloning
The ability to preserve a file system’s state at a particular time is known as a snapshot or point-in-time copy. The Mojo is conceptually simple:

  • A snapshot copy is created
  • Thereafter, any disk blocks that are changed are first copied
  • When returning to the snapshot copy, the copied blocks are substituted for the newly written blocks

This Copy-On-Write (COW) strategy minimizes data bloat – only the changed blocks are copied. Time Machine’s name and interface make snapshot copy management intuitive. TM’s snapshot copy capability is not easily added to Apple’s HFS+, which is why it hasn’t been done to date. Yet it is integral, and cheaper than overwriting existing data, in ZFS.

Clue #4: It Is “Set and Forget”
All backup software is fundamentally a kludge, since it is backing up files rather than blocks. So backups need to be scheduled, the application needs to run, and if the system is turned off or asleep, backup doesn’t happen. None of this applies to TM – so TM is not an application – but a system service. It has to be part of the file system, not some asynchronous agent that checks after the fact.

Clue #5: Remember When Apple Bought That Storage Software Company?
Me neither. Time Machine is non-trivial stuff, way beyond what Apple has done with file systems before. If they don’t claim they invented it, and they didn’t buy it, then they must have gotten it for free. ZFS is most advanced and solid open-source file system out there, so it gets my vote.

Any Caveats?
Well, there was the FireWire drive icon on the desktop of the demo system. ZFS would normally create a storage pool out of additional storage, rather than working with an individual virtual drive. However, if I were Apple, I’d want to reduce the changes users see, and having drives appear on the desktop is expected behavior.

Also, the option to not backup files surprised me – since ZFS works with file systems and manages data blocks. How could ZFS know to not back up a file? My theory is that TM accomplishes this trick using the individual file’s or folder’s Access Control List (ACL). I’m not a software engineer, so there may be a smarter method, but it is certainly plausible.

You Heard It Here First, Folks
So I conclude that Yes, Apple is incorporating ZFS into Leopard. Which is very cool, not only for us backup deprived users, but also for enterprise users of Xserve systems. ZFS offers many more capabilities than Time Machine uses. Over the next 10 months Apple will have a great opportunity to polish those up as well to give OS X users an unparalleled storage and data integrity experience.

w00t!

Comments always welcome!

Update: John C. Randolph says I’m wrong (see comments) and while he doesn’t identify himself further, a moment’s Googling reveals a John C. Randolph, Sr. Cocoa Software Engineer, in Apple Worldwide Developer Relations. Assuming they are one and the same I must reluctantly grant that he would know.

What I would love to hear now is how HFS+ will be re-engineered to actually deliver the data integrity and reliability implied by Time Machine.

And John, thanks for reading StorageMojo.com and commenting.