Apple’s touchless file system conversion patent
Make HFS to ZFS conversions fast and painless
Let’s say you wanted to convert millions of computers to a new file system. How would you do it? Back up, reformat the disk, and then reinstall everything?
Or run a handy file system converter that changes the file system without touching the files?
What? You can do that?
According to a patent filed by a couple of smart Apple engineers, yes you can. The hardworking Greg Keizer mentioned it in this Computerworld article. I thought it was worth a deeper look.
What it does - and how
The patent says:
. . . a first file-system can be converted to a second file-system while . . . files stored on the computer . . . remain virtually undisturbed in the same location. . . . the second filey-system can be generated primarily based on the location of the file(s) already stored on the computer . . . . It will also be appreciated that there is no need for rewriting or storing the file(s) while the file system is being converted.
The converter reads the existing file system to find out where all the files are on disk. Then it creates a new set of data structures, such as a catalog and file extents, for the new file system. After verifying the new data structures, the converter can then replace the first file system by, for example, modifying the disk’s partition map and and overwriting the volume headers of the first file system.
Here’s the flowchart for the conversion:
HFS+ to ZFS conversion is obvious
But is that all there is to it? The patent makes the point that the converter could be embedded in iTunes and used to convert the file system used on a media player. I can see the dialogue box now:
You have attached a Zune media player. Would you like iTunes to make it compatible with your iTunes music collection and the iTunes Music Store? All existing Zune music will be saved.
Pop in an AACS decryption program - or maybe this happens after the labels give up on encryption - and away you rock.
Set Reality Distortion Field to “global domination”
Or how about this. Apple supports Mac OS X on non-Mac hardware. Change the file system, use Rosetta technology and the open-source Wine Windows emulator to run existing Windows apps and it is a whole new world.
Instead of shipping a few million copies of OS X every year on low-margin hardware, Apple suddenly has the opportunity to sell tens of millions of high-margin copies. The hardware business continues, because Mac hardware is good, and suddenly there is a huge new market for Apple to conquer: the world’s installed base of 700 million PCs.
The StorageMojo take
Touchless file system conversion is a very cool capability. It makes it plausible that ZFS will become the default file system for Mac OS 10.5.n. I still don’t think ZFS will make be the default file system in Leopard from day one, if only because of the Q&A advantages of rolling it out first on OS X Server.
As for the less grounded speculation: Apple is a profitable and cash-rich company with a strong brand and high-quality hardware. They are about as well-positioned to offer Mac OS X on non-Apple machines as they will ever be. If they partnered with IBM for support they would have a ready entree into a business market tired of the foul stew of viruses and malware in which Windows machines swim.
It is a charming thought, no? Real competition on the world’s desktops.
Comments welcome, of course.
on June 10th, 2007 at 2:14 am
Probably not. Apple is not interested in giant leaps to world domination. They’re doing well at what their current strategy is, which is identify market areas that could be done better and apply some minor to intermediate innovations and profit.
Everyone keeps pointing to how Apple could and, in their opinion, will dominate the world of mass-market PCs. Delusions of grandeur I say. They’ll take baby steps into other people’s territory if anything.
I really think the iPhone is a baby step despite how the media has hyped it up to be. I’m excited about the iPhone too but Apple isn’t saying or acting like they plan to put Motorola, Nokia and LG out of business. They just want their implementation to do well because they believe it’s the best.
Back to ZFS, I give it an 95% chance that it will replace HFS+ as the default CLEAN install file system. As for the converter, I really hope it works out but I’d give that much lower odds. Guess we’ll find out on Monday!
on June 10th, 2007 at 9:05 am
Apple will NOT support the huge assortment of desktop PC hardware, period! IBM may well contract to support the software; but tackling the OS hardware issues and drivers is something Apple will continue to avoid. Apple hasn’t used Intel’s new desktop chips which are now deploying with Vista for instance. You’re suggesting that Apple fight for the business desktop market they lost in the ’80s while this market gets attacked by web 2.0 and Linux. Apple would have to provide forward guidance that they keep secret for marketing purposes now. It doesn’t make sense as Apple’s doing quite well without changing their culture as the majority of their desktop customers are business customes with their limited perspective of computing.
on June 10th, 2007 at 10:32 am
Gosh! Novel!
Oh wait… here is an implementation for Linux:
http://directory.fsf.org/ConvertFS.html
As you can see from the homepage, the first release was on August 23, 2001:
http://tzukanov.narod.ru/convertfs/
on June 10th, 2007 at 11:35 am
Matt,
The analogy is far from perfect, but instructive nonetheless. For a couple of decades Ford Motor was the world’s largest automobile manufacturer in unit volume and sales with the Model T. And then the market changed and Ford didn’t move fast enough. General Motors took the lead and been the world’s largest car company ever since. Yet now Toyota, with years of steady work, is ready to take the crown. My point is a modest one: just because Microsoft has been the desktop leader doesn’t mean they always will be. Steve Jobs is a hyper-competitive guy who loves to win. Apple is changing the competitive landscape and changing with it. With iTunes Apple is now on the huge majority of Windows machines, showing users how software could be. If they keep Microsoft on the defense, Apple plays offense.
There is nothing Toyota has done that GM couldn’t do. They just didn’t. Microsoft is under attack on numerous fronts, as they should be. Apple just happens to be the best funded and most creative contender.
Robert,
I agree that Apple wouldn’t support a lot Wintel configs. As I thought about it some more it occurred to me that perhaps the Vista Pro configs from 3-5 major vendors would be a manageable number. I don’t rate it as likely, but Steve had OS X running on Intel for five years before they released it. He is willing to invest in technology that gives Apple strategic flexibility. The same team probably has OS X running on HP, Dell and Lenovo PCs today, learning about the potholes they’d face. As Microsoft tries to improve customer satisfaction by requiring more uniform hardware from vendors, they play into Apple’s hands.
Ben,
Thanks for the links! I’d been wondering if anyone else had the same idea. I’d note that these utilities are limited to a single OS - Linux - and require support for sparse files. The Apple patent is a cross-platform technology. Like most such ideas there are worthy progenitors who point the way.
Robin
on June 10th, 2007 at 9:40 pm
‘convertfs’ is interesting, but misses a very important point from the Apple patent… No data copying.
From the convertfs page: “moves files from the primary filesystem to the mounted image”
From other stuff on the pages, it’s not clear to me that the original filesystem goes away either - it seems the new filesystem is created within the original FS… Not apples and apples.
For me the whole point of the Apple patent is that only the meta-data is changed - the data is left inplace.
It’s like doing an in-place sort - the clever bit is twiddling pointers, not copying massive amount of data.
on June 10th, 2007 at 9:48 pm
Apple seem to be a very focussed *hardware* company. I think it unlikely they will change soon.
They’ve had many chances to license their software - 68000 systems and the PPC reference platform (CHIRP?) - but haven’t.
The only way I can see them keeping their focus, ’staying pure’ perhaps, is to kick-out Mac OS/X into a separate company - but that leads to problems like Palm has had…
They will have a window of opportunity if Microsoft ever hits a financial crisis like IBM in 1989/90 - or if Microsoft gets into legal problems with consumer protection legislation (Operating Systems should *operate* if the hardware is OK. It can’t be that hard - hobbyists built Linux if their notional backyard/shed for nothing, and it passes that test much better.)
on June 11th, 2007 at 7:29 am
If Leopard was available to run on a range of specific hardware, e.g. nForce 4 and above, a few Intel chipsets, I’d line up over-night to get a license. I’ve got a G4/533 with OSX-Tiger and an Athlon64/nForce4 PC, and would dearly love to be able to run OSX on the PC, and the only reason I can’t is drivers.
on June 11th, 2007 at 9:57 am
Hmmm… so I’m not supposed to read patents ( http://storagemojo.com/?p=460 ) and yet the flowcharts and main points have been copied right into blog posts… this is getting tricky.
on June 11th, 2007 at 7:12 pm
Steve,
It was one thing to license low-volume companies to build Mac knock-offs. Selling Mac OS X to consumers and maybe to OEMs is another business entirely. Not that I rate it at all likely in the near term. But if Msoft keeps screwing up, Apple could swoop in five years from now.
Also, note the Safari on Windows announcement. Apple is starting to appreciate high volume.
“If you like iTunes you’ll love Safari!” Followed by: “If you like iTunes and Safari, you’ll love OS X!”
Jeremy,
Bwa-ha-ha-ha-ha!
Robin
on June 13th, 2007 at 5:03 am
Prior art: Alsoft’s DiskWarrior has behaved in this manner for years, although it doesn’t change the type of the file system. It repairs a possibly-corrupt file system by reconstructing all the metadata from scratch. It’s a short step from there to changing the type of the metadata.
on June 14th, 2007 at 6:20 am
[...] StorageMojo » Apple’s touchless file system conversion patent Potentially painless HFS to ZFS conversion patent (tags: pantent apple zfs storage computing) [...]
on January 23rd, 2008 at 3:17 am
[...] 29th, 2007 · No Comments I found this article on Storagemojo. Seemingly, Apple has come up with a filesystem converter. Aremarkable breakthrough! Just imagine the ease with which I could experiment, I have to keep aside an entire drive while trying out new things and this is a PITA. Instead this patent filed on the 1st of August is going to eliminate most of my problems (err this utility will be available on Linux right ?). I respect the fact that Apple is coming up with something like this and guess what, Apple is moving to ZFS ? I am of the opinion that Apple will have to mount its boot partition on something else as Sun itself has been unable to boot OpenSolaris off a ZFS partition. Still, by October, ZFS is going to be rid of that problem. Lots of cool stuff happening in the storage domain. [...]