Ding, dong.
PC file system progress took a giant step back this week with the news on MacOSforge that Apple’s ZFS project has been discontinued.
ZFS Project Shutdown 2009-10-23
The ZFS project has been discontinued. The mailing list and repository will also be removed shortly.
Apple announced in June ’08 that Snow Leopard server would support ZFS. But things came apart early this year.
What happened?
Jeff Bonwick, ZFS architect, posted Saturday on an earlier quoted comment:
> Apple can currently just take the ZFS CDDL code and incorporate it
> (like they did with DTrace), but it may be that they wanted a “private
> license” from Sun (with appropriate technical support and
> indemnification), and the two entities couldn’t come to mutually
> agreeable terms.I cannot disclose details, but that is the essence of it.
Jeff
Indemnification?
Sun is being sued by NetApp claiming that ZFS infringes on NetApp patents. If NetApp won, Apple would find itself in a tough position unless Sun shouldered the financial damage. That’s indemnification.
IMHO Sun has a good case that NetApp’s patents will be invalidated by prior art. But with all their other problems and the Oracle purchase it was a headache they, Oracle and Apple didn’t need.
Where does Apple go from here?
Apple has hired some smart file system engineers and wants to hire more to work on “state-of-the-art file system technologies for Mac OS X.”
I’m not convinced: it sounds like standard HR boilerplate and a snare for the unwary. But hey! it could happen.
But writing new file systems isn’t easy. It takes 5-7 years for a new file system to achieve the maturity needed to support large-scale deployment. Even replacing QuickTime is non-trivial.
So if Apple is starting from scratch we have a long wait for real innovation to appear. Like Mac OS XII.
What about Microsoft?
Meanwhile Redmond’s file system gurus are well aware of NTFS issues. They’re making stepwise enhancements.
But as the NTFS and HFS+ architectures age and the pace of storage innovation increases the gap between what is and what could be grows. It’s like putting a 1001 hp Bugatti engine in a Model T: the power is there but you can’t use it.
The StorageMojo take
I already hate software patents – but that’s another post. As long as law allows companies will try to enforce them.
Why didn’t Apple cut a deal with NetApp directly? Probably for the same reason Sun didn’t: money. Apple has a lot more of it than Sun, but Steve is a tightwad, especially when it comes to storage.
NetApp could have raised their visibility in the consumer market by cutting a deal with Apple, but NetApp’s management isn’t thinking strategically about the low-end of the market, as the rapidity of StoreVault’s entrance and exit demonstrated. True, they have bigger issues, but multi-tasking is supposed to be a corporate strength.
Consumers are generating masses of video and photos at an accelerating pace – and they’ll need reliable, available and dirt-easy storage. Lots of it.
Let EMC supply it!
Until the Next New Thing in file systems rolls out of Cupertino, Redmond or, maybe, Redwood City, consumers will stuck with too many BSODs, missing or corrupted files and app crashes. Let’s hope we don’t have to wait too many more years.
Comments welcome, of course. An earlier version of this was posted on Storage Bits. Can you spot the dozen or so differences?
And there is a Google code page for MacZFS for you diehards out there.
Unlike many others, I’m not ruling out btrfs yet. Filesystem developers are, of necessity, kernel developers. To the extent that btrfs has dependencies on Linux models or implementations, filesystem developers can resolve those dependencies either by providing the same facilities in another OS or by modifying btrfs to remove the dependency. The effort would be comparable to porting ZFS from Solaris, and either would be far *far* less work than creating a brand new filesystem from scratch. The fact that Apple’s interest in ZFS seems to have waned exactly as btrfs has become usable might not be mere coincidence.
What I don’t understand is why with a variety of filesystem types out there are people so determined to build their own? Save some development costs and use something already tried and true, or better yet, help set a standard so filesystems can be mounted on any architecture.
Isn’t ZFS open source? Can’t anyone use the code? Doesn’t it (or can’t it) run on Linux? Or, isn’t ZFS part of Solaris which is also OSS? And can be free?
Barry,
Sun open-sourced ZFS to keep the engineers happy, but NetApp says some of the IP in ZFS isn’t Sun’s. So yes, it is open-sourced, but there is an expensive lawsuit over who owns what has been open-sourced.
Once that gets sorted out we’ll know if Sun was within its rights to open-source ZFS. If they weren’t, the people who are using ZFS will be liable to NetApp.
Robin
I wonder what Implications that law suit has over Nexenta for example.
Which a really nice product, based on Open Solaris.
Apple dropping ZFS is a big disappointment. Judging by the comments on many other blogs (not here :-), most people wouldn’t know what a file system does, or why ZFS is better even on laptops, if it bit them someplace painful.
And that might be part of why Apple stopped struggling to integrate ZFS with Mac OS X. Why sink a bunch of engineering effort, legal fees, and expensive NetApp license fees (potentially useless depending on the lawsuit outcome), to implement a feature most consumers won’t understand?
Can you see the Mac vs PC ad that hypes “more reliable disk storage”? Neither can I. Snapshots — Time Machine does that. Other ZFS features are really obscure.
I don’t think consumers understand OpenCL or GCD (other than “it’s faster”), but Apple invested in those technologies.
Rex,
I would venture to guess that most Mac users still don’t know what their features are doing, they just know that they like them and that they work. If a consumer’s iPhotos don’t get silently corrupted, that’s a happy consumer; etc. I could see “operating system that makes sure your data is safe all the time” as a point in a Get a Mac ad.
And as far as Time Machine being snapshots, they’re hardly comparable. Time Machine requires a second drive (which is still the necessary for actual backup), and is a huge space hog.
Wes,
Good point. Apple (like Microsoft, Google, and other “platform” providers), really have two sets of customers: developers and purchasers. OpenCL and GCD enable developers to create better apps that will sell more Macs. What features of ZFS will attract a developer, other than the nebulous “less disk corruption”? Developers have lived with horrendous disk corruption on Windows for decades, and some make good money off of cleaning up the mess!
Alan,
“makes sure your data is safe all the time”, and snapshots are both features of a current product (Time Machine), that enables Apple and third parties to make more money by selling disk drive #2. Selling consumers on the somewhat subtle differences between ZFS and Time Machine would be tough. “New! Improved! With end-to-end checksums!”
I really want ZFS in Mac OS X. I’m unhappy Apple dropped ZFS. I’m pointing out that selling ZFS to developers and consumers is tough, and might be one factor in Apple’s decisions.
In all fairness, unless you were/are a Sun shop, the ZFS technology really didn’t bring all that much to the table. That’s why most people yawned. Everyone is already using some sort of abstract storage pooling mechanism… there are lots of solutions already out there (long before ZFS).
ZFS gets too many undeserved high marks for curing cancer, etc. It’s not a panacea. Follow the Sun forums and groups and you’ll see what I mean. ZFS is a glorious thing compared to what SUN had. But not all that compelling for the rest of us.
“ZFS is a glorious thing compared to what SUN had. But not all that compelling for the rest of us.”
ZFS would be an awesome thing to the Linux community, if they had it. None of the prevalent file systems, … ext3/4, reiser, or any of the others come close, and BTRFS is not ready.
Joe