With the release of Snow Leopard it is now official: no ZFS – anywhere – in Mac OS 10.6. Given that Apple went to the trouble of announcing it last year as part of Snow Leopard Server this is quite a reversal.
The question is why?
Many theories
I wrote this up on ZDnet Friday. At the time my theory was that the integration schedule or migration issues turned out to be less manageable than once thought. Or maybe NIH reared its parochial head.
ZDnet readers wrote in with ideas as well, the most popular that technical issues with ZFS itself forced the issue. I discounted this because, after all, ZFS is in production in large, I/O intense environments. If it is fundamentally broken we’d know by now.
I follow the ZFS discussion list and while there are issues, they aren’t show stopper bugs.
A new narrative
But then a couple of sources came in with a new angle: that Sun’s licensing demands killed the deal. Sun prefers the CDDL and may have asked for some extra protections, including Apple’s promise not seek damages should Sun lose the ZFS patent infringement suits initiated by NetApp, that caused Apple to reconsider the business risk of ZFS.
Sun could, of course, GPL ZFS, but it may also be that the ZFS engineering team – like other Sun engineers – rejected GPL. I’d love to get some comment from the ZFS team – very bright guys all – because this reminds me of the late ’80s at DEC when senior people begged DEC founder and CEO Ken Olsen to essentially open source some of DEC’s advanced software, like VMS, VMSclusters and DECnet.
Ken, a very smart engineer who shepherded DEC from a $70,000 startup to a $14 billion company, couldn’t see the business sense in giving away what the company had spent millions developing. So that leadership technology withered as DEC cratered.
The NetApp lawsuit may have come into play, making patent risk pertinent and potentially costly. Given that and the other CDDL-related risks, plus engineering opposition to GPL, Apple must have reluctantly stepped away. Apple would like bragging rights over Windows 7 that ZFS would give it, but in this narrative Sun’s pre-acquisition turmoil and tougher-than-expected licensing terms killed the deal.
Going forward
Now that Oracle is acquiring Sun things look brighter. Oracle is already bankrolling a GPL’d ZFS clone – btrfs – that will take years to reach the level of maturity that ZFS now enjoys. Once they own ZFS why wouldn’t they GPL it and call it good?
Update: Also, Oracle is in a stronger position to negotiate a settlement with NetApp over the ZFS/WAFL patent suits. After all, why would a storage company want the world’s largest database company as an enemy? End update.
The StorageMojo take
This is speculation of course and no doubt missing many specifics. But what is public – that Apple announced ZFS in June 2008, included a read-only CLI version in Leopard Server and is not shipping it in August 2009 – is evidence enough that things went awry. What other than a license issue would cause Apple to step away from even the read-only CLI version in Snow Leopard Server?
The ZFS team has produced a game-changing file system/volume manager. The chance to get it into the hands of 10s of millions of Mac users – and to influence Redmond’s file system strategy – seem to this outsider an opportunity of a lifetime.
If the ZFS engineering team opposed this – and I’d love to hear their take – I encourage them to reconsider. Marketers often ask the question “would you prefer 100% of nothing or 40% of something huge?”
Once the acquisition of Sun is complete, I hope Oracle quickly GPLs ZFS and cuts a deal with Apple. It will be good for them, for ZFS and for the entire industry.
Courteous comments welcome, of course. I worked for Sun for 3 years in the mid-90s and despite the many problems in the storage group I remain impressed by much of the company’s culture and accomplishments.
Update: I got the indemnification issue backwards in the original post and I thank those readers who deciphered my intent. For those who didn’t, I corrected it. While I was at it I made some other edits for clarity.End update.
Great article! I enjoy your posts and your dedication to fact finding it helps us all. I am a Mac user myself and I have been anxious so see ZFS more fully implemented into 10.6, but not that this will not happen it is what it is. I have moved on and have setup an Opensolaris machine with ZFS v19 and shared it to my macs to do sparse image backups with time machine and it works wonderfully as a replacement for money that would have gone apples way. Why am I telling you this?
First off, as I mentioned earlier I am a Mac user and, I should add, a strong SYS V derivative user/administrator/developer as well (ive been a fan of the BSD’s and Solaris since 7 linux as well) and as such this is a very torn subject for me. You see, I really love technology, the ideas that it spurs and working with people that share this passion and as such it is downing when something like this happens (as you mentioned with the DEC scenario, I remember the Alpha’s I had and how I loved them)
Secondly, still a bit torn here on GPL’ing anything at this point to be honest. I am really not trying to be nostalgic here, but I remember when times were different and Open Source didnt just mean downloading something from the internet slapping it together with some glue code and following the latest paradigm because someone wrote a book on it; to be fair alot of this stuff is really cool and some really smart people have done great work (Lucene, Hadoop et all), but all too often this falls into the hands of some who have no experience in the industry who dont understand why a “best practice” exists and why NOT to do something that has not been done. Apologies if this sound like philosophy here but I still believe that I should still have an opinion even if it is more of a gripe. I am sure you have thrown me right into the GPL/OpenSource hater category, but just wait before you throw that stone. What I am trying to say here (badly) is that people often learn the wrong things from things like DEC’s troubled leadership.. sometimes we swing too far in one direction, DEC had some great products and ideas just as Apple does now does that mean that they are both alike in this way? probably not. DEC was never really about consumers, Apple REALLY is and I think we have to compare apples to apples (damn sorry, no pun intended.. really).
All too often people say “just GPL it and it will take off, will fix our troubles”; this has never been proven to work, that feedback is anecdotal at best.
@Tom: recent zfs has options for case-insensitive behavior.
That would of course not be the default on Solaris, and I suspect
was mainly done for CIFS (SMB) serving to work more smoothly.
@Eric: AFP support would of course be Apple’s problem. It’s
up to their OS architecture how much the disk filesystem needs
to be aware of being served out via AFP, or vice versa (although
to be consistent with the rest of zfs’s behavior, adding a shareafp
property wouldn’t be unreasonable, IMO). While Sun doesn’t support
AFP, I’ve got netatalk running perfectly happily on Solaris 9 and
SXCE; it doesn’t take advantage of zfs features to do a nicer job
of storing resource forks and Finder attributes and such metadata,
but in principle it could be modified to do so, allowing OSs supporting
zfs to do a very nice job of AFP serving.
The most credible explanation I’ve seen (aside from that it
might not have been quite ready yet anyway) is
http://mail.opensolaris.org/pipermail/zfs-discuss/2009-October/033125.html
Unfortunately, I suspect that it boils down to another case of
lawyers trumping functionality…
Okay, a new thought after the release of the SSD only MBA (Oct 2010). Given that the Apple has gone the route of repackaging the SSD in the new MBA and that they made it removable, it would seem that Apple has interest in the repackaged SSD beyond the MBA. Taking SJ at his word that the new MBA IS the further of the Mac notebook, that would imply that the next refresh of the MBP lineup will likely be sans the hard drive and adopt SSD. But assuming that Apple needs and wants to hold the line on pricing of its SSD, assuming a 2X denser flash in the near future (ie to provide at least 500GB of storage), might not be the best approach. Rather, I suspect that Apple is planning to install 2+ of the existing new MBA SSDs into the next gen MBP and adding either ZFS or the equivalent so that the end user sees a single drive (ie just as they do currently).
This just makes so much sense that it has to be. I am betting 10.7 has many surprises yet to be discussed and a pooled volume managed file system based on multiple SSDs is just one.
dt
Now that the lawsuit between Oracle and NetApp has ended, what’s the possibility of Apple redoubling its effort (already quite advanced) to bring ZFS to the Mac OS X system??
@Eric D – I hope so! Time Machine is In desperate need of block level snapshots.
Also Dominic Giampaolo, who wrote the Be operating file system, is still working at Apple – I was hoping for some sort of successor to HFS long before now. I can only hope Apple is going “all in” with a radical SSD based file system approach and that’s what’s taking so long…