The convoluted path of OS X ZFS is taking another turn. Greenbytes, who bought Zevo from Tens Complement, will be transitioning the Mac ZFS product – Zevo – out of their company sometime this summer.
Where?
That’s the question. Can Zevo be part of a reasonable business model?
File systems aren’t robust businesses. Few users understand why they might want a better one and fewer are willing to spend money for one. Even one as advanced – especially compared to HFS+ – as ZFS.
Apple, of course, would be the best and most logical home for MaccZFS. But unless Jony Ive takes a sudden interest in Mac data integrity, that’s unlikely.
Five years ago they were hot to do it, but since then interest has been nil, despite Microsoft’s impressive investments in ReFS. Microsoft has an enterprise business to worry about. Apple doesn’t.
Who else might build ZFS into a competitive advantage?
- VMware or Parallels. These produce solid Mac virtualization products with not a lot of differentiation. Zevo would be a brag-worthy option for either.
- WD or Seagate. Other than Drobo, there’s not a lot of differentiation among SOHO arrays. With fast-growing external storage businesses, either WD or Seagate could corner Mac pros/prosumers, who have good reasons to care about data integrity. Since most over $1k PCs are Macs, that’s a real market that Apple has no interest in.
If no one steps forward, Greenbytes will likely open-source Zevo and was their hands of it. That’s the least good option.
The StorageMojo take
Greenbytes has a growing VDI business to focus on, and Zevo doesn’t fit that. While pleased to see them take Zevo on, it was never clear how it would advance their business.
Nonetheless, they invested in the product, producing a free community edition, and they’ve left it better than they found it. Can’t fault them for trying.
Greenbytes welcomes contact from interested parties.
Courteous comments welcome, of course. Rumor has it that engineering turnover in the OS X team is increasing. I wonder why?
It may be a bit too early in ReFS’s development, but I would love if you did a comparison of ZFS and ReFS sometime! Granted, it’s not an apples-to-apples comparison as some of the storage features (Storage Spaces, tiering, etc.) on the Windows side are handled outside the ReFS, but it would be interesting to see how they compare where they share functional areas. (An entire storage stack picture would also be very interesting, though!)
As free software, Mac ZFS was never in jeopardy. It has always existed, and has been working perfectly fine, without any press or corporate support after Apple got it started. ZFS is free software and ZEVO is the poster child of exactly why people should be concerned with software libre (source code), not just software gratis. Without the source code, the software that users entrust their private data and infrastructure to, will always be a dead end.
We have been developing a new generation of MacZFS, which is currently undergoing beta testing and code refactoring. Everyone is welcome to help at MacZFS.org . Thanks!
As Daniel Bethe said, with source code available it’s always possible for a few interested people to pick a project back up and at least handle bug fixes. Particularly for something as important as primary data storage, source code is an important final safeguard in being able to access that data down the road. At the same time though, it’s fair to say that MacZFS is also an example of how an pure volunteer project doesn’t always work that well either. Many people weren’t extremely excited about ZEVO for no reason, it was because MacZFS was horribly out of date and developing at a glacial pace, while Z-410 (later renamed ZEVO) was actually modern and fully functional. That’s not inherent per se, the FreeBSD project has done very well (better then either Mac project), but the amount of progress ZEVO made in a short time shouldn’t be casually dismissed either. I wouldn’t mind seeing a hybrid, with source code available but also a company behind it as well.
Additionally:
>>”File systems aren’t robust businesses. Few users understand why they might want a better one and fewer are willing to spend money for one.”
What do you mean by “robust” here? A directly related Mac-based example would be SoftRAID, a commercial alternative software raid solution that has been offered and updated continuously since 2002. Sometimes analysts get too caught up in “market share” and forget that while “few users” may care, when the installed base is measured in the tens of millions it doesn’t take a very big percentage to keep a small business going. A file system doesn’t take hundreds of people and a huge marketing budget to maintain, a very small business can do perfectly well by doing a very good job of serving a profitable niche (writ large, that would in fact describe how Apple survived for many years while restructuring and doing the R&D to get into consumer electronics and then handheld computers). Small niches with strong preferences tend to be much more willing to pay decent prices: SoftRAID goes for $130, and you see the same pattern over and over again in any specialist area (CAD, graphics design, etc can go for thousands to tens of thousands per license). These aren’t smart phone apps for a buck or two a piece.
> … source code available … a company behind it …
+1
It worked well for a while in the past, it could work well again.
I encourage people to think constructively and positively about this option, and others.
Unfortunately none of the Mac ZFS ports are sufficiently complete to allow you to boot from ZFS, and it’s not such a good idea to put your whole home directory on a ZFS volume either, due to differences between HFS+ and ZFS that various Mac things seem to rely on. Which is a shame – I’d pay for a rock solid ZFS boot filesystem.
It would be awesome if Apple were to buy Zevo and commit to a better filesystem but it seems unlikely.
Great points!
I’ve followed MacZFS forever. I have no insight into Apple decisions, but I think that clean integration with the OS X VFS layer required more resources — both developer/QA effort and system RAM — than Apple was willing to commit. They went ahead and shipped Time Machine by hacking HFS+ to allow hard links to directories in a restricted way, cycles not allowed, and the API for that will never be public.
I suspect it was mostly a RAM issue, then a tough project-management issue, where the filesystem would impact lots of special-case userland code, require effort from many different teams.
Apple has demonstrated that they can marshal the engineering effort when the want to do so. But just as they were coming to terms with the scope of this effort, Sun entered final-stage Oracle acquisition. Sun’s team could not tell Apple exactly which features would be licensed, because the Sun group responsible no longer had the authority to do so. At the time, engineers ran Sun and could commit to projects that they felt would pay off. Oracle was run by the sales people, who told Sun to shut up.
Pure speculation on my part.
Frankly, things have turned out better than I expected, with a still-active community of businesses built around open-source ZFS, zpool version 28.
But after Linux btrfs showed everyone that copy-on-write can work can work with tree-structured filesystems, I presume that Apple figured they could build on that for less money than the commitment to ZFS required, and have something perhaps better at the end of the process. Better technically, better in terms of Apple corporate control, better feature fit with OS X in 2016.
(I just found a good blog post that pretty much comes up with the same conclusions. Worth a read if you care to dig deeper.)
http://www.devwhy.com/blog/2009/10/24/the-loss-of-zfs.html
Developing a filesystem takes years. It’s not something that you can do overnight, and it’s clear Apple was only interested in ZFS if they could change the licensing, which failed. It took Microsoft years to deliver ReFS, it’s take Oracle and other Linux kernel developers years to develop Btrfs, and it’s still not released.
Unfortunately, HFS+ is the Achilles Heel of OS X and iOS operating systems, and Apple knows it. However, about 7 years ago, they had a job posting on their site for a filesystem engineer, so it wouldn’t surprise me to learn, that in a year or two, they announce their new copy-on-write filesystem.
I don’t understand the emphasis on copy-on-write with ZFS. Sure, very useful feature, snapshots and all. But what I’m a hundred times more worried about is silent data corruption aka bit-rot. ZFS allows proper on-disk check sums for discovery and repair of bit-rot.
All the backups in the world are useless, if what’s backed up is corrupted data. For the same reason ECC RAM is vastly underused and under-appreciated: a few sometimes-bad bits in RAM, and disk buffers being flushed write bad data “correctly” to disk.
Ooops, hit that “Submit” button too soon.
The other thing: why would ZEVO being open-sourced the worst alternative? MacZFS community has been fighting an uphill battle with getting an open-sourced version of ZFS to the community.
If they had a much more modern code base to work from, I’m sure maintenance and further development would be much easier, and things could be fixed a lot more quickly than with any company that at any given point asks if the revenue is worth the extra engineering resources.
From the LinkedIn profile of a former Manager, File Systems Engineering at Apple:
> CoreStorage (2012-2013) Drove innovative new storage features.
Let’s hope that it was good, and that the innovations continue under a new manager. Less closed source will be good.
EVERY IMPLEMENTATION OF ZFS IS FLAWED:
OpenZFS
Pool export is not automatic in Open ZFS on OS X. Before unplugging any devices you must export all pools used by the device.
https://openzfsonosx.org/wiki/Zpool
MacZFS is incompatible with version 5 implementations such as FreeBSD, OpenIndiana and ZEVO.
http://zevo.getgreenbytes.com/forum/viewtopic.php?f=4&t=13#p3839
ZEVO
Does not support Mac OS 10.9+
http://getgreenbytes.com/solutions/zevo/wiki/
MacZFS requires a GUID Partition Table (GPT)
https://code.google.com/p/maczfs/wiki/GettingStarted
USB drives are not safe to use with ZFS
https://code.google.com/p/maczfs/wiki/USB
_________
ZFS HAS SERIOUS LIMITATIONS:
– no versioning synchronization between different platforms
– no versioning synchronization between different implementations on the same platform
– no kernel support for ZFS in 32-bit Linux (although there is a FUSE extension)
– no Windows support for ZFS
– no GUI tools for managing ZFS volumes
So far as I am concerned, the practical application for ZFS is limited to NAS devices accessed through the network; it is not suitable for removable storage devices or cross-platform file transfer. That may be obvious to people who build RAID systems, but is not readily obvious to casual users who just saw the hype about ZFS being “the last word in file systems” and wanted more reliable backups. Call me picky, but I would not use ZFS on the desktop until there is a version-synchronized cross-platform solution with a GUI interface.