I’ve been a fan of ZFS from its early days and I’m a Mac user who has experienced data corruption thanks to its antiquated HFS+ file system. Thus I’m pleased to see that Greenbytes is releasing ZFS for Mac.
A post on ZDNet explains the details:
Now christened the “Zevo Community Edition” is coming out on Saturday the 15th. Works with Snow Leopard, Lion and Mountain Lion, 64 bit only. 4GB RAM required with 8GB recommended.
Has much of the original ZFS goodness, including snapshots, quotas, data scrubbing, mirrors & RAID Z – up to 3x failure protection – all heavy duty data protection features that the antiquated HFS+ never dreamed of. And, of course, the proven data integrity features of ZFS are standard.
But this release isn’t for everyone. It requires Terminal to set up – no fancy GUI. ZFS features such as dedup and hot spares aren’t supported yet. Most important: you can’t boot your system from it.
The StorageMojo take
It is depressing that Apple, the world’s most valuable company, takes such a cavalier attitude towards user data. Sure, few users are complaining about data loss – mostly because they don’t know what it looks like – but is this any different from shipping any known-defective product?
Back in the 60’s US car companies claimed that “safety doesn’t sell.” But today safety is a top issue.
Perhaps we are where car companies were 50 years ago. Let’s hope it doesn’t take 50 years to get more reliable file systems on consumer products.
Courteous comments welcome, of course. Kudos to Microsoft for their work on ReFS, which I hope to hear more about next week at SNIA’s 2012 Storage Developer Conference.
I think ZFS is good in concept, though my own personal experience with it has not been great, I’ve had data corruption on two different occasions in the past few months (purely software faults the hardware was fine). I’m not complaining as much about the corruption itself but how poorly zfs handles it. Just go looking online for how people handle data corruption in zfs, it’s really a nightmare. I had data corruption on Linux with ext3 once a few years ago on my home system with a crappy SSD and a UPS that had a dead battery, I was able to fairly easily fix it by booting off of a rescue cd, running a forced fsck, and deleting the directory tree that had the corruption. Before deleting the directory tree Linux would force the file system into read only mode when it came across the corruption – vs ZFS which seems to kernel panic the machine, a much less useful response, especially it means your system can be caught in a kernel panic reboot loop.
With ZFS, oh my god what a horror show it was, for a simple 200GB volume it took more than 24 hours to scan, I jumped through all sorts of hoops to try to repair or delete the corrupted data (I just wanted to fix the file system, I didn’t care about losing bits of data here or there, just delete the files that are affected).
In my journeys I came across many threads of others trying to recover from similar issues, and it is really apparent that, at the end of the day, if there does happen to be corruption in ZFS, more often than not your likely to lose everything, or perhaps wait weeks for commands to complete scanning the volume.
I believe both cases of corruption in my case was the result of clustering gone bad, I still have a support ticket open with the vendor to investigate the most recent case(after most recent case I just turned off clustering totally). I was hopeful I was able to save one of the volumes the last time around (I didn’t care about the data – I just didn’t want to go and re-create all the configuration associated with the volume), I even got it to a point where zfs scrub said it was clean (there was checksum errors but from what I read those are fine, as long as there are no read/write errors, checksum errors just means the data didn’t match up). No unrecoverable data errors though (this was after a good 48 hours of running various random zdb recovery tasks that I found online, undocumented commands in threads from Sun engineers). So I repaired this volume on another system, swung the LUN back over to the primary system, imported it, no problem. ran zdb scrub on it again, no new issues detected, I could access the data and stuff no big deal. I rebooted the system and BAM, kernel panic. The zfs vendor themselves said they don’t support cases where there is corruption, just delete the volume and restore from backups. If only there was a way to back up and restore the configuration associated with the volume (since that data is stored on the volume itself in hidden locations.
It seems to be a common complaint though that there is a lacking of tools and documentation to recover from corruption in zfs, at least from what I’ve read in various email threads, I suppose the more robust the thing is when it does fail it fails even harder.
I do think the integrated compression, and dedupe abilities of ZFS are nice, the file system snapshot ability is really nice (wish Linux had that – LVM snapshots don’t count).
My complaint is less about ZFS itself and how it operates and more about the lack of tools and docs provided to recover from serious problems.
Oh one more note – when it took me 24+ hrs to scan a 200GB volume I can’t IMAGINE the pain that people running TBs of data have to go through, I remember one post of a guy saying he waited more than a week for a command to return (I don’t recall the size of his volume).
One more thing I assume many ZFS folks will flame me for this story of mine..
Here’s one of the file results of zdb scrub for the volume I thought I could save (the other one was much more foobar)
root@TESTHOST:/volumes# zpool status
pool: scratch
state: DEGRADED
status: One or more devices has experienced an unrecoverable error. An
attempt was made to correct the error. Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
using ‘zpool clear’ or replace the device with ‘zpool replace’.
see: http://www.sun.com/msg/ZFS-8000-9P
scan: scrub repaired 42.5K in 0h22m with 0 errors on Tue Aug 28 08:18:01 2012
config:
NAME STATE READ WRITE CKSUM
scratch DEGRADED 0 0 0
c0t3d0 DEGRADED 0 0 39 too many errors
errors: No known data errors
Even in that state, I ran zpool clear to clear the errors (note “Applications are unaffected”). The system kernel paniced upon reboot trying to access the file system.
The fact that scrub found and fixed less than 50kB worth of problems (and had no problems fixing it apparently “0 errors”) was even more shocking to me how fragile it could be with such few issues.
I believe the zdb command I used originally was “zdb -e -bcsvL “, according to the log I see here – again zdb totally undocumented and I just happened to come across a post from another user that was having the exact same kernel panic as I was, and with help from a friendly Sun engineer he was able to get that command syntax to work for him.
ZFS without ECC memory – Perish the thought!
iirc only the macpro and xserve have ecc memory
nate, if you want snapshots on linux I have had an nilfs2 filesystem running fine for more than 2 years now, without any problem whatsoever (on server quality hardware, ECC RAM, raid-6 array). Nilfs2 really deserves some more loving 🙂
Nate, I think you have misunderstood how ZFS works and what it will do for you.
From what I can tell, you are running your zpool on a single disk. While ZFS works just fine on a single disk, you should expect more problems with it than with other file systems. Why? Well, it’s very simple, ZFS will detect a lot of errors that goes unnoticed in other filesystems and tell you about them.
No filesystem will really give you redundancy on a single drive. You may be able to repair things using fsck, but you should never entrust anything important to a single drive storage.
A ZFS filesystem without the backing of RAIDZ vdevs in your storage pool is pretty much useless for anything but stuff you want to play with. All the redundancy properties we love about ZFS comes from the redundancy of the storage pools.