I’ve been a fan of ZFS since I researched it over a year ago. I’ve also been happy with the progress ZFS is making on OS X.
So it was a bit of surprise when I saw (thanks Wes) that MacJournals, a developers web site, was all sideways about it.
A good conversation
Fortunately a former Mac file system developer, Drew Thaler, responded with Don’t be a ZFS Hater.
Another respected Mac developer, Michael Tsai, also responded with a thoughtful post.
The StorageMojo take
I follow the ZFS discussion on OpenSolaris, so I understand that the ZFS implementation has a ways to go. From a marketing perspective, ZFS or something like it is required if consumers are going to use computers as media centers for purchased content. Seeing a couple of thousand dollars worth of music, TV, movies and videos go poof! is a sure way to get tossed out of America’s living rooms.
I believe Apple developers have the Mojo to make ZFS use transparent for Mac customers. They certainly have the help of the Sun team and it is in the interest of both companies to make this work. Plus, don’t forget Apple’s “touchless” file system upgrade patent.
But MacJournals correctly points out that UFS was once thought – though without the level of support ZFS has enjoyed to date – to be the successor to HFS+ and that a similar fate may befall ZFS. While that is certainly a possibility – never say never around Steve Jobs – there are good business and marketing reasons for going forward with ZFS, regardless of what techies think. Apple will go forward with ZFS and make it the standard OS X file system within 2 years.
Comments welcome, as always.
Update: I’ve started editing comments on this post to keep them on topic and away from personalities. I regret not doing so sooner. Nonetheless the discussion is informative and if file systems interest you, well worth perusing.
Perhaps running off to SNW may have made you post a bit hastily. If “a couple of thousand dollars worth of music, TV, movies and videos go poof!” ZFS won’t do anything to stop it that conventional RAID won’t, since ZFS’s sole advantage in this area is in the detection and correction of otherwise-undetected and typically single-bit errors that won’t bother media data (which is specifically designed to tolerate such minor glitches) in any noticeable manner.
I’m beginning to suspect that you view ZFS primarily through a marketeer’s eyes, as something which can be sold (and in your case, over-sold) as “New, Improved, Shiny!”. If so, that helps explain why some more technical types (at least those not blinded by ZFS’s ‘coolness’ and open-sorcery) see things somewhat differently. So I’ll point to (rather than recapitulate here) the discussion we had over at your ZDNet blog (http://blogs.zdnet.com/storage/?p=202#comments).
– bill
I am constantly amazed that people underestimate ZFS. As a video producer finally switching to HD, how I long for a time when I can just add drives and scale up my file server without worry. There never will be a filesystem that replaces back-ups and I never understand why people think it should. The way I see it is that I have suffered through FAT, NTFS, XFS, and now HFS. I am sure that ZFS cannot be any worse and when it comes to technology, sometimes not being worse is the best you get.
Bill,
I am a marketing guy. Good marketing and a mediocre product beats poor marketing and a great product every day of the week.
Further, I believe that our existing storage paradigm is deeply broken. So architectural features such as combining the functions of a volume manager with a file system, the block pool and RAID Z’s performance benefits all appeal to me.
I also like competition, since that is the only way to get Microsoft to stop talking about innovation and actually doing something. ZFS is competition for NTFS and that is a Good Thing.
As I’ve said earlier, Steve Jobs is never going to get up in front of the faithful at Macworld and announce ZFS. It is plumbing and will never be sold or oversold by Apple. However, bringing the cost of high-performance and reliable storage down to commodity levels is another Good Thing. That is another worthy goal that I believe ZFS contributes to.
Is ZFS a perfect answer to all storage and data corruption problems? No. Will it require people to do some things differently to get its advantages? Yes. Is it something most people care about? No.
But it IS the first 21st century file system with an opportunity to ship millions of copies in a very short time and shake up the industry. That is another Good Thing. And I’m all for it.
Robin
It’s about being lucky or rather ‘unlucky’. If that single-bit error happens to hit your meta-data and not your picture or mp3, then depending on how unlucky you are you could loose just one file, entire directory, etc. ZFS will protect you in that case even without using RAID as meta-data are always kept in 2 or 3 copies (depending on type) kept separated on a disk. So you can lose your files and not only get ‘glitches’.
IMO there’s nothing necessarily wrong with being a ‘marketing guy’: even while we techies may sometimes abhor their behavior, we still know that we need them. However, it’s important to make that marketing viewpoint explicit, because a great deal of the time here (and at ZDNet) you’re talking about technical features as if they were intrinsically important rather than as being useful marketing tools.
And you do often (I hope not deliberately) present talking points that have nothing to do with the subject at hand. Just above, for example, “architectural features such as combining the functions of a volume manager with a file system, the block pool and RAID Z’s performance benefits” just aren’t applicable to a typical Mac single-disk or dual-mirrored-disk environment (which one might suspect covers most of Mac territory: while a lot of people probably use external Firewire or USB drives to supplement their storage, it would be silly to integrate them into a ZFS pool if you ever might want to move them around).
Furthermore, you seem impervious to criticism but rather just keep parroting your party line (yes, this is also a marketing trait, but not an admirable one): RAID-Z is brain-damaged, as I’ve noted before and then explained at considerably greater length in our recent ZDNet debate. At best (large reads and writes) it provides performance similar to RAID-5; at worst (small-to-medium-sized reads), performance only 1/(N-1) that of RAID-5 (where N is the array size).
And ZFS in other respects (RAID-Z limitations ignored) is AFAICT slower than ext2fs (and I suspect than some UFS variants, including those using ‘soft updates’) and at best no faster than ext3fs (mature now, with ext4fs just around the corner), XFS (a mature product which is probably faster handling the very large files for which it was designed by SGI), JFS (an equally mature product from IBM), and ReiserFS (well, we’ll have to wait and see where that goes now). So there’s no vacuum of existing, already-established ‘high performance’ NTFS competition for ZFS to step in heroically and fill.
I also covered your over-selling of the importance of ZFS’s edge in reliability (combined with your misinterpretation of CERN’s error analyses) in detail in the ZFNet discussion, so no need to reiterate it here. But (again) you appear to have been either incapable of understanding it or unwilling to assimilate it (given that your blog post still used the scary ‘poof’ scenario to assert that ZFS was ‘required’ for home media centers – and I notice that you adroitly avoided addressing my challenge to this example above…).
In sum, if you want to run a marketing blog, by all means do so. But please present it as such so we techies won’t take it too seriously (even if we still find it a convenient place to congregate).
– bill
Hi, Robert-of-the-Sun-ZFS-discussion-forum.
I think it’s more reasonable to characterize the question as whether you’re so unlucky that your metadata (and the rest of your system, and your house…) is likely to damaged by a meteor-strike (which is probably about as likely as it being hit by a 10^-14 incidence uncorrectable bit error): while the incidence of such errors on an entire disk may not be completely negligible, on a media server the metadata likely comprises something under one part in 10^5 even for low-bandwidth audio (more like one part in 10^8 – 10^11 for video, depending upon length and quality).
So if you’re asking whether I ‘feel lucky’ with those odds, the answer is that indeed I do – especially given that any user who doesn’t also back up the entire data set on such a media server (in which case it could all be recovered if such a low-probability event did occur) would be insane not to replicate it if it were valuable (in which case you drop from the uncorrectable error rate to the undetected error rate before encountering something that ZFS might handle but RAID – at least with scrubbing – would not).
– bill
Rats – I just did what I suspected Robin of doing: posted in haste before running out the door (in my case to get a pizza, which some people might not consider as significant as heading off to SNW, though others might not).
But I’m back now and feeling much better, thank you, so I’ll observe that while my low-quality audio metadata percentage above was about right (not that I expect many people to be filling up their 1 TB drive with 250,000 low-quality audio tracks) I for some reason grabbed uncompressed video sizes (which are fairly unlikely to be present on a home video server) out of my head and therefore the video metadata on such a server is more likely in the one part in 10^7 – 10^9 range.
Still, I would indeed feel comfortable living without ZFS even though one server in millions or tens of millions might thereby experience metadata loss – especially given how inexpensive simply replicating the drive is these days (which I’d want to do anyway for other far-higher-probability reasons, such as whole-drive failure).
– bill
Robert,
ZFS ability to detect & correct disk bit errors is fine, but not a great technical feat.
I am more concerned with High Availability features.
What is the point of all that software when the existing ‘Thumper” platform hardware (with 40 plus disks) runs off a single motherboard… i.e. a single point failure. Considering the number of disks, replication is an expensive way out.
Perhaps Robert could comment on this.
Richard – I can comment if you wan 🙂
Well, thumpers are great but not for everything – now I work in a site where it actually does make a sense to replicate data from one thumper to a another both in terms of performance and price. Why? Imaging millions of files and small percentage of them are beig changed. Traditional backup solutions have to scan all those files and it just takes like 11hours. But if you use built-in zfs option to send incrementals between snapshots you are down to minutes in the same environment. Not to mention that if you want to restore everything you just switch all traffic to second thumper. So both your backup and your restore is much faster.
Now if you want a cluster – sure you can do it – Sun Cluster 3.2 (which is for free) does support ZFS – I’ve been running ZFS under SC3.2 for quite some time now. I’ve just setup another two such clusters.
Of course there’s still valid question what are you protecting from and is it worth it in your environment. Most of the time you’ll run into disks failures and not mother board failure. I’ve been using thumpers since almost they hit the market and so far only disk failures occured. However in other enviromens where we don’t want to take chances we’ve actually gone with Sun Cluster solution.
Still, want a clustered solution and make it cheap? Grab 2x smal sparc or x86 servers, connect 2x externall jbods, put sun cluster and zfs -and if you don’t need support everythig except HW is for free. I’ve been doing something like this too. Any problems? Well, some silent data corruption on one of a scsi adapters – thanks to zfs it wasn’t an issue – was detected and corrected on the fly.
Bill – while on your desktop the probability is quite low I guess (although check on google for people reporting silent data corruption because of failing power supply and how they survived because of zfs). I’ve been used to manage quite a lot of data and for a last couple of years on zfs too. You would be surprised how many times we had to run fsck, get some data corrupted… and at the end it was a bug in firmware in Sun’s storage, IBM’s storage, EMC’s storage – each time with sata drives (interesting pattern here). Once we switched to zfs all we’ve started to observer were zfs checksum errors being reported and no more fsck, corrupted data, etc.
The other time we run in a bad scsi card in a cluster – it was corrupting all passing data on it with just a notice message in logs about firmware mismatch (but it pretended to be working) – again, thanks to zfs no harm was done. Strangely I run in a very similar problem again with exactly the same type of card lately…
Then I’ve been using zfs on my desktop in a office for quite some time – so far only one checksum error (corrected by zfs) detected.
Now how many times people lost some file on a desktop, or they can’t open it, etc…. in most cases I guess it’s a OS or application fault however I suspect from time to time we can actually blame disks.
On the other hand I’ve got yet to see any checksum detected on SAS/FC disks… no single one so far.
So it looks like zfs built-in end-to-end checksumming enables companies to use so called “cheap” storage safely. First it was google with their home grown google-fs which btw also does have built-in checksumming implement in a very similar way to zfs 🙂 Why? Check their published paper on googlefs. Ok, because on cheap disks you get your data corrupted.
You say there’s a backup – the problem is – what’s in your backup? Already corrupted data or good one?
Now zfs is not about end-to-end checkumming only.
Snapshots/clones – it’s a wonderful feature not only in enterprise but also on a desktop. Especially the way they are implemented in zfs – as they are really cheap in terms of resources (and free in terms of money).
Pooled storage with free file system creation, file system growing/shrinking on the fly, very fast RAID resynchronization after disk failure (only really used blocks), …
While zfs is not perfect (no file system is) it’s one of the best tools available on the market right now – and it’s for free.
I guess it’s like with Ferrari – before you’ve actually driven one you just hated or loved it, but after you only love it 🙂 (I’ve never driven Ferrari… :()
Richard:
ZFS’s ability to detect otherwise undetected errors via end-to-end validation is actually a technical feat of some note, given how many other systems can’t and given how frequently high-end environments like EMC’s Symmetrix use similar mechanisms but only within their block-storage boxes, rather than end-to-end from host RAM to disk and back again. Since WAFL has been doing something very similar for many years it doesn’t qualify as great innovation, and since their underlying mechanism that facilitates the required in-parent checksums is essentially ’60s shadow-paging technology on steroids that isn’t tremendously innovative either (nor a very efficient use of disk bandwidth, given the number of pages that must be written back on every update: WAFL can do somewhat better here, since it can accumulate updates in its NVRAM and IIUC must update only block pointers rather than the full path to the root when it actually does write data back to disk), and since the incidence of additional errors detected is very low it’s not all that important save in very controlled and critical environments where they aren’t swamped by other kinds of errors – but it’s still worthy of respect.
Robert:
Please remember that the subject under discussion here involves desktop storage, which is rather unlikely to experience any of the firmware problems you encountered in Sun, IBM, and EMC storage because virtually no desktops use it. Desktop ATA/SATA host-controller hardware is dirt-simple and very standardized – desktops aren’t even likely to be using SCSI. In fact, your own desktop experience (though anecdotal) tends to support this – as do the only quantitative studies that I’m aware of.
I’ve always acknowledged that *in controlled server environments* (such as your Google example) ZFS’s added integrity can become significant – though I don’t necessarily agree that it’s notably more significant for ‘cheap’ SATA disks than for enterprise disks, and (again) the two recent studies discussed here – one by Google itself – tend to support this (contrary to your own anecdotal experience).
For a media server, “what’s in your backup” is presumably the media files soon after they were added: in such an environment there’s no reason to back them up again and again, because they never change: you just add the new ones to the existing backup disk. Furthermore, by using such an incremental, file-based backup there’s no possibility that a disastrous metadata error on the original system can propagate to the backup copy, so the worst that can happen in the unlikely event that such a disastrous metadata error occurs before your system is hit by a meteor is that the media files added since your last incremental backup might become inaccessible. Again, remember the specific subject under discussion here.
Another ZFS feature that I’ve always acknowledged as worthwhile *in server environments* is its ease of multi-disk management (which, once again, is pretty irrelevant for most desktop environments, where if multiple fixed disk resources are present at all they’re usually in the form of simple disk-to-disk mirroring where ZFS has little incremental value to offer).
Snapshots are potentially more useful on the desktop, but ZFS is hardly the only desktop file system to offer them: FreeBSD with soft updates does (the fssnap facility seems to do so for other FreeBSD file systems and for pre-ZFS Solaris file systems as well), VxFS does, LVM-level snapshotting appeared in the Linux kernel in 2.6.8, and even Windows Vista (gag) supports an equivalent facility via its ‘previous versions’ feature, for example (the ‘Volume Snapshot Service’ which appeared in XP may be more limited to server environments).
I’m afraid that your enthusiasm for ZFS has made you less than objective. ZFS is no Ferrari, not even a Porsche/Mercedes/BMW (though it might eventually mature to that level of polish and performance): it may be beyond the basic Ford/Chevy/Chrysler arena, but is currently at best comparable to the mid-range Japanese offerings (which are quite respectable, of course, but so are quite a few of ZFS’s competitors – *especially* for the desktop).
– bill
Bill:
Google study wasn’t about silent data corruption on FC/SCSI disks IIRC it was about disk failure rates. If you want other users experience with silent data corruption on desktop servers – look at google, you’ll find them. My experience is quite good – I admit. Just one corruption in 2 years or so. Nevertheless no harm was done thanks to ZFS.
Backup – well most shops do actually full backups once in a while. I know TSM approach is to always do incrementals but that’s just one solution. And even if you backed up files soon after they were written the problem is they could already be corrupted… like in a case with failing SCSI card I encourted two times so far.
End-to-end checksumming for me is a must these days – and what you’ve got to pay for it in terms of CPU/disk IOPS is irrelevant in almost all cases in practice.
Snapshots on desktop – can you easily create sub filesystems in other solution so you can setup different snapshot policies for different “directories”? For example I don’t want a temp files or download directory to be snapshotted too at all, however I want my email to be snapshotted at least once a day (it is) the same goes for other data. With pooled storage approach it’s just so easy to achieve it and you don’t have to worry how you partition your disk or what size of your file system should be, etc.
Then what is a performance impact of using snapshots in other file systems? In ZFS it’s basically 0. Can you create a clone (writable snapshots)? What’s the disk space and performance impact? How easy is it to use? Do I have to pay for it?
Can you enable compression just for your emails or My Documents or whatever? I can and I do for my email – light lzjb so I don’t notice any CPU overhead and get 2:1 ratio.
I’m afraid your dislike for ZFS made you less than objective.
From my perspective I’ve been using VxVM/VxFS, LVM, UFS, SVM, ext2, ext3, xfs, …
And my experience (both desktop and server) with ZFS comparing to all the other solutions made me that enthusiastic about ZFS – it’s been making my life easier and solved lot of problems.
Apple has been clever picking up ZFS as its next generation file system. That some people don’t get it – well, it’s just the wayit is.
“I am a marketing guy. Good marketing and a mediocre product beats poor marketing and a great product every day of the week. ”
I’m trying to not take this out of context, but do you honestly mean that? Great marketing for something with lacking technical merit is better for who exactly???
Lots to respond to here, which I intend to do. But I wanted to respond to a.t. first:
-No, I think the fact that technically superior products – and let’s be real, not all technical “superiority” is something people will pay for – lose to inferior is NOT a good thing. But it is a reality.
Which is why it drives me nuts to see companies with good products assuming they’ll win because of their product. No, chances are good you will lose, unless you get good marketing.
Robin
You should be more careful about making assumptions about people you don’t know squat about, Robert. I don’t ‘dislike’ ZFS at all, in fact I’m quite enthusiastic about it (as I put it in Robin’s ZDNet blog, it’s a welcome breath of fresh air in a technical area that tends to be unduly stagnant) – I merely dislike unjustified hype, even associated with products that I think are good ones. But since your contributions here suggest that objectivity may not be one of your own strengths, it’s not surprising that you seem to have difficulty recognizing it in others.
You’re correct that the Google study neither addressed the incidence of ‘silent errors’ nor compared (S)ATA failure rates with FC/SCSI. The CMU study at the same FAST did the latter and found little difference; I don’t know of any studies that have explicitly targeted ‘silent errors’, but in the absence of evidence to the contrary (by all means present some if it’s not purely anecdotal) suspect that such errors likely follow the same pattern as other failure modes.
As for backup, you continue to confuse corporate environments (‘most shops’) with the subject under discussion here (home media servers). While the former have reason to perform full backups once in a while because their data *changes* and thus would require application of all incremental backups since the last full backup to reconstitute it, the latter do not (because data is *additive* and hence there’s no reason to do more than add it to the existing backup copy – which then becomes the current full-backup copy). Had you paid closer attention to the detail that I provided previously you would have understood that.
Furthermore (still with respect to backup) you’ve wandered away from the subject yet again: your early response was not about silent-error *file* corruption (which media files tolerate just fine for the typical case of single- or few-bit errors) but about the supposed hazard of catastrophic *metadata* (structural) corruption in the file system that would lose you the whole thing (Robin’s original doomsday scenario). If you’ll revisit my response with that in mind you will see that there’s no possibility of propagating such errors (in the unlikely event, I will remind you, that they occur before a meteor destroys your system) to your backup copy (and if every once in a while a bit in a backed-up media file is flipped, it will be profoundly irrelevant).
As for the personal value of end-to-end checksumming on your own desktop (I do hope that’s what you talking about), knock yourself out – just don’t expect the rest of the world (at least that portion which has any grasp of probability) to share your paranoia.
Since my observation that snapshot support was hardly unique to ZFS you seem to be trying to change the subject to using them with multiple file systems. If you create multiple file systems on your desktop so that you can snapshot them differently, that already puts you in a completely negligible minority of desktop users: when discussing the market relevance of that particular feature (which, I remind you, is the subject here) rather than just plain old whole-system snapshots, that minuscule percentage just doesn’t count. As for performance, I think that all the examples I presented perform copy-on-write of one form or another, rather than the antedeluvian mechanisms of the early ’90s where major metadata copying occurred when the snapshot was created, so performance is adequate, space use is reasonable, and at least for the open-source products I mentioned they’re free of charge as well. Writable snapshots for typical desktop use? You really do appear to be getting desperate here. Ditto for compression: with today’s processors compression (even good old Windows-style compression) can actually be a performance win, but disk space is so cheap that virtually no one bothers (just how much email are you talking about, anyway?).
That about covers it. I’m happy to leave it to anyone who’s sufficiently interested still to be reading this to assess which of us is being more objective (not that I care all that much: I tend to discuss things with people I disagree with more to try to elicit information that I may not be aware of, but in this case the effort is not proving very worthwhile).
– bill
I’ve spent the last few hours reading through this, the ZDNet blog comments and background information, and I want to commend Robin on the patience that he’s shown in dealing with what I — an ordinary computer user — regard as a sally of simply obnoxious, appallingly patronizing abuse. And the essential point of contention is really the question of whether “ordinary†desktop users do or do not have systems with multiple drives that could make use of the combined pool facility in ZFS, and other features. While I can’t speak about the Windows world, which of course dominates, Powermacs have for a while come with four SATA drive bays that are simple enough for the home user to use easily. So multiple drive desktop systems are a part of the Mac power users’ world right now, and have been for some time. The point is that the demand here can only grow — and will grow dramatically in the next two years as computers become the storage medium for home entertainment. So even if many Mac users, particularly iMac users, are currently restricted to one internal drive, Powermac users can have 4 Terabytes of disk space right now. And where there is manageable disk space there will be content that just begs to fill it. From that point of view ZFS is, as Jobs once said, “insanely greatâ€.
To say that that such things are just not important to average users gets average users completely wrong: even average users will tomorrow be making demands on their file systems that today look geeky. Robin understands that, Apple understands it, but somehow it is beyond the dirt-under-the-fingernail-tech guys here, who follow this issue around the net insisting that they, and only they, are right.
Not only is ZFS not being over-hyped here, in my view it is being undersold. It is needed on the desktop now, not in two years time. It’s failure to appear in Leopard is a significant disappointment.
1. My 9-year-old K6-2 box (the oldest Windows box that I still have kicking around without digging for one) has 4 5.25″ external drive bays, 1 3.5″ external drive bay, and 4 3.5″ internal drive bays, and each of the 4 Windows boxes we’ve purchased since then have at least as many (in total, anyway – a couple have only 3 external 5.25″ bays but more internal 3.5″ bays). So the Windows environment has had at least as much multi-disk potential as the Mac environment has had for a long time.
Guess how many of these machines came with more than a single hard drive standard? Guess how many typical users go to the trouble of ordering multi-drive systems, let alone adding an internal (i.e., fixed) drive later themselves? Guess how difficult it would be even to *find* a multi-drive system if you walked into a Best Buy or similar establishment (as most purchasers do) rather than configured one deliberately on line (as most on-line purchasers don’t)?
2. Your claim that computers are just about to become the focus of home entertainment might be a bit more credible if people hadn’t been trumpeting it for nearly a decade now. In fact, special-purpose devices have started hitting the market precisely because computers have *not* made good on this promise, and show no signs of doing so (for whatever reasons: some people just don’t want a computer in their living room, others had enough problems ‘programming’ their VCRs that the idea of an even more complex environment both in terms of wiring and of control appalls them, and in general a small, quiet box with a few simple buttons and plugs on it seems far more acceptable).
3. Pertinent to both of the preceding points is that laptop sales volumes are rapidly starting to dominate conventional desktop sales: just how prevalent do you expect multi-disk (especially beyond simple mirroring) laptop configurations to be, and just how popular do you expect laptops to be as the focus of home entertainment centers?
4. But it’s a lot less unrealistic to assume that *some* kind of storage-intense home entertainment usage will occur (just not much of it involving general-purpose desktops): what special advantages would ZFS bring to it? Closed ‘appliances’ would simply use matched drives if they wanted to support mirroring (leaving ZFS no value to add – people really aren’t going to want to have to ‘manage’ this storage, e.g., by mirroring material selectively) and use external USB drives for supplementary storage (no way for ZFS to add value there, either), especially for those people who might like to take their media files on the road once in a while.
5. Now, as a techie I can certainly see the value in having some kind of central home storage server that satisfies the needs of the home’s computers *and* its entertainment center, but I’m not sure that even that is likely to occur – even if wireless networking eliminates would otherwise be a wiring mess. Multi-use storage means that people have, at least to some degree, to understand how its multiple uses interact, and experience suggests that most people would be happy to pay something of a premium not to have to (i.e., to use separate appliances). If we get to the point where storage really does become a utility over the Internet and people don’t have to deal with it at all, then ‘home’ storage will become remote large-scale server storage and ZFS will start to become more interesting – but I don’t think that’s at all what’s been being discussed here.
Thinking about the evolution of audio equipment might provide some insight: my impression is that modular components peaked during the ’70s, after which most people save for audiophiles reverted to multiple stand-alone special-purpose boxes, even though they arguably provided inferior quality at a higher total cost. But I’m hardly well-acquainted with that market, so my impression could be completely wrong there.
Robin’s a big boy, Adrian: I expect (or at least hope) that he knows what he’s asking for by voicing his opinions as assertively as he does, which of course is why I’m similarly assertive when I disagree with him because – as I noted at his ZDNet blog – it sometimes takes liberal use of a heavy, blunt instrument to get his attention if he’s on a tear. The main faults I find with him are technical, since his depth in that area, while usually pretty respectable, sometimes just doesn’t match (or adequately support) his enthusiasm.
By contrast, your claims above seem to fall far more into the realm of uninformed speculation (you know what they say about opinions…) and I’ve responded in kind.
– bill
Bill:
I’m making no more assumptions about you than you making about me 🙂 Anyway it’s not that important.
Multiple file systems – yep, you’re right – most desktop users don’t use them or use just two (quite a common thing on windows is to have c: for system and another d: for data). The probel is that creating those file systems couses too many problems for desktop users like: I do have a space in one file system while I run out of space in another… so one big file system is easier to cope with.
The think you didn’t get is that if Apple would create one zfs pool for system disk then several separate file systems (for temporary files, for intermediate downloads, for documents, etc.) then user wouldn’t even know about it as he could still see all available space no matter where he wants it. But then Apple can enforce different snapshoting policy for those different filesystems, so your documents or media files and system itself is snapshotted quite frequently while temporary files are not snapshotted at all. It would be totally behinde the scenes from the user perspective. Then if user wants to create a “directory/folder” with different snapshot interval what would happen in practice is that new zfs file system would be created with different snapshotting policy. It’s that simple.
If you think about the old way of doing things – you’re right, too much hassle. With zfs it’s just natural. So having multiple file systems is actually very useful on desktop and user wouldn’t even know that he is actually using multiple file systems.
Performance of snapshots – yep, most of solutions in a marted are using CoW but in a different way – so if you’re going to change some files the performance would be actually impacted as you’ve got to wait for CoW which is generally not the case with ZFS.
Robert –
If Apple created a default multi-file-system configuration for the specific purpose of using different snapshotting policies that would indeed take the burden off the user. But doesn’t Leopard already have a somewhat similar mechanism (I forget the name, but it supposedly allows one to roll data back to fairly arbitrary points) in HFS+ – and does it suffer from inability to tune it such that it doesn’t snapshot things like temporary files?
While I use separate partitions for Windows and data myself, disk sizes these days are such that I just don’t have problems juggling space between them (though if I needed to Partition Magic or one of the free Linux/BSD utilities would make it trivial to do). So I still consider ZFS’s flexibility in this are to be more significant for corporate environments (where storage space must be juggled among multiple competing users and departments) than for the desktop.
– bill
Bill, you need to get to Fry’s and see what’s going on — with the low cost of external USB, Firewire, eSATA and (though not quite so cheap) Ethernet hard drives, I think multiple drives are already much more common than you think, and will become quite common – only a lot will be external (especially for laptop users).
Tony, you need to read more carefully before responding: I already acknowledged the growing popularity of external drives while noting that their main reason for existence is to allow storage to be added in an easily-portable and ad hoc manner (or in the case of Ethernet storage possibly to be shared among multiple clients), not to be integrated into a system from which they then can’t be removed without breaking it.
– bill
You just create separate pool for external disks… tha what I’ve been doing for quite some time.
Bill,
That’s a matter of opinion – until someone does an accurate survey of what typical home users do with external storage (and keeps re-doing it annually).
I suspect with CPU’s being way faster than necessary for most people (heck, my two year old CPU is still more than fast enough) and external storage being cheap and easy to add, a lot of external storage will end up being sedentary. In fact, I wonder what percentage of laptops end up staying in one place – I suspect it’s significant.
Are there not some security advantages to ZFS?