We have chosen a hybrid approach to availability that includes the best RAID (6) technology for large disks - combined with off-platform replication. I would love to see someone come out with a replication standard that allows rep-sets between different vendors’ arrays, etc - but there’s no vendor impetus to do such a thing. Why enable interoperability when you can enforce lock-in? I know there are tools out there, but how cool would it be to SRDF to a NetApp and Snap-Mirror that to a Hitachi? True data mobility - but there’s another post and subsequent 200 replies.

Replication plus diligent and appropriate RAID technologies (regardless of who the vendor is) protect better than any single vendor’s claim that they do “this” better than the next guy. NetApp’s approach (IMO) is blatant honesty (most times) which lends itself well to their credibility, and is reflected by their substantial footprint in our environment. When your disk vendor says “you need amount of disks space to accomodate - and here’s the exact reasons why” - that goes miles. When they tell you that you’re wasting money on to their own detriment (from a sales perspective) - they’re a true partner.

~~ YMMV ~~

James3678, AFAIK the NetApp response is a StorageMojo exclusive. Val is welcome to post it elsewhere if he likes.

Robin

Thanks,

James

1) consumer and enterprise drives has a lot more differentiation than claimed. Do you believe that the motors, bearings, and actuators are the same in these drives? They are not. They are in fact considerably different in design and reliability. I agree that the media (platters) is the same, but even they are not tested in the QA phase before shipment in the same manner. More time and tougher standards are used for enterprise drives.

2) I agree that failure rates are higher than one would expect reading MTBF figures. It might be worth noting that MTBF values for consumer drives are usually reported with a duty-cycle of 20%, meaning the drives are not spun-up 80% of every day (saving motor and bearings as much wear and tear as possible). Enterprise drives are in fact normally reported with duty cycle at 100%, meaning they are spun up 24 hours per day. I am not saying that either live up to their MTBF claims, and find that Vendor A has a good run for a year or two, then produces a bunch of duds, Vendor B the same, etc. Trying to determine who is producing good 400GB drives or 750GB drives at any point in time is quite a challenge.

3) RAID-5 - Most of the article, and some of the comments, talk about how RAID-5 is not a good thing today. My experience is that most failures and lost data, or even loss of access to data, are due to other factors, not classic dual-drive failures in the same parity group. This is a very complex subject, but most customers I know rely on RAID-5 for very high availability, but use other technologies including real-time data replication, backup/recovery, and transaction replication, to allow for recovery in those unfortunate times where an array does fail. I do feel it much more important to mention that most failures that cause data loss or loss of access are not the result of drive failures, but the firmware in the arrays that is supposed to handle the first failure. It is easy to implement RAID-5 today, but some vendors have much more sophisticated implementations of RAID-5. Case in point - One array vendor I know well has an implementation where a drive that is having reached some threshold for recoverable errors, will begin a recovery, but will not use RAID to rebuild. It is much faster and safer to just copy that drive to a spare before it does have hard errors, then flag it as bad, and go back into normal operation. If another drive has the unfortunate luck to ever have a hard error during this recovery, no problem, the first drive that was being copied is still online, and the parity group can recover using parity from the other drives to recovery the second, hard failed drive. This is just one very isolated example of how systems vendors implement very sophisticated methods to make RAID-5 “robust”. Not all vendors do this to the same level, and you only need to ask experienced storage admin’s to know which vendors do this type of work well, and which are not so well respected.

4) Many data loss situations, and lost of access to data, rests squarely on our shoulders, the users of these technologies. Do you keep your storage farm up-to-date on each level of microcode / firmware that is released? Leading edge or bleading edge, another topic to discuss, but if you are more than 6 months back and are thinking “if it ain’t broke, don’t fix it” you are a fool. There are hundreds of data integrity bugs fixed in most major firmware releases from the storage vendors listed (usually once very 4 to 12 weeks), and to not apply this new firmware is what is malpractice in my view. Also, do you monitor the heat inside the data center, or in the specific area of the datacenter where your arrays are located? What is the temperature inside the arrays themselves, and how stable is the temperature. Storage arrays hate heat, and they hate temperature fluctuation. This can have a major impact on MTBF observed in your shop!!! And guess what, you are in control of both of these variables.

5) Of the vendors you list, only Seagate and Hitachi actually build disk drives. The rest (including NetApp) all just buy drives from disk manufactures to use in their storage arrays. They can spin and FUD all they want, and talk about how they influence the specs or designs by these vendors, but that is hogwash. I would venture to say that both Seagate and Hitachi (and Maxtor and Western Digital and Samsung, etc.) all know more about drive MTBF than any systems vendor. Yes, that is only part of the story, and I would also say that storage systems vendors (including NetApp) know more about array firmware than the drive manufacturers.

6) The comment that native support for SATA is better than FATA or other implementations is not backed up by anything more than chest-thumping. Just to make my point, what if vendor-X can produce a more cost effective and reliable storage solution by using FATA instead of SATA, because it requires less re-design and coding (which costs lots of money) than using SATA. Is that bad? I don’t think users should care at the end of the day if they are using SATA or chicken-feed to store the data, as long as it works, reliably, with performance, etc.

Anyway, an interesting thread, but I don’t agre that the chest-thumping is under control in this post, it does raise some good points and dialog. I would not rate NetApp as the vendor I have been most imprssed with in terms of data protection capability, reliability, or field experience, but do respect their products and ability in the market.

I do wish I had more time to really read this article closely, and post a more comlete reply.

In any event, in any situation where you *do* need a remote copy, then that gives you an additional copy of the data that will allow primary-site RAID-5 to recover from encountering a bad sector during a rebuild, or even a second disk failure (the remote copy could either be a disk-level mirror of the primary site’s disks or an unRAIDed logical copy of the data). Of course, you need a RAID mechanism that understands how to incorporate this additional copy, or at least how to ask for higher-level help to do so.

Val’s response was a very worthwhile read, but I think does contain one error: ZFS’s ‘checksum-in-parent’ approach should definitely detect and allow recovery from lost writes.

- bill

Dale, I’m not sure I get where you are going with this question. FWIW, reducing the number of drives in a RAID 5 set reduces the AFR, so that’s good. Not sure what the benefit of a second hot spare would be. The risk is losing a second drive before the rebuild completes, which a second hot spare doesn’t help. Maybe you have something else in mind though.

John, thanks for the link. I’ve talked to a number of people who say DrunkenData and StorageMojo are the two storage blogs they read regularly.

Norm, you are correct. There is no substitute for offsite backup.

Robin

Would RAID 5 be acceptable IF:
You never had more than 6 drives in a RAID-5 set and you had 2 hot spares dedicated to the 6 drive set?

I am not asking whether this makes financial sense, only if it would make one more comfortable with using RAID-5.

Cheers!