Comments on: Why do storage systems fail?

By: You get what you pay for « Storage Effect

You get what you pay for « Storage Effect — Fri, 07 Mar 2008 19:15:38 +0000

[...] One failure in a million hours? It’s claims like these that seem extreme to some people when they look at enterprise disk drives. Yet a study of 39,000 NetApp systems by a researcher have found that these drives fail at a 1% annual failure rate (AFR). Robin Harris summarizes the study in his blog. [...]

By: Richard

Richard — Sat, 01 Mar 2008 09:48:46 +0000

Robin,
Yes… cabling plus connectors plus poor airflow plus power …. this is why the ‘commodity’ motherboard hardware apporoach does not work well.
NetApp controllers seem to be built on standard motherboard technology… right?

By: pmwut5

pmwut5 — Wed, 27 Feb 2008 10:12:53 +0000

I still find it interesting that a Netapp report into storage failure rates make no mention on out dated drive firmware as a cause of storage subsystem failure. Some disk drives being used have N14 releases of firmware - mostly to handle disk error handling which prevents the drive being marked as failed. If I was going to produce a report into 1.8 million disks the first thing I would look at is the firmware on those drives before making any conclusion

By: Pete Steege

Pete Steege — Tue, 26 Feb 2008 16:35:39 +0000

It’s probably not that NetApp systems are better, but that their application environment is different than other systems on average. I have no facts to support this, but my sense is the NetApp sweet spot is very high capacity apps that tend to stress individual drives less than other apps.

My observation is that the higher the capacity in a system, the lower the IOPS per Gigabyte (or drive).

By: Pete Steege

Pete Steege — Tue, 26 Feb 2008 16:32:03 +0000

Great perspective on disk failure. The fact that enterprise drives actually achieve their reliability numbers will surprise some people, given that they are so extreme (1.6 million hours between failures, for example).

Goes to show you get what you pay for.

By: Mad Morf

Mad Morf — Tue, 26 Feb 2008 12:56:13 +0000

pmwut5…
You’re mixing apples and oranges here.
Drive firmware is different from shelf module firmware.
Keeping drive and shelf firmware updated is the customer’s responsibility, as it is with any vendor.
LRCs provide no protection from LIP storms, so replace them if you don’t like the results you are getting. ESH2s and ESH4s do provide a lot of protection from LIP events, but no solution is perfect!
Dual connecting shelves requires no downtime if you have sufficient HBA ports available in the appliance.

By: pmwut5

pmwut5 — Tue, 26 Feb 2008 10:18:11 +0000

Cannot believe Netapp made no mention on out dated firmware running on their drives as a cause of storage failure. I have experienced several LIP storms from failed drives on LRC ESH2 and ESH4 modules storage interconnects on production Netapp Arrays. Netapp Solution is to dual connect all storage trays - great if you have enough I/O slots and downtime available.

By: Tony

Tony — Mon, 25 Feb 2008 17:33:24 +0000

Yes, connectors really suck (as I figure out where and what type of connectors to use in a new design) - but no connectors (hard-wired) is even worse!

There’s a lot more science and technology in good connectors than most people realize.