Regardless, a home user would not want to use RAID 1/5/10/50/100 as a backup solution. RAID is not appropriate for backups. Why? RAID does you absolutely no good against theft, viruses, accidental deletions – the kind of stuff you would want backups for.

For instance, if a virus comes along and zero’s out all the pictures on your machine… or someone steals your computers… or you delete an important folder permanently… how does RAID help you at all?

One of the key points in the argument against RAID here was made by David J. Heinrich in regard to “drive failures tend to be correlated due to similar use-patterns”. For the uninitiated this basically means that the likelyhood of your RAID drives failing simultaneously is increased due to the nature of how RAID works … an ideal RAID requires all of your RAID drives to be of the same model and firmware revision to be most effective. Let’s go back to the tyre analagy provided by George Ou, yes you may have 4 tyres and a spare however the spare is also being used at the same time, the tread depth wears out at the same rate and you are then running on 5 bald tyres rather than just 1, all mechanical drives share this same weakness and coupled with the precision of mass manufacture the likelyhood of multiple drive failure is compounded. In essence the greatest failure with RAID is it’s synchronicity. This statement is also true of mirroring.

Another , less noted, variant here is the problems with RAID 5. RAID 5 and it’s derivitives have the beauty of redundancy and speed built in however, as anyone who has dealt with the loss of a disk in a RAID 5 configuration can vouch, the performance stinks when a drive fails and you have all the issues of the resync to boot. A loss of a second disk duing this vulnerable phase and bang …. all your data is lost.

RAID 5 and Striping (Striping cannot really be considered as true RAID as there is no Redundancy): Plus point – is very fast access – the more disks you use is just short of equal to the number of drives multiplied by the access to one drive. Minus point – increasing the size of storage requires you to destroy what you have and rebuild from scratch. Also the need for this kind of performance shouldn’t really apply to the majority of SOHO users.

Taking all of the above into account I would look at the alternative of snapshots to alternative local storage to maintain data integrity – this would be similar to RAID 1 where you use a second disk for redundancy however only one drive would be actively providing access to the filesystem/disk the other drive could then be brought online in the event of a major failure on the first and service would be restored resulting in minimal data loss and a fast service restore time. To further improve in performance what could also be done is to have a matching partition layout on both drives (for simplicity we’ll just have 2) then on the first disk the first partition could hold one set of data with an equal amount of space on the second disk’s first partition being used to back this up. Meanwhile on the second disk the second partition could be used to hold a second set of active data while the corresponding partition on the first disk could be used to hold the backup for this. The upshot being that you can have two sets of data being accessed at a similar rate as opposed to contesting the same drive for access – hence the user see a performance increase as well as having redundancy meaning that they get some additional bang for their buck.

It is 14months later now, one died this weekend (the disk does not seem to be spinning at all), another has a noisy fan (luckily it’s in the “server room” in the basement) and the one on my desk intermittently makes some really strange noises of late. To say I’m somewhat disappointed in a Maxtor product that was highly rated last year is an understatement.

I was just looking into about buying a RIAD box for here at home, or stripping down one of the lesser use machines and building a RAID out of that when I came across this blog. While both my wife and I are tech-savvy enough to manage and secure our network neither of us is a network admin/specialist. So I was looking for one of those “dummed down” RAIDs. I know that I have to have twice the space and am willing to accept that as it part of the price you pay to get RAID.

We had, as of last week, close t0 3TB of disk storage on the network and as is almost always the case, the disk that died this weekend is the “critical” one for right now. I’m headed off to a data recovery specialist later this morning to see what the prognosis of getting that data back are.

I’m not a huge fan of the “online” backup solution as it put’s my customer’s data on servers that I have no control over from a security or reliability point of view, and upload speeds on a cable modem are just not that great on my provider.

Over the next week or two I’m going to be doing some research (until reading some of these posts, and the ones on ZDnet, I was just going to buy/build a RAID box) to see what would work best for us. I would be interested to hear what you and your readers think.

But, I can find no other method for keeping tabs on 3TB of data that are nearly as cost effective or pose less headaches. To date I have lost data, but only from optical media failure. Short of a DLT drive I can’t think of any viable option.

Sure, online backup is great, if you have a few hundred gigs of data at most, once you’re in the TB range it’s completely pointless for both speed of access and cost.

My only future idea is to go from Raid 5 to multiple mirrored drives to spread the URE possibility much thinner.

a

I’m a “prosumers” user, so I’m a little out of step with normal users, but I understand them. I had HD reliability issues on my main HD for my laptop. So what I did is bought a new 2nd HD, put it in the optical bay, and moved all my data there. My main HD filled up, and became less and less reliable (running WinXP BS, as I needed it for business school). Eventually, WinXP wouldn’t boot. Great! Got a 120GB replacement drive, and am now operating from the Ubuntu Linux CD. Haven’t bothered installing yet (actually, did, but ran into a stupid error 97% way done about GRUB install to MBR failing, then it borked out; haven’t had time to mess around with it yet). BTW, Ubuntu running from a CD boots up faster and loads programs faster than Winblows XP did for me for the last couple of years (from the HD). That’s really sad. And worse yet, apparently Win Vista is a downgrade from XP in terms of performance.

In the meanwhile, my data is pretty safe on an that data-drive, which is now wrapped in the HD static-proof bag that my new HD came with. And I have a lot of my photos on DVDs as well.

A few other comments…in RAID, drive failures tend to be correlated due to similar use-patterns. This greatly diminishes the value of RAID for home-users. It just doesn’t seem worth it. Because RAID isn’t a backup. So then, to really be protected, you need 3 hard-drives: 1 original, 1 for RAID, and 1 for backup. You’ll have a tough enough time convincing people to get 1 extra HD, let alone 2.

Ideally, backups to Gold Archival DVDs should be done whenever you know anything is fixed (e.g., not a work in progress), and have enough to fill up a DVD. But, although I’m willing to spend $80 for a 100-pack of gold archival DVDs, the average consumer will wonder why spend that kind of $$$, when a 100-pack of normal DVDs is like $20 or whatever.

There are other considerations when backing up, as well. A few ideas of mine:

* OS and apps should never be on the same HD as data. At the very least, they should be on a different partition. If on a different HD, the OS and apps would ideally be on a faster smaller HD…maybe even a small SAS drive instead of SATA for better performance. Also, if you want to make use of solid sate drives (like mtron), you can get small drives for your virtual memory, for your temporary files, and even for your /var files (if you’re a linux person).

*Your backup HD should be separated from your main HDs. Being inside the CPU case means more heat, which is bad for HDs. Also, if possible, it should be powered down when not in use by the automatic backup system (e.g., have software do this). That way, it has even less wear and tear.

*Use an ultra-high performance HD for your OS and apps, like a small Raptor or Cheatah.
*If possible, use an even better HD, like the mtron ssd’s, for your swap file, temp directory, /var directory, and anything else that gets a lot of read/writes all the time.
*Use an affordable high-performance high-capacity drive for your data, like the Hitachi DeskStar 7k1000, or Samsung Spinpoint F1. You can get these affordably in the 500, 750, or 1000GB sizes (by affordably, I mean more than 3.8 GB/$).
*Use an enterprise-class rated HD for your data backups. Something that’s rated for continuous 24×7 usage (even though you’ll have it powered down when not in use). E.g., the Hitachi Ultrastar, or Seagate Barracuda ES.2 SAS drives. These also come with 5 year warranties. Use them in an external enclosure. Or use an eSATA HD.

Ideally, all this stuff I’ve talked about should be done automatically, although that seems near-impossible. But Apple is always great with this stuff, making it easy for consumers. And they could even make all that hard-ware selection easy by selling premium reliability/performance grade macs that come with multiple HD’s, and the sys setup to use them appropriately for every-day use vs. backup.

PS: I find the arrogance displayed by some here flabbergasting. People are chastising the photographer woman for not knowing the difference between RAM and hard-drive? Ok, sure, she’s just a few notches above great-grandpa in terms of computer knowledge. But I bet a lot of the geeks harping about how imperative it is that people understand their computers, don’t know how to change a tire on a car, or deal with routine house-maintenance, or know a god-damned thing about their car for that matter. Also, I bet that that photographer — if she’s a professional — knows a lot more about Photoshop than anyone on this forum. Maybe if she saw people here doing some silly thing in Photoshop, she’d comment on how ignorant we were!

Not to go to far down Microsofts Road, I would imagine there are other vendors with similar products waiting for Microsoft to bloody the waters with the release of their usual unfinished product. They will then jump in with something Linux based that’s cheaper and has things like server backup built in.

The current selection of consumer NAS boxes are not worth the time to setup IMHO. They provide some useful services, but fail to provide true “server” abilities.

I made the following post originally on another Blog in response to a CAS post. No one ever replied. Maybe its worthless but I believe the “thousands of pictures” is very to the point.

Neither RAID nor backup would have helped without a Strategy and some Manageware skills.

Posted in response to:
“Oh, and one more thing. Robert Pearson’s first comment above ["CAS (Content Addressed Storage) requires organizing the Storage by Content. This requires some, if not a whole lot of, advance knowledge of the Information being stored."] truly invites a reaction, as the exact opposite is true. Real CAS explicitly does **not** require organizing Storage by Content.”

I guess I am a little confused by all of this?
You have my respect as an acknowledged expert in CAS.
I don’t claim to be.
My goal is to build consensus and make CAS better and more viable.

I fully agree with your statement about the Information, once it is in the CAS. My statement was directed to these areas:
1) Determining Stored Information that is a candidate for CAS
2) Getting it in the CAS
3) Once the Information is in the CAS dealing with the same
issues “non-CAS” Storage has like “hot-spots”, bottlenecks, updates, migrations, replications. and synchronizations.

Is CAS exempt from these “Management” issues?

How long would it take to migrate 30 TB of Seismic Information to CAS?
The same as for “regular” Storage?
I can tell you how long a 30 TB “snapshot” takes to commit and that is not being hashed for Content.

Here is an example:
A real world example is when my Uncle died. His wife preceded him by a few
years. My Uncle loved to take pictures. He had thousands of pictures.
He had some made into movies. He loved to show these at the family reunions. One whole room was his for the slide shows and the home theatre. And he took more pictures all the time.
He knew all the people in the pictures and had stories to tell about the person, the place, and the picture.

I helped his children transfer all these pictures to CD. Not every recipient had DVD capability. But we really didn’t know what to do with them. They seemed very valuable to us somehow. Like a valuable piece of history.
They should have been made with a VCR, or transferred to a VCR, so there would be audio but that technology wasn’t available until much later.
Between all of us we could probably identify 10% of the people and places.
We physically archived the pictures, movies and master DVD, made the CDs and disbanded.
That video Information has become valueless.
The technology is there to view them at any time. To what purpose?
Nobody derives any value from viewing them.

We put them on the CAS.
Suddenly they became hugely popular as background for TV shows. They were constantly being accessed. The TV people wanted to edit them and add comments.
Now I’ve got many versions of the originals that have become “fixed” that need to go on the CAS. I am out of CAS.
Does CAS offer versioning software?
Does it work with popular versioning software?

Then I got a chance to buy hundreds of hours of old TV sitcom video. We put it on the CAS figuring it would only be read.
Wrong! Thousands of “stills” have been edited out and produced.
These need to be stored in a related fashion to the CAS originals.
Is the hash on a still from a video the same as the video hash?
There is all kinds of Information related to the original Content that is stored on the CAS in a Content related fashion.
I keep hearing database. It was in a database.

Plus they want to insert “human realistic” pellet people in place of the
original actors to avoid paying any royalties. So now I have
the original, modified originals and “human realistic” copies. They are all the same content but play at different levels of value. Maybe these are not good candidates for CAS?

What about the level of Information High Availability? Information Integrity?
Disaster Recovery? Business Continuance?
Worst of all, Findability?
Each Unit of Information stored on the CAS is subject to all these demands.

“Vendor A” said CAS was the wrong solution. What I needed was “5 nines (99999)” active Storage. And lots of it. He bought me lunch.

p.s. Most dual failures happen when you buy a drive from a manufacturer that sells drives with higher failure rates. Some of these defects can be fixed with drive firmware updates, but unless that is an automatic process most consumers or even business will get the ugly failure.

p.p.s. Raid 6 does not fix the problem of “oops! All my drives have decided to stop reading or writing data”.