Robin

PS- Any EMC’ers want to weigh in? My guess is that dual-Symms, SRDF and interconnects would top out at about $50/GB. Last I heard, EMC does test their stuff.

http://openstoragepod.org

The article you posted doesn’t say the “solution will not work”. The author is a Sun Employee and was merely saying:

“No, it isn’t a system comparable to an X4540 … I see several problems, but i think it fits their need, so it’s an optimal design for them…the nice thing about custom-build is the fact, that you can build a system exactly for your needs. And the Backblaze system is a system reduced to the minimum.”

This is ideal for nearline storage and D2D backup, which is exactly the business Backblaze built it for. Their solution solves a key problem that no one in the industry has solved — namely this:

http://blog.backblaze.com/wp-content/uploads/2009/08/cost-of-a-petabyte-chart.jpg

I’m sorry, Joe, it was early in the morning. What I meant was that they are ALL less than $1/GB now.

Joe.

So … 16.000 pageviews later, i’ve read through several reactions to my article “Some perspective to this DIY storage server mentioned at Storagemojo”. I wrote that article after reading Robin Harris’ article about a DIY high density storage serve…

Hmmm, if I am reading this right, then you haven’t seen Peta-scale systems from any vendor less than $1/GB.

No offense intended, but this suggests to me that you are not talking to the right vendors. We have done numerous designs and quotes over the last 9 months for large scale PB+ systems coming in *well under* that target, with the parallel file system and management stack included.

Relative to vibration, while these drives are heavy, you still have to worry about higher frequency (non-bulk) oscillations, being transmitted through any support. Given 45 drives with a 120Hz signal (7200 RPM) arranged the way they are, it is left as an exercise for the student to calculate the frequency and normal mode power spectrum for this unit. We have seen units in racks situated near a “jumpy” section of a datacenter floor have higher failure rates than identical units situated elsewhere. Moving the rack or the device to a less “bouncy” section did eliminate the issues. Vibrations/impulses can/do impact raids. There is a famous Sun video where the engineer shouted at his drives, and caused some issues. You have to worry about that normal mode power spectrum. Happily, it should be easy to work around by doing some material alterations, and other things that are relatively low cost changes.

The design is interesting of these boxes. We had thought of something similar for our DeltaV line, but elected to go with a more standard approach, and made some radically different design decisions on the internal guts. They are minimizing cost of build above all other design decisions. We minimized cost of build while making sure if a bus hits our company (so to speak) our customers can still service and support the machines they bought from us, and that they will still perform well even in the face of low cost design decisions.

Well. It’s a doit-it-yourselfer. One could simply use a more basic SAS card, or set the controller to JBOD.

But as for the never swapping drives in a RAID set notion. Could one leave it unserviced? One could. If one is willing to lose an entire RAID set and leave it alone forever if lost, then one could go until one wanted to FRU or recap the whole unit. I.e., if you want and need a virtual LUN the size of 15 hard drives, one can make the decision to also have the 15 hard drive size LUN be a unit of failure. Personally, I would only be motivated to do this if I had a specific single stream speed requirement. I.e., if your performance requirement drives you to RAID, you RAID.

Various “cloud storage” technologies can all accomodate what I just said. One just needs to learn to think creatively about the storage problem. To wit: there is nothing about a RAID set that intrinsically requires service, once one has the appropriate replication glueware in place.

Joe.

Two points, first, they aren’t jbod’s. They specifically state they are doing raid-6 with LVM.

Second, they have 3 raid-6 raidsets with 15 disks each. They have 45 total 1.5 TB drives. IE: No hot spares.

There’s absolutely NO WAY they’re going entire quarters or more without swapping drives in that setup.

Some points.
1. Our pricing is public list pricing. Not a mistake. A starting point. Open. We can and do, of course, scale our pricing
based on scale. I could like most show you a private price list under NDA that is considerably higher
and then “dramatically” reduce the pricing based on a volume play.
2. The appliance product is priced for hardware and software. There is of course more
that MatrixStore has to offer than the above list but that is not for here. My first comment on this post was a genuine request from a client. I am happy
to go offline for the full sales pitch.

Like I said, the Backblaze approach is good in the right hands, for a team that has the skills
to build/support Pb scale. Simply not for everyone.

Ping me offline it you like. Be great to chat.
N

Thats interesting, Nick. It’s probably a mistake to simply post pricing without naming a volume point, as at the petascale there are in fact competitors that will grossly underprice you. I cannot tell you who they are, as every petascale negotiation I have every participated in has been NDA’d. However, I can tell you with certainty that I have to even once, in the last year, price out systems for less than $1/GB usable at the petascale from any vendor, and the pricing pressure has definitely been downwards since.

Auditing and regulatory compliance are, of course, important niche features. But specifically on that subject, we know of at least one vendor doing what you are doing (and more than what you have named above) pricing their SW at $0.25/GB when the procurements are on the order of 1PB/2 mos.

Do keep in mind that people looking at systems like the one named in this article are likely interested in unprecedented scales of deployment, generally also associated with unprecendented cost sensitivity.

Joe Kraska
San Diego CA
USA

It was in deed a quick tour that or our website needs to better reflect what you are getting for your money. The $1000 is for the Mac OS version which is for organisations who wish to re-use the Xserve RAIDs they are rolling out the from their SANs.

The $1000 per TB includes:

- 2 copies of data (RAID5 mirrored)
- Object based storage providing single namespace (not CAS) – Less management
- scalable in seconds (less Management)
- searchable (integrated metatdata support, metadata living in the archive with the content)
- replication
- self healing, self-managing (less Management)
- fully audited
- regulation compliance
- multiple-tenancy support
- migration in place to new platforms (less Management)

More than just disk.

We have a Linux based appliance also which has better economics. We are shortly to release our nodes using 2TB disks giving pricing from: From $729/Terabyte (single Copy) to $1460/TB (dual copy). that includes all hardware and the software. All this on non-proprietary platform, OS, Filesystem or File formats. Your data WILL outlive your vendor.

Please do let me know of any other solutions with that feature set for the same price as we are always keen to learn more about any competition.

Cloud argument is for another day.

Thanks

Nick