We run high end transaction systems running millions of revenue transactions a day with millions of online accounts.

We’ve used EMC and Netapps, but recently investigated “open storage.” running on 10GBe iscsi. Our benchmarks show that open storage on 10Gbe iscsi is comparable to EMC/Netapps. It is also a fltat architecture without the bottleneck of the storage processor. With the modern mirroring technologies provided by VM OEMs and by Database vendors, we get HA from them anyway. We can upgrade our hard drives and storage cards very cheaply – just swap them out.

From a cost perspective, the lifecyle cost of open storage is 1/15th that of closed systems. Its also less cost to maintain because any good SA can build these boxes or support them – its just a server with a lot of drives.

We moved all our non-prod systems to open storage and all our new production systems are now on open storage. It has saved us a ton of money and buying storage is now a signature process, not a procurement process.

I do recommend going with established open storage vendors as they have accumulated expertise with these systems. They wills save you some time and if something is messed up, they will fix it – otherwise you are on your own.

These are the ones we have used.

http://www.emdstorage.com/

http://www.aberdeeninc.com/

Robin

Keep up the good work, thanks.

–
David Strom

Today’s storage requirements are way different. Most of the data is unstructured (office documents, images, music and video). 100TB is normal to start with. Some applications demand beyond PBs. There is no way enterprise storage array vendors can scale their technology upwards (scale-up). There are physical limitations to number of CPUs, memory or I/O ports a single chassis can take.

Answer is to scale at the software layer on top of redundant array of inexpensive comodity servers. Let software do the job. It is scalable to multi Petabytes.

2 weeks before we deployed GlusterFS on 8 server storage cluster of 142TB capacity and 384GB memory cache. It delivered 12Gigabytes/s read throughput and 8Gigabytes/s write throughput with iozone benchmarking. 1000 clients pounded on the storage cluster.

You hit the problem right on. That’s exactly how NetApp and Sun’s open storage is going, using commodity servers with SAS adapters connected to loads of cheap SATA drives through SAS Expanders.

I am in love with ZFS. Unfortunately, Sun’s Open Storage strategy is being killed by their non-transparent marketing and advertising, and most Sun people are in denial. If Sun allowed Fishworks software to be purchased separately and installed on any SunFire X86 or Sun Sparc Machines, the open storage movement would instantaneously capture a majority of the enterprise storage market. Instead, as I have said before, it is ruined by “closed” hardware design choices, and basically non-realistic list prices.(Even though nobody pays list prices, I would prefer to have the actual price listed. )

Case in point, the 7310 is listed on Sun’s website for $40K for a single head node, here, $75K for a HA pair, and 111K for a loaded HA pair
http://www.sun.com/storage/disk_systems/unified_storage/7310/,
People can actually get the 7310 from resellers for $12.5K each from CDW.
http://www.cdw.com/shop/products/default.aspx?EDC=1897910

That is nearly 70% off of list prices. And that’s still not good enough. For people with a brain, they know that the 7310 is just a SunFire X4140, selling on CDW for as low as $2.5K
http://www.cdw.com/shop/products/default.aspx?EDC=1546283

On the storage front, the J4400 JBOD is only $3.6K
http://www.cdw.com/shop/products/default.aspx?EDC=1554430
Yet, filled with 22x 1TB Hitachi UltraStars + a couple of stecs,it suddenly becomes $32K
http://www.cdw.com/shop/products/default.aspx?EDC=1690801
People do know how much the 1TB drives are worth roughly $200 each for the Hitachi UltraStars. 1TB Western Digital and Seagates SATA can be had for $90 if you look around.

What it does is that it forces people to adopt NexentaStor’s OpenStorage appliance software, even though Nexenta piggy backs off of ZFS and Solaris Engineering costs.

So sad. I feel bad for ZFS engineering team. It is absolutely the greatest engineering feat, ruined by moronic MBAs. If Sun would wise up and truly open up the Fishworks software. All they have to do is sell the Fishworks as a software upgrade($2000) to any Sun Server, it would take off very quickly.

While I agree with the tone and tenor of this article, if one is paying more than $2.00/GB(usable) for one’s enterprise tier 2 SATA storage, one is paying too much, at least in any reasonably large deployment.

Joe Kraska
San Diego CA
USA

One goal is universal coverage – “good enough” solutions that reach every byte. For those that are willing & able, there are higher cost options that provide much more complete performance and protection.

While it would be nirvana if every data set could afford to be on the best and fastest solution, the math just doesn’t work.