Is it really possible to build enterprise-class storage using cheap commodity products? Some folks at HP certainly think so. They call their design FAB, short for a Federated Array of Bricks. Ignatz, meet Krazy Kat. Of course, HP is only the world’s largest data storage company, so what would they know?

And what is a brick? A server with disks. Instead of $10/GB, perhaps $1/GB? I’d love to see Storage Markets ask when this will come to market. Since HP doesn’t actually build big storage arrays – they OEM from Hitachi – they could do this. If they really want to take the data center away from IBM they have to do this.

Let me count the ways. . . .
Google has *almost* created enterprise-class storage from commodities. Microsoft has Boxwood. Amazon has, but they aren’t telling. Now HP has people saying “we can build enterprise class storage from commodity components.” And they’ve done it. They said it, and a lot more, in this paper, FAB: enterprise storage systems on a shoestring (pdf), by Svend Frølund, Arif Merchant, Yasushi Saito, Susan Spence and Alistair Veitch.

FAB: the mission
The authors begin by noting what should be obvious to everyone in the storage business: enterprise disk arrays are expensive. Why?

. . . disk arrays are constructed from customized, very reliable, hot swappable hardware components. Designing and building the hardware components is time-consuming and expensive, and this, coupled with relatively low manufacturing volumes, is a major factor in the high price of storage systems — high-end arrays retail for many millions of dollars.

Another cost factor, and problem for customers, is the lack of scalability of a single system. There is a high up-front cost for even a minimally configured array, and a single system is limited in both capacity and throughput. Many customers exceed these limits, resulting in poor performance or a requirement to purchase multiple systems, both of which increase management costs. The lack of scalability forces manufacturers to build multiple products, or even entire product lines, each targetted at different system scales. For example, Hewlett-Packard sells three different array lines, each of which effectively multiplies the engineering effort required — for hardware and firmware development, for integration and testing, etc, costs which are reflected in the price paid for each system.

Which doesn’t even mention the storage management nightmare this crazy-quilt engenders.

FAB: the hardware
Frølund et. al. set a high bar for enterprise-class storage:

. . . absolute reliability: a disk array must never lose data or stop serving data, under any circumstances short of complete disaster.

Using commodity components – a rack mount box containing disks, CPU, NVRAM and network cards – a FAB is a decentralized cluster that presents logical volumes to servers across Ethernet or Fibre Channel. The FAB interconnect is Ethernet, cheap and, with the arrival of 10Gb Ethernet, plenty fast. And that’s about if for the hardware. The magic is in the software.

FAB: the software
FAB is a decentralized cluster and clusters have challenges that uniprocessor or SMP systems – i.e. standard storage arrays – don’t. These challenges include:

• Failure tolerance: commodity hardware is less reliable than enterprise hardware, so FAB must handle hardware failure gracefully.
• Single-system view: FAB needs to look like a single array to the server, even though there is no centralized software that manages the presentation of FAB to servers.
• Asynchronous operation: Timeliness cannot be assumed, so data replication must be coordinated without assumptions about speed or connectivity.
• Hardware independence: Brick technology changes often, so the brick’s rev level or storage capacity must be discovered, not assumed, and then used appropriately.

Part II of Enterprise Storage On a Shoestring coming soon. Comments welcome, of course, with moderation turned on only to deter comment spam.