Sun is building, some might say finally building, its first complete storage system, Project Honeycomb, which they previewed at a Sun event last week. Researching further I found that Sun and others have dribbled out much more information on it than is common for major storage systems. Putting that info together I found an impressive effort that might, IMHO, be the disruptive technology the storage industry needs to kickstart a new round of growth.
Keeping in mind that features are frequently
head-shot decommitted in the final forced-march to shipment, here are the key features of the system:
- Scalable massively parallel cluster architecture made of cheap components: power supplies, Opteron processors, SATA disks
- Integrated metadata and search
- High-performance parallel disk rebuilds
- Autonomic system self-management of capacity and processor changes
- N + *many* redundancy
- Near real-time embedded Solaris OS enabling storage-based data preprocessing, SQL queries, metadata extensions, and custom application deployment in the storage
Likely features, hinted at but not explicitly stated, include:
- Open source ZFS file system, with vcool ZFS features including: monster 128-bit address space; fast, lightweight snapshots; dynamic striping; storage pools – no separate volume manager; boot from archives; very fast RAID-Z: no parity RAID write hole
- 10 Gb InfiniBand processor node interconnect
A Commercial Version of Google’s Infrastructure
In a nutshell, this will be the closest commercial product to Google’s internal infrastructure. Just as Google’s (see So Mr. Tucci, Where Are EMCâ€™s Google Application Notes?) infrastructure enabled the rapid roll out of data intensive services to millions of customers, Sun’s Honeycomb promises a largely autonomous storage infrastructure that manages multiple failures and capacity changes without operator intervention, high-performance RAID, exceptional data integrity, integrated continual (not continuous) data protection and advanced metadata management and extension capabilities.
Further, to the extent that these capabilities are based upon the open-source ZFS file system, other vendors will be able to quickly offer similar functionality, creating a competitive environment that will only benefit customers.
Danger, Will Robinson
Yet, despite the brilliance of the engineers, one must consider that Sun’s Network Storage Group has an unparalleled record of ineptitude, embracing bad ideas and blowing good opportunities. Danger areas include the STK sales force, which will have no idea how to sell this, Sun marketing, with no idea how to rev up the sales to sell it, and pricing, where there will be lots of brain dead ideas to “enhance” margins and “protect” the soon-to-be obsolete big iron arrays.
As a Sun alum I’d like to see the company be successful. The good news is that in the longer term it doesn’t matter if the Honeycomb implementation fails: the ideas it embodies are inevitable. Google has shown what can be done and the commercial pressure to top them is irresistible.
John Siracusa over at Ars Technica, has also written about ZFS and its implications for storage. If Sun doesn’t fumble the marketing Honeycomb could be huge. Note I thought differently a year ago. Open source ZFS, and its storage functionality (if Honeycomb is using it) are the major reasons I’m more optimistic now.