Our digital civilization requires data integrity and long-term preservation, and neither is assured by our current storage infrastructure. But progress continues.
Latest case in point: Amplidata. This 4 year old company, based in Belgium with a growing US footprint, brings a new level of erasure code goodness to the both problems with a cluster-based object store.
What erasure code goodness, you ask? The first – AFAIK – rateless erasure code, AKA fountain code, storage system in production use.
And that is a good thing because?
Robustness and efficiency.
Amplidata claims storage durability well beyond RAID 6: 10 9’s (spread across 16 drives with up to 4 failures) durability – though the spreads can be much larger logically and geographically. They do this by breaking the data object into segments and adding redundancy data.
The redundancy data adds about 50% to the object size – more efficient than mirroring or triple replication. The benefit is that the system can lose hundreds of segments and still reconstruct the data.
Each object is protected by checksums that can protect against more than 1000 simultaneous bit errors per object. And each write goes to at least to controllers before it is committed.
What kind of monster controller is able to perform all this magic? The minimum configuration is 3 Xeon-based commodity controller nodes with as many 10-drive Atom-based storage nodes as you need.
Amplidata is optimized for bandwidth, not IOPS. With their latest software update they now spec each controller at 750MB/sec, and you can have as many controllers as you can afford.
Sounds like Cleversafe
Cleversafe thought so too, and they’ve sued Amplidata for patent infringement. But Intel – who knows about patents and due diligence – invested after the suit.
Like NetApp’s suit against ZFS, this seems like a vanity project. Surely Cleversafe has more important things to invest in. If they don’t they’re in bigger trouble than we know.
The StorageMojo take
The need for robust, inexpensive and massive storage has been a theme of StorageMojo’s for years. Object storage is the best solution to the problem of scale, while the kind of redundancy and end-to-end checksumming that Amplidata uses seems as robust as anything on the market today.
As for inexpensive, that is in the eye of the beholder, but Amplidata tells me that their newest storage node lists for less than $0.60/GB while consuming only 60 watts. That should be attractive to people running tape silos who want faster access and better redundancy.
Courteous comments welcome, of course. I’m working with Amplidata to produce a video white paper on their technology, so stay tuned for more info on a promising company.
Recent Comments