Can object storage ever be as fast as block storage? It turns out the answer is yes.
And we already know how to do it.
I was speaking to the CTO of ClearSky Data, Laz Vekiarides, about their block storage system for enterprise applications. They offer
. . . a Global Storage Network that manages the entire enterprise data lifecycle, as a fully-managed service.
ClearSky is a cloud-based service that makes some usual and unusual promises:
- Eliminate storage silos.
- Pay as you grow – and populate thin-provisioned volumes.
- On-premise performance + cloud scale.
- Multi-site data access without replication.
- Fully managed, 24×7 support.
- Guaranteed 99.999% uptime.
- Consumption-based pricing.
- Substantially lower cost than legacy arrays AND AWS EBS.
It was the last promise that got me really interested. How do you provide cloud-based block storage at a substantially lower price than Amazon offers it, using Amazon’s infrastructure, while making it fast enough for transactional workloads?
The answer we already had
Cloud storage: high latency and limited bandwidth. Sounds like a disk, doesn’t it?
Let’s see, what did we do to make disk performance work? Oh, yes, caching.
Which is, essentially, what ClearSky does: they put a big, fast, scalable, SSD cache in front of cloud storage to provide Big Iron array performance, without Big Iron’s insupportable costs. The 2U rackmount caches – up to 32TB each – are highly redundant, clusterable for growth, and connect to a metro Ethernet POP over a private network.
Of course, there’s much more to what ClearSky does than this. Their Smart Tiering keeps track of hot, warm, and cold data. They have special POPs – in Boston, New York, northern Virginia, and Las Vegas for now – that dramatically reduce the latency that their edge appliances have to deal with.
They simplify storage management as well. Customers only have to manage LUNs and such, not the physical devices and interconnects. DR is built-in, if you have two or more IT locations. And more.
Bottom line: ClearSky offers a replacement for a VMAX array for a fraction of the cost.
But here’s the cool thing: ClearSky stores your data in the cloud as objects, not blocks. That’s how they can offer 4k block storage for a fraction of the cost of Amazon’s Elastic Block Storage.
Blocks into objects
So how do you serve blocks and store objects? While in theory there’s no reason why objects couldn’t be 4k each, the overhead required to keep track them would overwhelm the system with detailed metadata. Something has to give.
The local edge cache stores blocks. But when the blocks are moved into the POP object store, they are concatenated into 4MB objects. When a block is accessed, the system first goes to the 4MB object, which keeps track of its own 1,000 4k blocks, and extracts the block.
The POPs are equipped with SSDs to keep track of the metadata, so the lookups are fast and, if the data is warm (cached), the block read is too. Since the metro POP latency is 1-2ms, even the occasional block read from the POP is as good as traditional arrays.
The StorageMojo take
ClearSky should be on anyone’s shortlist for fast block storage with cloud pricing. I’ve only scratched the surface of what they’ve got. Their security story – end-to-end AES-256 in transit and at rest, with keys stored locally with TPM key management – is also impressive.
But making object storage really fast is a key problem for the coming decade. It looks like ClearSky has figured out how to do it.
Courteous comments welcome, of course. Updatee to correct ClearSky’s preferred spelling.