Can object storage ever be as fast as block storage? It turns out the answer is yes.
And we already know how to do it.
I was speaking to the CTO of ClearSky Data, Laz Vekiarides, about their block storage system for enterprise applications. They offer
. . . a Global Storage Network that manages the entire enterprise data lifecycle, as a fully-managed service.
ClearSky is a cloud-based service that makes some usual and unusual promises:
- Eliminate storage silos.
- Pay as you grow – and populate thin-provisioned volumes.
- On-premise performance + cloud scale.
- Multi-site data access without replication.
- Fully managed, 24×7 support.
- Guaranteed 99.999% uptime.
- Consumption-based pricing.
- Substantially lower cost than legacy arrays AND AWS EBS.
It was the last promise that got me really interested. How do you provide cloud-based block storage at a substantially lower price than Amazon offers it, using Amazon’s infrastructure, while making it fast enough for transactional workloads?
The answer we already had
Cloud storage: high latency and limited bandwidth. Sounds like a disk, doesn’t it?
Let’s see, what did we do to make disk performance work? Oh, yes, caching.
Which is, essentially, what ClearSky does: they put a big, fast, scalable, SSD cache in front of cloud storage to provide Big Iron array performance, without Big Iron’s insupportable costs. The 2U rackmount caches – up to 32TB each – are highly redundant, clusterable for growth, and connect to a metro Ethernet POP over a private network.
Of course, there’s much more to what ClearSky does than this. Their Smart Tiering keeps track of hot, warm, and cold data. They have special POPs – in Boston, New York, northern Virginia, and Las Vegas for now – that dramatically reduce the latency that their edge appliances have to deal with.
They simplify storage management as well. Customers only have to manage LUNs and such, not the physical devices and interconnects. DR is built-in, if you have two or more IT locations. And more.
Bottom line: ClearSky offers a replacement for a VMAX array for a fraction of the cost.
But here’s the cool thing: ClearSky stores your data in the cloud as objects, not blocks. That’s how they can offer 4k block storage for a fraction of the cost of Amazon’s Elastic Block Storage.
Blocks into objects
So how do you serve blocks and store objects? While in theory there’s no reason why objects couldn’t be 4k each, the overhead required to keep track them would overwhelm the system with detailed metadata. Something has to give.
The local edge cache stores blocks. But when the blocks are moved into the POP object store, they are concatenated into 4MB objects. When a block is accessed, the system first goes to the 4MB object, which keeps track of its own 1,000 4k blocks, and extracts the block.
The POPs are equipped with SSDs to keep track of the metadata, so the lookups are fast and, if the data is warm (cached), the block read is too. Since the metro POP latency is 1-2ms, even the occasional block read from the POP is as good as traditional arrays.
The StorageMojo take
ClearSky should be on anyone’s shortlist for fast block storage with cloud pricing. I’ve only scratched the surface of what they’ve got. Their security story – end-to-end AES-256 in transit and at rest, with keys stored locally with TPM key management – is also impressive.
But making object storage really fast is a key problem for the coming decade. It looks like ClearSky has figured out how to do it.
Courteous comments welcome, of course. Updatee to correct ClearSky’s preferred spelling.
Twin-strata (now EMC cloud array) I think had a roughly similar approach, with a front end block/file cache for fast accesses that synced to larger objects in the back.
That said,it was my experience working on the cloud provider side that these arrays are capable of producing very strange and problematic workloads for the backing object store. A few examples we saw: High rates of overwrites and deletes of objects, proliferation of huge numbers of objects in the same namespace container, other headaches for supporting services that try to maintain listings, scrubbing services, etc.
I’d be really curious/interested to see a codesign approach to try to mitigate some of these issues, assuming they still are a thing for these guys.
I’ve been following ClearSky Data for a while now, and I have to say that their solution is quite different from CloudArray…or any other storage solution that I’m aware of. They approached it from a totally different angle – ingenious, actually. Their architecture has solved the object storage problems that Ian discussed. From my understanding, ClearSky’s solution was designed from the beginning to address object storage and latency, thinking about each possible problem and solving them along the way. The result – super-fast storage, tiered caches (not just tiered storage, speaking to the speed of accessing the “remote” data), and efficient use of object storage without the churn commonly found in other solutions…distributed and optimized, without multiple copies everywhere.
That doesn’t mean that their storage isn’t enterprise-grade, though. ClearSky’s storage DNA brings all the features you would expect; and for VMware environments, full VAAI and VVOL support. While CloudArray claims that they support “familiar SAN and NAS interfaces”, they’re missing one for critical plug-n-play interoperability in the enterprise – Fibre-channel. Yep, ClearSky’s got that covered, too.
And you can’t beat their price/performance. No hardware investment up-front, and you only pay for storage you actually use…after layers of intelligent dedupe and compression without any performance hit…all fully managed.
So what happens when a read cache miss happens in the middle of a database transaction, and that uncached data is not in the “warm†tier either? Not trying to be snarky but just asking how does the architecture handle that inevitable scenario.
The size of the warm tier makes misses very unlikely on a continuously running application. The warm tier is intended to hold many weeks of ingested data and there are very clear signs that it has grown cold after a while.
Nonetheless, a miss that goes through both tiers will be serviced from the cloud, but over our high speed network. The latency here in the Northeast is 11 ms or less, so its not so bad. In fact, it isn’t noticeable unless there was an explicit management operation to revive a stone cold snapshot from a long time ago.