A new web scale – they claim linear scaling to 50,000 nodes — filesystem from MaxiScale has some interesting wrinkles.
Wrinkle #1 – like several of the largest Web scale filesystems MaxiScale does not use RAID. Instead, it replicates files among peer sets, a group of two or three disks on separate storage nodes that replicate each other’s data.
Wrinkle #2 – is a property of peer sets: distributed metadata. Unlike distributed lock manager’s that require is a low latency backend network for efficient management file metadata is attached to the peer set rather than the entire cluster.
In effect a MaxiScale filesystem can be thought of as potentially tens of thousands of two or three node clusters with a shared single namespace. is the file system does require a MaxiScale client then keeps track of the location of file objects.
Wrinkle #3 – MaxiScale offers three different repositories optimized for different workloads: normal (>1 MB) files; small (< And1 MB); and key value/object store.
More cool stuff
Their software also enables drive hot-swapping – critical for staying online.
They also run MapReduce to manage the cluster. A massively parallel data management tool running a massively parallel cluster. Cool.
And it runs on commodity hardware.
The StorageMojo take
Just when I’d thought that HP had cornered the massively scalable market with their IBRIX purchase, Maxiscale comes along with a new take on the problem. They aren’t a general purpose file system – the required client software eliminates that – but for the web-facing file serving market they appear to have a compelling solution.
Courteous comments welcome, of course. James Hamilton shares his take on MaxiScale here. Gary Orenstein of MaxiScale writes about Small Files, Big Headaches: Ensuring Peak Performance.