Our digital civilization requires data integrity and long-term preservation, and neither is assured by our current storage infrastructure. But progress continues.

Latest case in point: Amplidata. This 4 year old company, based in Belgium with a growing US footprint, brings a new level of erasure code goodness to the both problems with a cluster-based object store.

What erasure code goodness, you ask? The first – AFAIK – rateless erasure code, AKA fountain code, storage system in production use.

And that is a good thing because?
Robustness and efficiency.

Amplidata claims storage durability well beyond RAID 6: 10 9′s (spread across 16 drives with up to 4 failures) durability – though the spreads can be much larger logically and geographically. They do this by breaking the data object into segments and adding redundancy data.

The redundancy data adds about 50% to the object size – more efficient than mirroring or triple replication. The benefit is that the system can lose hundreds of segments and still reconstruct the data.

Each object is protected by checksums that can protect against more than 1000 simultaneous bit errors per object. And each write goes to at least to controllers before it is committed.

What kind of monster controller is able to perform all this magic? The minimum configuration is 3 Xeon-based commodity controller nodes with as many 10-drive Atom-based storage nodes as you need.

Amplidata is optimized for bandwidth, not IOPS. With their latest software update they now spec each controller at 750MB/sec, and you can have as many controllers as you can afford.

Sounds like Cleversafe
Cleversafe thought so too, and they’ve sued Amplidata for patent infringement. But Intel – who knows about patents and due diligence – invested after the suit.

Like NetApp’s suit against ZFS, this seems like a vanity project. Surely Cleversafe has more important things to invest in. If they don’t they’re in bigger trouble than we know.

The StorageMojo take
The need for robust, inexpensive and massive storage has been a theme of StorageMojo’s for years. Object storage is the best solution to the problem of scale, while the kind of redundancy and end-to-end checksumming that Amplidata uses seems as robust as anything on the market today.

As for inexpensive, that is in the eye of the beholder, but Amplidata tells me that their newest storage node lists for less than $0.60/GB while consuming only 60 watts. That should be attractive to people running tape silos who want faster access and better redundancy.

Courteous comments welcome, of course. I’m working with Amplidata to produce a video white paper on their technology, so stay tuned for more info on a promising company.

BlueArc and Drobo founder Geoff Barrall has a new perch: Gridstore, one of the companies I’ve been following for almost 3 years. Geoff is the new executive chairman. Formal announcement is expected this week.

Gridstore’s concept is a low-cost scale-out NAS appliance designed for office environments. Each box is a small, low-power node with a couple of TB. Stack ‘em for as much redundancy, capacity and performance you want.

Think of it as the consumerization of hyper-scale technology. Nutanix writ small.

Gridstore details
Gridstore is offering a low-cost, scale-out network file server for $500 a node. That is too cheap for the enterprise storage companies to sell directly.

Founded 5 years ago, Gridstore got a beta out in 2010, and have been shipping for well over a year. They are a Microsoft CIFS protocol file server, using Microsoft’s storage server software. Running on small, 25 watt Atom-based boxes, a 6 node configuration is the size of a bread box.


Like other scale-out NAS systems, the Gridstore NAS has no single point of failure and can survive multiple node failures without going down or losing data.

They call their redundancy scheme RAIDg. When you set up a volume you dial in how many faults you want to survive and the software handles the rest.

Today the number of faults they can handle is limited to half the number of nodes minus one. If you have a 6 node configuration it can handle the loss of 2 nodes. They expect to relax that requirement in the future.

The StorageMojo take
Haven’t spoken to Geoff about this, but Gridstore seems like a natural for him. If there’s a theme to his many endeavors, its making advanced NAS technology more accessible.

Gridstore fits the bill nicely. If there’s one complaint about Drobo, its the lack of box-level redundancy. Gridstore answers this objection, at a higher price point.

Drobo – over 200,000 units sold – has blazed a trail for bringing advanced storage technology to the masses at affordable prices. They may be the first, but as Gridstore and others demonstrate, they won’t be the last.

Courteous comments welcome, of course. Hoping to make it to CES later this week. Readers: anyone I should make a point to see?

A StorageMojo reader has a problem. Can you help?

Our mail hub (80,000+ mailboxes) is virtualized with vSphere 4.1 with Red Hat Enterprise Linux 5 x64 and Dovecot 2.0 [an open source IMAP/POP3 email server for Linux/UNIX-like systems]. We are using HP LeftHand Networks P4300 iSCSI storage in a “network RAID10 setup of RAID10 storage” for Dovecot indexes and multiple “networks RAID1 of RAID5 storage” for actual mailboxes.

This is my take: our Dovecot indexes are getting hammered with lots of small I/O requests, about 8,000 IOPS continuous during 8-working-hour days, 75% write. Indexes are fairly small (50 GB) and expected to grow to 100-150 GB, but need a lot of random I/O. We need real-time replication in storage (LeftHand is ok for us) and we think that SSD should shine in this situation. Bandwidth is not a problem (200-300 megabits of indexes traffic, but we need more IOPs).

The problem is the indexes, but our total mailbox capacity is expected to grow to 6 TB compressed using zlib compression in Dovecot.

We want to buy a storage appliance with the following requirements:

• Vsphere 4.1 & 5 certified storage, VAAI enabled (if possible)
• iSCSI (1 gbps)
• High number of IOPS (at least 12,000+, most of them writes)
• Small size (200 GB)
• Fault tolerant (RAID, battery-backed write cache, power supply, fans, multiple gigabit uplinks, synchronous replication)
• Cheap (less than $30k the full setup)

We want to buy at the beginning of 2012. Any product that fits?

The StorageMojo take
Suspect price will be the most significant limiter. But the respondent only needs index storage not the whole shooting match. He’s pretty happy with LeftHand for mailbox storage.

But if we can solve both problems for him, why not? If he should relax some constraint, feel free to suggest it.

He’ll be watching the comments, so if you have questions please ask them. I’ll be following the comments as well.

Courteous comments welcome, of course. His email was edited for clarity.

Most of us know what it is like when a relationship goes bad: the sinking feeling that this just isn’t going to work.

Can this configuration be saved?
Dear StorageMojo:

I joined a company last year that is running Oracle 10g on a NetApp NAS/SAN.

Immediately I asked why they were not using Clustering, Oracle RAC, Oracle ASM or Fiber Channel. No answer.

Fast fwd to a year later and they are asking me to deploy this to an I/O bound customer with hundreds of connections and lots of transactions to their DB over NFS.

Long story short it’s slow-w-w-w. They tried trunking multiple network connections. They tried tuning. They tried a bunch of stuff. And it’s still a dog.

How slow?

I have a screaming Dell r710 running a 7TB database attached over SAS to a set of MD3000 storage arrays. I am getting 450MBs…..and this barely suffices…..

The “new” system they showed me gets 50MBsec…the same screaming Dellr710 but connected over NFS (instead of SAS) to the NetApp NAS.

Do you have any suggestions?

Thank you for reading this nightmare.
Bob

Poor Bob! He’ll be getting grief from the client for months, maybe years to come, unless this gets fixed.

The StorageMojo take
Maybe Bob could have been better about developing a relationship with the guys configuring the systems. More questions, fewer conclusions, at first.

Suggestions to the customer for acceptance testing might be in order.

But there are 2 problems here:

1. What to do now.
2. How to keep this from happening again.

What would you suggest to Bob on either or both topics? I’ve asked him to watch the comments, so if more info would be useful, I hope he’ll provide it.

Courteous comments welcome needed. When a multi-billion dollar near-sighted telescope can get sent into orbit, it is surprising more IT projects don’t go wrong.

Talked to Nimble Storage a few months ago. The 1st time they sounded cool and now I know why.

What they do
Nimble builds a converged storage appliance out of commodity hard drives and SSDs that offers high performance – is there any other kind? – and iSCSI, backup, a form of dedup and WAN replication. The pitch is EqualLogic & Data Domain merged into a single low-cost appliance. Only better.

• iSCSI + dedup
• Capacity-optimized snapshots
• SATA + flash instead of high-rpm drives
• Can run off a remote snapshot

EL & DD sell a lot of kit, so this could work.

Claim to fame
Cache Accelerated Sequential Layout is what Nimble calls their secret sauce.

CASL combines a variable block size, in-line compression, application-specific block sizes and checksum and compression data kept in the block header. They coalesce the blocks and only write in full stripes to disk.

The box has a large flash-based cache where the full stripe writes are also written, overcoming the small write performance hit that flash shares with parity raid. This also insures a high percentage of cache hits on the first read.

The system maintains an index of where all the blocks are written. Typically, this index is also held in flash for maximum lookup performance.

App-specific block sizes
Nimble uses of variable block sizes to improve performance. For example, the last three versions of exchange have all used different block sizes. CASL recognizes the different versions of Exchange and dynamically adjusts its block size to the best fit.

They claim a 2x performance advantage on Exchange databases.

Coalesce
They take the variable size blocks then coalesce those blocks into big chunks and write to flash. They write in large blocks – full block writes to flash and in full stripe writes to disk. Result: fast reads & writes across both media

Their page sizes are variable but small, ranging from 4KB to 64KB. The greater granularity means that frequent snapshots are much smaller than large page size systems like EqualLogic.

The StorageMojo take
There’s no reason that data protection should be separate from data storage. We’ve been moving towards integration since the CDP craze.

The average business wants to store and protect their data and they don’t want to spend much time or money on it. Nor should they.

With powerful commodity processors and nickel-per-GB storage there’s a huge market for a box that – or 2 or 3 boxes – that

• Stores terabytes of data
• Protects that data with local replication and frequent snapshots
• Auto-connects to cloud storage for DR and archiving
• Doesn’t confuse users with LUNs and stripes
• Offers Time Machine like data recovery to end users

It will look like magic – as any sophisticated technology should – and you’ll buy it at Office Max. As with any volume product the key will be architecting to maximize the user experience at an affordable price point.

Nimble certainly has the right idea.

Courteous comments welcome, of course. I don’t know which analyst the Nimble guys are blowing their money on, but it isn’t me.

These requests came in over the transom in the last couple of days. Maybe some StorageMojo readers have wisdom to share.

I have a question I hope you can help me with. My boss asked me . . . to research HP Left-hand SANs and Dell Equallogic SANs. Do you have any special knowledge of these products and, if so, would you make an informal recommendation?

What say you, StorageMojo readers? If you evaluated both, why did you make the choice you did? Vendors welcome to comment, but please identify yourself as such.

The StorageMojo take
AFAIK, both products are good iSCSI systems. Both are backed by major corporations. EqualLogic may be stronger in the channel today, but HP has channel chops as well. HP’s blade servers may be a more expandable platform, but EqualLogic’s software portfolio may be more affordable.

Translation: you could do worse than either of these.

Part II
Another customer perplexity: service.

We have a pair of HP disk arrays, EVA 8000 and 6000 and I am looking for a consultant to help up with storage planning. Do you do such work or could you recommend someone to me. I am looking for someone who goes beyond just being a seller, I have plenty of potential sellers already.

The writer is in a small city in the Mountain West, so you should be used to working remotely with clients. No, not in Arizona.

The StorageMojo take
HP folks may be wondering: why doesn’t he call HP? My guess: not big enough for a direct engagement.

Courteous comments welcome, of course.

Imagine cloud storage that didn’t cost much more than bare drives. High density storage with RAID 6 protection, reasonable bandwidth and web-friendly HTTPS access.

And really, really cheap.

Raw disk cost is only 5-10% of a RAID systems cost. The rest goes for corporate jets, sales commissions, 3 martini lunches, tradeshows, sheetmetal, 2 Intel x86 mobos, obscene profits and a few pale and blinking engineers in a windowless lab who make the whole thing work.

Storage for ascetics
But let’s say you didn’t want the 3 martini lunch or the barely-clad booth babes. All you want is really cheap economical, reasonably reliable storage.

You aren’t running the global financial system – what’s left of it anyway – and you don’t have a 2500 person call center hammering on a few dozen Oracle databases 7 x 24. No, you’re thinking a quiet cloud storage business for SMB’s, maybe backup and some light file sharing, that will give you a nifty little revenue stream with annual renewals so you can see trouble coming 12 months in advance.

Enough redundancy so when something breaks you can wait until morning to fix it instead of an 0300 pajama run to the data center. Easy connectivity so you aren’t blowing the savings on Cisco switches.

Bliss
Well, you aren’t the only one. Backblaze, a new online backup provider, designed the Storage Pod for their own use and are sharing it with everyone. They aren’t in the hardware business and I think they figured sharing it would be a nice little attention-getting device.

It worked for me. Here’s the box – which they are using in production.

Here’s an exploded diagram with a simplified BOM:

And then there’s the (free) software. 64-bit Debian Linux, IBM’s open source JFS file system and HTTPS access. Put a stateless webserving front end on it and you’re good to go. Scale out the webserver and add storagepods to grow the system.

The StorageMojo take
This isn’t general purpose or high-performance storage. Nor is it backed by global network of 7 x 24 service professionals. But there are a lot of applications out there that just need a big bit bucket. This is it.

No one is manufacturing this for you either – which is a good thing. If you don’t know what you’re doing you can get in a lot of trouble with a lot of data real fast. Want to be the Bernie Madoff of cloud storage?

But the density is good, the performance is reasonable, the availability is decent and the price is right. This is a DC-3, not a 747. It is all you need for the right application.

Courteous comments welcome, of course. See the plans and get the box model in the paper Backblaze put together.

The video folks have an interesting set of problems: large needs; major bandwidth; time-critical collaboration; lots of metadata; and more. Like budgets. I do some video production myself and empathize.

They are today where most of us will be in 10 years: lots of large files; local and remote sharing; processor and bandwidth intensive operations; large archives of wanted and rarely accessed files. Today high-end video folks are working at 2k, 4k and, sometimes, 8k video resolutions – and 10 years from now I wouldn’t be surprised if home users weren’t too.

What prompts this is a note I received from, well, I’ll let him introduce himself.

I have a boutique post-production company and I’m a filmmaker. We are small, under a dozen, but swell to a few times that size with freelancers on a project-by-project basis. Because we work with very high resolution media, we need a lot of space, and very high throughput to each user. . . . [W]e’re all working with 2K and 4K media (300 and 1200MBps respectively to EACH user) and 3D animation rendering. . . . We use a mix of Linux, Windows, and OS X clients. In total, we could easily make use of 100TB+ right now, and prefer to stop archiving everything to tape and deleting it, but rather migrate to another tier of storage but keep in one global namespace with the tape just for disaster recovery. We also need security administration.

I can’t find a storage system that does all this. DataDirect Networks seems to be the du jour high-end storage for my industry, and supposing I’m willing to finance that big-ticket brand, they still don’t have a filing system answer. They’re suggesting StorNext or CXFS, and I know the multi-user scalability and expansion limitations well (can anybody say “forklift”?).

The closest I’ve come is Lustre. It seems like it would fit the bill nicely, especially since we’re savvy to integrate in-house, except that it is Linux only, and NFS/CIFS gateways don’t seem like a great idea. I keep hearing they’re working on at least a Windows client, but who knows when it will be ready?

Can you help at all? What have I overlooked? Doesn’t anyone make what I’m looking for?

Short answer to last question:
No.

Longer answer:
No. But there are workarounds.

For those new to video, here’s an abbreviated chart of some video rates in megabytes per second:
[Adapted from Integrity Data Systems which offers the whole chart. Aspect ratios and frame rates left out.]
Update: Larry Jordan, a writer and trainer in video editing, graciously wrote to let me know that the above data rates are uncompressed – and that most production houses would use compressed data. The amount of compression varies based on the codec as Larry explains in this informative post. End update.

Issue 1: Interconnects
GigE won’t even handle 32-bit RGB standard def video. And when you get into HD video it gets hairier fast. Trunk multiple GigE’s? 10GbE? 4x Infiniband? FC? eSATA or PCI-e direct attached storage?

Issue 2: Virtualization
A single address space is a wonderful thing. You’ll need a software layer that clusters multiple boxes. You’ll also probably want to build an archive infrastructure that is distinct from your higher performance working set storage, but some vendors will disagree.

Likely software suspects include IBRIX, Parascale, Caringo, MatrixStore, Bycast and Permabit.

On the combined HW/SW side there’s Panasas and Isilon. Something tells me there are some other options, like HP’s Extreme Data Storage 9100, that are also applicable.

Lustre is not a product I would recommend since it was designed for HPC, a market where PhDs work as sysadmins. Sun may have tamed it since they bought it, but it is a non-trivial piece of software.

Come one, come all
StorageMojo readers are invited to offer their 2¢ worth. Architecting is non-trivial, especially if money is an object.

Update:
Our interlocutor wrote in to add some detail:

thanks for the response. Here’s some answers:

– We can manage expensive interfaces like 10GigE and Infiniband QDR. We’ve been paying for dual-channel 4Gb FC for the past few years, after all. I just want to also allow standard Gigabit connections to the cheap seats without a lot of complexity. So I guess the jargon for that would be “multiprotocol” switching?

– The large naming space might be a luxury. The fact is that jobs come in one of three general sizes, and we could have volumes of that size waiting to take on new jobs as they come in, so at least there is one namespace per job. As you said, capacity is cheap…

– Truth is I am pretty savvy, but other than that we have a lot of power desktop users but not sysadmin types. I contract some people with steady part-time work, but it has been our business model to try to keep as many of our full-time people on the creative and producing side as possible, and not in support/administration.

The one thing I don’t understand is what you say about Infiniband not being so great when there’s lots of node churn?

I know what you mean about DAS, but I think I’ve ruled out distributing the data through push/pull from a central repository. The fact is jobs just move to fast through here for that, and we often have about two seconds notice that we need to bring a certain job’s data to System X, Y or Z to do work on it. It’s very dynamic.

I see some brands in your blog post I haven’t checked on yet.

What turned me onto Lustre is that Frantic Films in London has deployed it. They’re the only ones AFAIK.
End update.

The StorageMojo take
Some thoughts on the infrastructure issues.

Capacity is cheap, network bandwidth is expensive. Raw SATA disk is less than $0.10/GB. 10GbE switch ports are over a grand apiece. Infiniband is better from a price/performance perspective, but not as friendly for networks where there is much node churn – unless that’s been fixed in the last few years.

Direct attached storage will give you the best performance – especially with 4k. The new PCI-e attached arrays from JMR and others can offer up to 4,000 MB/sec bandwidth. Stripe across 4 of those and you’ll be able to handle 8k.

Transaction processing is well on its way to niche status, like mainframes and hierarchical databases that once ruled the earth. It is a big file world out there and the files are getting bigger every year.

Courteous comments welcome, of course. I’ve done work for many of these folks – but not all – at one time or another.

Hewlett-Packard’s acquisition of the LeftHand Networks shows how cluster storage is going mainstream – and how HP plans to be right in the middle of it. First PolyServe and now LeftHand.

This is about commodity-based clusters
Not iSCSI or GigE or 10 GigE as a storage interconnect. Fibre Channel’s failure to move downmarket – and Infiniband’s similar problem – means GigE is the only game in town.

Reaching the huge, not currently imploding, SMB market requires meeting people where they live. SMBs don’t live in Fibre Channel glass houses. GigE isn’t ideal, but it’s cheap and it works.

Did HP overpay?
$360 million isn’t pocket change, but it is only about 4x the $86 million investors put in. The investors get some nice coin, but it isn’t the 10-bagger they were hoping for.

Once the Lefties go through the interminable internal HP meat grinder, sales will grow rapidly. I suspect they weren’t up to Isilon’s $100M in sales – maybe $70M – but LeftHand was much closer to profitability. Net net: the price looks fair for a market leader in a high-growth market.

HP vs EMC
Battle of the competing cluster storage visions. Polyserve handles files; LeftHand blocks. EMC’s Maui is aimed at large-scale distributed file storage, a utility that ISP’s might resell to SMBs, but nothing an SMB would implement on their own.

Which will win – and there’s room for both – rests on the answer to the question Are there economies of scale in storage?. If there are, small-scale clusters sales will suffer and Maui should win.

The StorageMojo take
This is cluster storage market skirmishing, not a pitched battle. That will come but right now everyone is feeling their way, coming into the market from different directions, waiting to see what clicks.

Right now though, HP seems to have the strongest position. XIV is too new; Maui even newer; Lustre too complex; Isilon is digging out of a big hole. HP has the pole position with implementable products today and the services to back them up. Should be a powerful combination.

Courteous comments welcome, of course. Disclosure: I’ve done some work for HP, Isilon and Sun.

StorageMojo has long held the view that our storage workloads are changing: more file storage, less block storage; larger file sizes; and cooler data. While all the indicators said this was happening it’s good to find a study that confirmed this intuition.

In the Measurement And Analysis Of Large-Scale Network File System Workloads (pdf) researchers Andrew W. Leung and Ethan L. Miller from UC Santa Cruz and Shankar Pasupathy and Garth Goodson of Netapp measured 2 large file servers for 4 months. Their results are worth reviewing, since so many of the optimizations in storage infrastructures rely on workload assumptions.

Unstudied CIFS
The authors point out that there have been no major studies of the CIFS protocol, odd since it is the default on Windows systems. Furthermore, the last major study of network file loads was performed in 2001 – seven years ago – an interval in which average this drive sizes have gone from 20 GB to 500 and network speeds from 100 MB to 1 GB.

Most surprising, however is that no published study has ever analyzed large-scale enterprise file system workloads. Researchers have studied workloads closer to home: university and engineering workloads.

Enterprise workloads
One was a midrange file server with 3 TB of capacity with almost 3 TB used by over 1000 marketing sales and finance employees. The second server was a high end Netapp filer with 28 TB capacity – 19 TB used – supporting 500 engineering employees.

Yes, marketers, engineers get the good toys. You can cry about it over your next 3 martini lunch.

Some significant differences from prior studies:

• Workloads more write oriented. Read/write byte ratios and are now only 2 to 1 compared to the 4-1 or higher ratios reported earlier.
• Workloads less read-centric. Read/write workloads are now 30x more common.
• Most bytes transferred sequentially. These runs are 10x the length found in the old studies.
• Files 10x bigger.
• Files live 10x longer. Less than half are deleted within a day of creation.

Cool new findings

• Files rarely re-opened. Over 66% are re-opened once and 95% fewer than 5 times.
• Over 60% of file re-opens are within a minute of the first open.
• Less than 1% of clients account for 50% of requests.
• Infrequent file sharing. Over 76% of files are opened by just 1 client.
• Concurrent file sharing very rare. As the prior point suggests, only 5% of files are opened by multiple clients and 90% of those are read only.
• Most file types have no common access pattern.

And there’s this: over 90% of the active storage was untouched during the study. That makes it official: data is getting cooler.

Another interesting finding: 91% of VMWare Virtual Disk (vmdk) files accesses were small sequential reads – not the larger sequential accesses I’d expect.

The StorageMojo take
The writers rightly suggest that given the rarity of file reads after creation it makes sense to migrate files to cheap storage sooner than later.

Perhaps primary file storage should be thought of as a large FIFO buffer – tossing 3 month old files to an archive for long-term storage. A data flow architecture instead of a series ever-larger buckets.

Kudos to NetApp and UCSC for this work. It seems like NetApp has been doing the best job of leveraging academic researchers lately. I’d like to see them get more marketing mileage out of their good work.

Courteous comments welcome, of course.

I wrote a short piece on ZDnet about Los Alamos National Labs new Cell Broadband Engine based supercomputer, Roadrunner. With ~14k v.3 Cell processors – an earlier version powers the PS3 game console – and another ~7k dual core Opterons, the Roadrunner’s ~3,250 compute nodes pack a lot of compute cycles.

The key compute element is the new version of the PS3 chip – called a PowerXCell 8i Processor – features 8x faster double-precision floating point and over 25 GB/sec of memory bandwidth. And it can address 64 GB RAM. There are 4 8i’s per compute node.

Nothing I read mentioned the disk storage – until the friendly Panasas PR person suggested I talk to Larry Jones, VP Product Marketing. Panasas is providing the back end storage for Roadrunner.

I did, and here’s what I learned.

LANL storage infrastructure
LANL’s 6 supercomputers + Roadrunner share the Panasas storage through LANL-developed IO nodes. While Roadrunner itself uses dual-data-rate 4x Infiniband for internode communication, the I/O nodes attach to Panasas through trunked GigE.

The advantage of the I/O nodes is that the entire Panasas storage pool is available to each supercomputer. Lots of bandwidth.

Roadrunner currently has about 80TB of RAM, roughly 24 GB per compute node. That works out to about 4 GB RAM per processor.

The jobs these machines run are huge. A simulation can run 6 months or more. Depending on criticality a job gets checkpointed every hour or maybe once a day.

The Panasas installation at LANL, begun in 2003, is currently 2 PB. Assuming an average of 500 GB drives, that means 4,000 disk drives.

Panasas uses 5 trunked GigE links to each of the 8 controllers in a single rack. They are now in beta for 10 GigE, which reduce link count from 40 to 8 per rack while doubling bandwidth.

The hot rodders at LANL should like that.

The StorageMojo take
Roadrunner’s 80 TB RAM is a sizable storage infrastructure in its own right. Keeping it fed and backed up is a major job.

Consumerization of IT is a common concept – but what we see here is the consumerization of HPC: Playstation CPUs; SATA drives; Linux OS; air cooling. The old model of highly customized kit for HPC is dead.

Which is a good thing for the rest of us. We get some of the smartest people in computing working on platforms that we might also use, developing applications that otherwise would never be available to the consumer market.

I’ll never run molecular dynamics codes, but maybe my kids will. After all, I can now edit feature length movies on my desktop. Who would have believed that just 20 years ago?

Comments welcome, of course. Disclosure: I did some work for Panasas last year and – who knows? – might do some more in the future. I like the team and the way they are pushing pNFS.

I had a con call with Chris Gladwin and Russ Kennedy of Cleversafe a couple of weeks ago. They’ve come to market with a product line that seeks to deliver:

• Massive scalability to meet growing digital content requirements
• Unprecedented Security and Privacy for critical digital assets
• Survivability against disasters, dishonesty and time
• Extremely cost-effective infrastructure compared to traditional methods

That’s a quote from their pitch.

Cleversafe’s product line
Cleversafe, IIRC, started as a software company, but their announced products come in nice rack-mountable boxes. There are 3 of them:

• CS Slicestor – Dispersed Storage server – $11.3k
• CS Accesser – Dispersed Storage router – $12.3k
• CS Manager – Dispersed Storage network manager – $12.3k

The Slicestor is a 1U storage server containing 4 disks. The Accessor slices up the data and distributes it – think slice router. The Manager works out of band to monitor and manage the storage network components.

I assume the pricing includes some room for volume discounts. There is an open-source version (c. 2006) of the software. The company intends to offer a software-only version as well.

Why hardware?
The Conventional Wisdom in VC circles is that tin-wrapped software ramps revenues faster – hey, you’re selling tin + bits – at the cost of lower margins and loss of focus.

Qualifying hardware is non-trivial; so you tend to stay on one platform longer than you should. At liquidity event time, software companies fetch higher multiples, so it may be a net loss. VCs live by the Golden Rule: he who has the gold makes the rules.

What it does
Cleversafe has an iSCSI or block storage interface. It takes the data, slices it into small pieces using Information Dispersal Algorithms and then ships the slices off to storage either locally or around the world.

In the latest version you can specify how many slices the system makes and how many slices are required to rebuild the data. If you have 11 data centers around the world, you can specify that, say, 6 are required to recreate the data.

You could lose access to 5 data centers and still recover. If the local controlling authority busts into 3 or 4 data centers, they get nothing. Pretty cool if you worry about corrupt government officials getting hold of your company secrets.

The company is planning on adding FTP, CIFS and NFS in the fullness of time.

How well it works
Cleversafe claims that given sufficient low-latency bandwidth the dispersed storage is as fast as a local disk. That’s a tall order, but for now I’ll take their word for it.

Who should buy it?
The company is aiming the Dispersed Storage Network at ISPs to offer as a service and multinationals with round the clock operations and critical data.

How it works
Cleversafe uses Cauchy Reed Solomon erasure codes to slice and dice the data. These codes have several advantages:

• More capacity efficient and failure tolerant than parity codes
• Doesn’t require a license
• Code and decode are faster than other stack operations

If you’d like to play with Cauchy Reed Solomon, check out Dr. Jim Plank’s software page which includes

. . . Reed-Solomon coding, Cauchy Reed-Solomon coding, general bit-matrix coding, Reed-Solomon coding optimized for RAID-6, and Liberation coding. The documentation provides some tutorial material on matrix and bit-matrix based erasure coding.

I met the good doctor at FAST, where he was delighted to find that Clevesafe – also a FAST presenter – was using techniques he’d worked on a decade ago.

The StorageMojo take
I’m impressed with what Cleversafe has done. They will look even smarter after EMC’s Hulk/Maui announcement this spring. I suspect they’ll be bought by year’s end.

Kudos to the Cleversafe team.

Comments welcome, of course.

They haven’t reported financials for almost 3 quarters. Their stock is trading at about 20% of its peak. They fired their CEO and put founder Sujal Patel in his place. And NetApp was trying to strangle baby Isilon (see NetApp filers for $1/GB?) in its crib.

Are they goners?

I don’t think so.
I’ve been trying to read the tea leaves on the Peter van Oppen’s decision to join the board earlier this month.

Peter led the tape library company ADIC, also based in the Seattle area, for 12 years until its sale to Quantum. ADIC out-innovated Quantum – saddled with a cranky and slow DLT development group – in libraries and software as well.

If you think the folks who buy storage arrays are conservative, you haven’t sold any tape libraries. It is a tough market and ADIC did well.

So why would van Oppen join a sinking ship?
That’s why I don’t think Isilon is sinking. An external audit team is reviewing Isilon’s accounting to ensure that any financial dirty laundry – say, hypothetically, channel stuffing – gets cleaned up. They’ve been at it for months and must be about done.

The StorageMojo take
Based on the Isilon press release and pure speculation, here’s what I think is going down:

• Peter exercised some due diligence before accepting the directorship and isn’t terribly worried about the basic health of the company
• After he gets up to speed on company operations, he assumes the CEO role by July
• Sujal happily goes back to one of the best jobs in any company: CTO and Founder while the stock climbs in value

However it goes down, getting Peter on board is a real plus. Storage experience is thin in Seattle. Isilon has lots of smart people, but the storage market has many unique wrinkles that networking or software folks take a long time to learn.

Comments welcome, as always. Disclosure: I met Sujal 7 years ago and I’ve done some work for Isilon. I hope they do well.

Despite being written off for dead . . .
Isilon’s been putting their IPO money to good use: engineering the next gen of their platform that they’ve named the X-series. In the meantime they’ve been adding customers – over 600 so far – and they have 60 customers running the new kit.

Moving from an aging single-core Xeon to a dual-core Xeon – the second core isn’t turned on yet – with faster busses and more cache speeds things up. They claim up to 60% faster performance, 20% less power and heat and 10 GigE readiness. Once they get their software dual-core aware they’ll have another nice boost to offer.

The StorageMojo take
Turning over the platform more rapidly than traditional array vendors do is a good strategy. It keeps the competition off-balance and gives you something new to tell customers. What good is commodity hardware if you don’t follow Moore’s law?

That said, Isilon’s scale out architecture is the real differentiator vs NetApp and other traditional filers. More bang for the buck just underscores the differences.

Comments welcome.