Isilon #1 Supplier of Storage Clusters

by Robin Harris on Friday, 9 February, 2007

A reader piqued my curiosity
I got a comment today from a gentleman named Kevin Stay on an the Isilon’s Q4 Results. Kevin was a little miffed with me. He wrote:

“Isilon is validating the storage cluster market” ???

They are a tiny as yet unprofitable startup with zero market share and evidently a good implementation of a great idea.

I contrast them with Netapp and scratch my head at your assertion Isilon is doing the validating of the storage cluster market. Here we have a Fortune 1000 company growing like crazy compared to others with actual market share. Yes, Netapp gear is pricey and proprietary like all the other players at this point, but they have the established customer support and field reliability numbers to justify it if only against those other current players. If they can internally finish the acquisition and get back to 1 code base across the product line in the next year or so they should have a bright long term.

Good point!
So I went off in search of some reasonably hard data. I like SEC reports, since people can actually go to jail if they get too creative with the numbers, unlike, say, Gartner Group analysts, who might get a bonus instead.

Isilon claims #1
In the prospectus for their IPO (initial public offering) Isilon stepped up to the plate:

We believe we are the leading provider of clustered storage systems for digital content, based on customer adoption, breadth of product offerings and technology capabilities.

I think they mean that they sell more dollars worth of cluster storage than anyone else. Or something to that effect.

The dog that didn’t bark
Other than announcing they’ve started shipping their storage cluster, the Data ONTAP GX, which is based on the Spinnaker acquisition they made back in ’03, NetApp has been almost silent about GX. The most telling comment comes from their last 10-Q report, for the quarter ended October 27, 2006:

During the second quarter and first six months of fiscal 2007, our revenue grew year over year and our products have gained market acceptance. We continued to gain momentum through new product introductions in our high-end, midrange, NearStore Virtual Tape Library (“VTL”), and Decru®products and broadened distribution capabilities. We maintained our leadership position in the iSCSI and network-attached storage markets and showed growth in the Fibre Channel SAN market. Revenue growth occurred across all major geographies. . . .

Notice what’s missing? No Data ONTAP GX you say? Precisely.

If GX growing faster than the rest of the company, even from a small base, one would expect at least a nod. Perhaps the next 10-Q will contain that nod.

One number NetApp should fix:

According to International Data Corporation’s (IDC’s) Worldwide Disk Storage Systems 2006-2010 Forecast and Analysis, May 2006, IDC predicts that the average dollar per petabyte (PB) will drop from $8.53/PB in 2006 to $1.85/PB in 2010.

I rather doubt IDC said any such thing. Or I’ve been paying way too much for storage lately.

The StorageMojo take
According to their prospectus and quarterly report, Isilon sold about $62 million worth of storage clusters. From a standing start in mid-year I’d be very surprised if NetApp surpassed that.

I’ve introduced many products. Follow-on products are one thing. New architectures quite another. For one thing, smart sales people don’t want to be the first in the office to sell it, because they know they will catch all the arrows. So they wait for Mikey, the naive kid, to sell one and see how it pans out.

So what about PolyServe, the Oregon software company who has deals with HP, EqualLogic, and DataDirect Networks? The total sales of HW and SW using PolyServe might be bigger. Yet that revenue doesn’t flow to a single vendor as it does with Isilon.

It may be a TKO, but I give Isilon the nod. Number one storage cluster vendor in 2006. Congratulations!

Comments, disputation, or what have you welcome, as always. Comments moderated to protect the innocent. Or the guilty.

{ 14 comments… read them below or add one }

Richard February 10, 2007 at 5:18 am

From ” We believe we are the leading provider of clustered storage systems for digital content” … the key here is *digital content* i.e mostly sequential and read-only data …. mind you we are yet to see their sequential performance figures … and some comment on their IOP ?

Kevin Closson February 10, 2007 at 10:41 am

Richard,

Actually, sequential is what they do best. Look at their two biggest customers. It takes a lot of partitioning.

Robin Harris February 10, 2007 at 11:27 am

Richard,

In their prospectus they have a couple of claims that, on their face, seem contradictory:

Our OneFS operating system software can currently combine up to 96 separate nodes and up to 1,000 terabytes, or one petabyte, of storage capacity in a single cluster, and can deliver total data throughput of 10 gigabytes per second from a single file system and single pool of storage, providing linear scalability in both storage capacity and performance.

And then this, later:

The simplicity, ease of use and automation of our Isilon IQ storage systems have enabled individual customers to scale deployments from a few terabytes to more than 2,000 terabytes without any additional investment in IT staff.

Is there a 96 node limit or not? 96 IQ6000 nodes at 6TB each is 576 TB. Do the EX6000 expansion units count as nodes? I’d assume not.

Personally, I don’t believe they scale cost-effectively much beyond 100 TB due to the overhead of a fully distributed file system. But that is just my gut talking. My head doesn’t know. Thank you Steve Colbert.

Robin

Richard February 10, 2007 at 7:44 pm

Kevin,
I suggest that a “sequential & read-only traffic” is the *only* traffic they can cope with.

Robin,

Are we looking at … say… 200 users at 50 MB per 1Gbit Ethernet channel, accessing a “single file system and single pool of storage” i.e. a single file system spread over 100 nodes ?

A 100 node system will need a large dual fat-tree IB-switch topology i.e a large number of IB switches … additional latency / performance & management problems. How realistic is this?

These issues have been thoroughly researched and proven in computing clusters. There is a very good paper on …

http://www.appro.com/whitepaper/hpc_whitepaper_building%20a%20supercomputing%20cluster.asp … see page 20 for detail.

Perhaps Isilon could supply a block diagram for a 100 node system … with comments on performance, system management and cost… compare with similar switched FC SAN backend solutions from PolyServe, Ibrix, DataPlow, Sistina/RedHat GFS… etc.

Robin Harris February 10, 2007 at 8:56 pm

Richard,

AFAIK, any Isilon node can access any data, so as you add nodes you are also adding bandwidth. So 100 nodes would theoretically offer 100 x 1 Gb = 100 Gb – or about the 10GB Isilon claimed.

IB switches are quite cheap and very low latency compared to ethernet switches. Five years ago Mellanox, which IMHO is the leader in Infiniband technology (I worked with them at YottaYotta – great group) had an 18 port 4x chip. Switch latency was less than 100 nanoseconds. Which makes fibre channel look like a horse and buggy. So you could cascade multiple IB switches, create a large fabric, and still crush FC switches, while enet switches would still be handshaking and getting ready to work.

So, IMHO, IB-based are very realistic and why I think Isilon made a smart decision to offer them at the same price. Where they were weak several years ago, and I haven’t kept up on them lately, is the management software. If ethernet assumes an unreliable network, and builds in overhead to handle all the eventualities, IB assumes a reliable network and a relatively stable node population, so they don’t need all that overhead. So I am less certain about the current state of IB management. Maybe a clued-in reader will enlighten us.

Personally, I don’t expect Isilon to provide squat as far as technical info about their system. Makes it harder to shoot at them. Their resellers have the info and share it on an as-needed basis, but thats about it. I wouldn’t dignify their competition with a mention unless I had to, and they don’t. While I do think their marketing could use a big shot of adrenaline and imagination, I don’t fault them for not getting into pissing contests with PolyServe, et.al.

Robin

Kevin Closson February 11, 2007 at 11:11 am

YottaYotta…I like those guys. I tested Oracle Real Application Clusters on PolyServe with their distributed block server stuff. It was very cool and transparent. I had a 3-legged RAC setup with two servers co-located and the third node 2500km remote. Lot’s of good guys there…Brian, Greg… I wish them well.

Chris February 12, 2007 at 5:06 am

Hmm… go to now.netapp.com (customer login required) and look into the “Release Metrics” page, which shows the number of systems running a particular Netapp software release: there are currently >450 filers running 10.x (=GX) out there. Every node is either a FAS 3050 or a FAS 6070 and probably has at least a couple of TBs attached. Look into the price list. Do the math.

This release metrics stuff is really interesting: there are (Ontap “classic”) releases which are running on 900+ systems with an average (!) uptime of 627 days. IMHO using Netapp means buying reliability. Storage clusters for video/movie processing have lower reliability requirements as you can always re-calculate stuff if something breaks.

Kevin Stay February 12, 2007 at 8:13 am

Miffed? Naw…. I enjoy storagemojo too much for that 😉

However, I do agree with the previous poster, Chris, that there are indeed a fair number of GX boxes in the field. If so, why no fanfare from Netapp? The most simple explanation seems to me to also be the most likely; you do not eat your own children.

The current ‘mainstream’ OS from Netapp is in production in a lot of shops and runs very reliably. For the foreseeable future it will be what pays the bills. GX is the future and it is nice to have it out there for those who want or need it. IMHO Netapp is in a good position currently (certainly compared to the other established players) and they have time to fold the two code bases together.

It is also nice to see folks mentioning Polyserve.

Kevin Closson February 12, 2007 at 8:57 am

NetApp gear is very, very reliable.

Kevin Stay February 12, 2007 at 9:14 am

Oy, the grammar these days…

Clearly what I meant to write was, “The most simple explanation seems to me also to be the most likely;” (Evidently split infinitives are one of my specialities…)

Alan Commike February 14, 2007 at 11:48 am

What’s confusing to me is that it looks like they’re gearing for the HPC market, especially if streaming reads is their sweat spot. So why only the GigE front-end? There’s nothing in their marketing literature that talks about plugging directly into a HPC Infiniband Fabric that all the nodes are sitting on.

Robin Harris February 16, 2007 at 9:33 am

Alan,

Actually, I don’t believe Isilon is going for the HPC market at all. A wise move, IMHO.

HPC has been the graveyard of more companies than its small size would suggest. There are three systemic problems with the HPC market:

  • Many smart people with very specific problems
  • Funding, much of it government, that can gyrate wildly from year to year
  • Too much fun: like drag racing and NASCAR rolled into one!

The result of these factors is that you get a lot of home-rolled solutions that are tweaked to the nth degree for a particular problem, so there’s no economy of scale. You are essentially running on non-recurring engineering expense money, which doesn’t make a business. And once people get into HPC, they never want to leave, so there is tons of absurdly over-qualified expertise – like PhDs in Physics and Math – who are just waiting for their next chance to tweak whatever someone else has just developed.

Instead, it looks to me that Isilon is going after the commercial market. Sure, most of their business today is traditional large-file media – video-on-demand, photo sites, medical imaging, geophysical – but look at the language on their website: enterprise storage for all your needs!

Make no mistake: Isilon is aiming high, and like other disruptive technologies, I think people will be very surprised to see how far they able to take their platform. Dual-redundant arrays are going to start looking so 20th century in another couple of years.

Robin

Kevin Closson August 7, 2007 at 6:51 pm

Anyone still watching Isilon?

Bill May 1, 2008 at 9:26 am

I know this thread is basically over. But. Just in case: I’m looking for anyone who has used Polyserve + Mac OS X. We are testing that 10.4 can’t connect to our server via Polyserve, but 10.5 can do it. If you suspect there’s very little if any information, anywhere, about this, you are correct.

Leave a Comment

{ 2 trackbacks }

Previous post:

Next post: