The elephant’s graveyard is a metaphor for a place where old ideas go to die. For IT it is the flipside of the consumerization of IT.
The seductive glass house
As Sun’s troubles over the last few years have emphasized, enterprise IT is a Venus flytrap market: once you get into it, it can be very difficult to get out. What makes EIT so seductive?
- High margins
- Big budgets
- Low turnover
These conditions are the reverse of the consumer market, where margins are typically half of EIT’s, budgets are small and turnover of new technologies can be quite rapid.
Let’s look at Sun Microsystems
Sun started out as a commodity workstation company. Motorola 68000 processors, Seagate disks, Berkeley Unix, somebody’s ethernet, Sun did little more than design the circuit boards and packaging that everything plugged into. By not investing in manufacturing plants, semiconductor fabs and engineering and testing proprietary products, Sun achieved a sales per employee that was the envy of the industry in the mostly proprietary ’80s.
By the early ’90s though, the PC was starting to nip at Sun’s heels. Much less powerful, but much cheaper, PCs higher volume made them much more attractive to budget-conscious buyers and to market-conscious software developers. Sun began a long climb upmarket.
The upmarket is very pleasant – as long as it lasts
So workstation sales are getting harder, and meanwhile your very best customers are asking for larger and more powerful machines, network servers, to run the software they already know. These larger machines are more profitable and allow your salesforce to concentrate on fewer accounts to generate more revenue and profit. Engineering and marketing find it easy to justify fun new technology since a 10% goodness increase on a $500,000 machine is worth $50,000, while on a $1,000 machine it is worth $100 only if the customer is knowledgeable enough to notice. Which they aren’t, thank you very much.
Life is good. Until the same forces that chased you out of your original market start nipping at your heels again. You can maybe move a little more upmarket, as Sun did with E10000 series in the late ’90s, but eventually you are standing on the top of the mountain. And there is nothing left to climb. Now you have to climb down, a much less pleasant journey than the trip up.
IBM, EMC, NetApp have all been there
IBM faced the same problem in the early ’90’s as the company cratered under John Akers. High margins are extremely addictive and IBM had the shakes, bad. IBM was propped up by the high margins on its mainframes and storage, but mainframe demand was slowing and Amdahl, EMC and other plug-compatible vendors were growing. Lou Gerstner came in, saw that IBM had created a price umbrella under which its competition prospered, and cut prices big time. The umbrella snapped shut and the PCMs started dying off. IBM’s storage group didn’t respond to the RAID revolution fast enough (see Daddy, tell me again how little EMC beat giant IBM . . .) and suffered a major market share loss.
EMC rode the big iron gravy train through the ’90s, until the dot-bomb implosion. Not only did high-end demand shrink, but hundreds of millions of dollars of like-new storage came on the market at liquidation prices. EMC’s growth reversed abruptly. Since then EMC has sought to become a software company AND has moved into the low-end of the storage market throught its Dell channel. I spoke to an EMC engineering manager who’d worked on their first $5,000 product, and he told me that while customers were enthusiastic about the new price point, they were wondering when the $2500 version was coming out. Now that you can buy a 1 TB NAS box for $600, I’m sure the pressure is only growing.
NetApp, which has been growing fast in the enterprise, unveiled last year their $5k entry-price StoreVault product line. I doubt they are selling many entry-level systems, so NetApp is still missing the low-end of the market, which I would guess is the fastest growing part. But at least they are in there pitching.
The StorageMojo take
As I look around the storage landscape today, I see many companies focused on the enterprise, solving problems that only enterprises have. This isn’t a bad thing, and Lord knows enterprise IT can use all the help it can get. Yet I don’t expect those companies to be the NetApps or Veritas’s of tomorrow. The folks who solve Internet Data Center storage problems will do well, as will the folks who grow up out of the consumer media/SOHO storage businesses. Both markets build companies that thrive with low margins and high volume. They’ll be nipping at the big guy’s heels very soon now.
Comments welcome, as always. Moderation turned on to make spammers work a little harder.
So companies that solve the internet data center storage problems. Who does right now? My best bet is on Isilon. They do two things that I think make them as close as we’ll come to solving the problem.
First, what’s the problem. First, storage systems aren’t dense enough. I run out of space before I run out of power. A growing, or even a slightly growing web site needs to cram as much as possible per square foot in a colo. It’s one of the biggest cost as a company. Secondly, scaling. With 10m users uploading, downloading photos for example, or email, I can slaughter any storage device. You can cache only so much but some has to go to the origin server eventually. And I’m storing that data for probably forever. So a NetApp 168T 3050/3070 with 500g drives can probably handle like 5% or 10% of all the hits of 10m users if you cache enough. But if I need another 20T for spring break (for example) I’m gonna have to buy another head. Which requires power, and cooling and administration overhead.
1) They broke the RAID paradigm. No longer do I have to pay for a drive to spin that is only active for parity, or rebuilds, or even a spare. If I buy a EMC systems and shove a clustered nas ahead of it (such as ibrix or onstor or exabyte) I still have raid5 on the backend, with it’s one disk of overhead cost. With Isilon I can use all my spindles that I’m powering up and spinning. Each spindle is working
2) They get scaling. Each node in a cluster does work, and I can add it on the fly. If I need more power for photo servers, I add an accelerator. If I need storage, I add on an EX node to the storage node. I buy and power only what I need for my application.
Both of these features are not found in any other single vendor. That’s what makes Isilon get it to me. They are highly available far beyond any N+1 cluster from NetApp / EMC / HDS / IBM.
I don’t mean to totally gush on them, but I think they are closer then anyone in supporting the Internet Data Center storage problem.
storage is getting cheaper by the minute. Disks offering 1 TB are just around the edge. Cheap controllers SATA like are available and building a box with lots of cheap disks and offering lots of space are in the reach of anyone who has the guts to use a screwdriver.
The only thing that stands in the way of adopting these cheap installations in the enterprise is reliability. But even there developments like ZFS are lowering the bar.
If a company wants to succeed they need to adopt cheap consumer type products in reliable appliances.
Enterprise budgets for overpriced solutions are gone by the wind the recent years, so to eat your part of the cake you should slice thin 😉
I understand that the NetApp StoreVault product was designed by their Bangalore offices and engineers. Storevault sales may be a good indicator of whether their Indian offices understand the American storage marketplace.
As always, this blog has great mojo…
I see mention of the E10000. I’d like to point out that the E10K was not a Sun system. They walked right into a fully functional system that already ran Solaris. The heavy lifting was FPS->Cray->SGI … Floating Point Systems built the CS6400 just a stones throw away from the Sequent Computer Systems campus.
As for the comment about Isilon, I don’t see them as general purpose enough. I submit:
http://kevinclosson.wordpress.com/2006/12/19/wishing-your-competitors-well-isilon-isln-ipo-no-techno-religion-on-this-blog/
and:
http://kevinclosson.wordpress.com/2007/01/03/oracle-database-on-cas-nas-fcp-your-choice-why-not-some-of-each/
Storage seems to have finally reached a major turning point. Different blogs have different analyses of the problem and many different solutions. As with all of us, the analyses and solutions are from within our own experience.
I have often wondered who is ordering these “giant” capacity disks that lack any of the other improvements that are needed? I doubt we will ever know. We will have to incorporate them successfully into our Strategy without knowing who ordered them.
The only Strategy solution I see are the twin concepts of the “Storage Pool” and “Storage Domains”. These sound pretty trivial. They are very powerful. Easy to say, hard to do. Domains are carved out of the Pool and returned to the Pool. The Pool is virtual to all Domains and all Domains are virtual to Information.
Storage serves two masters. The “reason for being”, most important master, is the Company Portfolio Management through a specific Line of Business. The “care and feeding” master is the IT Portfolio Management to provide the associated Services. Conflicts between these two masters can arise when taking a Technology Centric approach to Storage. Using the Storage Pool and Storage Domains, and taking an Information Centric approach to Implementing these, eliminates that conflict.
The Line of Business (LOB) Storage needs presume the IT Portfolio Management Services. The Core Services are Findability (Search-Find_Obtain (SFO)), the User Experience (UX), Information High Availability (IHA), Information Integrity (II), Disaster Recovery (DR), and Business Continuity (BC).
Clustered Storage is a Technology Centric approach to Storage Pools and Domains. Technical purists love it. Easy to say, hard to do.
The Information Centric approach would be to define the Company Portfolio Management Storage needs.in gross terms. This gives the gross Storage Pool needs. Refine these needs into IT Portfolio Management Services terms. Then do a “black box” Design based on the Storage Pool needs and the Domain Strategy. Domains are virtual to Information. Always.
Kevin – I agree with you but for different reasons. We’re not looking for our clustered NAS systems to support production databases yet – we just want a large pool of data for our growing mass of digital content.
When we had Isilon in for discussions they seemed to ignore the fact they were actually using magnetic storage devices to store data. It’s all fine and dandy that they can do N+1 or N+2 node failover etc, but what about disk failures? I never got a clear answer what the cluster-wide performance impact is for rebuilding the filesystem after a disk failure, much less a multi-disk failure which is our real concern for ATA-based systems of this scale.
We don’t really want to frequently backup a cluster that could become multiple PB’s in size over time, so for us ironically fault-tolerance is very important even though we consider this a Tier-4 (archive) storage requirement.
Any idea of how Isilon IQ technology works, especially as mentioned by Blake, they are beyond N+1, so how are they handling redundancy? Looks like this is a promising company from their recent IPO. So if anyone could provide some more technical details as Robin’s wonderful articles on GFS, Boxwood and FAB, that would be great for tecnology lovers.
Robin / John / Blake,
What would truly be welcome is a “Full Disclosure” style Q&A on Isilon’s technology (beyond the PPT-level marketing hype) along the lines of Jon Toigo & NetApp regarding Data ONTAP GX:
http://www.drunkendata.com/?p=622
I’ve yet to see that level of detail come out of Isilon, so I’m skeptical they’ll step up to the plate. OTOH – I’d love to be proven wrong, so maybe this blog plus Jon’s DrunkenData can guilt Isilon into it.
Surely some of their wise PR people can see the value of more publicity in the blogoshpere?
Bvn,
My adimiration of JWT’s ability to get Dave Hitz to write his blog entries is unbounded. Damn, he’s good!
A couple of more serious points:
-Most architecture arguments are BS. IBM always had pretty lame architectures, back in their heyday, but damned if they didn’t know how to make them work. Functionally, a great architecture that doesn’t work is the same as a bad architecture that doesn’t work. You’re SOL either way. Don’t get me wrong – I love learning about architectures. I just don’t find architecture-based arguments conclusive.
-As far as Isilon is concerned, I’m planning to do an analysis of their pricing, to see where they REALLY stand. Pricing reflects a firm’s belief in what they can sell, so pricing analysis – a part of competitive analysis – can reveal all sorts of things about a company’s beliefs and deicsion-making.
I have great admiration for the Isilon guys – they’ve taken their company quite a ways into a market that a lot of VC’s even today don’t believe exists. But I’m looking forward to learning more.
Robert,
The basic answer to your question, is, I think, that data is cooling. As it cools, it needs fewer IOPS, so big disks work.
Think of the “storage pyramid” as a giant vacuum cleaner: data naturally gets sucked up to higher levels as costs decline. And more data gets stored that would have been discarded a few years ago.
The real issue, IMHO, is how will storage architectures adapt to this dynamic? Stay tuned!
Robin
Great thread, learning lots.
Conflict of interest disclosure: I sell Equallogic
Robin,
I agree that most architecture arguments are BS; but only if they are introducing a more complex system. Simpler technical and especially business designs will capture market share. A common technical thread between Isilon and Equallogic are that they are both “frameless”, unlike the traditional head and body we’re used to. To scale capacity you are also scaling network connections and cache. Ironically, the larger the units get, the faster they become. As for underlying transparency, I can’t speak for Isilon but Equallogic seamlessly grows under the same storage pool as you add units. Robert may even be pleasantly surprised at how pooling is handled by Equallogic.
Aside from architecture I will be most interested in the total cost of a fully featured Isilon config (please get every feature included). This has been the ugly side of a la carte storage functionality for years. Much money is to be made when an “Enterprise” vendor gives you a quote with 50 line items on it, some of which don’t make sense or may not even be needed. What’s worse is after you’ve gone through the 6 month budget and procurement process (longer for Govt) your needs have changed and you are now short functionality. Oh well, there’s always next year.
I applaud Equallogic for offering the final solution; bundle each unit with every software feature including any future “new” features forever, period. No licenses, no keys, no hassle. When you want multi-pathing I/O, remote site replication just start using it…it’s included.
Equallogic is my choice but I am now curious about Isilon.
Dale
I became interested in EqualLogic when I read Marc Farley’s blog comments. I posted these as a reply to “Economics of ZFS” at http://storagemojo.com/?p=337.
Marc Farley’s blog comments:
“Wade O’Harrow, one of our SE’s from the southeast was there and completely in his element as a VMware animal. He kept describing our technology as “doing the same thing for storage as VMware does for servers.†I admit it took me awhile to figure it out, but I finally did get it. In an EqualLogic SAN, the individual storage systems are really secondary to the logical storage pools that customers create. Volumes can be moved among the underlying storage hardware resources – more or less independently of what those resources happen to be. You need disk drives for virtual storage just like you need processors for virtual servers, but each disk drive just gets less and less important in the scheme of things.â€
I am thinking that an Isilon and EqualLogic combination would be a very powerful Storage Strategy.
One way to look at this is that the EqualLogic Storage pools are the new “LUN”. An Information Centric LUN instead of a Technology Centric LUN. Most people will want to think of them in a more comprehensive way. Like “one LUN fits all”. That is a mistake.
The “assigned” or “commissioned” Storage Pool consists of Storage Domains that are constructed of one or more EqualLogic Storage pools (Information Centric LUN). These are based on Content and business needs. They are totally Technology independent. There is a basic feature-function set requirement defined for each Storage Domain based on the Technology Centric Lower Metrics. The detail in this is another story.
For this to be Implemented and Managed, the IT shop homework needs to be done so that the Lower Metrics are understood. That is another story.
Dale,
I couldn’t agree more that shipping a solution with everything included *should* be a huge win for customers and, if the messaging is done correctly, EqualLogic. It should shorten the sales cycle, reduce revenue variability, increase customer satisfaction and get the software wrung out sooner and better. I commend EqualLogic for that policy and business model.
Yet I don’t see that business decision as an architecture decision. What’s the difference? Architecture decisions get nerdy guys – and the occasional gal – excited, while business decisions get the green-eyeshade set smiling.
I haven’t looked at EqualLogic’s architecture to see how it might compare to Isilon’s or other cluster-based storage products. It is on my ever-expanding list though.
Robin
I appreciate all the feedback. I confess I was playing devil’s advocate, or reluctant dragon, trying to draw out comments to maybe start a dialog. Technical blogs don’t seem to be a forum for that.
I posted my main position at http://www.drunkendata.com/?p=946.
My post has been brewing for some time as I have noticed the complete lack of comment about SOA (Service Oriented Architecture).
Once SOA starts to arrive even solutions like EqualLogic will be transparent and invisible to the business User. The Technology Centric solution will be even more obscure. The Storage Domains might be “Enabled” in a variety of Technology Centric ways. The ROI/TCO ratio will be the driver. The details will be in the Service Level Agreement (SLA). Then the Storage Pool will supply the closest fit “Enabling Technology” to satisfy the request.
If the “closest fit” is not close enough then a purchase request will be generated through the CRM to the ERP. All relevant personnel will be notified by the CRM. Approvals and acknowledgments will be managed by the CRM before the request is posted to the ERP.
In the interim, if required, the “closest fit” can be deployed. The performance variants will be noted and routed through the CRM for approval.
Once the ERP request is active and a supplier response received, the scheduling of the commissioning of the new “Enabling Technology”, redeployment of the Information (migration or replication) and de-commissioning of the interim “Enabling Technology” will be broadcast through the CRM to relevant personnel. The ERP and CRM will coordinate to ensure the event takes place and relevant personnel are notified of schedule changes.
All this means the IT personnel will have to put on some new hats just to use this system. I wonder what the IT Portfolio Management would look like to do this? Hmmmmm….
Lots of soft-spots here…
That sounds frighteningly complex?
Going from where we are now with Storage and infrastructure Management tools to where we should be, I am sure it looks that way. I have a little advantage.
I have long been a big fan of Information Architecture and the book “Information Architecture for the World Wide Web” by Peter Morville and Louis Rosenfeld published by O’Reilly. In particular I am a bigger fan of Visual Information Architecture. This is the marriage of Information Architecture and Visual Programming. Real Visual Programming.
I worked for a brief time on trying to adapt “Doom” to be a visual tool for Systems Administration. Dennis Chao started this at:
http://www.cs.unm.edu/~dlchao/flake/doom/
The Aftermath is at:
http://www.cs.unm.edu/~dlchao/flake/doom/after.html
The psDooM code is still at:
http://psdoom.sourceforge.net/
Use it at your own risk.
Today I read a posting that brought all those fond memories back at:
“Software Patent of the Week: Visual Programming”
http://www.techliberation.com/archives/041883.php
I have been waiting and wondering when Storage vendors would see the error of their ways and start down this road. The company that does will have a long development time and need a lot of funding but will absolutely dominate the market. Now I am wondering “Why?” Storage vendors avoid this Storage Management tools path.
This type of Visual Storage Management software would open many doors.
It would solve many problems. Each IT shop could “Roll Your Own” Management tools based on their needs.
These Visual Storage Management tools needed a coordinating body to establish standards, resolve conflicts and provide leadership for evolution. I had hoped the SNIA would assume this role. Perhaps they feel they have but with a different spin than the one I thought would be good for Storage.
The proof of concept was done with the SGI OpenVault project at:
http://www.sgi.com/products/software/openvault/
The OpenVault project got the API ball rolling. Anyone could connect and use any tape drive on any system that supported the OpenVault defined API. Any tape vendor can subscribe to it. It is in wide spread use today.
OpenVault is proof of concept for the API.
The API is the required Lower Metric.
Visual Management rides over the API.
One of the fun games in life I got to play was reading the Storage interface and microcode to get Configuration information. After a while all the vendor code begins to show startling similarities. I had my own API and my own StorageOpenVault. It took some doing.
I had developed an Intelligent Programmers Assistant to do all the Lower Metric functions when I ported code. I modified that to be the Intelligent Storage Assistant. Once finished and tested, it did all the work. Even on new boxen. It got real smart.
This is not a plug for me. Most people could have done this better and certainly faster than me.
Sun still just packages somebody else’s stuff