Note, I’m rushing out the door as one must when the nearest major airport is two and half hours away, and while I wanted to report on Amazon, Chubby and Joyent, I decided that one of my old posts about the rise of EMC – and that worthy institution, the Computer History Museum – was worth another look. My next post will be from Datacenter Ventures in San Jose.
The following was orginally published March 31, 2005
The Computer History Museum holds occasional lectures and panel discussions, and they finally had one on storage. Hosted by the venerable Jim Porter (whose firm, Disk Trends, did the most reliable storage market research in the industry until he folded it a few years ago due to a shrinking customer base), the panelists included Clod Barrera of IBM, Larry Boucher of Alacritech, Mike Gluck of Xiotech, and Randy Katz of UC Berkeley.
Titled “How Computer Storage Became a Modern Business” (translation: “a bloated oligopoly with really dull products”) each of the panelists gave a short presentation on their view of the industry.
Barrera reminded us that IBM had already patented and implemented many of the ideas contained in Patterson & Katz’s RAID paper, which begged for a question I later asked.
Boucher talked about founding Adaptec and the growth of SCSI as an open interconnect.
Gluck talked about marketing disks and later RAID. It was a bit of a yawner and reminded me that the one area CHM is falling down in is computer sales and marketing history. After all, if no one bought the beasts it wouldn’t be much of an industry. But I digress.
Katz talked about the heartbreak of having one’s name third on your most cited paper ever. Frankly though, he looked like he would get tired of the ceaseless adulation, so I think he did the right thing. He also discussed the genesis of the paper as the product of smart but poor guys who wanted a lot of disk space cheap.
Despite the venue in a museum of history, most of the questions were forward looking. It was encouraging to see that even these very bright guys, including at least one bright and rich guy, had very few ideas about where the industry was going. Of course someone from EMC could have told them.
So I asked my question: How did pathetic (back then) EMC kick mighty IBM’s butt during the 1990’s?
The gist of Barrera’s response was that it was a classic case of the innovator’s dilemma, that IBM was so successful with its existing model that it did not see the need to respond to EMC until it was too late.
I gave the answer a C-. I’ve competed with the IBM company in the trenches and strategically, and the answer didn’t compute. IBM is a very tough competitor. They don’t have the resources to do everything well, but rotating rust was a MAJOR profit center for the company. They had the finest storage R&D in the world. They also have a pretty dysfunctional relationship between marketing and engineering, but hey, how hard was it to take Katz’s RAID paper and build one? Other people did who knew a lot less than IBM’s Almaden research group.
The meeting ended shortly, and after speaking to the loveliest PhD storage geek I’ve ever met, I started to leave. A tall, greying gentleman of, I surmised, the engineering persuasion, complimented me on my question, and asked me if I wanted the real story. Of course I did.
With the contained passion and pain of long-festering wounds, he told me. His story was that many people inside Almaden were aware of EMC and the potential of large-scale RAID arrays. Alarms were raised, architectures debated, proposals made. And nothing was approved. The chief villain? Ellen Hancock, GM of IBM storage, who rejected these efforts in favor of the tried and true channel architecture that had served IBM so well for so long. Aided and abetted, of course, by the usual toadies and courtiers of the IBM imperial court.
Is this really so different than Barrera’s answer? Perhaps not. After all, behind the curtain of Christensen’s dispassionate analysis of “disruptive technology” are people, some of whom see what is happening in the real world and some of whom decide to ignore it for what seem like very good reasons.
But on balance I think there was greater than normal cupidity at work. i.e. Ellen really was a dunce. EMC wasn’t targeting some small niche of over-served customers, as in the classic disruptive technology case. EMC was targeting, and winning, IBM’s largest and most important customers. EMC’s marketing messages (in the early years focused on performance), explicitly called out IBM as the competitor, and baldly claimed EMC superiority. EMC was not nibbling on crumbs from IBM’s table, but loudly crashing the party, grabbing the caviar, chugging the champaign and propositioning the wife. Subtle they were not.
And IBM did not respond. For years. And when they finally did (with Shark) it was an under-baked too little, much too late.
Well.. in those early days they (EMC) were going broke selling add-on memory for IBM machines and some low-end add-on peripherals … so at least they new the IBM add-on business and some of the customers. This helped.
They were not the first to emulate DASD & CKD … and for many years they had only RAID1 (mirrors) and very poor packaging … it took them a long time to do graduate to RAID5, etc.
I guess IBM thought that it was always in a position to do a better job …. and to take the market back …. especially as they also made disk drives.
Its probably not too late, as nothing much has changed in the basic concept of RAID. IBM could do it… but it needs leadership.
I watched EMC as well starting in the early ’90s. The product was pretty pathetic, but their marketing of it was spot on and the sales force very aggressive. They hammered IBM on performance and that resonated with customers.
In addition to the issues you mention, they also had a single-point-of-failure cache as well. Yet they cleverly used all the memory grooming techniques they’d learned selling add-on memory to convince customers that the lack of redundancy was OK. Everybody else was spending time and money building redundant caches – not EMC.
Can IBM reinvigorate their storage business? Absolutely. But not by building more big iron storage arrays – they’ve lost that war. They need to take the same view they do with open source software: view storage as a means to provide long-term service and support rather than a big-ticket item by itself.
Hey, all you IBM’ers that read StorageMojo.com every day. I know how to fix your business – if you’ve got the guts to do something radical.
Yes… and we were all hard at work on coherent, mirrored caching.
I remember SUNs original RAID array … it had a 10-disk mechanical plug-in module (much like what 3PAR is doing now). If a disk failed you had to copy the 10 disks to another array… so that you could replace a single faulty disk. … then copy it all back…. and this was a very good product for SUN.
Their very new 48 disk chassis is based on a single controller front-end … and the disks can not be replaced at run-time. I hope Murphy is on Andy’s side.
I think IBM still have the technical capability, patents, etc… and there is value in ownership rather than reselling undifferentiated … ‘me too’ hardware.
The existing competitive ‘big iron’ is becoming obsolete … and not much new is happening in terms of architecture. It is an opportunity to innovate with new technology. IBM can deal with firmware and software …. they have it… or buy it… plus a great support organization. Your suggestion is good…. and it adds to this.
I think this needs to be addressed by Irving Wladowsky-Berger (chief of IBM strategy), as the rest of the organization seems to be just writing very very good papers.