Note, I’m rushing out the door as one must when the nearest major airport is two and half hours away, and while I wanted to report on Amazon, Chubby and Joyent, I decided that one of my old posts about the rise of EMC – and that worthy institution, the Computer History Museum – was worth another look. My next post will be from Datacenter Ventures in San Jose.

The following was orginally published March 31, 2005

The Computer History Museum holds occasional lectures and panel discussions, and they finally had one on storage. Hosted by the venerable Jim Porter (whose firm, Disk Trends, did the most reliable storage market research in the industry until he folded it a few years ago due to a shrinking customer base), the panelists included Clod Barrera of IBM, Larry Boucher of Alacritech, Mike Gluck of Xiotech, and Randy Katz of UC Berkeley.

Titled “How Computer Storage Became a Modern Business” (translation: “a bloated oligopoly with really dull products”) each of the panelists gave a short presentation on their view of the industry.

Barrera reminded us that IBM had already patented and implemented many of the ideas contained in Patterson & Katz’s RAID paper, which begged for a question I later asked.

Boucher talked about founding Adaptec and the growth of SCSI as an open interconnect.

Gluck talked about marketing disks and later RAID. It was a bit of a yawner and reminded me that the one area CHM is falling down in is computer sales and marketing history. After all, if no one bought the beasts it wouldn’t be much of an industry. But I digress.

Katz talked about the heartbreak of having one’s name third on your most cited paper ever. Frankly though, he looked like he would get tired of the ceaseless adulation, so I think he did the right thing. He also discussed the genesis of the paper as the product of smart but poor guys who wanted a lot of disk space cheap.

Despite the venue in a museum of history, most of the questions were forward looking. It was encouraging to see that even these very bright guys, including at least one bright and rich guy, had very few ideas about where the industry was going. Of course someone from EMC could have told them.

So I asked my question: How did pathetic (back then) EMC kick mighty IBM’s butt during the 1990’s?

The gist of Barrera’s response was that it was a classic case of the innovator’s dilemma, that IBM was so successful with its existing model that it did not see the need to respond to EMC until it was too late.

I gave the answer a C-. I’ve competed with the IBM company in the trenches and strategically, and the answer didn’t compute. IBM is a very tough competitor. They don’t have the resources to do everything well, but rotating rust was a MAJOR profit center for the company. They had the finest storage R&D in the world. They also have a pretty dysfunctional relationship between marketing and engineering, but hey, how hard was it to take Katz’s RAID paper and build one? Other people did who knew a lot less than IBM’s Almaden research group.

The meeting ended shortly, and after speaking to the loveliest PhD storage geek I’ve ever met, I started to leave. A tall, greying gentleman of, I surmised, the engineering persuasion, complimented me on my question, and asked me if I wanted the real story. Of course I did.

With the contained passion and pain of long-festering wounds, he told me. His story was that many people inside Almaden were aware of EMC and the potential of large-scale RAID arrays. Alarms were raised, architectures debated, proposals made. And nothing was approved. The chief villain? Ellen Hancock, GM of IBM storage, who rejected these efforts in favor of the tried and true channel architecture that had served IBM so well for so long. Aided and abetted, of course, by the usual toadies and courtiers of the IBM imperial court.

Is this really so different than Barrera’s answer? Perhaps not. After all, behind the curtain of Christensen’s dispassionate analysis of “disruptive technology” are people, some of whom see what is happening in the real world and some of whom decide to ignore it for what seem like very good reasons.

But on balance I think there was greater than normal cupidity at work. i.e. Ellen really was a dunce. EMC wasn’t targeting some small niche of over-served customers, as in the classic disruptive technology case. EMC was targeting, and winning, IBM’s largest and most important customers. EMC’s marketing messages (in the early years focused on performance), explicitly called out IBM as the competitor, and baldly claimed EMC superiority. EMC was not nibbling on crumbs from IBM’s table, but loudly crashing the party, grabbing the caviar, chugging the champaign and propositioning the wife. Subtle they were not.

And IBM did not respond. For years. And when they finally did (with Shark) it was an under-baked too little, much too late.