I got a note from David Flynn, co-founder and CTO of Fusion-io (disclosure: I’ve done work for them) in response to The new storage pyramid. He makes several points about the nature of the array model that I wish I’d made.
Well worth the read.
David Flynn’s note:
Geat analysis Robin.
And, great comments.
My $.02 ….
I think it’s not just about the proprietary nature, the somewhat better performance and features, and the high markups that differentiates “storage arrays†from “clustered storageâ€.
It’s actually more to do with the vertically integrated nature of the business model of the companies in the array building business. This leads to proprietary architectures, higher margins and, true, somewhat better performance and features.
Let me explain through an analogy…
We used to get graphics workstations from SGI, Apollo, and other vertically integrated vendors, who sold everything end-to-end, down to the monitors and their own proprietary OS’s. These guys commanded HUGE margins – partly to reward their risky investment in solving a worthy, complex problem.
Similarly, the military (and other few others who could afford a million dollar price-tag) used to get flight simulators from Evans&Sutherlands who were also vertically integrated and insanely expensive. You even had niche vendors like Intergraph doing 3D graphics information systems who could justify their own proprietary architectures.
At least for a while.
They were all doing 3D graphics in one form or another. And, now, they are all GONE – thanks to the emergence of a component, the 3D graphics card.
With enough capability to be applicable across all of these different verticals, the 3D graphics accelerator has now shattered the benefit of running a vertically integrated business.
Today, there are myriads of “integrators†who make graphics workstations, flight simulators, GIS systems, etc. at very low margin by comparison. And, they do it by pulling together off-the-shelf components – all commoditized down to the software that provides even the high-value features.
They might have been inferior to the proprietary solutions at first, but not anymore.
Now, what happens when you introduce to the storage industry a component that commoditizes and trivializes the linch-pin reason for expensive proprietary disk arrays, namely the caching tier – using NAND flash.
Once anyone can easily get the performance across any use case (OLTP, OLAP, Data Warehousing, BI, VOD, content caching, etc. etc.) you no longer need vertical specific, highly tuned, proprietary solutions from vertically integrated companies.
Every capability that doesn’t migrate into the component itself becomes nothing but commoditized software to be layered on top by any number of interchangeable integrators. Things like replication, disaster recover, backup, dedup, and so on just become commoditized software that can run anywhere.
This is a classic Adam Smithian market evolution. What used to be a single, vertically integrated provider becomes a layered market where some people build the components, others integrate them (with some bit of value add), and you go to having many players competing on many levels.
And prices go down.
But, thankfully, (for those of us in the business of creating this componentized building-block) volume, productivity, and efficiencies all go up.
So, actually everyone wins. Including society as a whole.
Well, almost everyone wins. Everyone, that is, except for the proprietary array vendors who get caught by the innovators dilemma and a business model that used to be the correct one, but no longer is.
This generally makes them the slowest to simplify their proprietary infrastructures around the commoditized component – to help justify their investment into their heroic proprietary solutions.
In an effort to protect their margins, they endeavor to make things seem as complicated as possible. They do this, say, by preferring that NAND be forced to pretend to be an HDD and be put into HDD drive bays behind HDD protocols, where it has little ability to simplify things or get much additional performance.
They are the last to come out and say it can be simplified. Instead they’ll tell you you must have features X, Y, Z. And, see, those aren’t as good as with our proven architecture.
Let’s take high availability as an example. They aren’t going to tell you that a “shared nothing†strategy – where two separate RDBMS servers with terabytes of direct attached NAND inside of each use off-the-shelf log-shipping for asynchronous replication (or query replication to do it synchronously) to get fault tolerance.
No, they aren’t going to tell you that it’s actually simpler, more cost effective, and, here’s the real kicker… more fault tolerant to share nothing, than to use shared storage – no matter how fault tolerant they claim their monolithic storage array is, it’s still shared.
I’m not saying this market transformation is going to happen by tomorrow. But, given the geometric growth of the performance gap between processors and storage, and the geometric decline in cost of NAND flash – leading to a “Moore’s Law Squared†effect in the benefit to cost ratio – it is going to happen faster than people would think. Even considering the “stodgy†nature of storage folks who are in the business of obsessively caring for precious bits.
It doesn’t hurt that in this global recession companies are looking for ways to reduce costs while still needing to grow throughput. So, there’s more of a willingness to look at different, innovative ways to skin the cat.
I agree with you Robin. It will be a fait accompli by 2015.
David Flynn
CTO, Fusion-io
The StorageMojo take
Technology diffusion is a complex mashup of secular trends, technology development, individual creativity and happenstance. But the current direction of the high-end storage market points to the greatest change we’ve seen since the early 90’s and the advent of arrays.
The “Moore’s Law Squared” effect is particularly intriguing. Humans are terrible at estimating the impact of power functions, so this one is likely to be even more surprising than we dream.
Courteous comments welcome, of course.
Shared-nothing works well for some environments, but not others. Most storage folks aren’t going to tell you that shared-nothing is simpler, more cost effective, or more fault tolerant because, for the deployments and applications they care about, it’s not true.
Why does Fusion now have dual high-speed network ports on their SSD’s if they don’t plan on sharing them over a highly available network?
fait accompli
Robin asked that I expand a bit on my last point. Below is a slightly edited version of what I sent via email.
===
Well, for example, a lot of people use filesystems instead of RDBMSes. Many of the biggest systems in the world, driving the highest I/O rates, are machines at the national labs that never saw an RDBMS. Flynn’s recommendation to use log or query replication is completely inoperative in such situations, and where are the shared-nothing parallel filesystems? Panasas or Ibrix might claim to fit this bill, but (a) they’re rather expensive offerings competing with free ones, (b) they’re not dominant players in these markets, and (c) they have their own problems. What problems? Well, #1 that I mentioned in my own blog entry on the subject before you even posted Flynn’s advertorial is that once you’re doing that kind of thing you just gave up the in-host flash’s supposed advantage of nearness to the CPU crunching the data. If I, on a client/compute node, have to access storage on some server node through some sort of external interconnect, how does it benefit me any more for that storage to be flash? Disk offers more capacity per dollar, RAM more performance per dollar, and if we’re already distributing across servers because those are our unit of failure and recovery then the volatility of RAM ceased to be a serious disadvantage. Add in load on the interconnect (more for redundant copies, and even more for the same kind of rebuild that’s supposedly so awful when done internally on a disk array but is typically even more disruptive in a shared-nothing environment) and you’re pretty much back to treating that flash as a disk. Maybe it speaks Lustre or pNFS object-based protocol instead of block protocol, but what you will have built looks an awful lot like disk to the rest of the system.
===
P.S. Pounce, one way or another, you can be sure they’re doing it to satisfy customer demand. They might think customers are misguided e.g. in preferring shared storage over shared nothing, but customers are after all customers. They will build their systems according to how their architects, not any vendor’s, feel is best, and if a vendor’s product doesn’t fit well into that model then that’s an opportunity lost. The particular demand in this case is likely to be availability. I suspect that many customers who actually care about availability would like this fast storage to be replicated, either transparently or explicitly, and having an external port makes that more efficient than if the data had to go all the way up to a software agent on the remote/secondary system and back down to its card.
Hi Robin,
I have been a silent reader of your blog and insights for quite some time. You have opened up a conversation on a topic that I feel is important. At Luminex we have found that the Storage Pyramid itself has outlived its usefulness. We now promote the Storage World which can be found and discussed at http://www.luminex.com/solutions/default.htm
The Storage World takes a customer view of what job is trying to be done that uses storage – in recognition of the insights detailed in the Innovator’s Dilemma as David Flynn mentions. The four primary jobs, or quadrants of the Storage World, are Primary Storage, Backup and DR, Archiving and Compliance, and Data Sharing. Using a storage technology may make perfect sense with respect to one quadrant, but not another. At any cost, flash isn’t better for off-site Backups than tape, just like tape isn’t a naturally great storage choice for Primary copies of production data. And the concepts perpetuated by the Storage Pyramid that data should migrate between storage media over time only works within a given storage quadrant, if at all, and don’t adequately address the true nature of customer storage concerns.
I’m interested in your comments and perspective.
Art Tolsma
CEO
LUMINEX
Nice post. Thank you for the info. Keep it up.