Why is enterprise infrastructure so costly and inflexible while warehouse-scale computing is cost-effective and flexible? Is it:
a) Enterprise infrastructure is too capital intensive?
b) Warehouse-scale people are smarter?
c) High-scale systems can’t be reduced to enterprise-scale cost effectively?
Lucky for you, this is an open-blog exam. Read on.
People, machines and scale
Greg Ferro wrote a provocative piece on his blog Ethereal Mind titled Human Infrastructure Poverty & Over-Capitalisation In The Enterprise. He argues that enterprises have been conned into spending too much on product and too little on people.
The purchase promise offered by existing IT vendors is that increasing Capex will reduce the cost of ownership through faster performance, better features, better software and usability. Yet the public cloud uses conventionally ‘over staffed’ IT teams creating automation and orchestration that reduces capital spend up to 70%. Modest investments in human infrastructure instead of physical or software has built an entire product category that is growing in excess of 40% per annum.
A fair critique. But why are enterprises unable to follow the IaaS industry’s lead?
Money, money and money
Data center budgets are 60-70% salaries, while Google-scale computing cost is 3-5% salaries. Yet Google, Amazon or Microsoft can afford a lot more PhDs than any enterprise because OpEx isn’t the crucial metric.
Why? Scale.
OpEx is the wrong measure for the big players. The real investment is in R&D budgets, not OpEx.
One data point: Google spent $7.95 billion on R&D in 2013; Goldman Sachs spent less than 1/10th that – $776 million – on communications and technology, most of which was OpEx, not R&D. How many IT shops even have an R&D budget?
Goldman’s IT folks are rumored to be among the best paid and brightest in the industry. But they don’t build their own infrastructure either.
The StorageMojo take
A number of companies make Google-style, scale-out infrastructure. Buy software only or integrated hardware and software.
But adoption is a problem. In the short run these are new systems that incur new system startup costs. But the savings are – mostly – in the long run.
CIOs have been playing the OpEx savings card for so long that CFOs don’t believe them. Given IT’s dismal record for new projects, who can blame them?
Mr. Ferro contends that adding intelligence to IT rather than trusting vendor R&D is a Good Thing. He’s no doubt correct.
But how will its ROI be validated? Where will this intelligence come from? Most vital: how will CFOs and CEOs be convinced?
Because of their scale Amazon, Google, Azure and Facebook can afford to put PhDs on problems that no enterprise would see a return for. The right answer for enterprises is to implement resilient scale-out architectures from committed vendors, rather than attempt to reinvent a warehouse-sized wheel. Focus scarce resources on evaluating best-of-breed solutions and the cultural changes required to best implement them.
Courteous comments welcome, of course. Where do you think the biggest payback from greater IT intelligence would come from?
one observation: the big scale guys are delivering opex as product, where those in charge of opex for smaller scale are not often exposed to making business wide decisions about their employment / deployment of abilities. I don’t think it’s necessarily that shops can;t hire umpteen PhDs to develop infrastructure, I usually meet plenty of intelligence any place anything sophisticated is deployed. Simply those brains do not have a mandate to develop their own “product”. That is very much the R&D gap, but I think – cursorily only, mind you – that one can create incentives that greatly encourage that opex to human capital ratio improve. Okay, i strongly dislike half hearted attempt to “productize” internal functions, because they tend towards being cost plus utility / usage accounting, and nothing more. But, just how much “virgin territory” is there to play with, if you are a even large regular company, buying from the more recognizable names? As opposed to being able to ODM switches or servers to spec? I don’t think it’s a fair comparison. Not to mention big cloud people can tune so much to utilize features or capabilities they have. I don’t see that going down well with who is LOB in charge of a OLAP system, for example.
I’m unable (or insufficiently caffienated) to get into speculating on thoughts here how gains can be made, but one has to start with a much freer culture than most companies believe / have the confidence that they can manage.
Robin,
I worked on some of those warehouse scale designs, and believe the answer would be to combine commodity based high density storage hardware, with software defined storage (SDS) as it matures to the enterprise standards, probably in 1-2 years.
can read the latest article in my Blog (http://sdsblog.com/2014/10/13/lowest-cost-storage-for-bigdata-and-cloud/), which demonstrates how you can achieve a redundant $0.14/GB, without requiring PHds, those are imaginary costs in the Enterprise world.
this trend is re-enforced by the latest moves of HGST, Seagate, .. who try to move higher in the stack and provide disk enclosures and optionally some software at much lower margins, in addition large enterprises can source the same hardware used in web directly from Quanta and alike for about the same price.
Opex can benefit from this trend as well, much of the Opex is directly related to Capex (e.g. backups, snapshots, over provisioning, migration costs), DR and Backup can leverage cloud economy, and by having higher density systems you would need less IT overhead, the missing link is good SDS software stacks which can match EMC and NetApp, and potentially further simplify/automate the provisioning and life-cycle management overhead. With all the VC $ going into SDS i hope we will see some who manage to hit the mark in the coming 1-2 years.
I know a bunch of Enterprise guys already exploring those ideas with Ceph for the non mission critical stuff.
Yaron
That’s why most Free Open Source software is not widely adopted on the enterprise, most OSS is what some people call “half baked”, you have to put some thought on making it work to fit your organization’s needs, but most IT people don’t have skills beyond the common vendor branded products to make this happen, yet CTOs keep throwing money at vendors buying the “ready to use” solutions, which usually lack functionality and integration with other vendors, contrary to OSS which is usually open to the industry standards.
Another challenge is political/organizational, Enterprise storage is owned by the storage department, not by the business unit or application guys, they don’t have a huge incentive to settle for lower budget and work harder to make it work, or learn new products/technologies/vendors. They can charge the Biz units imaginary $/GB/Year prices and get away with it.
If you look at the Web/Cloud infrastructure it is all part of a holistic view/system, with a clear goal to reduce the overall $/VM or $/GB or $/TPs while maintaining the user SLA/Experience targets.
So I believe part of the change need to come from collapsing the Enterprise IT structure and avoid the silos, this is also a must in light of trends such as converged infrastructure.
I am very much with Yaron, on this.
There is a problem when someone sells you certain capabilities to a price, a certain metric, versus trying to achieve that metric, the other way around with a production system.
A interesting aspect of cloud utility compute, is when it introduces discipline into systems design, that may not have been there, with e.g. department heads and internal sub CIO fragments politicking for resources…
Some of this is classical value of benchmark debate.
Benchmark “publishing” is something that gets to me, lately more so, but always. I cannot think how a license agreement can within the law restrict publishing benchmarks of real systems. It can only be enforced heavy handedly. So far as I am concerned, the rate of any metric on my system is a number created by my own efforts, and hence is my copyright, to do with as I please, and this is a argument that I and a number of attorneys I have discussed this with, believe to be validatable if there were ever litigation. But, when I have had the opportunity to discuss such esoterica in person, I confess that the idea has been about rather heavy handed power, also: negotiating a license price.
Cloud compute is throwing out some interesting challenges to monolithic silos, not only storage.
re: Ricardo’s thoughts, well I think any sufficiently sophisticated system is going to involve a sufficient amount of work, that “out of box” versus “roll your own” is a minor factor in final cost, and I would start applying Amdhal’s Law across the board, before I freaked at either sticker price or the potential necessity to hire a additional team.
. . .
Sometimes I think the only thing we know, is that storage systems are very highly priced for what they really do. I hazard also, that at any price, there is no product that takes away even a sufficient amount of pain, and there is absolutely no linear correlation whatsoever with price increase and ease of facility deployment.
It is a little harsh, to say that just as I advocate the need to have a distinct job function of “storage expert”, there ought to be no need, and if it is “storage experts” who first identify the “holy grail” of superlative tools or equipment, we must surely question the turkey as to their voting for Christmas.
With true object storage, blobs and any data, structured or otherwise, is a genuine devops issue. I might say it is higher level than that, because above devops is the processing architecture and business logic.
What I see as the boundary of all innovation and economy in this field, is the installed base of EMC style sales bods, and expense account steaks. I’ve been taken aback by the number of “stealth” companies, coming out, recently. But sometimes I wonder if the only way to disrupt is to risk selling enterprise kit at SOHO prices. Even SOHO prices are all fat, IMO. Companies like Drobo live off the uninformed, if I am pessimistic. A great sales system or channel is such a wonderful buffer/buffet, and a enormous amount of inertia to overcome. Selling anything technical to do with storage is plain tough, so who would do that for a $10K box? There is potential for boom, and real bust. Hence, I believe the number of new entrants.
I hope that was not too grumpy.. the thing that sucks, with all high texch is just getting the ability to buy it. Not money and able to afford, but getting attention even when all you want to do is sign a PO **at list price**, **signt unseen** **because that is cheaper** than mucking about with the whole process – something I am experiencing again, but it used to be a weekly problem I had to solve.. but now I am tilting at industry wide bugaboos.
(minor rant, no I do not like spending thousands to visit a trade show to buttonhole someone to ask if I can simply buy something, but if you happen to be a small and rather private shop, this is what happens)