James Hamilton, Amazon architect and a very smart guy, recently blogged about private clouds. In Private Clouds Are Not The Future he argues that economies of scale make public clouds much more efficient than private clouds.

I think we agree that several effects make web scale public clouds more efficient:

  • Higher quality services. Large clouds can economically employ experts to design and optimize their services and infrastructure. Security and server/storage design are two areas where deep expertise can provide more reliable and efficient service.
  • Utilization. Power systems and power cost are optimized when data centers are run at 100% utilization. As utilization rises across the board so does the capital efficiency, i.e. work per invested dollar.
  • Cost. Large-scale investments create their own lower-cost dynamic. Public cloud providers save money on infrastructure acquisition through volume buys. In addition, their volume enables them to acquire optimized components, such as high-efficiency power supplies or custom cost-reduced motherboards, that offer little economic advantage to small volume buyers.
  • Portfolio advantages. With a mix of customers and jobs web-scale clouds have a more stable aggregate load. Some customers are growing, some are shrinking, but the net demand becomes more stable with size. This, in turn, enables public cloud managers to drive utilization higher with less risk of pegging the system.

With all these advantages it is obvious that private clouds are not the future. Or is it?

It isn’t all about the Benjamins
Economics is not the driver many assume. Individuals and companies often select less economic choices. Some people buy cars that cost $200,000 and get 12 miles to the gallon. Some companies buy $6/GB storage and then utilize just 1/3rd of that costly capacity.

Often perceived benefits are not well measured in dollars. Convenience, availability, consistency and control often relate to emotional needs and wants that are rarely quantified or questioned.

But we don’t have to invoke those to understand why private clouds will be part of the computing landscape. Just a quick look at one of the large Internet data centers will tell us what we need to know.

Show me the power
All the advantages of public clouds have analogs in the world of power generation and distribution. Power generation is cheapest when centralized and large-scale distribution systems move power at the lowest cost per watt.

Electrical power generation and distribution is over 125 years old. The technology is well understood, the industry is mature, and a massive infrastructure — including mile-long coal-hauling trains — supports production and distribution.

And yet, Google’s massive Dalles, Oregon data centers, built next to a substation a few miles from the nation’s largest hydropower system – one of the world’s most reliable power sources – flanks each data center with generators. I expect Amazon does the same.

Clearly, access to data is at least as important as access to power or why would data centers spend the money on uninterruptible power supplies?

Despite the maturity of the power industry people realize it cannot be relied upon 100%. Therefore they maintain their own power storage, generation and distribution systems.

Is the Internet that different?

We cannot rely 100% on Internet access to our data. If the application is important enough, as judged by often subjective human criteria, we will keep our data as close as Google keeps its generators.

Even if it isn’t the most economic choice.

The StorageMojo take
My thanks to James Hamilton and his post for a lucid justification for an all cloud IT infrastructure future. He helped me see why that isn’t going to happen and for that I thank him.

I’ve grappled with the question of private clouds for the last couple of years. The advantages of web scale systems became more obvious, but the human desire for reliable data access and control has not receded.

Public and private will not displace each other: they will coexist just as public and private power sources coexist today. No doubt public clouds will claim the majority of the market whether measured in dollars or exabytes, but private clouds will remain significant contributors to our data infrastructure for decades, if not centuries, to come.

Courteous comments welcome, of course.