In part I some of the reasons the current model of enterprise IT is broken were explored. In the concluding part, I make a radical suggestion about how IT can be fixed – and why Internet Data Centers are better positioned to implement the fix and perhaps force enterprise IT into a long slow spiral of decline.

This section discusses why Service Oriented Architectures (SOA) are a long-term secular trend:

  • Accelerating demand for IT services
  • Commoditization of IT infrastructure
  • Automation of all business processes
  • Efficient utilization of IT investment

Accelerating demand for IT services
Rapidly growing IT resource demand show no signs of slowing and may even accelerate due to the advent of automated web services. The growth results from the mass commoditization of supply and the automation of demand: vastly increased supply at much lower prices combined with increasing economic demand for easily accessible information. Web 2.0 – or whatever you want to call it – automates IT demand well beyond anything we’ve ever seen.

The fundamental economic driver of increased IT demand is that electronic data is cheaper and easier to access, process and store. The ease and low cost enable higher quality services and better decision making, pushing competitors to exploit the technology in order to keep up. The rise of E-commerce has dramatically increased the volume of easily collected business data. Web services will further increase the volume and scope of such data, as well as further pushing business into 7/24 operations since machines never sleep. In addition the continuing decrease in infrastructure and programming costs makes it economic to automate business services that were too costly even five years ago such as

Commoditization of IT infrastructure
Commoditization of key infrastructure is driving and will continue to drive IT technology costs lower than even Moore’s law would suggest. As IT buyers concentrate their spending on fewer architectures (CPU, network, storage, database, OS etc.) these technologies get the benefit of increased volume and lower cost if there is competition. Hardware costs have benefited most, but even software pricing has been affected.

The widespread adoption of Linux has hastened the commoditization of operating systems and brought the industry down to two basic camps: Unix and Windows. Pre-packaged software and sophisticated development tools have commoditized most application and development software. Even twenty years ago many companies wrote their own business applications for inventory, order entry and other critical business processes. Today the focus is on integrating across applications using tools and standards, such as XML, that were unimaginable 10 years ago.

The commoditization of the IT infrastructure means it is feasible for the first time to build SOA infrastructures. By reducing the number of variables (storage, server, network, OS and applications) infrastructures are now ripe for the kind of centralized provisioning and management that telcos have had for decades.

Automation of all business processes (Resiliant infrastructure now required)
The virtuous cycle of lower cost infrastructure enabling the automation of more business processes thereby driving demand for IT products continues unabated. As more business processes are automated, the infrastructure is expected to handle greater variation in demand at the same time that uptime becomes more critical. Most systems even 10 years ago had manual backup processes that could handle IT outages of a few hours to a few days. Today however few companies have enough experienced employees to buffer IT outages. The automated systems have to work or business simply stops until the system gets fixed.

Flat budgets and exploding IT demands will push more enterprise executives to consider an SOA model – whether services are deployed internally or entrusted to an outsourcer. Not all enterprise data is equally sensitive so the LOB’s will begin testing the waters with less critical applications that debug the infrastructure. Internal IT organizations are simply another service provider. Expect LOBs to invite external service providers to take part of the business to provide a competitive benchmark. The growing economic pressure to move to the SOA model redefines the infrastructure challenges whether IT services are provided by a captive internal organization or an external supplier.

Resource Allocation in Service Architectures
Service architectures change the way resources are allocated – moving from the element to the event. Events define service: a server added to a cluster; a logical volume capacity increase; a packet sent or received; a cache hit; or a database access. In fact any event that can be captured could be used to help allocate resources. Tie these events to specific resources and users, through source and destination addresses, system or file names, and the event is placed in its economic context: who generated the event and what was its impact on the infrastructure. This algorithm contrasts sharply with the taxation model that supports enterprise IT today.

The SOA pricing model
Traditionally a LOB purchases the elements required to run a given application (server, storage, database and application software) and IT would operate the system. In service architectures the elements change rapidly: an application might require 6 blades and 100 GB at the end of the quarter and only 2 blades and 50GB a week later. Now the LOB is purchasing the service provided by the infrastructure, not the infrastructure itself. If the resources are going to vary based upon demand the best way to measure and pay for them is through events rather than ownership.

In practice some elements are more fungible (interchangeable) than others. Network bandwidth is fungible: a packet is a packet, and availability is similar across the enterprise. Servers and storage are less so, especially at the higher end of the market. While 1U servers are reasonably similar, an MVS mainframe is quite different from a Solaris Sunfire server, even though both may be running Unix. Rather than attempt to ignore the differences a rational resource allocation system would reflect these differences so the LOBs would have the incentive to maximize the business benefit of the various systems.

The SOA advantage
In The Capacity Illusion one of my commenters noted that the intelligent thing to do is to waste what is cheap in order to preserve what is expensive. Tax-based funding – what enterprise IT has today – fails in two dimensions:

  • True costs are concealed from IT consumers, so expensive shared resources get over-consumed
  • Business benefit are hidden from IT providers, so IT doesn’t know where best to invest in improved services

Taxes vs. a free market
The most obvious alternative to paying for corporate IT with taxes on the LOBs, is a free market, where the LOBs order services from either internal or external IT providers. This is where IDC’s such as Amazon have a key advantage: they already charge for their services in a fairly granular fashion, and are highly competitive to boot.

Markets aren’t perfect for allocating resources, but over the long-haul they’ve proven to be the best mechanism we have. The big advantage of SOA’s that they are well suited for event-based pricing and enterprise IT isn’t. This goes well beyond today’s clumsy chargebacks that peanut-butter costs across a few areas, such as storage or network accesss. Pricing needs to be as granular as possible so that high-value services cost more and low-value services cost less.

Enterprise IT, if it saw the need, could turn itself into a price-based SOA. Current cell-phone rating software, which routinely handles millions of events with thousands of pricing models could be adapted to handle enterprise SOA pricing. The technology exists to turn enterprise IT into a bazaar instead of a cathedral – to steal a phrase from Eric Raymond. A bazaar where LOBs finally have a reason to take responsibility for their IT usage and where IT gets unambigous feedback on what is important, measured in dollars.

Does there need to be a part III? Comments welcome. Moderation is turned on to control comment spam, and registration is turned off so you can comment faster.