The redoubtable Kevin Closson has a post entitled “Introducing the “Unstructured Data Administrator”. In it he refers to a study put out by the Independent Oracle Users Group called “Managing the Storage Equation: The Converging Roles of Data and Storage Professionals”. Very provocative title, which, sadly, the study doesn’t back up. But Kevin raises some very good questions.

Kevin also points to a Wikipedia article which quotes a Merrill Lynch study saying that 85% of all business data is unstructured. Clicking down the rabbit hole one comes to a website, sponsored by IBM, on something called UIMA (Unstructured Information Management Architecture).

OK, 15% of your data is managed professionals, and the 85% is ???
Several threads here.

  • My reading of the IOUG study is that DBAs have no idea what is happening with unstructured data, which doesn’t bode well for the putative convergence of data and storage professionals
  • If only 15% of business data is in relational databases, and some presumably larger amount is in email, why is such a high percentage of business data kept on the most costly, high-performance storage?
  • IBM’s UIMA working group is co-led by DARPA, which, given its history with TIA, is probably putting the technology to work to spy on US citizens, in addition to its use in the MADCAT program for tactical foreign language document translation.

Another losing battle for IT?
A simple and persuasive narrative of the IT/LOB fight over the last 50 years is that IT loses every time. IT’s high-volume command-and-control mindset meets business unit needs for ad hoc tools, and after much techie mumbo-jumbo, IT is forced to concede that making money is more important than saving money.

If the combination of expensive DBAs, expensive storage and expensive DBMS rises on to the CFO’s radar, changes will occur. It will take some years, but the target will be too fat to ignore. How the dollar split will shake out over time is anyone’s guess. Oracle will be saying “we’ve automated managment and use cheap storage, buy us!”; EMC will say “we can make any bloated pig of a mismanaged database scream, buy us!”; and DBA’s will be saying “we optimize for the real world in real time, buy us!” They’ll all get whacked, and I’d guess storage the most and DBA’s the least.

The StorageMojo take
Thanks, Kevin, for pointing out the growth and importance of unstructured data. It suggests more reasons for the coming hard landing in data storage.

Comments welcome, of course.