A new storage market is being born. Will it survive?

As infrastructure continues to adjust to a data-centric world, the ability to manage data – not just storage – is poised to become a must-have capability. Traditionally, of course, data management has meant databases. But file data – confusingly called unstructured data – is by far the predominant business data type today.

Reading the tea leaves
The recent launch of DataGravity – who perform deep file inspection in their array – and a discussion with Quaddra Software’s John Howarth and Marc Farley suggests a mini-wave of activity in this area.

This is not a new idea. I talked to a startup a decade ago that proposed to do the same thing across an entire LAN – and then went nowhere.

ZL Technologies is 15 years old, prospering in the enterprise archive space. Their web site says:

ZL Technologies’ Unified Archive® utilizes a unique, unified architecture that breaks down data siloes in favor of one robust, centralized repository for managing all enterprise unstructured data and performing records management, eDiscovery, and compliance functions.

The thread
All three companies handle both metadata and content. Users can sort based on types of files – PDFs, MP3s – as well as content – social security or credit card numbers – in those files.

The use cases are similar as well. For legal discovery, DataGravity and ZL handle it, but Quaddra is looking to empower others to deliver that service. Chargeback is another common use.

An open question of growing importance is the data generated outside of corporate storage: tweets; Facebook entries; IMs; voicemails; and other types and formats that may not exist today. And you thought email was hard.

The StorageMojo take
What makes business markets take flight? I like the pain theory: when enough people feel the pain AND that pain is high relative to other pains, then people seek relief.

We’re reaching that point with deep file analytics. E-discovery is one driver. Sheer volume of data is another, which in part, drives chargeback.

The bottom line, with commodity and virtualized computes, cheaper networks, security issues and the newly visible costs of storage – thanks, AWS! – the pain of storage, which was always there, is now more visible and higher relative to other pains.

There will be backing and filling over the next 5 years, but deep file analytics is here to stay.

Courteous comments welcome, of course. Agree or disagree as you like – just tell me your reasons.