Jeff Darcy has a good post on key data stores, like Amazon’s Dynamo, and how they differ from filesystems and databases. He relates his transition from a filesystem purist to a more flexible perspective.
The thing that really changed my mind about this was an observation in the Dynamo paper: strong consistency reduces availability. I’ve always thought of data availability in terms of data not being lost or stranded on the other side of a failed network connection. The Dynamo insight is that many applications have to do a lot of work within a small acceptable-response-time window, and to make sure that they fit into that window they have to impose deadlines on all sub-operations including data access. If consistency issues make data unavailable within that deadline then they’ve made it unavailable period, with practically the same effect as if the data were unavailable in any other sense.
In short, while there is a class of applications where traditional consistency is important, there is an emerging class where strong consistency isn’t affordable or necessary. Good stuff.
Another point
Many of the features that make up these non-FS/non-DB stores seem to have a lot in common with object storage. In a highly mobile world the whole idea of placing a file in cyberspace by a path name is anachronistic at best: it could be, physically, almost anywhere and is most likely in several places at once.
The StorageMojo take
While the name “object” is problematic for market acceptance, the concept of managing objects in a flat address space – like the web itself – is a better fit for a mobile networked world. There is a major opportunity to move file management infrastructure forward to reflect the world we now live in rather than a 35 year old server environment.
Courteous comments welcome, of course. Thanks to Wes Felter’s Hack the Planet blog for the link to Jeff’s post.
Robin,
In what way is the web ‘flat’? To me, Ethernet MAC addresses on a LAN are flat. Within the LAN I can move the host around and other nodes can still find it. But if I move a file from one web server to another or even from one directory to another on the same web server, its url will change.
We could have flat web addresses if any of the search engines would create a hash code for every page, image and document AND publish it. Then we could store that hash code and later retrieve the page, image or document, no matter where it had been moved, as long as at least one (indexed) copy remained on-line. The web would become content-addressable, with enormous implications.
There would be a lot less need to store stuff from the net. Instead, we would store a url of the form http://www.searchengine.com/hash?76125339765. A lot of our excessive data growth is attributable to downloading and storing data ‘just in case’ we need it later. A study has shown that a large percentage of files are never opened in the first 90 days after their creation.
@Ernst Lopes Cardozo
“We could have flat web addresses if any of the search engines would create a hash code for every page, image and document AND publish it. Then we could store that hash code and later retrieve the page, image or document, no matter where it had been moved, as long as at least one (indexed) copy remained on-line. The web would become content-addressable, with enormous implications.”
But then there is the problem – what exactly is a web page these days? With so much content generated dynamically using both explicit arguments and implicit ones (like your source location, the date and time, not to mention how your local environment renders the output and how it applies all that embedded context-sensitive page logic), then a web page is a mutable thing, constantly changing and difficult to pin down. About all you can say is that the version cached by Google is some sort of snapshot of a particular incarnation, but any hash generated from that is going to be specific to that version.
Of course there are hashed “content addressable” (I’m always uneasy really describing these as such) proprietary stores, but they work precisely because they aren’t open and flexible in the way of the web.
I must admit to using content addressable email and documents – simply I seem either unable or unwilling to spend time organising my file and email systems and rely on search tools to find what I want. Google does a reasonable job of doing that across the Web, but it’s hardly deterministic. Maybe a “find this old page” wherever is was facility would be nice, but If I need a local history of these pages to find it after it has moved between domains, then a hash on its own won’t do as I need some form of contextual reminder.
Bobble bobble blobble,, eehh, eeeh,, aahh …… twitter…. ahhh….
All of our memories are now framed by twitter level detail and left to bounce around in a bobble head doll of Donald Trump.
Does any one remember lotus notes or the work done at PARC (Xerox) on the Bayou project ? Amazon’s Dynamo is a regergitated chumming of really great works done at SRI, CMU, PARC, U of Ill Urbana, Ray Ozzie’s work and dozens of others. You’ll notice in the amazon white paper references to those seminal works missing. I guess those memories are like Amazon’s $475/share stock and the retirement funds that invested in them— gone.. I also liked how the avertisers are shamelessly sucking up to Wener’s blog (which is not Wener’s Vogels’ fault).
We been using highly distributed, very scalable key/value pair technologies for a long time DNS and LDAP are a good examples.
The data consistency in a high latency or disconnect mode environment is not the same as making it unavailable. Some data will have maturation dates associated with it, is that data unavailable, sure it is. Just because its ready doesn’t mean you should have immediately.
The value of data and the value of the most recent copy may be over stated. The value of data avalibility is a very complex issue. In part, its set by the consumer, not the provider; integrity, accuracy and relevance of the data comes into question as well as certification of the sources. Being able to deliver the data within a time frame and other specified metrics gives you a level service. The amazon model is very far from that.
The emerging class of “new” applications have been around for decades.
@ Steve Jones. I think you hit it right on the head. We as a user communtiy do not want to take the time to organize our information. Search is a more desirable action to take, it follows the laws of conservation and it can be more fun. I think a more important issue thei not organizing out info and relying on search technologies is the limits of organization technologies. You can take an email and file in one folder or another, but not both. Search is our cross reference tool, because we don’t have another alternative.
@ Robbin -perceptive drawing a comparison to object storage, especially with the emergence of cloud technologies. Ultimately, on a coarsely granular scale, we end up with addressable content on geographically transparent media. We need to lose the concepts of “geographically static” and “mobility” and remove them from our lexicons. Consumer use models have rapidly transcended those concepts.to personal multicast. The same person accessing the same information on different devices with radically different capabilities transparent to geography. Not multi user collaberation, but device level interoperability.
For example, you have your PDA and your reading your email. You walk in the office, and there is your email read by your pda now sitting next to your keyboard. You type your email and save it, run to a meeting where you continue editing the email on your pda while waiting to your turn.
The value of those capabilities are determned by the consumer. The value of syncronization, whether in uS or days, is still set by the consumer, not the provider. Today the provider is telling you what the cost is. As data.gov and similar private and public initives develop, that form of business will change.
If you would like to see real progress made and change this antiquated storage and data paradigm, hope the SNIA cloud computing proposal get rejected by the cloud communties and room is left open for some true innovation. Its here, but being edged out.
Think of Information as a vector.
Once it is a vector it becomes independent of Operating Systems and File Systems. It does require the minimum working Storage environment.
Moving from the legacy hardware based scalar addressing requires some new perspectives.
Think of all Information as unstructured in ad hoc spaces. The amorphous mass is not without meaning, coherence and cohesion. It does have a life and mind of its own.
The Unit of Information is what is important. Not the Unit of Technology. In the real world it is the ROI of the Unit of Information and the TCO of the Unit of Technology and the games people play with them.
How do you make money from the amorphous mass?
Your choice of method of personal transportation is the Unit of Technology. You are the Unit of Information and the only Unit that can make money (ROI). Your TCO is the method of personal transportation chosen.
What does vector object addressing buy you?
In Robin’s post Configure a 100 TB HD video infrastructure under “Issue 2: Virtualization” he outlines some “product” solutions that bridge the gap between what we can do now and what we want to do tomorrow, if not sooner.
What makes all this possible?
The maturity of virtualization software and the increasing power of hardware (Unit of Technology) processing. These two advances now make it possible
to run enough virtual instances to easily process even complex vector objects to profitable solutions.
When people talk about Cloud Computing they have their own set of definitions in mind. The true definition may be in the mind of the creator. A “Cloud” running enough virtual instances to process my TP (Transaction Processing) requirements may be a much less powerful “Cloud” Unit of Technology than the one processing my video/image/graphics Units of Information.
Then somewhere in all those “Clouds” is the “Cloud” Unit of Technology delivering Standard Shared Services (like the SLA, Service Level Agreement server) which defines the “Rules of the Road” for “How We Do Things Around Here”, one of which would be Storage. There are a host of others.
Information Management becomes the nightmare it is today until the global name space exists and objects plus their metadata are directly addressable.
The Information survival up to now has been to divide and conquer with physical segregation. That will change. Any segregation and all object metadata processing will be done in the virtual mass (Clouds and Clouds of virtual processes) after retrieval from the virtual name space. The rules designed by humans will be applied there.
@ Rob interesting perspective. This week I’m talking to a govt. agency that is looking to place thousands of 1TB disk drives around their facilities. I think Walmart’s name came up as a source for the drives.
The CTO of this group doesn’t believe in the UOT model. They believe in the Unit of Consumption. A significant deviation from traditional business models, where the manufacturers set how services and capabilities are procured, managed and fulfilled.
I agree that all storage space is abstract and amorphous until its purposed. I do disagree that there needs to be a global name space, in fact, I believe there needs to one namespace for each person in the world for each of their roles, consumer, personal accountant, financial advisor, social planner and each of their access devices… The name space, how we identify things, changes according to what we are doing and the capabilities of the tools we use.
There is also a language, cultural and social component that cannot be ignored. Each influencing our “world view” and the way people relate to our environment. All affects how we name “things”. Today, North America has about 500M people. China 1.4 b and India 1.2b. In this group there are thousands of cultures, millions of social groups each will have behaviors in common, but each will and have created their own identities, consume name and organize with different patterns.
A single name space simply will not work.
It’s important to keep in mind the difference between “distributed” in the sense of a local cluster with fast interconnects and “distributed” in the sense of long-haul latencies and failure rates. I’ve done both. They have some similarities, but also some very significant differences; anybody who thinks DNS and LDAP are comparable to Dynamo just doesn’t get it and shouldn’t be holding forth on the topic. Yes, Dynamo and the others owe a debt to a lot of the previous work mentioned. It’s not a direct debt, though, because the technology has advanced not one but two or three generations past some of those works. It’s common in papers to acknowledge only direct debts; it’s neither necessary nor useful, for example, for every computing paper to acknowledge Turing and Babbage so it’s up to the reader to understand the underlying tree of knowledge.
On the concept of availability vs. consistency, BTW, I highly recommend studying Eric Brewer’s CAP Theorem. I neglected to cite it in my article (mea culpa) but it’s pretty essential reading for anyone who wants to understand why so many modern architectures are moving away from the strongly-consistent hierarchical-namespace model. Understanding those reasons, even if one ultimately disagrees with them, is a fruitful exercise.
Now that I’m not on a pda, I can properly post…
In recent years there has been a notable lack of innovation in computer technologies. I don’t mean in the areas of semiconductor or magnetic physics, but in systems architecture and design. Since media adoption of cloud computing, one of those random events that changes the course of history, there has been a rapid re-branding of products and technologies as cloud based products. Dynamo is one of those technologies.
One could argue the point that there is/was incremental improvements over other solutions.. and historically significant innovation. To draw and analogy to storage, Dynamo is not any different than one manufacturer’s point solution, raid controller vs another manufacturer’s. This is an very important distinction that computing community and media tends to ignore. Its like comparing CIFS to NFSv4. There are some application solutions that do gain slight benefits deploying one over the other, both can work equally well for file sharing applications.
Once technology selection becomes a purely emotional, like the Sex Pistols, the industry is in big trouble. Next, we’ll be hearing about how creationism invented computer technologies.
Unlike others, I’ve never been a file system purist or a database purist, My world view of data storage has always been an amalgam of both with storage system placement based on media properties. Sort of where we are starting to move again, after 20years. BTW, I include networking in the media models..
Great find noting the cap model, I was there for Eric Brewer’s presentation in 2000. There are many factors not accounted for in the model, but it was an excellent articulation and starting point with a solid foundation.
Also, today a blog (http://beta.technologyreview.com/computing/22611/) posted Nick McKeown’s opinion on recent trends in the internt computing. Nick is the director of the Clean Slate Design for the Internet project at Stanford. “Until now, the Internet’s solution to mobility has been a hack, retrofitted to an old and ossified architecture. This isn’t going to be good enough for cloud computing–we need more innovation in the architecture.”
It seems I’m not alone.
To paraphrase Ralph Waldo Emerson, a foolish data consistency is the hobgoblin of IT minds. However, there’s a reason for this. We only really know how to produce systems with consistent, deterministic results using mathematically consistent models. Once that fails, or is foregone for scale, performance, complexity or other reasons we are driven closer to the real world of uncertainty and non-deterministic behaviour. Once non-linearity is introduced into your systems, then (quite literally) chaos theory reigns.
For a long time I’ve suspected that human intelligence and perception is inherently dependent on inconsistent data and logic models and the ability to hold approximations and contradictions in one mind. Now whether we are going to be prepared to accept increasing levels of inconsistent and unpredictable behaviour be deliberately designing systems that are tolerant of contradictions, ambiguities and the like, yet continue to try and work, I don’t know. Perhaps the days when computers will need the equivalent of a psychiatrist or therapist to sort them out may not be too far away.