by Robin Harris on Monday, 29 June, 2009
As the economics of data storage push more and more data onto disks, the energy efficiency of data storage is ever more critical. Storage is anti-entropic, so keeping bits organized requires energy. How can we minimize that energy input?
Data cooling is the major reason disk drives have remained a viable storage strategy for 50 years. The IOPS/MB has dropped steadily for decades, yet disks remain the preferred tool outside of very low latency or high-bandwidth applications.
Looking forward to massive scale-out storage infrastructures the data will get even cooler. Copan’s MAID architecture, which turns disks off when not in use, is a rational extension of the cool data concept.
As data continues to cool we will eventually see millions of disk drives – along with tapes – sitting idle. But even if we have cold archive disks, one of disk’s big advantages over tape is the ease with which data can be spread over multiple drives for data protection.
Not RAID 5
You can’t count on any one hard drive actually restarting after a few months or years of idle time. Nor can you expect that any specific sector will be readable. Cold data requires even more advanced – energy efficient, disaster-tolerant – storage techniques than RAID arrays offer today.
Oh, and they need to be cheap too. Which means RAID arrays won’t get this business. What about open source software?
Erasure coding
In A Performance Evaluation and Examination of Open-Source Erasure Coding Libraries For Storage (pdf) James S. Plank, Jianqiang Luo, Catherine D. Schuman, Lihao Xu, and Zooko Wilcox-O’Hearn examine 5 open source implementations of 5 different erasure codes: Reed-Solomon, Cauchy Reed-Solomon, Even-Odd, Row Diagonal Parity and Minimal Density RAID 6 codes.
Typical storage system with erasure coding – figure from the paper
Several companies – including Cleversafe, NetApp and Panasas – use erasure codes today to ensure higher data availability. What Plank et. al. wanted to know is how well these codes work and what system designers need to know to use them effectively.
The OSS implementations tested are:
- Luby, a C version of CRS.
- Zfec, a highly tuned Reed-Solomon library.
- Jerasure, a GNU LGPL C library that includes RS, CRS and 3 MDR6 among others.
- Cleversafe
released an open source version of their dispersed storage system, from which the authors used just erasure coding parts.
- EVENODD/RDP
, patented codes not available to the public and included for performance comparison.
Most important result
The study found that while tuning boosts performance and some architectures are much faster than others,
Given the speeds of current disks, the libraries explored here perform at rates that are easily fast enough to build high performance, reliable storage systems.
Translation: this isn’t string theory.
Other findings include:
- The RAID 6 codes out-perform the general purpose codes.
- For non-RAID 6 codes, the Cauchy Reed-Solomon performs much better than straight RS
- CPU architectural features, such as cache size and memory behavior, make it hard to predict an optimal data structure for a given code configuration.
- The code’s memory and cache footprint can have a large impact on performance.
- Specialized RAID 6 codes hold promise for creating efficient storage that can withstand numerous concurrent disk failures.
- Multicore performance issues are largely unexplored.
The StorageMojo take
The architect for a planned commercial 200 PB cold-storage infrastructure confessed that he can see how to get to 25 PB today, but not beyond. Yet they have no choice but to start building now.
This market’s eventual structure may parallel that of today’s tape silo market: everal hundred large customers who are continuously churning through rolling upgrades of media and servers.
Right now, tape silos still enjoy an economic advantage over disks. But it looks like disks have more degrees of freedom to improve their cold storage economics than tape.
In just 5 years the first exabyte cold storage systems will be on the drawing boards. It is time for disk companies to get serious about a tape-replacing archival disk. And for clever startups to focus on this emerging market.
Courteous comments welcome, of course.
by Robin Harris on Wednesday, 17 June, 2009
Jeff Darcy has a good post on key data stores, like Amazon’s Dynamo, and how they differ from filesystems and databases. He relates his transition from a filesystem purist to a more flexible perspective.
The thing that really changed my mind about this was an observation in the Dynamo paper: strong consistency reduces availability. I’ve always thought of data availability in terms of data not being lost or stranded on the other side of a failed network connection. The Dynamo insight is that many applications have to do a lot of work within a small acceptable-response-time window, and to make sure that they fit into that window they have to impose deadlines on all sub-operations including data access. If consistency issues make data unavailable within that deadline then they’ve made it unavailable period, with practically the same effect as if the data were unavailable in any other sense.
In short, while there is a class of applications where traditional consistency is important, there is an emerging class where strong consistency isn’t affordable or necessary. Good stuff.
Another point
Many of the features that make up these non-FS/non-DB stores seem to have a lot in common with object storage. In a highly mobile world the whole idea of placing a file in cyberspace by a path name is anachronistic at best: it could be, physically, almost anywhere and is most likely in several places at once.
The StorageMojo take
While the name “object” is problematic for market acceptance, the concept of managing objects in a flat address space – like the web itself – is a better fit for a mobile networked world. There is a major opportunity to move file management infrastructure forward to reflect the world we now live in rather than a 35 year old server environment.
Courteous comments welcome, of course. Thanks to Wes Felter’s Hack the Planet blog for the link to Jeff’s post.
by Robin Harris on Monday, 15 June, 2009
DataSlide has come out of stealth mode with a very creative SSD replacement technology. They call it a Hard Rectangular Disk or HRD.
Here’s their quick overview:
DataSlide applies technology in new, patented ways to achieve unprecedented high performance 160,000 IOPS & 500MB/sec and low power <4 Watts for a magnetic storage device:
- A piezoelectric actuator keeps the rectangular media in precise motion
- A diamond solid lubricant coating protects the surfaces for years of worry free service
- A massively parallel 2D array of magnetic heads reads from or writes to up to 64 embedded
heads at a time
Here’s a diagram, courtesy DataSlide:
But that’s not all. According to the redoubtable Chris Mellor at The Register a
. . . 2-dimensional array of 64 read-write heads, operating in parallel, . . . positioned above an piezo-electric-driven oscillating rectangular recording surface. . . .
The data organization compared to a disk drive look like this:
courtesy DataSlide
Chris also reports that Oracle’s Embedded Global Business Unit is working with DataSlide to incorporate a database to create a “smart” storage device for use in I/O intensive “multiple concurrent stream” applications.
The company says the drive is at the prototype stage and uses existing high-volume production technologies, including perpendicular recording media, semicondutor lithographic heads and LCD glass treatments.
The StorageMojo take
DataSlide has taken much from IBM’s Millipede concept and reimagined it using common technologies. While much remains to be done to productize the prototype, the fact of such architectural creativity should spur new thinking at the hard drive companies.
Of course, just like SSDs, with such low latencies it doesn’t make much sense to stick the device at the end of a long, complex, high-latency interconnect chain. PCI-e HRD card, anyone?
Also, the relatively low capacity – 36GB – of the prototype device suggests it may slot in between larger capacity SSDs and DRAM. Until we know the economics though that is almost baseless speculation.
Let’s hope they can get it to market in less than 3 years. And let the based speculation begin!
Courteous comments welcome, of course. This post was updated from the original with the digrams and some minor edits.
by Robin Harris on Wednesday, 27 May, 2009
They are lifeless ideas nearly devoid of meaning – but they just won’t die. Like real – if a mythical creature can be called “real” – zombies they take a long time to kill.
In a time of economic crisis, when IT organizations are begging for resources from gimlet-eyed CFOs and IT marketers are trying to win attention to new products, the industry needs new ideas. And that means getting rid of some old ones.
And the nominees are:
- Grid. StorageMojo declared “Grid is dead” 18 months ago, but some people didn’t get the memo. Call it a cluster and make life easier for your prospects to understand what you’re trying to sell them.
- CAS. Content Addressable Storage has the germ of a good idea – extended metadata – but as a feature it has been overtaken by the more recent need for e-discovery and compliance features. EMC didn’t help by attaching the name to a costly, proprietary and hackable platform.
- Object. Objects come in 2 varieties: files with extended metadata wrapped around them; and data slices of consistent size -like the Google FS chunks. Extended metadata isn’t a useful differentiator – see CAS above – so calling a file with a metadata wrapper an object may be correct but it it doesn’t help prospects understand what you’re doing. It’s a file with extended metadata.
- RAID. The term RAID started losing its mojo years ago and the trend has accelerated with the adoption of “file-based RAID” and other data protection strategies. Yes, customers have been well-trained to associate RAID with data protection goodness and that’s worth something. But if you have redundant data protection that isn’t RAID you’re missing a chance to say to say “better than RAID.” RAID 5/6 controller costs and reconstruction hassles are worth some snark if you don’t have them.
- Cloud. Yep, the word du jour is, because of its popularity, rapidly being drained of meaning. Public? Private? Compute? Storage? Cheap? Secure? Software? Infrastructure? What??? Sure, talk about cloud, but for goodness sake don’t put the word cloud in a product name. You’ll regret it within 2 years if you do. If you simply must then follow EMC’s oblique approach with Atmos.
The StorageMojo take
Marketers are exhorted to create a niche and dominate it, which leads to much unneeded niche creation. Most die unmourned, except by the people who wasted time and money trying to instill life and meaning into them.
Once we have terms like RAID that are accepted we are wise to use them for as long as they make sense. But for many vendors RAID is no longer what they do. The term obscures the real advantages of their architecture.
Extended metadata is an idea for which we have no good label today. Yet it is now needed in many markets because we are realizing that electronically stored information needs more metadata the longer we keep it.
A clever, catchy name for extended metadata would help the industry educate customers on the need and the benefit. Any ideas?
Courteous comments welcome, of course. No, I don’t have any ideas for what to name extended metadata. I’m not very good at naming.
by Robin Harris on Wednesday, 25 March, 2009
The Wall Street Journal (subscription probably required) reports that e-books appear to be taking off:
Barnes & Noble Inc. has launched a free electronic-reader application for Research In Motion Ltd.’s BlackBerry as general book sales flag and the e-book market heats up. . . .
Consumers spent about $100 million on e-books in 2008, according to one estimate. German media concern Bertelsmann AG noted in its annual results Tuesday that its e-book sales “increased tremendously” in 2008 and that its Random House publishing arm will have 15,000 e-book titles in the U.S. by year-end. . . .
Sony Corp., which also sells a dedicated e-book reader, recently struck a partnership with Google Inc. that provides users of the Sony Reader with free access to more than 500,000 public-domain titles that Google has digitized.
A practical lesson in consumer technology diffusion
For 20 years E-books have been a so-obvious-it-hurts application:
- Books are heavy and expensive to ship and store – and returns kill profits
- Many books are words only: easy to convert to ASCII
- Shop at home convenience
- Instant download gratification
But despite all the Obvious Goodness ebooks haven’t left the building – until now.
A confluence of factors is behind the take-off:
- A growing supply of ebooks
- Wider adoption of readable large screen handheld devices
- Simplified purchase and wireless download – thanks to improved infrastructure
- Popular awareness of ebook advantages
Ebooks could have taken off on notebook computers – but they didn’t. The unseen need for a paperback-sized reader, along with much simpler access, has catalyzed a market.
The StorageMojo take
Emerging and evolving technologies are StorageMojo’s stock-in-trade. Comments back sometimes betray the notion that because it hasn’t happened yet, it will never happen. Which is always true – until it isn’t.
Apple’s Newton was a great idea hampered by technoloy limitations. The less-ambitious Palm Pilot got the form factor and consumer pricing right, and the handheld device market took off. But not the Palm ebook market.
Fast forward 10 years and iPhone has more power than the Palm Pilot ever dreamed of and it’s an acceptable – though not ideal – ebook reader. As well as a gaming platform, browser, PIM and more.
In prospect technology change always takes longer than we think it should. In retrospect it is amazing how quickly the technology evolves.
Courteous comments welcome, of course.
by Robin Harris on Monday, 9 March, 2009
Butler Lampson, Irwin Jacobs and Vint Cerf walked into a bar . . . .
Actually, they spoke at a symposium held at Google’s HQ sponsored by the American Academy for Arts and SciencesThe Public Good: The Impact of Information Technology on Society. (See the Stanford News Service report here.)
They mused on where technology would be in 20 years. One prediction: self-driving cars.
“In less than 20 years I think we’re definitely going to have…cars that drive themselves. I mean for real,” said Lampson. “This means you can read the paper while commuting.”
“What paper?” someone shouted. These guys still read daily printer papers? That is so-o last millennium.
Google-scale bit rot
More to the point was Vint Cerf talking about widespread data loss:
It’s the year 3000. You’ve just done a Google search and you turned up a 1997 PowerPoint file. You’re running Windows 3000,” he theorized. “The question is: Does it know how to interpret a thousand-year-old PowerPoint file? And the answer is probably no.”
If all of our generation’s technology is digital, and 1,000 years from now the current digital information is “rotten,” all of the information about our time period will be lost, he warned.
“By the year 2100 everyone will wonder about all of you and the beginnings of the 21st century because all the bits about you will be rotten and no one will be able to interpret them
The audience and Lampson disagreed, saying clever engineering could solve the problem. Engineers always think that.
The StorageMojo take
This isn’t so much about clever engineering as it is getting someone to do the scut work. Save all .ppt files as .pdf, for example.
But I like how both Vint and Butler implicitly assume that Google will be around in 1000 – or even 100 – years and will have that data preserved. That’s one prediction they didn’t even talk about.
Courteous comments welcome, of course. Let me know if Google posts a video of the event. I’d like to see it.
by Robin Harris on Tuesday, 3 February, 2009
Researchers at Stanford University have demonstrated quantum holographic storage, shattering long-held assumptions about the information limits of matter. Moving into the sub-atomic realm, they permanently stored 35 bits in the quantum space surrounding a single electron.
Moreover, the technique allows holograms to be “stacked” in 3 dimensions, enabling 2 35-bit storage elements to occupy the same space. Encoding data using mere atoms would be less than half as space efficient.
Holodeck backstory
Holograms use 2 coherent laser beams – a reference beam and an illumination beam – to create an interference pattern that is recorded on photo sensitive media. Shine a laser on the recorded interference pattern and the original image is reconstructed in glorious 3D. As the laser moves around – or you do – you see the image from different perspectives..
A InPhase Technologies, a spin-off from Bell Labs has been working on commercial holographic storage for several years. They may even be shipping it – who knew?
Quantum holography
The researchers (Christopher R. Moon, Laila S. Mattos, Brian K. Foster, Gabriel Zeltzer and Hari C. Manoharan) an interdisciplinary team from the departments of Physics, Electrical Engineering and Applied Physics at Stanford, used a gas of 2D surface state electrons held on the face of a copper crystal. Using atomic manipulation the team place individual electrons in closed quantum corrals – a common research tool.
The tricky part was encoding a specific pattern around the electrons. Using simulated annealing, they controlled the amplitude and phase of the electrons to encode the bits.
Baseline quantum holography
Since the writing “surface” is a gas, the team was then able to encode holograms in the same space by embedding them volumetrically in 3D. Here’s a picture:
Volumetric holography
A scanning tunneling microscope, a standard tool of atomic level research, was used to read the information and create the images.
iPod sub-Nano
The authors conclude:
We have experimentally demonstrated that 35 bits can be permanently encoded into a time-independent fermionic state, and that two such states can be simultaneously prepared in the same area of space. . . . In all experimental attempts, extending down to the subatomic regime, the encoding was successful and the data were retrieved at 100% fidelity. We believe the limitations on bit size are [.025 of a nanometer], but surprisingly the information density can be significantly boosted by using higher-energy electrons and stacking multiple pages holographically. Determining the full theoretical and practical limits of this technique — the trade-offs between information content (the number of pages and bits per page), contrast (the number of measurements required per bit to overcome noise), and the number of atoms in the hologram — will involve further work.
I hope they get the money they need to continue this research.
The Storage Bits take
This is far frontiers research – not something you’ll see in a commercial product in 5 years or even 25 years. But by demonstrating that quantum holography can store massive amounts of data in a very small space, the scientists have pushed out our conception of how much data mankind may eventually be able to process and store.
Comments welcome, of course. The complete article is available online at Nature Nanotechnology (pdf).
by Robin Harris on Thursday, 22 January, 2009
Some quick impressions from the SNIA cloud storage symposium.
Not everyone believes in economies of scale
At least one presenter questioned whether there are economies of scale that justify the higher latency and lower bandwidth of cloud storage. I recently wondered about that myself.
But since then I’ve checked James Hamilton’s work on cold bulk storage. I’m now comfortable that there are significant economies of scale – at least for rarely accessed data – in a well-architected and managed large-scale data center.
There’s one more problem with economies of scale: some strategists and analysts are unclear on the concept. “Why can’t any enterprise do what Amazon or Google do?” they ask.
If 500 petabytes costs less per GB than 10 PB does, then the economic pressure to build 500 PB data centers is constant. If your company only needs 10 PB you will never be as cost-effective as the 500 PB data center.
Every class A data center has its own diesel generator set, but they get 99% of their electricity from the power company. Why? Because it’s cheaper.
ZFS flagged
Several presenters mentioned Sun’s ZFS as a significant enabler of cloud storage, none more enthusiastically than Joyent’s Ben Rockwood, author of the excellent Cuddletech blog. Ben made a compelling point about OpenSolaris: given Solaris’ industrial strength and many cool features – like Dtrace and ZFS – why wouldn’t you use it instead of Linux?
Most surprising company
You know the cluster storage software company with hundreds of customers, the leader in healthcare image storage and archiving, whose resellers include HP and IBM, with 10s of petabytes under management? Me neither.
Meet Bycast. They’ve been in business 10 years. Coolest feature: you can set it up so you don’t have to back up the data. Yes, people are doing that in production today.
The StorageMojo take
Ever since Google built a huge, low-cost storage infrastructure from commodity parts, the proprietary array business has been living on borrowed time. Optimized for structured transactional data, traditional cached RAID arrays will be around for many years to come, but expect a long decline.
The growth in file data, especially consumer digital content, has made data both cooler and massive. As network bandwidth improves, remote storage becomes more attractive.
There are 3 key elements to the cloud puzzle:
- Economies of scale. The steeper the slope the faster data will migrate to remote storage.
- Network bandwidth. A faster network makes remote storage more compelling.
- Component, product, solution? People don’t want to buy storage – they want save and protect their important data.
Courteous comments welcome, of course. Get copies of the presentations here.
