I saw the future at NAB
No, it isn’t a storage company. Or a system company. It is a workload.
The dominant storage workload of the 21st century. Large file sizes, bandwidth intensive, sequential reads and writes.
What about OLTP, small random reads and writes?
Not going away. The industry is well supplied with kit for OLTP. It will simply be a steadily shrinking piece of the entire storage industry. OLTP will keep growing, just not as fast as big file apps.
Robin, did the bright lights and eye-candy of NAB addle your brain?
Bear with me a moment.
Think about the microprocessor – back in the early 1970’s. Few recognized the potential of really cheap computer chips. Calculators were the first big application. $4,000 calculators became handheld $400 calculators and then $40 calculators. Demand skyrocketed. Slide rules died. Wang shifted to mini-computers from calculators.
Storage today is in the $4,000 calculator stage.
OLTP isn’t the way world thinks
Around the corner from the Isilon booth was a film scanner. It was scanning 35mm movie film, one frame at a time. Once a second it went “ka-thunk” and shifted to the next frame. Each ka-thunk meant a new 2k scan was completed. Each scan, another 13 MB file.
Next to it was another scanner. Once every seven seconds it went “ka-thunk” as it complete a 4k scan. And a 53 MB file.
6k and 8k scanners are on the horizon, because film makers don’t want to lose any data of the original frame.
Uh, Robin, there’s more to the world than movies
Indeed there is. This isn’t about movies. It is about digitizing, and storing, our analog world.
Massive storage is getting cheaper and will soon get much cheaper. Just as with microprocessors, as storage gets cheaper more applications open up. As networks and display options improve, there are more places to send the “assets” as media folks call content.
Isilon VP Brett Goodwin told me about their new deal with NBC Universal. Isilon’s self-managing storage is making it possible for NBC to dramatically increase their Olympic games broadcast hours without increasing people costs. Gordon Bell’s idea for recording every moment of his life isn’t limited by our ability to capture pixels and audio: it’s the storage and retrieval that’s the killer.
As bandwidth intensive apps grow, more investment will improve bandwidth
I’ve been planning to write about NFS v4.1, which adds parallel support to the NFS standard. Making parallel I/O a standard is going to free up vendor innovation for a market that is starved for bandwidth. I don’t think CIFS is a player in HPC, and pNFS will ensure it never is.
The other shoe
Disk drives: rapidly growing capacity; slowly growing IOPS. Small I/0s are costly. Big sequential I/0s are cheap. Databases have long used techniques to turn small I/Os into larger ones. With big files, you don’t have to.
The StorageMojo take
The combination of pervasive high-resolution media, consumer-driven storage needs, expensive random I/0s and cheap bandwidth point to a new style of I/O and storage. The late Jim Gray noted that everything in storage today will be in main memory in ten years. A likely corollary is that everything analog that is stored today will be digital in 10 years. “ka-thunk”
Comments welcome, of course.