The world of data storage is changing faster than it has since the mid-90’s amid the rise of hardware arrays and storage networks. Looking back 2008 will be seen as a pivotal year. The big news, in rough ascending order:
FCoE
Though production-ready products are still in the future, the broad vendor embrace of Fibre Channel over Ethernet signaled the beginning of the end for the Fibre Channel physical layer. The storage companies who profited from a decade of Balkanizing the storage network market will have Cisco calling the shots.
Brocade, in particular, needs good strategy advice. Maybe one of these days they’ll get it.
Blu-ray tanks
Call me old-fashioned, but I have a soft spot for removable media. So I’m sorry to see Sony screw the pooch with Blu-ray’s big-studio-friendly licensing and hate-the-customer DRM. And sinking the PS3 as well.
The good news: you can put HD content on a standard DVD – just not as much; and there’s an upscaling dvd player – the Oppo Digital DV-983H that upscales ordinary DVDs to near Blu-ray quality. Yes, even better than the upscaling on a Blu-ray player. One less reason to pay the Blu-ray tax.
2.5″ drives
Rumor has it that Seagate is designing its last generation of 3.5″ drives, which augurs the switch to SFF in desktop and enterprise systems. 3 years ago 2.5″ drives were 1/5th the capacity; today the gap is 1/3 the capacity and a much smaller price differential.
At some point it will occur to Seagate’s top management that 1.8″ and 2.5″ drives are the disk industry’s best answer to flash. Now, if Seagate were in the I/O business, it would be a different story.
Zero-maintenance storage
Xiotech and Atrato introduced storage boxes that guarantee capacity, performance and uptime with no maintenance for 5 and 3 years respectively. These are storage game-changers.
That Seagate sold ISE to Xiotech after spending years developing it has to be one of their biggest blunders ever, several notches above buying Xiotech in the first place. The ISE is, in effect, a super disk that Seagate could have sold to all its enterprise disk customers.
Flash
2008 is the year that every major vendor – with the laudable exception of laser-focused WD – announced alliances and/or plans to enter the flash drive market. High-end SSDs will displace 15k high-end disks in the next 3 years.
But flash-in-disk-clothing is the near/medium-term solution. Fusion-io and Violin are on the winning architectural track. Flash belongs between the CPU and disk layers: that’s where we’ll get the most benefit for the added cost.
Hey, disk vendors: want to stick it to Intel, Micron and Samsung? Buy one of them. You are in the I/O business, not the disk business.
Commodity-based cluster storage
EMC’s Atmos, HP’s Extreme Storage 9100 and IBM’s XIV are commodity-based cluster storage. The important thing is the storage mainstream has embraced storage clusters based on commodity hardware and mostly open-source software. That’s what Google did years ago and soon many companies will.
Yes commodity hardware saves real money, as I and Bill Mottram of Data Mobility Group found out when we ran the numbers on HP’s 9100 vs Isilon, NetApp and Sun. We’ll see if Atmos is on the latest EMC price list when I do the updates later this month.
The StorageMojo take
2009 will be a great year for the hungry and flexible. The ongoing financial train wreck is trouble for Big Iron fans in the data center.
Fortunately, help is on the way. Look for my 2009 forecast before the end of 2009.
Courteous comments welcome, of course. Of the companies mentioned I’ve done work for HP and Fusion-io.
regarding FCoE: this is probably one of the reasons Brocade bought Foundry
You forgot OpenStorage in the list.
Why do I care about Blu-Ray’s DRM for data storage? It doesn’t affect me when I’m storing files on the disc. I’m not interested in HD video until the (mid-range) players are under $100 and the titles under $20, but Blu-Ray writers are coming as an option on most laptops today. Now if only the blanks weren’t $10/each.
As Brainy said,
I think the Sun OpenStorage strategy from Sun and related products is a very intresting approach, it’s not mature enough to solve enterprise storage needs now, but i think they can deliver good poducts in the near future.
So with violin: What exactly am I missing here? It seems like they’re doing exactly what Texas Memory Systems has been doing for years now with the RamSan. And on that front, it looks lke TMS also has a product to compete with Fusion-IO. It would seem to me they deserve at least a mention being (somewhat) pioneers and all 😉
**edit, I lied, they don’t have any pci-E cards, those are all DSP’s. I thought I had read they were coming out with a PCI-E based flash card as well, but I guess not.
Interesting picks. Personally I think Flash is getting screwed by not knowing what it wants to be when it grows up, the 21st century’s answer to magnetic core memory or fixed-head drums. It has certainly become the removable media standard for sure.
I’m also not as optimistic as you are on zero maintenance storage. I remember the IBM demo product (stackable cube of storage) where you were left with dead storage bricks in the middle because you didn’t want to unstack 20 – 30 TB of storage to get to the place where you could remove it. I do agree the days of carefully removing one drive at a time out of a shelf of drives may be on the decline. Give me a 10 – 15 drive “sled” I can pull out and “remanufacture” after its more than 33% dead that would be fine.
And finally you haven’t yet touched on what seems like a growing trend which is SiTH (my silly marketing acronyn for Storage in The Home). Cable DVR penetration plus game consoles plus media servers and having your media in central place is starting to make sense to non-nerds. The who netflix/blockbuster/amazon/itunes media on demand markets, the availabilty of an “always on” appliance with a relatively large amount of storage in it and a fat download channel available (cable/sat). Given Intel’s recent numbers and the proliferation of notesbooks (vs desktops) nearly everyone I know now has at least 2 USB attachable external hard drives, some folks many more, and the pain is growing around managing that storage.
Interesting times, especially with money being so tight.
–Chuck
I think you missed BIG on your completely biased and inaccurate scale-out commodity storage piece. Stating that Isilon is not flexible is nuts. How about mentioning the ease of setup and management…far superior to these other vendors. I wonder how much HP paid you to publish this crap!
Getting the best performance out of Flash by putting it directly onto server buses for maximising performance (especially latency) is fine as far as it goes, but in large centres that’s not how storage is accessed by applications. In large scale data centres storage is provided via some form of network service, either as block-mode using FC or iSCSI or network file services. Reverting back to the position that we had pre-networked storage with dedicated storage in each server is not going to happen in large centres. Storage will remain networked.
In general, applications tend to use files, and that’s the abstraction layer that matters for most purposes. It will, of course, be possible to design storage arrays where flash is more closely coupled with the internal bus structures of the devices. It would be somewhat surprising if the likes of EMC, EDS and NetApp were not looking at exactly that. That way they could hide away the flash hardware interface from the more standardised storage services offered to applications. It’s also possible to write a “flash friendly” file system (OnTap and ZFS are a very good fit for storage devices with very good random read performance as both have a tendency to fragmentation with some access patterns and they are also more tolerant of poor random write performance).
Applications are notoriously slow to change, and there are plenty of legacy applications out there that could do with a simple speed boost, especially in the area of reduced latency. Existing network storage protocols and paths through device drivers and so on probably means that it will be hard to get lower than about 0.5ms, but that’s still an order of magnitude or more better than we see on random access on 15K drives (and the IOPs possible increase still more). There is, of course, nothing stopping a commodity approach to this, once the appropriate hardware, software and standards emerge.
Outside the realm of dedicated storage array hardware and embedded server storage, then I don’t see and obvious winner just yet. We still need this shared access model, and unless a very high speed, low-latency shared flash storage hardware model appears (with drivers), then I still perceive that the vast majority of large applications will access their storage over existing networked storage protocols.
If we want to see the persistence of legacy access methods, then look at how the vast majority of removable flash media is formatted. It doesn’t matter if it’s USB, CompactFlash, SD, memoryStick, you will find that the various generations of FAT dominate. It’s perfectly possible to embed abstraction layers into storage devices to optimise performance for an alien storage type. Quite what the physical interace to the actual storage device is then an internal only matter.
What do you think of the future of Blu-Ray jukeboxes for archival storage? My company is just starting to look into it & have found very few vendors…
Hi Robin,
For top stories 2008, you said:
>>High-end SSDs will displace 15k high-end disks in the next 3 years.
I don’t think 15K disks are a long term good bet, but I don’t think it’s because Flash SSDs will replace them. There is a large and growing body of evidence that write performance in synchronous workloads — where application IO streams are transactional “threads” of interdependent reads and writes — are so problematic as to reduce performance of a Flash-SSD often to as little as the equivalent of just a few 7,200 RPM SATA disks.
Gene Ruth recently tripped over this problem when he observed write performance of a RAID5 array made up Intel’s much-ballyhooed X25-E “enterprise” Flash SSD.
The asymmetry of read vs. write performance is nothing new. But what has gone unnoticed by too many for too long is not the asymmetry of performance when reads and writes are measured in isolation, it’s the dramatic FURTHER decline in performance that happens when Flash SSDs encounter mixed read-write workloads — especially when these are encountered in the “synchronous IO” context of ordered and interdependent reads and writes typical of transactional workloads. This effect is so dramatic that in RAID-5 (which is a good model of this type of IO), write performance of a four-SSD array has repeatedly been observed to drop BELOW that of a similar sized array of 7,200RPM SATA disks.
At an average now of 300 15K spinning disks for each CPU socket in the server now, TPC is probably the most IO intensive, actuator-bound workload on the planet.
So allow me to offer a couple of my own predictions.
1) 2009 will come and go, as 2008 did, without a single published, audited TPC result using Flash SSD in place of disk, and the silence will become deafening.
2) The industry will learn a lot more about the intrinsic and fundamental limitations of Flash at the silicon architecture level, and we’ll begin looking for other alternatives.
Flash SSD will find lots of new uses, but not among brick-and-mortar enterprise applications. The enterprise data-center markets for Flash are going to be much smaller and applications far more limited than anyone expects.
I agree with Enrico on the Sun Open Storage and Storage 7000 series. It is a very exciting direction. It can be as big as Linux in the Open Source OS world. What is your take on it Rob?
I don’t understand why the Big Box of Battery Back RAM (B4R) is not a widely used alternative to 15k disks, x25-e’s, and so forth. Commodity dram is around $10/GB, half the price of an x25-e and orders of magnitude faster, especially for random write.
Well 2009 is not yet over and the “deafening silence” ended.
On October 10th at Oracle Openworld, Larry Ellison announced a new world
record (audited) for TPC-C using enterprise grade Flash.
Flash is the new disk and disk is the new tape. That’s how 2009 ends.