How flash is really going to affect the storage industry is becoming clear. The short take: not as big a deal as flash vendors hoped. The longer take: There won’t be much of a mid-range flash market; instead we’ll see either costly fast flash or cheap slow flash.

There are lots of theories about how flash will alter the mass storage landscape. This is mine.

The flash write problem
The fundamental flash problem is the slow writes. There are 3 elements to the slow write problem.

  • Flash has to be erased before it can be written. Every write operation is really 2 write operations.
  • The writes are large. Typical block sizes are 128KB to 256KB. Writing a single page requires writing – after erasing it first – the entire block.
  • The write bandwidth to a single block is less than a slow disk. High bandwidth writes requires parallel paths to multiple blocks.

These problems can all be engineered around.

  • Garbage collection-like algorithms can be extended to enable a supply of erased blocks
  • RAM backed by a small battery or capacitor can buffer writes for later re-writing to flash
  • Controller chips can be built in high volume with multiple data paths

But at what cost? The first two require well-engineered software and some sort of CPU to run it. Since it is software it will have bugs. Can it be any more reliable than current drive firmware?

The dilemma
For enterprise use, flash-based SSDs need to be rock-solid, which implies a lot of careful and costly engineering. For consumer use, they need to be very high volume, which means low-cost.

It is a similar problem to RAID controllers: very low-end RAID controllers aren’t reliable enough for enterprise use. They also aren’t cheap enough – or easy enough – for consumers to buy in volume. RAID controllers have engineering problems similar to flash translation layers.

Making flash drives look like disks makes them easy to integrate, but if you really need performance it also makes them costly – like the $10k for the flash drive EMC is using in the Sym.

Flash in the disk controller?
As I’m writing this a NetApp exec says that flash will be disruptive because by placing flash in a disk controller they will reduce the need for the costly and highly profitable fibre channel disks. That could be correct. It sounds smarter than sticking flash on a disk.

The StorageMojo take
Despite the miracles of cost-reduction and integration the industry regularly performs, some things, like power provisioning, don’t get cheaper. High-quality software engineering doesn’t either. That is what high-performance flash drives require.

The high-performance consumer flash drive seems to be a mirage. I’d like to be proven wrong, but today’s notebook SSDs don’t offer superior application performance and cost 10x as much. Hardly a recipe for success.

Update: Intel is planning to offer “high-performance” flash drives with partner Micron. I saw an impressive demo – is there any other kind? – at the Storage Visions conference. But with the early marketing missteps of Samsung, it looks like the consumer flash drive may fall off the hype cycle into a deep ditch. Flash drive marketers: now is the time for precision marketing if you ever hope to establish a mass market. Consumers remember unkept promises. Until you are cheaper. End update.

Comments welcome, as always. Also check out BPLRU: A Buffer Management Scheme for Improving Random Writes in Flash Storage by two Samsung researchers, Hyojun Kim and Seongjun Ahn for a nice intro to flash issues.