StorageMojo

What’s behind the hype?
Congrats to the Xiotech team on generating the most interest at SNW. Their demos were crowded with the curious. Their claims bordered on the implausible, but the credibility of the engineering team kept derision in the corners.

I talked to Ellen Lary, engineering VP, and Steve Sicola, CTO, as well as taping the very helpful Chad. Before going any further, let’s roll the 103 second - less if you skip the credits - tape:

How do they do it?
Darned if I know - they weren’t talking. Reading between the lines:

• Systems thinking: each disk drive is more powerful than that 1980’s workhorse VAX 11-780 supermini. Put that intelligence to work!
• Clean code: Xiotech has had free run of Seagate’s best thinking - so they’ve gotten rid of the firmware hairballs inside disk drives to create a distributed architecture where components cooperate in a trusted environment instead of competing. Their disks won’t work with your Brand X controller.
• Spare no expense: the Xiotech team is going for the gold with a top-of-the-line resource-intensive architecture. If you have to ask how much it costs you can’t afford it.

With 350 IOPS per 15k FC drive claimed - and Sicola said more was coming - this is a lot of bang. When we see some pricing we’ll know about the bucks.

The value proposition
Xiotech’s bet is this: all is forgiven if it kicks butt 7×24 for 5 years. Each ISE is a storage utility writ small. With these building blocks, they promise, you can build an infrastructure whose availability and performance - still the storage ne plus ultra - will beat anything from EMC, IBM or HP.

A worthy goal, indeed.

The StorageMojo take
Just when EMC is assuming that Maui’s new Über-layer will win them the undying cashflow of multinationals, Xiotech comes along and exposes EMC’s feet of clay.

That sucking sound you hear is EMC emptying the datacenter’s coffers to run 7×23.999. If Xiotech can win even 10% of EMC’s business, they’ll be a $1 billion company sooner than they dreamed. And their VCs will be high-fiving in Aspen this winter.

NetApp, IBM and HP should worry as well. It sounded like Xiotech was OEM’ing the ISE to others - if so it makes sense to add them to the product line.

The disk-in-a-box model needed a thorough rethink and kudos to Xiotech for doing it. But many promising - on paper - products have failed. Once Xiotech is shipping and there is independent testing - then we’ll know what they’ve really got.

Comments welcome, of course. The indefatiguable Beth Pariseau homes in on the Atrato/Xiotech nexus.

It looks like Xiotech is going to cop the “Best Announcement at Spring SNW ‘08″ prize. See the nifty flash intro.

I did speak to Ellen Lary, Engineering VP last night after going through their mobbed booth. Later today I have an appointment with Steve Sicola, Xiotech’s CTO. I’ll have a more complete report later. Here’s what I’ve gleaned so far.

Remember Atrato?
Interesting stuff:

• Sealed unit starting at 1.5 TB. They had a 1 PB system on display in 3 54 RU - i.e. bigger than you use - racks.
• 5 year warranty and nifty blue LED light. Are we in a data center or a cocktail lounge?
• Uses the draft T10 DIF (Data In Flight or Data Integrity Field, Data Integrity Feature - depending on where you read it - evidence that humans have a far greater problem with data integrity than computers do) standard to protect data within the array.
• Uses Seagate’s own drive test software to attempt repairs on drives in place. Ellen said that about 70% of drives work normally after a power cycle.
• If power cycling doesn’t work, the box can perform a complete reformat of the drive, starting with laying down tracks and proceeding on to what you and I consider “formatting”.
• If a particular head is the problem, they can electrically disable that side of a platter while continuing to use the rest of the capacity of the drive.
• It is cheaper to put in a couple of extra high-end drives than it is to make a service call. This won’t be true in China of course.

The best announcement that WASN’T made at Spring SNW
A company has figured out how to enable long distance synchronous replication. Here in America we like things big - including our idiots in Washington - and our disasters are no exception.

Hurricanes, earthquakes, volcanos, floods, blizzards, tornados and fires - and purblind ideologues - can lay waste to hundreds or thousands of square miles. So normal synchronous replication distances don’t cut it for gotta-have-it infrastructure.

The still-in-stealth-mode company’s Chief Engineer, Montgomery Scott, explained that by running dilithium crystals a little hot, a special hyperspace “tunnel” is created enabling . . . .

Just kidding. Their actual solution looked good in principle but the devil is in the details. I asked all the hard questions I could think of and they had answers for all of them, so it looks like they have something real.

Look for a fall announce.

The StorageMojo take
Those of you wondering if this year would be more of the same old, same old, fear not. The spirit and fact of invention is still strong in the ever-more-vital storage industry.

Comments welcome, of course. Would you use 1,000 mile synchronous replication if you could get it?

On the occasion of announcing a new HPC modular array, the Engenio-based 4600, SGI commissioned me to do a StorageMojo video for them.

Some interesting comments about modular vs cluster storage and CXFS. And I got to practice my radio voice.

We spoke for about an hour and I boiled down the comments of Raj Das of SGI and LSI’s Flavio Santoni - before putting the StorageMojo take on it.

Must get video page up soon
One thing about video: every syllable counts. This one gets into Apple’s Motion for the first time. Nothing wild though.

Update: The video came down for a couple of modest tweaks. Now it’s back - new and improved.
End update.

The StorageMojo take
Video is another way to reach people who aren’t going to plow through a white paper. In 4 minutes you meet some people, get exposed to some new ideas and maybe learn something. And you can be drinking coffee in your bathrobe at the same time!

Comments welcome, of course.

Already?
Didn’t we just have one of those?

Leaving sunny Arizona for sunny Florida today. Say hi if you see me.

NAB next week
I’m driving to Las Vegas next week to see what’s cool in media storage.

And EMC World next month!
Never been to one before. Hope to hear all about Maui, the new global digital repository.

Squeezing in a trip to Chicago next month too.
No business planned, but I think I will have some downtime.

Self-service pricing is coming
A reader wrote in the other morning to say:

I wanted to take a moment to thank Robin for this invaluable tool. As an architect I often use this site to get Rough Order of Magnitude costs for components and set these up in spreadsheets that allow others to apply thier vendor discounts to get my ROM costs closer to their actuals. The site lets me rough out a Vendor A solution and a Vendor B solution and quickly get to comparative pricing without waiting for negotiations around pricing.

You are very welcome.

But now there is something better
An approval-based system that gives you up-to-date pricing. Go to pricing pages for 3par, CommVault, DataDomain, Lefthand, VMWare or EqualLogic and click on the reseller ad.

That will take you to their site where you can request a quote. This isn’t an anonymous system - you have to provide personal information - and the reseller has to approve sending you the quote.

Normally that should be a quick process. I tried it once and got a response in 5 minutes. The service is available from a company named Echoquote.

The StorageMojo take
Marketing, and many sales people, like to keep a tight hold on pricing for reasons that may have once made sense but don’t today. As we saw with the latest “Dear Uncle StorageMojo” a qualified buyer limited his research because pricing and feature opaqueness made it too difficult to compare many vendors.

That is just wrong if you’re the company losing the business. And since smaller companies are the ones losing most of the business, they’re the ones who should be easiest to deal with.

At DEC I used to hand out our thick price lists - marked “Company Confidential” - to all the regular customers. With a $20,000 per working day budget I didn’t have time to quote every cable and network interface. My value was product knowledge - not pricing.

And not to worry, I will still update the StorageMojo price lists every 6 months or so.

Comments welcome, of course. Who is the easiest storage company you’ve dealt with?

The 2nd installment of an occasional feature . . .
A reader writes:

I think your input would be valuable in helping me make a decision on storage for my company. I’ve done loads of research and I’m fairly certain I have good players narrowed down, but have reservations about both. . . .

Players:
-Datacore SANMelody H/A solution on HP hardware.
-Equallogic PS3800XV

The app
It’s is an up-to-the-minute commercial application supporting virtual machines. The VM’s run proprietary messaging/transactional servers that spend 99% of their disk I/O time appending very small messages - ~300 bytes - to transaction logs.

Update: After the initial comments, the prefers-to-remain-anonymous reader (BTW, I did check him out and his company is for real) added this clarification:

• Yes, there are DR and HA requirements.
• Each VM has its own transaction logs that can grow to GBs in size. These transaction logs are not for archival purposes, rather to recover state in the event of an application restart
• Traffic: Traffic will come in bursts and maximize at about 1500 iops between 10 separate hosts.
• Reservations: Is Equallogic a “true” H/A solution considering it does not support synchronous replication between completely separate hardware? Are the competitors claims of Datacore’s “unprotected cache” well-founded? (Datacore insists in H/A mode that all cache is synchronously written and requires a commit from its H/A partner before committing to client.)
• Storage size requirements are small, so I’ll pay for SAS performance over SATA terabytes.

End update.

Update II: The anonymous reader comes back with more crucial detail:

Let’s pretend the budget is around $60k-$70k. I know the two finalists can provide an acceptable degree of HA, DR, and iSCSI performance at that price. What products should one be looking at from HDS/EMC/NetApp? They were not considered initially for the perception of being unaffordable.

End update II.

Update III:

The plan is for an H/A setup in a class 1 datacenter with asynchronous replication over an existing DS3 (..but dark fiber is in the works) to a remote site.

All things considered, the question could be framed, “Whom/What should be demanded for trial?”

End update III.

The StorageMojo take
It is interesting that this customer is NOT looking at the traditional OLTP storage vendors. This is a business-critical application - the company is handling Other People’s Money.

What are the questions the reader should be asking of vendors? How should the problem be framed? I surmise that price is an issue. Where else might the reader go?

I welcome comment from vendors, but please do us the courtesy of identifying yourself as such.

Comments welcome, of course.

6 weeks ago StorageMojo covered the leaving-stealth-mode non-announce of Atrato’s new storage box. I spoke to Dan McCormick, Atrato’s co-founder and CEO a few days ago for an update.

They’ll have more details at SNW. But here’s what I found interesting.

Density and capacity
The new Atrato box is 3U, not 5, and has about 200 2.5″ drives, for 50 TB raw. With the new 500 GB 2.5s coming out they’ll be able to do 100 TB.

That blows away the density of EMC’s soon-to-be-announced Hulk box. And with the declining delta between 3.5″ and 2.5″ drive capacities, the Atrato box should increase their capacity per rack unit lead.

Performance
In a refreshing change from normal industry practice Atrato quotes IOPS to disk, not cache. Thus their quoted 10,000 IOPS is a real-life number. Dan said that one user got up to 20,000 IOPS after tuning their app.

Apps with big files and large I/Os need disk I/Os, not cache I/Os. Most controllers turn off cache when they see large I/Os anyway. Quoting cache IOPS to their market would be a mistake.

Power
Atrato claims an 80% reduction in power per I/O. 80% of that is due to the power efficiency of 2.5″ drives. The remaining third though is their own special sauce.

Virtual drive hospital
When a drive starts acting up - and with 200 drives that doesn’t take very long - their software “pulls” the drive and tests it. If the drive is failing they leave it alone, but Atrato has found that over half the problem drives can be put back into service.

The StorageMojo take
Still cool. An interesting metric will be uptake into space and power constrained enterprise data centers. If power really is an issue - and while I’m sure it is at some level, the priority is the question - I’d expect to see all the big NYC data centers testing these things within 90 days.

Comments welcome, of course. Dan also commented that StorageMojo’s original Atrato post was the best researched and most insightful of all the reportage they saw. Flattery works.

Mobile computing. Cloud computing. Client-server computing. Green computing.

A new mainframe. A 9U supercomputer. Scale-out clusters. High-bandwidth RAID controllers. Multi-core processors. Massive memory servers.

Facebook. YouTube. Twitter. Blogging. MySpace. Google apps.

The Next Big Thing: there is no Next Big Thing
Punctuated equilibrium is an evolutionary theory that posits that long periods of “normal” evolution - stepwise enhancements that fine-tune environmental adaptation - are interrupted by big events - asteroid strikes, climate change - that engender explosions of mutation and variety. These variations then get whittled down by the pressures of the new normal.

The current hype around “cloud computing” is a case in point. Much over-heated prognostication about how this changes everything. But does it?

Cloud computing will host a certain class of applications that

• Have low bandwidth requirements
• Only require ~99% uptime
• Are latency insensitive

Both “low bandwidth” and “latency insensitive” are relative measures. They will change over time. We’ve always had those applications and always will.

In the 1980’s those requirements fit PCs and Novell LANs. In the 1990s they fit browsers and 56k modems. Today they fit smart phones, sociall media, some web-hosted productivity apps and cool data storage

But there will always be important apps that don’t meet these restrictions and never will. Plus there will be new products that provide “cloud” advantages of cost and scale without the disadvantages of security, latency and bandwidth costs. Is a local “cloud” still a cloud?

The StorageMojo take.
Our human pattern-recognition hardware craves simple patterns and big stories - even if they aren’t there.

What is actually happening is that we are seeing an explosion of new computing forms to take advantage of many new market niches. Old forms will either bend - as the mainframe has - or break - as the minicomputer companies did.

Implicit requirements are becoming explicit. Market demand is great enough to support a larger number of niches. Application users are gradually understanding what they need - as opposed to what they’ve always wanted.

Out of this stew will come the new normal. For a few years anyway.

Comments welcome, of course.

Do you help buy, spec or evaluate storage and related products?
Please take a few minutes to take a survey that I hope will make potential advertisers thrust large piles of green my way. Click on the “We value your opinion” button under my picture to the right.

It is a standard reader demographic survey. Its great virtue is that it is short. 5 minutes tops.

Update: Olga from IDC tells me that the StorageMojo reader response has been outstanding:

. . . congratulations on having such a supportive audience!

Kevin predicted it could take several weeks to get to the right sample size. Instead it took 3 days. So I’m taking the survey down.

Thanks to all who responded! End update.

You can tell me what you like best, too
I put in a couple of questions too: what you like about StorageMojo; what you want more - or less - of in fine storage blogging.

There’s a serious side too
The analyst community gets the huge majority of its money from vendors. That’s not a bad thing, unless you are looking for unvarnished vendor analysis. Then you have to wonder about the analytical rigor. Some firms are simply mouthpieces for their clients. What’s the point?

I take money from vendors too. Mostly the younger, more innovative vendors, yet vendors nonetheless. So part of my intent has been to build a revenue stream that is advertising-based. Yes, money from vendors again, but mediated by a sales force and competition for the page views. If somebody gets mad they pull their adverts and somebody else takes their place. I’m cool with that.

That’s where you come in. Click on the link, click the check boxes, and strike a blow for independent storage blogging.

The StorageMojo take
If you like what StorageMojo does, please complete the survey. I’ll be watching the results, trying to figure out how to make you love StorageMojo even more. Or at least a little. Namaste.

Comments welcome, of course.

Back in ‘96, when I was flogging FC networks for Sun under NDA, the most common objection was “I don’t want another layer to manage.” Despite that FC became successful in big enterprise IT shops. But the objection is still valid and a major factor, with price, in the low uptake of FC in smaller shops.

Is FCoE (Fibre Channel over Ethernet) the answer?

FC vendors are - reluctantly - hoping it is
The future of pure FC looks pretty bleak in the long term. 10 GigE is coming down the cost curve just as earlier generations of Ethernet did. The volume Force is with them.

As 10 GigE gets cheaper its total available market gets larger. It may not be optimal, but for many shops “good enough” is good enough.

FC partisans aren’t quitting. 8 Gbit has just started shipping, 16 Gbit is on the drawing boards and there are noises about future generations beyond that.

FCoE follows in the footsteps of VTLs
When 1 Gbit FC started rolling out in ‘97, it was 10x-20x the speed of the then hot 100 Mbit Ethernet in either its full or half duplex flavors. And today - 8 Gbit FC is slower than 10 GigE. It is cheaper, but for how long?

An Emulex VP explained at a recent conference that enterprise shops have well-developed processes for managing FC SANs. FCoE enables shops to continue using those processes minus the fibre. The problem: FCoE won’t be ready for volume deployment until 2010 - if you believe the current schedules.

Any technical problems could easily drop FCoE into 2011, leaving Emulex, Qlogic and Brocade with a 3+ year chasm to cross. The Emulex VP tried to sound enthusiastic about FCoE but wasn’t succeeding. Maybe his teeth hurt.

The StorageMojo take
Enterprise data center inertia is a powerful market driver. Witness the success of VTLs. It’s understandable: they have work to do. Can’t be overhauling the engines in mid-flight.

But Wall Street isn’t as understanding as StorageMojo. FC is topping out, so where is the growth going to come from for FC companies? Especially when new iSCSI, Infiniband and pNFS products are coming to market in the near term.

The current economic malaise will force companies to get tough on data center requirements. The “good enough” standard will be the only standard for apps that aren’t absolutely core to business success.

Comments welcome, of course.

Moving to a small town in northern Arizona from Silicon Valley has enriched my perspective on many things, including how the industry develops products. The consensus is that if we take datacenter technology and put in enough defaults it will be “simple” enough for consumers. Wrong.

Memo to developers: it is ALL consumer IT
The consumerization of IT is usually means the adoption by IT of high volume consumer technologies. The PCI bus, Microsoft Windows, USB, x86, SATA disks and Wi-Fi all started in the consumer space and displaced more sophisticated and expensive IT.

But consumerization also means taking tech first developed for IT and making it easy enough for consumers. Ethernet LANs, symmetric multi-processing, external disk systems (well, really only Drobo) and what we used to call “office automation” software are now usable by non-geeks.

Pro vs amateur
Amateurs like GUIs. Pro’s like CLIs. Why do we have both on “enterprise” products? Because we are all amateurs - at something.

The third shift guys are all amateurs. They may want to be “professional” but they aren’t now.

Backup: the highest failure rate in IT?
Who knows how good the numbers are. A 40% enterprise backup failure rate is frequently bandied about. Whatever the “real” number is, it isn’t good enough.

If “professionals” with “industrial strength” backup hardware and software can only achieve a 60% completion rate - a failing grade anywhere - why does it surprise us that only a tiny percentage of small office/home office people backup regularly?

And further, why do we assume that SOHOs will never backup? “Americans will never wear seatbelts.” “People will never recycle.” “SOHOs will never backup.”

Yet the record is clear. If you take an education and ease of use approach, people will change their behavior. They will wear seatbelts. They will recycle. They will even learn to deal with PITA child seats. And they will backup.

But not if it is presented as a “junior” enterprise backup. Make it easy and affordable. Mostly easy. And people will do it.

A couple of backup products that work
On ZDnet I reviewed a Windows backup product that I could recommend to any small business here in the red rock-strewn desert, Backupkey. Plug it in, hit “enter” twice, and all your valuable Windows data gets copied.

Did this simple, useful product come from Boston? Silicon Valley? Redmond? Denver? Nope. Charleston, South Carolina? Bingo!

I suspect Backupkey got built there because the developer actually knows small business people. Knows their frustration and their intolerance for stuff that doesn’t work as they think it should.

Most Windows backup software is simply dumbed-down “real” backup. Backup sets. Incrementals. Images. Bootable. Whatever. But non-IT folks don’t know those words or concepts. Why can’t it just work?

On a Mac both Carbon Copy Cloner and SuperDuper work great and are almost easy enough for complete idiots to use. Partial idiots only, please. Apple’s Time Machine, which I finally set up last night on a new 500 GB USB/eSATA drive, is totally easy. Mindless bliss.

The StorageMojo take
I cringe every time I hear the big companies proclaim a new focus on the SMB market. Usually it is some shrunk-down enterprise product with incentives for the channel.

But what doesn’t change is the thinking behind the product. The assumptions about the consumer - “like us, only dumber” - and how the problem they are trying to solve rarely get the kind of re-think that went into Time Machine.

But the logic is inescapable: the more pervasive IT becomes, the more the technology must adapt to people. Backupkey does that for low-end Windows backup. Time Machine does that for Mac OS X. Who, and what, is next?

Comments welcome, as always.

The P4P working group demo’d their work Friday at the Distributed Computing Industry Association show in New York. Not only did they show 2-3x faster downloads, but they also cut the average number of inter-metro hops - the expensive kind - from over 5 to less than 1. Cool.

The P4PWG idea is that if P2P is both cheaper for ISPs and faster for users we will all have a happier Internet. Folks from the Yale CompSci department - Haiyong Xie, Y. Richard Yang and Avi Silberschatz - along with Verizon and Pando Networks, cooperated on the demo.

The P4PWG includes AT&T, Verizon, Pando, BitTorrent, Cisco and LimeWire among others. The cable companies are there as observers. The P4P work is an open standard with the hope that all ISPs and P2P networks will endorse it.

How does it work?
The tech papers aren’t available yet on the web, but this is what I’ve pieced together from an afternoon’s websurfing. Update: Wide-awake reader Paul found this P4P Overview on Ars Technica. Thanks Paul! End update.

P2P is network oblivious. When you start downloading streams they might be from anywhere, regardless of network cost. The problem is that big routers are costly and smaller routers are much cheaper, not to mention undersea fiber.

What P4P is inject some knowledge into the P2P network so peering decisions are made more intelligently. It looks like a network version of locality of reference.

Implementation
There are at least 2 ways to deliver network awareness to peers. Here’s one of them.

A peer-tracker (pTracker) and an Internet tracker (iTracker) are added to the P2P network. A peer requests peering information of the pTracker, which has knowledge of local (metro area) and recent non-local resources. The pTracker sends back an edited server list and the peer goes its merry way.

If the resources aren’t local and the pTracker doesn’t know the network topology, it pings the iTracker, which returns high-level peering suggestions. If locality of reference works as well in cyberspace as it does with other data the pTracker won’t be querying the iTracker very often.

It is expected that the pTracker will be maintained by the P2P network, while the iTracker could be maintained by the ISP, network or a trusted 3rd party. This should preserve help P2P user privacy, although the *Tracker names certainly won’t reduce user paranoia.

Guys, how about something less Big Brotherish? PeerServer and RoutServer? Just a thought.

The StorageMojo take
As file sizes continue their secular trend upward the need for P2P will continue to grow. By aligning ISP, telco and user needs for faster and more efficient P2P the P4PWG has pulled off a win/win/win situation.

A less obvious benefit of this work is on VoIP networks, which are also P2P. It doesn’t take much to degrade VoIP quality. To the extent that it enables improvement in P2P network node selection, the P4P project will benefit the rapidly growing population of VoIP users as well.

Kudos to the P4PWG and especially the Yale team.

Comments welcome, of course. Images courtesy of the P4PWG.

It didn’t win Best Paper honors at FAST 08 - IIRC it was An Analysis of Latent Sector Errors in Disk Drives (the link is to the StorageMojo review of that excellent paper last month) but I really like the thinking behind Pergamum: Replacing Tape with Energy Efficient, Reliable, Disk-Based Archival Storage.

Written by Mark W. Storer, Kevin M. Greenan, Ethan L. Miller (UC Santa Cruz) and Kaladhar Voruganti (NetApp) the paper discusses a prototype that

. . . is a distributed network of intelligent, disk-based, storage appliances that stores data reliably and energy-efficiently. While existing MAID systems keep disks idle to save energy, Pergamum adds NVRAM at each node to store data signa- tures, metadata, and other small items, allowing deferred writes, metadata requests and inter-disk data verification to be performed while the disk is powered off.

They call the appliances tomes.

Tape: where data goes to die
One of tape’s big advantages is that it uses no power at rest. Any disk-based tape replacement will have to come as close to the same ideal.

The tomes use a single hard drive, an ARM-based processor board with NIC and NVRAM. Total power use - when powered up - about 11.5 watts, less than 15k FC drive. With tighter code, a slower drive and more integration, I’d bet they could cut that in half.

The single disk drive means that tomes must be used in groups to enable distributed RAID techniques and exchange of algebraic signatures to ensure inter-disk recovery. The paper goes into those techniques in detail.

NVRAM

The purpose of the NVRAM is to provide low-power, persistent storage; operations such as metadata searches and signature requests do not require the unit’s drive to be spun up.

. . . the NVRAM primarily holds metadata such as algebraic signatures and index information, flash writes are relatively rare; flash writes coincide with disk writes.

The Ethernet interconnect is important - by using cheap unmanaged switches for fan out, high aggregate bandwidth, exceeding that of current tape libraries, is easily and inexpensively achieved. The use of power-over-Ethernet would further reduce costs, especially if the system used 4200 RPM drives.

The StorageMojo take
Most of the disk vs tape discussions look at the disk device vs tape cartridge cost issue - and they aren’t that different even today. But the tape library market is a $4-5 billion market. A disk-based alternative to slow tape libraries could take a big chunk of that.

Further, this design could be integrated into a single disk controller board, creating a disk with a single Ethernet port and incredible packaging and manufacturing economies.

If Seagate were smart they’d jump on this. This is a major opportunity to drive another significant consumer of disk drive units - without encroaching on existing OEM customer businesses. That doesn’t happen very often.

Comments welcome, as always. Pergamum was an ancient Greek city known for its sizable library, second only to the library of Alexandria.

NetApp has formally changed its name from Network Appliance Inc to NetApp Inc. The name change is part of a larger effort to raise their awareness among what they call the Storage 5000 - the 5000 largest storage customers worldwide.

I’m in NYC today and tomorrow at the annual NetApp analyst meeting, flown out and put up at NetApp’s expense, wined and dined, charmed by their capable director of analyst relations. It works - a warm feeling about NetApp suffuses every atom of my being. Don’t worry, it won’t last.

The StorageMojo take
Great technology poorly marketed loses to mediocre technology well-marketed almost every time.

NetApp is very serious about taking more market share, just as EMC is ramping up a strategy to take more NAS market share as well. The market also-rans will take most of the initial damage, but both companies mean to take a piece of the other.

Storage consumers will benefit the most.

Comments welcome, of course.

How flash is really going to affect the storage industry is becoming clear. The short take: not as big a deal as flash vendors hoped. The longer take: There won’t be much of a mid-range flash market; instead we’ll see either costly fast flash or cheap slow flash.

There are lots of theories about how flash will alter the mass storage landscape. This is mine.

The flash write problem
The fundamental flash problem is the slow writes. There are 3 elements to the slow write problem.

• Flash has to be erased before it can be written. Every write operation is really 2 write operations.
• The writes are large. Typical block sizes are 128KB to 256KB. Writing a single page requires writing - after erasing it first - the entire block.
• The write bandwidth to a single block is less than a slow disk. High bandwidth writes requires parallel paths to multiple blocks.

These problems can all be engineered around.

• Garbage collection-like algorithms can be extended to enable a supply of erased blocks
• RAM backed by a small battery or capacitor can buffer writes for later re-writing to flash
• Controller chips can be built in high volume with multiple data paths

But at what cost? The first two require well-engineered software and some sort of CPU to run it. Since it is software it will have bugs. Can it be any more reliable than current drive firmware?

The dilemma
For enterprise use, flash-based SSDs need to be rock-solid, which implies a lot of careful and costly engineering. For consumer use, they need to be very high volume, which means low-cost.

It is a similar problem to RAID controllers: very low-end RAID controllers aren’t reliable enough for enterprise use. They also aren’t cheap enough - or easy enough - for consumers to buy in volume. RAID controllers have engineering problems similar to flash translation layers.

Making flash drives look like disks makes them easy to integrate, but if you really need performance it also makes them costly - like the $10k for the flash drive EMC is using in the Sym.

Flash in the disk controller?
As I’m writing this a NetApp exec says that flash will be disruptive because by placing flash in a disk controller they will reduce the need for the costly and highly profitable fibre channel disks. That could be correct. It sounds smarter than sticking flash on a disk.

The StorageMojo take
Despite the miracles of cost-reduction and integration the industry regularly performs, some things, like power provisioning, don’t get cheaper. High-quality software engineering doesn’t either. That is what high-performance flash drives require.

The high-performance consumer flash drive seems to be a mirage. I’d like to be proven wrong, but today’s notebook SSDs don’t offer superior application performance and cost 10x as much. Hardly a recipe for success.

Update: Intel is planning to offer “high-performance” flash drives with partner Micron. I saw an impressive demo - is there any other kind? - at the Storage Visions conference. But with the early marketing missteps of Samsung, it looks like the consumer flash drive may fall off the hype cycle into a deep ditch. Flash drive marketers: now is the time for precision marketing if you ever hope to establish a mass market. Consumers remember unkept promises. Until you are cheaper. End update.

Comments welcome, as always. Also check out BPLRU: A Buffer Management Scheme for Improving Random Writes in Flash Storage by two Samsung researchers, Hyojun Kim and Seongjun Ahn for a nice intro to flash issues.