No CES for me this year, but did attend Storage Visions 2013. Some cool stuff there.

Panasonic’s Blu-ray RAID archive robot
Panasonic has a Blu-ray-based storage system that looks interesting. Imagine 12 Blu-ray discs in the striped RAID configuration. The RAID gives the Blu-ray discs speed and competitive capacity.

Specs:

• 200MB/sec throughput
• 1.2TB per cartridge of 12 discs
• 108TB per archive of 90 cartridge
• 60 second access time
• 6 Watts standby, 100 Watts R/W
• 6U rackmountable box

The 12 Blu-ray discs are in a cartridge about the size of a tape. The disc unloader places them into 12 different Blu-ray drives for reading and writing.

But can Panny’s Blu-rays with a claimed 50 year life be trusted? I’d like something better.

The 1000 year DVD
That something better may be here. It is a DVD, and soon to be a Blu-ray disc, disc, with a claimed life of 1000 years.

The key is a writable layer that, unlike existing writable DVDs, uses an inorganic mineral layer to store the data. Doug Hansen, CTO, told me that they use a stone-like material.

Most rocks are silicates, oxides and metalloids. M-DISC’s material at nanoscale looks like igneous rock. Extremely thin layer and fine-grained – 100nm thick – horizontal scale 1/2 micron and smaller for bluray.

The laser burn heats the mineral layer so there’s a bit of melting and surface tension creates a hole. The material migrates to create a berm.

Because it is physically burnt into that inorganic mineral layer of the DVD media, it cannot and will not shift or change over time. The layer will last as long as the disc’s tough polycarbonate plastic. Most credible optical story I’ve heard.

Ridata is producing the discs, with Blu-ray versions expected this summer. All current LG DVD burners are warranteed to properly burn M-DISCs, which means you take your chances with other brands today.

Quotium
Quotium’s StorSentry software analyzes the quality of tape storage on a real-time runtime basis. With LTO tapes warranteed for only 200 complete read/write cycles, and costing almost as much as a disk drive, anything that can extend the useful life of a tape while protecting your data is a Good Thing.

Media Entertainment & Scientific Storage
MESS is a new group looking at long-term storage stack from media all the way to Digital Asset Management. Paul Evans, a long-time big systems guy, is heading it up.

The StorageMojo take
For some reason my attention was on the long-term storage issues that media and entertainment folks must grapple with. Given the massive carelessness of the movie industry over the last 100 years – and the loss of thousands of movies – I don’t believe that republishing all data every 5 years is a viable long-term strategy.

Long-term readability is a major problem with any digital medium. Assuming the media is good, can you count on a machine able to read it?

Optical has an important advantage because optical technology is everywhere and easy to replicate. Compare that to the specialized head/media engineering of tape – typically promising compatibility over 2-3 generations – and you’ll have a much more difficult time reading an LTO-5 tape in 20 years.

Panasonic should get together with M-DISC to produce a true long-term, high-performance storage system. But in the meantime, tape is still a major piece of the long-term storage puzzle, and Quotium’s software seems like a step in the right direction.

But MESS is correct to focus on the entire stack. Even if you can read the media, can you make sense of it?

One of these days the relentless growth in the density and speed of digital media will slow. When that happens it will become practically impossible to maintain existing data without a long-term strategy.

While we can hope that slowdown is decades away, the earlier we start thinking about it the better prepared we will be. Our digital civilization depends on it.

Courteous comments welcome, of course. Come to think of it, maybe the fact that I moderated a panel on long-term storage put me in that frame of mind. Go figure.

“Call home” support has been standard in large arrays for 15 years. But Nimble Storage has kicked it up a notch with their advanced telemetry data from installed systems. It gives new meaning to the term “after-sale support.”

Talk to me
Their system gathers configuration details and more. Feature – such as snapshots and backups – use. Volume protection. Application performance. Updated every 10 minutes.

Here’s a partial screen shot of representative data:

Nimble now has over 4TB of customer use data. Customers opt in to the program. Over 80% have.

Uses
When a problem is detected, Nimble’s software creates a trouble ticket. Then a human gets involved.

It might be as simple as having more Ethernet links on one of the active-passive storage controllers, causing asymetrical performance. Perhaps a volume is not protected. Or the replication policy won’t meet RPO objectives.

Email alerts flag issues to customers. Nimble support engineers can login remotely for real-time troubleshooting.

But that’s not all!
Detailed information on usage allows customers to compare their usage to average usage. Nimble can also look at how customers with the most efficient utilization manage their systems, automating the documentation of best practices.

For example, backup: most customers are retaining snapshots for more than a month. Over 50% of customers replicate workloads for DR.

The StorageMojo take
This is what 21st century support should look like. The best infrastructure is invisible – until it breaks – and the best support keeps the infrastructure from breaking.

The bad news: customers don’t want to buy and manage storage arrays. The good news: they want fast and reliable access to their data.

The Apple model of “it just works” breaks down if the applications are too complex, as many data center applications are. But that doesn’t require vendors to throw all the load on customers.

Automating the capture, review and disposition of system data gives a vendor important advantages:

• Perceived reliability goes up, a fact established with early phone-home experience.
• A stronger customer relationship makes follow-on sales easier and is a competitive barrier.
• The “virtual user group” of shared data enables users to get smarter, faster using their Nimble arrays.
• The real-time remote troubleshooting gives customers help when they need it most – not 4 hours later.

Courteous comments welcome, of course. What other support strategies have you experienced that either worked well – didn’t? If you want to learn more about Nimble, I did a video white paper on Nimble’s architecture last year.

Several readers have written in lately with roughly the same question: what’s the best way to build cheap home bulk storage?

Here’s how 1 writer put it:

I was hoping you could provide me with some advice. I have so many external drives that I have to swap. I have almost 10 Terabytes of data – mostly movies that I would like to consolidate into one “volume”. I was thinking about building a 5-bay JBOD Raid (a cheap enclosure) and was also thinking of using MacZFS to handle the storage pool part.

Price is important. Performance isn’t.

How would readers propose to do this on their favorite OS? The folks asking this question aren’t full time sysadmins – they already have their ideas – so let’s not get too esoteric.

The StorageMojo take
This question has been vexing me as well. I have a small Thunderbolt array – a Promise Pegasus R4 – hooked up to a new iMac for video editing. I’d like to reconfigure it from RAID 5 to multiple RAID 0 stripes for speed and capacity. The question is how to back up those vulnerable RAID 0 partitions.

It’s looking like large FireWire drives are the right answer. But maybe you have a better one.

Courteous comments welcome, of course.

I sat down with Nimble Storage co-founder and VP of engineering Varun Mehta to discuss their architecture – and shoot some video. Varun has been part of several Valley success stories – NetApp, Sun, Data Domain – and has a first hand perspective on disruptive technologies.

Varun and co-founder Umesh Maheshwari – a brilliant architect and a very nice guy – designed the Nimble product that he discusses. Take 4 minutes to learn more about Innovations in Storage Architecture at Nimble Storage:

Or you can see it in HD on YouTube.

The StorageMojo take
The Nimble guys have great technology, but they’ve also put together a compelling value proposition: collapse 3 time-consuming and complex workflows – primary storage, backup and archiving – into 1 appliance. Include all the needed software, price it well, target under-served mid-sized companies and you have a recipe for another Valley success.

The tech trends they’re riding will only get better. But the business trends are in their favor as well. SMB’s today have many TB of data and little staff to manage it – or capital to invest. With Congress ensuring that America operates well below capacity for years to come, the times favor thrifty solutions like Nimble’s.

Courteous comments welcome, of course.
Nimble bought my time for this video, but I made all editorial decisions.

Quantum announced a new deduplication appliance series – the DXi 6701 and 6702 – that claims exceptional scalability. Why? Because it uses technology from Quantum’s StorNext cluster file system.

Scale out
Quantum says the units grow from 8 to 80TB of usable RAID 6 capacity with no subtractions for landing areas, hidden reserves or multiplication. And they say they’re fast: 5.8TB/hr using VTL or OST; 5TB/hr for NAS; all dedup at wire speed.

The only difference between the 2 models is that one has 1Gig Ethernet and the other 10Gig. All the software is included in the price: NAS, OST, VTL, tape support, replication and client side dedup option.

List prices start at $56k. Quantum sells through the channel, so that’s a maximum.

The StorageMojo take
The DXi 670x is a good example of the power and economy of scale-out vs scale-up. The cluster file system technology underlying it enables the 10x capacity expansion with high performance and low-costs.

With a scale-up approach hardware volumes would be lower with hardware and software qual and support costs higher. That Quantum owns the underlying technology makes their job that much easier.

Quantum’s market power is a fraction of EMC’s Data Domain. But the power of their architecture’s advantages in performance, flexibility and cost point to a larger trend.

The problem with Quantum’s marketing is that they only play the price/performance card. Important, no doubt, but by ignoring the fundamental advantages of their scale-out architecture, they let the competition sidestep their long-term problem: they don’t scale.

Winning in the development lab is only part of the battle. Helping customers appreciate – and making competitors react to – the differences, is needed to win in the market.

Courteous comments welcome, of course.

Forget the flame wars over moving window versus fixed block de-duplication. A recent paper, A Study of Practical Deduplication (pdf) from William J. Bolosky of Microsoft Research and Dutch T. Meyer of the University of British Columbia found that whole file deduplication achieves about 75% of the space savings of the most aggressive block level de-dup for live filesystems and 87% of the savings for backup images.

Presented at FAST 11 – and winner of a “Best Paper” award – the researchers looked at file systems from 857 Microsoft desktop computers over 4 weeks. Researchers asked permission to install rather invasive scanning software.

The scanner took a snapshot using Window’s volume shadow copy service and then recorded metadata about the file system itself. The scanner recorded each file’s metadata, retrieval and allocation pointers as well as the computer’s hardware and systems configuration. They excluded the pagefile, hibernation file, the scanner itself and the VSS snapshots the scanner created.


During scanning each file was broken into chunks using both fixed block or Rabin fingerprinting. They also identified whole file duplicates.

Rabin uses dynamically variable block sizes to maximize compression. Figuring out where to break the file adds to the overhead.

The resulting data set was 4.1 TB compressed – too large to import into a database – and was further groomed to lose unneeded data.

De-dup issues
De-duplication is expensive. You’re giving up direct access to the data to save capacity.

The expense is in I/Os and CPU cycles. Comparing each chunk’s fingerprint to all other chunks is nontrivial. De-duplication indirection adds to I/O latency. A file’s chunks are scattered around, requiring small and expensive random I/O’s to read.

Older techniques, such as sparse files and Single Instance Storage, are more economical even if their compression ratios aren’t as high. Fewer CPU cycles, less indirection and good compression.

The StorageMojo take
If capacity is expensive – read “enterprise” – and I/Os cheap – SSD or NVRAM in the mix – fancy dedup can make sense. It is at the margin of capacity cost and I/O availability that the value prop gets dicey.

Low duty cycle storage – SOHO – with plenty of excess CPU and light transactions could use deduped primary storage. But with a 10 TB of data to backup, most users would’t notice the difference between whole file and 8KB Rabin.

It’s the price tag and user reviews the SOHO/SMB crowd will be looking at.

Courteous comments welcome, of course. The paper also included some interesting historical data about Windows file system that I covered on ZDNet.

Talked to Nimble Storage a few months ago. The 1st time they sounded cool and now I know why.

What they do
Nimble builds a converged storage appliance out of commodity hard drives and SSDs that offers high performance – is there any other kind? – and iSCSI, backup, a form of dedup and WAN replication. The pitch is EqualLogic & Data Domain merged into a single low-cost appliance. Only better.

• iSCSI + dedup
• Capacity-optimized snapshots
• SATA + flash instead of high-rpm drives
• Can run off a remote snapshot

EL & DD sell a lot of kit, so this could work.

Claim to fame
Cache Accelerated Sequential Layout is what Nimble calls their secret sauce.

CASL combines a variable block size, in-line compression, application-specific block sizes and checksum and compression data kept in the block header. They coalesce the blocks and only write in full stripes to disk.

The box has a large flash-based cache where the full stripe writes are also written, overcoming the small write performance hit that flash shares with parity raid. This also insures a high percentage of cache hits on the first read.

The system maintains an index of where all the blocks are written. Typically, this index is also held in flash for maximum lookup performance.

App-specific block sizes
Nimble uses of variable block sizes to improve performance. For example, the last three versions of exchange have all used different block sizes. CASL recognizes the different versions of Exchange and dynamically adjusts its block size to the best fit.

They claim a 2x performance advantage on Exchange databases.

Coalesce
They take the variable size blocks then coalesce those blocks into big chunks and write to flash. They write in large blocks – full block writes to flash and in full stripe writes to disk. Result: fast reads & writes across both media

Their page sizes are variable but small, ranging from 4KB to 64KB. The greater granularity means that frequent snapshots are much smaller than large page size systems like EqualLogic.

The StorageMojo take
There’s no reason that data protection should be separate from data storage. We’ve been moving towards integration since the CDP craze.

The average business wants to store and protect their data and they don’t want to spend much time or money on it. Nor should they.

With powerful commodity processors and nickel-per-GB storage there’s a huge market for a box that – or 2 or 3 boxes – that

• Stores terabytes of data
• Protects that data with local replication and frequent snapshots
• Auto-connects to cloud storage for DR and archiving
• Doesn’t confuse users with LUNs and stripes
• Offers Time Machine like data recovery to end users

It will look like magic – as any sophisticated technology should – and you’ll buy it at Office Max. As with any volume product the key will be architecting to maximize the user experience at an affordable price point.

Nimble certainly has the right idea.

Courteous comments welcome, of course. I don’t know which analyst the Nimble guys are blowing their money on, but it isn’t me.

This came in over the transom from a semiconductor engineer. He’s wants home archive storage and is wondering why no one seems to sell it. I’ve been grappling with the same issue.

Here’s an edited-for-length excerpt from his letter:

I use RAID server products from Netgear and QNAP and have been searching for my ideal server product. I don’t understand why it doesn’t exist. My hunt is for a server with integrated error checking to ensure that bit errors can be caught and rectified. My goal is a system that secures the integrity of the files stored upon it. As far as I am aware this kind of functionality does not exist and isn’t discussed anywhere.

For example, once I have a video file (which can be the result of many hours of editing) it isn’t subsequently modified, just read on occasion as required. I don’t want any bit errors on this file – every change is just a corruption. I backup my files, I just want to make sure that what I am backing up is the same as when it was first written.

I don’t want to run a piece of software over the network, I want the server to run a check and fix any errors that it finds. It is something that I would gladly pay a premium for. I think there is a market for this as there must be a lot of users who have files that never change.

Frustrated in California

Dear Frustrated
You’ve identified the key problem: RAID systems aren’t for archives. RAID keeps your data available after a disk failure – sometimes 2 disk failures – but they do not ensure long-term data integrity. Or even short-term integrity. Not their thing.

This is what archives – traditionally tape – are for.

But tape is a tough sell to the home & SOHO market. Low-end drives – DAT, SVR, VXA – cost several hundred dollars plus the tapes. DLT/LTO drives start around $1200 with $40 tapes.

You can buy an external Blu-ray burner for those prices and 50 GB media for a few bucks each. On sale BR media is starting to reach the 5¢/GB level of 2 TB drives, and the longevity should be better.

But I have over 500 GB of video alone. Shuffling 10 or more BR media – 20 if I’m paranoid – reminds me of floppy backups. Yuck.

The current plan:

1. Create zip archives of files and folders I want to preserve.
2. Back them up to 2 local hard drives.
3. And ship them off to my online backup provider.

What I don’t know is how robust zip archives are. There is a 32-bit CRC, but what does that do for a 10 GB folder of PDFs?

Also, I wonder about the advisability of zipping compressed formats such video and audio files. It might be worth the computational overhead and the possible larger files if the zip file is robust.

The StorageMojo take
Frustrated isn’t the first home user to want an archive and he won’t be the last. Hundreds of millions of home users will see the need over the next decade.

The question is whether or not someone can design a commercially viable system for home and SOHO use. It is obvious that drive vendors have the cost advantage, especially with the advent of easy and cheap USB drive docks, if they build a disk drive designed for that purpose.

An archive drive can be slower – 4200 or even 3600 RPM – and less dense. Optimized for large transfers. Slower, cheaper actuators and drive electronics.

Single platter 2.5″ 7mm drives could be the sweet spot: minimal head cost; slim cartridge-like form factor; and much faster than optical. Then it is just a matter of getting the volumes up and the costs down.

But that’s just one idea. Please comment on how you would solve the home and SOHO archive problem.

Courteous comments welcome, of course.

Storage Newsletter reports that Tape Drive and Media Revenues Decreased by 25% in 2009. The data comes from a report by the Santa Clara Consulting Group.

The numbers show us how old tape formats die: slowly. While the overall market for drives and media was $1.58B it was split among LTO, DLT, DAT, 8mm and even, gasp, QIC.

The good news: drive sales were $629M, suggesting that media sales will continue for years to come. LTO had over 83% of drive sales – $534M – with DAT (!) drives making most of the rest – $69M – and DLT much of the remainder.

The media numbers are revealing. Overall, media sales were only about 50% greater than drive sales or $955M. But in the case of DAT, media sales of $45M were less than drive sales. Buyers aren’t making much use of their new drives.

8 mm and QIC bring up the rear. Somebody bought over a million units of AIT media and over $16M of QIC media.

The StorageMojo take
The long tail of tape is longer than I’d thought. There must be ancient systems out in retail or OEM equipment that use the media. Military, too.

But why that 25% drop in the overall tape market? I’d need more time series data to draw any firm conclusions, but here’s what I’d look at:

• The Great Recession. The overall slowing in business and capital expenditures is a piece of that. But the world economy did not decline 25%, thank goodness, so that can’t be the full cause.
• D2D. Data de-duplication is aimed at making disk competitive with tape. Looks like 2009 was the year it took a byte out of tape.
• Tape capacity growth. The LTO folks have been increasing LTO tape capacity at a rate near that of disk. More data, fewer tapes. Disks, of course, wear out, so the replacement market is huge.
• Drive cost. At $3-$4k for an LTO 5 drive and $125 per 3 TB tape, the use of tape is moving upmarket, which means smaller volumes.

Some people love what tape does. But others don’t: I haven’t seen a new tape or disk-based camcorder introduced in over a year. Everyone is going to flash.

$1.5B markets don’t die overnight – even dropping 25% a year. Tape will be around for a long time to come.

Courteous comments welcome, of course. I kicked off DLT for DEC back in 1991 and have always wondered why Quantum just rolled over for LTO instead of fighting. Oh well.

After 9 years and $100,000,000, holographic storage pioneer InPhase Technologies has shut down without ever shipping a product. Their office building was also seized for non-payment of back taxes.

They assured me that the product would ship in May, 2008. It didn’t. Reportedly many employees took pay cuts – or no pay at all – to help keep the company going.

It is a sad and ignominious end to a brave technology experiment. And a warning to anyone trying to replace disk drives as random access storage.

The 40% problem
At a 40% annual capacity growth rate hard drives are difficult to catch. When InPhase started showing their initial prototype, 300 GB wasn’t much less than hard drives. But 3 years later 300 GB is less than 1/6th the capacity.

Nor was it very speedy: 20 MB/sec. You can do almost as well with a USB thumb drive.

InPhase planned to take the drives to 1.6 TB and 120 MB/sec. If they could ship that today, they’d have a competitive product.

In the meantime, cheap hard drives and cheaper hard drive docks make it easy to use bare drives for backup and data transfer. The market for 300 GB removable drives withered before it had a chance to grow.

As I wrote 4 years ago:

I love holographic technology and wish InPhase the best, but I don’t believe they have a viable business with their technology – yet. The problem: 3.5″ disk drives will reach 750GB by the end of this year with much faster transfer rates. InPhase’s 20 Mbps is only 2.5 million bytes per second or only 9GB per hour. It will take over 30 hours just to fill one disk! I predict that hard drives will still be more convenient and fairly cost-competitive than this promising new technology.

But keep at it guys. Lightning will strike if your investors are patient enough.

They weren’t.

The StorageMojo take
The disk industry spends over $1B a year improving hard drives. Thousands of PhD scientists and engineers are busy researching drive problems.

That kind of momentum is hard for a startup to overcome. NAND flash did so only because it built a large business in mobile applications where disk drives couldn’t compete.

For a startup to succeed with optical storage they’ll need to:

• a) build a multi-billion dollar business where disks and now flash don’t compete, such as Blu-ray’s movie distribution, or
• b) start with a product that is 10x – 5 years – ahead of current disk drive capacity, and
• c) have a clear grasp of what continued 40% annual growth means for disk drive capacity pricing – and product delays.

There are several firms pushing optical storage forward. Blu-ray is the least ambitious – now that HD DVD is (almost) gone – and that huge investment is almost certain to have a negative ROI, even if 3D content succeeds in making it the preferred consumer physical medium.

Nor is the outlook for other optical drives promising. Like removable magnetic drives before them, they are being crushed by substitute technologies like USB flash drives, 2.5″ drives & drive docks and downloading and wireless networks.

With the InPhase demise we may never see holographic storage commercialized. Especially if disk vendors start building archive-quality disks.

Comments welcome, of course. I was rooting for InPhase’s success, to no avail. Another version of this post was published on my ZDnet blog.

IBM and Fujifilm have demonstrated a technology that, if productized, could give us a 70 TB LTO tape cartridge. Tape isn’t dead – that will be a long time coming – but its vital signs aren’t good, either.

Vacuum column, 800bpi tape drives
Magnetic tape is the oldest digital storage technology still in use. Once mass storage meant tape because drums – and later, disks – were tiny and absurdly expensive.

IBM and Fujifilm demonstrated a density of 29.5 billion bits per square inch on linear tape. Disks are approaching 1 T/bit in a controlled environment and much less media area.

Theoretically this supports a single tape cartridge with a 35 TB of uncompressed data capacity – or 70 TB of compressed data in a single LTO (linear tape open) cartridge.

Current LTO tapes, even with compression, are at about 2 TB per cartridge — the same as high-end disk drives. In nine months those 2 TB disks will cost about the same as single LTO cartridge. Why store data on tape where it is so much slower to access?

Defenders point to tape’s energy efficiency — write once and shelve without consuming more energy for decades — but people like the convenience of random-access data. If this drive industry woke up and started offering archive quality disks — Seagate sold an automotive hard drive that carried a 10 year warranty — much of the remaining tape market would disappear.

Lifespan is another benefit of tape technology. I recently transferred a 20-year-old VHS tape that hadn’t been looked at in at least 10 years to my computer. There was some drop out but the picture was very watchable. Try that with a 20 year old disk drive.

Technology
Whether it is commercially feasible or not, the IBM/Fuji technology is impressive:

• Advanced nano particle technology — they limited the size of the barium ferrite particles to 1600 nm³ — approximately 1/3 of current metal particle volume.
• Advanced nano coating technology — a smooth and thin magnetic layer with very low variability reduced signal fluctuation significantly, enabling more accurate signal processing.
• Advanced nano dispersion — a new material controlled agglomeration enabling more uniform dispersion of the nano particles.
• Nano perpendicular orientation — taking advantage of the barium ferrite particles crystal magnetic anisotropy, a perpendicular orientation improved high-frequency characteristics.

But the remaining obstacles are daunting: mass production of tiny uniform nanoscale particles; mass production of an extremely smooth and thin magnetic layer; and careful control of the particle dispersion and orientation. Plus heads and transports accurate enough to take advantage of the density.

That added technology raises tape’s entry price – further restricting the market – and it isn’t easy to see what, if anything, can reverse that dynamic.

The StorageMojo take
Regardless of whether you think tape has a long-term future, this is an impressive demonstration. When I introduced DLT at DEC, customers were thrilled to get to 2.6 GB on a tape cartridge.

If they can get the cartridge to market in the next 5 years, they’ll can charge 5x what a disk costs – because the capacity is so much higher than any single disk. If they can’t – well, it was a neat tech demo.

Drive marketers should see that a massive archive disk market is fast approaching. Cheap USB 3 SATA drive docks will enable millions to store their memories on rarely used disks – and to rapidly access all the data.

Nevertheless, tape remains the most proven archival storage medium for digital data. Tape may yet live to see that 70 TB cartridge delivered.

Courteous comments welcome, of course. I had an audio cassette recorder for storage on my first computer. Couldn’t afford $800 for a 144 KB floppy disk. I now have 11 disks – and 2 optical drives – on my Mac Pro. That cassette recorder was my 1st – and last – tape drive.

TDK recently demo’d an impressive technical achievement: a 10 layer optical disk with 320 GB capacity – using standard Blu-ray (BD) drive technology. Each layer has better than 90% light transmission and writing required no more than 20 mW of the 30 mW Blu-ray spec.

Too bad it will never be a commercial success. Optical is at the end of the line.

When do formats die?
When their combination of reliability, capacity, performance, density and cost aren’t competitive. Which is where optical is now – even 320 GB optical.

Some of you may remember punched paper tape – hot in the 60s and early 70s – and popular on 16 bit minicomputers back when 4k of RAM was respectable and 64k unaffordable. It was limited to a few dozen KB of capacity and unreliable in long-term use, so when 240KB 8” floppies arrived in 1973 paper tape was toast.

Floppies had to improve to compete with removable disk pack drives – like DEC’s RK05 family – with their 2 MB capacity and a screaming 150 KB/sec transfer rate, and floppies did by increasing capacity – what TDK demonstrated – and decreasing size, from 8” to 5.25” to 3.5”, and cost from over a thousand dollars for a drive to less than $20.

But floppies couldn’t keep up with the growing size of applications and data sets. The 100 MB Zip drive was insanely popular when introduced in 1994 – a woman offered me a $100 premium on the spot to buy mine at a Palo Alto sushi bar – but by 1999 the format was on the way out thanks to cheaper and more capacious CD-R drives.

Despite heroic efforts to increase removable magnetic disk capacities – culminating in 2001 with the 5.7 GB Orb drive – removable magnetic disk media is dead, killed by cheaper optical and more convenient flash media.

Removable: backup and transfer
Removable media has 2 major use cases: data backup and data transfer. Tape dominates removable media backup today with capacities rivaling the largest disks.

Thumb drives long ago replaced floppies for smaller file transfers – “sneakernet” – with external hard drives handling large capacities. With 1 TB 2.5” hard drives, even a writeable 50 GB Blu-ray (BD-R) can’t compete with a small hard drive in transfer speed or capacity.

TDK’s problem
Which gets us to the 10x Blu-ray problem: even if it were commercialized there would be no market. Why?

• Capacity. Successful optical media capacities have been competitive with current disks – CD-ROM in the early 90s; DVD-R in the early 2000s. Multi-layer Blu-ray will never be more than a small fraction of hard drive capacities.
• Performance. 24x Blu-ray transfer rates are half that of today’s disks. And as capacities increase, disks get faster. Not so with Blu-ray.
• Reliability. Early adopters report that BD burner disks often don’t play on many commercial players. That will get fixed someday, but multi-layer DB-R will have to solve it again.
• Density. Managing a single piece of media is much simpler than managing 6 or 10. External hard drive density makes them much more convenient.
• Cost. BD-playing DVD drives haven’t been popular on PCs, and BD burners are way more expensive, as is the media. A FireWire or USB 2 or 3 hard drive can be had for less than $100, has much faster access times, higher capacity and faster data transfer. With volume BD-R costs will come down – but where will the volume come from?

Multi-layer BD-R has advantages, especially if current BD players can be updated to use it. But there is no commercial justification for distributing content on 320 GB optical disks and there isn’t likely to be one.

Hollywood has a real chance to make 3D work this time, but 3D HD movies will fit fine on BD. Put a 3D “Band of Brothers” on a single disk? OK, but really, getting up every 50 minutes to change disks isn’t so hard, is it?

The Storage Bits take
New optical formats will get introduced – like 750 MB Zip drives and 5.7 GB Orb drives – but they’ll stumble around the fringes of consumer acceptance before a quiet death off stage. Many of the same forces that are killing BD – downloading, upconverting, cost – are closing in on optical media in general.

DVDs will be around for years – even as CDs still are – but the focus is shifting to online storage and local disks. The industry still hasn’t cracked the code on massive home disk storage, but that day is coming.

You’ll buy HD 3D content online, download it, store it in your digital library, and watch it when and where you want. If your house burns down your content suppliers will let you download again. Who needs the hassle to burn disks?

The one remaining piece is for hard drive vendors to get serious about building archive-quality hard disks. I love their technology, but they aren’t the most forward looking group.

Courteous comments welcome, of course. Anyone interested in buying a vintage USB Zip drive?

Joe, how about “Hawaiian shirt Fridays?”
Data Domain’s board is has rejected EMC’s all cash offer in favor of NetApp’s enhanced cash + stock offer. But shareholders get the ultimate say.

EMC is continuing with its tender offer for outstanding stock at $30/share through June 29. If it gets a majority of the ~62 million shares, it’s all over.

Except for the fallout.

No non-competes in California
While everyone has been polite so far, EMC’s hostile offer will turn up the heat. And the all cash offer and EMC’s stagnant stock price means that DD employees have to consider their options – which, BTW, are usually fully vested in event of an acquisition.

NetApp has nothing to lose by formally pushing the anti-trust issues. Engineers are getting calls from old friends. Business plans are being sketched on cocktail napkins at Birk’s.

Saw VI – “DDUP, are you grateful to be alive now?”
EMC could end up with a shell company: some patents, products and a brand, but much of the newly-affluent talent gone. And maybe they can earn their cash back in the next few years before something better comes along.

The StorageMojo take
EMC has irritated more folks than the occasional blogger. Their sales-driven culture doesn’t mesh well with Valley technophiles. And their stock price is a turn-off.

Emotions will cool. We’ll have the answer by month’s end – if EMC doesn’t extend their offer. There’s been so much turnover in DDUP stock in the last 2 weeks that it’s hard to know – for outsiders – who still owns what.

Financial investors will take EMC’s money if they haven’t already sold. People who see the chance to rebuild NetApp’s fortunes – and stock price – haven’t.

Hostile takeovers have a poor record in high-tech. But that’s a risk Tucci is willing to take.

Courteous comments welcome, of course. StorageMojo will be returning to its irregularly unscheduled programming shortly.

Chris Mellor has the story on NetApp’s new bid. Basically NetApp has sweetened their offer of combined cash and stock to match EMC’s all-cash offer.

I commented yesterday that

If NetApp merely matches EMC’s offer they may still win the deal. Valley folks generally view them more favorably than they do EMC.

This being finance, however, the emotional decision needs to be covered in a cloth of gold. As NetApp’s CEO said in a letter to Data Domain’s Chairman, the new proposal:

. . . offers Data Domain’s stockholders a superior combination of risk-adjusted value and transaction certainty than EMC’s unsolicited acquisition proposal.

Let’s parse that.
“Transaction certainty” is NetApp’s polite way of promising to raise anti-trust issues should EMC’s bid be accepted. It’s doubtful that anti-trust would derail the deal – after all, there are a number of dedup vendors and no one is dominant – given a new administration it might take longer than usual for Washington to OK the deal – and some strings might get attached along the way.

EMC will say “Hey! it’s all cash, so who cares if the deal closes now or in 9 months?”

The nut of NetApp’s argument is the “superior combination of risk-adjusted value” line. The cash in the offer means “value certainty.” The stock in the deal offers “. . . the potential for long-term value upside through the ongoing ownership of NetApp stock.”

There are 2 elements in favor of stock. First, the stock portion is tax-free, so if you’ve already made a ton of money on DDUP you won’t have to take your capital gains all at once. Second, maybe the 2 companies really will make a killing and drive the stock price up substantially.

If you like your DDUP stock now NetApp offers a way to keep some of it. Both companies have traded in the $40s in the last 3 years, maybe DDUP+NTAP could be trading in the -simplistically – $80s in another 3 years.

Sure, you could buy EMC stock, but other than the unwarranted VMware mania in late ’07, EMC stock has been stuck in neutral for years.

NetApp’s checkered acquisition history
EMC partisans will point to NetApp’s acquisition problems: Internet Middleware (later NetCache); Spinnaker Networks; and Topio. Somehow, NetApp was never able to make those go.

The difference here is that both companies sell appliances. This isn’t a technology integration play or a an orthogonal business venture.

NetApp’s pitch should be “look, NAS is a faster growing market than EMC’s block-based storage and DDUP makes a perfect back-end for our customer base. We’ll make a killing!”

And they probably will.

The StorageMojo take
Data Domain’s top 5 investors hold more than 50% of the shares. This isn’t going to turn into a long, drawn-out proxy fight.

If they like NetApp more than EMC – which they probably do – a few head nods around a conference table will seal the deal. EMC can counter again, but if NetApp’s cash+stock offer is attractive now, it will only get more so if they match EMC again.

NetApp will close this deal and EMC will be left looking a little foolish. The good news: there are plenty of other good acquisition candidates that could power EMC’s growth in the next decade.

Courteous comments welcome, of course.