StorageMojo

A video networking company in StorageMojo?
Omneon isn’t new to StorageMojo. Their price list has been on price list page since January 2007.

Their booth was about 50 yards from Isilon’s and EMC’s and it was a madhouse each time I walked by. Partly that was because they were holding all their meetings there, but it also seemed like there was lots of traffic.

Building storage into an app
Founded in 1998, Omneon started offering storage in response to customer demand. They decided on a commodity-based cluster and built their own storage software, MediaGrid.

Their architecture hews to the post-array Google-style storage model:

• No RAID - slices are replicated one or more times based on policy or demand
• Single global namespace
• Out-of-band meta-data servers manage content servers

<strike>They can rebuild a failed 1 TB drive in less than an hour.</strike> They can replicate the data from a failed 1 TB drive in less than an hour.  Just add 4 or 24 drive content servers to scale capacity. <strong>Update:</strong> My original wording was incorrect. Thanks to Bill Todd for elucidating Omneon’s mechanism.<strong> End update.</strong>

But that’s not all!
Omneon’s content servers do more than serve content. They put their unused CPU power to work doing jobs like transcoding - translating content from one format like HD to iPhone-suitable QuickTime.

Given the growth in multi-core processors that will become a more important part of their market appeal over time. Since they process files, not blocks, they have many more opportunities to add value than a modular array.

The StorageMojo take
Omneon made a lot of smart choices with their MediaGrid architecture. It shows how a company with a few bright engineers can build a basic storage utility to take advantage of low commodity costs.

Where they win is their integration with the application and the workflow. They’ve created a video utility that integrates ingest, post, media management and playout with the smart and scalable storage needed to make it all work.

Application specific storage writ large. They’ve taken the same storage the rest of us use and wrapped broadcast interfaces around it that broadcasters already know.

Comments welcome, of course.

Isilon at NAB
Stopped by the Isilon booth a couple of times. Traffic seemed steady. Isilon held their meetings away from the booth, so it didn’t have the level of activity of, say, Omneon’s booth.

NAB is their biggest show of the year and the market where they have the biggest footprint. Their booth was the same size as EMC’s nearby - and much quieter booth - despite a Hulk display that looked like an embarrassed afterthought.

Personnel changes at Isilon
Isilon CEO Sujal Patel was there. We discussed my theory that Peter van Oppen had joined the board as a prelude to becoming CEO. Sujal assured me I’d gotten it wrong - that he was in it for the long haul, with Peter as a senior and trusted advisor.

Looking at him I believed. Sujal has developed the gravitas of a leader. Watching his company almost die - and his net worth drop from $75 million to $12 million - seems to have concentrated his mind.

He’s also hired a CTO. Looks like Sujal has moved on for good.

The StorageMojo take
Anyone waiting for Isilon to lay down and die has a long wait. While they may have alienated Wall Street - for good reason - customers seem to like what they have. They’re coming through the storm.

Comments welcome, of course. Isilon was also doing the “shown but not announced” thing with some products due later this year. Sujal asked that I not write about them and I said I wouldn’t. But the engineers have been busy.

They haven’t reported financials for almost 3 quarters. Their stock is trading at about 20% of its peak. They fired their CEO and put founder Sujal Patel in his place. And NetApp was trying to strangle baby Isilon (see NetApp filers for $1/GB?) in its crib.

Are they goners?

I don’t think so.
I’ve been trying to read the tea leaves on the Peter van Oppen’s decision to join the board earlier this month.

Peter led the tape library company ADIC, also based in the Seattle area, for 12 years until its sale to Quantum. ADIC out-innovated Quantum - saddled with a cranky and slow DLT development group - in libraries and software as well.

If you think the folks who buy storage arrays are conservative, you haven’t sold any tape libraries. It is a tough market and ADIC did well.

So why would van Oppen join a sinking ship?
That’s why I don’t think Isilon is sinking. An external audit team is reviewing Isilon’s accounting to ensure that any financial dirty laundry - say, hypothetically, channel stuffing - gets cleaned up. They’ve been at it for months and must be about done.

The StorageMojo take
Based on the Isilon press release and pure speculation, here’s what I think is going down:

• Peter exercised some due diligence before accepting the directorship and isn’t terribly worried about the basic health of the company
• After he gets up to speed on company operations, he assumes the CEO role by July
• Sujal happily goes back to one of the best jobs in any company: CTO and Founder while the stock climbs in value

However it goes down, getting Peter on board is a real plus. Storage experience is thin in Seattle. Isilon has lots of smart people, but the storage market has many unique wrinkles that networking or software folks take a long time to learn.

Comments welcome, as always. Disclosure: I met Sujal 7 years ago and I’ve done some work for Isilon. I hope they do well.

Despite being written off for dead . . .
Isilon’s been putting their IPO money to good use: engineering the next gen of their platform that they’ve named the X-series. In the meantime they’ve been adding customers - over 600 so far - and they have 60 customers running the new kit.

Moving from an aging single-core Xeon to a dual-core Xeon - the second core isn’t turned on yet - with faster busses and more cache speeds things up. They claim up to 60% faster performance, 20% less power and heat and 10 GigE readiness. Once they get their software dual-core aware they’ll have another nice boost to offer.

The StorageMojo take
Turning over the platform more rapidly than traditional array vendors do is a good strategy. It keeps the competition off-balance and gives you something new to tell customers. What good is commodity hardware if you don’t follow Moore’s law?

That said, Isilon’s scale out architecture is the real differentiator vs NetApp and other traditional filers. More bang for the buck just underscores the differences.

Comments welcome.

What could be more appropriate?
HP’s lackluster showing in The Info Pro’s survey - see yesterday’s post - reminded me that I’d written about HP shortly after after I’d started blogging (see HP’s Storage Grid-lock: panic-stricken execs promise fix in four years).

Here’s a quote from the September 2004 post:

HP’s storage shortfall this past quarter has Carly pulling out the stops to save one of HP’s few high-margin non-printer businesses. Firing executives, check. Inspirational speech to resellers, check. Roll out ambitious product roadmap for delivery in 2008, check.

2008!?! Why not just put out a press release titled ” HP execs panic over storage shortfall, have no clue how to fix business in less than four years”? Then, at least, they could start facing up to their real problems.

Issuing long-range roadmaps is always a move of pure desperation, so the HP storage business must be considerably weaker than they have let on.

So what were they going to deliver this year?
Get this wacky idea: a storage grid. As an HP whitepaper described it:

The HP StorageWorks Grid will be built from intelligent building blocks called smart cells. These elements will incorporate commodity hardware. In addition to the expected control and storage hardware, smart cells will incorporate a flexible operating environment that allows storage functions to be downloaded as needed—and allows smart cells to be repurposed if necessary.

While the 4 year timeline was laughable the actual proposal was smart and forward-looking. The technology was there. But the paper didn’t address the business dynamics that would enable HP to bring the project to fruition. In other words, the marketing problem.

Clearly no one else did either.

The StorageMojo take
Today clustered storage is the obvious successor to big-iron arrays. When HP wrote the paper it wasn’t. If they’d followed through they’d be the thought leaders in the industry and well-positioned to take commercial leadership as well.

As I noted then,

But HP’s roots as a device company have always made the systems approach [. . .] difficult to execute. Even though HP bought a strong storage business with Compaq (which won it when they bought DEC) and the StorageWorks line, the feckless storage mavens in Palo Alto (best idea ever: “Let’s introduce EMC into all our top corporate accounts! It’s cheap, easy and with no engineering its all profit!”) have managed to run it into the ground.

HP should take a leaf from Sun’s storage strategy: move storage software from tin-wrapped boxes into the OS group where commodity servers and networks can provide cost-effective infrastructure.

HP had a great idea 4 years ago. Now it’s time to deliver.

Comments welcome, of course.

No, this isn’t about pork bellies
In just the last few weeks EMC and IBM have announced their intentions to offer commodity server-based storage. EMC with Hulk/Maui and IBM with XIV.

Sun already offers its Thumper product, a high-density server and storage chassis with 48 disk drives, at prices competitive with commodity servers from a GB perspective. You can put OpenSolaris on anything you choose.

A few more shoes will be dropping
So EMC, IBM and Sun have commodity-hardware based storage. HDS, HP LSI and NetApp don’t. How long will they choose to hold out?

HP is part way there. Polyserve runs on HPs popular blade servers. But if density isn’t your goal and scalability is important, you may lean to another solution.

Pressure on the margin
It is a simple question, really. How will expensive hardware compete with cheap hardware? After the FUD and hand waving are said and done, customers of enterprise storage will have a new choice with a lower cost structure.

If HDS and NetApp aren’t in the new game, where exactly will they be?

The StorageMojo take
HDS is not nimble, so I’d be pleasantly surprised if they did anything in 08. NetApp also has some breathing room as none of the big boys are offering commodity cluster NAS. But that could change in a week.

NetApp appears to be the most vulnerable. Their largest customers are ripe for conversion to a more scalable architecture and lower costs. No matter how much NetApp discounts, their costs are higher than commodity hardware. They can fight for a while, but not forever. They have to be competitive and their big customers have to believe they will be competitive.

Expect to see NetApp make a cluster storage software acquisition in 2008.

Comments welcome. Is Polyserve going to meet HP’s needs going forward, or are they too going to have to buy or create a new cluster storage product?

Enough of Google’s bathtub brew
IBM’s purchase of XIV makes it official: cluster storage is on a roll.

XIV’s website could have been ripped from the webpages of StorageMojo:

. . . enterprise-class storage systems typically comprise proprietary, special-purpose hardware, such as backplanes, shared memory architecture, and disk shelves. Huge amounts of resources are spent on developing and testing these products — with the associated costs passed on to the user. Moreover, special-purpose hardware quickly becomes obsolete, with a long wait time until new-generation processors, switches, and other components are integrated.

It also appears that the Nextra product has done away with RAID 5:

The Nextra system uses innovative RAID-X design, in which each disk is split into small pieces, and each piece is mirrored on a different disk. As a result, when a disk does fail, all disks in the system participate in the rebuild.

They don’t like ILM:

The ILM concept is rendered redundant, saving on ILM-related software license costs, administration efforts, and management attention, and sparing users migration-related downtime and other service issues related to ILM

Payback time?
Moshe Yanai, the executive chairman of XIV, was the chief engineer for the original Symmetrix that EMC used - along with raging incompetence at IBM - to destroy IBM’s lock on enterprise storage. Not only was he the engineer, he also got a percentage of the sales price of each Symm sold, making him a wealthy man.

But when EMC bought Data General to acquire the Clariion storage division, Moshe didn’t like it. After a long fight, CEO Joe Tucci pushed Moshe out of EMC and continued the successful Clariion product line. Moshe went back to Israel and eventually developed the Nextra product for XIV.

Gee, do you think his tie-up with IBM might be aimed at his former employer? A little?

The StorageMojo take
EMC’s Hulk/Maui and IBM’s XIV products are aimed at different parts of the market. IBM doesn’t have a large high-end array business to protect so the Nextra’s positioning as

A winning new storage paradigm for the enterprise
XIV Ltd., creator of Nextra™, has undertaken to design and produce the next generation of enterprise-class SAN (Storage Area Networks) systems. Nextra was created based on the principle of providing a simple solution for meeting the herculean IT challenges of today and tomorrow.

isn’t the problem for IBM that it is for EMC, desperate to protect the margins and revenue of the Symm line.

But both products are built on (quality) commodity hardware, so if one or the other needs to make mid-course corrections they can do it in software. Positioning Nextra as enterprise storage puts the heat on EMC and the Symm.

It will be interesting to see who gets to a boil first.

Comments welcome, of course.

For some reason I volunteered to write something about vendors after the Wikibon con call today. That follows.

Vendors: responding to EMC’s cluster storage initiative

Context:
EMC’s support of cluster storage for archiving and backup will legitimize the technology. Vendors with competitive products have a window of opportunity to position themselves as a superior alternative. Make no mistake: EMC plans to own this market and will commit significant resources to the effort.
EMC’s market entry will be hobbled by several problems that competitors can exploit.

• Immature software: limitations, bugs and the eval cycle that implies
• Maintaining a bright line positioning between Hulk/Maui and Symms
• 60% gross margin requirement

EMC will be NDA’ing strategic customers starting mid-January to build major sales to reference at announcement. Smart customers will be calling other vendors, including the smaller, innovative ones, for perspective. Luck favors the prepared.

Strategy:
IBM, Hitachi, HP, NetApp IBM Global Services should be open to reselling/integrating suitable substitutes. There are efforts within IBM’s storage group to create a scalable, commodity storage infrastructure, but the chasm between IBM’s brilliant technologists and IBM marketing makes success problematic.

Hitachi doesn’t seem to be doing anything in this area. They will be looking at an acquisition and will take their time.

HP’s Polyserve acquisition may convince them that they have the cluster thing under control, but Polyserve isn’t competitive with EMC’s initiative. HP has a deep well of technology expertise from the DEC cluster products. Expect a cluster acquisition in 2008.

NetApp is vulnerable. ONTAP GX has missed the cluster market and their controller-based architecture has all the cost disadvantages of traditional arrays without the flexibility of clustering. Putting ONTAP 7G on commodity hardware bricks with software “mortar” - as Google does with GFS - would preserve their significant advantages with WAFL at a lower $/GB.

New competitors
Now is the time to get serious about what your product really does and what its appeal is to customers. Focus is critical to building a defensible position that can be used to win F500 business in areas where EMC is less competitive. There is also an opportunity to shift the terms of the customer debate. This market is still fluid and customers don’t have a clear mental map of the terrain. Smart, focused marketing can take advantage of that.

Action Item:
Small/new vendors: if you want to be acquired, now is the time to be shopping yourself to the big guys. If you want to build a big business, get your marketing focused on verticals and business justification.

Big vendors: start shopping now. EMC wants your scalp so you’ll want to be well-armed.

All: there is a lot more to know about Hulk/Maui. A focused competitive analysis effort will pay dividends.

Update: The audio is available here. If you are wondering if I mentioned your company, I probably didn’t.

Comments welcome, of course.

Here’s an invite to a Wikibon Peer Incite discussion I’ll be leading
Tuesday, 11 December. Call in on Skype and listen and ask questions. EMC’s AR and competitive analysts will be there. Shouldn’t you?

If you aren’t hip to Wikibon it is worth checking out.

Here’s the blurb from the invite with the call in number and passcode.

This is a reminder that the next Peer Incite research meeting is scheduled for Tuesday December 11, 2007. The topic for this meeting is: EMC’s Maui and the Future of Clustered Storage.

Google’s development of one of the world’s largest storage infrastructures based on commodity components, without reliance on traditional array technology, was a huge wakeup call for the storage industry in general and EMC in particular. Recent comments by EMC’s CEO Joe Tucci indicate two new products from the company, Hulk and Maui will address the market for so-called ‘Cloud Computing’ and hit the market in mid-2008. It is estimated that 85% of corporate data is unstructured yet organizations continue to spend billions optimizing storage for the 15% of information that is traditional database-oriented. Will this continue to make sense?

Key issues we’ll address on the call include:

• What does it mean to users that EMC is about to legitimize clustered storage?
• What do these advancements mean for user investments in traditional array technology?
• How will the industry likely respond to EMC’s attempt to lead this trend?

Here’s how to participate in the discussion:

• Date: Tuesday December 11, 2007
• Time: 12:00pm EST (9:00am PST)
• Call in #: 218-486-1300 Passcode: 509215

Moderator: Peter Burris (http://www.wikibon.org/User:PBurris)

See you at the meeting.

The StorageMojo take
The IT consulting business - as in Gartner and everybody else - needs a good shaking. Wikibon is a good idea that may be part of the solution. Let me know what you think after you check it out.

Update: If you miss the call check out the Wikibon archive.

Comments welcome, of course

Expect to hear a lot more about the SMB segment over the next 6 months.

Because the high-end market is sucking wind. NetApp and EMC are both reporting problems in the high-end. HP and IBM don’t break out as much detail but I’m sure they are feeling the chill as well.

This is a great time to buy
Vendors are scrambling.

The subprime mortgage mess - midwifed by political hack and former Fed Chairman Alan Greenspan (remember when he was encouraging people to take ARMs?) - has reached Wall Street and every major European financial center. IIRC, the financial services industry takes about 20% of the high-end storage kit.

A slowdown in 20% of the market may not sound like much. But EMC’s profits are about 11% of revenue, so a slowdown in Symm sales hits the bottom line pretty hard.

With CEOs getting bounced amid multi-billion dollar write-downs - now forecast to reach $400 billion - bank CIOs aren’t about to pony up for the gold-plated, diamond-studded storage of yore. Except for Goldman, who saw this coming and unloaded their CDOs early this year.

Ask for what you want - you’ll probably get it
EMC is very interesting. With Hulk/Maui coming in Q2CY08, you should hold off on any 2nd tier storage purchases you can. I estimate that H/M will be about 30% per GB less than the current gear.

The StorageMojo take
Structures tend to stand until a storm hits. The financial storm is unfolding now and it could get very bad. With EMC’s endorsement of cluster-based storage in Q2 the other storm will arrive.

The next 2 years promise to be very interesting for the industry and customers.

Update: Just came across this tidbit from the IT Jungle about a survey taken at the SMB Summit last month:

Among those responding to the survey, which was conducted in October, 45 percent said they expected to grow their annual sales by more than 10 percent over the next 12 months, and a stunning 21 percent of those surveyed said they expected to grow their businesses by more than 20 percent over the next year.

Storage was the number one area for capital investment.

Comments welcome, as always.

Was it ever alive?
Techies have been excited about grid computing for years. The rest of the world never caught on. They probably never will.

Who killed grid?
A slide from Sun’s recent HPC meeting at SC’07 implicated marketing:

The Grid is Dead…

• Term falling into discredit
• Misuse by Corporate marketing departments, including multiple contradictory definitions
• Failing to live up to expectations
• Built for specific community
• Unscalable
• Expansion efforts only make the situation more unwieldy

The death of grid had 2 causes: a) nobody could agree on what a grid was and b) they were too specialized and didn’t scale.

Yup, marketing killed the grid. If only they’d agreed on a single contradictory definition.

Simple is better
Techies seem to have a mania for inventing a new term when a perfectly good term already exists. In the case of “grid” the term “cluster” would have served admirably.

Among aficionados, a grid is not much like a cluster. To the folks who approve capital authorizations it is hard to tell the difference. Except grids are new and unproven while clusters have been around for decades.

Polite laughter
At IDEMA there was some back and forth between Seagate and other vendors over what to call the recording method that uses lasers to heat the media. Seagate researchers were pushing the catchy “HAMR” - pronounced hammer - an acronym for Heat Assisted Magnetic Recording. Everyone else seemed to like “TAR” for Thermal Assisted Recording.

At one point a senior researcher said he’d been talking to a civilian who asked about the “hummer” technology. Most of the assembled laughed and took this as evidence that HAMR was a bad acronym.

In truth, it is simply evidence that a research PhD does not a marketer make. Disk drives suffer from a dearth of sex-appeal, especially against cool newcomers like NAND flash. HAMR is a much sexier acronym than TAR. And a much better acronym to help keep consumers interested in disk drives.

You know, consumers. The people who buy most of the world’s disk drives.

What if RAID had been SMAD?
Would a Spare Matrix of Autonomous Disks have taken the storage industry by storm the way RAID did? No way. People, even techie people, are social creatures. We use language - even jargon - to signal to others our values and knowledge.

People like RAID so much that even after the original words ceased to have commercial meaning - all disks are inexpensive today - they just changed the word to “independent”. All to keep a cool acronym.

“God is dead.” -Nietzsche. “Nietzsche is dead.” -God.
The Sun presentation’s point was that a new middleware for grid computing was needed so users didn’t need to worry about where and how their jobs would get done. Just ship it up to that big batch mainframe in the cloud, sip your Red Bull and Jack, and let thy work be done.

The StorageMojo take
Product names aren’t all that important. But paradigm-shifting names, like RAID, client-server, clusters, NAS, pNFS and SAN are. Which is the real reason “grid” is dead.

Grid just isn’t suggestive of anything to people who aren’t intimately involved with it already. RAID and HAMR suggests something forceful while TAR sounds like a place I don’t want to go.

So yes, bad marketing killed grid computing. They should have asked the guy who came up with RAID.

Update: The technique of lashing together lots of boxes to perform work isn’t dead. We’re at the beginning of that trend. What is dead is the term “grid”. Its unfortunate connotation of rigid structure and the concomitant “gridlock” don’t do the technology justice. “Cluster” whether hetero- or homogenous, local or wide area, storage, compute or both, is a much better understood term.

Update II: Reading the comments in The Blog of Scott Aaronson, a postdoc at the Institute for Quantum Computation at the University of Waterloo, about the much disputed D-Wave quantum computing demonstrations, he commented:

I think that computer scientists have been much too eager to coin acronyms, and that this has damaged the public perception of our field relative to physics (which has cool names like “quark”, “supersymmetry”, “black hole”…) . . . .

He’s right. If you had to convince yawning non-scientists to fund billions of dollars in sub-atomic physics research you’d probably develop better marketing muscles too.

Comments welcome, of course. If you don’t like cluster, what would you call grid computing?

Joe Tucci let slip, on purpose, that EMC will be coming out with a cluster storage system for backup and archive purposes at a press event this week.

Hulk is the code name for the hardware. Maui is the software. Expect to see large green guys in grass skirts at the fall SNW. The subtext: think Big Green for EMC storage clusters. Hey, that sales force doesn’t come cheap!

Kudos to EMC
Storage clusters are the coming thing for the 85-90% of all data that is unstructured. EMC would like to sell Syms for that data, but they’ve twigged to the fact that that won’t happen. Smart.

Surf’s up!
StorageMojo’s most regular readers are the bright folks in EMC’s competitive analysis group. Some of them are fans, but it isn’t a good idea to admit it. How much fun is it to read analysts repeating what you’ve paid them to say?

Pipeline
EMC’s challenge is to move to cluster storage while maintaining the margins that Wall Street loves. IBM is somnolent, HP complacent and Sun, well, Sun is a wild card.

The real wild card is the yet to be announced, VC-backed, cruise-missile into the heart of the whole bloated enterprise storage market. Someone who sees the advantage of turning a $30 billion array market into a $3 billion storage cluster market, as long as they get 80% of it.

The StorageMojo take
EMC has no choice, but my hat is off to Mark Lewis for getting Joe Tucci to recognize that fact. The desiccated corpses of once-great Boston minicomputer companies should be an object lesson. Surf or drown, guys.

Now if they can just lose Egan’s frat-boy aggression and chip-on-the-shoulder attitude, they may find the admiration they crave.

Comments welcome.

New Product Introduction
As part of my campaign to increase the world’s consumption of disk capacity - see yesterday’s post - I’ve developed a new capacity gobbling product. For lack of a better term I call it a video white paper.

The impetus? No one reads anymore. Especially white papers. We’d much rather watch videos.

Enter Gear6
When I looked around for a launch customer, Gear6 came to mind. Their marketing VP, Gary Orenstein, has one of the few marketing blogs, Thoughtput with real content instead of “aren’t we wonderful” happy talk. He’s done a number of podcasts as well. He’s a new-media, large file size kind of guy.

Happily, he agreed to be the launch customer.

Here’s your part
Gear6 and Gary responded favorably to this new product and now I would like to hear from all of you. I am continuing to enhance the concept with the goal of bringing more value to everyone that views it.

What’d I’d like you to do is to watch the 4.5 minute video and tell me what you think. What works, what doesn’t work. What you’d like to see more of and what you’d like to see less of.

Meet Nisha Talagala, CTO

Nisha is not just really smart - smarter than the average Silicon Valley CTO - she is also a very nice person. I was impressed.

Yes, I was paid for this. And I’d like to be paid to do more of them! But only if they are worthwhile for you. So help me figure out how to make that happen.

Comments welcome - more than ever. My goal is to create something that is genuinely useful for information seekers in a 3-5 minute package. Tell me how well you think it works. How would *you* get more valuable content into 3-5 minutes in a way that people will watch?

Update: I tweaked the wording a bit. Same video.

I don’t normally link and run but this is a good article on the Next Big Thing in NFS v4.1.

Written by 3 NetApp engineers, Garth Goodson, Sai Susarla, and Rahul Iyer, Standardizing Storage Clusters offers a good overview of what’s new. It’s on the ACM Queue web site.

If paragraphs like

protocol operations

The pNFS protocol adds a total of six operations to NFSv4. Four of them support layouts (i.e., getting, returning, recalling, and committing changes to file metadata). The two other operations aid in the naming of data servers (i.e., translating a data server ID into an address and getting a list of data servers). All the new operations are designed to be independent of the type of layouts and data-server names used. This is key to pNFS’s ability to support diverse back-end storage architectures.

get you interested the article is well worth a read.

The StorageMojo take
pNFS is going to commoditize parallel data access. In 5 years we won’t know how we got along without it.

Is Parascale new or old?
There were many good reader questions about Parascale’s announcement. Even though I’ve done some work for them I didn’t know the answers so I invited their CTO, Cameron Bahar, to respond. He sent me a text only email, which I’ve decorated with some HTML to improve readability.

CTO Cameron Bahar:

Hi Robin,

We are delighted by the interest shown in both the file management challenges that Parascale seeks to address…and in our newly-announced solution. Your readers bring up many important issues, especially in regards to how existing solutions compare to Parascale. Permit me to try to group these questions into categories and to highlight how Parascale is different.

HPC solutions High Performance Computing (HPC) solutions are typically implemented with kernel code and employ custom client-side software to achieve high bandwidth. For example, Lustre has been successful at many national labs as mentioned in one post. Parascale is targeting a different market. Parascale is all about industry standards. We support NFS, HTTP, and FTP protocols because we don’t expect our customers to recompile their applications. We want our software to be simple to use, as well as to scale in capacity and bandwidth for our target digital content applications.

Archival solutions. Several companies, including Archivas, have delivered archival systems. These solutions are generally WORM (write once read many) systems and disallow updates to existing files. By comparison, Parascale is POSIX-compliant and designed to support large read/write bandwidth—not always a requirement for archiving. Finally, if a large vendor has acquired these technologies (e.g. HDS-Archivas), they’re usually shipped as a rack of pre-installed appliances, limiting choice of hardware provider and hardware configuration.

Clustered file systems. Shared-disk clustered file systems such as Red Hat GFS have the characteristics of traditional distributed file systems such as tight cache coherency, distributed lock management, symmetric topology. Scalability of these file systems is generally limited to 16 or 32 nodes due to heavy cache coherency traffic and message passing between nodes.

Members of our engineering team have written several clustered file systems in previous undertakings. From that experience we elected to adopt a very different architecture for Parascale. For starters, we elected to adopt a loosely-coupled architecture for scalability. Further, we chose not to write a new file system. File systems are very delicate (as we know by having written them in the past) and they take 5-7 years to fully stabilize and stop corrupting data. We simply aggregate existing file systems to present a “virtual file system” layer to clients/applications over standard protocols.

Appliances versus software. NAS appliances are ideal for many markets, like SMBs and enterprise workgroups, that need simplicity of installation and for which scalability in volume and bandwidth are not key requirements. Appliances generally employ hardware highly-customized for serving files, including hardware features like NVRAM to boost write-performance and RAID controllers for data redundancy.

Parascale seeks to solve a different problem, that for management of large digital content repositories. Think of video on demand, photo archives, medical imaging, seismic data, and genomics data. Don’t fault us for being inappropriate as secondary storage for an RDBMS. We didn’t design Parascale for block storage because many excellent products already address this market.

We’ve constrained our solution to run as an application (with no kernel code) on industry-standard servers, as qualified only by Red Hat. We want our customers to enjoy the very latest advances in server hardware (motherboards, processors, memory, disks) available from Dell, HP and others. And we want our customers to be able to buy servers from their “regular hardware vendor”

Parascale’s software-only solution lets our customers to tune the disk capacity, CPU, RAM, I/O and network bandwidth independently—as required by the application at hand. Growth can be incremental—one disk drive or server at a time. You never have to discard hardware or licenses. Another useful benefit of a software-only solution is that other applications can coexist on the Parascale storage nodes, allowing data mining, trans-coding, encryption, or compression on the servers where the data resides. This is not possible with closed appliances.

What qualifies as “software-only” file storage solution? Our perspective is, first, that the software has to support standard network file access protocols like NFS, HTTP, or CIFS. You can store files in an RDBMS, but that doesn’t make it a software-only file management solution. Second, the disk drives must be direct-attached to the servers. Shared disk distributed or parallel filesystems (over SAN) are software products, but don’t qualify because they require specialized SAN hardware on the back end.

Finally, because all our engineering resources are focused on software, we’ve been able to innovate (with patents to prove it) and to deliver features like transparent, automated file migration (to eliminate server hot spots) and replication (to raise read bandwidth). And our roadmap promises a lot more innovation to follow!

Asked another way, where does Parascale fit in the market? Choose us if:

• You want industry-standard hardware (e.g. because you want to run applications on the storage nodes, or because you have corporate hardware standards).
• You need more bandwidth than one server/head can provide.
• You need the benefits of data mobility across servers (e.g. migration to balance data and eliminate hot spots, replication to increase read bandwidth, smart load balancing to optimize system performance).

Lastly, Parascale aspires to be new and modern in its business model. When our product goes production, we plan to allow you to download our software to try it out at no cost. We’re confident you’ll like it. Our pricing is per-spindle, so you never have to deploy or pay for storage capacity before you need it. And if a drive fails, replace it with a new drive in the manufacturers’ current sweet-spot; we’re not trying to make money on advances by the disk drive manufacturers.

Hope I’ve addressed some of the questions posted. I applaud the thoughtful discussion that your post has prompted.

Best,

Cameron

Comments welcome, of course.