but it’s really a shame that Atmos is implemented, at least currently, using technology that does not include ANSI T10 DIF. Without DIF, the nice, large cloud of petabytes of data is going to eventually end up as nice, large, cloud of petabytes of data, many gigabytes of which is corrupt, and you don’t know which ones. file or object replication doesn’t help here – it doesn’t really protect the data, it merely makes potentially corrupt copies of it. Read the CERN paper from the spring of 2007.

but it’s really a shame that Atmos is implemented, at least currently, using technology that does not include ANSI T10 DIF. Without DIF, the nice, large cloud of petabytes of data is going to end up as nice, large, cloud of petabytes of data, many gigabytes of which is corrupt, and you don’t know which ones. file or object replication doesn’t help here – it doesn’t really protect the data, it merely makes potentially corrupt copies of it. Read the CERN paper from the spring of 2007.

It shouldn’t be assumed that this sort of service has to be constrained to commercial providers.

More heavyweight commercial DP is more of an issue. The killer there is often performance – high latency is inherent to anything geographically spread. Global latency is truly awful – up to about 400ms round-trip, but even continental (typically 50-100ms) is unusable for many functions. In that case you want your compute capability close to the data.

There are also issues over consistency of multiple copies. Still, it’s an interesting approach and I can see it’s use for low access rate storage, archival stores (email looks a lot like that), remote backups etc.

EMC did use IBRIX for some 9+ months during the initial development of Maui/Hulk. But the use of IBRIX merely bought EMC time to develop their own (theoretically) scalable “local data store”. The verdict is still very much out on EMC’s ability to compete in the local clustered storage market (let alone cloud).

Mike Feinberg, vice president of EMC’s cloud infrastructure group, shared the following with techtarget.com:

“The talk in the storage industry is that EMC was using Ibrix software for its clustered file system. And while Atmos does include capabilities (such as global namespace) that Ibrix offers, Feinberg said that no other vendor’s IP is used outside of EMC. “It’s all EMC technology,” he said. “Frankly, there’s no Ibrix software and no other people’s software. It’s built from the ground with home-grown technology, although we do use open source technology as we see fit.”

love reading your articles.

just to note that Atmos is NOT using IBRIX (or Clustered XFS) for the underlying filesystem. This is a truly organic, object based filesystem designed by Eaton and Co.

cheers,

Dave Graham

(1) It’s highly likely to be a new codebase even if it implements the same ideas.

(2) Atmos doesn’t quite have the same goals as OceanStore (especially wrt multi-tenancy) and the state of the industry has changed in quite a few ways since OceanStore was being actively developed.

(3) Even if Atmos is exactly like OceanStore, which OceanStore? Some of the basic approaches, including in areas you mention above, changed quite a bit during the project lifetime.

From what I’ve read so far, Atmos seems more clearly related to an EMC Cambridge project I used to know as RAIDiant than to OceanStore, and might even be related to at least one other that I’ll keep quiet about. I think I’ve succeeded in prodding Steve Todd to write more about the aspects that are most likely to be OceanStore-like, though, so that’s probably where we should look for a more authoritative answer.

That said, dare I ask – do you think this means EMC actually *IS* innovative?

Or is recognizing, funding and then converting a research project into a real-world product no more innovative than say, acquiring a smart-but-struggling startup and turning their technology into more than it ever was going to be on its own?