An architect and regular StorageMojo reader faces a perplexing problem: should he move his organization’s archive from tape to object storage? The first question is, of course, is it technically feasible?
The economics come next.
problem opportunity, lightly edited:
I’m really curious about where to turn next with yet another infrastructure project.
Picture a large HSM environment currently storing about 4PiB+ of data, with all of the bells and whistles as far as data validation at the tape layer (Data Integrity Validation via Oracle T10K-C/D) and ZFS at the top for disk consistency. SAM-FS (Storage & Archive Manager) is the data mover/stack in use.
What of object storage in this space? Are we there yet? Can we consider something like a commercialised/packaged up Ceph to be able to pull off the preservation and “forever” feats of big HSM environments with multiple tape copy scale?
My gut feeling is that Ceph has a place and will be a rockstar filesystem in the end, and might just take a place alongside ZFS as a stalwart of modern “just use it” thinking, but I do wonder if Ceph has been used at that kind of storage scale with the sole intent of a very high throughput, high performance (think: 10+GB/sec, 100k+ IOPS) file serving platform ATOP an object storage container BEING RELIED UPON FOR a long term archiving and preservation infrastructure.
Are we there yet? Is the tapeless world now possible with all the goodness and data integrity capabilities that tape offers without the vendor smarm/marketing and “tape is dead” rhetoric? It is almost embarrassing to hear such words at press and conference events. Not sure how company execs still get away with saying it without being pelted with tomatoes.
What about it, object storage vendors, can you meet his needs without bashing tape?
The StorageMojo take
Tape has been ceding market share for the last 60 years. But it’s still here. The recent work by IBM and Fujifilm holds the potential for tape to begin to gain market share, if they can meet their aggressive density growth rates.
Until that happens, the rising economic threshold for tape makes scale out object storage every more attractive. But attractive even at 4PiB?
Vendors, start your engines.
Courteous comments welcome, of course. Vendors encouraged to comment but, please, identify your company and try to add as much value as you can. Please ask any questions in the comments or email StorageMojo and they’ll be passed on to the architect.