The official best paper winner at FAST ’10 isn’t one of the several I excerpted. I’m listening to the presentation as I write – trying live blogging – while following a fast talking presenter.
The winning paper is quFiles: The right file at the right time by Kaushik Veeraraghavan, Jason Flinn and Brian Noble of the University of Michigan and Edmund B. Nightingale of Microsoft Research.
From the abstract:
A quFile is a unifying abstraction that simplifies data management by encapsulating different physical representations of the same logical data. Similar to a quBit (quantum bit), the particular representation of the logical data displayed by a quFile is not determined until the moment it is needed. The representation returned by a quFile is specified by a data-specific policy that can take into account context such as the application requesting the data, the device on which data is accessed, screen size, and battery status.
One application is video files that may be played back on a variety of devices with differing resolutions, compute and graphics engines, codecs, editing capability and storage. There is one quFile that encapsulates several versions of the file – even different versions of the same file – and which is returned depends on the device requesting the file.
The key is that every device asks for the same file name, simplifying file management on the server and file distribution. quFiles are space efficient, adding little to file size, while their compute overhead is in the single-digit percents. And no application changes are required.
The StorageMojo take
FAST’s Best Paper isn’t necessarily StorageMojo’s Best Paper, yet this is a worthy candidate. Hiding the gory details of file types and network requirements from users is a Good Thing. I particularly like the support for file versioning, a feature I grew to love on the VMS operating system, but not widely appreciated today.
Congratulations to the team on their win.
Courteous comments welcome, of course. I’ll provide a link to the conference papers once the USENIX folks make them public.
I’ve thought about things like this as well. Holographic storage is a natural fit, plus you may end up being able to do amazing things. Imagine getting the md5 checksum of a file without having to compute it on the fly. It would just be a property of the file. Want an encrypted version? No need to do the work, it’s all another property of the file. Hopefully the real world works like my dreams 😉
This sounds like a mix between some uses of extended attributes, named pipes, unix sockets, BSD’s portalfs (not production-worthy) and FUSE, and current web dev best practices. E.g. making your unix $HOME/.plan a named pipe to give a different message each time someone finger’s your account; delivering location- or browser- specific content from the same url.
@Matt
It might be handy to define an extended attribute to contain the md5 checksum of the file, though you need to either update this on each write (ugh) or re-compute at regular intervals (ugh). An encrypted version of the file will require approximately duplicate space. Filesystems/storage are beginning to handle checksums/encryption internally; how about an open(2) that allowed access to that info (yeah, very problematic).