If memory serves – and mine often doesn’t – I asked a panel at the NVM Workshop at UCSD their opinion on using Thunderbolt as a cheap, fast, and flexible interconnect. After all, I thought, academics always need more than they can afford, so these guys would have been looking into it.
Nope! They laughed at the very thought, then confirmed their ignorance by assuming that Thunderbolt was limited to Macs. Even very smart people make mistakes!
So I was pleased to see Thunderbolt used as a cluster interconnect by a couple of companies at NAB 2016. The furthest along – though not yet shipping – was Symply.
Started by Alex Grossman – a former Apple storage guy – with assists from Promise and Quantum, Symply is aiming at the media and entertainment space with Thunderbolt-based storage for workgroups and individuals.
Why Thunderbolt?
Let me count the reasons.
- Fast. Thunderbolt 3 is 40Gb/s, making it one of the fastest shipping interconnects on the market. Low latency too.
- Free. It’s built into all Macs and a rapidly growing number of motherboards from the likes of Lenovo and HPE.
- Robust. I’ve been using Thunderbolt 1 for four years and it is very solid.
- Support. Intel is pushing it hard and Microsoft supports it in Windows Server. Linux supports it as well.
- Faster. When Intel announced Thunderbolt their roadmap went up to 100Gb/s. That’s still their plan.
- Flexible. I think of Thunderbolt as a layer 1 and 2 pipe that enables all kinds of other protocols such as IP, PCIe, USB 3, DisplayPort and more.
But will it switch?
While I came to love Thunderbolt after I started using it, it’s limited to eight nodes. That’s not much of a cluster.
So I was curious about how Symply broke that limit to build a cluster large enough to handle eight editing stations plus storage. Very simple!
They use PCIe as the protocol and PCIe switches. Thunderbolt 3 provides the fast physical pipe, essentially replacing the PCIe physical layer. Since PCIe sends data in packets, architecturally it can run over multiple physical interconnects.
Quantum’s StorNext cluster file system, which is built into Mac OS and available for many other systems, provides the file access and locking mechanisms that clusters require. Potentially, then, Symply – or poor academics – could build much larger clusters using Thunderbolt.
The StorageMojo take
When DSSD’s plan to use PCIe first appeared, I was dubious since there weren’t any large scale PCIe switches available. But that’s changed in the last two years, with IDT, Avago and Broadcom all offering cheap 24 port switch chips.
I recall hearing that originally PCIe was intended as a server interconnect, not a local bus, but Intel took it to where the demand was as PCI ran out of steam. It seems that using Thunderbolt, PCIe is about to deliver on its early promise.
Courteous comments welcome, of course.
While I’m sure symply will find some niche customers, I don’t see this spreading much further unless servers start shipping with thunderbolt more often. Most SANs connect to servers, not directly to clients. It will be hard to compete against the incumbent TCP/IP networks even if it’s cheaper and faster.
This is probably a good play for Mac based media teams, but I think they need to ditch the complexity of Xsan/stornext to make that convenient enough over an off the shelf NAS solution or direct attach storage. Even wrapping that in a gui with support is not sufficient in my mind… We all know how that went last time…