Just when everyone agreed that scale-out infrastructure with commodity nodes of tightly-coupled CPU, memory and storage is the way to go, Facebook’s Jeff Qin, a capacity management engineer – in a talk at Storage Visions 2015 – offers an opposing vision: disaggregated racks. One rack for computes, another for memory and a third – and fourth – for storage.
The rationale: applications need different amounts of each resource over time. Having thousands of similarly configured servers ignores this fact and leads to substantial – at FB scale – waste.
They’ve also found that different components reach functional obsolescence at different rates. Refreshing hardware at the rack level is simpler than opening thousands of servers and replacing dusty bits.
Enabling this dramatic change is their new network. No details on this network, but it must offer high bandwidth and extraordinarily low latency.
Another rack resource coming soon: optical cold storage racks starting at 1PB and expected to go to 3-4PB with the advent of 400GB optical discs.
The StorageMojo take
Holy disaggregation, Batman! The hooded crusaders at Facebook are roaring out the Zuckcave with architectures blazing. Maybe hyperscale is even odder than we imagined.
What does this mean for the rest of us? A first approximation: very little.
Facebook is an amalgam of services with very different requirements: instant messaging; friend news feeds; gaming; video; long-term photo storage; and oodles of advertising and user tracking.
An Amazon home page draws on over 100 distributed asynchronous services, but the focus is your shopping cart and payments. Facebook is, in comparison, a realtime feed mashed up with a massive personal archive.
Facebook is popular culture and its application resource requirements reflect that. Apps, like memes or fads, ebb and flow with user’s whims. Search, by contrast, is almost static.
To the extent that there is a larger lesson, it’s the network that FB has designed. If they can actually make disaggregation work the network is key.
The advantages of stripped down, warehouse-optimized LANs recall the earlier battle between RISC and CISC in CPUs. Simpler, cheaper and faster vs complex, costly and slower.
That is an idea with legs.
Courteous comments welcome, of course. As is traditional, Internet access at CES is spotty.