Talking to Bryan Cantrill, one of the ZFS engineers, in San Francisco a few weeks ago, I mentioned that it seemed that Google could really kick Ebay’s tail and Bryan, selected by MIT’s Review as one of the TR35 quickly responded that no one can “spray on” transaction processing (TP) capability. No doubt he is correct.
As noted in the Google File System review, GFS is not optimized for small reads and writes, so there is no way it is the basis for a big TP system. Yet with the advent of Google Checkout, a heavy-duty TP application, the company must have one.
What architecture is Google using to provide high-performance, large-scale transaction processing? Unlike GFS, there are no papers covering this architecture or even hinting around that Google has done anything at all with transaction processing. Are they using the open-source MySQL or Postgres? If so, it would be by far the biggest installation of either of these databases in the world. If anyone could make it work, it would be Google, yet it would be a huge risk.
The most likely conclusion is that Google is using a commercial solution for TP – one that no one in the industry – let alone Google – is talking about. I’ve combed through Google searching for any indication they’ve built a TP infrastructure and haven’t found it. Unless they’ve developed TP technology so amazing that they aren’t even letting their researchers publish, which doesn’t seem to fit with their free-wheeling PhD seminar culture.
Anyone have any insights into who the lucky vendor is getting many tens of millions from Google for big TP systems? Sun is an obvious candidate, since Schmidt used to work there. Yet Oracle prefers simple disk architectures, which we know Google favors, so are they getting big PO’s from Google?
Email me or comment with your thoughts.
A couple of smarter-than-me folks point out in the comments that there is no reason for Google Checkout to be a TP application. My apologies.
But that made me dig deeper, using links suggested by other readers, and I discovered a story of Google using MySQL for their AdWords database – followed by lots of entertaining (if you’re a nerd) discussion of database gack. Personally I love to listen to engineers argue because you can learn a lot about engineers (and a little about engineering) without the bother of being one.
Net-net: So it may very well be that Google is still preserving its open-source purity. I root for underdogs, so I hope so.
Bryan is NOT a member of the ZFS team. They brought him along and he talked a lot, so I just assumed he was. He is actually the Jedi Master of Dtrace at Sun.