Comments on: The Amazon keynote at FAST ‘09 http://storagemojo.com/2009/03/04/the-amazon-keynote-at-fast-09/ Data storage info & analysis Thu, 18 Mar 2010 02:52:50 +0000 http://wordpress.org/?v=2.9.2 hourly 1 By: How Amazon builds the world's most scalable storage | Storage Bits | ZDNet.com http://storagemojo.com/2009/03/04/the-amazon-keynote-at-fast-09/comment-page-1/#comment-199460 How Amazon builds the world's most scalable storage | Storage Bits | ZDNet.com Mon, 09 Mar 2009 17:19:15 +0000 http://storagemojo.com/?p=1131#comment-199460 [...] welcome, of course. There’s a longer version of this on StorageMojo. And there’s the Amazon CompSci paper Dynamo: Amazon’s Highly Available Key-value Store. [...] [...] welcome, of course. There’s a longer version of this on StorageMojo. And there’s the Amazon CompSci paper Dynamo: Amazon’s Highly Available Key-value Store. [...]

]]> By: Robin Harris http://storagemojo.com/2009/03/04/the-amazon-keynote-at-fast-09/comment-page-1/#comment-199458 Robin Harris Mon, 09 Mar 2009 02:30:29 +0000 http://storagemojo.com/?p=1131#comment-199458 Don, I've been noodling around that very problem for a while. The CapEx advantages don't seem large enough to justify it - sure, scale out clusters are cheaper, but it would take a lot for the average enterprise to notice due to legacy overhead - but the management advantages are another story. If the enterprise has a large volume of not-very-hot data that they want online - think media, health, geophysical - then the management advantages make sense. More later. Robin Don, I’ve been noodling around that very problem for a while. The CapEx advantages don’t seem large enough to justify it – sure, scale out clusters are cheaper, but it would take a lot for the average enterprise to notice due to legacy overhead – but the management advantages are another story.

If the enterprise has a large volume of not-very-hot data that they want online – think media, health, geophysical – then the management advantages make sense. More later.

Robin

]]> By: Donald B. http://storagemojo.com/2009/03/04/the-amazon-keynote-at-fast-09/comment-page-1/#comment-199456 Donald B. Mon, 09 Mar 2009 01:32:39 +0000 http://storagemojo.com/?p=1131#comment-199456 Great notes Robin - makes me wish I had went. I'll have to read your blog more frequently again. Raises the question, at what point does it make sense for the average enterprise to look at 'internet-scale' tactics? Great notes Robin – makes me wish I had went. I’ll have to read your blog more frequently again. Raises the question, at what point does it make sense for the average enterprise to look at ‘internet-scale’ tactics?

]]> By: Storagezilla http://storagemojo.com/2009/03/04/the-amazon-keynote-at-fast-09/comment-page-1/#comment-199446 Storagezilla Fri, 06 Mar 2009 22:44:57 +0000 http://storagemojo.com/?p=1131#comment-199446 "In a later conversation with Alyssa I noted that Google seems to have a problem scaling their clusters beyond 7-8 thousand nodes and asked where Amazon’s clusters topped out. She said they hadn’t seen that problem yet. Hmm-m-m. . . ." Look at Google now and think back to Google in 1999. What's changed? I'd say not much. The reason they've topped out is that they've stopped innovating. The worst thing happened to them, they won. If they had evolved search beyond firing some keywords into a text box they'd need another scale jump and maybe even a rethink of their architecture, but they're not so what they have is good enough. Their innovation cycle is over and their biggest initiatives are a web browser (So 1990s) and a mobile phone operating system (So early 2000s) “In a later conversation with Alyssa I noted that Google seems to have a problem scaling their clusters beyond 7-8 thousand nodes and asked where Amazon’s clusters topped out. She said they hadn’t seen that problem yet. Hmm-m-m. . . .”

Look at Google now and think back to Google in 1999. What’s changed? I’d say not much. The reason they’ve topped out is that they’ve stopped innovating. The worst thing happened to them, they won.

If they had evolved search beyond firing some keywords into a text box they’d need another scale jump and maybe even a rethink of their architecture, but they’re not so what they have is good enough. Their innovation cycle is over and their biggest initiatives are a web browser (So 1990s) and a mobile phone operating system (So early 2000s)

]]> By: Tim http://storagemojo.com/2009/03/04/the-amazon-keynote-at-fast-09/comment-page-1/#comment-199421 Tim Wed, 04 Mar 2009 14:31:31 +0000 http://storagemojo.com/?p=1131#comment-199421 james hamilton has posted the slides: http://perspectives.mvdirona.com/2009/02/25/FAST2009KeynoteAWSS3.aspx james hamilton has posted the slides:

http://perspectives.mvdirona.com/2009/02/25/FAST2009KeynoteAWSS3.aspx

]]> By: Justin Mason http://storagemojo.com/2009/03/04/the-amazon-keynote-at-fast-09/comment-page-1/#comment-199420 Justin Mason Wed, 04 Mar 2009 13:51:02 +0000 http://storagemojo.com/?p=1131#comment-199420 Hi -- James Hamilton posted the slides here: http://perspectives.mvdirona.com/2009/02/25/FAST2009KeynoteAWSS3.aspx Hi — James Hamilton posted the slides here:

http://perspectives.mvdirona.com/2009/02/25/FAST2009KeynoteAWSS3.aspx

]]> By: Pete Steege http://storagemojo.com/2009/03/04/the-amazon-keynote-at-fast-09/comment-page-1/#comment-199419 Pete Steege Wed, 04 Mar 2009 13:39:49 +0000 http://storagemojo.com/?p=1131#comment-199419 Two things jumped out: 1. More sites to deal with Internet latency. They can't speed things up with more or faster storage in a data center, so they build more data centers. The power of having a big enough plan! 2. Seat belts and air bags vs. belts and suspenders. Great analogy! Can you expand on this part of her presentation? Thanks for sharing, very interesting topic and speaker. Two things jumped out:
1. More sites to deal with Internet latency. They can’t speed things up with more or faster storage in a data center, so they build more data centers. The power of having a big enough plan!
2. Seat belts and air bags vs. belts and suspenders. Great analogy! Can you expand on this part of her presentation?
Thanks for sharing, very interesting topic and speaker.

]]> By: Steve Jones http://storagemojo.com/2009/03/04/the-amazon-keynote-at-fast-09/comment-page-1/#comment-199416 Steve Jones Wed, 04 Mar 2009 10:43:11 +0000 http://storagemojo.com/?p=1131#comment-199416 Extremely interesting stuff - and what it says very strongly is that you need an application architecture which can exploit (and withstand) infrastructure "glitches". However, very many businesses don't have the scale and the engineering ability to do this - often their systems are integrations of "best of breed" and legacy applications which don't easily deal with issues such as geographic separation, latency, inconsistent data and so on. For many companies, the mantra used ot be to buy and not build applications. Now it might be that a really advanced service provider could extend the SaaS model to provide easy self-build of WEB-based application components on a common infrastructure to allow more modest sized organisations to utilise these sort of platforms. However, I still think that's a major challenge for many companies. It would also appear that many of the other massively-scalable e-businesses on the Internet have similar general approaches, some of whom have been more successful than others (take something like YouTube and they clearly have big issues of keeping data in step in a timely manner, but maybe that doesn't matter for their services - if eBay goes down, tens of thousands of traders lose business; if there is a YouTube glitch, it;'s just a minor annoyance). Extremely interesting stuff – and what it says very strongly is that you need an application architecture which can exploit (and withstand) infrastructure “glitches”. However, very many businesses don’t have the scale and the engineering ability to do this – often their systems are integrations of “best of breed” and legacy applications which don’t easily deal with issues such as geographic separation, latency, inconsistent data and so on.

For many companies, the mantra used ot be to buy and not build applications. Now it might be that a really advanced service provider could extend the SaaS model to provide easy self-build of WEB-based application components on a common infrastructure to allow more modest sized organisations to utilise these sort of platforms. However, I still think that’s a major challenge for many companies.

It would also appear that many of the other massively-scalable e-businesses on the Internet have similar general approaches, some of whom have been more successful than others (take something like YouTube and they clearly have big issues of keeping data in step in a timely manner, but maybe that doesn’t matter for their services – if eBay goes down, tens of thousands of traders lose business; if there is a YouTube glitch, it;’s just a minor annoyance).

]]>