28 January 2008 - 21:13Amazon’s EC2

It is really interesting what is happening right now in the grid or cloud computing field: the biggest player in this field is Amazon.com, a retailer. You would have expected grid computing or cloud computing to be championed by a tech company, I personally would have thought that Sun would be pretty active in this field, at the time I believed this 3-year old announcement to be more a bit more than hot-air.

You could speculate that Google is not lifting the curtain on parts of its infrastructure for fear that it would undermine one of its competitive advantage: the ability so scale out almost effortlessly. You could speculate that the large traffic sites which use grids for strategic reasons will be reluctant to open up their architectures to the outside world to the degree that Amazon has done and to a certain extent I see their absence from this space as a the cost of keeping inner workings of your organization away from your competitors eyes. Amazon probably has gone thru the same stage where its proprietary architecture was one important differentiator between itself and its competitors, but it looks like it has gone beyond it, it has grown beyond . Probably as Google, eBay, Yahoo, Microsoft will mature they will also start opening parts of their infrastructure which are not creating a difference between their services.

But I still don’t understand why Sun’s is silent in this space. Part of its strategy to move towards open source would have been to move towards services at the same time (since your code doesn’t pay you need to make money from running services on this code). Support is one type of service, but I see grid computing as one service which is far more profitable to Sun.
Cloud computing comes with a pretty high entry barrier: you need know-how, operation teams, investments in hardware, etc… in order to run the large data-centers that are hosting this service. Sun could have tried to do that and in the article at the beginning of the post they seemed pretty determined to open up such an environment. Judging by the size of their investment in MySql you could say that Sun it committed to software. I don’t see hardware playing a big role in the future, but rather infrastructure. Once you solve the problems of scaling out the need for big iron boxes diminishes, however the infrastructure starts to matter a lot. The only thing that Sun seems to do is to sell various grid-solutions but not to host grids, I think they are missing out on a big piece of what will be demanded in the future IT landscape.

So how do I see Amazon’s grids doing in the future? I see them doing pretty well, for now the market for grid computing on a credit card (to quote Cameron Purdy) has only one seller, that is Amazon. Amazon will bank on the integration between small companies and its grid capabilities, integration which translates in in the number of VM images stored on EC2, on the different configurations that different hosts are using, etc… This integration is essentially an infrastructure lock-in and once Amazon has acquired these customers it is a safe bet that they will not migrate to its competitors (which are yet to appear) easily.

Later edit: I was thinking about using Amazon`s EC2 to host Facebook applications which start having operational problems because of becoming immensely popular (as illustrated on this post on Marc Andressen`s blog). It would probably make sense, but I didn`t dwell too much on it.