Now days there are more and more cloud services available to the public. The diversity of these services are welcomed by the users and the developers, but this also makes the choice harder.
One of the tools which could make the choice easier is benchmarking. With the help of it, both the user and the owner could gain useful informations about the properties of the system. However because of said diversity, benchmarking isn’t as straight forward as it used to be with traditional systems.
Because of the young age of the field and the robust conformation compared to the traditional systems, there are no sophisticated, accepted method for the benchmark of cloud services. Most of the times random or intuitive metrics are taken into consideration, making it hard or nearly impossible to compare the results.
In this essay, I’m trying to compare two of the current Platform as a Service (PaaS) providers (Google App Engine and Windows Azure) through a somewhat real life example (TPC-W, e-commerce application).