By Dirk Paessler • May 17, 2010
"Real World" Performance Comparison of CDN (Content Delivery Network) Providers
Since February visitors of our website have the possibility to measure "website asset delivery speed" of 24 CDN and cloud providers using their own internet connection (see the blog post). The results for each user are stored in our database and we now have gathered the performance results of 340,000 requests.
There is something special about our test: We have "real surfer" measurements for the 24 providers because we have actually used the "real world" Internet connections of our website visitors to run the test requests. Most CDN comparison tests that we have seen published on the Internet were created by using just a few measurement points, and often - to make things even more unrealistic - the measurements were taken out of professional data centers (which, in most cases, have high speed connectivity anyway). 24% of the tests were run from inside the USA, 9% each from Germany and the UK, the rest came from countries all over the world. So now it is time to dive into the data and draw some conclusions.
Introduction: How Providers Were SelectedFirst we have created accounts with a selection of 24 CDN and cloud computing providers. We chose those who did not require entry barriers and offered pay-as-you-go pricing. We did this together with our friends from cloudharmony.com (we leverage each other's accounts for CDN testing). Thank you, guys! We wanted to know how much faster it is to use dedicated CDNs for website assets compared to serving these assets directly from the website's web server. That is the reason we not only tested CDN providers but also cloud computing (PaaS) providers.
Lesson #1: Using Any CDN is Better Than Not Using OneWhen we compared the CDN sites with the cloud computing based sites we found that almost all CDNs we have tested are faster than all servers hosted by cloud computing providers. Even if you are using the fastest cloud server in our test, using a CDN for website assets will likely slash asset load times by up to 50%, especially for users outside the U.S. Using a CDN is the second recommendation in Yahoo's "Best Practices for Speeding Up Your Web Site".
Lesson #2: First Requests Are Much SlowerWhen we looked at the data we found that for all sites the first request is always the slowest. The initial requests took up to twice as long as the 9 subsequent requests. Reasons for this behavior are:
- The hostname must be resolved into an IP using the DNS system (and this can take even longer for a CDN than for normal systems) before the server can be contacted for the first time
- Some CDNs support http/1.1 keep alives (so for subsequent requests no new http connection must be created)
- The edge servers of the CDNs in most cases have to fetch the file from the origin server (they are then cached for subsequent requests)
It is obvious that all providers need more time for the initial request.
Our Actual MeasurementsIf you have a website with heavy traffic from one region then your assets are likely already cached in the edge servers of a CDN because they were recently requested by a previous website visitor (most CDNs keep a file in the cache for 24h). For websites with only little traffic from one region the chance to have website assets already cached on the edge server closest to you becomes smaller and smaller. This must be taken into account when we compare the average request times of the CDN providers: The list of the fastest providers look different for "fastest initial request" and "fastest average request". First the results from a global perspective:
Secondly, the results from the USA only:
Lesson #3: Which Provider Wins The Comparison?Well, to some extent it depends on the amount of traffic your site receives from its target region. If your site has enough traffic to have your assets likely stored in the CDNs' edge servers for most users, look at the upper graphs. If you have less traffic and most users will likely cause the CDN to pull the asset from the origin server, look at the lower graph which is sorted by the request times for the first request. Anyway, the following 4 providers all hold solid positions in the top 10 of all four lists. They provide the best overall performance from a global and a north American perspective and I would consider all of them solid choices to serve website assets through a CDN:
- CacheFly CDN (#1, #2, #1, #4)
- SoftLayer/Internap CDN (#3, #3, #2, #4)
- GoGrid/Edgecast CDN (#4, #2, #10, #3)
- Google Appspot (#1, #6, #2, #8)