Skip to content

Initcwnd settings of major CDN providers

In November 2011, we first published the initcwnd values of CDNs, following our blog post Tuning initcwnd for optimum performance that showed how tuning the initial congestion window parameter (initcwnd) on the server can have a significant improvement in TCP performance.

CDNs constantly tune their platform to improve performance. When we updated this article on August 27 2014, we measured higher initcwnd values for many CDNs (compared to the 2011 measurements). The most recent measurements (February 13 2017) again show some CDNs have increased the initcwnd size but interestingly for other CDNs we measured a lower value than in 2014. Also, quite a few CDNs have still not changed to be higher than the default 10 in Linux kernel.
First we show you the data, then some conclusions and finally we describe our test methodology.

CDN initcwnd values

CDN Initcwnd value (Feb 2017) Initcwnd value (Aug 2014)
Cachefly down46 66
Level3 (now Lumen) up46 12
Highwinds up44 36
Akamai up32 16
Edgecast (now Edgio) up30 10
CloudFront up25 10
CDNetworks up22 10
Tata Communinications same10 10
MaxCDN (now Stackpath) down10 32
Fastly same10 10
Cloudflare same10 10
CDN77 same10 10
Leaseweb same10 10
The initcwnd value is not the only parameter that determines CDN performance. Don't think CacheFly is the fastest CDN globally because their initcwnd is highest. The initcwnd value is just *one* of the performance parameters.

Conclusions

Six out of the measured 15 CDNs have a initcwnd of 10. This is the default value in Linux kernel and apparently many these CDNs have found this to work well. Most the others set the initcwnd to be higher than 20 except for Limelight which is at 14. CacheFly still is at top with a value of 46. This is 20 lower than the 2014 measured value but very likely this is a direct result of how we measured initcwnd this time which is different from last time. Read on ...

Test Methodology

During the 2014 measurements, the tests were conducted on a Macbook Air in The Netherlands with an initial receive window (initrwnd) set to a high 262144. For each CDN, we made requests to some far-away POP (US-West or APAC) to ensure a high RTT to make it easier reading tcpdumps. For each test, we made few hits to prime the cache at the edge servers. We then studied the tcpdumps, ran the entire process several times for sanity check. For the global IP Anycast CDNs like Cachefly and Cloudflare We added an extra 500ms latency using Dummynet. No dummy packet loss was added.
We used this python script to run tests and capture results using tcpdump. The dumps were manually analyzed in wireshark as described here

That was all in 2014. In 2017, we got lazy and simply used the Initcwnd Checker tool on this CDN Planet website. We measured the initcwnd of every CDN a few times. The tool works just fine. The only differences with the 2014 testing methodology are:

  1. the initial receive windown was 65000
  2. the tests ran from a AWS EC2 instance in US-East
  3. we did not take the trouble to figure out what the IPs are of far-away POPs and/or add latency to have higher RTTs

Differences 2 and 3 are not so relevant but the lower initial receive window is important and explains why we measured 'just' a initcwnd of 46 for Cachefly. 46 packets at ~1400 bytes each is ~65000 bytes and that is the initrwnd on the test machine. In other words, the test machine could not receive more than 65000 bytes in the first roundtrip and so Cachefly served no more than 46 packets. Had we advertised a higher initrwnd, maybe Cachefly would have sent out more packets.

Webmentions

1 Reply

ふくい.tsx ふくい.tsx
ただしこの14KBというのは、大体の配信環境で10パケット(- ヘッダー = 約14KB)というだけで、CDNによってはもっと高い値に設定していることもあるらしい cdnplanet.com/blog/initcwnd-…