Circumventing browser connection limits for fun and profit

December 18, 2006 on 11:14 pm | In ajax |

A few days ago, this video hosted by metacafe popped up on digg, explaining how to increase site download times by tweaking your browser settings to increase connection parallelism. To explain why this works, let’s step back a bit to discuss how browsers manage server connections.

In building any application, developers are often required to make ‘utilitarian’ choices. Pretentiously paraphrasing Jeremy Bentham, ‘utilitarian’ describes an approach that ‘does the greatest good for the greatest number.’ Many times, sacrifices in performance are made for a subset of users so that the average expected performance of all users will be better.

Since all browsers were originally developed for a time when the vast majority of users were on bandwidth-constrained dial-up links, it made sense to restrict users to a small number of connections. The overhead of juggling many connections over dial-up makes it difficult for progress to be made servicing any of the individual requests. Also, web servers and proxy servers in that era were less robust, so keeping the per-browser connection pool small reduced the risk of overwhelming the network infrastructure.

To find a decent balance, IE and Firefox by default restrict users to 6 connections total and 2 connections per host for HTTP 1.1 connections. HTTP 1.0 is a slightly different story, but the benefits of persistent connections means that you should be (and probably are) using HTTP 1.1 anyway.

Of course, in the real world, these utilitarian decisions have a tendency to rot, outliving the time when they are relevant. Today, the majority of users have broadband connections, so client side bandwidth is not the gating factor. Typically, latency in retrieving individual objects is dominated by the time required to setup a connection and send a request. By increasing the number of concurrent connections, we can parallelize that cost and churn through the list of pending objects more quickly, leading to an increase in user perceived download time. “Lightning fast,” to echo the hyperbole of the metacafe clip.

Unfortunately, relying on your users to modify their browsers is not a reasonable optimization strategy, so what can the concerned developer do to tap into these gains?

Most sites only use one host, so requests are forced to share the 2 connections to that host. One effective strategy to improve parallelism is to spread your content out across multiple hosts. This is not hard to do as browsers look at the hostname, not the ip. That is, images1.yoursite.com and images2.yoursite.com would each be allowed 2 connections.

As a practical example, check out this sample application Buddy and I built for our workshop at Web Builder 2.0. The album thumbnails are by default all loaded from the parent host, so they all share from two connections. Below is a sample ‘waterfall’ chart of the objects loaded for this page, collected by Gomez’ external performance testing service.

two conn waterfall small

The full version of the chart is available behind the link.

You can see from the graphic that only two connections (C0 and C2) are opened for musicstore.ajaxperformance.com. We are using HTTP 1.1, so we do not need to open a separate connection for each image, but we are still spending the majority of our time servicing object requests. The object times are dominated by the cost of requesting the images (the blue first byte time) and very little time is spent downloading the content (the red content time).

To improve performance, we created CNAMEs for images1.ajaxperformance.com, images2.ajaxperformance.com, and images3.ajaxperformance.com, all of which point back to our main host. This small snippet in our Rails album layout code spread each album image across a different host:

img_url = "/images/album_art/#{album.id}.jpg"
if @perf_multiplex_images
  idx = (album.id % 3) + 1
  img_url = "http://images#{idx}.ajaxperformance.com/images/album_art/#{album.id}.jpg"
end

You can see the results by hitting the updated version. First load performance should be much better. As you can see from the below waterfall (again, the full chart is available behind the link), we are now using 6 connections to grab our images.

six conn waterfall small

So what did that buy us? A lot, given the relatively simple change. Below is a head-to-head comparison of total page load time, collected over 24 hours from 8 locations around the country.

daily chart

The average load time when using 2 connections is 7.919 seconds. The average load time when using 6 connections is 4.629 seconds. That’s a greater than 40% drop in page load time. This technique will work anywhere that you have a large block of object requests currently served by one host.

There is plenty of precedent for this approach in real world Ajax apps. To exploit connection parallelism, the image tiles at Google Maps are served from mt0.google.com through mt3.google.com. Virtual Earth also uses this technique.

You can also use this connection management approach to sandbox the performance of different parts of your application. If you have page elements that require database access and may be more latent than static objects, keep them from clogging up the 2 connections for image content by putting them on a subdomain. This trick won’t cause a huge improvement in the total load time of your page, but it can significantly improve the perceived performance by allowing static content to load unfettered.

46 Comments »

RSS feed for comments on this post. TrackBack URI

  1. At Lingr, rather than specifically listing certain CNAMEs, we just use wildcard dns entries, so that, for example, .images.ajaxperformance.com would resolve to the same ip address. For , we just use a random number.

    Comment by Danny Burkes — December 19, 2006 #

  2. Oops, that should read <anything>.images.ajaxperformance.com

    Comment by Danny Burkes — December 19, 2006 #

  3. Damn I hate unexpected markup :-) Lets’ try again, from the top:

    At Lingr, rather than specifically listing certain CNAMEs, we just use wildcard dns entries, so that, for example, anything.images.ajaxperformance.com would resolve to the same ip address. For anything, we just use a random number.

    Comment by Danny Burkes — December 19, 2006 #

  4. Yeah, I should have been more explicit on that. Wildcard dns entries are a much more elegant way to solve this. We just modified the CNAME because we were too lazy to figure out how to enable wildcards with our hosting provider ;-)

    Comment by Ryan Breen — December 19, 2006 #

  5. For serving images i’ve found that it helps to use a function that will take a filename and output the same number every time, since with random numbers you risk eliminating caching.
    Thus background.gif always gets served from images2.mywebsite.com, as opposed to from images0 on one page and then images1 on another.

    Comment by Tim — December 19, 2006 #

  6. [...] Ajax Performance » Circumventing browser connection limits for fun and profit [...]

    Pingback by 304: No Mod Required » Blog Archive » Emptying the web junk drawer of my mind — December 19, 2006 #

  7. Tim, that’s a great point. Using a consistent hostname is a definite win for caching.

    Comment by Ryan Breen — December 19, 2006 #

  8. To elaborate on what Tim was saying: it’s probably better to hash the whole path to the file, not just the file name, in case you have many files with the same name in different directories.

    Also, the original post didn’t mention hostname resolution times. Shouldn’t that be factored in?

    Comment by Ned Baldessin — December 19, 2006 #

  9. Ned, good question, but I don’t think DNS times are important. In my experience, DNS resolution is typically an order of magnitude faster than connection setup and object retrieval time.

    Also, the DNS queries will only occur once for each host, and they will occur in parallel since all of the connections are created at the same time, so there shouldn’t be any effective difference between making 1 additional DNS query and making 3.

    The graphs linked from this page include DNS time as a yellow bar, but it’s so small that I can’t see any DNS time for any of the initial connections to a host.

    Comment by Ryan Breen — December 19, 2006 #

  10. Have broadband? Do this trick to make your browser load pages faster…

    There is this old hack (which i knew only today) about configuring browsers to load pages faster in faster connections. By default, most browsers seem to have configuration that fits a dialup connection better. The video in the link below will show how…

    Trackback by Harish Palaniappan's — December 20, 2006 #

  11. First we talk about most sites using a single server. Then we show some “simple” code from multiple servers and call it a small change. Adding multiple servers could hardly be considered a small change.

    Now we have several posts talking about how to name your hosts to make those 4 lines of code more efficient. Trees vs Forest and the Trees are winning.

    Comment by David — December 20, 2006 #

  12. David, not sure what you’re saying here. We are only using one server throughout this exercise, the simple code is just tricking the browser into thinking we are spanning multiple hosts.

    Comment by Ryan Breen — December 20, 2006 #

  13. I love this idea. I’d like to implement it on the site I run, but my only problem is that it’s a secure site, and every hostname needs a security certificate (which must be purchased) or else warning prompts will rear their ugly heads. Anyone know of costless workaround? Thanks.

    Comment by Georges — December 20, 2006 #

  14. David, I think you’re missing the point entirely…

    The idea is that you make the browser believe the images are coming from multiple hosts… even though images1.host.com and images2.host.com and images3.host.com are all pointing to the same folder on the same host.

    Get it? It seems redundant but the browser doesn’t realize you aren’t using multiple servers.. and thats the whole point.

    Comment by Brendan — December 20, 2006 #

  15. Georges, that’s a great question. I didn’t mention SSL at all, and that’s largely because there are other costs to multiplying your connections in this way for SSL. The connection initialization step with HTTPS is much higher than HTTP, typically handshaking takes at least as long as creating the underlying connection, so using 3 times as many connections increases that cost enough to offset some of your gains from this technique. Also, the certificate cost you mention is another issue.

    If you are downloading large objects over HTTPS, and content download time is a much higher percentage of transfer time than SSL handshaking time, one possible solution is to use HTTP 1.0 for your SSL connections. IE will open 4 HTTP 1.0 connections at a time to a given host, and Firefox will open 8.

    This is a costly approach. From a server overhead perspective, handshaking is compute intensive, and you would be doing a lot more handshaking since every request for every object would now require its own connection. This would only really make sense if you have a relatively small number of objects that take the server a long time to generate or transfer.

    I am working on a post about HTTP 1.0 to cover this in more detail, but it’s the only thing I can think of in answer to Georges’ question. I don’t think it’s likely to be a palatable answer. Anyone else have better ideas?

    Comment by Ryan Breen — December 20, 2006 #

  16. Brendan: Thanks — you said that much more succinctly than I managed. :-)

    Comment by Ryan Breen — December 20, 2006 #

  17. Just curious, what happens with DNS load balancers? I don’t know very much about them, since I only work on bitsy sites, but I’ve seen SIGNIFICANT performance degradations introduced by DNS load balancers (which seems to defeat the purpose, but some ISPs have load balancers, even for single hosts).

    Comment by Chris — December 20, 2006 #

  18. By the way, I’ve always considered Hot Rats to be Jazz *

    Comment by Chris — December 20, 2006 #

  19. Chris, that’s a great question. My belief is that any sort of DNS resolution pain, whether it’s caused by lame load balancers, etc, will be felt more with this solution because we are ratcheting up the number of queries we need to make. As I said in another response above, though, these requests are likely to fall in parallel, so the latency hit will only be felt once though it will cause a stall in all object requests.

    But another outcome of this approach is that it will put you more at risk to expose intermittent name resolution issues. Since each user is potentially making 3 extra queries, an issue that crops up N% of the time is that much more likely to be felt by any given user. I still don’t think it’s a huge concern, but if you’ve had DNS pain it’s something to keep in mind.

    And I passed on your Zappa pointer to Buddy. I think Outkast and the Beatles are the token entries he gave me — the rest are his. :-)

    Comment by Ryan Breen — December 21, 2006 #

  20. Chris, DNS Load Balancing is one of those things that you have to be careful of dissing. Most of the major CDNs use advanced algorithms when a client makes a DNS request to determine which server location to send the TCP request too.

    In the case of CDNs, the performance “price” they pay at DNS is more than compensated for by the improved performance achieved by sending client requests to a server location closer to the endpoint.

    Round-robin DNS load-balancing is ok for smaller sites, but once there are more than 3-4 servers in the mix, other options should be examined.

    DNS load-balancing is a great solution to ensure data-redundancy, but there are other solutions available that will serve the same purpose.

    smp

    Comment by Stephen Pierzchala — December 21, 2006 #

  21. Chris, DNS Load Balancing is one of those things that you have to be careful of dissing.

    Oh, I wasn’t dissing it. As I stated, I write dinky NPO sites that seldom have a GPR greater than 3.

    However, I did used to use an ISP that boasted load balancers as one of the reasons you should use them, then kept on telling me that the reason our site was a dog was because of their load balancer.

    Comment by Chris — December 21, 2006 #

  22. “increase site download times” ?

    You mean . . . decrease, ya?

    I don’t really want to increase my site download time. :)

    Comment by Danny Howard — December 21, 2006 #

  23. Ryan, you might be interested my findings on the topic of multiple hostnames from a few months ago:
    http://www.die.net/musings/page_load_time/

    My simulations suggest that there will be much greater benefit for users further away in terms of round-trip time. You mention your tests were done from “8 locations around the country”. Did the improvement vary with the ping times for these locations?

    Comment by Aaron Hopkins — December 21, 2006 #

  24. Danny, I’m an idiot, but in my defense no one else pointed that out in the last few days, so at least it was sorta subtle ;-)

    Aaron, I’ll take a look at the data and post a by-location breakdown, but it’s certainly true that there should be a strong location correlation. We can add some international testing to further explore that angle.

    Comment by Ryan Breen — December 22, 2006 #

  25. [...] Ajax Performance » Circumventing browser connection limits for fun and profit great and simple to implement performance tip (tags: programming javascript howto performance http browser connections) [...]

    Pingback by tijs.org » Blog Archive » links for 2006-12-24 — December 24, 2006 #

  26. [...] Ryan Breen has written up a detailed post on Circumventing browser connection limits for fun and profit in which he discusses the old-fashion limits of 2 connections per HTTP/1.1 per host, and the benefit you get from a simple CNAME hack. [...]

    Pingback by Using CNAMES to get around browser connection limits — January 9, 2007 #

  27. Ryan,

    This is an interesting analysis.

    Concerning the HTTPS behaviour, I confirm that disabling keepalive gives good results. On a big site (4000 hits/s), we had to do this simply because the servers could not cope with the amount of connections (apache, you know…). We’ve been running without keepalive for about 3 years now and people never complained about page loading time. Concerning the CPU, it’s not a problem. CPU is far cheaper than sockets these days. Maintaining a connection for 2 seconds instead of 1 is far more expensive than upgrading to a 20% more powerful CPU, because when you reach software limits (eg: 2000 apache processes fighting for the accept() lock) or protocol limits (eg: source ports on reverse proxies), you have virtually no solution except adding more servers. Today, you can count on about 125 req/s per GHz per opteron with 128 bit SSLv3 without keepalive. This means 2000 req/s on a quad opteron dual-core 2GHz. Depending on the application, this means anything between 5000 and 10000 users. This is really cheap. In contrast, with 16 GB of RAM, such a machine supports between about 10000 processes, which limits you to 5000 users if keepalive is enabled. It means that you can saturate your machine with 50% CPU free. Let’s use it for SSL!

    Willy

    Comment by Willy Tarreau — January 10, 2007 #

  28. Willy,
    Thanks for the feedback. That’s very interesting. I’m curious how many objects you are serving per page. There is still overhead from the roundtrips for handshaking, but it sounds like that is not as significant an issue for your application as juggling all of those connections.
    Cool stuff.
    Thanks,
    Ryan

    Comment by Ryan Breen — January 10, 2007 #

  29. [...] Offloading images ties in well to my discussion of connection parallelism back in December. Loyal readers will recall the tip for using DNS wildcards to trick the browser into assigning more connections to your host, but that does nothing to reduce your bandwidth costs. The solution I suggested is appropriate when you want to maximize connection parallelism and have bandwidth to spare. [...]

    Pingback by Ajax Performance » Real world case study in slimming down bandwidth consumption — March 8, 2007 #

  30. Great thread… re: the SSL issue… anyone tried wildcard certificates?

    Comment by Mark — May 3, 2007 #

  31. The connection chart (1st one) is neat ? Can i ask what tool are you using ?

    Comment by sagar — June 22, 2007 #

  32. Sagar, I work for Gomez, and those are screenshots from our web performance management service.

    Comment by Ryan Breen — June 22, 2007 #

  33. Oh…thats okay…are you familiar with any freeware tool that gives such details about connection, TTFB, actual processing time and TTLB

    Comment by sagar — June 22, 2007 #

  34. IBM Page Detailer might be your best bet. There is a free version that I believe provides this functionality.

    Comment by Ryan Breen — June 22, 2007 #

  35. Thanks a bunch Ryan. I will check that out.

    Comment by sagar — June 22, 2007 #

  36. Any ideas on how to implement this on windows server 2000 DNS servers? Creating CNAMES in the domain didnt work, nor did creating sub-domains and placing the CNAME record in those…

    Comment by Nate — July 3, 2007 #

  37. [...] optimization in this case, in my opinion, is to exploit connection parallelism using the CNAME hack. Given that there are hundreds of images, using 6 (IE max) or 8 (Firefox/Safari max) persistent [...]

    Pingback by Ajax Performance » Happy Sprite Day! — August 13, 2007 #

  38. [...] can read more about this at Ajax Performance and Optimizing Page Load Time Popularity: 1% [...]

    Pingback by Anirudh Lohia - Me and My Theories » Blog Archive » Optimize Site Loading Time — September 1, 2007 #

  39. [...] Ajax Performance » Circumventing browser connection limits for fun and profit (tags: performance ajax http browser connections server optimization dns) [...]

    Pingback by mcdave.net » links for 2007-09-23 — September 23, 2007 #

  40. ryan,

    hi, i’m writing something. i’d like permission to reproduce the third figure on this page, can you contact me off-blog re this? tx

    - andy

    Comment by andy king — October 24, 2007 #

  41. Does anyone have some example code that can be used to cycle objects across multiple hostnames but ensure that a given object always goes to the same hostname? Thanks

    Comment by marcus — December 7, 2007 #

  42. [...] upped the connection limit to 6 per host from the default of 2. I’ve talked before about DNS tricks to get around the 2 connection limitation, but having this support out of the box will be a great assistance in the war on round-trip latency [...]

    Pingback by Ajax Performance » IE8: The Performance Implications — March 7, 2008 #

  43. I tried to implement this using ajax calls instead of image calls but I am receiving a cross domain popup. While I understand I can suppress this by enabling Access data sources across domains in the Security zones that is not an option. Do you have any recommendations?

    Comment by Mike — April 4, 2008 #

  44. [...] more, using simple technique you can have your browser use 6 concurrent connections to parallelize fetching data as much as it [...]

    Pingback by data URI theory and practice « Sharovatov’s Weblog — May 11, 2008 #

  45. [BP-2065] Club widget downloads…

    Yes there are enough widgets that are re-used across pages. Your argument is based on the assumption that, on all pages, all widgets are requested from the server and hence browser caching is an efficient way. But in BP2.0 only required widgets (the….

    Trackback by JIRA: Build & Price 2.0 — May 22, 2008 #

  46. I tried this technique on cross-domain ajax (the script tag method). it won’t work. the limitation is still per browser, instead of per domain.

    My IE is MS IE 6.

    I don’t know how you did it, or you just did it with img tag instead of script tag?

    Comment by JX — May 28, 2008 #

Leave a comment

XHTML: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Powered by WordPress with Pool theme design by Borja Fernandez.
Entries and comments feeds. Valid XHTML and CSS. ^Top^