Ajax Performance

Thanks for the help! And now, some results.

A couple weeks ago I posted a request for clicks to a Dojo Chart benchmark in the hopes of determining the relative performance of SVG (or VML, in the case of IE) engines across browsers. Thanks to your help (and special thanks to a link from our good friends at Ajaxian), we have a couple thousand data points to work with. That’s enough volume to start drawing some statistically relevant conclusions, so let’s dig into the numbers.

First, the high level — the totals by browser:

Interesting stuff. Safari’s sample size was rather small since only recent Nightlies support SVG, but the early returns are very impressive. At 37ms, Safari is easily the fastest implementation, over 3 times faster than Firefox. Internet Explorer is by far the slowest, but this may be a generic statement for the speed of VML operations. Opera was the 2nd fastest browser, narrowly losing out to Safari and turning in a time over twice as fast as Firefox.

There may be some bias in the data given that the majority of people running WebKit Nightlies are likely doing so on sexy multicore Mac hardware. Let’s take a look at the breakdown by OS to see how Safari performs against other Mac browsers:

Firefox on Mac is significantly slower than Firefox on Windows. OK, so either all Firefox Mac users are on boring old PowerPC tech, the Firefox SVG implementation on Mac is slower than Windows, or my Safari hardware bias theory is totally busted. Perhaps a combination of the three. Any way you slice it, Safari’s dominating performance can’t be rationalized away — the WebKit guys have done some seriously good work on the SVG front.

Finally, let’s look at differences between browser versions to see if there is any movement in the trends release to release:

Looks like Microsoft made some improvements to the VML engine in IE7 — there is a significant improvement from IE6. In the SVG space, Firefox appears to have made some marginal improvements in 2.0. Safari SVG support is new, and the per-version sample size on Opera is too low to draw any trend conclusions.

In summary, the WebKit team should be applauded for the performance of their SVG implementation. Now, let’s get that spiffy new engine in the hands of more users. If the Safari numbers hold up once the new version ships, the Firefox team will have their work cut out for them trying to catch up. As for Microsoft, the VML implementation is twice as slow as the average SVG counterpart. This may be a case where more tuning in necessary on the Dojo side of things, but if it reflects a true limitation of their VML engine, Microsoft also has some work ahead.

Isn’t competition great?

Click a link, help a worthy cause

Frequent readers will recall that, a few months ago, I built a performance instrumentation package which supported the collection and aggregation of performance metrics from inside an end user’s browser. Timings are communicated up to a Rails app at aelite.ajaxperformance.com and graphed client side using the Dojo Chart widget.

A couple of the Dojo developers are interested in using this technique to compare the performance of Dojo Chart on different browsers. Loading this page in your browser will time the performance of rendering a chart within your browser. To contribute to our cross-browser analysis, just hit that link in Firefox, Internet Explorer, Opera, or a recent WebKit nightly (sorry, the stable Safari build is not yet supported). We are looking to get a decent sample size across real browsers, so feel free to hit that link repeatedly from every browser you have over the next few days. Tell your friends! It’s fun for the whole family!

Once we have a decent sample size, I’ll post a report on the relative performance of each browser.

Thanks for your help!