Ajax Performance

Happy Thanksgiving, to those in the US

I’m heading to Chicago tomorrow, but I hope to have a report on the benchmark data posted Friday for your post-Thanksgiving consumption . . .

If you are reading this post, you are contributing to a benchmark

In several earlier posts, I talked about the increasing migration of complexity to the edges of the network, to within the web browser. Today, tools to manage and understand that complexity and how it relates to real user experience don’t really exist, and that’s a problem we have been thinking about for the past year or so.

We realized that the best way to understand what is happening inside the end user’s browser is to be inside the end user’s browser, so we are building a JavaScript based technology, dubbed Actual Experience XF, which does just that. I previewed this approach during my presentation at the Ajax Experience, but now I’m finally allowed to demonstrate it publicly.

An early demo, which I am calling Actual Experience Lite until my marketing department punches me in the head, is available at aelite.ajaxperformance.com. To instrument a page for collection of metrics, you first include a small .js file in HEAD. Out of the box, the tag will track the DOM Ready and onload times as well as additional metadata (browser type and version number, etc).

The tag also provides 3 JavaScript functions — nameEvent, startInterval, and endInterval — which can be used to add timing instrumentation anywhere within your client side code. The first, nameEvent, defines a point-in-time event, while the latter two are used to measure a span of time. Here’s a simple use case:

 var xhReq = createXMLHttpRequest();
 xhReq.open("GET", "someGet.aspx?giveMe=stuff&iWant=stuff", true);
 xhReq.onreadystatechange = onSomeResponse;
 startInterval('some request');
 xhReq.send(null);

 function onSomeResponse() {
   if (xhReq.readyState != 4)  { return; }
   endInterval('some request');
   var serverResponse = xhReq.responseXML;
   // do a bunch of stuff with the xml response
   nameEvent('finished response parsing');
 }

This example would create two events: an interval event called ‘some request’ with a start time and a duration, and a point in time event called ‘finished response parsing’ with a start time and no duration. All start times are relative to the load of the aelite.js tag.

Code samples are all well and good, but real world data is what keeps my 3 readers coming back. In the hopes of providing a valuable service to the community, I instrumented a copy of the W3C vs. innerHTML table building benchmark from quirksmode. I embedded my aelite-ified benchmark as a hidden iframe in the footer of this blog, so any visit to any page contributes to the benchmark. Results are available here.

I’m hoping that the aggregation of hits over time from a variety of browsers will give us a more meaningful understanding of the costs of different scripting techniques, and I will use this platform to build a suite of interesting benchmarks over time. Suggestions are always welcome, of course.

All of the graphs are created client side by Dojo Chart, packaged as a kitchen-sink build plus either the svg (Mozilla/Opera) or vml (IE) packages. The charting page itself uses aelite, so any visit to a chart page is also a contribution to a benchmark of Dojo Chart. Since the Dojo libs [can] have a very robust girth (350kB or thereabouts [when packaged in a completely unoptimized fashion for the purposes of establishing a worst-case benchmark]), I instrumented the charting page to measure the time it takes for the browser to parse the entire dojo-[svg,vml].js. I also added an interval measurement around the render time for the chart. It will be interesting to watch performance trends as I track new Dojo releases.

I will be at Web Builder 2.0 in Las Vegas, December 4-6, presenting on the 5th and conducting a full day workshop on the 6th. The workshop will include hands on labs covering the instrumentation approach, so I plan to have many more samples available in the coming weeks.

[Update] I have replaced the dojo-[svg,vml].js with a single dojo.js provided by Alex Russell. It includes both renderers and is less than half the size of my previous build. Feel free to pound away at the charting to give us some nice comparison data

It’s a mad, mad, mad, mad, composite world

In my last post, I spoke of the evolving understanding of Ajax performance within organizations and mentioned some new challenges resulting from reliance on third party content and code. Today, I will give that discussion the consideration it deserves. And sexy graphics!

Circa 2000

Let’s look back to the heart of the .com boom. Web development was a pure three tier model with a data layer, a business layer, and a presentation layer. The browser was typically a thin and dumb (even obtuse, often) client, so all three tiers were encapsulated within a server infrastructure under the direct control of the organization. Also, since most organizations only built for one browser, there was little variability to worry about at that level.

The internet cloud adds latency and uncertainty to the delivery of that content, so most organizations treated this as a cost of doing business, an unavoidable reality of the web, and contracted with a company such as my employer, Gomez, to help monitor and manage the internet cloud. Since the network components were beyond the control of the organization, working with Gomez was considered an insurance policy, typically a reactive measure to minimize downtime.

Circa 2007

click image for full size

Leap forward to 2007, and we have a very different landscape. To an individual organization, the three tier model still applies. The underlying components within the datacenter are likely quite similar, containing some mix of databases, application servers, web containers, and web servers. However, a broad range of third party components are now involved in the system, so there are many more variables impacting end user experience.

SOA components may provide business logic and data to your systems from a third party. You may be using CDNs or DNS trickery to move your content and applications closer to the end user. And you may be doing application composition within the browser itself, ranging in complexity from advertising and CDN hosting of images or media up to entire chunks of the business process being provided by a third party through portlets.

Another complicating factor is the evolving role of the browser. Now there are at least three mainstream browsers that need to be supported, and the browser is increasingly treated like a smart client, an application runtime environment, within which presentation logic is executed to compose the user-facing application from all of these disparate components. Ajax and the trend toward RESTian development is moving more of your application complexity to the end user.

What hasn’t changed is the portion of this system owned by you, the owners of www.acme.com. Everything that happens outside of your firewall, whether it is code or data from third party sites that make up the composite application or presentation logic running within the browser, is outside of your control. But the end user only sees www.acme.com. If any piece of the elaborate web of interdependent services suffers, it is your team that is blamed for the poor user experience.

A cool Ajax-based performance tool: OctaGate SiteTimer

Just to set expectations, this performance tool does not execute JavaScript and therefore can’t analyze your Ajax applications, but OctaGate SiteTimer has a dead sexy interface for viewing the results of a basic performance test of your page.

What is a page? Part 3

Active measurement of web application performance involves simulating a real end user hitting a series of pages of the application and performing some action to move from one step to the next. An example transaction may start with a user loading up www.ebay.com, searching for ‘Beanie Babies’, and clicking the first result of the Beanie Baby search. That transaction would be 3 steps — home page, search results, and listing for the first search item. Organizations who perform this type of testing on their production applications have developed an understanding for how long each step should take and use this expectation to benchmark their product over time. This is the page-by-page model I discussed in Part 2.

So, along comes Ajax, blowing to hell all of our cozy institutional familiarity with measuring application performance. We need to update the model by adding in an understanding of the times perceived by the end user. In the old world, that was easy — the user clicked a link or submitted a form, the page refreshed, and the user waiting for content to finish loading. To provide an Ajax analogue, deconstruct the current model — the user takes an action, something happens, and the user receives a result. The whole page refresh thing goes out the window, but in its place we just need the flexibility to ascribe appropriately granular meanings to ‘something happens’ and ‘user receives’ a result.

So, let’s now call a page of a transaction a ‘step’, and let’s call a step:

Any user interaction that the developer considers a meaningful milestone within the business process.

The key word in there is developer. Historically, development hasn’t been very involved in performance testing, accepting the occasional load test. Ongoing performance testing has typically been the domain of the operations group. In the Ajax world, the developer is drawn into the performance discussion because no one else in the organization has a comparable understanding for how the user will interact with the application, where steps begin and end.

Also, Ajax developers are pushing more complexity to the edges of the networks, to the web browsers. In the past, it made sense for the operations group to have the exclusive focus on performance since most of what could contribute to degradation (servers, switches, upstream bandwidth) lived within operational control. If a database started to go wonky, operations would see it and get development involved. Ajax and the increasing reliance on third party content and code are making it harder to know how a site will perform once it goes live, so the era when developers could ship an application without doing rigorous performance testing is rapidly drawing to a close.

Our most mature customers design applications with this changing dynamic in mind. Developers build a testing plan for measuring the ‘steps’ of the application, and this testing becomes release criteria for the application. Until a good user perceived experience can be demonstrated, the application doesn’t ship.

Oops, make that 4 seconds

Looks like my algorithm for calculating user-acceptable latency needs to be updated. Akamai says it’s now 4 seconds, at least for retail sites.

I’m sure increasing broadband adoption plays a huge role in the downward trend of user toleration for latency, but I also expect that Ajax is a driver as well. Ajax gives developers finer grained controls for hiding network latency from the user, and the technique is mainstream enough that users can’t help but include their experiences on speedy Ajax sites in their expectations.

Wow, professional hosting!

I moved this site to DreamHost today. Until now, it was running from one of my servers at home. Hopefully the new hosting will give us more speed and reliability.

Network Computing coverage of speedbumps in Ajax adoption

Yet another interesting article, this time from Lori MacVittie at Network Computing, discussing the risks as enterprises start to incorporate Ajax and RIA technologies.

I love these snippets:

Web 2.0 and RIA (Rich Internet Applications) will dramatically change your infrastructure in terms of monitoring, management, deployment and availability.

[T]he developers who blazed the way rarely concerned themselves with details like management, security, scalability or support. They were in it for the thrill of the hunt.

Most customers I’ve worked with run into exactly the issues listed in this article. Because we’re still in a very immature phase of adoption, Ajax is often introduced into applications without a lot of thought about manageability, performance, etc. I touched on this some in the first section of What is a page?, and it’s gratifying to see more discussion of this in the mainstream enterprise press.

Another great article about performance

Aaron Hopkins of Google wrote an article on Optimizing Page Load Time which, while not Ajax specific, provides some great lessons on tweaking the performance of web applications. In particular, the code he providers for measuring user bandwidth within the browser and reporting it up to his logs is very similar to a technique I will discuss in great detail in the coming weeks. Just waiting for the gag order to expire . . .