Performance Tuning Tip: Turn on time-taken in your web logs (IIS, Apache, Tomcat) to identify performance hot spots

Everyone knows that the web logs generated by web servers like IIS, Apache, and Tomcat (JBOSS / Catalina) are a well-known goldmine of information.  They allow you to track site usage, find content errors, gather browser statistics, and all sorts of other incredibly useful types of information.  Slap together some basic linux (or cygwin) commands like grep, sort, cut, and uniq, and you can do some very sophisticated analysis.  If that’s not enough, there are thousands of free and paid tools to help with the analysis.

One often overlooked benefit is the ability to gather detailed performance information.  Each hit record how long it took to process a request in milliseconds, and this can be a goldmine of information.  You can use the data answer important performance-related questions like:

• How does your response time fluctuate based on time-of-day?
• How does your response time fluctuate based on server load?
• How does the response time of your servers compare with each other (assume you have a web farm)?
• What pages are the slow pages?
• Does response time correlate with the size of the page being returned?
• Do specific IP addresses (i.e. customers) see higher response times than normal?
• My customers are complaining about a performance blip – how long did it last? was it all pages or just specific ones?

Being able to answer questions like this is vital to being able to identify performance hot spots.  Even more importantly, it allows you to objectively measure performance improvements after you make them.

Unfortunately, for reasons I cannot fathom, time-taken is not turned on by default in either IIS or Tomcat.  It is incredibly useful and easy to collect, so why not turn it on by default?

Here is how to do it in IIS 6.0:

1. Open your website properties
2. Make sure “Enable Logging” is turned on, and then click Properties
   Image may be NSFW.
   Clik here to view.

    

3. Click on the “Advanced” tab
4. Turn on the Time Taken option
   Image may be NSFW.
   Clik here to view.

Here is how to do it in Tomcat / JBOSS:

This will vary depending on your specific Tomcat / JBOSS configuration, but here is what worked for me:

1. Open up <JBOSS_HOME>serverdefaultdeployjboss-web.deployerserver.xml
2. Find the entry for “org.apache.cataline.valves.AccessLogValue”
3. Make sure that “%D” is included in your format string

Image may be NSFW.
Clik here to view.

Image may be NSFW.
Clik here to view.