The Scientific Method (sciencebuddies.org)

In this article I will explain a basic, but often overlooked foundation for improving the performance of any software application.

Much of software development is an art, but performance tuning is a science.  I’ve seen a lot of good developers waste time significant amounts of time on performance with little to show for it, or just as bad, improve performance without knowing exactly which change had the desired effect.

Do you remember talking about the Scientific Method from your high school science class?  The diagram on the right is a refresher.  The scientific method is the repeatable process on which all scientific exploration is based.  It gives scientists across the world a common language and framework to compare the process and outcomes of experiments.

The scientific process provides a few of important points that can be applied to software performance optimization:

1. Repeatable process – use the same process for every performance enhancement you make
2. Only modify one variable at a time – Do not make multiple tweaks at the same time.
3. Record the results of each optimization.  Track what you did and how much it helped.

Performance Optimization Method (clickonchris.com)

This sounds simple right?  It is.  The tough part for software developers is to never break these rules during a round of optimizations.  To the right I’ve also included a more detailed diagram of what the scientific process looks like when applied to performance optimization.  Let’s call it the Performance Optimization Method.

But I know what I’m doing!  Why shouldn’t I make multiple tweaks at once?

Lets say you do make two changes at once.  You optimize two queries and drop the page load time from 3s to .1s.  Do you know how much relative impact the changes had?  Did each change reduce the cost by the same amount (50%/50%)?  Did one query account for most of the cost (75%/25%)?  Or did one of the changes not even have any impact (100%/0%)?  What if the two changes were somehow interdependent?  For the most part these questions are impossible to answer unless you use a repeatable process and only modify one variable at a time.  There are exceptions *(there are always exceptions.  If you have a good profiling tool that tells you exactly what two different method calls cost and you are absolutely sure they are not somehow related then you could cut a corner and make multiple changes at once.  If the results do not turn out as expected you still need to go back and make the changes one at a time).  By the way, I hope you are testing against a volume of data you expect in production.

Don’t forget to record the result of each optimization.  This way you can throw your results into a table, and with a little explanation about the process and results you turn it into a report and send it to management so they can see how you’re spending their budget (and how good you are at science).  Having these sorts of metrics reports also makes it easy for stakeholders to justify the time spent on performance optimization activities.

The law of diminishing returns applies to performance enhancements.  At some point you will have picked all of the low-hanging fruit and enhancements start to get progressively more expensive.  Stakeholders need insight into how this is progressing on your project so they can make decisions on how much more to spend on performance.  Metrics reports should provide sufficient detail for stakeholders to make those decisions.

Ultimately you will end up with a faster application and a clear story of how you got there.  Isn’t science fun?

Environment Information:

JDK 1.5+
Eclipse 3.4.0
maven 2.0.10
m2eclipse 0.9.7 (maven plugin for eclipse)
Jetty 6.1.10
Spring
JPA,Hibernate

Maven Jetty Configuration:

In your maven project’s pomx.xml, in your <build> section, add the jetty plugin.  An example can be found at the Jetty website here:

http://docs.codehaus.org/display/JETTY/Maven+Jetty+Plugin#MavenJettyPlugin-WebappConfiguration

It is very important to keep <scanIntervalSeconds> set to zero.  This setting tells Jetty how often (in seconds) to scan the webapp for changes and if changes are found, it re-cycles the web container.  You don’t want to do this and setting it to zero will disable it.

Configuring Jetty to start within Eclipse:

Next, create an easy way to launch your jetty server.  I’m using Jetty through Maven and Eclipse.  Here is how I setup an Eclipse External Tool to launch my Jetty server:

To make sure it is listening for a debugger, make sure that a MAVEN_OPTS environment variable is set with the following options:

-Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,address=4000,server=y,suspend=n

The address parameter is what port the jetty server will be listening on for the remote debugger.

Click ‘Run’ and your jetty server should start running in Eclipse’s console window.  It should look something like this (depending on your log4j config):

Listening for transport dt_socket at address: 4000

[INFO] Scanning for projects...
[INFO] Searching repository for plugin with prefix: 'jetty'.
[INFO] ------------------------------------------------------------------------
[INFO] Building myProject
[INFO]    task-segment: [jetty:run]
[INFO] ------------------------------------------------------------------------
[INFO] Preparing jetty:run
2010-05-27 15:39:25.733::INFO:  Started SelectChannelConnector@0.0.0.0:8080
[INFO] Started Jetty Server

Attaching the Debugger:

Next, setup a debugger.  In Eclipse open ‘Debug Configurations’, and create a new ‘Remote Java Debugger’.   Select your eclipse project, set the host to localhost, and set the port to 4000, or whatever you defined earlier.

Press the Debug Button and the remote debugger should attach to your jetty server.

Finally, make sure Build Automatically is enabled in eclipse (Project->Build Automatically).

At this point your environment is enabled for debugging code on your Jetty server through Eclipse.  Breakpoints, watch variables, you name it.

Hot deploy is also enabled.  If you modify some java code, the automatic builder should compile the .java file to a .class file.  The remote debugger will see it and use it, all without restarting your jetty server or its web container.  This will not work for things like adding static variables, new domain classes, or new injectable service methods that require the application to acknowledge them on startup.