What I wanted was something to “spider” the site and follow links for a certain period of time simulating lots of users.  Hitting the same page over and over again just doesn’t cut when you are caching pages and want to test the load required to bring them into cache and deliver them.

I found JMeter. Its is a great application from the Apache Jakarta project but confusing to say the least when it comes to configuration.

After digging around I found a nice set up to do exactly this:

* Thread Group (Continue on error)
* HTTP Request (Site URL, path [enter homepage or seed url] )
* While Controller (Condition = “LAST”)
* * HTTP Request (Site URL, path [.*])
* * HTML Link Parser
* * Gaussian Random Timer

This gives me a good spread of URL’s at good random intervals.

Huge thanks to this jmeter newsgroup post by Jason.