‘Starting small’ sounds a bit awkward in the context of load testing. Isn’t it all about many virtual users, large volumes of data and massive traffic? It is. But still, starting small is the only way to ensure successful load testing: a smoother process, more accurate results analysis, and no wasted time.
The Dangers of Starting Too Big
A while back I was advising a company, helping them resolve performance issues. I came in after they spent two months testing without any progress. Initially, it seemed their load testing project was just perfect. They had detailed plans, a dedicated lab, and a test system that was copied from one of their customers, complete with real-life data.
When I started drilling down into their tests, I realized why no progress was made: tests were so complex, it was impossible to isolate issues. Their initial test had over 10,000 users, executed 40 different scenarios from 10 different locations. If that wasn’t enough, because the system was a real one, there were many periodical processes running, of which the testing team was not aware.
Wrong approach. Although the objective is, indeed, to reach such large-scale, complex test scenario simulating real life, it can only be done gradually. With so many parameters right from the start, you are bound to fail.
Zero Load
Truth be told, your first step (prior to using your load testing tool) should be zero load, testing your website performance without any fancy scripts whatsoever.
What’s the point? You want to verify that your web application and associated servers function smoothly with a single user. If, for example, a single page loads in 20 seconds when using a single user, there’s no point of moving on to a load test before the problem is first fixed.
There are several free tools you can use, such as WebPageTest.org. Remember not to simply login and browse through a few pages, but actually test each of the pages and actions you’ll be running in your load testing.
Single User, Single Scenario
Next, run each load test scenario with a single user. Run each scenario on its own, repeatedly, for a full hour. This will help you identify issues such as memory leaks, errors, test data, and your actual test script. You want to clear as many issues out of the way, which are not related to the actual performance of the servers and network.
Primarily, this simple setup will validate your test scripts. You’ll verify that scripts are robust, that they use the right data and logic, and that they actually test what you’ve planned.
Single User, Minimal Load
Once you have verified that a scenario works smoothly with a single virtual user, increase the number of threads – for example, run it with 5 or 10 virtual users. Don’t attempt yet to increase the load. (Only one more step left).
Why is this needed? It helps you ensure that there are no deadlocks or other issues that affect your application, just because a few users are working in parallel. You want to discover these issues early on, rather than erroneously associate them with a problem related to massive load.
Increase Load or Mix Scenarios<
Now, after you’ve verified that your entire test setup is working properly, you’re ready for real load testing.
You can start by either mixing scenarios, or increasing the load. There is a debate which one should come first – increasing the load for each scenario till reaching its maximum requirements vs. starting to mix scenarios and only then increasing the load. Each of these has its advantages and disadvantages. But at this point, you’ve already grown into the big league and can decide on your own.
Good luck.
