3 years ago I spoke about my experience with performance testing at Songkick, and to be honest I loved this specialism and discipline. It combines two of may favourite things, how users behave and how the system behaves in different conditions. At songkick it was fascinating to flick the switch for a an artist’s ticket sale and seeing how different fans would use the system differently, not just if they were using mobiles or certain browsers but what did they do? Do they keep refreshing, going back and forth through the journey? Are they happy to just buy 1 ticket or always going for multiples?
I just love combining the data of what actual users do and how they behave to model different types of performance tests.
And covid-19 has actually meant for a few reasons that I get to think about this again. The reasons are:
- I no longer commute so have a bit more time (my commute was very small though).
- Our users used our systems more for a period of time due to several reasons
- I have changed teams for a short while -giving me a tiny bit of breathing and much needed thinking space.
While we do not do much performance testing in my team, at least there isn’t a proper strategy (it has been my retro item for a few weeks to create a strategy), we have got external teams with the ability to, and it is their role to test all the systems together.
What excites me is that performance testing let’s you get under the skin of the application. You need monitoring and analytics in place to watch what happens and then you get to look at all the layers of your architecture. Being from a non technical background, I find this fascinating and I learn a lot each time. As I don’t have actual tales yet I thought I would describe the 4 performance testing types I have experienced and talk about in my talk of 2017. Enjoy.
Performance testing types
Performance testing in my mind is not a type of test but a collective description for many types of tests that provide information about how your system performs under different conditions.
The ones I’d like to mention are these below. (Thanks to Mark Tomlinson for the drawings 3 years ago, as part of my presentation mentoring).
- Load Testing
- Stress Testing
- Soak Testing/Endurance Testing
- Spike Testing
Load Testing
Load Testing is type of performance test to check how a system deals with a sustained load. It can inform your system’s threshold before it starts to degrade.
When we say we create load we mean the number of concurrent users going through the system and its transactions behaviour.
Load testing is used to inform and determine:
- Heavy load conditions – informs other tests you can run to determine bottlenecks
- Throughput rates
- Response times – especially do they degrade?
- Resource utilisation
Stress Testing
Stress Testing is often performed to check how a system deals with an ever increasing load (number of concurrent users). This can give information on the stability of software when hardware resources are not sufficient like CPU, memory, disk space etc.
This performance test type is used to determine or validate an application’s behavior when it is pushed beyond normal or peak load conditions. This sounds similar to load testing but the idea here is to actually push the system beyond its limit so it is “stressing”. You want to find out when the system fails, how it fails and keep an eye on how the system recovers (if it can/is meant to).
By running a stress test you can discover things about:
- The limits of the system
- How does the system fail?
- Is the failover strategy adequate?
It may also uncover:
- Race conditions
- Memory leaks
Soak Test/Endurance Test
Soak testing or endurance testing involves testing a system with a normal amount of load over a long period of time. In some companies this isn’t possible, bt we have found memory leaks using this technique, even after just 2 hours. However some companies that can run these tests for 2 days. But remember generally load is not high but the test runs for a long time.
This approach will help find:
- Any problems that only manifest themselves after long periods of use.
- Memory leaks
- How long will it take for my system to break?
- What are the request error rates over time?
And now the most fun!
Spike Test
A spike test is carried out to validate the performance characteristics when the system under test is subjected to high load volumes repeatedly. A common model is to test the load increasing and decreasing several times over a period of time. This can be really useful if you want to test the scaling of your system and also how it scales back down.
Spike tests can uncover:
- Issues with concurrent transactions at the same time – race conditions
- Does the application scale quickly?
Why do I find this the most fun? The sort of projects and products I have worked on, this type of test found the most issues. Hitting a system hard suddenly, can quickly show you bottlenecks and other strange issues that you may not find otherwise, more often than not caused by race conditions.
These are the types of performance tests I have experienced and used in the past. What I will do next, I am not sure about yet but I will try to share my journey. Are you a seasoned performance tester, or new to it? What strategy do you have at your work? Does anyone do performance driven development?
P.S. Thanks Mark Tomlinson from @perfbytes http://www.perfbytes.com
punkmiktests 