concepts on software performance testing
DESCRIPTION
Concepts on Software Performance Testing. Presented by Chris SAUNDERS - 2014 @chrisrsaunders. Presentation. Presentation. Presentation's Purpose [1%] What’s Software Performance ? [ 4 %] General Measures [4%] The Business Case [1%] Performance Goals - 5 Examples Types [45%] - PowerPoint PPT PresentationTRANSCRIPT
testrun.co.nz@testrunnzConcepts on
Software Performance
Testing
Chris SAUNDERS 2014
testrun.co.nz@testrunnz
testrun.co.nz@testrunnz
Presentation's Purpose
testrun.co.nz@testrunnz
Presentation's Purpose
• To talk about software performance testing
• To explore concepts, ideas and things to consider
• Not talking about tools
testrun.co.nz@testrunnz
What is Software Performance?
testrun.co.nz@testrunnz
What is Software Performance?Software performance is the manner in which
software functions, operates, or behaves; based on a given set of factors.
Software performance testing is to determine how a system performs in terms of responsiveness and stability with a defined set of factors.
What factors?
Each person takes two sticky notes and writes their answer down, and places it on the wall
Note this question may be asked again later
testrun.co.nz@testrunnz
General Measures
testrun.co.nz@testrunnz
General Measures
Measures Description
1. Known measures Measure where the system meets a given predefined performance criteria
For example: The CPU & Memory resources will not exceed 80% for more than 5 second duration.
testrun.co.nz@testrunnz
General Measures
Measures Description
1. Known measures Measure where the system meets a given predefined performance criteria
2. Comparable measures Measure between the performance of two systems to determine which has better performance
For example: The new calculation engine will have a 20% increase in processing time and data volume
throughput.
testrun.co.nz@testrunnz
General Measures
Measures Description
1. Known measures Measure where the system meets a given predefined performance criteria
2. Comparable measures Measure between the performance of two systems to determine which has better performance
3. Unknown measures Measure where the system has a possible performance issue in a given area
(One does not exactly no where or how it will fail)
For example: Determine the system performance on the system due to teamed network card failure.
testrun.co.nz@testrunnz
The Business Case
testrun.co.nz@testrunnz
The Business Case
“What is the business case for performance testing?”
“How does it add business value?”
Need to define “realistic performance related goals”
testrun.co.nz@testrunnz
Performance Goals5 Examples Types
testrun.co.nz@testrunnz
Performance Goals:Water pipe analogy
Consider software performance testing like Christchurch’s water pipe system
testrun.co.nz@testrunnz
Performance Goals:Response Time
Measuring the duration of time between two points
For example: The duration for a user waiting on a file to upload
A BDuration
0:00:00.29 or 290ms
testrun.co.nz@testrunnz
Performance Goals:Concurrency
Measuring the effect on the system with concurrent processes
For example: The effect of the different number of users trying to upload files at almost the same time
testrun.co.nz@testrunnz
Performance Goals:Throughput
Measuring the effect on the system with throughput of information (volume)
For example: The effect of different sized files being uploaded
20 l/s98 l/s
testrun.co.nz@testrunnz
Performance Goals:Interdependence
Measuring how the system responds to different factors
For example: Consider the relationship between the following1. Response time 2. Concurrency 3. Throughput
A BDuration
testrun.co.nz@testrunnz
Performance Goals:Interdependence
Compare these two scenarios:1. 100 users are each uploading 20MB files, start
uploading files within a 5 minutes duration.2. 10 users are each uploading 200MB files, start
uploading files within a 10 seconds duration.
Which scenario has better performance?
testrun.co.nz@testrunnz
Performance Goals:MTBF & MTTR
Mean Time Between Failures & Mean Time To Repair
Determine the performance of when the system fails and how long does it take to repair?
ConsIder the relationship of the system to other systems or internal components
testrun.co.nz@testrunnz
Performance Goals:MTBF & MTTR
Source : http://www.greenm3.com/gdcblog/2012/7/31/netflix-sets-chaos-monkey-free-for-all-to-use-next-comes-mor.html
The Chaos Monkey is a software tool that was
developed by Netflix engineers to test
the resiliency and recoverability of their Amazon Web Services
Open Source https://github.com/Netflix/SimianArmy
Netflix engineers develop automatic recovery mechanisms
testrun.co.nz@testrunnz
Performance Goals:Summary
1. Response Time2. Concurrency3. Throughput4. Interdependence5. MTBF & MTTR6. …
testrun.co.nz@testrunnz
Some Performance Testing Approaches
testrun.co.nz@testrunnz
Performance Testing Approach:Load Testing
Measure how the system behaves under a specific expected load.
Load characteristics: Define number of concurrent users, with a determined throughput for a given duration
Could be looking for bottlenecks
testrun.co.nz@testrunnz
Performance Testing Approach:Stress Testing
Measure how the system behaves with loads beyond normal operational capacity.
Measure modes of failure.
Determine the robustness of the system: How, when & where is it going to fail
testrun.co.nz@testrunnz
Performance Testing Approach:Spike Testing
Measure how the system behaves with the sudden increase in load
Determine system response; performance profile or system failure
testrun.co.nz@testrunnz
Performance Testing Approach:Endurance (Soak) Testing
Measure how the system operates under a continuous expected load
Determine performance degradation
testrun.co.nz@testrunnz
Environmental changes to determine system performance
System controlled failure
System configuration changes
testrun.co.nz@testrunnz
Performance Testing Approach:Summary
1. Load Testing2. Stress Testing3. Spike Testing4. Endurance Testing5. Environmental changes to
determine system performance6. …
testrun.co.nz@testrunnz
What’s Software Performance?
testrun.co.nz@testrunnz
What is Software Performance Testing?
Software performance testing is to determine how a system performs in terms of responsiveness and stability with a defined set of factors.
What factors?(Each person can change their two sticky notes on the wall)
Discuss software performance testing…
testrun.co.nz@testrunnzConcepts on
Software Performance
Testing
Chris SAUNDERS 2014