Download - BlazeMeter- Effective Performance Reporting
EFFECTIVE PERFORMANCE REPORTING USING
APACHE JMETER
JULY 31, 2012
THE LOAD TESTING CLOUD
A DEV-TEST CLOUD SERVICE 100%
COMPATIBLE WITH THE OPEN-SOURCE
APACHE JMETER
AGENDA
Performance Attributes
Creating Load Test Reports
Understanding Performance KPIs
JMeter Reporting Elements
BlazeMeter Reporting Plugin
Generating Advanced JMeter Reports
PERFORMANCE ATTRIBUTES
• Speed / Responsiveness
• How fast does the page load?
• How quickly can the system process a transaction?
• Scalability
• Can the application handle the expected end user load?
• Does the application throughput degrade as the user load increases?
PERFORMANCE ATTRIBUTES…
• Efficiency and Capacity Planning
• Are you using the right resources
• Can your infrastructure carry the
load?
• Reliability/Availability/
Recoverability
• What is the mean time between
failure (MTBF)?
• Does the application recover after
a crash? Does it lose user data
after crash?
UNDERSTANDING PERFORMANCE KPIS
Internet
Application Metrics • Response Time • Throughput • Error Rate
Browser Rendering Metrics* • Total Rendering Time • Heavy Images/CSS/JS • DNS Lookup
System Metrics • CPU • Memory • Disk / IO • Network
Platform Metrics • DB • App-server • Application
User Load
Re
sp
on
se
Tim
e
User Load
Re
qu
es
ts / s
ec
End User
Server
UNDERSTANDING PERFORMANCE KPIS…
Throughput
Error
•Measured from the end-user perspective
•Time taken to completely respond to request
•TTLB TTFB
Response Time Web
Server
Inter
net App
Server
DB
Server
DB
Server
Response Time
Total Response Time =
Network latency + Application latency + Browser Rendering Time
•Transactions are specific to applications
•In its simplest form, it is requests / sec
Throughput =
[TRANSACTIONS] / Second
•Defined in terms of the success of the request
•Error at HTTP level (404, 501)
•Application level error
CREATING LOAD TEST REPORTS
Capture Application Metrics • Response Time • Throughput • Errors
Capture Server Metrics • CPU / Memory / Disk / IO • Network • Application • Platform
Tables • Response Time
(avg/min/max/%/stddev) • Throughput (average) • Errors (success % / types)
Graph / Charts • Scatter / Line • Overlay
Correlate Application Metrics • User Load - Response Time • User Load - Throughput • User Load - Errors
Correlate System Metrics • User Load - Server Metrics • User Load - Network • User Load - Platform
Summarize • Overall Performance • Important Trends • Threshold Violations
Trends / Thresholds • Response Time Trends • Throughput Trends • Threshold Violation • Utilization (Server Metrics) Trends
1. Capture
2. Correlate
3. Plot / Tabulate
4. Trends / Thresholds
5. Customize / Summarize
6 . Compare
SAMPLE REPORT ELEMENTS (SNAPSHOTS)
Photo Credits:
• http://msdn.microsoft.com/en-us/library/bb924371.aspx
• Sanitized past projects
JMETER REPORTING ELEMENTS (LISTENERS)
• JMeter Listeners
• JMeter elements that display performance test metrics / output
• Various types of Listeners (Raw / Aggregated / Graphical)
• Doesn’t have inherent capability to measure system metrics*
• Useful for basic analysis
JMeter Report using xslt stylesheet
• Style-sheet under ‘extras’ folder
• .jtl output must be in xml format
– jmeter.save.saveservice.output.for
mat=xml
• Integrate using ant
GENERATING ADVANCED JMETER REPORTS
Other Reporting Options
• JMeter CSV results + Excel
• Process results programmatically
(perl / python etc.)
• BlazeMeter Reporting Plug-in
Photo Credits:
• http://www.programmerplanet.org/pages/projects/jmeter-ant-
task.php
WHAT HAPPENED?
TO LABEL A AND KPI B AT TIME C
BLAZEMETER REPORTING PLUGIN
BENEFITS
• Store a report per test run,
including
• Script that was used to run the
test
• Logs & JTL file
• Compare results of two test runs
• See an improvement trend
• Compare current with previous in
real time
• Share with co-workers
KPIS AVAILABLE IN A JMETER TEST RESPONSE TIME - THE TIME IT TAKES A REQUEST TO FULLY LOAD
• Indicates the performance level of the entire system under test (web server +
DB).
• Represents the average response time during a specific minute of the test.
BLAZEMETER REPORTING PLUGIN COMPARE TWO REPORTS
BlazeMeter - Changing the Economics of Load Testing via the Cloud’
‘BlazeMeter - Code probing, not Angry Birds will define cloud’s success’
‘BlazeMeter - Startup Offers JMeter Cloud Load Testing at Scale’
THANK YOU!
HTTP://BLAZEMETER.COM/