introduction to mobile performance testing - myloadtest.com · © copyright 2013 hewlett-packard...
TRANSCRIPT
![Page 1: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/1.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Introduction to mobile performance testing Igor Markov, Shlomi Zalma, Shane Evans, Wilson Mar June 11 & 13, 2013
![Page 2: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/2.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 3
HP LoadRunner components
Virtualize the Network 4
Stay Connected
Analysis, Diagnostics and Reporting
Database Internet/WAN Application Server(s) Client(s) End User
Test Scenario
Measure Impact
6
Generate Load 3 Author Scripts 1
Plan and Execute Tests 2 HP Live Network 7
5
![Page 3: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/3.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 4
Incremental integrated development workflow
Deploy app
Run test cases
Provision devices
Execute build
Application Security Assessment
Test results
Check-in & Code Scan Fortify Source Code Analysis
UFT / LoadRunner
WebInspect
ALM Lab Management
Cloud Delivery Automation
Jenkins
HP integrated suite
LR Analysis
Executive Scorecards
Analytics Portal
![Page 4: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/4.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 5
Workflow for performance engineering
1. Plan 2. Provision 3. Verify 4. Script 5. Scenarios
Workload Templates Data Scenarios Attributes Cost vs.
Benefit
Domains Servers Ports IP Addresses SSL certs. Licenses
Batch Jobs Network
Variability Report
Samples
Transactions Resource
Usage Variations Retry and
Recovery
Report Templates
Reruns Baselines Comparisons Archival
![Page 5: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/5.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 6
Create a new script, mobile protocols
HP LoadRunner VuGen
![Page 6: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/6.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Author mobile TruClient scripts
https://connections.houston.hp.com/docs/DOC-15086
![Page 7: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/7.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 8
Recording methods
TruClient Ajax
HTTP/HTML
HP LoadRunner VuGen
App web server
User Agent >
![Page 8: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/8.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 9
Create a new script, mobile TruClient protocol
HP LoadRunner VuGen
![Page 9: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/9.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 10
TruClient recording
HP LoadRunner VuGen 11.52
![Page 10: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/10.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 11
Devices with User Agent strings
Count: 27
HP LoadRunner VuGen11.52
![Page 11: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/11.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 12
Sample User Agent for Samsung Galaxy S3
User Agent string example
Mozilla/5.0 (Linux; U; Android 4.0; xx-xx; GT-I9300 Build/IMM76D) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30
720 x 1280
HP LoadRunner VuGen11.52
![Page 12: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/12.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 13
Invoke Firefox in VuGen for TruClient
HP LoadRunner VuGen11.52
![Page 13: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/13.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
HTTP/HTML mobile recording
https://connections.houston.hp.com/docs/DOC-15086
![Page 14: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/14.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 15
Create a new script, HTML recording protocol
HP LoadRunner VuGen11.52
![Page 15: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/15.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 16
HP LoadRunner VuGen11.52
![Page 16: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/16.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 17
HTTP/HTML
TruClient
HTTP/HTML recording methods wizard
Proxy Recording
Record Emulator
Analyze Traffic
Record & Analyze Traffic
TruClient Ajax HP LoadRunner VuGen11.52
![Page 17: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/17.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 18
Record emulator recording method
Android
Device Emulator
HTTP/HTML
Record Emulator
APK
APK
TruClient Ajax HP LoadRunner VuGen11.5x
App web server
![Page 18: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/18.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 19
Record emulator requirements
Java Development Kit http://www.oracle.com/technetwork/java/javase/downloads/index.html
Android SDK with ADT http://developer.android.com/sdk/index.html
HTTP/HTML
Record Emulator
![Page 19: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/19.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 20
Record emulator
HP LoadRunner VuGen11.52
HTTP/HTML
Record Emulator
![Page 20: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/20.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 21
Record emulator options
HP LoadRunner VuGen11.52
HTTP/HTML
Record Emulator
![Page 21: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/21.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 22
Proxy recording
Android
LoadRunner Proxy
iOS
Port 8080
HTTP/HTML
Proxy Recording
Port 80/443
APK
HP LoadRunner VuGen11.5x
App web server
![Page 22: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/22.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 23
Proxy recording
HP LoadRunner VuGen11.52
HTTP/HTML
Proxy Recording
![Page 23: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/23.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 24
Record and analyze traffic
Android
Wireshark
iOS
HTTP/HTML
Record & Analyze Traffic
Ad hoc network
APK .pcap
HP LoadRunner VuGen11.5x
App web server
![Page 24: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/24.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 25
Record and analyze traffic
HP LoadRunner VuGen11.52
HTTP/HTML
Record & Analyze Traffic
![Page 25: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/25.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 26
Record and analyze traffic
HP LoadRunner VuGen11.52
HTTP/HTML
Record & Analyze Traffic
![Page 26: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/26.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 27
Analyze traffic
Mobile Recorder
Android
.lrpcap
HTTP/HTML
Analyze Traffic
Android 4.1+ (rooted)
App web server
APK
HP LoadRunner VuGen11.52
![Page 27: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/27.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 28
Analyze traffic
HP LoadRunner VuGen11.52
HTTP/HTML
Analyze Traffic
![Page 28: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/28.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 29
Analyze traffic—options
HP LoadRunner VuGen11.52
HTTP/HTML
Analyze Traffic
![Page 29: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/29.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Scenario runs
https://connections.houston.hp.com/docs/DOC-15086
![Page 30: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/30.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 32
Concurrency and rendezvous
[P10]
![Page 31: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/31.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 33
Wi-Fi vs. 3G response time
[P01] HP Network Virtualization by Shunra
![Page 32: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/32.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 34
3G vs. 4G latency spikes
HP Network Virtualization by Shunra
![Page 33: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/33.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 35
http://www.youtube.com/watch?v=7CGCief0EaU&hd=1&t=19m35s
Bandwidth Utilization In (Down)
Bandwidth Utilization Out (Up)
Response Time
Login with more VUsers
![Page 34: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/34.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 36
Ramp up and down
Resources
• Processes, Threads
• Memory (leaks?)
• CPU (saturation?)
• Disk I/O (thrashing?)
• Network I/O
• Transaction Throughput
• Etc.
[P04]
Response time
Resources per user
Clean-up Resources over time
Back-end Resources
HP LoadRunner Analysis 11.52
![Page 35: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/35.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 37
Capacity in 3 dimensions
[P04]
![Page 36: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/36.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 38
Scalability
[P01] http://wwwperftest.com
![Page 37: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/37.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 39
Isolate end-user performance by carrier
HP Business Service Management
![Page 38: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/38.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 40
Isolate end-user performance by device
HP Business Service Management
![Page 39: Introduction to mobile performance testing - myloadtest.com · © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without](https://reader031.vdocuments.mx/reader031/viewer/2022041312/5e1581504fbf050efe679b28/html5/thumbnails/39.jpg)
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Thank you