introduction to mobile performance testing - myloadtest.com · © copyright 2013 hewlett-packard...

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Introduction to mobile performance testing Igor Markov, Shlomi Zalma, Shane Evans, Wilson Mar June 11 & 13, 2013

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 3

HP LoadRunner components

Virtualize the Network 4

Stay Connected

Analysis, Diagnostics and Reporting

Database Internet/WAN Application Server(s) Client(s) End User

Test Scenario

Measure Impact

6

Generate Load 3 Author Scripts 1

Plan and Execute Tests 2 HP Live Network 7

5

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 4

Incremental integrated development workflow

Deploy app

Run test cases

Provision devices

Execute build

Application Security Assessment

Test results

Check-in & Code Scan Fortify Source Code Analysis

UFT / LoadRunner

WebInspect

ALM Lab Management

Cloud Delivery Automation

Jenkins

HP integrated suite

LR Analysis

Executive Scorecards

Analytics Portal

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 5

Workflow for performance engineering

1. Plan 2. Provision 3. Verify 4. Script 5. Scenarios

Workload Templates Data Scenarios Attributes Cost vs.

Benefit

Domains Servers Ports IP Addresses SSL certs. Licenses

Batch Jobs Network

Variability Report

Samples

Transactions Resource

Usage Variations Retry and

Recovery

Report Templates

Reruns Baselines Comparisons Archival

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 6

Create a new script, mobile protocols

HP LoadRunner VuGen

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Author mobile TruClient scripts

https://connections.houston.hp.com/docs/DOC-15086

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 8

Recording methods

TruClient Ajax

HTTP/HTML

HP LoadRunner VuGen

App web server

User Agent >

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 9

Create a new script, mobile TruClient protocol

HP LoadRunner VuGen

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 10

TruClient recording

HP LoadRunner VuGen 11.52

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 11

Devices with User Agent strings

Count: 27

HP LoadRunner VuGen11.52

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 12

Sample User Agent for Samsung Galaxy S3

User Agent string example

Mozilla/5.0 (Linux; U; Android 4.0; xx-xx; GT-I9300 Build/IMM76D) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30

720 x 1280

HP LoadRunner VuGen11.52

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 13

Invoke Firefox in VuGen for TruClient

HP LoadRunner VuGen11.52

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

HTTP/HTML mobile recording

https://connections.houston.hp.com/docs/DOC-15086

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 15

Create a new script, HTML recording protocol

HP LoadRunner VuGen11.52

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 16

HP LoadRunner VuGen11.52

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 17

HTTP/HTML

TruClient

HTTP/HTML recording methods wizard

Proxy Recording

Record Emulator

Analyze Traffic

Record & Analyze Traffic

TruClient Ajax HP LoadRunner VuGen11.52

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 18

Record emulator recording method

Android

Device Emulator

HTTP/HTML

Record Emulator

APK

APK

TruClient Ajax HP LoadRunner VuGen11.5x

App web server

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 19

Record emulator requirements

Java Development Kit http://www.oracle.com/technetwork/java/javase/downloads/index.html

Android SDK with ADT http://developer.android.com/sdk/index.html

HTTP/HTML

Record Emulator

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 20

Record emulator

HP LoadRunner VuGen11.52

HTTP/HTML

Record Emulator

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 21

Record emulator options

HP LoadRunner VuGen11.52

HTTP/HTML

Record Emulator

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 22

Proxy recording

Android

LoadRunner Proxy

iOS

Port 8080

HTTP/HTML

Proxy Recording

Port 80/443

APK

HP LoadRunner VuGen11.5x

App web server

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 23

Proxy recording

HP LoadRunner VuGen11.52

HTTP/HTML

Proxy Recording

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 24

Record and analyze traffic

Android

Wireshark

iOS

HTTP/HTML

Record & Analyze Traffic

Ad hoc network

APK .pcap

HP LoadRunner VuGen11.5x

App web server

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 25

Record and analyze traffic

HP LoadRunner VuGen11.52

HTTP/HTML

Record & Analyze Traffic

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 26

Record and analyze traffic

HP LoadRunner VuGen11.52

HTTP/HTML

Record & Analyze Traffic

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 27

Analyze traffic

Mobile Recorder

Android

.lrpcap

HTTP/HTML

Analyze Traffic

Android 4.1+ (rooted)

App web server

APK

HP LoadRunner VuGen11.52

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 28

Analyze traffic

HP LoadRunner VuGen11.52

HTTP/HTML

Analyze Traffic

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 29

Analyze traffic—options

HP LoadRunner VuGen11.52

HTTP/HTML

Analyze Traffic

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Scenario runs

https://connections.houston.hp.com/docs/DOC-15086

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 32

Concurrency and rendezvous

[P10]

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 33

Wi-Fi vs. 3G response time

[P01] HP Network Virtualization by Shunra

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 34

3G vs. 4G latency spikes

HP Network Virtualization by Shunra

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 35

http://www.youtube.com/watch?v=7CGCief0EaU&hd=1&t=19m35s

Bandwidth Utilization In (Down)

Bandwidth Utilization Out (Up)

Response Time

Login with more VUsers

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 36

Ramp up and down

Resources

• Processes, Threads

• Memory (leaks?)

• CPU (saturation?)

• Disk I/O (thrashing?)

• Network I/O

• Transaction Throughput

• Etc.

[P04]

Response time

Resources per user

Clean-up Resources over time

Back-end Resources

HP LoadRunner Analysis 11.52

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 37

Capacity in 3 dimensions

[P04]

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 38

Scalability

[P01] http://wwwperftest.com

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 39

Isolate end-user performance by carrier

HP Business Service Management

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 40

Isolate end-user performance by device

HP Business Service Management

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Thank you