introduction to mobile performance testing to mobile performance testing shlomi zalma, ... hp...

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Introduction to Mobile Performance Testing Shlomi Zalma, Wilson Mar DT3338 at Noon, June 13, 2013

HOL330 in Sands 305

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 3

Mobile apps are now:

Why performance test mobile apps?

• how customers locate stores and information

• how brands get exposure many times a day

• the “face” of the organization

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 4

Slow app = Slow product?

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 5

Baseline vs. impact of network latency, etc.

[P01] from HP Network Virtualization by Shunra

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 6

Possible points of risk in landscape

Storage DB

Servers App

Servers Web

Servers

Load Balancers

3rd Party

Cloud Services

CDNs & CoLo.

Carriers

Internet Browsers

Platforms & Versions

Firewalls VPNs

Device Features

2G,3G,4G Cellular

Mobile Components

Screen Sizes

Web Services

LANs

ISPs

DB Indexes & Logs

Resource Usage

Latency & Loss

Pooling & Threading

Server Config.

SysAdmin: DBA: Back-end: Dev: Network:

Data Caching

mDev: Encrypt, Compress.

Load Bal. Algorithm

Network: Network:

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 7

HP Live Network

Collaborate 2

HP LoadRunner components and workflow

Virtualize Network 5

End User Application Server(s) Internet/WAN Database

Analyze, Compare, Report 8

Client(s)

Generate Load 6

Author Scripts 1

Measure Impact 7

Test Scenario

Plan and Execute Tests 3

Virtualize Services 4

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 8

Scan for more information

LoadRunner Network Virtualization

Service

Virtualization Mobile

Testing

HP Codescan app

http://www.hp.com/go/...

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 9

Apple App

Store

Play. google

.com

iPhone, iPad Android “feature phones”

Windows PC

Mobile technology terms

Safari browser

URL

m.ft.com mobile

site

ft.com/ws web

services

ft.com HTML5

responsive design

Native app

internet explorer

ft.com HTML4

URL

minimal browser

URL Hybridapp

Chrome browser

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Mobile LoadRunner Scripting

https://connections.houston.hp.com/docs/DOC-15086

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 11

Create a New Script, Mobile Category

HP LoadRunner VuGen 11.5 SP1+

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 12

Multiple Approaches Available

Browser or hybrid app:

Ajax? > TruClient Ajax Mobile scripting protocol using User Agent spoofing in Firefox

Simple > Mobile HTTP/HTML scripting protocol

Native app: > Mobile HTTP/HTML scripting protocol

Android:

No device > Record Emulator using Android Virtual Device (AVD)

Have rooted device > Mobile Recorder generate lrpcap to Analyze

iOS & others:

> LoadRunner Proxy on same subnet/Wi-Fi hub (the newest technique)

> Record & Analyze WinPcap capture on the mobile app server

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 13

Recording Method TruClient Ajax

TruClient Ajax HP LoadRunner VuGen 11.51+

App web server

User Agent in Firefox >

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 14

TruClient recording

HP LoadRunner VuGen 11.51+

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 15

Devices with User Agent strings

Count: 27

HP LoadRunner VuGen11.52

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 16

Sample User Agent for Samsung Galaxy S3

User Agent string example

Mozilla/5.0 (Linux; U; Android 4.0; xx-xx; GT-I9300 Build/IMM76D) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30

HP LoadRunner VuGen11.52 More at http://www.useragentstring.com/

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 17

Android Versions

4.1 (16)

4.2 (17)

2.3 (9,10)

4.0 (15)

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 18

Mobile Device Manager Tool in VuGen

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 19

VuGen Scripting Process

from HP Virtual User Generator User Guide 11.52 page 73

1. Configure recording settings

2. Consider run-time settings

3. Object identification using script levels

4. Specify Transaction Names

5. Specify Completion End Events Logic

6. Insert flow control

7. Run-time logic

8. Vary data based on parameters

9. Using C script functions in a library

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 20

Random Think-time in TruClient

Arguments:

http://www.perftesting.co.uk/ajax-truclient-tips-and-tricks-part-ii/2012/05/02/

Math.floor( 11 * Math.random()+5 );

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 21

TruClient Toolbox drag and drop

window.alert( “randWait=” + randWait );

Arguments:

http://www.perftesting.co.uk/ajax-truclient-tips-and-tricks-part-ii/2012/05/02/

Math.floor( 11 * Math.random()+5 ); var randWait =

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

HTTP/HTML Mobile Recording

https://connections.houston.hp.com/docs/DOC-15086

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 23

Mobile HTTP vs. HTML

HP LoadRunner VuGen

mode=“HTML” mode=“HTTP”

Parse HTML to automatically identify additional resources to download

For use with web services calls. Does not parse for resource requests.

• CSS • JavaScript

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 24

Record Mobile HTTP/HTML

HP LoadRunner VuGen11.52

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 25

HTTP/HTML

TruClient

HTTP/HTML Recording Methods Wizard

Proxy Recording

Record Emulator

Analyze Traffic

Record & Analyze Traffic

HP LoadRunner VuGen11.52

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Proxy Recording

https://connections.houston.hp.com/docs/DOC-15086

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 27

Proxy Recording

Android

LoadRunner Proxy

HTTP/HTML

Proxy Recording

HP LoadRunner VuGen11.5x

App web server

Port 80/443

Proxy Port 8080

APK

iOS IPA

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 28

Same subnet via Wi-Fi router

HTTP/HTML

Ethernet cables

Wi-Fi router on private LAN

Proxy Recording

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 29

Proxy Recording

HP LoadRunner VuGen11.52

HTTP/HTML

Proxy Recording

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 30

Proxy Recording Setup on iOS

1.

Source: http://www.amsys.co.uk/2012/blog/how-to-setup-proxy-servers-in-ios/

4. 3. Scroll down 2.

Eleanor

Password:

ncjm-q3ey-t1dd

Eleanor 192.168.1.2

255.255.255.0

192.168.1.1

192.168.1.3

8080

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 31

Proxy Recording Setup on Android Galaxy S3 (4.1.2)

1. 4. 3. 2. Long press

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Record Emulator

https://connections.houston.hp.com/docs/DOC-15086

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 33

Record Emulator Android Recording Method

Android

device emulator

HTTP/HTML

Record Emulator

APK

HP LoadRunner VuGen11.5x

App web server

Async.

APK

@device ADT file

Android… \sdk\tools

File manager

Backup

Play. Google

.com

Install

Copy

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 34

Record Emulator Requirements

1. USB cable to device

2. USB driver for device

3. Android app APK file for Android app under test

4. Android file download manager app to extract APK file from device

5. JDK (Java Development Kit)

6. ADT (Android Development Toolkit) plug-in for Eclipse

7. AVD (Android Virtual Device) within Eclipse for device under test

8. JAVA_HOME and PATH Windows environment variables

9. HP LoadRunner 11.50 and SP2 patch to upgrade to 11.52

HTTP/HTML

Record Emulator

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 35

Record & Analyze Traffic

Android

network sniffer

iOS

HTTP/HTML

Record & Analyze Traffic

Ad hoc network

.cap, .pcap

HP LoadRunner VuGen11.5x

App web server

APK

IPA

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Analyze Traffic

https://connections.houston.hp.com/docs/DOC-15086

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 37

Analyze Traffic

Mobile Recorder

Android

.lrpcap

HTTP/HTML

Analyze Traffic

Android 4.1+ (rooted)

App web server

HP LoadRunner VuGen11.52

APK

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 38

Android Mobile Recorder 1. Start Recording 3. Stop and send email 2.

Perform user steps.

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 39

Android Mobile Recorder Settings

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 40

Recording Methods Recap

Mobile Recorder

Android

.lrpcap

LoadRunner Proxy

device emulator

network sniffer

iOS

Proxy Port 8080

HTTP/HTML

Proxy Recording

Record Emulator

Analyze Traffic

Record & Analyze Traffic

Port 80/443

User Agent in Firefox >

Android 4.1+ (rooted)

Ad hoc network

APK

APK

.cap, .pcap

TruClient Ajax HP LoadRunner VuGen11.52

App web server @device ADT file IPA

Play. Google

.com

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Mobile scenarios

https://connections.houston.hp.com/docs/DOC-15086

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 42

How can we help?

Run test cases

Provision & deploy environment

Execute automated build

Application Security Assessment

Test results

Security Code Scan Fortify Source Code Analysis

UFT / BPT LoadRunner

WebInspect

Continuous Delivery Automation (CDA)

(Jenkins)

HP integrated testing suite

ALM Analysis

Executive Scorecard

Performance Center

Service Virtualization

Network Virtualization

Development Agile Manager

Business Service Management

Monitoring & Diagnostics

Analytics

App. Lifecycle Intelligence

ALM Lab Management

Tear down environment Continuous Delivery Automation (CDA)

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 43

Shopping process

Interconnected component services

Pay Buy Ship Track

Product

Search

Customer profiles Payment Shipping

Composite Managed cloud

Public cloud

Traditional

Track package

Private cloud

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 44

Concurrency and rendezvous

[P10] Illustrative data

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 45

Key take-aways

• For mobile browser apps (jQuery Mobile, Sencha, PhoneGap, etc.), little scripting is needed using User Agent spoofing via Firefox TruClient Mobile

• JavaScript executes against a DOM for each virtual user

• Traffic captured between native apps and server are converted to VuGen scripts

• Android native app traffic on rooted devices can be recorded on device using LoadRunner Mobile Recorder

• iPhone and iPad (iOS) -- all non-rooted apps – can be recorded using LoadRunner Proxy

• Monitor and Diagnose using HP Business Service Management

• Consider HP Network Virtualization and Services Virtualization

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Thank you