roland van leusden mobile performance testing rtc 2014 v0.6

Mobile Performance Testing

Roland van Leusden

1

Working for the FITT project, which consist of migration from several datacenters into one, more then 400 applications will be migrated.

Study researchers found three-quarters (77%) of mobile users are concerned about app performance before they buy. (Kristina Knight)

However mobile devices typically have: slower CPUs, smaller screens, and less memory than the typical desktop PC.

Connections can come from cell phones, tablets, and various other devices with widely different capabilities. Then there is the diverse and growing range of platforms used in the mobile world, including Android, BlackBerry, iPhone, and Microsoft Windows Phone. To complicate matters further, there are many existing versions of platforms in use today.

There aren’t enough hours in the day to test on every device….

http://opensignal.com/coverage-maps/Netherlands/

Devices

• Different platforms & Hardware

Network

• Connections from variousnetworks with different quality

Apps

• Other applications usingresources.

Where is the app used ?

•At home and office•At the train station•In the train

Which devices ?

•Android Phone•Android Tablet•iPhone•iPad

Networks used ?

•Home / Office: 73% Wifi 27% Mobile•Train station: 47% Wifi 53% Mobile •In the train: 26% Wifi 74% Mobile

Architecture questions

• Required functionality ?• Quality of Mobile Network ?• User profile and expectations ?• Amount of data in bundle ?• Network speed ?• Devices used ?• Experience of Developers ?

http://wiki.developerforce.com/

11

0 5

Device use

Network use

Server use

Resource Allocation

0 5

Device use

Network use

Server use

Resource Allocation

0 5

Device use

Network use

Server use

Resource Allocation

0 5

Device use

Network use

Server use

Resource Allocation

0 5

Device use

Network use

Server use

Resource Allocation

12

Best Practice Guidelines for Developing Quality Mobile Apps

In house,Need to buy devices all the time / For older models secondhand .In controll of network and installed apps.Good for Performance testing

In the cloud,

Pay per useGood for functional testingNewest devices availableNetwork control ?Installed apps ?Measuring recources ?

AppInsight: Mobile App Performance Monitoring in the Wild

http://research.microsoft.com/en-us/projects/appinsight/

Closed Beta

Bought by Facebook

AT&T Application Resource Optimizer (ARO)

https://developer.att.com/application-resource-optimizer

$3500/ year

Modify Headersadd-on

NeoLoad

Silk Performer

Spirent Avalanche

Shunra

WANemDummy Cloud

Dummy Net

m.nshighspeed.nl

IP 1: 192.168.0.20IP 2: 192.168.0.30IP 3: 192.168.0.40

192.168.0.62

Architecture

• Users, Devices, Functionality

• Mobile Networks

Development

• Web, Hybrid or Native.

• Instrumentation, Emulators

Functional & Performance

testing

• Real devices

• Mobile Network emulation