mpte - software testing...
TRANSCRIPT
M.COM
9985282049
9962979696
MPTE
AUTHORS
HAREESH MUNAGANURI
PRATIBHA ROHILLA
MALLIKARJUN GANIGI
ORGANIZATION –
IBM INDIA PVT Ltd.
MOBILE PERFORMANCE
TESTING EXPEDITE
SYNOPSYS
The phenomenal growth of mobile devices cannot be left
unnoticed by any Organization, intending to grow. Increasingly
Organizations, wanting to increase the reach of their
Applications.
However, this also poses challenges on the cost optimization
due to the ability to support various types of
mobile devices, and operating systems like Android, iOS, and
Windows.
This paper will explain below key points which will be useful for
industry to obtain the right tool to conduct performance testing
and monitoring for different mobile OS and devices also the
typical challenges
o KPIs to be considered for mobile performance testing.
o Different tools for android and IOS mobile testing.
o Setting up cost effective lab for mobile performance
testing.
o Monitoring tools to get the statistics to identify the
tailbacks.
Above best practices can overcome the challenges in Mobile
Performance Testing.
2
INTRODUCTION
Testing of mobile application must be performed according to certain industry standards to ensure its
stability and reliability, so that the applications can be accepted globally by the people using different
mobile operating systems, which helps user friendly and popular.
CHALLENGES IN MOBILE TESTING
Some of the typical challenges faced in any Mobile application are as shown in the diagram
KPI’S FOR MOBILE PERFORMANCE TESTING
Below are the important KPIs to be considered for mobile performance testing.
▪ App crash: The typical crash rate is around 1-2%, but this varies widely depending on the type of
app, its usage, maturity, etc.
▪ API latency: It is best practice to optimize the APIs to meet the turnaround time of 1 second. As
data generally shows that anything over 3-4 seconds total response time and the majority of user
(60% or greater) will abandon the transaction and may even delete your app altogether.
3
▪ App load per period: It is critical metric because you want to make sure that as the load increases,
your application performance doesn’t degrade. Load can be very spiky in nature for some apps,
so you need to know that your app can handle sudden changes in load without slowing down.
▪ Application time running without connection to permanent power (battery) (App + device + OS)
▪ Frequency of connections to server (every 5, 10, 30 minutes)
▪ Number of bytes needed for App data packet to server.
▪ Storage capacity for app data.
KNOW WHEN TO USE EMULATORS OR REAL DEVICE TESTING
While developers typically leverage emulators for testing, traditional QA knows there is no substitute for
real devices. This means both have their place as part of the process. For early testing and distributed
teams, nothing can replace the convenience of emulators. However, they typically don’t replicate many
core capabilities that should be tested. Recent advances with test cloud libraries of online devices replaced
many of the emulator advantages and eliminated the need to purchase and manage your own library of
devices. The bottom line is that both emulators and real device testing should play a key role.
Real Device Testing: Testing on real device allows you to run your mobile applications and checks its
functionality. Real device testing assures you that your application will work smoothly in customer
handsets.
Emulators: Emulator is a software program that allows your mobile to imitate the features of another
computer or mobile software you want them to imitate by installing them to your computer or Mobile.
PERFORMANCE TESTING TOOLS AND APPROACHES
Device Any Where
A revolutionary solution providing instant remote access to more than 2000 real devices in live global
networks, DeviceAnywhere empowers mobile application and content developers to create better, higher-
quality content faster than ever before. DeviceAnyWhere provides access to real mobile handsets that
are each connected to live networks. These devices are accessible remotely over the Internet, allowing
users to interact in real time to ensure all development, porting, testing, and monitoring needs are met.
The DeviceAnywhere service uses actual, physical handsets to provide developers with the same
experiences as having the devices in hand.
4
Test Quest
TestQuest is the leader in automated test solutions for mobile and wireless applications and devices. By
using TestQuest solutions, companies reduce test cycle times, improve test accuracy, collaborate to
accelerate new products to market and ensure a quality end user experience.
Mobile Device Perspective
Keynote Mobile Device Perspective enables you to measure the true end-to-end subscriber experience of
your mobile quality service from actual mobile devices. It allows you to validate service availability,
delivery times, throughput, as well as accuracy of content and services.
Steps for Recording in I-OS device through LR
1) Installation of Hotspot in Vugen Machine
2) Proxy Recording in LR - Protocol for Recording (Mobile Application Protocol or WebApp with Older
recording engine)
3) I-Phone /I-OS device setup
Installation of Hotspot:
Download and Install Connectify Hotspot to vugen machine
Click on “Start Hotspot” button to trigger the HotSpot
5
Proxy Recording in LR –
through Mobile Application Protocol open New Mobile app script in LR 11.52, click on Recording button
& Select “Proxy Recording”
Specify Port in the next window as below (e.g.7777 )
6
Click Finish button to start recording
I-Phone /I-OS device setup
Go To Settings – Wifi - & Select Connectify
Click Join Network, Enter password & Click Join
7
Configure HTTP proxy in the Wifi from OFF to Manual & provide Vugen machine’s Host Name & Port
Details
Install the certificate in mobile device by hitting the URL (http://<Hostnameof VuGenmachine>:
<port>/proxyroot.cer). Then Click Install as shown below.
Sample mobile browser based recording snapshot
8
EMULATORS
Android 1.5 SDK
What is Emulator ?
The Android SDK includes a mobile device emulator — a virtual mobile device that runs on your
computer. The emulator lets you develop and test Android applications without using a physical device.
Configure and Launch Emulator –
• once after downloading the package. Click on SDK (Software Development kit)page to upgrade to
latest version of software.
• Click on AVD.exe to create Android Virtual device
• Give AVD name (by this name the LR tool will recognize the emulator), Device, SD card Size: min
of 512 mb
• Click on start to Open the Device
9
Installing APK file in Emulator
• Go to Command Prompt
• Give the path of the Android Application using Command – ADB install filename.pak
Mapping AVD to LoadRunner:
• Open Vugen, select Mobile Http/Html protocol
• Click on Record Through Emulator
• In the recording wizard browse the path of Emulator and start Recording
Monitoring Tools
Spectra 2
- IMS/TISPAN, VoIP, and legacy/mobile core network test solutions for load/stress testing,
conformance testing, Media/QoS, and network monitoring.
Spectra2SE
- A cost-effective, software-only edition of Spectra2 for your PC or laptop
Spectra2XL
- High-performance chassis that produces extreme load/stress testing capacity
Spectra2XL
- Powerful 2G, 3G and WiMAX Functional and Load tester
K12-G35
- Powerful 2G, 3G and WiMAX Functional and Load tester
K15 Classic
- Powerful 2.5 and 3G Protocol Monitoring Tool
K18
- Monitors UMTS, UTRAN, and WiMAX
NSA
- To help you better optimize your 3G networks, Tektronix brings you the UTRAN Network
and Service Analyzer, a suite of real-time and off-line software applications designed to
troubleshoot and optimize those 3G networks and services supported by the K18 and K15
platform
10
K2Air
- The K2Air air interface probe provides real-time, promiscuous mode, air interface monitoring
capabilities via CPRI and is seamlessly integrated into the Network & Service Analyzer™
(NSA™) to provide vendor and carriers alike with a 360 degree, end-to-end view of LTE
network and services.
Mobile Testing Lab Setup Approach & Usage
Key challenges in Mobile testing are Increasing handset, application functionality compatible on
diversified platform and devices Mobile lab enable us to carry out detailed analysis on various tools,
simulators and develop innovative solution for our existing customers and potential customers
11
Remote Access: There is also opportunity to make the machines in these environment remotely
accessible from onsite and create virtual lab at offshore.
Lab Setup Process
➢ Identification of Lab Requirements
o Infra
▪ Tables
▪ Chairs
▪ Branding
o Network
▪ Cabling
▪ Connectivity
▪ Voice(vnets)
➢ Identification of Lab Location
➢ Approvals
➢ Raising PO & Procurement of Requirements
➢ Establishing Lab setup(infra & network)
➢ Setting up Lab Test Environments
➢ Pilot
➢ Live
Infrastructure & Network team need to get below things
12
Mobile Lab rough sketch
Benefits Of Mobile Lab
13
Suggested Reference
1. http://articles.techrepublic.com.com/5100-10878_11-5207561.html
2. http://www.safecomprogram.gov/NR/rdonlyres/5C74C631-ACF6-433F-B313-
C04D041A5489/0/Look_Future_Wireless_Communications_Beyond3G.pdf
3. http://www.forrester.com/Research/Document/Excerpt/0,7211,42749,00.html
4. http://www.tradingmarkets.com/.site/news/Stock%20News/2344437/
5. http://www.sybase.com/files/White_Papers/IDC_MobilePlatform_WhitePaper_Oct2008.pdf
6. http://gnokii.org/
7. http://www.pressreleasepoint.com/open-source-saas-collaboration-solution-provides-push-
email-and-cloudbased-mobile-sync-15-billion-sm
8. http://mite.keynote.com/product_overview/index.php
9. http://www.ciol.com/Developer/Testing/News-Reports/DeviceAnywhere-integrates-mobile-
testing/2609120397/0/
10. http://www.testquest.com/
11. http://www.testquest.com/objects/Library/appnote_net_perform.pdf
12. http://www.deviceanywhere.com/
13. http://www.mverify.com/resources/resources.htm
14. http://www.mverify.com/technology/mVerify-MTS-PerformanceTesting.pdf