srinivas-smart- mobile application testing strategy · 2013-06-10 · testing mobile applications...

"SMART” Mobile Application Testing Strategy for

End to End Coverage and Automated Solutions

Srinivas Rao Labhani

Mobile Trends and Challenges

2©Alliance Global Services 2013

Mobile in Enterprise – Fortune 500 Companies

3©Alliance Global Services 2013

*Annual State of Mobile In Enterprise analysis in December 2012 by Alliance Global Services

Mobile in Enterprise – Fortune 500 Companies

4©Alliance Global Services 2013

*Annual State of Mobile In Enterprise analysis in December 2012 by Alliance Global Services

Mobile Technology’s Growth and Trends

• User base of location-based services has grown

from 96M in 2009 to more than 526 million in 2012

(Gartner Research)

• Worldwide advertising and content revenue

forecast to reach $67B

• Smartphone shipments will reach 978M in 2014

(IDC)

• Smartphones and tablets will put power in the

hands of a billion global consumers (Forrester

Research, Inc)

5©Alliance Global Services 2013

Research, Inc)

• Mobile apps for the first time are providing more

compelling functionality than desktop apps

Mobile Development ApproachesC

om

ple

xity

& C

ost

of

Test

ing

Hybrid Mobile App

Native Mobile Apps

Cross-device, HTML5 based functionality delivered via Cross-device, HTML5 based functionality delivered via

Native iOS and Android apps with full access to

playback, and app integration.

Native iOS and Android apps with full access to

device sensors, camera, microphone, video

playback, and app integration.

6©Alliance Global Services 2013

Capability & Experience

Co

mp

lexi

ty &

Co

st o

f Te

stin

g

Mobile Compatible Website

Mobile Optimized Web App

Configure existing websites and apps for compatibility.

Ensure most commonly used transactions work correctly.

Create separate website targeted to mobile. Design for

touch, small screens, mobile UI pattern, task focused use.

functionality – location, network support.

Cross-device, HTML5 based functionality delivered via

installed App through common App Stores. Some device

functionality – location, network support.

Testing Mobile Applications

Mobile application testing on Smart Phones and Tablets is especially challenging

given the variety of combinations in which applications interface with devices and

networks.

• Device & OS Diversity– Oses & Networks

– Form factors

• Application Features– Location-aware

7©Alliance Global Services 2013

– Location-aware

– Local Storage

– Offline Access

• Device Hardware– Memory

– Processing etc.

• Network Service Providers

• Rapid updates of devices

and Oses

• Testing tools and

ecosystems still evolving

Mobile test automation challenges

Tools Explore multiple tools to evaluate feasibility of tools to you specific portfolio of applications

Skillsets Given the complexity of test automation and testing on mobile applications, resources on these projects need cross-functional skills

8©Alliance Global Services 2013

Time & Effort Traditional models for estimation of Time & Effort may not be useful for mobile testing and automation efforts – use POCs and jumpstarts

Infrastructure Test Environment and Automation Infrastructure for mobile is complex but can be built incrementally

Challenges for Enterprise App Testing

Infrastructure – Limited

resources to develop End-to-

End environmentsData Driven –

Application

functionality driven

by data

9©Alliance Global Services 2013

Tools –

Proliferation of

testing tools that are

not integrated

Workflow based –

Typical applications

flow through

numerous stages

Enterprise Mobile Testing Strategy

10©Alliance Global Services 2013

Mobile Testing – Optimization Factors

Quantitative prioritization enables the Business and Testing teams to prioritize Mobile

testing strategy according to importance

• Application Architecture

• Identify the development

approach and options

• HMTL5, Native app

development, HTML5 toolkits

• Integration with back-end

functionality

11©Alliance Global Services 2013

functionality

• Target OS’es

• Identify target OS and any OS

specific functionality

• iOS, Android, WinMo

• Device Specific Features

• Identify device specific features

• Local Storage, Camera, Offline

Access, Network dependencies,

Security

Mobile Testing – Optimization Process

Optimization Factor Identification

• Define optimization Factors for Probability and Impact of Failure

• Assign Weightage

Mobile Functionality Analysis

User Rating

• Assign scores according to the application, device, browsers, OS, business criticality combinations

Functionality Coverage

• Intermediate Test Coverage Decision for each Device, OS and Browser combination

Mobile Test Planning

• Mobile Test Assessment

• Mobile strategy preparation

• Prioritization Guidelines Overall Test

Coverage

12©Alliance Global Services 2013

Coverage

• Final coverage decision

• Test suite identification

• Test effort estimationFramework Setup

• Import requirements

• Update Optimization Factors

Mobile Device Testing Analysis

Optimization Evaluation

• Grading of Optimization Factors on a numeric scale

• Calculation of the Risk Exposure value

Classification

• Classification of combination of Devices, OS and browser based on Risk Exposure Value

Mobile Device Testing Coverage

• Mobile Device coverage based on Test coverage Analysis or user experience

Mobile Automation Strategy

13©Alliance Global Services 2013

SMART Mobile Application Testing StrategyThree-tier “SMART” Mobile Application Testing Strategy for end to end coverage,

automated solutions for each tier and testing on actual devices

Device TestingDevice TestingAutomated and Manual

on actual device in

Alliance Mobile Test Lab

and Cloud

Functional Testing

Device Management Testing

Installation Testing

Compatibility Testing

Device Profiling

UI Testing

14©Alliance Global Services 2013

Functional TestingFunctional Testing

Business Workflow TestingBusiness Workflow Testing

Automated testing of business workflows and backend

application and enterprise using SOA and Integration testing

Automated function testing of mobile application

using device simulators

SOA Testing

Integration Testing

Device Management Testing

Security Testing

Authentication Testing

Integration Testing

Align to Business Objectives

Mobile Automation Framework

15©Alliance Global Services 2013

Mobile Test Automation Framework

Test Scripts Automation

Engine Mobile Test

Results

Test Management System

Mobile Test Automation should be integrated with rest of the enterprise test

automation tools to ensure Return on Automation Investment

• Common Test

Management System

across enterprise, web

and mobile testing

• Organize test plans, test

data and reports by

application and functional

16©Alliance Global Services 2013

EngineResults

Au

tom

ati

on

To

ols

Mobile Test

Execution

Report

Modularized

Apps

Modularized

Components for

specific Mobile

Apps

Test Data

application and functional

hierarchy rather than

end-user access

• Develop risk identification

and defect root cause

analysis plans based on

this information

• Performance

measurements are a nice

to have

Thank YouSrinivas Rao Labhani

Technical Director – Software Testing Services

17©Alliance Global Services 2013

Technical Director – Software Testing Services

slabhani@allianceglobalservices.com

www.allianceglobalservices.com