a guided approach to testing - a mobile first strategy for your enterprise!

©2014 InfoStretch Corporation. All rights reserved.

A Guided Approach to Testing - A Mobile First Strategy for your Enterprise


InfoStretch Overview Enable Customers to Monetize Mobile Channel

Helping you Achieve Enterprise-Grade Mobile Applications

2


The Company

Unique Apps Tested

Employees

US, UK, India

150+ Apps developed and deployed for Enterprises

Real devices in mobile labs 3000+

Global Footprint

Mission

Locations Corporate HQ: Silicon Valley

Offices: Conshohocken (PA), Ahmedabad(India), Pune(India), London(UK)

500+

200K

Active Mobile Customer Engagements 60+

To Create Meaningful Customer Engagement and Improve

Employee Enablement and Productivity with Mobile Applications

3


App Store Certification Authority (2007 onwards)

4

Tested 200,000+ mobile applications in more than 16

languages and 20+ carrier networks around the globe

Serving 4 Largest Mobile App Stores for Certification


InfoStretch – 2013 Onwards

Deploy Deep

Mobile Skills

Design, UI &

Development for

Enterprise

Applications

Enterprise

Testing and

Automation with

Mobile Center of

Excellence(mCOE)

Optimization and

Post Production

Sustenance

Unified Platform for

Building Mobile

Solutions

Enterprise Mobility

Tools Stack to

Power Service

Engagements

5


Test Challenges for Enterprise Mobile Applications

6


Quick Poll

• Do you have a separate team for Mobile Testing or You rely on End-Users?

• Do you have a separate test lab with all mobile devices you support?

• Are you familiar with Mobile Testing tools?

7


Mobile Testing Challenges

• Proliferation of Devices, Platforms and Carriers

• Compressed Release and Development Cycles

• Generating Real World Testing Conditions

• Limitations of Manual Testing

9


#1 Device and Platform Proliferation

DEVICES

CARRIERS

OPERATING SYSTMES

10


#2 Compressed Release Cycles

v1.0 v2.0 v3.0 v4.0 v5.0 v2.1

Nov 2013

Mar 2014

Apr 2014 Aug 2014

Dec 2014

Feb 2015

Mobile Applications

Web Applications

v1.0 v2.0 v3.0 v4.0 v5.0 v2.1

Nov 2013 Dec 2014 April 2015 Jan 2016 Sept 2016

Feb 2017

11


#3 Generating Real World Conditions

• How will the application behave under 70% CPU utilization?

• How will connection speed change the application’s performance?

• What is the performance impact of 50 other applications running on the device?

• How will it work with interrupts like SMS and MMS?

12


#4 Limitations of Manual Testing

• Running the code manually – tedious

• Effort required to run manual test cases – expensive

• Process of manual testing – boring and error prone

• Non repeatability and reuse – time consuming

13


Déjà vu

14


Need for a Strong Quality Program

Pace of Change

Device and

Platform

Fragmentation

Cost of Failure

Backdoor

Vulnerabilities

Employee

Satisfaction and

Productivity

15


Enterprise

Mobile

Infrastructure

Testing

Application

Functionality

Testing

Enterprise Mobility and Quality Needs

Maintenance &

User Support

Testing


Quality Strategy Needs For Enterprise MDM

Enroll Configure Secure Service

• Test VPN to enterprise server, Email

• Verify OTA content delivery under varying network conditions

• Verify App Deployment • Verify enforced password

policies • Verify data encryption • Test device compatibility with

anti virus

• Device Analysis • Lab Setup with possible

OEM, OS and Carrier combinations

• Device Compatibility with identified policies

• Test enrollment via SMS, OTA, Email

• Passcode policies • Device Unlock • Containerization/Virtualization • Enforce Encryption • Security updates

• Verify Security updates • Testing data backup • Verify device conformity • Verify Logging/Reading • Verify app-to-app

communication • Verify data leakage

• End of life • Passcode Reset • Tracking Device • Update Software • Device Usage Data • Restore data

• Verify device passcode reset • Verify that device data restore

& remote access • Test end to end SMS/Cloud

Notification • Verify successful app

upgrade/patch to existing application on device

• Verify end of life

• BYOD • Enroll via SMS • Enroll via Email • Enroll via web URL

• Exchange Mail • VPN • Network Settings • Install/configure antivirus

MD

M F

EATU

RE

TEST

STR

ATE

GY


QA Lifecycle of Smart Enterprise Apps Infrastructure

Define Test Strategy

•Align QA objectives to Enterprise Mobility Vision

•Define Scope and Limitations of QA

Test Plan

•Device Analysis and Lab Setup

•Data Access and Security

•Testing Containerization

Functional Testing

•App functionality test

•App interoperability test

Device Lifecycle Testing

•Non-functional device specific testing

•Device provision testing

•Device End of life testing

New App Certification Process

•Application Certification on Enterprise App Store

•Enterprise Framework compliance

•Security testing

Ongoing Testing of Service Management

•No harm to data and Networks

•New device validation

•New OS validation

18


Types of Mobile Application Testing

Functional

• Validation of Functionality

• Smoke / Regressions Testing

• Off line access testing

• Negative Testing

Non Functional

• Network Strength / Outage / Recovery

• Different NW Types

• Peripheral Testing

Interrupt

• Voice / SMS interrupts

• Notifications

• Battery /Cable Removal

Memory Leak

• Memory Usage

• Memory Leaks

• Garbage Collection

Performance Testing

• CPU Usage testing

• Network Usage

• Page Render time or activity Render time

Usability Testing

• User Experience

• Competitive Analysis

• Expert Review

Installation Testing

• New App Install

• Uninstall and Reinstall

• Upgrade testing

Security Testing

• OWASP Vulnerabilities

• Dynamic Testing

• Static Code Analysis

• Data Encryption

Language Testing

• Validation for Locales

• Images and Text

• Currencies, time zones etc.

• Context


Quality – An Integral part of your Enterprise Mobile Strategy

• Comprehensive Testing Infrastructure

• Agile Testing Practices

• Market Change and Application Impact Monitoring

20


Enterprise Mobility Quality Enablers

QAS

Selenium

based Mobile

Testing

Solution for

End-to-End

Functionality

Testing

QMetry

A SaaS based

mobile

application

lifecycle

management

and

maintenance

tool

Synchro

Synchronized,

geo distributed

testing on real

devices for

optimal user

experience

management

Attesto

Intelligent UI

heuristics tool

to aggregate

and analyze

user reviews

and perform

verification

and validation


UX

Designer

Product

Manager

Developer

Support

Functional

Testing

Business

Use Case

Validation

Synchro

Device

Testing

UI & Layout

Validation

Synchro

Automation

Functional /

Regression

Automation

QAS

Diagnostics

Debug

Issues

Synchro /

QAS

Support

Monitor &

Sustenance

Attesto

Private Device Cloud – 3000+ Devices

Test Management Platform – QMetry /QC

Security

Testing

Load Testing

Monitoring

Non-

Functional

Testing

Retail

Finance

Re

ad

ym

ad

e T

est

Pa

cks

User

Roles

InfoStretch Integrated Solutions – Mobility QA


Case in Point Enterprise Mobile MDM QA Strategy for Large Global Insurance company’s Smart App

The Client Profile Company: Large Global Insurance Company

Industry: Insurance and Financial Services

Details:

The Challenge

Main Drivers - Reliability, security and cost

- 70,000 employees around the world

- Mobile applications integrated with other enterprise

apps and field apps

- One of the most aggressive and widespread BYOD program

- Global implementation challenges with wide and diverse

requirements across Geos

- Significant need to use third party certified applications.

- Stringent on field requirements, causing unique networking

challenges

- Validation of the MDM and MAM implementation plan

- Significant post go live changes


Phase 1

•Phase 1 – Mobile Testing Strategy (8 weeks)

•Identified overall scope and plan of testing

•Defined multiple aspects of quality to align with MDM and MAM implementation

•Understood current QA team and environment limitations

•Deliverables: Mobile QA plan, high level test strategy for each area (module) of MDM and MAM deployment, project and resource plan, cost estimates

Phase 2

•Phase 2 – Definition of Test lab, Test Areas and Test cases, Testing and Go live (3 months)

•Defined set of devices that meet company requirements and provision them

•Identified functional and non functional tests (300+), relevant test environment and conditions

•Specific test methodology to address unique MAM and MDM implementation testing

•Go live on-time with adequate quality and certification program for external apps

Phase 3

•Phase 3 – Post go live – Special Events Testing (ongoing)

•Post go live test plan

•Identify specific post go live QA trigger points and exit criteria

•Execution of App store certification, Security testing program, specialized events testing

Case in Point Enterprise Mobile MDM QA Strategy for Large Global Insurance company’s Smart App


Testing of Event Mgmt. App. for a Large Hi-Tech Company

25

• Event Management Platform Testing

• Self Service – Auto Generated App Testing

• Event Content Testing


Thank you ! Email：[email protected]

URL：www.infostretch.com

Blog : blog.infostretch.com

http://www.infostretch.com/

http://blog.infostretch.com/

a guided approach to testing - a mobile first strategy for your enterprise!

Software

infostretch corporation

enterprise mobile applications

mobile testing tools

device data

mobile devices

infostretch overview

mobile labs3000

mobile channel