Mohit Anand, Software EngineerAdobe

1

Selecting GUI Automation Testing Tool for Mobile Domain

2

Abstract

With the growing number of mobile apps and their agile development life

cycle, Automation testing around them will no longer be optional. The intent of

this paper is to present a comparison between the widely used mobile

automation tools in order to determine their ease of use & effectiveness. We

picked up a typical Adobe mobile app, tested its GUI manually & then followed

it up by a round of automation testing using these tools. This paper list down the

technical nuances involved while choosing any such tool for user specific

scenario.

3

Do you know your apps?

4

Complexities in Mobile Landscape

5

Operating System & Device Manufacturers

6

Screen Size & Resolution

7

GPU: Heart of Creative Apps

8

Localization: The Big hurdle

9

Our main consideration

10

Need of Automation in Mobile Testing

Automation is the key

7(Q1’14) – 40(Q1’16)

11

Mobile Automation Tools

12

Selection criteria

We want to have a combination of tools which satisfy the following criteria:

Can automate Android/iOS/Both mobile apps

First party/Third party tool

Bundled with the OS IDE/ Should be installed separately

Coordinate based/ Object based automation

13

Tools Selected

14

Key Features

Components of this tool

Google• Provides an API for writing programs to control an Android device/emulator.

• Can be used for both functional and UI testing.

• Entirely based on coordinates based automation.

• MonkeyRunner

• MonkeyDevice

• MonkeyImage

MonkeyRunnerMonkeyRunner

15

MonkeyRunnerMonkeyRunner

16

Key Features

Components of this tool

Google• It is a java library containing API’s to create customized functional UI tests.

• Integrated with Google’s own Android development IDE

• Entirely based on object based automation.

• Uiautomatorviewer

• API framework.

UI AutomatorUI Automator

17

UI AutomatorUI Automator

18

Key Features

Components of this tool

• It is an integrated environment for recording, customizing and managing test suites.

• Open source, eclipse based IDE.

• Used for automation of both iOS & Android apps.

• Limited features are available in the free version of this tool. Paid version is costly.

• MonkeyTalk IDE

• MonkeyTalk Agent

Cloud Monkey

19

20

Key Features

Components of this tool

• Appium Server

• Appium Client libraries

• Appium Inspector

• It is a HTTP server that takes HTTP commands from the test script and executes the command on the device.

• Based on WebDriver JSON protocol.

• Used for automation of both iOS & Android apps.

• Totally derived from selenium code + mobile gesture handling

Sauce Labs

21

22

Deep dive into Appium

23

Test Setup

24

The Process: Recipe of Success

25

Demo of Adobe Shape GUI Automation

26

Takeaways

Use real devices: In Adobe, we say no to simulators.

All elements in the UI should have a unique resource id.

The App UI has changed: Get ready for modifying your test script.

Say NO to sleeps and xPath.

iOS Automation: You need code access or debug build.

27

Choose the RIGHT tool

28

Use case scenario

Monkey Runner

Android app only Basic UI testing Time Crunch Low effort/investment

UI Automator

Android app only Intensive UI testing.

Don’t want to use third party tool.

Have time to build a stable automation

solution.

Monkey Talk

Android & iOS app

Your app doesn’t supports web

view.

Image comparison

requirement.

Can install agents in your app / can invest money per

month

AppiumAndroid & iOS app

Intensive UI testing

Don’t have code / Don’t want to

install any agent.

Stable automation

solution (free) for suite of apps

29

Bibliography

o http://developer.android.com/tools/help/monkeyrunner_concepts.html

o https://www.cloudmonkeymobile.com/monkeytalk

o http://developer.android.com/training/testing/ui-testing/index.html

o https://developer.android.com/about/dashboards/index.html

o https://developer.apple.com/support/app-store/

30

Question and Answers

31