nakal think test_2015

Enrich Your Mobile Automation

With Visual Assertions

http://clipartzebraz.com/cliparts/eye-clipart/cliparti1_eye-clipart_10.jpg

Rajdeep VarmaSr. Consultant, ThoughtWorks

About Myself• Rajdeep Varma

• 6 years in Quality Engineering. I work at ThoughtWorks

• Active open source contributor

• Passionate about test automation and agile practices

• Tested a number of web apps and recently started testing mobile apps

I will talk about• The Problem: Limitation of functional

testing tools and Need of Automated visual regression.

• Our Approach: “Nakal”

• Some Challenges and solutions

• Future Plans

WHY ??

version-3.1

version-3.2

Difference

Common Codebase

App 1 App 2

The Problem• UI changes done for App-1,

frequently break UI of App-2

• There is no automated regression to catch this!

Tech Radar

• CSS Critic

• dpxdt

• Huxley

• PhantomCSS

• Wraith

NOT FOR

MOBILE

Options• Manual UI regression against past build:

Error prone. It SUCKS!

• Proprietary cloud based tools

• Create a tool that meets our need

http://images.clipartpanda.com/choice-clipart-fork-choice-hi.png

My Need• No new framework. Use existing tests.

• Easy to extend any test for visual validations

• Easy to integrate in our CI

• On-premise and easy to debug failures locally

My tool-set

“nakal”

Demo

How it works?

• ADB for capturing screen on android.

• AppleScript for capturing iOS Simulator screenshots

• ImageMagick for comparison

Installation and Configuration

•Install ImageMagick•Include in Gemfile

gem ‘nakal' •In env.rb or any equivalent file:

require “nakal/cucumber" OR require "nakal" ; include Nakal::DSL

Nakal.platform = :android OR Nakal.platform = :ios

To add visual assertion

nakal_execute(“home_screen")

•For the very first time, to create baseline, run your test with environment variable NAKAL_MODE=build

•Now onwards, run your test with environment variable NAKAL_MODE=compare

Some Improvements• Implicit wait till current screen matches baseline

• Mask a dynamic region

• Crop Notification and scroll area

• Embedded screenshots in cucumber reports

• Driver independent. Can work with appium or calabash etc

• Easily integrate in CI by setting NAKAL_MODE environment variable

• Use ImageMagick fuzz for better comparison

Nakal Configuration

default_device: top: 70 right: 0 left: 0 bottom: 0 login_screen: {region_foo: [390, 112,512,156]}

Best Practices•Use mocking to make your components reliable

•Use directories to organize your baseline. e.g:

nakal_execute(“login_flow/home_screen”)

•Don’t use if UI is under very active development

•Use only when there is genuine need

Coming Soon..

• Better masking• Java version

https://github.com/rajdeepv/nakal

rubygems.org/gems/nakal

nakal

References:

https://www.thoughtworks.com/radar

https://rubygems.org/gems/rmagick/versions/2.15.4

http://rmagick.github.io/

https://developer.android.com/tools/help/shell.html

https://github.com/Huddle/PhantomCSS

?Rajdeep VarmaSr. Consultant, ThoughtWorksEmail: mail.rajvarma@gmail.com