nakal think test_2015
TRANSCRIPT
Enrich Your Mobile Automation
With Visual Assertions
http://clipartzebraz.com/cliparts/eye-clipart/cliparti1_eye-clipart_10.jpg
Rajdeep VarmaSr. Consultant, ThoughtWorks
About Myself• Rajdeep Varma
• 6 years in Quality Engineering. I work at ThoughtWorks
• Active open source contributor
• Passionate about test automation and agile practices
• Tested a number of web apps and recently started testing mobile apps
I will talk about• The Problem: Limitation of functional
testing tools and Need of Automated visual regression.
• Our Approach: “Nakal”
• Some Challenges and solutions
• Future Plans
WHY ??
version-3.1
version-3.2
Difference
Common Codebase
App 1 App 2
The Problem• UI changes done for App-1,
frequently break UI of App-2
• There is no automated regression to catch this!
Tech Radar
• CSS Critic
• dpxdt
• Huxley
• PhantomCSS
• Wraith
NOT FOR
MOBILE
Options• Manual UI regression against past build:
Error prone. It SUCKS!
• Proprietary cloud based tools
• Create a tool that meets our need
http://images.clipartpanda.com/choice-clipart-fork-choice-hi.png
My Need• No new framework. Use existing tests.
• Easy to extend any test for visual validations
• Easy to integrate in our CI
• On-premise and easy to debug failures locally
My tool-set
“nakal”
Demo
How it works?
• ADB for capturing screen on android.
• AppleScript for capturing iOS Simulator screenshots
• ImageMagick for comparison
Installation and Configuration
•Install ImageMagick•Include in Gemfile
gem ‘nakal' •In env.rb or any equivalent file:
require “nakal/cucumber" OR require "nakal" ; include Nakal::DSL
Nakal.platform = :android OR Nakal.platform = :ios
To add visual assertion
nakal_execute(“home_screen")
•For the very first time, to create baseline, run your test with environment variable NAKAL_MODE=build
•Now onwards, run your test with environment variable NAKAL_MODE=compare
Some Improvements• Implicit wait till current screen matches baseline
• Mask a dynamic region
• Crop Notification and scroll area
• Embedded screenshots in cucumber reports
• Driver independent. Can work with appium or calabash etc
• Easily integrate in CI by setting NAKAL_MODE environment variable
• Use ImageMagick fuzz for better comparison
Nakal Configuration
default_device: top: 70 right: 0 left: 0 bottom: 0 login_screen: {region_foo: [390, 112,512,156]}
Best Practices•Use mocking to make your components reliable
•Use directories to organize your baseline. e.g:
nakal_execute(“login_flow/home_screen”)
•Don’t use if UI is under very active development
•Use only when there is genuine need
Coming Soon..
• Better masking• Java version
https://github.com/rajdeepv/nakal
rubygems.org/gems/nakal
nakal
References:
https://www.thoughtworks.com/radar
https://rubygems.org/gems/rmagick/versions/2.15.4
http://rmagick.github.io/
https://developer.android.com/tools/help/shell.html
https://github.com/Huddle/PhantomCSS
?Rajdeep VarmaSr. Consultant, ThoughtWorksEmail: [email protected]