android e2e testing at mercari

マスタータイトルの書式設定

• マスターテキストの書式設定

VishalAndroid Test Night #1

Android e2e Testing at Mercari


• マスターテキストの書式設定 Software Engineer in Test

Case Study (Mobile Automation)

SWET (past)

Core Member

Vishal Banthia @vbanthia @vbanthia_

https://github.com/vbanthia

https://twitter.com/vbanthia_





Agenda

• About Mercari US Android App• Android UITest Architecture & Design• Continuous Integration


• マスターテキストの書式設定In June, we had a major update in mercari US app.

mercari US app

Native Native Native React-Native

Not Just UI Changes, but complete architecture shift


• マスターテキストの書式設定• While deciding new architecture, easy testability was always a priority

• One of the reason to use dependency-injection pattern is above and we use Dagger2 for this. With this, mocking objects is pretty easy and can increase UnitTest coverage

• We always try to keep low-level UnitTest coverage high following Test Pyramid Principle

New Architecture

Note: More technical details can be read here (http://tech.mercari.com/entry/2017/07/19/123443)



But UITesting is always challengingand was so for US


• マスターテキストの書式設定• After joining mercari, this was my first big project. Following objectives

were in mind:• Adding new test scenarios and maintaining old ones should be painless• High reliability so that flaky tests are not wasting developers and QA

engineers time• Scalable architecture to minimize test run time• Tests should run on real devices instead of emulators• Minimum operational cost for infrastructure and CI management• Test reports should have all the important information required for

debugging if test fails• Super portable, should be easy to copy for other products

Android UITesting



UITest High Level Architecture

Ruby

RSpec

AppiumKit

Appium

UIAutomator2

Test Script Automation tool App Under Test

(MercariKit)



appium_kit (Ruby Library)

A simple ruby gem which extends famous selenium testing ruby library such as Capybara, SitePrism for Appium



Selenium Web Testing in Ruby

selenium-webdriver

capybara watir

site_prism page-object

rspec cucumber test-unit



Quick Overview

● selenium-webdriver is the ruby binding for Selenium

● capybara provides DSL to help writing web tests easily

● page-object design pattern helps in maintaining test easily

● site_prism provides DSL to write pages easily

These libraries have proved themselves for so long, we wanted to use something which we can

rely and can also use same stack for web-testing that is why, simple appium_kit gem.



appium_kit

selenium-webdriver

capybara

site_prism

rspec

appium-driver

capybara-appium

site_prism

rspec helpers / formatter

appium_kit



UITest Low Level Architecture

user_profile_spec.rb (rspec)

UserProfilePage < SitePrism::PageUtils

Helpers



Overview

• Pages• All the application related information such as ids, static text etc are tied

with Page Object. Test will talk to application using either helpers or pages. Test can not access application directly.

• 1 Activity ~= 1 Page• Helpers

• Helper binds pages together• Utils

• Utilities such as RandomGen, AdbUtil etc. Not directly related to AUT• Configs

• Static testdata



Running Tests



Rake Tasks

bundle exec download_apk bundle exec rake spec

Environment Variables

● BUILD_NUMBEREnvironment Variables

● ENVIRONMENT

● FEATURES

● ...



Dockerizing UITests

• Dockerizing tests helps in sandboxing environment. • Deploying / Running tests becomes very easy. Do not

need to worry about run environment• Many more benefits …

More Details: https://github.com/vbanthia/appium-docker-demo



Dockerfile

https://github.com/vbanthia/docker-appium



run_uitest.sh

• Reserve a device on STF• Remote connect• Download apk• Start appium server• Run test



Continuous Integration



Mercari Android CI (Tools & Service)

• Bitrise: To build application• Google Cloud Storage: To store built apks• Google Container Registry: To store docker images• OpenSTF: For Android Device Farm• CircleCI 2.0: Build docker image & run tests in parallel containers• CircleCI Artifacts: To store test reports• AWS Lambda: To trigger nightly builds• Slack: Notification & manual triggers



Application Build Pipeline

Developer Github Bitrise GCS

1. Developer pushes code to Github

2. Github triggers Bitrise

3. Bitrise build app and upload it to GCS

This make sure that we have all the apks

in GCS with their build number linked



UITest docker image build pipeline

Developer Github CircleCI GCR

1. Developer pushes code to Github

2. Github triggers CircleCI

3. CircleCI build docker image for uitest

and pushed it to google container

registry

This make sure that we have all the

docker image of latest test code any time



UITest Pipeline



OpenSTF



UITests Pipeline

CircleCI Job

AWS Lambda

Slack

Connect remote deviceDownload apk

Run UITests

(device1)

Run UITests

(device2)

Run UITests

(device3)

Test reports

CircleCI Artifact



Benefits

• Everything is on cloud, we only have devices connected to mac-mini which only needs adb to work

• Always ready to run• No need to worry about on-premise infra (Jenkins etc...)



Test Reports



RSpec HTML Formatter2

● https://github.com/vbanthia/rspec_html_formatter2

● Features:

o Beautiful html report using Bootstrap css/js

o Supports screenshots & screenrecords

https://github.com/vbanthia/rspec_html_formatter2



Thank you

android e2e testing at mercari

Internet