moet: mobile end-to-end testing
DESCRIPTION
TRANSCRIPT
![Page 1: MOET: Mobile End-to-End Testing](https://reader036.vdocuments.mx/reader036/viewer/2022081800/549296b4b479596a4d8b464b/html5/thumbnails/1.jpg)
1
MOET
https://github.com/eing/moet Mobile End-to-End Testing Mobile Exploratory Testing
Eing Ong
Intuit, Inc. @eingong|[email protected]
1
![Page 2: MOET: Mobile End-to-End Testing](https://reader036.vdocuments.mx/reader036/viewer/2022081800/549296b4b479596a4d8b464b/html5/thumbnails/2.jpg)
2
Mobile'automation'technologies1
• Two categories • Instrumented technique • Non-instrumented technique
• What is instrumentation? • Tests are compiled with the app • Tests are installed & launched with the app • Source code is required and may need to be modified • Only one app can be executed at a time • White box approach
![Page 3: MOET: Mobile End-to-End Testing](https://reader036.vdocuments.mx/reader036/viewer/2022081800/549296b4b479596a4d8b464b/html5/thumbnails/3.jpg)
3
Advantages'of'both'techniques1
Non$instrumentation- Instrumentation-
• Device'platform'agnostic1• Test'code'reuse1• Test'language'&'test'harness'
autonomy1• Support'for1
Multi>applications'testing1Custom'composite'UI'elements1Database/server'API'assertions1Use'of'external'libraries1'(e.g.'hardware/image'libraries)1
• Elements'can'be'accessed1• Debugging'ease1• Test'verification'ease1• Reduce'tools'dependencies1• Support'for1
Installing'application1Launching'application1Cleanup'(kill'application)1Test'execution'on'device1Code'coverage1
![Page 4: MOET: Mobile End-to-End Testing](https://reader036.vdocuments.mx/reader036/viewer/2022081800/549296b4b479596a4d8b464b/html5/thumbnails/4.jpg)
4
Which'technique'should'I'use'?1
Text$based3features-
Image3based3features-
Hardware3interaction-
External3interaction-
Instrumentation Non-instrumentation
![Page 5: MOET: Mobile End-to-End Testing](https://reader036.vdocuments.mx/reader036/viewer/2022081800/549296b4b479596a4d8b464b/html5/thumbnails/5.jpg)
5
Mobile'automation'tools1
Mobile3-OS-
Non$instrumentation-
Instrumentation-
Android1 eggPlant,'Sikuli,'MOET,'MonkeyRunner1
Robotium,'Calabash,'MonkeyTalk1
iOS1 eggPlant,'Sikuli,'MOET1 UIAutomation,'KIF,'iCuke,'Frank,'UISpec,'Zucchini,'Bwoken,'Calabash,'MonkeyTalk,'Applecart,'iOS>driver1
![Page 6: MOET: Mobile End-to-End Testing](https://reader036.vdocuments.mx/reader036/viewer/2022081800/549296b4b479596a4d8b464b/html5/thumbnails/6.jpg)
6
touch(100,100)
enter(username) touch(100,200)
enter(password)
touch(150, 300)
void enter()
void touch(x,y)
Login(“user1”,”passwd1”) Test
iPhone Sikuli library
iPhone implementation
MOET1
• Think design o Interfaces o Creational pattern
• Think reuse o Device independent tests
• Think One o IDE o Test harness and language
![Page 7: MOET: Mobile End-to-End Testing](https://reader036.vdocuments.mx/reader036/viewer/2022081800/549296b4b479596a4d8b464b/html5/thumbnails/7.jpg)
7
Anatomy'of''MOET1
iPhone3application3implementation-
Android3MonkeyRunner3Library-
Device3Independent3Tests-
Simulator/Device libraries
Mobile3Application3Interface-
Runtime binding
iPhone3Sikuli3Library-
Android3application3implementation-
![Page 8: MOET: Mobile End-to-End Testing](https://reader036.vdocuments.mx/reader036/viewer/2022081800/549296b4b479596a4d8b464b/html5/thumbnails/8.jpg)
8
Top Ten Wish List
a.k.a.
Mobile test automation
Wall of Pain
8
![Page 9: MOET: Mobile End-to-End Testing](https://reader036.vdocuments.mx/reader036/viewer/2022081800/549296b4b479596a4d8b464b/html5/thumbnails/9.jpg)
9
Android'Wall'of'Pain1
10. There’s more to test execution o Command with report file generation :
adb shell am instrument -w -e reportFile junit-report.xml com.intuit.test/com.zutubi.android.junitreport.JUnitReportTestRunner
o Date-time fix if using device snapshots : adb shell date -s `date +"%Y%m%d.%H%M%S”`
9. Where are the test results ? o Pulling results from device :
adb pull /data/data/com.intuit.test/files/junit-report.xml junit-report.xml
![Page 10: MOET: Mobile End-to-End Testing](https://reader036.vdocuments.mx/reader036/viewer/2022081800/549296b4b479596a4d8b464b/html5/thumbnails/10.jpg)
10
Android'Wall'of'Pain1
10. There’s more to test execution o Command with report file generation :
adb shell am instrument -w -e reportFile junit-report.xml com.intuit.test/com.zutubi.android.junitreport.JUnitReportTestRunner
o Date-time fix if using device snapshots : adb shell date -s `date +"%Y%m%d.%H%M%S”`
9. Where are the test results ? o Pulling results from device :
adb pull /data/data/com.intuit.test/files/junit-report.xml junit-report.xml
![Page 11: MOET: Mobile End-to-End Testing](https://reader036.vdocuments.mx/reader036/viewer/2022081800/549296b4b479596a4d8b464b/html5/thumbnails/11.jpg)
11
iOS'Wall'of'Pain1
8. iOS simulator…the singleton pattern o Multiple instances of simulator o Headless execution
7. What test harness ? o Test grouping, test assertions, base test class o Results needs to be displayable in browser
6. How about a CI plugin o Test execution o Code coverage
![Page 12: MOET: Mobile End-to-End Testing](https://reader036.vdocuments.mx/reader036/viewer/2022081800/549296b4b479596a4d8b464b/html5/thumbnails/12.jpg)
12
iOS'Wall'of'Pain1
8. iOS simulator…the singleton pattern o Multiple instances of simulator o Headless execution
7. What test harness ? o Test grouping, test assertions, base test class o Results needs to be displayable in browser
6. How about a CI plugin o Test execution o Code coverage
![Page 13: MOET: Mobile End-to-End Testing](https://reader036.vdocuments.mx/reader036/viewer/2022081800/549296b4b479596a4d8b464b/html5/thumbnails/13.jpg)
13
iOS'Wall'of'Pain1
8. iOS simulator…the singleton pattern o Headless execution o Multiple instances of simulator
7. What test harness ? o Test grouping, test assertions, base test class o Results needs to be displayable in browser
6. How about a CI plugin o Test execution o Test results o Code coverage
![Page 14: MOET: Mobile End-to-End Testing](https://reader036.vdocuments.mx/reader036/viewer/2022081800/549296b4b479596a4d8b464b/html5/thumbnails/14.jpg)
14
iOS'Wall'of'Pain1
5. Where’s the IDE plugin ? o Code formatting o API completion o Breakpoints, debugger o “javadoc” help o One click test execution of test method/testsuite
![Page 15: MOET: Mobile End-to-End Testing](https://reader036.vdocuments.mx/reader036/viewer/2022081800/549296b4b479596a4d8b464b/html5/thumbnails/15.jpg)
15
All'mobile'platforms1
4. Problem ? Search ? o Blogs, stackoverflow, developer groups for devices, etc. o Mobile testing one-stop tips & tricks (not questions) o Community driven effort
Tip – Remove authorization prompt
Administrative rights Update /etc/authorization
<key>system.privilege.taskport</key> <dict> <key>allow-root</key> <!-- previous value <false/> --> <true/>
Tip – Clean cache, preferences, SQLite ~/Library/Application Support/iPhone Simulator/{SDK}/Applications/<uuid>
Library: Preferences, Caches Documents: sqlite3 <app>.sqlitedb
![Page 16: MOET: Mobile End-to-End Testing](https://reader036.vdocuments.mx/reader036/viewer/2022081800/549296b4b479596a4d8b464b/html5/thumbnails/16.jpg)
16
All'mobile'platforms1
3. How about a mobile test report ?
o Mobile test report standard e.g. mobile-test-report.xml • Screenshots and video recording of failed tests • Device logs • Device detection
2. Wireless?
o OTA app installation o Testing via wifi and carrier networks o Selenium grid-like test infrastructure
![Page 17: MOET: Mobile End-to-End Testing](https://reader036.vdocuments.mx/reader036/viewer/2022081800/549296b4b479596a4d8b464b/html5/thumbnails/17.jpg)
17
All'mobile'platforms1
3. How about a mobile test report?
o Mobile test report standard e.g. mobile-test-report.xml • Screenshots and video recording of failed tests • Device logs • Device detection
2. Mobile testing is anything but mobile
o OTA app installation and execution o Testing via wifi and carrier networks o Test tool independent component o Selenium grid-like test infrastructure
![Page 18: MOET: Mobile End-to-End Testing](https://reader036.vdocuments.mx/reader036/viewer/2022081800/549296b4b479596a4d8b464b/html5/thumbnails/18.jpg)
18
All'mobile'platforms1
1. Yet another proprietary language?
o Open sourced test harness & language for • Instrumented tests • Non-instrumented tests • Hybrid apps tests
![Page 19: MOET: Mobile End-to-End Testing](https://reader036.vdocuments.mx/reader036/viewer/2022081800/549296b4b479596a4d8b464b/html5/thumbnails/19.jpg)
19
Next'steps1
Monkey$Runner-
Device3Independent3Tests-
Simulator/Device libraries
Mobile3Application3Interface-
Runtime binding
Sikuli-
Non$instrumented3application3implementation-
Instrumented3application3implementation-
?Android?- ?iOS?3-
Eclipse mobile testing plugin
Test development
Test execution
Test reporting
Continuous integration