guide to the jungle of testing frameworks
TRANSCRIPT
![Page 1: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/1.jpg)
![Page 2: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/2.jpg)
Unit Testing Experience on Android
![Page 3: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/3.jpg)
![Page 4: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/4.jpg)
Android apps are difficult to test
![Page 5: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/5.jpg)
Types of Android tests
![Page 6: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/6.jpg)
Types of Android tests
Instrumentation tests
Local unit tests
![Page 7: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/7.jpg)
Android test code• project sources
• ${module}/src/main/java
• instrumentation tests • ${module}/src/androidTest/java
• unit tests • ${module}/src/test/java
• full Gradle and Android Studio support
![Page 8: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/8.jpg)
![Page 9: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/9.jpg)
• the essential piece of both instrumentation and unit tests
• alone can be used only for pure Java
• doesn’t provide any mocks or Android APIs
![Page 10: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/10.jpg)
Instrumentation Tests
![Page 11: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/11.jpg)
Instrumentation Tests
• running on physical device or emulator
• gradle connectedAndroidTest
• ${module}/build/reports/androidTests/connected/index.html
![Page 12: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/12.jpg)
![Page 13: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/13.jpg)
Instrumentation Tests
Legacy instrumentation tests or
Testing Support Library
![Page 14: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/14.jpg)
Legacy Instrumentation Tests• JUnit3 • Tests extend from TestCase
• AndroidTestCase • ActivityInstrumentationTestCase2 • ServiceTestCase • …
deprecated since API level 24
![Page 15: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/15.jpg)
Testing Support Library• JUnit4 compatible
• AndroidJUnitRunnerandroid { defaultConfig { testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner" }}
dependencies { androidTestCompile 'com.android.support.test:runner:0.5'}
![Page 16: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/16.jpg)
Testing Support Library• JUnit test rules
• AndroidTestRule
• ServiceTestRule
• DisableOnAndroidDebug
• LogLogcatRule
• …
androidTestCompile 'com.android.support.test:rules:0.5'
![Page 17: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/17.jpg)
![Page 18: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/18.jpg)
• framework for functional UI tests
• part of Android Testing Support Library
androidTestCompile 'com.android.support.test.espresso:espresso-core:2.2.2'
![Page 19: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/19.jpg)
@Testpublic void sayHello() { onView(withId(R.id.edit_text)) .perform(typeText(STRING_TO_BE_TYPED), closeSoftKeyboard());
onView(withText("Say hello!")) .perform(click());
String expectedText = "Hello, " + STRING_TO_BE_TYPED + "!"; onView(withId(R.id.textView)) .check(matches(withText(expectedText)));}
![Page 20: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/20.jpg)
Problems
• testing on device is not isolated
• device state affects the result
• e.g. screen on/off might affect test result
onView(withId(R.id.my_view)) .check(matches(isDisplayed()));
![Page 21: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/21.jpg)
Some annoyances
android.support.test.espresso.NoActivityResumedException: No activities in stage RESUMED. Did you forget to launch the activity. (test.getActivity() or similar)?
![Page 22: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/22.jpg)
Instrumentation tests are
kindaSLOOOOOW
![Page 23: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/23.jpg)
Unit Tests
![Page 24: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/24.jpg)
Unit Tests
• run on JVM
• mockable android.jar
• gradle test
• ${module}/build/reports/tests/${variant}/index.html
![Page 25: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/25.jpg)
…...
![Page 26: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/26.jpg)
![Page 27: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/27.jpg)
• Helps rarely • returns 0, false, null, …
Method ... not mocked.
android { testOptions { unitTests.returnDefaultValues = true } }
![Page 28: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/28.jpg)
![Page 29: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/29.jpg)
• mocking framework • easy to use • compatible with Android unit testing
testCompile 'org.mockito:mockito-core:2.2.11'
![Page 30: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/30.jpg)
• can be used also in instrumentation tests • needs dexmaker
androidTestCompile 'org.mockito:mockito-core:2.2.11'androidTestCompile 'com.google.dexmaker:dexmaker:1.2'androidTestCompile 'com.google.dexmaker:dexmaker-mockito:1.2'
![Page 31: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/31.jpg)
@RunWith(JUnit4.class)public class ContextManagerTest {
@Mock Context mAppContext;
@Before public void setUp() { MockitoAnnotations.initMocks(this); }
@Test public void testWithContext() { … }}
![Page 32: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/32.jpg)
@RunWith(MockitoJUnitRunner.class)public class ContextManagerTest {
@Mock Context mAppContext;
@Test public void testWithContext() { … }}
![Page 33: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/33.jpg)
@RunWith(JUnit4.class)public class ContextManagerTest {
@Test public void testWithContext() { Context appContext = mock(Context.class); Mockito.when(appContext.getPackageName()) .thenReturn(“com.example.app”); … }}
![Page 34: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/34.jpg)
• Mockito.spy()
• wrapping a real object
• Mockito.verify()
• verify that special condition are met
• e.g. method called, method called twice, …
![Page 35: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/35.jpg)
Limitations
• final classes
• opt-in incubating support in Mockito 2
• anonymous classes
• primitive types
• static methods
![Page 36: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/36.jpg)
![Page 37: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/37.jpg)
• functional testing framework
• runs on JVM
• at first, might be difficult to use
• the ultimate mock of Android APIs
• provides mocks of system managers
• allows custom shadows
![Page 38: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/38.jpg)
• possible to use for UI testing
• better to use for business logic
@RunWith(RobolectricTestRunner.class)public class MyTest { …}
![Page 39: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/39.jpg)
• Robolectric
• RuntimeEnvironment
• Shadows
• ShadowApplication
• ShadowLooper
![Page 40: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/40.jpg)
Potential problems
• difficult to search for solutions
• long history of bigger API changes
• many obsolete posts
![Page 41: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/41.jpg)
![Page 42: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/42.jpg)
• Can mock static methods
• Can be used together with Mockito
![Page 43: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/43.jpg)
@RunWith(PowerMockRunner.class)
@PrepareForTest(Static.class);
PowerMockito.mockStatic(Static.class);
Mockito.when(Static.staticMethod())
.thenReturn(value);
PowerMockito.verifyStatic(Static.class);
![Page 44: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/44.jpg)
![Page 45: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/45.jpg)
• “matchers on steroids”
• offers more complex checks
assertThat(myClass, isInstanceOf(MainActivity.class));
assertThat(myManager.getValue(), isEqualTo(someValue));
assertThat(value, isIn(listOfValues));
assertThat(value, not(isIn(listOfValues)));
![Page 46: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/46.jpg)
![Page 47: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/47.jpg)
• cross-platform BDD framework
• human-like test definitions
testCompile ‘junit:junit:4.12'testCompile ‘info.cukes:cucumber-java:1.2.5'testCompile 'info.cukes:cucumber-junit:1.2.5'
![Page 48: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/48.jpg)
• describe the desired behaviour
Feature: CoffeeMaker Scenario: a few coffees Given I previously had 3 coffees When I add one coffee Then I had 4 coffees
![Page 49: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/49.jpg)
• create the mapping
public class CoffeeMakerDefs { CoffeeMaker mCoffeeMaker = new CoffeeMaker();
}
![Page 50: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/50.jpg)
• create the mapping
public class CoffeeMakerDefs { CoffeeMaker mCoffeeMaker = new CoffeeMaker(); @Given("^I previously had (\\d+) coffees$") public void hadCoffeesPreviously(int coffees) { mCoffeeMaker.setCoffeeCount(coffees); }
}
![Page 51: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/51.jpg)
• create the mapping
public class CoffeeMakerDefs { CoffeeMaker mCoffeeMaker = new CoffeeMaker(); @Given("^I previously had (\\d+) coffees$") public void hadCoffeesPreviously(int coffees) { mCoffeeMaker.setCoffeeCount(coffees); } @When("^I add one coffee$") public void addCoffee() { mCoffeeMaker.addCoffee(); }
}
![Page 52: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/52.jpg)
• create the mapping
public class CoffeeMakerDefs { CoffeeMaker mCoffeeMaker = new CoffeeMaker(); @Given("^I previously had (\\d+) coffees$") public void hadCoffeesPreviously(int coffees) { mCoffeeMaker.setCoffeeCount(coffees); } @When("^I add one coffee$") public void addCoffee() { mCoffeeMaker.addCoffee(); } @Then("^I had (\\d+) coffees$") public void hadCoffees(int coffees) { Assert.assertEquals(coffees, mCoffeeMaker.getCoffeeCount()); }}
![Page 53: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/53.jpg)
• place definition and mapping at the same paths! • ${module}/src/test/java/com/example/MyMapping.java
• ${module}/src/test/resources/com/example/MyDefinition.feature
@RunWith(Cucumber.class) public class RunCucumberTest {}
![Page 54: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/54.jpg)
Code Coverage
![Page 55: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/55.jpg)
Code Coverage
• instrumentation tests
• JaCoCo
• EMMA
• obsolete
• unit tests
• JaCoCo
![Page 56: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/56.jpg)
Instrumentation Tests & Code Coverage• has to be explicitly enabled
• gradle createDebugCoverageReport
• ${module}/build/reports/coverage/debug/index.html
• ${module}/build/outputs/code-coverage/connected/${deviceName}-coverage.ec
• doesn’t work on some devices!!!
buildTypes { debug { testCoverageEnabled true }}
![Page 57: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/57.jpg)
JaCoCo
![Page 58: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/58.jpg)
JaCoCo
• enabled by default for unit tests
• gradle test
• generates binary report in build/jacoco
• ${module}/build/jacoco/testDebugUnitTest.exec
• it’s necessary to create a JacocoReport task to obtain a readable report
![Page 59: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/59.jpg)
![Page 60: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/60.jpg)
![Page 61: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/61.jpg)
Good tests
![Page 62: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/62.jpg)
Good tests
• run in any order
• run in isolation
• run consistently
• run fast
• are orthogonal
![Page 63: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/63.jpg)
How to write testable apps?
![Page 64: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/64.jpg)
Rules of thumb• prefer pure Java • abstract away from Android APIs • separate business logic and UI
• don’t write business logic into activities and fragments • MVP, MVVM is a way to go
• try avoid static and final • use dependency injection
![Page 65: Guide to the jungle of testing frameworks](https://reader031.vdocuments.mx/reader031/viewer/2022030303/587c16e41a28abb5068b4761/html5/thumbnails/65.jpg)
Questions?