Lessons Learned in Testability

Scott McMasterGoogle

Kirkland, Washington USAscott.d.mcmaster (at) gmail.com

About Me• Software Design Engineer @ Google.

– Building high-traffic web frontends and services in Java.– AdWords, Google Code

• Ph.D. in Computer Science, U. of Maryland.• Formerly of Amazon.com (2 years), Lockheed Martin (2 years),

Microsoft (7 years), and some small startups.• Frequent adjunct professor @ Seattle University, teaching

software design, architecture, and OO programming.• Author of technical blog at http://www.scottmcmaster365.com.

Testing and Me• Doing automated testing since 1995.• Ph.D. work in test coverage and test suite

maintenance.• Champion of numerous unit, system, and

performance testing tools and techniques.• Co-founder of WebTestingExplorer open-source

automated web testing framework (www.webtestingexplorer.org).

Agenda• What is Testability? • Testability Sins

– Statics and singletons– Mixing Business and Presentation Logic– Breaking the Law of Demeter

• Testability Solutions– Removing singletons.– Asking for Dependencies– Dependency Injection– Mocks and Fakes– Refactoring to UI Patterns

Testability: Formal Definition

• Wikipedia: “the degree to which a

software artifact (i.e. a software

system, software module,

requirements- or design document)

supports testing in a given test

context.”http://en.wikipedia.org/wiki/Software_testability

Some Aspects of Testability

• Controllable: We can put the software in a state to begin testing.

• Observable: We can see things going right (or wrong).• Isolatable: We can test classes/modules/systems

apart from others.• Automatable: We can write or generate automated

tests.– Requires each of the previous three to some degree.

Testability: More Practical Definition

• Testability is a function of your testing goals.• Our primary goal is to write or generate

automated tests.• Therefore, testability is the ease with which we

can write:– Unit tests– System tests– End-to-end tests

Testers and Testability• At Google, test engineers:– Help ensure that developers build testable

software.– Provide guidance to developers on best

practices for unit and end-to-end testing.– May participate in refactoring production

code for testability.

Example: Weather App

Weather App ArchitectureRich Browser UI

(GWT)Frontend Server (GWT

RPC servlet)Remote Web Service (XML-

over-HTTP)User Database

Original Weather App Design

Testability Problems?

• Can’t test without calling the cloud service.– Slow to run, unstable.

• Can’t test any client-side components without loading a browser or browser emulator.– Slow to develop, slow to run, perhaps unstable.

Mission #1: Unit Tests for WeatherServiceImpl

• Problem: Uses static singleton reference to GlobalWeatherService, can’t be tested in isolation.

• Solution:– Eliminate the static singleton.– Pass a mock or stub to the WeatherServiceImpl

constructor at test-time.

WeatherServiceImpl: Beforeprivate static GlobalWeatherService service = new GlobalWeatherService();

public List<String> getCitiesForCountry(String countryName) { try { if (countryName == null || countryName.isEmpty()) { return new ArrayList<String>(); } return service.getCitiesForCountry(countryName); } catch (Exception e) { throw new RuntimeException(e); }}

What if we try to test this in its current form?1. GlobalWeatherService gets loaded at classload-time.

1. This itself could be slow or unstable depending on the implementation.2. When we call getCititesForCountry(“China”), a remote web service call gets made.3. This remote web service call may:

1. Fail.2. Be really slow.3. Not return predictable results. Any of these things can make our test “flaky”.

Proposed Solution• First we need to get rid of the static

singleton.• Then we need something that:– Behaves like GlobalWebService.– Is fast and predictable.– Can be inserted into

WeatherServiceImpl at test-time.

A Word About Static Methods and Singletons

• Never use them!• They are basically global variables (and

we’ve all been taught to avoid those).• They are hard to replace with alternative

implementations, mocks, and stubs/fakes.– They make automated unit testing extremely

difficult.

Scott’s Rules About Static Methods and Singletons

1. Avoid static methods.2. For classes that are logically

“singleton”, make them non-singleton instances and manage them in a dependency injection container (more on this shortly).

Singleton Removalpublic class WeatherServiceImpl extends RemoteServiceServlet implements WeatherService {

private final GlobalWeatherService service;

public WeatherServiceImpl(GlobalWeatherService service) { this.service = service; } ...

• Also, make GlobalWeatherService into an interface.

• Now we can pass in a special implementation for unit testing.

• But we have a big problem…

We’ve Broken Our Service!

• The servlet container does not understand how to create WeatherServiceImpl anymore.– Its constructor takes a funny parameter.

• The solution?

Dependency Injection• Can be a little complicated, but here is what you

need to know here:• Accept your dependencies, don’t ask for them.

– Then your dependencies can be replaced (generally,

with simpler implementations) at test time.

– In production, your dependencies get inserted by a

dependency injection container.• In Java, this is usually Spring or Google Guice.

Dependency Injection with Google Guice

• Google Guice: A Dependency Injection framework.• When properly set up, it will create your objects

and pass them to the appropriate constructors at runtime, freeing you up to do other things with the constructors at test-time.

• Setting up Guice is outside the scope of this talk.– This will get you started:

http://code.google.com/p/google-guice/wiki/Servlets

Fixing WeatherServiceImpl (1)

• Configure our servlet to use Guice and tell it about our objects:

• When someone asks for a “GlobalWeatherService”, Guice will give it an instance of GlobalWeatherServiceImpl.

public class WeatherAppModule extends AbstractModule { @Override protected void configure() { bind(WeatherServiceImpl.class); bind(GlobalWeatherService.class).to(GlobalWeatherServiceImpl.class); }}

Fixing WeatherServiceImpl (2)

• At runtime, Guice will create our servlet and the object(s) it needs:

@Singletonpublic class WeatherServiceImpl extends RemoteServiceServlet implements WeatherService {

private final GlobalWeatherService service;

@Inject public WeatherServiceImpl(GlobalWeatherService service) { this.service = service; } ...

• The “@Inject” constructor parameters is how we ask Guice for instances.

After Testability Refactoring #1

Finally! We Can Test!• But how?• We want to test WeatherServiceImpl in

isolation.For GlobalWeatherService, we only care about

how it interacts with WeatherServiceImpl.To create the proper interactions, a mock

object is ideal

Mock Object Testing• Mock objects simulate real objects in ways specified by the tester.• The mock object framework verifies these interactions occur as

expected.– A useful consequence of this: If appropriate, you can verify that an

application is not making more remote calls than expected.– Another useful consequence: Mocks make it easy to test exception

handling.

• Common mocking frameworks (for Java):– Mockito– EasyMock

• I will use this.

Using Mock Objects1. Create a mock object.2. Set up expectations:

1. How we expect the class-under-test to call it.2. What we want it to return.

3. “Replay” the mock.4. Invoke the class-under-test.5. “Verify” the mock interactions were as-expected.

Testing with a Mock Objectprivate GlobalWeatherService globalWeatherService;

private WeatherServiceImpl weatherService;

@Beforepublic void setUp() { globalWeatherService = EasyMock.createMock(GlobalWeatherService.class); weatherService = new WeatherServiceImpl(globalWeatherService);}

@Testpublic void testGetCitiesForCountry_nonEmpty() throws Exception { EasyMock.expect(globalWeatherService.getCitiesForCountry("china")) .andReturn(ImmutableList.of("beijing", "shanghai")); EasyMock.replay(globalWeatherService); List<String> cities = weatherService.getCitiesForCountry("china"); assertEquals(2, cities.size()); assertTrue(cities.contains("beijing")); assertTrue(cities.contains("shanghai")); EasyMock.verify(globalWeatherService);}

Observe:• How we take advantage of the new WeatherServiceImpl constructor.• How we use the mock GlobalWeatherService.

Mission #2: Unit Tests for GlobalWeatherService

• Problem: Talks to external web service, does non-trivial XML processing that we want to test.

• Solution:– Split the remote call from the XML processing.– Wrap external web service in an object with a known

interface.– Pass an instance to the GlobalWeatherServiceImpl constructor.– Use dependency injection to create the real object at runtime,

use a fake at test-time.

After Testability Refactoring #2

Fakes vs. Mocks• Mocks

– Verifies behavior (expected calls).– Implementation usually generated by a mock object framework.– Often only usable in a single test case.– Often fragile as the implementation changes.

• Fakes– Contains a simplified implementation of the real thing (perhaps using static data, an in-

memory database, etc.).– Implementation usually generated by hand.– Often reusable across test cases and test suites if carefully designed.

• Mocks and Fakes– Either can often be used in a given situation.– But some situations lend themselves to one more than the other.

Use a Fake, or Use a Mock?

• Problem: Setting up a mock object for GlobalWeatherDataAccess that returns static XML is possible, but ugly (and perhaps not very reusable).

• Idea: Create a fake implementation of GlobalWeatherDataAccess.– We can give the fake object the capability to return

different XML in different test circumstances.

Implementing the Fake Objectpublic class FakeGlobalWeatherDataAccess implements GlobalWeatherDataAccess {

// Try http://www.htmlescape.net/javaescape_tool.html to generate these. private static final String CHINA_CITIES = "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<string xmlns=\"http://www.webserviceX.NET\"><NewDataSet>\n <Table>\n <Country>China</Country>\n <City>Beijing</City>\n </Table>\n <Table>\n <Country>China</Country>\n <City>Shanghai</City>\n </Table>\n</NewDataSet></string>"; private static final String BEIJING_WEATHER = "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<string xmlns=\"http://www.webserviceX.NET\">&lt;?xml version=\"1.0\" encoding=\"utf-16\"?&gt;\n&lt;CurrentWeather&gt;\n &lt;Location&gt;Beijing, China (ZBAA) 39-56N 116-17E 55M&lt;/Location&gt;\n &lt;Time&gt;Oct 27, 2012 - 04:00 PM EDT / 2012.10.27 2000 UTC&lt;/Time&gt;\n &lt;Wind&gt; from the N (010 degrees) at 9 MPH (8 KT):0&lt;/Wind&gt;\n &lt;Visibility&gt; greater than 7 mile(s):0&lt;/Visibility&gt;\n &lt;Temperature&gt; 39 F (4 C)&lt;/Temperature&gt;\n &lt;DewPoint&gt; 28 F (-2 C)&lt;/DewPoint&gt;\n &lt;RelativeHumidity&gt; 64%&lt;/RelativeHumidity&gt;\n &lt;Pressure&gt; 30.30 in. Hg (1026 hPa)&lt;/Pressure&gt;\n &lt;Status&gt;Success&lt;/Status&gt;\n&lt;/CurrentWeather&gt;</string>"; private static final String NO_CITIES = "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<string xmlns=\"http://www.webserviceX.NET\"><NewDataSet /></string>";

@Override public String getCitiesForCountryXml(String countryName) throws Exception { if ("china".equals(countryName.toLowerCase())) { return CHINA_CITIES; } return NO_CITIES; }

@Override public String getWeatherForCityXml(String countryName, String cityName) throws Exception { return BEIJING_WEATHER; }}

Testing with a Fake Objectprivate GlobalWeatherServiceImpl globalWeatherService;

private FakeGlobalWeatherDataAccess dataAccess;

@Beforepublic void setUp() { dataAccess = new FakeGlobalWeatherDataAccess(); globalWeatherService = new GlobalWeatherServiceImpl(dataAccess);}

@Testpublic void testGetCitiesForCountry_nonEmpty() throws Exception { List<String> cities = globalWeatherService.getCitiesForCountry("china"); assertEquals(2, cities.size()); assertTrue(cities.contains("beijing")); assertTrue(cities.contains("shanghai"));}

@Testpublic void testGetCitiesForCountry_empty() throws Exception { List<String> cities = globalWeatherService.getCitiesForCountry("nowhere"); assertTrue(cities.isEmpty());}

The fake keeps the tests short, simple, and to-the-point!

Mission #3: Unit Tests for WeatherHome

• Problem: UI and business logic / service calls all mixed together.– The view layer is difficult and slow to instantiate at unit

test-time.– But we need to unit test the business logic.

• Solution:– Refactor to patterns -- Model-View-Presenter (MVP).– Write unit tests for the Presenter using a mock or stub

View.

Mixing Business and Presentation@UiHandler("login")void onLogin(ClickEvent e) { weatherService.getWeatherForUser(userName.getText(), new AsyncCallback<Weather>() {

@Override public void onFailure(Throwable caught) { Window.alert("oops");

}

@Override public void onSuccess(Weather weather) { if (weather != null) { fillWeather(weather); unknownUser.setVisible(false); } else { unknownUser.setVisible(true); } } });}

How NOT to write a UI event handler for maximum testability:• Have tight coupling between the UI event, processing a remote

service call, and updating the UI.

Model-View-Presenter (MVP)

• UI pattern that separates business and presentation logic.

• Makes the View easier to modify.• Makes the business logic easier to

test by isolating it in the Presenter.

Model-View-Presenter Overview

Model-View-Presenter Responsibilities

• Presenter uses the View interface to manipulate the UI.

• View delegates UI event handling back to the Presenter via an event bus or an interface.

• Presenter handles all service calls and reading/updating of the Model.

Passive View MVP• A particular style of MVP where the View is

completely passive, only defining and layout and exposing its widgets for manipulation by the controller.– In practice, you sometimes don’t quite get here, but

this is the goal.

• Especially if you use this style, you can skip testing the View altogether.

After Testability Refactoring #3

Presenter Unit Test Using EasyMock

@Testpublic void testOnLogin_unknownUser() { weatherService.expectGetWeatherForUser("unknown"); EasyMock.expect(weatherView.getUserName()).andReturn("unknown"); weatherView.setUnknownUserVisible(true); EasyMock.expectLastCall(); weatherView.setEventHandler(EasyMock.anyObject(WeatherViewEventHandler.class)); EasyMock.expectLastCall();

EasyMock.replay(weatherView);

WeatherHomePresenter presenter = new WeatherHomePresenter(weatherService, weatherView); presenter.onLogin();

EasyMock.verify(weatherView); weatherService.verify();}

This test uses a manually created mock to make handling the async callback easier.

Question

• Why does the View make the Presenter do this:weatherView.setUnknownUserVisible(true);

• Instead of this:weatherView.getUnknownUser().setVisible(true)

AnswerweatherView.getUnknownUser().setVisible(true)

• Is hard to test because it is hard to mock:– To mock this, we would have to mock

not only the WeatherView, but also the UnknownUser Label inside of it.

• The above code is “talking to strangers”.

Law of Demeter• Also known as the “Don’t talk to strangers” rule.• It says:

– Only have knowledge of closely collaborating objects.– Only make calls on immediate friends.

• Look out for long chained “get”-style calls (and don’t do them:a.getB().getC().getD()

• Your system will be more testable (and maintainable, because you have to rework calling objects less often).

What’s the Point?• To write good unit tests, we need to be able to insert mocks

and fakes into the code.• Some things that help us do that:

– Eliminating static methods and singletons.– Asking for dependencies instead of creating them.– Using design patterns that promote loose coupling, especially

between business and presentation logic.– Obeying the Law of Demeter.

• Code that does not do these things will often have poor test coverage.

What Can I Do?• Developers:

– Follow these practices!

• Testers:– Educate your developers.– Jump into the code and drive testability

improvements.• A good way to motivate this is to track test coverage

metrics.

Questions?Scott McMaster

GoogleKirkland, Washington USA

scott.d.mcmaster (at) gmail.com

Bonus Slides

Model-View-Controller (MVC)

Model-View-Controller (MVC)• View directly accesses the Model and

fires events to the Controller.• Controller performs operations on the

Model.• Controller doesn’t really know about

the View other than selecting the View to render.

Which to Use?

• Many web MVC frameworks exist (Struts, Rails, ASP.NET MVC).

• But these days, we work more with MVP.

Manually Created Mockpublic class MockWeatherServiceAsync implements WeatherServiceAsync {

private List<String> expectGetWeatherForUserCalls = Lists.newArrayList(); private List<String> observeGetWeatherForUserCalls = Lists.newArrayList();

// More @Overrides not shown on the slide.

@Override public void getWeatherForUser(String userName, AsyncCallback<Weather> callback) { observeGetWeatherForUserCalls.add(userName); if ("scott".equals(userName)) { callback.onSuccess(new Weather()); } else { callback.onSuccess(null); } }

public void expectGetWeatherForUser(String userName) { expectGetWeatherForUserCalls.add(userName); }

public void verify() { assertEquals(expectGetWeatherForUserCalls, observeGetWeatherForUserCalls); expectGetWeatherForUserCalls.clear(); observeGetWeatherForUserCalls.clear(); }}