loopt unit test experiences

Unit Testing for startups

Experiences from Loopt

Heine Frifeldt <heine@loopt.com>

• Server Team Manager

• Got Introduced to extreme programming in

2000 @ Adomo DK• Have previously tried to have lots of manual tests

• Realized the value of unit tests

• Continued Agile Development in Adomo US

• Joined Loopt in 2008• Code examples are in C#, but topics should apply

regardless of language

• Feel free to ask questions

2

Loopt – What do we do?

3

Connecting You with Friends and Family

Loopt – What do we do?

4

Connecting You with the Places You Go

Loopt – What do we do?

5

Connecting You with Your Local Businesses

Time for Unit Test in Startup?

• Turn one-off tests into automated tests

• Takes extra time upfront but it’s worth it

• Legacy clients likely for mobile companies

• Confident deployments

• Unit tests work as documentation

• Easy way for new employees to get

familiarized with code

6

Overview

• Initial state of Loopt tests

• Untestable code

• Code structure

• Improved code

• Lessons learned

• Tools

7

Initial Experience

• My first UT experiences we wrote the tests

along with code• You structure your code for testability

• Adding tests for existing code can be much

harder

• Believe common startup problem• You get to a point where you realize it would be nice

with unit tests

8

Unit Tests at Loopt

• General support behind Unit Test• Most eng. wanted to add unit tests, but it was hard, so in

practice new tests were rarely added

• VPE had previous good experiences with unit tests and

our deployment did not have good track record

• The existing tests were neglected

• One test project for all projects

• Few tests

• End to end functional tests

• Complex architecture and code that requires mobile to

invoke

9

Code Example (Database Access Layer)

• One of my first tasks was to add a LockedOut

property to our User class

• In theory to test it, Initialize LooptUser, Emulate

failed login N times, Verify field got set

• In practice• Constructor takes phone number (or session, or …) which reads

and sets all relevant properties directly from DB

• Login is a static method in a different class which makes direct

DB calls

• Checks your phone make/model

• Makes external billing checks to certain carriers

10

Code Example (Business Logic)

• Loopt Cell Server takes binary stream

from clients and processes request

• Stream passed down through the flow –

and maybe modified

• Some tests used the raw binary stream to

verify functionality

11

Code Example (Business Logic)

public class DeleteJournal {

private DeleteJournal() {}

public static void ProcessRequest(byte[] content)

{

using (SqlData sdp = new SqlData("sp_Delete_Journal"))

{

byte deltype = content[pos++];

byte num = content[pos++];

entry = BitUtil.ReadInt(content, ref pos);

sdp.AddParameter("@EntryID", SqlDbType.Int, entry);

[…]

12

Code Example (Static Initializers)

• Carrier class has internal constructor

• Get it from carrier factory• Has static initializer which reads carrier settings from a config

file

static CarrierFactory()

{

SmppMap = new Dictionary<string, SmppConnectionElement>();

CarrierSettings configSection =

(CarrierSettings)ConfigurationManager.GetSection("carriers");

foreach (SmppElemtn sce in configSection.SmppConnectionElements)

{

[…]

13

Argh!!

• Ways of testing suggested to me

• LooptUser - Create a test account and use number and

password in tests

• CellServer - Manual test with real phone

• Carrier - Use phone number of wanted carrier

• Manually create test data

• Didn’t know what to change and where to begin

and end

14

Testable Code Talk

• Hosted unit test talks about testable code• Misko Hevery @ Google

• http://misko.hevery.com/2008/11/11/clean-code-talks-

dependency-injection

• Key points• Avoid use of statics (your own and base class libraries)

• Keep constructors simple - they cannot be overridden.

• Separate object graph from logic – remove new operators

• Ask for what you need (dependency injection) and have

DI framework to initialize root objects

• Hollywood Principle

15

Improvements

• Immediate steps (no tools)• Start using dependency injection in new code

• Refactor existing code when changes are required

• Use our own BuildObject initializers

• Use our own mock object implementations for testing

• After ~6 months

• Core code was more nicely structured

• Unit tests and root objects were cluttering up with object

graph initializations

• Starting using Ninject

• After ~2 years

• Got Moq demoed. Just started using that.

16

Example Class using Dependency Injection

public class GrouponAdapter : IGrouponAdapter

{

private readonly IPoiController _poiController;

private readonly DataContextProvider _contextProvider;

private readonly ILooptWebClient _looptWebClient;

[Inject]

public GrouponAdapter(IPoiController poiController,DataContextProvider contextProvider,ILooptWebClient looptWebClient)

{

_poiController = poiController;_contextProvider = contextProvider;_looptWebClient = looptWebClient;

}

…

17

Example Ninject Bindings

public class LooptLogicModule : NinjectModule

{

public override void Load()

{

Bind<IPoiController>().To<PoiController>().InSingletonScope();Bind<ILooptWebClient>().To<LooptWebClient>().InSingletonScope();

…

private IKernel _kernel;

_kernel.Get<IGrouponAdapter>();

18

Example Unit Test using Moq

[TestMethod]

public void ParseGrouponDeals()

{

// Results in 6 deals being returned.

var webClient = new Mock<ILooptWebClient>();

webClient.Setup(c => c.DownloadString(It.IsAny<Uri>())).

Returns(Resources.Groupon_Deals_In_Austin);

IGrouponAdapter ga = new GrouponAdapter(null, webClient.Object);

Deal[] deals = ga.GrouponDeals(new Coordinate(30.44595, -97.79016));

Assert.AreEqual(6, deals.Length, "Expected 6 deals");

}

19

Lessons Learned - Test Barriers

• Hook up Ninject immediately in new projects

• New operators can quickly creep back in

• Add helpers when “impossible” or impractical to

change

• Use IDispose interface for TransientXXXXX test classes

• Users

• Phone numbers

• Carriers

• Config files

• Non-ideal tests are better than no tests

• Well, usually ;-)

20

Lessons Learned – Adding Tests

• No strict test requirements; test can be written up

front or after

• Tests added later are better than no tests

• Add tests when you encounter a bug

• Bookmark/revisit hard to test code and set goal to

write one test

• Restructure / helpers will result in many more tests

• Tools can be introduced gradually

• Introduce DI from top to bottom to avoid cascading

changes

21

Current Tools

• Source Control - Mercurial

• Unit Test Framework – Visual Studio

• Automated Build Environment - Jenkins

• Code style – Dependency Injection

• Dependency Injection Framework - Ninject

• Mock Framework - Moq

22

State of our tests

• 34 Test Projects• 1008 Unit Tests. Executed on commit. Takes ~10 min

• 23 Functional tests. Executed nightly. Takes ~1 min

• Too long execution time

• Flaky tests tend to put us into bad streaks

of several days with failing tests• Due to timing

• Due to functional test nature

23

Jenkins (aka Hudson)

• Nice overview of projects of their state

• Age of failing unit tests

• Execution time of unit tests

24

Next steps

• Continue refactoring existing code to use

DI/Ninject

• Refactor existing tests to be more unit and less

functional test• Faster

• Not flaky

• Better process for dealing with broken tests in

crunch mode• Don’t want to ignore false negatives

• Don’t notice new failures

• Wait for better Ninject integration with ASP.NET,

MVC, WCF, NT Services.

25

We are hiring ($5000 referral bonus)

• System Administrator

• Build Engineer

• Metrics Engineer

• Multiple QA

• Server

• iPhone

• Android

• Web

26