loopt unit test experiences
DESCRIPTION
We all hear how unit tests can ensure higher quality code and help us in day to day refactoring, but is it feasible to write and maintain unit tests in a fast paced startup company?This is a presentation by server lead, Heine Frifeldt, on how unit tests was gradually introduced into the server code base at Loopt, which tools are used in the continuous build environment, coding techniques and lessons learned.TRANSCRIPT
Unit Testing for startups
Experiences from Loopt
Heine Frifeldt <[email protected]>
• Server Team Manager
• Got Introduced to extreme programming in
2000 @ Adomo DK• Have previously tried to have lots of manual tests
• Realized the value of unit tests
• Continued Agile Development in Adomo US
• Joined Loopt in 2008• Code examples are in C#, but topics should apply
regardless of language
• Feel free to ask questions
2
Loopt – What do we do?
3
Connecting You with Friends and Family
Loopt – What do we do?
4
Connecting You with the Places You Go
Loopt – What do we do?
5
Connecting You with Your Local Businesses
Time for Unit Test in Startup?
• Turn one-off tests into automated tests
• Takes extra time upfront but it’s worth it
• Legacy clients likely for mobile companies
• Confident deployments
• Unit tests work as documentation
• Easy way for new employees to get
familiarized with code
6
Overview
• Initial state of Loopt tests
• Untestable code
• Code structure
• Improved code
• Lessons learned
• Tools
7
Initial Experience
• My first UT experiences we wrote the tests
along with code• You structure your code for testability
• Adding tests for existing code can be much
harder
• Believe common startup problem• You get to a point where you realize it would be nice
with unit tests
8
Unit Tests at Loopt
• General support behind Unit Test• Most eng. wanted to add unit tests, but it was hard, so in
practice new tests were rarely added
• VPE had previous good experiences with unit tests and
our deployment did not have good track record
• The existing tests were neglected
• One test project for all projects
• Few tests
• End to end functional tests
• Complex architecture and code that requires mobile to
invoke
9
Code Example (Database Access Layer)
• One of my first tasks was to add a LockedOut
property to our User class
• In theory to test it, Initialize LooptUser, Emulate
failed login N times, Verify field got set
• In practice• Constructor takes phone number (or session, or …) which reads
and sets all relevant properties directly from DB
• Login is a static method in a different class which makes direct
DB calls
• Checks your phone make/model
• Makes external billing checks to certain carriers
10
Code Example (Business Logic)
• Loopt Cell Server takes binary stream
from clients and processes request
• Stream passed down through the flow –
and maybe modified
• Some tests used the raw binary stream to
verify functionality
11
Code Example (Business Logic)
public class DeleteJournal {
private DeleteJournal() {}
public static void ProcessRequest(byte[] content)
{
using (SqlData sdp = new SqlData("sp_Delete_Journal"))
{
byte deltype = content[pos++];
byte num = content[pos++];
entry = BitUtil.ReadInt(content, ref pos);
sdp.AddParameter("@EntryID", SqlDbType.Int, entry);
[…]
12
Code Example (Static Initializers)
• Carrier class has internal constructor
• Get it from carrier factory• Has static initializer which reads carrier settings from a config
file
static CarrierFactory()
{
SmppMap = new Dictionary<string, SmppConnectionElement>();
CarrierSettings configSection =
(CarrierSettings)ConfigurationManager.GetSection("carriers");
foreach (SmppElemtn sce in configSection.SmppConnectionElements)
{
[…]
13
Argh!!
• Ways of testing suggested to me
• LooptUser - Create a test account and use number and
password in tests
• CellServer - Manual test with real phone
• Carrier - Use phone number of wanted carrier
• Manually create test data
• Didn’t know what to change and where to begin
and end
14
Testable Code Talk
• Hosted unit test talks about testable code• Misko Hevery @ Google
• http://misko.hevery.com/2008/11/11/clean-code-talks-
dependency-injection
• Key points• Avoid use of statics (your own and base class libraries)
• Keep constructors simple - they cannot be overridden.
• Separate object graph from logic – remove new operators
• Ask for what you need (dependency injection) and have
DI framework to initialize root objects
• Hollywood Principle
15
Improvements
• Immediate steps (no tools)• Start using dependency injection in new code
• Refactor existing code when changes are required
• Use our own BuildObject initializers
• Use our own mock object implementations for testing
• After ~6 months
• Core code was more nicely structured
• Unit tests and root objects were cluttering up with object
graph initializations
• Starting using Ninject
• After ~2 years
• Got Moq demoed. Just started using that.
16
Example Class using Dependency Injection
public class GrouponAdapter : IGrouponAdapter
{
private readonly IPoiController _poiController;
private readonly DataContextProvider _contextProvider;
private readonly ILooptWebClient _looptWebClient;
[Inject]
public GrouponAdapter(IPoiController poiController,DataContextProvider contextProvider,ILooptWebClient looptWebClient)
{
_poiController = poiController;_contextProvider = contextProvider;_looptWebClient = looptWebClient;
}
…
17
Example Ninject Bindings
public class LooptLogicModule : NinjectModule
{
public override void Load()
{
Bind<IPoiController>().To<PoiController>().InSingletonScope();Bind<ILooptWebClient>().To<LooptWebClient>().InSingletonScope();
…
private IKernel _kernel;
_kernel.Get<IGrouponAdapter>();
18
Example Unit Test using Moq
[TestMethod]
public void ParseGrouponDeals()
{
// Results in 6 deals being returned.
var webClient = new Mock<ILooptWebClient>();
webClient.Setup(c => c.DownloadString(It.IsAny<Uri>())).
Returns(Resources.Groupon_Deals_In_Austin);
IGrouponAdapter ga = new GrouponAdapter(null, webClient.Object);
Deal[] deals = ga.GrouponDeals(new Coordinate(30.44595, -97.79016));
Assert.AreEqual(6, deals.Length, "Expected 6 deals");
}
19
Lessons Learned - Test Barriers
• Hook up Ninject immediately in new projects
• New operators can quickly creep back in
• Add helpers when “impossible” or impractical to
change
• Use IDispose interface for TransientXXXXX test classes
• Users
• Phone numbers
• Carriers
• Config files
• Non-ideal tests are better than no tests
• Well, usually ;-)
20
Lessons Learned – Adding Tests
• No strict test requirements; test can be written up
front or after
• Tests added later are better than no tests
• Add tests when you encounter a bug
• Bookmark/revisit hard to test code and set goal to
write one test
• Restructure / helpers will result in many more tests
• Tools can be introduced gradually
• Introduce DI from top to bottom to avoid cascading
changes
21
Current Tools
• Source Control - Mercurial
• Unit Test Framework – Visual Studio
• Automated Build Environment - Jenkins
• Code style – Dependency Injection
• Dependency Injection Framework - Ninject
• Mock Framework - Moq
22
State of our tests
• 34 Test Projects• 1008 Unit Tests. Executed on commit. Takes ~10 min
• 23 Functional tests. Executed nightly. Takes ~1 min
• Too long execution time
• Flaky tests tend to put us into bad streaks
of several days with failing tests• Due to timing
• Due to functional test nature
23
Jenkins (aka Hudson)
• Nice overview of projects of their state
• Age of failing unit tests
• Execution time of unit tests
24
Next steps
• Continue refactoring existing code to use
DI/Ninject
• Refactor existing tests to be more unit and less
functional test• Faster
• Not flaky
• Better process for dealing with broken tests in
crunch mode• Don’t want to ignore false negatives
• Don’t notice new failures
• Wait for better Ninject integration with ASP.NET,
MVC, WCF, NT Services.
25
We are hiring ($5000 referral bonus)
• System Administrator
• Build Engineer
• Metrics Engineer
• Multiple QA
• Server
• iPhone
• Android
• Web
26