test infected presentation
DESCRIPTION
TTD talk I did with Chris RoffTRANSCRIPT
Test Driven DevelopmentTools and techniques
Talk overview• From idea to requirements, to development, to
deployment Topics • Stories to Acceptance Tests• Unit Testing • Integration testing• Deployment testing
What is testing...Verification • Are we building
the software right? Phases • Unit, Integration,
System, User Acceptance, Deployment
Validation • Are we building
the right software ?
Phases • User Acceptance
Test or ALL?
What is testing...Verification • Are we building the
software right?
Engineering focused
Tools• MbUnit, Watin,
RhinoMocks, FXCop, NDepend, NCover...
Validation • Are we building
the right software ?
Analysis focused
Tools• Fitnesse, Story
Teller, xBehave...
Classic developmentQuality is a side effect of the process not the driver of the process. • Developers, Business Analysts, Testers
throw requirements, code and bug reports over the wall
• No shared quality/testing strategyoDifferent groups use different tools
and techniques • Often manual• Regression tests labour/time intensive
Test Driven DevelopmentThe day to day philosophy Red, Green, GreenRed - Analysis/Design Hat• Create a test that defines the expected outcome• Asking "What I want is..."Green - Creative Hat (Fun part)• Experiment/Prototype/Build until the test passesGreen - Engineering• Refactor, refactor, refactor
Testing Dimensions
Modern TDDVerification and Validation performed continually, during all phases with everyone involved.
• xUnit/CI is only the start• TDD mindset is pushing the boundaries of where
tests can become automated• New focus on down stream testing during
Requirements gathering • Breaking project into Stories promotes detailed
requirements over classical analysis which promotes "sweeping open ended statements"
Requirements
Precise and clear over verbose
We want requirements that • Can be tested!!!
• Are attainable!!!
• Can be easily mapped to User Stories!!!
• Are traceable into code!!!
Acceptance Tests
Using Fitnesse to Verify & ValidateFitNesse enables customers, testers, and programmers to learn what their software should do, and to automatically compare that to what it actually does do. It compares customers' expectations to actual results.
http://fitnesse.org/FitNesse.OneMinuteDescription
FitnesseWhat is it?
• A wiki for writing acceptance tests
• Allows Text, Images and Test case data to be intermingled on the same page
• Promotes shared terminology between analysis and development (Ubiquitous language)
Its almost Christmas...
Midland Organic Foods are thinking about Easter - Jack Plays Snap
• Jack climbs beanstalk when correctly snapping
• Jack wins when he reaches the top of the Beanstalk
• Giant snaps when Jack takes too long
Fitnesse Demo
FitnesseDemo code was horrible...
• No transactions
• Not N-Tiered
• Inadequate error handling
• Prototype...
Fitnesse
As code is productioised
• Fixtures are updated • Wiki pages ARE NOT TOUCHED!!• Fixtures are a bridge between analysis and
implementationoComplex fixture implies translation of
ideasoStrive for Ubiquitious language...oRefactor, refactor, refactor...
Unit Testing
What is Unit Testing
• Testing individual units of code • Take smallest piece of testable
software in the application & isolate it from the remainder of the code
• Simple, Repeatable, Fast
Advantages \ Disadvantages
• Client & Developer confidence
• Refactoring • Self documenting
• Have to write more codeoBut..is this
REALLY a disadvantage?
Tools• Testing frameworks
o MBUnit, NUnit, xUnit...many more • Test runners
o Testdriven.Net, Resharper, Gallio, TeamCity • Mocking
o RhinoMocks, NMock, Typemock • Code Coverage
o NCover
Unit Test Syntax
• Arrange • Act • Assert
Example [Test] public void FindByName_Returns_FarmyardAnimal_When_In_Collection() {
//Arrange string Animal_Name = "test"; FarmYardAnimal testAnimal = new FarmYardAnimal {Name = Animal_Name}; FarmYardAnimals farmYardAnimals = new FarmYardAnimals(); farmYardAnimals.Add(testAnimal);
//Act FarmYardAnimal foundAnimal
= farmYardAnimals.FindByName(Animal_Name); //Assert Assert.AreEqual(testAnimal,foundAnimal); }
Mocking
• Simulating objects that mimic the behavior of real objects in controlled ways.
• Isolate dependancies • Only used in UNIT testing
Mocking cont...• RhinoMocks
o currently version 3.5o written by Ayendeo well maintainedo lots of exampleso can mock
everything on interfaces or virtuals on classes
Mocking cont...• Stubs
o Supply the SUT with data
o No constraints
o Assert on SUT • Mocks
o For interaction testing
o "Record" actions on the mock
o Assert on Mock using constraints
Stubbing - What we’re testing public bool GameFinished
{
get
{
return
(Beanstalk.AtTop ||
(Jack.Cards.Count == 0 && Giant.Cards.Count == 0));
}
}
Example of Stubbing[Test] public void GameFinished_True_When_BeanstalkAtTop_And_Neither_Jack_Or_Giant_HaveCards(){ //Arrange Queue<IPlayingCard> noCards = new Queue<IPlayingCard>(); IBeanstalk stubBeanstalk = MockRepository.GenerateStub<IBeanstalk>(); stubBeanstalk.Expect(sb => sb.AtTop).Return(true);
IPlayer stubJack = MockRepository.GenerateStub<IPlayer>(); IPlayer stubGiant = MockRepository.GenerateStub<IPlayer>(); stubJack.Expect(j => j.Cards).Return(noCards); stubGiant.Expect(j => j.Cards).Return(noCards);
Game gameToTest = new Game(stubJack , stubGiant , stubBeanstalk ); //Act bool isFinished = gameToTest.GameFinished;
//Assert Assert.IsTrue(isFinished);}
Mocking - What we're testing
public void Save(IGame game){ var status = new GameStatus { Giant = game.Giant.Name, Jack = game.Jack.Name, CanSnap = game.CanSnap, IsFinished = game.GameFinished, ActivePlayer = game.ActivePlayer.Name }; try { _session.SaveOrUpdate(status); } catch (Exception ex){ throw new GameRepositoryException("Game Save Error", ex);
}}
Example of Mocking cont..
[Test] public void GameRepository_Should_Convert_GameToGameStatus_And_Use_Session_To_Save(){ //Arrange GameStatus expectedStatus = new GameStatus{ ActivePlayer = "active",CanSnap = true, Giant = "giant",Jack = "jack",IsFinished = false}; //Dont use stubs if creating the object directly is easier IPlayer jack = new Player(expectedStatus.Jack); IPlayer giant = new Player(expectedStatus.Giant); IPlayer activePlayer = new Player(expectedStatus.ActivePlayer);
//Create Game stub IGame stubGame = MockRepository.GenerateStub<IGame>(); stubbedGame.Expect(g => g.Giant).Return(giant); stubbedGame.Expect(g => g.Jack).Return(jack); stubbedGame.Expect(g => g.ActivePlayer).Return(activePlayer); stubbedGame.Expect(g => g.CanSnap).Return(expectedStatus.CanSnap); stubbedGame.Expect(g => g.GameFinished).Return(expectedStatus.IsFinished);
Example of Mocking....cont //create mock ISession mockedSession = MockRepository.GenerateMock<ISession>();
GameRepository repository = new GameRepository(mockedSession); //Act repository.Save(stubGame);
//Assert mockedSession.AssertWasCalled(ms => ms.SaveOrUpdate(null), options => options.Constraints(Property.AllPropertiesMatch(expectedStatus));
}
Constraints• Iso Is.Null(), Is.NotNull(), Is.GreaterThan() etc
• Property oValue(“Status”,Status.Valid),
IsNotNull(“User”), IsNull(“User”) etc • List
o constraints on lists e.g List.Count(2) • Text
o constraints on text e.g Text.StartsWith(“foo”)• Many more...
Exception Path TestingstubbedSession .Expect(s => s.SaveOrUpdate(null)) .IgnoreArguments() .Throw(new Exception());
var gameRepository = new GameRepository(stubbedSession);
//Acttry{ gameRepository.Save(stubbedGame);}catch(GameRepositoryException exception){ //Assert }
State vs Interaction Testing
• Arrange and Act • Verify state of SUT as expected
• Uses only Stubs
• Arrange and Act • Verify SUT has interacted
with dependancy correctly • Can use Stubs & Mocks
MbUnit
• Testing framework • Currently at version 3 • Bundled with Gallio
Row Attribute[Test][Row("name1", "name2")][Row("name1", "name1", ExpectedException = typeof (ArgumentOutOfRangeException))]
public void Can_Only_Add_Animals_With_Unique_Names_To_FarmyardAnimals(string firstAnimalName,string secondAnimalName){//ArrangeFarmYardAnimals animals = new FarmYardAnimals();
//Actanimals.Add(new FarmYardAnimal {Name = firstAnimalName});animals.Add(new FarmYardAnimal {Name = secondAnimalName});}
IntegrationTesting
What is Integration Testing
• Testing components working together • No Mocking
• Long running
• Never sacrifice repeatability for speed
• Large setup, multiple Asserts
Database Integration Testing[Test] public void Using_The_DTC_To_Rollback_Transactions(){using (new TransactionScope()){using (ISession session = _sessionFactory.OpenSession()){ var status = new GameStatus { ActivePlayer = "active", CanSnap = true, Giant = "giant", IsFinished = true, Jack = "jack" };
session.SaveOrUpdate(status);Assert.IsTrue(status.Id > 0);}}
Database Integration Testing cont...[Test, Rollback]public void Using_MbUnit_To_Rollback_Transaction() { using (ISession session = _sessionFactory.OpenSession()){ var status = new GameStatus { ActivePlayer = "active", CanSnap = true, Giant = "giant", IsFinished = true, Jack = "jack" };
session.SaveOrUpdate(status); Assert.IsTrue(status.Id > 0);}
Code Coverage - NCover• Commercial & Free versions available • Bundled as part of Testdriven.Net • Good indication of code requiring tests BUT • High test coverage != Fully tested
NCover cont...
ProductionTesting
Deployment Verification Tests
Automated tests in production
Is the deployment ok?• Verify resources have been installed• Verify software has permission to resources
(file, databases, services, etc...)• Verify that manual software environment is
correct (Windows version etc)
Deployment Verification Tests
• DVT Hippocratic Oatho DO NO DAMAGE!o How much mess can you live with
Log changes?Test data in production?
• Not testing functionality that is done in lower environments
Deployment Verification Tests
Steps• Create a mbUnit project specifically for DVT• Execute test runner console in production• Ensure that the Unit tests load the SAME
configuration files as the SUT• Ensure test runner is executed using the
same Windows credentials as the SUT
Deployment Verification Tests
Create a library of common tests... • DatabaseAssert library
oDatabaseExists();oStoredProcedureExists();
• FileServerAssert libraryoCanOpen();oCanDelete();
Monitoring and Alerting - HeartbeatsContinuous tests in production
• If a server is down, how long does it take to find out?oWithin 5 minutes or when the users
complain• Create a service that can be remotely
executed • Ensure it touches all the systems it depends
upon
Merry ChristmasThanks for listening
• http://www.fitnesse.org• http://www.mbunit.com/• http://ayende.com/projects/rhino-mocks.aspx
Code @ www.pebblesteps.com
Back Story for Jack Plays Snap
Its almost Christmas...
Midland Organic Foods are thinking about Easter • Temporary product for Easter called
Beanstalk Munchies• Organic biscuits coated in chocolate• Biscuits shaped like Farm Yard Animals
based on Jack and Beanstalk fairy tail• Golden Gooses Egg in the middle of the
box
The Pitch - VisionIncrease sales by creating Buzz...
• Client wants to create buzz around product launch
• Want a Web game that they can advertise on TV and in retail stores
• Ideas... Puzzle game or Education game