unit testing

33
Unit testing in Visual Studio 2010 Because it is a matter of quality!!!

Upload: gian-maria-ricci

Post on 01-Nov-2014

10 views

Category:

Technology


1 download

DESCRIPTION

This is a draft of a presentation for a course on Visual Studio 2010 Unit Testing, I've uploaded mainly because I tried to create a Metro Style presentation, so if everyone like it, he can use as base for own presentation.

TRANSCRIPT

Page 1: Unit Testing

Unit testing in Visual Studio 2010

Because it is a matter of quality!!!

Page 2: Unit Testing

Different Type of Tests

Unit Tests

Manual Tests

Web Tests

Coded UI Tests

Automated tests aimed to verify the functionality of a little piece of code, run very often by developers and from automated build.

Test Suites managed with Microsoft Test Manager, manually executed from testers and associated with User Stories. Can be automated with Action RecordingWeb Request record and replay to simulate the interaction with web sites. Can be used to do load or stress test to verify the behavior of software under heavy usageRecord and replay user interaction with UI, plugin-based recorder to maximize reproducibility, generation of UiMap that contains all the data used to reproduce the automation.

Page 3: Unit Testing

Test and Lifecycle - Waterfall

Requirements

Design Develop DeployTest

In waterfall processes testing has a well-defined phase, located between Develop and Deploy. With such a process developers tend to do manual test to verify that requirements are satisfied

Modern waterfall approaches tend to distribute testing in all the phases of the process.

Page 4: Unit Testing

Test and Lifecycle - Waterfall

Test

Requirement are software artifacts that can be tested following basic principle of testingThe purpose of testing a program is to find problems

in it

The purpose of finding problem is to get them fixed(Testing Computer Software – Cem Kaner et Al.)Requirements

Requirements should be verified with stakeholders and users to verify that we are going to create the right software

A mismatch between the program and its specification is an error in the program if and only if the specification exists and is

correct

A program that follows a terrible specification is terrible not perfect

(Testing Computer Software – Cem Kaner et Al.)

Page 5: Unit Testing

Test and Lifecycle - Waterfall

Test

Even with Emerging Architecture you need to validate the design of the software.The design phase should produce a «proof of concept» for each «typical» part of the softwareEarly testing on POC should be used to validate the infrastructureTechnique used: Unit Test, Performance Test

Design

Testing of the Design phase should take place when: No existing architecture, start from scratch First project with new technology Strict performance requirements External service Verify testability of the architecture

Page 6: Unit Testing

Test and Lifecycle - Waterfall

Test

During developement all form of testing are usedUnit test: used by developers to validate new codeManual test: Used by test team to validate code for «done» requirementsCoded UI test: Used by developers and test team to automate testing through the interface

Develop

The main purpose of testing during development is guarantee quality of the codebase.When code is out of the developing phase, it should be ready to be “beta tested” to find bug that are not caught during development.

Page 7: Unit Testing

Test and Lifecycle - WaterfallTesting phase is where you find bug during real-usage of the softwareThe software is usually in «beta» stage, meaning that it can be used by real user.This is the moment where you mainly use Acceptance test with Microsoft Test Manager

Develop

All defect found during this phase should be filled up in issue/bug tracker to be investigated by the teamNo developer should be allowed to «pick a bug» to fix it without a formal review.The aim of this phase is to fix all most important bugs of the software, as well as gather user impression of the overall product, to make little modification before the real shipping.All steps should be reported.

Real-usage Formal bugfix

Quality

Page 8: Unit Testing

Test and Lifecycle - Agile

Each agile process has its own development cycle, but quite all of them have an interactive cycleEach interaction value is added to the software and this value should be immediately available to StakeholdersAcceptance and Unit Testing are the skeleton of this lifecycle, without tests it is difficult to be really Agile. Testing permits to

Establish Definition Of Done (Acceptance) Refactor and do incremental development (Unit testing) Clarify User Stories (Acceptance) Assure internal quality of the codebase (Unit testing in

CI)

Requirements

Design Develop DeployTest

Gian Maria Ricci
Page 9: Unit Testing

Unit Testing

xUnit test pattern and typical techniques of unit testing in

VS2010

Page 10: Unit Testing

Unit test principleAutomated: Test can be executed without user interaction

Develop

Integrated: Written in the same language of the code to test, available in the same environment

Standard: Many frameworks available

DevelopAgile: Support for refactoring and agile lifecycle

DevelopPattern support: www.xunitpattern.org estabilish a ground language and series of pattern for unit testing in all language/environments

Page 11: Unit Testing

xUnit framework pattern

Setup

Setup: Initial conditions are set to maximize reproducibility of the test

Exercise: Code to be tested is executed, usually is a call to a single function or a series of calls.

Verify: An expectation is checked, if the expectation fails, the whole test is considered to be failed

Teardown: If the test modify the system/environment everything is restored to a good / clean state

Exercise

Verify

Teardown

Page 12: Unit Testing

Quality of good testsAutomated: Test can be executed without user interaction

Develop

Indipendent: The test does not depend on any other test

Focused: The test should verify a single condition

DevelopRepetable: Test depends only on precondition contained in the setup phase and it should give the same result at each executionDevelopFast: Slow tests are not executed frequently

Page 13: Unit Testing

Test is First-Class code

Part of the project

Part of the project: testing code has the same importance of tested code

Refactored: Time is spent refactoring and adapting test code to the modifications of tested code.

Time: writing good tests takes time but having good Unit Test suite dramatically raise quality

Long run ROI: Sadly enough, Unit Testing usually does not pay in the short time, so it is difficult to take the path to unit testing

Refactored

Hard to write

Plan for the future

Page 14: Unit Testing

DEMO

Basic of 4-phase test in Visual Studio 2010

Page 15: Unit Testing

Test smells

High maintenance

Symptom: too much time to maintain testsImpact: test are not executed and removed from suiteCause: Code duplication, low quality test code

Symptom: test fails but the code under test is correctImpact: time wasted to look for inexistent bugsCause: Test is too complex

Symptom: test has conditional logic insideImpact: it is not clear what it is testingCause: Dependence from environment, wrong logic

Symptom: it is difficult to understand what is testedImpact: it is difficult to understand why it failedCause: Eager test, mystery guest, irrelevant information, hard coded test data

Buggy tests

Conditional test logic

Obscure test

Page 16: Unit Testing

Avoid test smellsUsers does not execute the test often

Develop

Continuous integration automatically executes tests

Test are slow, they needs several minutes to be executed

DevelopCategorized test permits execution of only a little part of a test, continuous test tools (Mighty Moose, Test Impact) can automate thisDevelopManual intervention before test run

DevelopWrite unit test first, write test that is testable

Page 17: Unit Testing

Fixture

The most difficult part to create Unit Tests without smells is fixture

management

Page 18: Unit Testing

The fixture

Setup Exercise

Verify Teardown

Everything is needed to create preconditions for the execution of test, it can be data in database, setting global variables, cleaning up file on disks, etc.

This is the most complex part of the whole Unit-Testing stuff, because it is the source of many smell

Fresh Fixture – transient: The fixture is recreated at each test, at the end of the test the system is restored to the original stateFresh Fixture – persistent: Same as transient, but the system is not restored at the end of the testShared Fixture: The fixture is shared for many test, it is created, then a series of test is executed

Page 19: Unit Testing

Fresh transientFixture is created during the setup phase, take track of every modification that is done to the system.It is useful to create a mechanism that permits to schedule automatic cleanup at the end of the test.

Setup

Exercise

Verify

TeardownFixture is removed during the teardown phase, everything is restored. Garbage collector should not be used because it is not-deterministic.If multiple restore operations should be executed, use try-catch clause to be sure to cleanup the most of them

Page 20: Unit Testing

Fresh persistentFixture is created during the setup phase, no need to track operationsSetup

Exercise

Verify

Teardown

Setup

Exercise

Verify

Teardown

Next test will create another fixure, but every modification done by the previous test is still thereThis can augment the risk of interacting-test smell.This technique is useful only if we are sure that the fresh persistent fixture will not impact other tests.It is useful mainly if the restore operation is slow and you do not want to lose time.

Page 21: Unit Testing

Shared fixtureFixture is created during the setup phase, modification are trackedSetup

Exercise

Verify SetupExercise

Verify

Teardown

After the latest verification the fixture is removed.

SetupExercise

Verify

Teardown

SetupExercise

Verify

Subsequent tests execution will use the same fixture

First test is executed, the test is composed only by Exercise and Verify steps

Page 22: Unit Testing

Shared fixtureFixture Setup Setup

Exercise

Verify

Teardown

Setup

Exercise

Verify

Teardown

Setup

Exercise

Verify

Teardown Teardown

A shared fixture is created before the execution of each testEach test can create another specific fixture and remove it in Setup/TeardownAfter the latest test of the suite is executed the final Teardown is run to remove the shared fixture.

Page 23: Unit Testing

Assertion

How to write good verification code that maximize the

advantage of unit testing

Page 24: Unit Testing

AssertionSingle assertion: A test should verify only one fact of the code under test

Develop

Clear assertion: Reading verification code should immediately clarify what fact we are testing

Test named assertion: Name of the test should clarify the purpose and what is verified by the test.

Page 25: Unit Testing

Writing good assertionsToo many assertions in the test

Develop

Expected Object, create an object that reflect the status you want to check, name that object in clear way.

Complex assertion code, that is not readable.

DevelopUse FluentInterface based assertion frameworks (SharpTestEx)

DevelopMagic values to verify return values or object properties

DevelopUse good names for variables and expected values

Page 26: Unit Testing

Test doubles

Creates fake component to be sure to exercise only a single part

of the system

Page 27: Unit Testing

Depend On ComponentThe System Under Test dialogates with external component and the verification phase has not access to it.

Setup

Exercise

Teardown

VerifySUT DOC

If the test fails, we are not sure if the reason is in bug in the SUT or for a bug or different behavior of the DOC.It leads to erratic test and frequent debugging test smell.

Page 28: Unit Testing

Test Double

We need to design the SUT to loose coupling with the DOC, with this technique you can create a double of the DOC in the test.The Test Double simulates the real component, but it has a well defined and reproducible behavior that is defined in the Setup phase

Setup

Exercise

Teardown

VerifySUT

Test Double

Page 29: Unit Testing

Verifiable Test Double

With a verifiable Test Double the Verify step can ask to Test Double to verify how the SUT interacted with him during Exercise phase.As an example, if you create a Test Double of an E-Mail sender component, you can verify that during Exercise the SUT asked to send an E-Mail.

Setup

Exercise

Teardown

VerifySUT

Test Double

Page 30: Unit Testing

Types of TestDoubles

Stub

Stub: its only purpose is to answer to the requests of the SUT with well defined and known answers, or with a default one

Dummy: It does nothing, simply avoid the SUT to crash for missing components.

Mock and Spy: Behave like a Stub, but also permits to make assertion on how the SUT interacted with them.

Fake Object: It implements a DOC with minimal functionality.

Dummy

Mock Object and Test Spy

Fake Object

Page 31: Unit Testing

Database Testing

Testing a database is complex and needs specific techniques

Page 32: Unit Testing

Database Testing Basic

Database Sandbox

Sandbox: Each tester/environment should have a dedicated database to execute testing. Minimize dependency, maximize execution time avoid conflicts

Database Fixture: Shared fixture to minimize setup time, use of Back Door Manipulation and backup/restore attach/detach method. Heavy use of Shared Fixture.Smart teardown: Use preload at each test and make every test transactional so you can rollback after each verification phase

Database Projects: Specific project type introduced with VS2008 that contains a dedicated section to unit testing.

Fixture Setup

Teardown

Database Project

Page 33: Unit Testing

……