table-driven acceptance testing

Table-Driven Table-Driven Acceptance TestingAcceptance Testing

Mario AquinoMario Aquino

Principal Software EngineerPrincipal Software Engineer

Object Computing, Inc.Object Computing, Inc.

[email protected]@ociweb.com

Table-Driven Acceptance Testing 2

Session Goals…Session Goals…

Define the concept of Acceptance Testing Define the concept of Acceptance Testing (AT)(AT)

Describe its role in agile software Describe its role in agile software developmentdevelopment

Discuss the toolkits that facilitate testingDiscuss the toolkits that facilitate testing Demonstrate the writing and execution of a Demonstrate the writing and execution of a

table-driven acceptance testtable-driven acceptance test


Test Types & FocusTest Types & Focus

Unit testsUnit tests Exercise the public contract of single unit of code, Exercise the public contract of single unit of code,

isolated from the larger systemisolated from the larger system Integration testsIntegration tests

Interact with a working system at some levelInteract with a working system at some level Utilize “public” interfaces into a systemUtilize “public” interfaces into a system Confirm integration between components in a Confirm integration between components in a

working systemworking system


Test Types & Focus (cont.)Test Types & Focus (cont.)

Acceptance TestsAcceptance Tests Demonstrate the implementation of customer Demonstrate the implementation of customer

defined system requirementsdefined system requirements Confirm that domain/business rules are enforced Confirm that domain/business rules are enforced

by the system as defined by the customerby the system as defined by the customer Utilize business language to represent domain Utilize business language to represent domain

conceptsconcepts


Fostering CollaborationFostering Collaboration

The process of writing ATs has goalsThe process of writing ATs has goals Q & A between the customer and developerQ & A between the customer and developer Transfer of understandingTransfer of understanding Decision makingDecision making Examples to satisfy requirements capturedExamples to satisfy requirements captured Production of the executable contract to which all Production of the executable contract to which all

are accountableare accountable


Objective Gauges of ProgressObjective Gauges of Progress

Features are complete when their ATs passFeatures are complete when their ATs pass Percentage completion estimates become Percentage completion estimates become

irrelevantirrelevant Add ATs to continuous integration process & Add ATs to continuous integration process &

publish the resultspublish the results


Level Playing FieldLevel Playing Field

ATs should be in a mutually accessible ATs should be in a mutually accessible formatformat Easy for customers to write, change, extendEasy for customers to write, change, extend Easy for developers to follow and executeEasy for developers to follow and execute


FitFit

Framework for Integrated Tests (Fit)Framework for Integrated Tests (Fit) http://fit.c2.comhttp://fit.c2.com Originally developed by Ward CunninghamOriginally developed by Ward Cunningham

Uses a table-based approachUses a table-based approach Test details appear in several varieties of tablesTest details appear in several varieties of tables Tests can be written in HTML filesTests can be written in HTML files


How It Works…How It Works…HTML

Test file

FitTest Runner

referencesreferences Fixtures(compiled

source files)

HTML TestReport

parsesparses executesexecutes

generatesgenerates


Framework FixturesFramework Fixtures

fit.ColumnFixturefit.ColumnFixture Calculation-style testingCalculation-style testing

fit.RowFixturefit.RowFixture Equality comparison for groups of valuesEquality comparison for groups of values

fit.ActionFixturefit.ActionFixture Workflow-style testingWorkflow-style testing


““Calculation” TestingCalculation” Testing

Business rules that depend on multiple Business rules that depend on multiple conditionsconditions Varying inputs trigger different business rulesVarying inputs trigger different business rules

Appointments in a schedule must haveAppointments in a schedule must havestart dates/times occurring before thestart dates/times occurring before the

end dates/timesend dates/times

Overlapping appointments are Overlapping appointments are not allowed for some eventsnot allowed for some events

Certain events are only allowedCertain events are only allowedbetween specific hours of the daybetween specific hours of the day

ColumnFixtureColumnFixture


Groups of ValuesGroups of Values

Testing equality for collections of objectsTesting equality for collections of objects Useful for checking values before/after Useful for checking values before/after

something is done in a testsomething is done in a test Simple or complex objects can be usedSimple or complex objects can be used Can be domain objects or utility typesCan be domain objects or utility types Evaluation is order independentEvaluation is order independent

RowFixtureRowFixture


Sequences of InteractionsSequences of Interactions

Application accessible via “device” metaphorApplication accessible via “device” metaphor Devices responds to 4 keywords:Devices responds to 4 keywords:

startstartBegins use of fixtureBegins use of fixture

enterenterAdds data to system deviceAdds data to system device

presspressIssues command to deviceIssues command to device

check*check*Reads information from deviceReads information from device

* triggers comparison* triggers comparisonActionFixtureActionFixture


Using Domain TypesUsing Domain Types

ATs should express concepts using business ATs should express concepts using business languagelanguage

Domain types can be used directlyDomain types can be used directly Fixtures translate strings in test tables to Fixtures translate strings in test tables to

concrete domain typesconcrete domain types Parsing delegates can be registered as wellParsing delegates can be registered as well


FitLibraryFitLibrary

Adds new fixtures, utilities and keywords to Adds new fixtures, utilities and keywords to Fit Fit DoFixtureDoFixture – “flow” style testing – “flow” style testing SetUpFixtureSetUpFixture – convenient test setup – convenient test setup FolderRunnerFolderRunner – batch test execution – batch test execution setUp() & tearDown()setUp() & tearDown() SetUp & TearDownSetUp & TearDown Keywords: Keywords: rejectreject, , not not (and others)(and others) Tests can be written in MS Excel spreadsheetsTests can be written in MS Excel spreadsheets


““Flow”-style TestingFlow”-style Testing

A single fixture (DoFixture) organizes the testA single fixture (DoFixture) organizes the test Testing conditions setupTesting conditions setup Supporting fixtures share common resourcesSupporting fixtures share common resources

Parsing delegates for domain typesParsing delegates for domain types References to the system under testReferences to the system under test


““Flow”-style Testing (cont.)Flow”-style Testing (cont.)

MyGrandUnifyingFixtureMyGrandUnifyingFixture

AddMoviesAddMovies AddCustomersAddCustomers

CalculateLateFeesCalculateLateFees

ViewActiveRentalsViewActiveRentals

CheckOutMoviesCheckOutMovies

Subclasses fitlibrary.SetUpFixtureSubclasses fitlibrary.SetUpFixture

Subclasses Subclasses fitlibrary.DoFixturefitlibrary.DoFixture

fitlibrary.DoFixturefitlibrary.DoFixture

fit.RowFixturefit.RowFixture

fit.ColumnFixturefit.ColumnFixture


SetUpFixtureSetUpFixture


DoFixtureDoFixture


RowFixtureRowFixture


ColumnFixtureColumnFixture


Batch Test ExecutionBatch Test Execution

fitlibrary.runner.FolderRunnerfitlibrary.runner.FolderRunner Specify folder where tests are locatedSpecify folder where tests are located Specify output folderSpecify output folder

Prints a “pretty” reportPrints a “pretty” report


KeywordsKeywords

Fit supplies the Fit supplies the errorerror keyword keyword Use when an exception is expected (only in Use when an exception is expected (only in

ColumnFixture and ActionFixture)ColumnFixture and ActionFixture) FitLibrary adds several more for DoFixtureFitLibrary adds several more for DoFixture

rejectreject & & notnot Behave identicallyBehave identically Cause cell evaluation to be reversedCause cell evaluation to be reversed

ensureensure Behaves the opposite of Behaves the opposite of rejectreject & & notnot


The big demo…The big demo…


Patterns and PracticesPatterns and Practices

Testing the system via APITesting the system via API True domain objects interactionTrue domain objects interaction Unit-style testing of the whole systemUnit-style testing of the whole system

Testing the system via UITesting the system via UI Test how the real users wouldTest how the real users would Need UI driverNeed UI driver


But What About???But What About???

FitNesseFitNesse Fit/FitLibrary testing via a wiki-serverFit/FitLibrary testing via a wiki-server Adds suites similar to JUnit tests suitesAdds suites similar to JUnit tests suites Other featuresOther features

Variable substitution / Security / SetUp & TearDownVariable substitution / Security / SetUp & TearDown

DebuggingDebugging FitLibrary steals System.out & System.error FitLibrary steals System.out & System.error Use log file output (log4j)Use log file output (log4j) Launch tests with JPDA activeLaunch tests with JPDA active


Questions???Questions???


ReferencesReferences

Fit Website: Fit Website: http://fit.c2.comhttp://fit.c2.com FitLibrary: FitLibrary: http://fitlibrary.sourceforge.net/http://fitlibrary.sourceforge.net/ FitNesse: FitNesse: http://fitnesse.orghttp://fitnesse.org Fit article:Fit article:

http://www.ociweb.com/jnb/jnbDec2005.htmlhttp://www.ociweb.com/jnb/jnbDec2005.html

table-driven acceptance testing

Documents