through a glass, darkly - people.eecs.ku.edu · through a glass, darkly: object-oriented testing...

Through a Glass, Darkly: Object-Oriented Testing

Mark Stockmyer December 1, 2005

1

Slide 1 of 110

Through A Glass, Darkly:Through A Glass, Darkly:Object-Oriented TestingObject-Oriented Testing

●

●

●

●

●

● Mark StockmyerMark Stockmyer● EECS 814 PresentationEECS 814 Presentation

● December 1, 2005December 1, 2005●

●



2

Slide 2 of 110

Personal InfoPersonal Info

● Mark StockmyerMark Stockmyer● B.S. Computer Science – MWSUB.S. Computer Science – MWSU● B.S. Chemistry – MWSUB.S. Chemistry – MWSU● 15 years professional programming 15 years professional programming

experienceexperience● Reuters – Financial market analysis softwareReuters – Financial market analysis software● US Navy – Weapons/Propellant simulationsUS Navy – Weapons/Propellant simulations



3

Slide 3 of 110

SourceSource

● A Practical Guide to Testing Object-Oriented A Practical Guide to Testing Object-Oriented SoftwareSoftware by John D. McGregor and David A. by John D. McGregor and David A. SykesSykes



4

Slide 4 of 110

QuoteQuote

● Never ascribe to malice that which can be Never ascribe to malice that which can be explained by incompetence.explained by incompetence.

Napoleon (attributed)



5

Slide 5 of 110

Road MapRoad Map

● IntroductionIntroduction● Object-Orientation OverviewObject-Orientation Overview● Specifications/TestingSpecifications/Testing● ClassesClasses● InteractionsInteractions● HierarchiesHierarchies● ConclusionConclusion



6

Slide 6 of 110

Why Do I Care?Why Do I Care?

● Quality is importantQuality is important● No one likes fixing bugsNo one likes fixing bugs● Prevent them from occurringPrevent them from occurring● Questions on the final examQuestions on the final exam● PracticalityPracticality



7

Slide 7 of 110

Question #1Question #1

● How many of you write code for a living?How many of you write code for a living?



8

Slide 8 of 110


● If you write code for a living, how many of If you write code for a living, how many of you write Object-Oriented code?you write Object-Oriented code?

● C++?C++?● Java?Java?● Smalltalk?Smalltalk?



9

Slide 9 of 110

Question #3 (Rhetorical)Question #3 (Rhetorical)

● Have you noticed that most of the in-class Have you noticed that most of the in-class examples have been imperative examples have been imperative programming?programming?– Non Object-OrientedNon Object-Oriented

● Does this mean that the techniques we've Does this mean that the techniques we've learned aren't applicable to OO?learned aren't applicable to OO?– No, they're still very applicableNo, they're still very applicable



10

Slide 10 of 110

Practical ('prak-ti-kul) Practical ('prak-ti-kul) adjectiveadjective

● 4a: disposed to action as opposed to 4a: disposed to action as opposed to speculation or abstractionspeculation or abstraction

● Something you can accomplishSomething you can accomplish● Realistic goalsRealistic goals● Everyday usesEveryday uses



11

Slide 11 of 110


● Do you work in a “Practical Development Do you work in a “Practical Development Environment”?Environment”?



12

Slide 12 of 110

So What's an Impractical So What's an Impractical Environment?Environment?

● Space Shuttle on-board software teamSpace Shuttle on-board software team– Must be 100% goodMust be 100% good

– They test everythingThey test everything

– Realistically, fast and cheap are ignoredRealistically, fast and cheap are ignored

– No such thing as “good enough”No such thing as “good enough”

● ““Real” worldReal” world– Don't have that luxuryDon't have that luxury

– Must set prioritiesMust set priorities



13

Slide 13 of 110

3-Way Practicality Process Matrix3-Way Practicality Process Matrix**

● There are three things that can define any There are three things that can define any processprocess– GoodGood

– FastFast

– CheapCheap

*Pick any two



14

Slide 14 of 110

●QuoteQuote

● Everybody in the real world will agree - the Everybody in the real world will agree - the moment a project is behind deadline, quality moment a project is behind deadline, quality assurance tends to go out the window. assurance tends to go out the window.

Alan Cox



15

Slide 15 of 110

Road MapRoad Map

● IntroductionIntroduction

● Object-Orientation OverviewObject-Orientation Overview● Specifications/TestingSpecifications/Testing● ClassesClasses● InteractionsInteractions● HierarchiesHierarchies● ConclusionConclusion



16

Slide 16 of 110

Object-Oriented SystemsObject-Oriented Systems

● ObjectObject● MessageMessage● InterfaceInterface● ClassClass● InheritanceInheritance● PolymorphismPolymorphism



17

Slide 17 of 110

ObjectObject

● Operational entityOperational entity– As opposed to definitional entityAs opposed to definitional entity

● EncapsulatesEncapsulates– Specific data valuesSpecific data values

– Code that manipulates data valuesCode that manipulates data values

● Basic building block of OO systemsBasic building block of OO systems



18

Slide 18 of 110

Object: ExamplesObject: Examples

● CourseCourse● StudentStudent● BillBill● GradeGrade



19

Slide 19 of 110

Object: Testing ImplicationsObject: Testing Implications

● EncapsulationEncapsulation– Build up object and pass it aroundBuild up object and pass it around

– Easy to manipulateEasy to manipulate

● Information hidingInformation hiding– Difficult to change objectDifficult to change object

– Changes in object difficult to observeChanges in object difficult to observe

● Has stateHas state– Can become inconsistentCan become inconsistent

● Has a lifetimeHas a lifetime– Construction/DestructionConstruction/Destruction



20

Slide 20 of 110

MessageMessage

● Request that an operation be performed by Request that an operation be performed by an objectan object

● Allows objects to collaborate to solve Allows objects to collaborate to solve problemsproblems

● Different languagesDifferent languages– C++: member function callC++: member function call

– Java/Smalltalk: method invocationJava/Smalltalk: method invocation



21

Slide 21 of 110

Message: ExamplesMessage: Examples

● Student.AddCourse(Course)Student.AddCourse(Course)● Student.SendBill(Bill)Student.SendBill(Bill)● Student.MakePayment(CreditCard)Student.MakePayment(CreditCard)● Grade.SetGrade(“A”)Grade.SetGrade(“A”)● Student.SetGradeForCourse(Grade,Course)Student.SetGradeForCourse(Grade,Course)● Student.SendGrades()Student.SendGrades()



22

Slide 22 of 110

Message: Testing ImplicationsMessage: Testing Implications

● Has a senderHas a sender– Sender determines when to send messageSender determines when to send message

– May send incorrectlyMay send incorrectly

● Has a receiverHas a receiver– Receiver may not be readyReceiver may not be ready

– Unexpected message may be ignoredUnexpected message may be ignored

● May include parameters/return valueMay include parameters/return value– Used and/or updated by receiverUsed and/or updated by receiver

– Object state is importantObject state is important

– Receiver may throw exceptionReceiver may throw exception



23

Slide 23 of 110

InterfaceInterface

● Aggregation of behavior declarationsAggregation of behavior declarations● Different languagesDifferent languages

– C++: abstract base class with public pure C++: abstract base class with public pure virtual methodsvirtual methods

– Java: interface (defined just like a class)Java: interface (defined just like a class)

● Building block for specificationsBuilding block for specifications



24

Slide 24 of 110

Interface: ExamplesInterface: Examples

● EnrollmentEnrollment– Behavior associated with enrolling in coursesBehavior associated with enrolling in courses

● ResidenceResidence– Behavior associated with living in on-campus Behavior associated with living in on-campus

housinghousing

● BillableBillable– Behavior associated with being able to make Behavior associated with being able to make

paymentspayments● StudentStudent● ParentParent● CompanyCompany



25

Slide 25 of 110

Interface: Testing ImplicationsInterface: Testing Implications

● Possible poor designPossible poor design– Some interfaces may have 'extra' behaviorSome interfaces may have 'extra' behavior

● Interface as parameterInterface as parameter– An object may be passed merely as its An object may be passed merely as its

interfaceinterface

– Unwanted baggage may be carried alongUnwanted baggage may be carried along



26

Slide 26 of 110

ClassClass

● Set of objects (members) that share a Set of objects (members) that share a common conceptual basiscommon conceptual basis

● Class definition Class definition – What the members have in commonWhat the members have in common

– What they will do togetherWhat they will do together

● ObjectsObjects– Elements for executing a programElements for executing a program

● ClassesClasses– Elements for defining a programElements for defining a program

● An Object is an instance of a ClassAn Object is an instance of a Class



27

Slide 27 of 110

Class: SpecificationClass: Specification

● Operation: action applied to an object to Operation: action applied to an object to obtain a certain effectobtain a certain effect

● Two categories of operationsTwo categories of operations– AccessorAccessor

● Returns information about the objectReturns information about the object● Changes nothing in the objectChanges nothing in the object

– ModifierModifier● Changes the state of an objectChanges the state of an object



28

Slide 28 of 110

Preconditions/PostconditionsPreconditions/Postconditions

● PreconditionPrecondition– Condition that must be true before an Condition that must be true before an

operation is attemptedoperation is attempted

● PostconditionPostcondition– Condition that must be true after an operation Condition that must be true after an operation

is completedis completed

● InvariantInvariant– Condition that must be true before Condition that must be true before andand after after

an operationan operation

– Sort of a pre/post combinationSort of a pre/post combination



29

Slide 29 of 110

PreconditionsPreconditions

● State and attributes of the object containing State and attributes of the object containing the operationthe operation

● Attributes of Attributes of parameters in parameters in the messagethe message



30

Slide 30 of 110

PostconditionsPostconditions

● State and attributes of the object containing State and attributes of the object containing the operationthe operation

● Attributes of Attributes of parameters in parameters in the messagethe message● Return value (if any)Return value (if any)● Any exception that might ariseAny exception that might arise



31

Slide 31 of 110

InvariantsInvariants

● Attributes that must be the same before and Attributes that must be the same before and afterafter

● OperationalOperational– Invariants during a single operationInvariants during a single operation

● ClassClass– Invariants for the lifetime of an objectInvariants for the lifetime of an object

– Implies: during ALL operationsImplies: during ALL operations



32

Slide 32 of 110

Road MapRoad Map

● IntroductionIntroduction● Object-Orientation OverviewObject-Orientation Overview

● Specifications/TestingSpecifications/Testing● ClassesClasses● InteractionsInteractions● HierarchiesHierarchies● ConclusionConclusion



33

Slide 33 of 110


● Is OO Testing:Is OO Testing:– Blackbox testing?Blackbox testing?

– Whitebox tesing?Whitebox tesing?

● Why?Why?



34

Slide 34 of 110

Trick Question: BothTrick Question: Both

● WhiteboxWhitebox– Looking at how the code is writtenLooking at how the code is written

– Analyzing code structureAnalyzing code structure● ClassClass● MessageMessage● InheritanceInheritance

● BlackboxBlackbox– How objects behave from the outsideHow objects behave from the outside

● InterfaceInterface● PolymorphismPolymorphism



35

Slide 35 of 110

OO Is More Than the CodeOO Is More Than the Code

● It can also include the entire development It can also include the entire development process*process*– RequirementsRequirements

– Analysis and DesignAnalysis and Design

– ImplementationImplementation

*Please see EECS 816: OO Analysis and Design



36

Slide 36 of 110

The Big Zoom OutThe Big Zoom Out

● Many levels of testingMany levels of testing● Each has 'challenges'Each has 'challenges'

– ClassClass

– Class InteractionsClass Interactions

– Class HierarchiesClass Hierarchies



37

Slide 37 of 110

So, What's a Class Again?So, What's a Class Again?

● Fundamental unit of OO programmingFundamental unit of OO programming● First level of information-hidingFirst level of information-hiding● Key to encapsulationKey to encapsulation● Building block of inheritanceBuilding block of inheritance



38

Slide 38 of 110

How Can You Test a Class?How Can You Test a Class?

● Two ways:Two ways:– Code reviews (static)Code reviews (static)

– Execution of test cases (dynamic)Execution of test cases (dynamic)



39

Slide 39 of 110

Code Reviews (Pro/Con)Code Reviews (Pro/Con)

● ProsPros– Ability to shift with changing specificationsAbility to shift with changing specifications

– Exposes more of the team to the Exposes more of the team to the implementationimplementation

● ConsCons– Humans involved (mistakes)Humans involved (mistakes)

– Time-consuming (especially for regression Time-consuming (especially for regression testing)testing)



40

Slide 40 of 110

Test Case Execution (Pro/Con)Test Case Execution (Pro/Con)

● ProsPros– Ability to shift with changing specificationsAbility to shift with changing specifications

– Exposes more of the team to the Exposes more of the team to the implementationimplementation

– Easily repeatable (Regression testing)Easily repeatable (Regression testing)

● ConsCons– Humans involvedHumans involved

– Difficult to identifyDifficult to identify

– Time-consuming to writeTime-consuming to write● If coverage is good, maybe more time than If coverage is good, maybe more time than

writing the code itselfwriting the code itself



41

Slide 41 of 110

So, Which One's the Best?So, Which One's the Best?

● That depends. . . . That depends. . . . – How often do you need to test?How often do you need to test?

– What development model do you use?What development model do you use?

– Aren't they complementary?Aren't they complementary?

● Yes, they're complimentaryYes, they're complimentary● Technically, a lot of overlapTechnically, a lot of overlap● Better to do both, but may reduce the Better to do both, but may reduce the

practicalitypracticality



42

Slide 42 of 110

Our FocusOur Focus

● Execution-based testingExecution-based testing– Easily repeatableEasily repeatable

– More useful in non-waterfall modelsMore useful in non-waterfall models



43

Slide 43 of 110

Main TroubleMain Trouble

● Identifying test casesIdentifying test cases



44

Slide 44 of 110



45

Slide 45 of 110

Assumption QuoteAssumption Quote

● ““We assume that a class to be tested has a We assume that a class to be tested has a complete and correct specification. . . . . We complete and correct specification. . . . . We assume the specification is expressed in a assume the specification is expressed in a specification language such as the Object specification language such as the Object Constraint Language (OCL) or a natural Constraint Language (OCL) or a natural language and/or as a state transition language and/or as a state transition diagram.”diagram.”

● Does that sound practical to you?Does that sound practical to you?



46

Slide 46 of 110

Complete Specifications?Complete Specifications?

● Not me!Not me!



47

Slide 47 of 110

But, Seriously. . .But, Seriously. . .

● In order to write tests, you need a In order to write tests, you need a specification of some kindspecification of some kind



48

Slide 48 of 110

So, What's a Specification So, What's a Specification Exactly?Exactly?

● Link between real world and model Link between real world and model – Part of the problemPart of the problem

– Part of the solutionPart of the solution

● Description of operationDescription of operation– Class a wholeClass a whole

– OperationsOperations

– Preconditions/postconditionsPreconditions/postconditions

– InvariantsInvariants



49

Slide 49 of 110

What If I Don't Have a Spec?What If I Don't Have a Spec?

● Get one Get one



50

Slide 50 of 110

No, Really, What If I Don't Have a No, Really, What If I Don't Have a Spec?Spec?

● Reverse-engineer from the codeReverse-engineer from the code– And how think the code is supposed to workAnd how think the code is supposed to work

● Test-driven developmentTest-driven development– Write the testsWrite the tests

– Write the codeWrite the code

– RepeatRepeat

– When do you write comments?When do you write comments?



51

Slide 51 of 110

Better To Write As You Go. . .Better To Write As You Go. . .

● Tests are codeTests are code● Consider test code linked with Consider test code linked with

implementationimplementation● Unit testingUnit testing



52

Slide 52 of 110

What is Unit Testing?What is Unit Testing?

● Unit TestUnit Test– procedure used to verify that a particular procedure used to verify that a particular

module of source code is working properlymodule of source code is working properly

● Goal is to catch regression errorsGoal is to catch regression errors● Unit test frameworksUnit test frameworks

– Sunit (Smalltalk)Sunit (Smalltalk)

– Junit (Java)Junit (Java)

– CppUnit (C++)CppUnit (C++)

– NUnit (C#)NUnit (C#)

● PracticalPractical



53

Slide 53 of 110

eXtreme Programming (XP)eXtreme Programming (XP)

● Agile development modelAgile development model● ““Requires” test-first modelRequires” test-first model● Must write the test before the code that will Must write the test before the code that will

be testedbe tested● We're getting off track. . .We're getting off track. . .



54

Slide 54 of 110

Road MapRoad Map

● IntroductionIntroduction● Object-Orientation OverviewObject-Orientation Overview● Specifications/TestingSpecifications/Testing

● ClassesClasses● InteractionsInteractions● HierarchiesHierarchies● ConclusionConclusion



55

Slide 55 of 110

Finding DefectsFinding Defects



56

Slide 56 of 110

Dimensions of Class TestingDimensions of Class Testing

● WhoWho● WhatWhat● WhenWhen● HowHow● How much?How much?



57

Slide 57 of 110

WhoWho

● Usually developers (of the code itself)Usually developers (of the code itself)● Minimizes public knowledge (good)Minimizes public knowledge (good)● Minimizes peer review (bad)Minimizes peer review (bad)



58

Slide 58 of 110

WhatWhat

● Code meets requirementsCode meets requirements– No moreNo more

– No lessNo less



59

Slide 59 of 110

WhenWhen

● NeverNever● MilestonesMilestones● When you have a specificationWhen you have a specification● When changes occurWhen changes occur● AlwaysAlways



60

Slide 60 of 110

HowHow

● Drivers and stubsDrivers and stubs● Unit test frameworksUnit test frameworks● Instance vs. staticInstance vs. static



61

Slide 61 of 110

How Much?How Much?

● Spec vs. implementationSpec vs. implementation● Simple answer: Enough!Simple answer: Enough!● Impractical answer: Everything!Impractical answer: Everything!● Risk-based analysisRisk-based analysis



62

Slide 62 of 110

Back To Not Having a Spec. . .Back To Not Having a Spec. . .

● Writing testable codeWriting testable code● Contract ApproachContract Approach

– Preconditions over postconditionsPreconditions over postconditions

● Defensive ApproachDefensive Approach– Postconditions over preconditionsPostconditions over preconditions

● Problems?Problems?



63

Slide 63 of 110

Contract ApproachContract Approach

● Preconditions are an obligation of the Preconditions are an obligation of the sendersender

● If met, receiver If met, receiver is obligated to meis obligated to meet the et the requirements and postconditionsrequirements and postconditions

● Design implicationsDesign implications– Need sufficient preconditions so receiver can Need sufficient preconditions so receiver can

fulfill postconditionsfulfill postconditions– a sender can determine a sender can determine whether all whether all

preconditions are met before sending a preconditions are met before sending a messagemessage

– May need extra accessor methods to check May need extra accessor methods to check for postcondition successfor postcondition success



64

Slide 64 of 110

Defensive ApproachDefensive Approach

● Interface defined in terms of the receiverInterface defined in terms of the receiver● Operation typically returns a valueOperation typically returns a value

– Return codeReturn code

– Status objectStatus object● Primary goalPrimary goal

– Sanity checkingSanity checking

– Returning status of processingReturning status of processing



65

Slide 65 of 110

Test Writing ImplicationsTest Writing Implications

● Contract approachContract approach– simplifies class testingsimplifies class testing– complicates interaction testingcomplicates interaction testing

● must ensure any sender meets the must ensure any sender meets the preconditionspreconditions

● Defensive approachDefensive approach– complicates class testingcomplicates class testing

● test cases must address all possible test cases must address all possible outcomesoutcomes

– complicates interaction testingcomplicates interaction testing● must ensure all possible outcomes are must ensure all possible outcomes are

produced and that they are properly handled produced and that they are properly handled by the senderby the sender



66

Slide 66 of 110

Road MapRoad Map

● IntroductionIntroduction● Object-Orientation OverviewObject-Orientation Overview● Specifications/TestingSpecifications/Testing● ClassesClasses

● InteractionsInteractions● HierarchiesHierarchies● ConclusionConclusion



67

Slide 67 of 110

Programming TipProgramming Tip

● Testing is for cowardsTesting is for cowards– A brave coder will bypass that stepA brave coder will bypass that step

– If we have full confidence in our coding If we have full confidence in our coding ability, then testing will be unnecessary ability, then testing will be unnecessary

- from How to Write Unmaintainable Code



68

Slide 68 of 110

Class InteractionsClass Interactions

● OO ProgramOO Program– Collection of objects Collection of objects

– Collaborate together to solve a problemCollaborate together to solve a problem

– Ensure that messaging happens correctlyEnsure that messaging happens correctly

● AssumptionsAssumptions– Objects have been individually testedObjects have been individually tested

– Unidirectional messagesUnidirectional messages

– Interactions are sequential (not concurrent)Interactions are sequential (not concurrent)



69

Slide 69 of 110

What's an Interaction?What's an Interaction?

● Object 'A' calls a method 'X' of Object 'B'Object 'A' calls a method 'X' of Object 'B'● Object B's method X executesObject B's method X executes● Reply from method X is returned to AReply from method X is returned to A



70

Slide 70 of 110

Impact of a “Single” InteractionImpact of a “Single” Interaction

● Possible cascade of other object interactionsPossible cascade of other object interactions– State of the receiverState of the receiver– Associated objectsAssociated objects– Possible outcomesPossible outcomes

● ““No change”No change”● Changes in attribute valuesChanges in attribute values● State changesState changes● Creation or deletion of objectsCreation or deletion of objects



71

Slide 71 of 110

Identifying InteractionsIdentifying Interactions

● Two types of classesTwo types of classes● Primitive classesPrimitive classes

– SimpleSimple

– No other classes neededNo other classes needed

– Usually limited in numberUsually limited in number

● Non-primitiveNon-primitive– Support/require other objectsSupport/require other objects

– Most common in OO programmingMost common in OO programming



72

Slide 72 of 110

Two Categories of ClassesTwo Categories of Classes

● CollectionCollection● CollaborationCollaboration



73

Slide 73 of 110

Collection ClassesCollection Classes

● Collection classesCollection classes– Have classes associated with themHave classes associated with them

– Never actually Never actually use use them them

– Store/Create/DeleteStore/Create/Delete

– ExamplesExamples● StackStack● QueueQueue● MapMap● VectorVector



74

Slide 74 of 110

Testing Collection ClassesTesting Collection Classes

● Fairly straightforwardFairly straightforward● Capacity limitationsCapacity limitations● Store/retrieve order correctnessStore/retrieve order correctness● Negative testsNegative tests



75

Slide 75 of 110

Collaboration ClassesCollaboration Classes

● More complex than collection classesMore complex than collection classes● IdentificationIdentification

– Uses other objects in operationsUses other objects in operations

– When, after an operationWhen, after an operation● State is possibly affectedState is possibly affected● An attribute is used or modifiedAn attribute is used or modified

● Handout # 1Handout # 1– Identify the classesIdentify the classes



76

Slide 76 of 110

What To Test?What To Test?

● BestBest– Every collaboration interactionEvery collaboration interaction

– Not reasonableNot reasonable

● BetterBetter– Interactions most likely to have faultsInteractions most likely to have faults

● GoodGood– Probability distributionProbability distribution

● Good enough (Not much choice here)Good enough (Not much choice here)– Less than ideal probability distributionLess than ideal probability distribution



77

Slide 77 of 110

Test Case SamplingTest Case Sampling

– RandomRandom

– Use ProfilingUse Profiling

– Stratified SampleStratified Sample

– OATS (Orthogonal Array Test System)OATS (Orthogonal Array Test System)



78

Slide 78 of 110

Use ProfileUse Profile

● Associate each use of the system with a Associate each use of the system with a frequencyfrequency

● RankedRanked● Probability of useProbability of use● Higher frequency of use ---> More testingHigher frequency of use ---> More testing



79

Slide 79 of 110

●Stratified SampleStratified Sample

● Tests selected from categoriesTests selected from categories● Example: Test subsetsExample: Test subsets

– Select a set of test cases that exercise every Select a set of test cases that exercise every component in the systemcomponent in the system

– Divide those into subsets based on Divide those into subsets based on componentcomponent

● Example: Use Case ActorsExample: Use Case Actors– Rank uses by actor/frequencyRank uses by actor/frequency

– Handout #2Handout #2



80

Slide 80 of 110

Orthogonal Array Test System Orthogonal Array Test System (OATS)(OATS)

● Method to limit explosion of test casesMethod to limit explosion of test cases● Uses pair-wise combination of factorsUses pair-wise combination of factors

– Two-way interactions cause most faultsTwo-way interactions cause most faults

● Sometimes called “pairwise testing”Sometimes called “pairwise testing”● Used in Taguchi MethodsUsed in Taguchi Methods



81

Slide 81 of 110

OATS: Basic TechniqueOATS: Basic Technique

● 1) Determine number of independent 1) Determine number of independent variablesvariables

● 2) Determine number of values it will take 2) Determine number of values it will take (states)(states)

● 3) Find an minimum-pair array (book)3) Find an minimum-pair array (book)● 4) Map factors into array4) Map factors into array● 5) Build test cases5) Build test cases● These steps may vary slightly from user to These steps may vary slightly from user to

useruser



82

Slide 82 of 110

OATS: ExampleOATS: Example

● Consider a web page with three sections Consider a web page with three sections (Top, Middle, and Bottom).(Top, Middle, and Bottom).

● Sections that can be individually shown or Sections that can be individually shown or hidden by the user. hidden by the user.

● You wish to test the interactions of the You wish to test the interactions of the different sections.different sections.



83

Slide 83 of 110


● How many test cases would there be if you How many test cases would there be if you wanted 100% coverage?wanted 100% coverage?



84

Slide 84 of 110

Answer #6Answer #6

● 8 tests, 2^3 = 88 tests, 2^3 = 8



85

Slide 85 of 110

OATS 1-Independent VariablesOATS 1-Independent Variables

● Three independent variables (page sections)Three independent variables (page sections)– TopTop

– MiddleMiddle

– BottomBottom



86

Slide 86 of 110

OATS 2-Variable StatesOATS 2-Variable States

● Each variable can take two statesEach variable can take two states– HiddenHidden

– VisibleVisible



87

Slide 87 of 110

OATS 3-Minimum Pair ArrayOATS 3-Minimum Pair Array



88

Slide 88 of 110

OATS 4-Map Factors Into ArrayOATS 4-Map Factors Into Array



89

Slide 89 of 110

OATS 5-Build Test CasesOATS 5-Build Test Cases

● 1) Hide all sections1) Hide all sections● 2) Display all but the top section2) Display all but the top section● 3) Display all but the middle section3) Display all but the middle section● 4) Display all but the bottom section4) Display all but the bottom section



90

Slide 90 of 110

OATS: OO ProgrammingOATS: OO Programming

● Independent variables are class familiesIndependent variables are class families● Values are members of the class familyValues are members of the class family● See Handout #3See Handout #3



91

Slide 91 of 110

OATS: Adequacy CriteriaOATS: Adequacy Criteria

● Exhaustive Exhaustive – all combinations of all factorsall combinations of all factors

● MinimalMinimal– only interactions between the base classes only interactions between the base classes

● RandomRandom– tester haphazardly selects cases from tester haphazardly selects cases from

several of the classesseveral of the classes● RepresentativeRepresentative

– uniform sample -- every class is tested uniform sample -- every class is tested ● Weighted representativeWeighted representative

– adds cases of relative importance or risk adds cases of relative importance or risk



92

Slide 92 of 110

Road MapRoad Map

● IntroductionIntroduction● Object-Orientation OverviewObject-Orientation Overview● Specifications/TestingSpecifications/Testing● ClassesClasses● InteractionsInteractions

● HierarchiesHierarchies● ConclusionConclusion



93

Slide 93 of 110

Programming TipProgramming Tip

● Never, Ever Do Any Performance TestingNever, Ever Do Any Performance Testing– Hey, if it isn't fast enough, just tell the Hey, if it isn't fast enough, just tell the

customer to buy a faster machine customer to buy a faster machine

- from How to Write Unmaintainable Code



94

Slide 94 of 110

Class Hierarchies (Inheritance) Class Hierarchies (Inheritance)

● Relationship between classesRelationship between classes● Allows new class to be based on existing Allows new class to be based on existing

classclass● Reuse of specification and implementationReuse of specification and implementation● Parent (source) classParent (source) class

– SuperclassSuperclass

– Base classBase class

● New ClassNew Class– SubclassSubclass

– Derived classDerived class



95

Slide 95 of 110

Inheritance (cont.)Inheritance (cont.)

● Inheritance hierarchyInheritance hierarchy● Each class has ancestors (except roots)Each class has ancestors (except roots)● Good OO DesignGood OO Design

– is-a is-a relationshiprelationship

– is-a-kind-ofis-a-kind-of relationship relationship

● Best when used to propagate specificationBest when used to propagate specification– Not implementation (code reuse)Not implementation (code reuse)



96

Slide 96 of 110

Inheritance: Testing ImplicationsInheritance: Testing Implications

● Provides a bug propagation mechanismProvides a bug propagation mechanism– Inherit defects from parentInherit defects from parent

– Good reason to test class hierarchy as it Good reason to test class hierarchy as it developsdevelops

● Allows for reuse of test casesAllows for reuse of test cases– Reuse of spec and implementationReuse of spec and implementation



97

Slide 97 of 110

A Tale of Two ClassesA Tale of Two Classes

● CC– Parent or superclassParent or superclass

● DD– Child or subclassChild or subclass

● Handout #4Handout #4



98

Slide 98 of 110

Is-Kind-Of RelationshipIs-Kind-Of Relationship

● D is-a-kind-of CD is-a-kind-of C● D can be used in place of C in a programD can be used in place of C in a program● Behavior of D must match (or supersede) CBehavior of D must match (or supersede) C● Class BehaviorClass Behavior

– Observable statesObservable states

– Operational semanticsOperational semantics

● Enforced by the Substitution PrincipleEnforced by the Substitution Principle



99

Slide 99 of 110

Substitution PrincipleSubstitution Principle● Only these changes are allowed in a Only these changes are allowed in a

subclasssubclass– PreconditionsPreconditions

● Must be the same or weakerMust be the same or weaker● Less constraining to a clientLess constraining to a client

– PostconditionsPostconditions● Must be the same or strongerMust be the same or stronger● Must expect as much as superclassMust expect as much as superclass

– Class InvariantClass Invariant● Must be the same or strongerMust be the same or stronger● Can add more constraintsCan add more constraints

● Assume Substitution PrincipleAssume Substitution Principle



100

Slide 100 of 110

Refinement Possibilities Refinement Possibilities (Subclass)(Subclass)

● 1) Add new operations/methods to D1) Add new operations/methods to D● 2) Change spec or implementation of and 2) Change spec or implementation of and

operation declared by Coperation declared by C– Change in D an operation declared in CChange in D an operation declared in C

– Override in D a method inherited in COverride in D a method inherited in C

● 3) Add new instance members in D3) Add new instance members in D● 4) Change the class invariant4) Change the class invariant



101

Slide 101 of 110

Hierarchical, Incremental Testing Hierarchical, Incremental Testing (HIT)(HIT)

● Incremental changes from C to D can guide Incremental changes from C to D can guide test case selectiontest case selection

● All of the spec-based tests of C should apply All of the spec-based tests of C should apply to Dto D– If not, substitution rule has been brokenIf not, substitution rule has been broken

● Simple process to determineSimple process to determine– Which C tests are still applicableWhich C tests are still applicable

– What new tests need to be writtenWhat new tests need to be written



102

Slide 102 of 110

1) HIT: Add Operations1) HIT: Add Operations

● New operation implements new functionalityNew operation implements new functionality● New functionality needs to be testedNew functionality needs to be tested● Also need to addAlso need to add

– Class/Object implementation testClass/Object implementation test

– Class Interaction testsClass Interaction tests



103

Slide 103 of 110

2) HIT: Change Spec2) HIT: Change Spec

● a) Change in D the spec for a method a) Change in D the spec for a method declared in Cdeclared in C– Must add new spec-based testsMust add new spec-based tests

– Must test new pre/post conditionsMust test new pre/post conditions

– Must re-run C test casesMust re-run C test cases

● b) Override a C method in D that uses a C b) Override a C method in D that uses a C operationoperation– Can reuse C's spec-based testsCan reuse C's spec-based tests

– Must review all implementation based testsMust review all implementation based tests

– Possibly new coveragePossibly new coverage



104

Slide 104 of 110

HIT: Change OperationsHIT: Change Operations

● New instance variables addedNew instance variables added– Most likely tested by new codeMost likely tested by new code

– Already covered by new tests?Already covered by new tests?

– If a new variable is not used, it doesn't need If a new variable is not used, it doesn't need to be testedto be tested



105

Slide 105 of 110

HIT: Change InvariantHIT: Change Invariant

● New postconditionsNew postconditions● Simply test the new invariant conditionsSimply test the new invariant conditions● Must re-run all inherited test cases to assure Must re-run all inherited test cases to assure

that the invariant holdsthat the invariant holds



106

Slide 106 of 110

ConclusionConclusion

● OO brings new tool to the programmerOO brings new tool to the programmer● New way of programmingNew way of programming● New ways of testingNew ways of testing● Old testing methods still applyOld testing methods still apply



107

Slide 107 of 110

Thoughts?Thoughts?

● How do you test now?How do you test now?● How can you change the testing of code in How can you change the testing of code in

your environment?your environment?● Will these techniques help you?Will these techniques help you?



108

Slide 108 of 110

Questions?Questions?



109

Slide 109 of 110

ReferencesReferences

● A Practical Guide to Testing Object-Oriented A Practical Guide to Testing Object-Oriented SoftwareSoftware by John D. McGregor and David A. by John D. McGregor and David A. Sykes; 2001, Addison-Wesley.Sykes; 2001, Addison-Wesley.

● The Object PrimerThe Object Primer by Scott W. Ambler; by Scott W. Ambler; 2001, Cambridge University Press.2001, Cambridge University Press.

● ””JUnit Primer”: JUnit Primer”: http://www.clarkware.com/articles/JUnitPrimhttp://www.clarkware.com/articles/JUnitPrimer.htmler.html

● ““Unit Testing”: Unit Testing”: http://en.wikipedia.org/wiki/Unit_testinghttp://en.wikipedia.org/wiki/Unit_testing



110

Slide 110 of 110

ReferencesReferences

● ““Orthogonal Array Testing System (OATS)”: Orthogonal Array Testing System (OATS)”: http://www.cvc.uab.es/shared/teach/a20370/http://www.cvc.uab.es/shared/teach/a20370/Temas/TemarioEnCurso/Prova orientada a Temas/TemarioEnCurso/Prova orientada a objecte/orthogonal arrays.pdfobjecte/orthogonal arrays.pdf

through a glass, darkly - people.eecs.ku.edu · through a glass, darkly: object-oriented testing...

Documents