Ivan MarsicRutgers UniversityLECTURE 7: Software Testing

*TopicsOverview of Software TestingTest Coverage and Code CoveragePractical Aspects of Unit TestingIntegration and System Testing

*Overview of Software TestingTest chng t s c mt ca li, khng chng t rng khng c li. Edsger W. DijkstraMt li (fault, defect, bug,) l mt phn t phn cng hoc phn mm c sai st m c th gy s c h thngTest-Driven Development (TDD)Mi bc trong qu trnh pht trin phi bt u bng mt k hoch v cch kim tra xem kt qu t c mc tiu hay chaDeveloper khng nn to ra xut phm no (a system requirement, a UML diagram, or source code) m khng bit cch test nMt test case (ca kim th) l mt la chn c th v input c dng khi kim th mt chng trnhMt test l mt tp hu hn cc test case

*Why Testing is HardVn quan trng cn cn nhc:Test c nhiu trng hp cng tt, nhng chi ph ch c hnMc tiu l tm li c cng nhanh cng tt v cng r cng tt.L tng: vi mi mt li, ta ch cn thit k mt test case pht hin li ri chy nThc t, ta phi chy nhiu test case tht bi - chng khng pht hin c li

*Logical Organization of TestingUnit testUnit testUnit testIntegration testComponent codeComponent codeComponent codeTested componentIntegrated modulesFunction testQuality testAcceptance testInstallation testSystem testSystem in useEnsure that each component works as specifiedEnsures that all components work togetherVerifies that functional requirements are satisfiedVerifies non-functional requirementsCustomer verifies all requirementsTesting in user environment( Not necessarily how its actually done! )

Cc loi test*Unit test: test ring r c lp tng component (phng thc, lp, mt vi lp nh). Khng ng n c s d liu, a cngIntegration test kim th tch hp: test vic tch hp cc component vi nhau xem chng c lm vic c vi nhau khngSystem test kim th h thng: test ton b h thng xem c tha mn cc yu cu chc nng v phi chc nngAcceptance test kim th chp nhn: Khch hng test ton b h thng (thit k ti pha phn tch yu cu)Regression test kim th hi quy: Khi h thng phi sa do thm hay thay i yu cu, ton b cc chc nng c phi c test li m bo phn sa i mi khng nh hng n phn c.

*Acceptance Tests - Examples[ Recall Section 2.2: Requirements Engineering ]Test with the valid key of a current tenant on his/her apartment (pass)Test with the valid key of a current tenant on someone elses apartment (fail)Test with an invalid key on any apartment (fail)Test with the key of a removed tenant on his/her previous apartment (fail)Test with the valid key of a just-added tenant on his/ her apartment (pass)Input dataExpected result

*Example: Test Case for Use Case*[ Recall Section 2.3.3: Detailed Use Case Specification ]

Test-case Identifier:TC-1Use Case Tested:UC-1, main success scenario, and UC-7Pass/fail Criteria:The test passes if the user enters a key that is contained in the database, with less than a maximum allowed number of unsuccessful attemptsInput Data:Numeric keycode, door identifierTest Procedure:Expected Result:Step 1. Type in an incorrect keycode and a valid door identifierSystem beeps to indicate failure; records unsuccessful attempt in the database; prompts the user to try againStep 2. Type in the correct keycode and door identifierSystem flashes a green light to indicate success; records successful access in the database; disarms the lock device

Cc kiu thit k test*Black box testing kim th hp en: phn tch mt chng trnh bng cch th cc input khc nhau, khng quan tm ci t nh th no. Thng dng cho acceptance testWhite box testing kim th hp trng: chn d liu test vi kin thc v ci t (kin trc, thut ton, m chng trnh). Thng dng cho unit test.

*Test CoverageTest coverage o xem bao nhiu phn ca c t hoc code c kim tra bi cc testCode coverage o xem bao nhiu phn ca code c kim tra bi testCc tiu ch o code coverage:equivalence testingboundary testingcontrol-flow testingstate-based testing

*Code Coverage: Equivalence TestingEquivalence testing l mt phng php kim th hp en, trong khng gian ca tt c input c chia thnh cc nhm tng ng sao cho chng trnh hot ng ging nhau i vi input trong mi nhmHai bc: Phn hoch khng gian input thnh nhng nhm/lp tng ngChn cc gi tr input cho testEquivalence classes:

*Heuristics forFinding Equivalence ClassesFor an input parameter specified over a range of values, partition the value space into one valid and two invalid equivalence classesFor an input parameter specified with a single value, partition the value space into one valid and two invalid equivalence classesFor an input parameter specified with a set of values, partition the value space into one valid and one invalid equivalence classFor an input parameter specified as a Boolean value, partition the value space into one valid and one invalid equivalence class

*Code Coverage: Boundary TestingBoundary testing l trng hp c bit ca equivalence testing, nhng tp trung vo cc gi tr bin ca cc inputLp trnh vin thng qun/b qua cc trng hp c bit nm ti bin ca cc lp tng ngChn cc phn t input nm ti bin ca lp tng ng, chng hnzero, min/max, empty set, empty string, nullNhm ln gia > v >=...

*Code Coverage: Control-flow TestingStatement coverage: mi lnh c chy t nht mt ln bi mt test case no Edge coverage: mi nhnh (branch) ca control flow c chy qua t nht mt ln bi mt test case no .Condition coverage: mi iu kin nhn cc gi tr TRUE v FALSE ti t nht mt test case no Path coverage: mi path (ng chy) qua chng trnh u c i qua t nht mt ln bi test case no control graph of a program for Edge Coverage:

*Code Coverage: State-based TestingState-based testing xc nh mt tp cc trng thi tru tng m mt unit c th c v test hnh vi ca unit bng cch so snh trng thi thc vi trng thi trng iThng dng vi cc h thng hng i tngTrng thi (state) ca mt i tng c nh ngha bi mt iu kin xc nh trn gi tr ca cc thuc tnh i tngDo cc phng thc tnh ton hnh vi ca i tng da trn cc thuc tnh, nn hnh vi ph thuc trng thi i tng

*State-based Testing Example

*State-based Testing Examplestateeventguard conditionactiontransition

Cc phn t trong State Diagram ca Controller04 trng thi - state { Locked, Unlocked, Accepting, Blocked }02 s kin - event { valid-key, invalid-key }05 transition hp l { LockedUnlocked, LockedAccepting, AcceptingAccepting, AcceptingUnlocked, AcceptingBlocked }*

m bo State CoverageTest ph mi trng thi xc nh t nht 01 ln (mi trng thi nm trong t nht 01 test case) Ph mi transition hp l t nht 01 lnKch hot mi transition khng hp l t nht 01 ln*

*Quy trnh testArrange / setup: To th cn test v mi trng testAct: Kch hot thao tc cn test ca fixtureAssert / verify: Kim tra xem trng thi thc (actual state) c bng trng thi trng i (expected state) hay khng

*Cc kha cnh thc tin ca Unit TestingUnit cn test c gi l fixture, hay SUT (system under test)Mock objects:Mt test driver gi lp phn ca h thng s kch hot hot ng ca SUTMt (vi) test stub gi lp cc component m SUT s giQuy trnh test:To SUT, test driver, v cc stubCho test driver kch hot mt thao tc ca SUTKim tra xem trng thi thc (actual state) c bng trng thi trng i (expected state) hay khng

*Testing the KeyChecker (Unlock Use Case)(a)(b)

*xUnit / JUnitCc phng thc dng assert_*_() thng c dng kim tra trng thi. Chng nh ngha trng thi trng i v bo li nu trng thi thc khng ging nh vyhttp://www.junit.org/V d:assertTrue(4 == (2 * 2));assertEquals(expected, actual);assertNull(Object object);v.v..

Example Test Case*

Listing 2-1: Example test case for the Key Checker class.public class CheckerTest { // test case to check that invalid key is rejected @Test public void checkKey_anyState_invalidKeyRejected() { // 1. set up Checker checker = new Checker( /* constructor params */ ); // 2. act Key invalidTestKey = new Key( /*setup with invalid code*/ ); boolean result = checker.checkKey(invalidTestKey); // 3. verify assertEqual(result, false); }}

t tn phng thc cho test case*V d tn phng thc cho test case vi ni dung: kim tra hm checkKey() lun lun reject kha khng hp l

checkKey_anyState_invalidKeyRejected()

Another Test Case Example*

Listing 2-2: Example test case for the Controller class.public class ControllerTest { // test case to check that the state Blocked is visited @Test public void enterKey_accepting_toBlocked() { // 1. set up: bring the object to the starting state Controller cntrl = new Controller( /* constructor params */ ); // bring Controller to the Accepting state, just before it blocks Key invalidTestKey = new Key( /* setup with invalid code */ ); for (i=0; i < cntrl.getMaxNumOfAttempts(); i++) { cntrl.enterKey(invalidTestKey); } assertEqual( // check that the starting state is set up cntrl.getNumOfAttempts(), cntrl.getMaxNumOfAttempts() 1 ); // 2. act cntrl.enterKey(invalidTestKey); // 3. verify assertEqual( // the resulting state must be "Blocked" cntrl.getNumOfAttempts(), cntrl.getMaxNumOfAttempts() ); assertEqual(cntrl.isBlocked(), true); }}

*Integration TestingHorizontal Integration TestingBig bang integrationBottom-up integrationTop-down integrationSandwich integrationVertical Integration Testing

*Horizontal Integration TestingSystem hierarchy:Bottom-up integration testing:Top-down integration testing:

*Horizontal Integration TestingSystem hierarchy:

*Horizontal Integration TestingBottom-up integration testing:

*Horizontal Integration TestingTop-down integration testing:

*Vertical Integration TestingPht trin cc user story:Mi story c pht trin trong mt chu trnh tch hp cc unit test ti vng feedback trong v acceptance test ti vng feedback ngoiTon b h thnginner feedback loopouter feedback loop

*Logical Organization of TestingUnit testUnit testUnit testIntegration testComponent codeComponent codeComponent codeTested componentIntegrated modulesFunction testQuality testAcceptance testInstallation testSystem testSystem in useEnsure that each component works as specifiedEnsures that all components work togetherVerifies that functional requirements are satisfiedVerifies non-functional requirementsCustomer verifies all requirementsTesting in user environment( Not necessarily how its actually done! )

***************