introduction to programming ibbm101/fall18/... · lecture overview •debugging •exception...

Lecture#08– TestingandDebugging

AykutErdem,Fuat Akal&AydınKaya//Fall2018

BBM101IntroductiontoProgrammingI

TheAdventuresofSherlockHolmes(1939)

Lasttime… Sorting,ListComprehension,Visualization

Datavisualization

Sortingprint("hamlet:", hamlet)

print("sorted(hamlet):", sorted(hamlet))print("hamlet:", hamlet)

print("hamlet.sort():", hamlet.sort())print("hamlet:", hamlet)

M = MonthAverages('Ithaca') bar(range(12),M,facecolor='magenta') xlim(-.2,12)ylabel('Average Hours of Sunlight') title(A.City,fontsize=16)show()

2

[i*2 for i in range(3)]

Listcomprehension

LectureOverview• Debugging• ExceptionHandling• Testing

3

Disclaimer:Muchofthematerialandslidesforthislecturewereborrowedfrom—R.Anderson,M.ErnstandB.HoweinUniversityofWashingtonCSE140


4https://www.reddit.com/r/ProgrammerHumor/comments/1r0cw7/the_5_stages_of_debugging/

TheProblem

Whatyouwantyourprogramtodo Whatyourprogramdoes

Notthesame!

5

Thereisabug!

WhatisDebugging?• GraceHopperwasoneofU.S.’sfirstprogrammers.• ShefoundamothintheMarkIcomputer,whichwascausingerrors,andcalleditacomputer“bug”

• Thus,theworddebuggingiscoinedJ

6

DebuggingTools• Pythonerrormessage• assert• print• Pythoninterpreter• PythonTutor(http://pythontutor.com)• Pythondebugger• Besttool:

7

1. Thescientificmethod2. Divideandconquer

Ifyoumasterthose,youwillfinddebuggingeasy,andpossiblyenjoyable;-)

8

TwoKeyIdeas

TheScientificMethod

1. Createahypothesis

2. Designanexperimenttotestthathypothesis– Ensurethatityieldsinsight

3. Understandtheresultofyourexperiment– Ifyoudon’tunderstand,thenpossiblysuspendyourmainlineofworktounderstandthat

9

TheScientificMethod

Tips:

• Besystematic– Neverdoanythingifyoudon'thaveareason– Don’tjustflail

• Randomguessingislikelytodigyouintoadeeperhole

• Don’tmakeassumptions(verifythem)

10

ExampleExperiments1. Analternateimplementationofafunction– Runallyourtestcasesafterward

2. Anew,simplertestcase– Examples:smallerinput,ortestafunctioninisolation

– Canhelpyouunderstandthereasonforafailure

11

YourScientificNotebookRecordeverythingyoudo• Specificinputsandoutputs(bothexpectedandactual)• Specificversionsoftheprogram

– Ifyougetstuck,youcanreturntosomethingthatworks– Youcanwritemultipleimplementationsofafunction

• Whatyouhavealreadytried• Whatyouareinthemiddleofdoingnow

– Thismaylooklikeastack!

• Whatyouaresureof,andwhy

Yournotebookalsohelpsifyouneedtogethelporreproduceyourresults.

12

ReadtheErrorMessage

Traceback (most recent call last):File "nx_error.py", line 41, in <module>print(friends_of_friends(rj, myval))

File "nx_error.py", line 30, in friends_of_friendsf = friends(graph, user)

File "nx_error.py", line 25, in friendsreturn set(graph.neighbors(user))#

File "/Library/Frameworks/…/graph.py", line 978, in neighborsreturn list(self.adj[n])

TypeError: unhashable type: 'list'

Listofallexceptions(errors):http://docs.python.org/3/library/exceptions.html#bltin-exceptionsTwootherresources,withmoredetailsaboutafewoftheerrors:http://inventwithpython.com/appendixd.htmlhttp://www.cs.arizona.edu/people/mccann/errors-python

Callstackortraceback

Firstfunctionthatwascalled(<module> meanstheinterpreter)

Secondfunctionthatwascalled

Lastfunctionthatwascalled(thisonesufferedanerror)

Theerrormessage:dauntingbutuseful.Youneedtounderstand:• theliteralmeaningof

theerror• theunderlying

problemscertainerrorstendtosuggest13

CommonErrorTypes

• AssertionError– Raisedwhenanassertstatementfails.

• IndexError– Raisedwhenasequencesubscriptisoutofrange.

• KeyError– Raisedwhenamapping(dictionary)keyisnotfoundinthesetofexistingkeys.

• KeyboardInterrupt– Raisedwhentheuserhitstheinterruptkey(normallyControl-CorDelete).

14

CommonErrorTypes

• NameError– Raisedwhenalocalorglobalnameisnotfound.

• SyntaxError– Raisedwhentheparserencountersasyntaxerror.

• IndentationError– Baseclassforsyntaxerrorsrelatedtoincorrectindentation.

• TypeError– Raisedwhenanoperationorfunctionisappliedtoanobjectofinappropriatetype.

15

DivideandConquer

• Whereisthedefect(or“bug”)?• Yourgoalistofindtheoneplacethatitis• Findingadefectisoftenharderthanfixingit

• Initially,thedefectmightbeanywhereinyourprogram– Itisimpracticaltofinditifyouhavetolookeverywhere

• Idea:bitbybitreducethescopeofyoursearch• Eventually,thedefectislocalizedtoafewlinesoroneline– Thenyoucanunderstandandfixit

16

DivideandConquer

• 4waystodivideandconquer:– Intheprogramcode– Intestcases– Duringtheprogramexecution– Duringthedevelopmenthistory

17

• Localizethedefecttopartoftheprogram– e.g.,onefunction,oronepartofafunction

• Codethatisn’texecutedcannotcontainthedefect

18

DivideandConquerintheProgramCode

Threeapproaches:1. Testonefunctionatatime

19


Threeapproaches:2. Addassertionsorprintstatements– Thedefectisexecutedbeforethefailingassertion(andmaybeafterasucceedingassertion)

20


Threeapproaches:3. Splitcomplexexpressionsintosimplerones

Example:Failureinresult = set({graph.neighbors(user)})

Changeittonbors = graph.neighbors(user)

nbors_set = {nbors}

result = set(nbors_set)

Theerroroccursonthe“nbors_set ={nbors}"line21


DivideandConquerinTestCases

• Yourprogramfailswhenrunonsomelargeinput– It’shardtocomprehendtheerrormessage– Thelogofprintstatementoutputisoverwhelming

• Tryasmallerinput– Chooseaninputwithsomebutnotallcharacteristicsofthelargeinput

– Example:duplicates,zeroesindata,…

22

DivideandConquerinExecutionTimeviaPrint(or“logging”)Statements

• Asequenceofprint statementsisarecordoftheexecutionofyourprogram

• Theprint statementsletyouseeandsearchmultiplemomentsintime

• Printstatementsareausefultechnique,inmoderation• Bedisciplined

– Toomuchoutputisoverwhelmingratherthaninformative– Rememberthescientificmethod:haveareason(ahypothesistobetested)foreachprintstatement

– Don’tonly useprintstatements

23

DivideandConquerinDevelopmentHistory• Thecodeusedtowork(forsometestcase)• Thecodenowfails• Thedefectisrelatedtosomelineyouchanged

• Thisisusefulonlyifyoukeptaversionofthecodethatworked(usegoodnames!)

• Thisismostusefulifyouhavemadefewchanges• Moral:testoften!– Fewerlinestocompare– Yourememberwhatyouwerethinking/doingrecently

24

AMetaphorAboutDebugging

Ifyourcodedoesn’tworkasexpected,thenbydefinitionyoudon’tunderstandwhatisgoingon.

• You’relostinthewoods.• You’rebehindenemylines.• Allbetsareoff.• Don’ttrustanyoneoranything.

Don’tpressonintounexploredterritory-- gobackthewayyoucame! (andleavebreadcrumbs!)

You’retryingto“advancethefrontlines,”not“trailblaze”25

• Thegameistogofrom“workingtoworking”• Whensomethingdoesn’twork,STOP!

– It’swildoutthere!• FIRST:Gobacktothelastsituationthatworkedproperly.

– Rollbackyourrecentchangesandverifythateverythingstillworksasexpected.

– Don’tmakeassumptions– bydefinition,youdon’tunderstandthecodewhensomethinggoeswrong,soyoucan’ttrustyourassumptions.

– Youmayfindthatevenwhatpreviouslyworkednowdoesn’t– Perhapsyouforgottoconsidersome“innocent”orunintentional

change,andnoweventestedcodeisbroken

26

Time-SavingTrick:MakeSureYouareDebuggingtheRightProblem

ABadTimeline

• Aworks,socelebratealittle• NowtryB• Bdoesn’twork• ChangeBandtryagain• ChangeBandtryagain• ChangeBandtryagain…

27

https://xkcd.com/1739/

ABadTimeline

• Aworks,socelebratealittle• NowtryB• Bdoesn’twork• ChangeBandtryagain• ChangeBandtryagain• ChangeBandtryagain…

28

fromgiphy.com

ABetterTimeline• Aworks,socelebratealittle• NowtryB• Bdoesn’twork• RollbacktoA• DoesAstillwork?

– Yes:FindA’thatissomewherebetweenAandB– No:Youhaveunintentionallychangedsomethingelse,andthere’sno

pointfutzingwithBatall!

These“innocent”andunnoticedchangeshappenmorethanyouwouldthink!• Youaddacomment,andtheindentationchanges.• Youaddaprintstatement,andafunctionisevaluatedtwice.• Youmoveafile,andthewrongoneisbeingread• Youareonadifferentcomputer,andthelibraryisadifferentversion

29

OnceYouareonSolidGroundYoucanSetOutAgain• Onceyouhavesomethingthatworksandsomethingthatdoesn’twork,itisonlyamatteroftime

• Youjustneedtoincrementallychangetheworkingcodeintothenon-workingcode,andtheproblemwillrevealitself.

• Variation:Perhapsyourcodeworkswithoneinput,butfailswithanother.Incrementallychangethegoodinputintothebadinputtoexposetheproblem.

30

SimpleDebuggingToolsprint

– showswhatishappeningwhetherthereisaproblemornot

– doesnotstopexecutionassert

– Raisesanexceptionifsomeconditionisnotmet– Doesnothingifeverythingworks– Example:assert len(rj.edges()) == 16

– Usethisliberally!Notjustfordebugging!

31


34

WhatisanException?• Anexceptionisanabnormalcondition(andthusrare)thatarisesinacodesequenceatruntime.

• Forinstance:– Dividinganumberbyzero– Accessinganelementthatisoutofboundsofanarray– Attemptingtoopenafilewhichdoesnotexist

35

WhatisanException?• Whenanexceptionalconditionarises,anobjectrepresentingthatexceptioniscreatedandthrowninthecodethatcausedtheerror

• Anexceptioncanbecaughttohandleitorpassiton

• Exceptionscanbegeneratedbytherun-timesystem,ortheycanbemanuallygeneratedbyyourcode

36

WhatisanException?test = [1,2,3]test[3]

37

IndexError: list index out of range

WhatisanException?successFailureRatio = numSuccesses/numFailures

print('The success/failure ratio is', successFailureRatio)

print('Now here')

38

ZeroDivisionError: integer division or modulo by zero

WhatisanException?val = int(input('Enter an integer: '))

print('The square of the number', val**2)

> Enter an integer: asd

39

ValueError: invalid literal for int() with base 10: 'asd'

HandlingExceptions• Exception mechanism gives the programmer achanceto dosomething against anabnormal condition.

• Exception handling isperforming anaction inresponseto anexception.

• This action may be:– Exiting the program– Retrying the action with or without alternative data– Displaying anerror message and warning user to dosomething

– ....

40

HandlingExceptionstry:

successFailureRatio = numSuccesses/numFailures

print('The S/F ratio is', successFailureRatio)

except ZeroDivisionError:

print('No failures, so the S/F is undefined.')

print('Now here')

41

• Uponenteringthetry block,theinterpreterattemptstoevaluatetheexpressionnumSuccesses/numFailures.

• Ifexpressionevaluationissuccessful,theassignmentisdoneandtheresultisprinted.

• If,however,aZeroDivisionError exceptionisraised,theprintstatementintheexcept blockisexecuted.

while True: val = input('Enter an integer: ') try:

val = int(val) print('The square of the number', val**2) break #to exit the while loop

except ValueError: print(val, 'is not an integer')

42

HandlingExceptions

ChecksforwhetherValueError exceptionisraisedornot

KeywordsofExceptionHandling• There are five keywords inPython to deal withexceptions:try,except,else,raise and finally.

• try:Creates ablock to monitor if any exceptionoccurs.

• except:Follows the try block and catches anyexception which isthrown within it.

43

AreThereManyExceptionsinPython?

• Yes,someofthemare…– Exception– ArithmeticError– OverflowError– ZeroDivisonError– EOFError– NameError– IOError– SyntaxError

44

Listofallexceptions(errors):http://docs.python.org/3/library/exceptions.html#bltin-exceptions

MultipleexceptStatements

45

• It ispossible that more than one exception canbethrown inacode block.– Wecanusemultipleexcept clauses

• Whenanexceptionisthrown,eachexceptstatementisinspectedinorder,andthefirstonewhosetypematches thatoftheexceptionisexecuted.– Type matching means that the exception thrown must beanobject ofthe same class or asub-class ofthe declared classinthe except statement

• Afteroneexcept statementexecutes,theothersarebypassed.

MultipleexceptStatements

46

try:

Youdoyouroperationshere;except Exception-1:

Executethisblock.except Exception-2:

Executethisblock.except (Exception-3[, Exception-4[,...ExceptionN]]]):

Ifthereisanyexceptionfromthegivenexceptionlist,thenexecutethisblock.

except (ValueError, TypeError):…

Theexceptblockwillbeenteredifanyofthelistedexceptions israisedwithinthetryblock

MultipleexceptStatementstry:

f = open('outfile.dat', 'w')dividend = 5divisor = 0division = dividend / divisorf.write(str(division))

except IOError:print("I can't open the file!")

except ZeroDivisionError:print("You can't divide by zero!")

47

Youcan'tdividebyzero!



except Exception:print("Exception occured and handled!")



48

Exceptionoccured andhandled!



except:print("Exception occured and handled!")



49

SyntaxError: default 'except:' must be last

except-elseStatements

50

try:You do your operations here

except: Execute this block.

else: If there is no exception, execute this block.

try:f = open(arg, 'r')

except IOError:print('cannot open', arg)

else:print(arg, 'has', len(f.readlines()), 'lines')

finally Statement

51

• finally creates ablock ofcode that will beexecuted afteratry/except block hascompleted and before the codefollowing the try/except block

• finally block isexecuted whether or notexception isthrown

• finally block isexecuted whether or notexception iscaught

• It isused to guarantee that acode block will beexecuted inanycondition.

finally Statement

52

Youcanuseittocleanupfiles,databaseconnections,etc.

try:You do your operations here

except: Execute this block.

finally: This block will definitely be executed.

try: file = open('out.txt', 'w') do something…

finally: file.close()os.path.remove('out.txt')

Nestedtry Blocks• When anexception occurs insideatry block;

– If the try block does nothave amatching except,then the outertry statement’s except clauses are inspected for amatch

– If amatching except isfound,that except block isexecuted– If no matching except exists,execution flow continues to find a

matching except by inspecting the outer try statements– If amatching except cannot befound atall,the exception will be

caught by Python’s exception handler.

• Execution flow never returns to the line that exceptionwas thrown.This means,anexception iscaught andexcept block isexecuted,the flow will continue with thelines following this except block

53

Let’s clarify itonvarious scenarios

54

try:statement1try:

statement2except Exception1:


statement4;try:


statement6statement7;

except Exception3:statement8

statement9;

Information: Exception1andException2aresubclassesofException3

Question:Whichstatementsareexecutedif1- statement1throwsException12- statement2throwsException13- statement2throwsException34- statement2throwsException1andstatement3throws Exception2

Scenario:statement1throws Exception1

55

try:statement1try:



statement4;try:




statement9;

Exception1Step1:Exceptionisthrown

Step2:except clauses ofthetryblockareinspectedforamatching except statement.Exception3issuperclassofException1,soitmatches.

Step3:statement8isexecuted,exceptionishandledandexecutionflowwillcontinuebypassingthefollowing except clauses

Step4:statement9isexecuted


56

try:statement1try:



statement4;try:




statement9;


Step2:except clauses ofthetryblockareinspectedforamatching except statement.Firstclausecatchestheexception

Step3:statement3isexecuted,exceptionishandled

Step4:executionflowwillcontinuebypassingthefollowing except clauses.statement5isexecuted.

Step5:Assumingnoexceptionisthrownbystatement5,programcontinueswithstatement7andstatement9.


57

try:statement1try:



statement4;try:




statement9;


Step2:except clauses ofthetryblockareinspectedforamatching except statement.Noneofthese except clauses matchException3

Step3:except clauses oftheoutertrystatementareinspectedforamatching except .Exception3iscatchedandstatement8isexecuted


Scenario:statement2throws Exception1and statement3throws Exception2

58

try:statement1try:



statement4;try:




statement9;


Step2:Exceptioniscatchedandstatement3isexecuted.

Step3:statement3throwsanewexception


Exception2

Step4:Except clauses oftheoutertrystatementareinspectedforamatching except.Exception2iscatchedandstatement8isexecuted

raise Statement• Youcanraiseexceptionsbyusingtheraisestatement.

• Thesyntaxisasfollows:raise exceptionName(arguments)

59

raise Statementdef getRatios(vect1, vect2):

ratios = []for index in range(len(vect1)):

try:ratios.append(vect1[index]/vect2[index])

except ZeroDivisionError:ratios.append(float('nan')) #nan = Not a Number

except:raise ValueError(’getRatios called with bad arguments’)

return ratios

try:print(getRatios([1.0, 2.0, 7.0, 6.0], [1.0,2.0,0.0,3.0]))print(getRatios([], []))print(getRatios([1.0, 2.0], [3.0]))

except ValueError as msg:print(msg)

60

[1.0, 1.0, nan, 2.0][]getRatios called with bad arguments

raise Statement• AvoidraisingagenericException! Tocatchit,you'llhave

tocatchallothermorespecificexceptionsthatsubclassit.

61

def demo_bad_catch(): try:

raise ValueError('a hidden bug, do not catch this')raise Exception('This is the exception you expect to handle')

except Exception as error:print('caught this error: ' + repr(error))

>>> demo_bad_catch()caught this error: ValueError('a hidden bug, do not catch this',)

raise Statement• andmorespecificcatcheswon'tcatchthegeneralexception:..

62

def demo_no_catch(): try:

raise Exception('general exceptions not caught by specific handling')except ValueError as e:

print('we will not catch e')

>>> demo_no_catch()Traceback (most recent call last):

File "<stdin>", line 1, in <module> File "<stdin>", line 3, in demo_no_catch

Exception: general exceptions not caught by specific handling

CustomExceptions

63

• UserscandefinetheirownexceptionbycreatinganewclassinPython.

• Thisexceptionclasshastobederived,eitherdirectlyorindirectly,fromtheExceptionclass.

• Mostofthebuilt-inexceptionsarealsoderivedformthisclass.

CustomExceptions

class ValueTooSmallError(Exception):"""Raised when the input value is too small"""pass

class ValueTooLargeError(Exception):"""Raised when the input value is too large"""pass

64

CustomExceptions

number = 10 # you need to guess this number

while True:try:

i_num = int(input("Enter a number: "))if i_num < number:

raise ValueTooSmallErrorelif i_num > number:

raise ValueTooLargeErrorbreak

except ValueTooSmallError:print("This value is too small, try again!")

except ValueTooLargeError:print("This value is too large, try again!")

print("Congratulations! You guessed it correctly.")

65


66

Testing• Programmingtoanalyzedataispowerful• Itisuselessiftheresultsarenotcorrect• Correctnessisfarmoreimportantthanspeed

67

FamousExamples• Ariane 5rocket– OnJune4,1996,themaidenflightoftheEuropeanAriane5launchercrashedabout40secondsaftertakeoff.

– Mediareportsindicatedthattheamountlostwashalfabilliondollars

– Theexplosionwastheresultofasoftwareerror

• Therac-25radiationtherapymachine– In1985aCanadian-builtradiation-treatmentdevicebeganblastingholesthroughpatients'bodies.

68

TestingdoesnotProve Correctness• Edsger Dijkstra:“Programtestingcanbeusedtoshowthepresenceofbugs,butnevertoshowtheirabsence!”

69

Testing=Double-CheckingResults• Howdoyouknowyourprogramisright?– Compareitsoutputtoacorrectoutput

• Howdoyouknowacorrectoutput?– Realdataisbig– Youwroteacomputerprogrambecauseitisnotconvenienttocomputeitbyhand

• Usesmallinputssoyoucancomputebyhand

• Example:standarddeviation– Whataregoodtestsforstd_dev?

70

Testing≠Debugging• Testing:Determiningwhether yourprogramiscorrect– Doesn’tsaywhere orhow yourprogramisincorrect

• Debugging:Locatingthespecificdefectinyourprogram,andfixingit2keyideas:– divideandconquer– thescientificmethod

71

WhatisaTest?

• Atestconsistsof:– aninput:sometimescalled“testdata”– anoracle:apredicate(boolean expression)oftheoutput

72

WhatisaTest?

• Exampletestforsum:– input:[1,2,3]– oracle:resultis6– writethetestas:sum([1, 2, 3]) == 6

• Exampletestforsqrt:– input:3.14– oracle:resultiswithin0.00001of1.772– waystowritethetest:• -0.00001 < sqrt(3.14) – 1.772 < 0.00001• math.abs(sqrt(3.14) – 1.772) < 0.00001

73

TestResults• Thetestpasses ifthebooleanexpressionevaluatestoTrue

• Thetestfails ifthebooleanexpressionevaluatestoFalse

• Usetheassert statement:– assert sum([1, 2, 3]) == 6– assert True doesnothing– assert False crashestheprogramandprintsamessage

74

WheretoWriteTestCases• Atthetoplevel:isruneverytimeyouloadyourprogram

def hypotenuse(a, b):…

assert hypotenuse(3, 4) == 5assert hypotenuse(5, 12) == 13

• Inatestfunction:isrunwhenyouinvokethefunctiondef hypotenuse(a, b):

…def test_hypotenuse():

assert hypotenuse(3, 4) == 5assert hypotenuse(5, 12) == 13

75

AssertionsarenotJustforTestCases• Useassertionsthroughoutyourcode

• Documentswhatyouthinkistrueaboutyouralgorithm

• Letsyouknowimmediatelywhensomethinggoeswrong– Thelongerbetweenacodemistakeandtheprogrammernoticing,theharderitistodebug

76

AssertionsMakeDebuggingEasier• Common,butunfortunate,courseofevents:

– Codecontainsamistake(incorrectassumptionoralgorithm)– Intermediatevalue(e.g.,resultofafunctioncall)isincorrect– Thatvalueisusedinothercomputations,orcopiedintoothervariables

– Eventually,theusernoticesthattheoverallprogramproducesawrongresult

– Whereisthemistakeintheprogram?Itcouldbeanywhere.

• Supposeyouhad10assertionsevenlydistributedinyourcode– Whenonefails,youcanlocalizethemistaketo1/10ofyourcode(thepartbetweenthelastassertionthatpassesandthefirstonethatfails)

77

WheretoWriteAssertions• Functionentry:Areargumentslegal?

– Placeblameonthecallerbeforethefunctionfails

• Functionexit:Isresultcorrect?

• Placeswithtrickyorinterestingcode

• Assertionsareordinarystatements;e.g.,canappearwithinaloop:

for n in myNumbers:assert type(n) == int or type(n) == float

78

Wherenot toWriteAssertions• Don’tclutterthecode

– Sameruleasforcomments

• Don’twriteassertionsthatarecertaintosucceed– Theexistenceofanassertiontellsaprogrammerthatitmightpossiblyfail

• Don’twriteanassertionifthefollowingcodewouldfailinformatively

assert type(name) == strprint("Hello, " + name)

• Writeassertionswheretheymaybeusefulfordebugging

79

WhattoWriteAssertionsAbout• Resultsofcomputations

• Correctly-formeddatastructures

assert 0 <= index < len(mylist)assert len(list1) == len(list2)

80

WhentoWriteTests• Twopossibilities:–Writecodefirst,thenwritetests–Writetestsfirst,thenwritecode

81

WhentoWriteTests• Ifyouwritethecodefirst,youremembertheimplementationwhilewritingthetests– Youarelikelytomakethesamemistakesintheimplementation

82

WhentoWriteTests• Ifyouwritethetestsfirst,youwillthinkmoreaboutthefunctionalitythanaboutaparticularimplementation– Youmightnoticesomeaspectofbehaviorthatyouwouldhavemadeamistakeabout

– Thisisthebetterchoice

83

WritetheWholeTest• Acommonmistake:

1. Writethefunction2. Makeuptestinputs3. Runthefunction4. Usetheresultastheoracle

• Youdidn’twriteatest,butonlyhalfofatest– Createdthetestsinputs,butnottheoracle

• Thetestdoesnotdeterminewhetherthefunctioniscorrect– Onlydeterminesthatitcontinuestobeascorrect(orincorrect)asitwasbefore

84

TestingApproaches• Blackboxtesting- Choosetestdatawithoutlookingatimplementation

• Glassbox(whitebox,clearbox)testing -Choosetestdatawith knowledgeofimplementation

85

InsideKnowledgemightbeNice• Assumethecodebelow:

c = a + bif c > 100

print("Tested”)print("Passed”)

• Creatingatestcasewitha=40andb=70isnotenough– Althougheverylineofthecodewillbeexecuted

• Anothertestcasewitha=40andb=30wouldcompletethetest

86

TestsmightnotRevealanErrorSometimes

def mean(numbers):"""Returns the average of the argument list.

The argument must be a non-empty number list."""return sum(numbers)//len(numbers)

# Testsassert mean([1, 2, 3, 4, 5]) == 3assert mean([1, 2, 3]) == 2

Thisimplementationiselegant,butwrong!

mean([1,2,3,4]) à should return 2.5!!!

87

LastbutnotLeast,Don’tWriteMeaninglessTestsdef mean(numbers):"""Returns the average of the argument list.

The argument must be a non-empty number list."""return sum(numbers)//len(numbers)

Unnecessarytests.Don’twritethese:

mean([1, 2, "hello"])mean("hello")mean([])

88