charlie garrod michael hiltoncharlie/courses/15-214/2017-fall/slides/20171003... · 03/10/2017  ·...

115-214

SchoolofComputerScience

PrinciplesofSoftwareConstruction:Objects,Design,andConcurrency

Designing(sub-)systems

Incrementalimprovements

CharlieGarrod MichaelHilton

215-214

Administriva

• HW4PartAdueOct5th

• Mandatorydesignreviewmeeting• FinalFriday,December15,201705:30-08:30p.m.

315-214

ExamresultsMidterm1

• Mean• 43.73/72• NOTE:Thiscoursedoesnothaveafixedlettergradepolicy;i.e.thefinallettergradeswill not beA=90-100%,B=80-90%,etc.

• StandardDeviation• 9.26

415-214

ReviewStrategyPattern

• Problem:Clientsneeddifferentvariantsofanalgorithm• Solution:Createaninterfaceforthealgorithm,withan

implementingclassforeachvariantofthealgorithm• Consequences:

– Easilyextensiblefornewalgorithmimplementations– Separatesalgorithmfromclientcontext– Introducesanextrainterfaceandmanyclasses:

• Codecanbehardertounderstand• Lotsofoverheadifthestrategiesaresimple

515-214

DesignProblem

public class EgyptianTranslator {int n;EgyptianTranslator(int n) {

this.n = n;…

}public String translate() {

…}

}

615-214

CODESMELLS

715-214

CodeSmells

• Acodesmellisahintthatsomethinghasgonewrongsomewhereinyourcode.

• Asmellissniffable,orsomethingthatisquicktospot.

• Asmelldoesn’talwaysindicateaproblem

815-214

BadSmells:Classification

• MostCommon:codeduplication• Class/methodorganization

– Largeclass,LongMethod,LongParameterList,LazyClass,DataClass,...

• Lackofloosecouplingorcohesion– InappropriateIntimacy,FeatureEnvy,DataClumps,...

• Toomuchortoolittledelegation– MessageChains,MiddleMan,...

• NonObject-Orientedcontrolordatastructures– SwitchStatements,PrimitiveObsession,...

• Other:Comments

915-214

Codeduplication(1)

code

code

code

code

Class

Method 1

Method 2

Method 3

code

Class

Method 1

Method 2

Method X

MethodX();

Method 3MethodX();

MethodX();MethodX();

• Extract method

• Rename method

1015-214

Codeduplication(2)

code

Subclass A

Method codeMethod

Subclass BClass

Sameexpressionintwosiblingclasses:

• Samecode:Extractmethod+Pullupfield

• Similarcode:Extractmethod+FormTemplateMethod

• Differentalgorithm:Substitutealgorithm

1115-214

Codeduplication(3)

code

ClassA

MethodA codeMethodB

ClassB

1215-214

Codeduplication(3)

ClassA

MethodA MethodB

ClassB

Sameexpressionintwounrelatedclasses:

• Extractclass

• Ifthemethodreallybelongsinoneofthetwoclasses,keepitthereandinvokeitfromtheotherclass

code

ClassX

MethodX

ClassX.MethodX(); ClassX.MethodX();

1315-214

Longmethod//700LOCpublic boolean foo() {

try {synchronized () {

if () {} else {}for () {

if () {if () {

if () {if ()?{

if () {for () {}

}}

} else {if () {

for () {if () {} else {}if () {} else {

if () {}

}if () {

if () {if () {

for () {}

}}

} else {}

}} else {}

}}

}}

Source: http://thedailywtf.com/Articles/Coding-Like-the-Tour-de-France.aspx

• Rememberthis?

1415-214

Solution:Refactoring

• Refactoringisachangetoaprogramthatdoesn’tchangethebehavior,butimprovesanon-functionalattributeofthecode(notreworking).

• Examples:– Improvereadability– Reducecomplexity

• Benefitsincludeincreasedmaintainability,andeasierextensibility

• Fearlesslyrefactorwhenyouhavegoodunittests

1515-214

Refactoringalongmethod

void printOwing() {Enumeration e = _orders.elements();double outstanding = 0.0;// Print banner System.out.println(“******************“);System.out.println(“***** Customer *****“); System.out.println(“******************“);// Calculate outstandingWhile (e.hasMoreElements()) {

Order each = (Order) e.nextElement(); outstanding += each.getAmount();

} // Print details System.out.println(“name: “ + _name); System.out.println(“amount” + outstanding);

}

1615-214

Refactoringalongmethod

void printOwing() {Enumeration e = _orders.elements();double outstanding = 0.0;// Print banner System.out.println(“******************“);System.out.println(“***** Customer *****“); System.out.println(“******************“);// Calculate outstandingWhile (e.hasMoreElements()) {

Order each = (Order) e.nextElement(); outstanding += each.getAmount();

} // Print details System.out.println(“name: “ + _name); System.out.println(“amount” + outstanding);

}

1715-214

Refactoringalongmethod

void printOwing() {Enumeration e = _orders.elements();double outstanding = 0.0;

printBanner();

// Calculate outstandingWhile (e.hasMoreElements()) {

Order each = (Order) e.nextElement(); outstanding += each.getAmount();

} // Print details System.out.println(“name: “ + _name); System.out.println(“amount” + outstanding);

}

void printBanner(){System.out.println(“******************“);System.out.println(“***** Customer *****“); System.out.println(“******************“);

}

Extractmethod

CompileandtesttoseewhetherI'vebrokenanything

1815-214

Refactoringalongmethod

void printOwing() {Enumeration e = _orders.elements();double outstanding = 0.0;

printBanner();

// Calculate outstandingWhile (e.hasMoreElements()) {

Order each = (Order) e.nextElement(); outstanding += each.getAmount();

} // Print details System.out.println(“name: “ + _name); System.out.println(“amount” + outstanding);

} void printBanner(){…}

1915-214

Refactoringalongmethod

void printOwing() {Enumeration e = _orders.elements();double outstanding = 0.0;

printBanner();

// Calculate outstandingWhile (e.hasMoreElements()) {

Order each = (Order) e.nextElement(); outstanding += each.getAmount();

} printDetails(outstanding);

}void printBanner(){…}void printDetails(outstanding){

System.out.println(“name: “ + _name); System.out.println(“amount” + outstanding);

}

Extractmethodusinglocalvariables

CompileandtesttoseewhetherI'vebrokenanything

2015-214

Refactoringalongmethod

void printOwing() {Enumeration e = _orders.elements();double outstanding = 0.0;

printBanner();

// Calculate outstandingWhile (e.hasMoreElements()) {

Order each = (Order) e.nextElement(); outstanding += each.getAmount();

} printDetails(outstanding);

}void printBanner(){…}void printDetails(outstanding){

System.out.println(“name: “ + _name); System.out.println(“amount” + outstanding);

}

2115-214

Refactoringalongmethod

void printOwing() {Enumeration e = _orders.elements();double outstanding = getOutstanding(); printBanner(); printDetails(outstanding);

}void printBanner(){…}void printDetails(outstanding){…}

double getOutstanding() { Enumeration e = _orders.elements();double result = 0.0; While (e.hasMoreElements()) {

Order each = (Order) e.nextElement(); result += each.getAmount();

} return result;

}

Extractmethodreassigningalocalvariable

CompileandtesttoseewhetherI'vebrokenanything

2215-214

ManyMoreBadSmellsandSuggestedRefactorings

• Topcrime:codeduplication• Class/methodorganization

– Largeclass,LongMethod,LongParameterList,LazyClass,DataClass,...

• Lackofloosecouplingorcohesion– InappropriateIntimacy,FeatureEnvy,DataClumps,...

• Toomuchortoolittledelegation– MessageChains,MiddleMan,...

• NonObject-Orientedcontrolordatastructures– SwitchStatements,PrimitiveObsession,...

• Other:Comments

2315-214

ANTI-PATTERNS

2415-214

Anti-patterns

• “Anti”-pattern• PatternsofthingsyoushouldNOTdo• Oftenhavememorablenames.

2515-214

Commonanti-patterns

• Spaghetticode

2615-214

Commonanti-patterns

• Spaghetticode• TheBlob

2715-214

Commonanti-patterns

• Spaghetticode• TheBlob• GoldenHammer

2815-214

Commonanti-patterns

• Spaghetticode• TheBlob• GoldenHammer• LavaFlow

2915-214

Commonanti-patterns

• Spaghetticode• TheBlob• GoldenHammer• LavaFlow• SwissArmyKnife

3015-214

EVALUATINGFUNCTIONALCORRECTNESS

3115-214

Reminder:FunctionalCorrectness

• Thecompilerensuresthatthetypesarecorrect(typechecking)– Prevents“MethodNotFound”and“CannotaddBooleantoInt”errorsat

runtime

• Staticanalysistools(e.g.,FindBugs)recognizecertaincommonproblems– WarnsonpossibleNullPointerExceptions orforgettingtoclosefiles

• Howtoensurefunctionalcorrectnessofcontractsbeyond?

3215-214

FormalVerification

• Provingthecorrectnessofanimplementationwithrespecttoaformalspecification,usingformalmethodsofmathematics.

• Formallyprovethatallpossibleexecutionsofanimplementationfulfillthespecification

• Manualeffort;partialautomation;notautomaticallydecidable

3315-214

Testing

• Executingtheprogramwithselectedinputsinacontrolledenvironment(dynamicanalysis)

• Goals:– Revealbugs(maingoal)– Assessquality(hardtoquantify)– Clarifythespecification,documentation– Verifycontracts

"Testing shows the presence, not the absence of bugs

Edsger W. Dijkstra 1969

3415-214

TestingDecisions

• Whotests?– Developers– OtherDevelopers– SeparateQualityAssuranceTeam– Customers

• Whentotest?– Beforedevelopment– Duringdevelopment– Aftermilestones– Beforeshipping

• Whentostoptesting?

(More in 15-313)

3515-214

TESTCOVERAGE

3615-214

Howmuchtesting?

• Yougenerallycannottestallinputs– toomany,usuallyinfinite

• Butwhenitworks,exhaustivetestingisbest!

• Whentostoptesting?– inpractice,whenyourunoutofmoney

3715-214

Whatmakesagoodtestsuite?

• Provideshighconfidencethatcodeiscorrect• Short,clear,andnon-repetitious

– Moredifficultfortestsuitesthanregularcode– Realistically,testsuiteswilllookworse

• Canbefuntowriteifapproachedinthisspirit

3815-214

• Alsoknowasfuzztesting,torturetesting• Try“random”inputs,asmanyasyoucan

– Chooseinputstotickleinterestingcases– Knowledgeofimplementationhelpshere

• Seedrandomnumbergeneratorsotestsrepeatable• Successfulinsomedomains(parsers,networkissues,…)

– But,manytestsexecutesimilarpaths– But,oftenfindsonlysuperficialerrors

Blackbox:RandomInputsNextbestthingtoexhaustivetesting

3915-214

Blackbox testingBlackbox:CoveringSpecifications

• Lookingatspecifications,notcode:

• Testrepresentativecase• Testboundarycondition• Testexceptionconditions• (Testinvalidcase)

4015-214

TextualSpecification

public int read(byte[] b, int off, int len) throws IOException

§ Reads up to len bytes of data from the input stream into an array of bytes. An attempt is made to read as many as len bytes, but a smaller number may be read. The number of bytes actually read is returned as an integer. This method blocks until input data is available, end of file is detected, or an exception is thrown.

§ If len is zero, then no bytes are read and 0 is returned; otherwise, there is an attempt to read at least one byte. If no byte is available because the stream is at end of file, the value -1 is returned; otherwise, at least one byte is read and stored into b.

§ The first byte read is stored into element b[off], the next one into b[off+1], and so on. The number of bytes read is, at most, equal to len. Let k be the number of bytes actually read; these bytes will be stored in elements b[off] through b[off+k-1], leaving elements b[off+k] through b[off+len-1] unaffected.

§ In every case, elements b[0] through b[off] and elements b[off+len] through b[b.length-1] are unaffected.

• Throws:§ IOException - If the first byte cannot be read for any reason other than end of file, or if

the input stream has been closed, or if some other I/O error occurs.§ NullPointerException - If b is null.§ IndexOutOfBoundsException - If off is negative, len is negative, or len is greater

than b.length - off

4115-214

StructuralAnalysisofSystemunderTest

– Organizedaccordingtoprogramdecisionstructure

public static int binsrch (int[] a, int key) {

int low = 0;int high = a.length - 1;

while (true) {

if ( low > high ) return -(low+1);

int mid = (low+high) / 2;

if ( a[mid] < key ) low = mid + 1;else if ( a[mid] > key ) high = mid - 1;else return mid;

}}

Whitebox testing

4215-214

StructuralAnalysisofSystemunderTest

– Organizedaccordingtoprogramdecisionstructure

public static int binsrch (int[] a, int key) {

int low = 0;int high = a.length - 1;

while (true) {

if ( low > high ) return -(low+1);

int mid = (low+high) / 2;

if ( a[mid] < key ) low = mid + 1;else if ( a[mid] > key ) high = mid - 1;else return mid;

}}

Whitebox testing

Will this statement get executed in a test?

Does it return the correct result?

4315-214

StructuralAnalysisofSystemunderTest

– Organizedaccordingtoprogramdecisionstructure

public static int binsrch (int[] a, int key) {

int low = 0;int high = a.length - 1;

while (true) {

if ( low > high ) return -(low+1);

int mid = (low+high) / 2;

if ( a[mid] < key ) low = mid + 1;else if ( a[mid] > key ) high = mid - 1;else return mid;

}}

Whitebox testing

Could this array index be out of bounds?

Will this statement get executed in a test?

Does it return the correct result?

4415-214

StructuralAnalysisofSystemunderTest

– Organizedaccordingtoprogramdecisionstructure

public static int binsrch (int[] a, int key) {

int low = 0;int high = a.length - 1;

while (true) {

if ( low > high ) return -(low+1);

int mid = (low+high) / 2;

if ( a[mid] < key ) low = mid + 1;else if ( a[mid] > key ) high = mid - 1;else return mid;

}}

Whitebox testing

Could this array index be out of bounds?

Does this return statement ever get reached?

Will this statement get executed in a test?

Does it return the correct result?

4515-214

Codecoveragemetrics

• Methodcoverage– coarse• Branchcoverage– fine• Pathcoverage– toofine

– Costishigh,valueislow– (Relatedtocyclomatic complexity)

4615-214

MethodCoverage

• Tryingtoexecuteeachmethodaspartofatleastonetest

• Doesthisguaranteecorrectness?

4715-214

StatementCoverage

• Tryingtotestallpartsoftheimplementation• Executeeverystatementinatleastonetest

• Doesthisguaranteecorrectness?

4815-214

StructureofCodeFragmenttoTest

Flow chart diagram forjunit.samples.money.Money.equals

4915-214

StatementCoverage

• Statementcoverage– Whatportionofprogramstatements

(nodes)aretouchedbytestcases

• Advantages– Testsuitesizelinearinsizeofcode– Coverageeasilyassessed

• Issues– Deadcodeisnotreached– Mayrequiresomesophisticationto

selectinputsets– Fault-toleranterror-handlingcode

maybedifficultto“touch”– Metric:Couldcreateincentiveto

removeerrorhandlers!

5015-214

BranchCoverage

• Branchcoverage– Whatportionofconditionbranchesare

coveredbytestcases?– Or:Whatportionofrelationalexpressions

andvaluesarecoveredbytestcases?• Conditiontesting(Tai)

– Multicondition coverage– allbooleancombinationsoftestsarecovered

• Advantages– Testsuitesizeandcontentderived

fromstructureofbooleanexpressions– Coverageeasilyassessed

• Issues– Deadcodeisnotreached– Fault-toleranterror-handlingcode

maybedifficultto“touch”

5115-214

PathCoverage

• Pathcoverage– Whatportionofallpossiblepathsthroughthe

programarecoveredbytests?– Looptesting:Considerrepresentativeandedge

cases:• Zero,one,twoiterations• Ifthereisaboundn:n-1,n,n+1iterations• Nestedloops/conditionalsfrominsideout

• Advantages– Bettercoverageoflogicalflows

• Disadvantages– Infinitenumberofpaths– Notallpathsarepossible,ornecessary

• Whatarethesignificantpaths?– Combinatorialexplosionincasesunless

carefulchoicesaremade• E.g.,sequenceofniftestscanyield

upto2^npossiblepaths– Assumptionthatprogramstructureisbasically

sound

5215-214

TestCoverageTooling

• Coverageassessmenttools– Trackexecutionofcodebytestcases

• Countvisitstostatements– Developreportswithrespecttospecificcoveragecriteria– Instructioncoverage,

linecoverage,branchcoverage

• Example:Cobertura andEclEmma forJUnit tests

5315-214

5415-214

Checkyourunderstanding

• Writetestcasestoachieve100%linecoveragebutnot100%branchcoverage

int foo(int a, int b) {if (a == b)

a = a * 2;if (a + b > 10)

return a - b;return a + b;

}

5515-214

Checkyourunderstanding

• Writetestcasestoachieve100%linecoverageandalso 100%branchcoverage

int foo(int a, int b) {if (a == b)

a = a * 2;if (a + b > 10)

return a - b;return a + b;

}

5615-214

Checkyourunderstanding

• Writetestcasestoachieve100%linecoverageand 100%branchcoverageand 100%pathcoverage

int foo(int a, int b) {if (a == b)

a = a * 2;if (a + b > 10)

return a - b;return a + b;

}

5715-214

Coveragemetrics:usefulbutdangerous

• Cangivefalsesenseofsecurity• Examplesofwhatcoverageanalysiscouldmiss

– Datavalues– Concurrencyissues– raceconditionsetc.– Usabilityproblems– Customerrequirementsissues

• Highbranchcoverageisnot sufficient

5815-214

Testsuites– idealvs.real

• Idealtestsuites– Uncoverallerrorsincode– Test“non-functional”attributessuchasperformanceandsecurity– Minimumsizeandcomplexity

• RealtestSuites– Uncoversomeportionoferrorsincode– Haveerrorsoftheirown– Arenonethelesspriceless

5915-214

STATICANALYSIS

6015-214

StupidBugs

public class CartesianPoint {private int x, y;int getX() { return this.x; }int getY() { return this.y; }public boolean equals(CartesianPoint that) {

return (this.getX()==that.getX()) && (this.getY() == that.getY());

}}

6115-214

Fin

dB

ug

s

6215-214

Stupid Subtle Bugs

public class Object {public boolean equals(Object other) { … }

// other methods…}

public class CartesianPoint extends Object {private int x, y;int getX() { return this.x; }int getY() { return this.y; }public boolean equals(CartesianPoint that) {

return (this.getX()==that.getX()) && (this.getY() == that.getY());

}}

classes with no explicit superclass implicitly extendObject

can’t change argument type when overriding

This defines a different equalsmethod, rather than overriding Object.equals()

6315-214

Fixing the Bug

public class CartesianPoint {private int x, y;int getX() { return this.x; }int getY() { return this.y; }

@Overridepublic boolean equals(Object o) {

if (!(o instanceof CartesianPoint)return false;

CartesianPoint that = (CartesianPoint) o;

return (this.getX()==that.getX()) && (this.getY() == that.getY());

}}

Declare our intent to override;Compiler checks that we did it

Use the same argument type as the method we are overriding

Check if the argument is a CartesianPoint.Correctly returns false if o is null

Create a variable of the right type, initializing it with a cast

6415-214

Fin

dB

ug

s

6515-214

Fin

dB

ug

s

6615-214

Ch

eckS

tyle

6715-214

StaticAnalysis

• Analyzingcodewithoutexecutingit(automatedinspection)• Looksforbugpatterns• Attemptstoformallyverifyspecificaspects• Pointouttypicalbugsorstyleviolations

– NullPointerExceptions– IncorrectAPIuse– Forgettingtocloseafile/connection– Concurrencyissues– Andmany,manymore(over250inFindBugs)

• IntegratedintoIDEorbuildprocess• FindBugsandCheckStyleopensource,manycommercial

productsexist

6815-214

ExampleFindBugsBugPatterns

• Correctequals()• Useof==• Closingstreams• Illegalcasts• Nullpointerdereference• Infiniteloops• Encapsulationproblems• Inconsistentsynchronization• InefficientStringuse• Deadstoretovariable

6915-214

Bugfinding

7015-214

Canyoufindthebug?

if (listeners == null)

listeners.remove(listener);

JDK1.6.0, b105, sun.awt.x11.XMSelection

7115-214

Wrongbooleanoperator

if (listeners != null)

listeners.remove(listener);

JDK1.6.0, b105, sun.awt.x11.XMSelection

7215-214

Canyoufindthebug?

public String sendMessage (User user, String body, Date time) {

return sendMessage(user, body, null);

}

public String sendMessage (User user, String body, Date time, List attachments) {

String xml = buildXML (body, attachments);

String response = sendMessage(user, xml);

return response;

}

7315-214

Infiniterecursiveloop

public String sendMessage (User user, String body, Date time) {

return sendMessage(user, body, null);

}

public String sendMessage (User user, String body, Date time, List attachments) {

String xml = buildXML (body, attachments);

String response = sendMessage(user, xml);

return response;

}

7415-214

Canyoufindthebug?

String b = "bob";

b.replace('b', 'p');

if(b.equals("pop")){…}

7515-214

Methodignoresreturnvalue

String b= "bob";

b = b.replace('b', 'p');

if(b.equals("pop")){…}

7615-214

Whatdoesthisprint?

Integer one = 1;Long addressTypeCode = 1L;

if (addressTypeCode.equals(one)) {System.out.println("equals");

} else {System.out.println("not equals");

}

7715-214

Whatdoesthisprint?

Integer one = 1;Long addressTypeCode = 1L;

if (addressTypeCode.equals(one)) {System.out.println("equals");

} else {System.out.println("not equals");

}

7815-214

ASIDE:FINDBUGS NULLPOINTERANALYSIS

Detector foo = null;foo.execute();

7915-214

FindBugs

• Workson“.class”filescontainingbytecode

– Recall:Javasourcecodecompiledtobytecode; JVMexecutesbytecode

• Processingusingdifferentdetectors:

– Independentofeachother

– Maysharesomeresources(e.g.,controlflowgraph,dataflowanalysis)

– GOAL:Lowfalsepositives

– Eachdetectorisdrivenbyasetofheuristics

• Output:bugpatterncode,sourcelinenumber,descriptivemessage(severity)

HIGHSEVERE RISK OF

PROGRAM FAILURE

MEDIUMELEVATED RISK OF PROGRAM FAILURE

LOWLOW RISK OF

PROGRAM FAILURE

8015-214

Nullpointerdereferencing

• Findingsomenullpointerdereferencesrequiresophisticatedanalysis:– Analyzingacrossmethodcalls,modelingcontentsofheapobjects

• Inpracticemanyexamplesofobvious nullpointerdereferences:– Valueswhicharealwaysnull– Valueswhicharenullonsomecontrolpath

• Howtodesignananalysistofindobviousnullpointerdereferences?– Idea:Lookforplaceswherevaluesareusedinasuspiciousway

From: https://www.cs.umd.edu/class/spring2005/cmsc433/lectures/findbugs.pdf

8115-214

SimpleAnalysis

Detector foo = null;foo.execute();

Dereferencing Null

Detector foo = new Detector(…);foo.execute();

Dereferencing NonNull

HIGHSEVERE RISK OF

PROGRAM FAILURE

J

8215-214

Ifonlyitwerethatsimple…

• Infeasiblepaths(falsepositives)

• Isamethod’sparameternull?

boolean b;if (p != null)

b = true;else

b = false;if (b)

p.f();

void foo(Object obj) {int x = obj.hashcode();…

}

8315-214

Dataflowanalysis

• Ateachpointinamethod,keeptrackofdataflowfacts– E.g.,whichlocalvariablesandstacklocationsmightcontainnull

• Symbolicallyexecutethemethod:– Modelinstructions– Modelcontrolflow– Untilafixedpointsolutionisreached

8415-214

Dataflowvalues

• Modelvaluesoflocalvariablesandstackoperandsusinglatticeofsymbolicvalues

• Whentwocontrolpathsmerge,usemeet operatortocombinevalues:

8515-214

Dataflowvalues

• Modelvaluesoflocalvariablesandstackoperandsusinglatticeofsymbolicvalues

• Whentwocontrolpathsmerge,usemeet operatortocombinevalues:

Null⬦ Null=Null

8615-214

Dataflowvalues

• Modelvaluesoflocalvariablesandstackoperandsusinglatticeofsymbolicvalues

• Whentwocontrolpathsmerge,usemeet operatortocombinevalues:

Null⬦ NotNull=MaybeNull

8715-214

Null-pointerdataflowexample

x=y=z=null

y=new…z=new...

y.f()

x.f() z.f()

x = y = z = null;if (cond) {

y = new …;z = new …;

}y.f();if (cond2)

x.f();else

z.f();

8815-214

Null-pointerdataflowexample

x=y=z=null

y=new…z=new...

y.f()

x.f() z.f()

x = y = z = null;if (cond) {

y = new …;z = new …;

}y.f();if (cond2)

x.f();else

z.f();

8915-214

Null-pointerdataflowexample

x=y=z=null

y=new…z=new...

y.f()

x.f() z.f()

x = y = z = null;if (cond) {

y = new …;z = new …;

}y.f();if (cond2)

x.f();else

z.f();

x = nully = not nullz = not null

9015-214

Null-pointerdataflowexample

x=y=z=null

y=new…z=new...

y.f()

x.f() z.f()

x = y = z = null;if (cond) {

y = new …;z = new …;

}y.f();if (cond2)

x.f();else

z.f();

x = nully = not nullz = not null

9115-214

Null-pointerdataflowexample

x=y=z=null

y=new…z=new...

y.f()

x.f() z.f()

x = y = z = null;if (cond) {

y = new …;z = new …;

}y.f();if (cond2)

x.f();else

z.f();

x = nully = not nullz = not null

x = nully = nullz = null

9215-214

Null-pointerdataflowexample

x=y=z=null

y=new…z=new...

y.f()

x.f() z.f()

x = y = z = null;if (cond) {

y = new …;z = new …;

}y.f();if (cond2)

x.f();else

z.f();

x = nully = maybez = maybe

9315-214

Null-pointerdataflowexample

x=y=z=null

y=new…z=new...

y.f()

x.f() z.f()

x = y = z = null;if (cond) {

y = new …;z = new …;

}y.f();if (cond2)

x.f();else

z.f();

x = null

9415-214

Null-pointerdataflowexample

x=y=z=null

y=new…z=new...

y.f()

x.f() z.f()

x = y = z = null;if (cond) {

y = new …;z = new …;

}y.f();if (cond2)

x.f();else

z.f();

z = uncertain

9515-214

Abstract Interpretation

• Staticprogramanalysisisthesystematicexamination ofanabstractionofaprogram’sstatespace

• Abstraction– Don’ttrackeverything!(that’snormalinterpretation)– Trackanimportantabstraction

• Systematic– Ensureeverythingischeckedinthesameway

Details on how this works in 15-313

9615-214

COMPARINGQUALITYASSURANCESTRATEGIES

9715-214

Errorexists No errorexists

ErrorReported Truepositive(correctanalysisresult)

Falsepositive(annoying noise)

NoErrorReported Falsenegative(falseconfidence)

True negative(correctanalysisresult)

Sound Analysis: reports all defectsà no false negativestypically overapproximated

Complete Analysis:every reported defect is an actual defect à no false positivestypically underapproximated

9815-214

Checkyourunderstanding

• Whatisatrivialwaytoimplement:– asoundanalysis?– acompleteanalysis?

9915-214

DefectsreportedbySoundAnalysis

AllDefects

DefectsreportedbyCompleteAnalysis

UnsoundandIncompleteAnalysis

10015-214

Errorexists No errorexists

ErrorReported Truepositive(correctanalysisresult)

Falsepositive(annoying noise)

NoErrorReported Falsenegative(falseconfidence)

True negative(correctanalysisresult)

How does testing relate? And formal verification?

Sound Analysis: reports all defectsà no false negativestypically overapproximated

Complete Analysis:every reported defect is an actual defect à no false positivestypically underapproximated

10115-214

The Bad News: Rice's Theorem

• Everystaticanalysisisnecessarilyincompleteorunsoundorundecidable(ormultipleofthese)

• Eachapproachhasdifferenttradeoffs

"Any nontrivial property about the language recognized by a Turing machine is undecidable.“

Henry Gordon Rice, 1953

10215-214

Soundness/Completeness/PerformanceTradeoffs

• Typecheckingdoescatchaspecificclassofproblems(sound),butdoesnotfindallproblems

• Compileroptimizationsmusterronthesafeside(onlyperformoptimizationswhensureit'scorrect;->complete)

• Manypracticalbug-findingtoolsanalysesareunsoundandincomplete– Catchtypicalproblems– Mayreportwarningsevenforcorrectcode– Maynotdetectallproblems

• Overwhelmingamountsoffalsenegativesmakeanalysisuseless• Notall"bugs"needtobefixed

10315-214

Testing, Static Analysis, and Proofs

• Testing– Observableproperties– Verifyprogramforoneexecution– Manualdevelopmentwith

automatedregression– Mostpracticalapproachnow– Doesnotfindallproblems

(unsound)

• StaticAnalysis– Analysisofallpossibleexecutions– Specificissuesonlywith

conservativeapprox.andbugpatterns

– Toolsavailable,usefulforbugfinding

– Automated,butunsoundand/orincomplete

• Proofs(FormalVerification)– Anyprogramproperty– Verifyprogramforallexecutions– Manualdevelopmentwith

automatedproofcheckers– Practicalforsmallprograms,may

scaleupinthefuture– Soundandcomplete,butnot

automaticallydecidable

What strategy touse in your project?

10415-214

Take-HomeMessages

• Therearemanyformsofqualityassurance• Testingshouldbeintegratedintodevelopment

– possiblyeventestfirst

• Variouscoveragemetricscanmoreorlessapproximatetestsuitequality

• Staticanalysistoolscandetectcertainpatternsofproblems• Soundnessandcompletenesstocharacterizeanalyses