correctness correctness. quality perceptions the perception of quality associated with your code is...

CorrectnessCorrectness

Quality PerceptionsQuality Perceptions

The perception of quality associated with your code is typically bound to: Correctness Efficiency (speed of execution usually) Cost (if it costs more, it must be better right?) Robustness Flexibility Functionality Maintainability Security Usability Whatever the user likes.

The perception of quality associated with your code is typically bound to: Correctness Efficiency (speed of execution usually) Cost (if it costs more, it must be better right?) Robustness Flexibility Functionality Maintainability Security Usability Whatever the user likes.


We will focus on correctness in this class. Often has an indirect impact on:

Cost Robustness Flexibility Functionality Maintainability Security

What is correctness?

We will focus on correctness in this class. Often has an indirect impact on:

Cost Robustness Flexibility Functionality Maintainability Security

What is correctness?

Definition of Software Correctness

Definition of Software Correctness

Correct software must accomplish the following: Compute accurate results Operate safely Implement the requirements and meet the

specifications Achieve the above for all possible inputs Recognize input from outside its domain

What can we do to help us achieve this?

Correct software must accomplish the following: Compute accurate results Operate safely Implement the requirements and meet the

specifications Achieve the above for all possible inputs Recognize input from outside its domain

What can we do to help us achieve this?

Standard TechniquesStandard Techniques

Use design patterns – don’t reinvent the wheel Use standard libraries Code re-use

If it we know something is correct from past experience, then why start from scratch to re-invent it? Need to be mindful of legal issues – are you

allowed to use the software, are there licensing issues (use of freeware in software you want to sell)?

Use design patterns – don’t reinvent the wheel Use standard libraries Code re-use

If it we know something is correct from past experience, then why start from scratch to re-invent it? Need to be mindful of legal issues – are you

allowed to use the software, are there licensing issues (use of freeware in software you want to sell)?

Choice of Programming Language


Choose a language with strict type checking. A type checker is like a theorem prover which

confirms your assertions about the functions/methods and variables you use in the program.

Integer f (float x) {return …

}

When the type checker checks the code, it confirms: f is called with one argument and it is a value of

type float If f produces a result, then that result belongs to

the class Integer.

Choose a language with strict type checking. A type checker is like a theorem prover which

confirms your assertions about the functions/methods and variables you use in the program.

Integer f (float x) {return …

}

When the type checker checks the code, it confirms: f is called with one argument and it is a value of

type float If f produces a result, then that result belongs to

the class Integer.



While type checking (and other useful support from programming languages, environments, etc) does not directly check that we match the requirements and specification of the application, it does help reduce coding errors.

You no longer have to check that the function f maps a float onto an Integer.

Does not alleviate the need for testing. Only true for languages with sound type systems

(Ada, Java, ML, Haskell, …). Not true for languages such as C/C++ because the function f could consume a bit pattern which is another type (i.e., string) and interpret it as a float.

While type checking (and other useful support from programming languages, environments, etc) does not directly check that we match the requirements and specification of the application, it does help reduce coding errors.

You no longer have to check that the function f maps a float onto an Integer.

Does not alleviate the need for testing. Only true for languages with sound type systems

(Ada, Java, ML, Haskell, …). Not true for languages such as C/C++ because the function f could consume a bit pattern which is another type (i.e., string) and interpret it as a float.

AssertionsAssertions

Type checking is limited in the support it provides. We want something more.

An assertion is a claim about the state of our program (values of variables and their relationship to each other) at various points in the execution of our program.

We are probably used to writing comments that states assertions:int p; // p is a prime numberint x;...x = x * x; // x is now positive

Problem – these are just statements and are not checked for us in the same way that the type checker checks things automatically.

Type checking is limited in the support it provides. We want something more.

An assertion is a claim about the state of our program (values of variables and their relationship to each other) at various points in the execution of our program.

We are probably used to writing comments that states assertions:int p; // p is a prime numberint x;...x = x * x; // x is now positive

Problem – these are just statements and are not checked for us in the same way that the type checker checks things automatically.


Assertions are often the result of the execution of statements.

Used to argue that code fragments execute as expected.

Often referred to as pre-conditions and post-conditions (of statements).

Use logic to express or assert facts that we believe or expect to be true:Sort (A);// for all i, j such that 1 i j p// A[i] A[j]

PRECONDITION{ Code fragment }POSTCONDITION

Assertions are often the result of the execution of statements.

Used to argue that code fragments execute as expected.

Often referred to as pre-conditions and post-conditions (of statements).

Use logic to express or assert facts that we believe or expect to be true:Sort (A);// for all i, j such that 1 i j p// A[i] A[j]

PRECONDITION{ Code fragment }POSTCONDITION

ExampleExample

Consider:interface Queue {

// is the queue empty?boolean empty();

// add Item x to the end of the QueueQueue push (Item x);

// return and remove the Item at the head of the QueueItem pop ();

// how many elements in the Queue?int size();

}

Consider:interface Queue {

// is the queue empty?boolean empty();

// add Item x to the end of the QueueQueue push (Item x);

// return and remove the Item at the head of the QueueItem pop ();

// how many elements in the Queue?int size();

}

Adding AssertionsAdding Assertions

Questions: Can the method be called in all situations. If not,

when can it be called? Is there anything special about the result value or

state? It is always possible to call empty. It always returns

a true|false value. It is always possible to call push. The argument must

be of type Item (checked by type checker). The result is a non-empty queue whose size is one more than it was before the call. The call only makes sense if the queue is not already full.

Questions: Can the method be called in all situations. If not,

when can it be called? Is there anything special about the result value or

state? It is always possible to call empty. It always returns

a true|false value. It is always possible to call push. The argument must

be of type Item (checked by type checker). The result is a non-empty queue whose size is one more than it was before the call. The call only makes sense if the queue is not already full.

Adding AssertionsAdding Assertions

It is only possible to call pop when the queue contains at least one element. At the end of the call, the queue has one less element than at the start of the call.

It is always possible to call size. The result is an integer in the range 0 to the maximum possible size of the queue.

How do we describe this?

It is only possible to call pop when the queue contains at least one element. At the end of the call, the queue has one less element than at the start of the call.

It is always possible to call size. The result is an integer in the range 0 to the maximum possible size of the queue.

How do we describe this?

ExampleExample

size´() is the value of the size function BEFORE the current method was called – the old value if you will.

interface Queue {// is the queue empty?// PRE: noneboolean empty();//POST: none

// add Item x to the end of the Queue//PRE: size() < MAX_LENGTHQueue push (Item x);// POST: !(empty()) && size() = size´() + 1

size´() is the value of the size function BEFORE the current method was called – the old value if you will.

interface Queue {// is the queue empty?// PRE: noneboolean empty();//POST: none

// add Item x to the end of the Queue//PRE: size() < MAX_LENGTHQueue push (Item x);// POST: !(empty()) && size() = size´() + 1

ExampleExample

// return and remove the Item at the head of the Queue// PRE: !(empty())Item pop ();// POST: size() = size´() - 1

// how many elements in the Queue?// PRE: noneint size();// POST: 0 RESULT MAX_LENGTH

}

// return and remove the Item at the head of the Queue// PRE: !(empty())Item pop ();// POST: size() = size´() - 1

// how many elements in the Queue?// PRE: noneint size();// POST: 0 RESULT MAX_LENGTH

}


Q: How do we automatically enforce these assertions?

A: Write a class to help you. See sample Correctness.java in ~michael/CS351 on

esus.class Correctness {

public static void pre(boolean expr, Error ex) {…}public static void post (boolean expr) {…}public static void beginRequire () {…}public static void endRequire () {…}public static void beginEnsure () {…}public static void endEnsure () {…}

}

Q: How do we automatically enforce these assertions?

A: Write a class to help you. See sample Correctness.java in ~michael/CS351 on

esus.class Correctness {

public static void pre(boolean expr, Error ex) {…}public static void post (boolean expr) {…}public static void beginRequire () {…}public static void endRequire () {…}public static void beginEnsure () {…}public static void endEnsure () {…}

}


Pre-conditions must be within a beginRequire() … endRequire() bracket.

Post-conditions must be within a beginEnsure() … endEnsure() bracket.

This helps identify a post-condition that fails within a pre-condition evaluation compared to post-condition that fails at the end of a method. (Read the code).

May have multiple pre- and post-conditions within appropriate bracket.

May have a pre-conditions and post-conditions associated with any group of statements – it is not restricted to the entire method.

Pre-conditions must be within a beginRequire() … endRequire() bracket.

Post-conditions must be within a beginEnsure() … endEnsure() bracket.

This helps identify a post-condition that fails within a pre-condition evaluation compared to post-condition that fails at the end of a method. (Read the code).

May have multiple pre- and post-conditions within appropriate bracket.

May have a pre-conditions and post-conditions associated with any group of statements – it is not restricted to the entire method.

Assertions - ExampleAssertions - Example

public final Edge getEdge(int id) {Edge Result; // variable declarations// pre-condition checkingCorrectness.beginRequire();Correctness.pre( id >= 0 && id < numEdges(),

new IdOutOfRange() );Correctness.endRequire();Result = impl.getEdge(id); // evaluate result to be returned// post-condition checkingCorrectness.beginEnsure();Correctness.post( hasEdge(Result() == true );Correctness.endEnsure();return Result; // finally return the result.

}

public final Edge getEdge(int id) {Edge Result; // variable declarations// pre-condition checkingCorrectness.beginRequire();Correctness.pre( id >= 0 && id < numEdges(),

new IdOutOfRange() );Correctness.endRequire();Result = impl.getEdge(id); // evaluate result to be returned// post-condition checkingCorrectness.beginEnsure();Correctness.post( hasEdge(Result() == true );Correctness.endEnsure();return Result; // finally return the result.

}

ExpectationExpectation

Use assertions when developing your code. When testing your code, if a pre- or post-condition

fails you will get an error message. It helps you identify the failure and correct it far more quickly.

Use them in your project.

Use assertions when developing your code. When testing your code, if a pre- or post-condition

fails you will get an error message. It helps you identify the failure and correct it far more quickly.

Use them in your project.


Two basic techniques for attempting to produce programs without bugs: Testing: run the program on various sets of data

and see if it behaves correctly in these cases. Proving correctness: show mathematically that

the program always does what it is supposed to do.

Both techniques have their particular problems: Testing is only as good as the test cases

selected. A proof of correctness may contain errors.

Two basic techniques for attempting to produce programs without bugs: Testing: run the program on various sets of data

and see if it behaves correctly in these cases. Proving correctness: show mathematically that

the program always does what it is supposed to do.

Both techniques have their particular problems: Testing is only as good as the test cases

selected. A proof of correctness may contain errors.


A detailed formal proof is typically a lot of work. However, even an informal proof is helpful in clarifying your understanding of how a program works and in convincing yourself that it is probably correct.

Informal proofs are little more than a way of describing your understanding of how the program works – such proofs can easily be produced while writing the program in the first place Excellent program documentation!

A detailed formal proof is typically a lot of work. However, even an informal proof is helpful in clarifying your understanding of how a program works and in convincing yourself that it is probably correct.

Informal proofs are little more than a way of describing your understanding of how the program works – such proofs can easily be produced while writing the program in the first place Excellent program documentation!

Program CorrectnessProgram Correctness

Before looking at program proving in detail, there is something else that must be pointed out: A program can only be judged correct in relation

to a set of specifications for what it is supposed to do.

All programs do something correctly; the question is: does it do what it is supposed to do?

A really formal proof amounts to showing that a (mathematical) description of what the program does is the same as a (mathematical) description of what it should do.

Before looking at program proving in detail, there is something else that must be pointed out: A program can only be judged correct in relation

to a set of specifications for what it is supposed to do.

All programs do something correctly; the question is: does it do what it is supposed to do?

A really formal proof amounts to showing that a (mathematical) description of what the program does is the same as a (mathematical) description of what it should do.

Program CorrectnessProgram Correctness

Aspects of a program's correctness include:(1) Partial correctness: whenever the program

terminates, it performs correctly.(2) Termination: the program always

terminates.(1) + (2) ⇒ Program is totally correct.

Aspects of a program's correctness include:(1) Partial correctness: whenever the program

terminates, it performs correctly.(2) Termination: the program always

terminates.(1) + (2) ⇒ Program is totally correct.

Program Correctness ProofsProgram Correctness Proofs

Consider the handout "Proof of Program Correctness" and the function "exponentiate" on the first page.

function exponentiate (x: in integer) return integer is–– Evaluates 2**x, for x0 –– {1}

i, sum: integer;begin

sum := 1;–– sum = 2**0 ` –– {2}for i in 1 .. x loop

sum := sum + sum–– sum = 2**i, i>0 –– {3}

end loop;–– sum = 2**x, x 0

–– {4}return sum;

end exponentiate;

Consider the handout "Proof of Program Correctness" and the function "exponentiate" on the first page.

function exponentiate (x: in integer) return integer is–– Evaluates 2**x, for x0 –– {1}

i, sum: integer;begin

sum := 1;–– sum = 2**0 ` –– {2}for i in 1 .. x loop

sum := sum + sum–– sum = 2**i, i>0 –– {3}

end loop;–– sum = 2**x, x 0

–– {4}return sum;

end exponentiate;


{1} lists the goals of the function{2} asserts the initial value of "sum"We can prove {3} by induction.

The first time {3} is reached we havei = 1sum = 1 + 1 = 2 = 20 = 2i

Assume that the nth time {3} is reachedsum = 2n

then the (n+1)th time setssum' = sum + sum

= 2n + 2n = 2n+1

therefore {3} always holds.

{1} lists the goals of the function{2} asserts the initial value of "sum"We can prove {3} by induction.

The first time {3} is reached we havei = 1sum = 1 + 1 = 2 = 20 = 2i

Assume that the nth time {3} is reachedsum = 2n

then the (n+1)th time setssum' = sum + sum

= 2n + 2n = 2n+1

therefore {3} always holds.


If {4} is ever reached, there are two possibilities:a)The loop was never executed, in which case x=0,

and sum remains unchanged from {2}, i.e., sum = 1 = 20.

b)The loop was executed, in which case {3} was reached x times. Hence at {4}, sum = 2x.

See handout for further examples involving induction.

If {4} is ever reached, there are two possibilities:a)The loop was never executed, in which case x=0,

and sum remains unchanged from {2}, i.e., sum = 1 = 20.

b)The loop was executed, in which case {3} was reached x times. Hence at {4}, sum = 2x.

See handout for further examples involving induction.


For large programs, a major obstacle of program correctness proofs is an inability of the human to visualize the entire operation.

The remedy is modularization. As we can not write a large program without the aid of modularization and top-down design, we can not understand an algorithm and prove correctness unless it is modularized.

As a module is designed, an informal proof of correctness can be produced to show that the module matches the specification which describes its inputs and outputs.

For large programs, a major obstacle of program correctness proofs is an inability of the human to visualize the entire operation.

The remedy is modularization. As we can not write a large program without the aid of modularization and top-down design, we can not understand an algorithm and prove correctness unless it is modularized.

As a module is designed, an informal proof of correctness can be produced to show that the module matches the specification which describes its inputs and outputs.


A proof of correctness for a module relying on "lower level" modules is only interested in what they do and not how they do it. The lower level modules are assumed to meet the specifications which state what they do.

The specification of a module consists of two parts: specification of the range of inputs of the module. desired effect of the module.

In addition to pre- and post-conditions, a complex algorithm should contain assertions at key points. The more complex the algorithm, the more assertions that are necessary to bridge the gap between pre- and post-conditions.

The assertions should be placed so that it is fairly easy to understand the flow of control from one assertion to the next. In practice, this usually means placing at least one assertion in each loop.

Consider...

A proof of correctness for a module relying on "lower level" modules is only interested in what they do and not how they do it. The lower level modules are assumed to meet the specifications which state what they do.

The specification of a module consists of two parts: specification of the range of inputs of the module. desired effect of the module.

In addition to pre- and post-conditions, a complex algorithm should contain assertions at key points. The more complex the algorithm, the more assertions that are necessary to bridge the gap between pre- and post-conditions.

The assertions should be placed so that it is fairly easy to understand the flow of control from one assertion to the next. In practice, this usually means placing at least one assertion in each loop.

Consider...


procedure binary is –– binary search algorithmN: constant ...; –– some number 1}x: array (1..N) of float;key: float; L, R, K: integer; found: boolean;

begin key := ...;–– (x[I]x[J] iff 1IJN) and (X[1]keyx[N}) {0}L := 1; R := N; found := false;-- 1LRN and x(L)keyx(R) {1}while (LR) and (not found) loopK := (L+R) div 2;

–– 1LKRN and (x(L)keyx(R)) {2}found := (x(K) = key);if not found then –– x(K)key {3}

if key<x(K)then R := K–1; –– keyx(R) {4}else L := K+1; –– x(L)keyend if; –– {5}

–– x(L)keyx(R) {6}end if;

end loop;–– found= and (x(K)=key) {7}

end binary;[ is “key is present in the array x”]

procedure binary is –– binary search algorithmN: constant ...; –– some number 1}x: array (1..N) of float;key: float; L, R, K: integer; found: boolean;

begin key := ...;–– (x[I]x[J] iff 1IJN) and (X[1]keyx[N}) {0}L := 1; R := N; found := false;-- 1LRN and x(L)keyx(R) {1}while (LR) and (not found) loopK := (L+R) div 2;

–– 1LKRN and (x(L)keyx(R)) {2}found := (x(K) = key);if not found then –– x(K)key {3}

if key<x(K)then R := K–1; –– keyx(R) {4}else L := K+1; –– x(L)keyend if; –– {5}

–– x(L)keyx(R) {6}end if;

end loop;–– found= and (x(K)=key) {7}

end binary;[ is “key is present in the array x”]


{0} is a pre-condition describing what this module expects of its input.

{1} is a pre-condition describing the initial conditions before entering the loop.

{2} is an assertion true at that point for each iteration of the loop.

{3} is an assertion true whenever the if condition evaluates to true.

{4} holds if the then clause is executed. {5} holds if the else clause is executed. {6} holds after the if statement. It is true irrespective

of whether the then or else clause was executed. {7} is the post-condition of the module.

{0} is a pre-condition describing what this module expects of its input.

{1} is a pre-condition describing the initial conditions before entering the loop.

{2} is an assertion true at that point for each iteration of the loop.

{3} is an assertion true whenever the if condition evaluates to true.

{4} holds if the then clause is executed. {5} holds if the else clause is executed. {6} holds after the if statement. It is true irrespective

of whether the then or else clause was executed. {7} is the post-condition of the module.

TerminationTermination

A proof of partial correctness gives a reasonable degree of confidence in the results produced by an algorithm. Provided a result is output, we can be reasonable confident that it will be correct. However, a proof of partial completeness does not guarantee that a result is produced.

In order to provide such a guarantee, one must produce a proof of total correctness, i.e., it is also necessary to prove termination.

In order to prove termination it is necessary to show that conditions on loops are eventually satisfied, that recursive calls eventually stop, etc.

A proof of partial correctness gives a reasonable degree of confidence in the results produced by an algorithm. Provided a result is output, we can be reasonable confident that it will be correct. However, a proof of partial completeness does not guarantee that a result is produced.




A proof of partial correctness gives a reasonable degree of confidence in the results produced by an algorithm. Provided a result is output, we can be relatively confident that it will be correct. However, a proof of partial completeness does not guarantee that a result is produced.


A proof of partial correctness gives a reasonable degree of confidence in the results produced by an algorithm. Provided a result is output, we can be relatively confident that it will be correct. However, a proof of partial completeness does not guarantee that a result is produced.




Consider the following function:


Consider the following function:

function Ackermann(x, y: in integer) return integer is–– x and y must be nonnegative integersbegin –– Ackermann

if x = 0 then return (y+1);elsif y = 0 then return Ackermann((x-1), 1);else return Ackermann((x-1), Ackermann(x, (y-1)));end if;

end Ackermann;

ReadingReading

Rex Page, “Engineering Software Correctness”, ACM, FDPE’05, September 25, 2005, Tallinn, Estonia, pp 39-46.

Bertrand Meyer, ‘Applying “Design by Contract”’, IEEE Computer, October 1992, pp 40-51.

Cormac Flanagan, K. Rustan M. Leino, Mark Lillibridge, Greg Nelson, James B. Saxe, Raymie Stata, “Extended Static Checking for Java”, ACM, PLDI’02, June 17-19, 2002, Berlin, Germany.

Rex Page, “Engineering Software Correctness”, ACM, FDPE’05, September 25, 2005, Tallinn, Estonia, pp 39-46.

Bertrand Meyer, ‘Applying “Design by Contract”’, IEEE Computer, October 1992, pp 40-51.

Cormac Flanagan, K. Rustan M. Leino, Mark Lillibridge, Greg Nelson, James B. Saxe, Raymie Stata, “Extended Static Checking for Java”, ACM, PLDI’02, June 17-19, 2002, Berlin, Germany.

SummarySummary

Correctness is an important aspect of software quality.

Use appropriate tools (including programming language choice) to help.

Use assertions. Reason about the code to prove correctness when

necessary. It is easier and quicker to use assertions etc than it

is to test the code to demonstrate compliance with the requirements and specifications.

Still need to test the code – could have flaws in your logic!

Correctness is an important aspect of software quality.

Use appropriate tools (including programming language choice) to help.

Use assertions. Reason about the code to prove correctness when

necessary. It is easier and quicker to use assertions etc than it

is to test the code to demonstrate compliance with the requirements and specifications.

Still need to test the code – could have flaws in your logic!

correctness correctness. quality perceptions the perception of quality associated with your code is...

Documents