1

Formal Formal MethodsMethods

Lawrence ChungDepartment of Computer ScienceThe University of Texas at Dallas

2

Desirable Properties of Desirable Properties of SpecificationsSpecifications

Clarity Clarity – void of any ambiguities– void of any ambiguities Completeness Completeness – nothing important missing– nothing important missing Consistency Consistency – no contradictions; no inconsistencies– no contradictions; no inconsistencies … … no over-specification, no mixed levels of abstractionno over-specification, no mixed levels of abstraction

Objectives – achieve the desirable properties of a specificationObjectives – achieve the desirable properties of a specification Formal semantics enables requirements or design to be interpreted in Formal semantics enables requirements or design to be interpreted in

only one way, hence enhancing clarity by eliminating ambiguity that only one way, hence enhancing clarity by eliminating ambiguity that often occurs with a natural language (e.g., English) or a graphical often occurs with a natural language (e.g., English) or a graphical notation.notation.

Consistency is enhanced by mathematical proofs - Facts and inference Consistency is enhanced by mathematical proofs - Facts and inference rules do not lead to counter facts.rules do not lead to counter facts.

Formal Formal SpecificationSpecification

3

Formal Specification Languages

Three primary components: Three primary components: a a syntaxsyntax that defines the specific notation with which that defines the specific notation with which

the specification is represented - often based on a the specification is represented - often based on a syntax that is derived from standard syntax that is derived from standard set theoryset theory notation and notation and predicate calculuspredicate calculus..

semanticssemantics to help define a "universe of objects" to help define a "universe of objects" [WIN90] that will be used to describe, and interpret, [WIN90] that will be used to describe, and interpret, the systemthe system

a set of relations/a set of relations/rulesrules which should be obeyed by all which should be obeyed by all objects and propositions.objects and propositions.

example:

example:

example:

4

Basics: A Brief ReviewBasics: A Brief ReviewSet TheorySet Theory

Sets:Sets: oneTwoThree == {1, 2, 3}oneTwoThree == {1, 2, 3} Person == {Adam, Eve}Person == {Adam, Eve}

S: P X == 2 S: P X == 2 XX --- S is a set of X’s powerset, i.e., the set of all subsets of X--- S is a set of X’s powerset, i.e., the set of all subsets of X

oneTwoThreeSet == P oneTwoThree == P {1, 2, 3} == oneTwoThreeSet == P oneTwoThree == P {1, 2, 3} == ?? personSet == P person == P {Adam, Eve} == personSet == P person == P {Adam, Eve} == ?? |P X| ==|P X| == ? ?

From [Chung, SA Lecture Notes]

5

BasicsBasicsSet TheorySet Theory

Sets (cont’d)Sets (cont’d) x memberOf Sx memberOf S

?? 1 memberOf {1, 2, 3} 1 memberOf {1, 2, 3}

?? 1 memberof P {1, 2, 3} 1 memberof P {1, 2, 3}

?? {1} memberof P {1, 2, 3} {1} memberof P {1, 2, 3}

?? Adam memberOf P Person Adam memberOf P Person

?? Adam memberOf Person Adam memberOf Person

?? {Adam, Eve} memberOf P Person {Adam, Eve} memberOf P Person

6

BasicsBasicsSet TheorySet Theory

Sets (cont’d)Sets (cont’d) S subsetOf S’S subsetOf S’

?? 1 subsetOf {1, 2, 3} 1 subsetOf {1, 2, 3}

?? {1, 2} subsetOf P {1, 2, 3} {1, 2} subsetOf P {1, 2, 3}

?? {{1, 2}} subsetOf P {1, 2, 3} {{1, 2}} subsetOf P {1, 2, 3}

?? Adam subsetOf P Person Adam subsetOf P Person

?? Adam subsetOf Person Adam subsetOf Person

?? Person subsetOf Person Person subsetOf Person

?? {Person} subsetOf P Person {Person} subsetOf P Person

7

BasicsBasicsSet TheorySet Theory

Sets (cont’d)Sets (cont’d) S X S’ (cross/cartesian product)S X S’ (cross/cartesian product)

oneTwoThree X person == {1, 2, 3} X {Adam, Eve} ==oneTwoThree X person == {1, 2, 3} X {Adam, Eve} ==

{{1, Adam}, {1, Eve}, {2, Adam}, {2, Eve}, {3, Adam}, {3, Eve}}{{1, Adam}, {1, Eve}, {2, Adam}, {2, Eve}, {3, Adam}, {3, Eve}}

?? {1, 2} subsetOf {1, 2} X {1, 2} {1, 2} subsetOf {1, 2} X {1, 2}

S U S’, S intersect S’, S\S’, etc. (skip)S U S’, S intersect S’, S\S’, etc. (skip)

8

BasicsBasicsFunctionsFunctions

FunctionsFunctions dom f --- The set of values x for which f(x) is defineddom f --- The set of values x for which f(x) is defined

f(x) = x f(x) = x 2 2 , dom f = {n memberOf N| 1 <= n <= 5}, dom f = {n memberOf N| 1 <= n <= 5} ran f --- The set of values yielded by f(x), ran f --- The set of values yielded by f(x),

where x memberOf dom fwhere x memberOf dom f

ran f = ran f = ?? f: X -> Y f: X -> Y --- f is a total function from X to Y--- f is a total function from X to Y

i.e., f is defined for all x memberOf dom(f), i.e., i.e., f is defined for all x memberOf dom(f), i.e., dom(f) = Xdom(f) = X

f: X -|-> Y --- f is a partial function from X to Yf: X -|-> Y --- f is a partial function from X to Y i.e., f is defined for some values in Xi.e., f is defined for some values in X

if f(x) = 1/x, if f(x) = 1/x, ?? dom(f) = Z dom(f) = Z

?? spouse: Person -> Person spouse: Person -> Person

9

BasicsBasicsFunctionsFunctions

Functions (cont’d)Functions (cont’d) (lambda x: T . t) returns the value of the term t(lambda x: T . t) returns the value of the term t

(lambda x: N . X (lambda x: N . X 2 2 ) 5 == 25) 5 == 25

(lambda x: N . (X (lambda x: N . (X 2 , 2 , 1/x) == 1/x) == ??

(lambda x, y: N . (X (lambda x, y: N . (X 2 +2 + y, y - y, y - 1/x) 5 1 == 1/x) 5 1 == ??

10

BasicsBasicsFirst Order LogicFirst Order Logic

First Order LogicFirst Order Logic Logical connectives:Logical connectives: AND, OR, NOT, =>, <=> AND, OR, NOT, =>, <=> QuantifiersQuantifiers

?? Exists n: N . n = n Exists n: N . n = n 22

?? Exists p: Person . P == father (Adam) Exists p: Person . P == father (Adam)

?? Forall i: N . I Forall i: N . I 22 >= I >= I

?? Forall i, j: N . i > j => i Forall i, j: N . i > j => i22 > j > j 22

?? Forall x, y: Person, x == spouse(y) <=> y == spouse(x) Forall x, y: Person, x == spouse(y) <=> y == spouse(x)

11

Introduction to ZIntroduction to Z Based on typed set theory and first order logicBased on typed set theory and first order logic SchemasSchemas

A schema consists of a set of declarations of variables and a A schema consists of a set of declarations of variables and a predicate constraining these variables (i.e., state space and predicate constraining these variables (i.e., state space and operations)operations)

----- BirthdayBook --------------------------------------------- BirthdayBook ----------------------------------------| known: P Person| known: P Person| birthday: Person -|-> Date| birthday: Person -|-> Date----------------------------------------------------------------------------------------------------------------------------------| known = dom birthday| known = dom birthday----------------------------------------------------------------------------------------------------------------------------------

One possible state:One possible state:known = {Adam, Caine, Eve}known = {Adam, Caine, Eve}birthday = {Adam |-> Apr/01, Eve |-> Apr/01}birthday = {Adam |-> Apr/01, Eve |-> Apr/01}

12

BlockHandler in ZBlockHandler in Z

——————BlockHandler——————————————BlockHandler——————————————used, freeused, free : : PP BLOCKSBLOCKSBlockQueueBlockQueue : seq : seq PP BLOCKSBLOCKS——————————————————————————————————————————————usedused >> freefree = = \\ usedused << freefree = = AllBlocksAllBlocks ii: dom : dom BlockQueueBlockQueue BlockQueueBlockQueue ii ## usedused i, ji, j : dom : dom BlockQueueBlockQueue ii ≠≠ jj => => BlockQueue iBlockQueue i >> BlockQueue jBlockQueue j = = \\ ————————————————————————

The following example of a schema describes the state of The following example of a schema describes the state of the block handler and the data invariant:the block handler and the data invariant:

See Section 28.6.2 for further expansion of the specification

notation described in Table 28.2notation described in Table 28.2

13

14

¬¬

15

Larch: LinearContainerLarch: LinearContainerLinearContainer (E, C): trait

introducesnew: -> Cinsert: C; E -> Cfirst: C -> Erest: C -> CisEmpty: C -> BoolisIn : E;C -> Bool

assertsC generated by new, insertC partitioned by first, rest, isEmptyforall c: C, e, e’: E

first(insert (new, e)) == erest(insert (new, e)) == newisEmpty(new)~isEmpty(insert (c, e))~isIn (e, new)isIn (e, insert (c, e’)) == e = e’ | isIn (e, c)

impliesforall c: C, e: E

isEmpty (c) => ~isIn (e, c)converts isIn, isEmpty

16

17

Larch: QueueLarch: QueueQueue : trait

introducesnew -> Qinsert: Q, E -> Qfirst: Q -> Erest: Q -> QisEmpty: Q -> BoolisIn: Q, E -> Bool

constrains Q so thatQ generatedby [ new, insert ]for all q: Q, e, e1: E

first(insert (new), e)) =elfirst (insert(q, e)) = if isEmpty(q) then e else first(q)rest(insert(q, e)) = if isEmpty(q) then new else

insert(rest(q), e)isEmpty(new) = trueisEmpty(insert(q, e)) = falseisIn(new, e) = falseisIn(insert(q, e), e1) = (e = e 1 | isIn(q, e1))

18

Object Constraint Language (OCL)

A formal notation for adding more precision to UML specificationsA formal notation for adding more precision to UML specifications

All of the power of logic and discrete mathematics is availableAll of the power of logic and discrete mathematics is available

Only ASCII characters (not conventional mathematical notation) Only ASCII characters (not conventional mathematical notation) - See - See Table 28.1 for summary of OCL notationTable 28.1 for summary of OCL notation

Like an OOPL, an OCL expression involves operators Like an OOPL, an OCL expression involves operators operating on objects.operating on objects.

Each complete expression is a predicate, i.e., Boolean Each complete expression is a predicate, i.e., Boolean valued (true or false). valued (true or false).

Objects can be instances of the OCL Objects can be instances of the OCL Collection Collection class, class, of which of which SetSet, , SequenceSequence and Bag are subclasses. and Bag are subclasses.

19

OCL Quick ReferenceOCL Quick Reference[http://www.eoinwoods.info/doc/ocl_quick_reference.pdf][http://www.eoinwoods.info/doc/ocl_quick_reference.pdf]

Invariants:context Class1

inv attr1 > 100context Class2

inv secondInvariant: attr1 < 10

Pre and Post Conditions:context Class1::method1(v1: Integer) : Integerpre valueIsLargeEnough: v1 >= 100post: attr1 >= attr1@pre + 100 and result > v/10

Can have as many “inv” statements as required, optionally named, and the resulting invariant is their conjunction (“i1 and i2 and ...”).

The “@pre” notation refers to the “before” state (VDM’s “hook” notation) and “result” is a reserved word for the result of the operation (if it has one).

Query Definitionscontext Class1::query1(v: Integer) : Integerbody: v + 100 + attr1

Queries don’t change state and so pre/post-condition form isn’t used to define them. Instead, they use a single expression in a “body” statement.

20

OCL Quick ReferenceOCL Quick Reference[http://www.eoinwoods.info/doc/ocl_quick_reference.pdf][http://www.eoinwoods.info/doc/ocl_quick_reference.pdf]

Loop Operationscollect(expr) Returns a bag containing the value of the expression for each of the items in the collection (e.g.

items->collect(value)). A simpler synonym for this operation is the period (“.”) operator (e.g. items.value).

forAll(expr) Does expression expr hold for all items in the collection?

select(expr) Returns the sub-collection of items in a collection for which expression expr holds.

set1->select(attr1 > 10) These two examples are equivalent.

set1->select(i | i.attr1 > 10) “i” is an “iterator” variable and can be thought of as being set to each of the elements of set1 in turn.

Collection Manipulation Operationscount(obj) Number of times that obj appears in a collection.

excludes(obj) Does count(obj) = 0 ?

excludesAll(coll) Does count(obj) = 0 hold for all items in collection coll?

first() The first item in the ordered collection.

includes(obj) Is count(obj) > 0 ?

includesAll(coll) Does count(obj) > 0 hold for all items in collection coll?

isEmpty() Is collection’s size() = 0 ?

size() Number of items in the collection.

Operations are applied to collections using the “->” operator (e.g. items->isEmpty(), where “items” is a collection).

21

ExamplesExamplesPerson

22

Examples: OCL and CodingExamples: OCL and CodingPerson

Class sex{ static final int MALE = 1; static final int FEMALE = 2;}

class Person{ public int sex; public Person spouse;

public void marry(Person p) {

assert p != this;assert p.sex != this.sex;assert this.spouse = null &&

p.spouse = null;this.spouse = p;p.spouse = this;

}}

class Person{ public int sex; public Person spouse;

public void marry(Person p) throws ArgumentException { if(p == this) {

throw new ArgumentException("cannot marry self");}if(p.sex == this.sex) { throw new ArgumentException("spouse is same sex");}if((p.spouse != null || this.spouse != null) { throw new ArgumentException("already married");

} this.spouse = p; p.spouse = this; }}

Defensive programming style

23

ExamplesExamplesBanking

24

Examples: OCL and CodingExamples: OCL and CodingBanking

class Account { private float balance = 0;

public void withdraw(float amount) { assert amount <= balance;

balance = balance - amount; }

public void deposit(float amount) { balance = balance + amount;

}

public float getBalance() { return balance;

}}

public void testWithdraw() { Account account = new Account();

account.deposit(500);

float balanceAtPre = account.getBalance();

float amount = 250;

account.withdraw(amount);

assertTrue(account.getBalance() == balanceAtPre . amount);}

25

ExamplesExampleshttp://maude.sip.ucm.es/itp/ocl/tutorial.html

Train

context Train inv atLeastOneWagon:self.wagon->size() >= 1

context Wagon inv belongToTheSameTrain:self.succ->notEmpty() implies self.succ->forAll(w | w.train = self.train)

context Train inv sameNumberOfWagons:Train.allInstances->forAll(t1 | (self.wagon->size() = t1.wagon->size()))

context Wagon inv notInCyclicWay:(Wagon.allInstances)->forAll(w2 | self <> w2 implies not ((self.succ)->includes(w2) and (w2.succ)->includes(self))

simplification?

26

Specifications vs. ModelingSpecifications vs. ModelingNancy Leveson wrote: Nancy Leveson wrote:

<<< Is there data that software developed without FM does not make money for the customers? <<< Is there data that software developed without FM does not make money for the customers? >>> For many customers, yes. See: J. Johnson, “Chaos: The Dollar Drain of IT Project Failures,” Application Development Trends, >>> For many customers, yes. See: J. Johnson, “Chaos: The Dollar Drain of IT Project Failures,” Application Development Trends, pp.41-47, Jan. 1995.( $80B/yr soon adds up to real money). That's not to say that FMs alone would have prevented this waste (although pp.41-47, Jan. 1995.( $80B/yr soon adds up to real money). That's not to say that FMs alone would have prevented this waste (although that's not what you asked); probably most of it was the fault of people ignoring other aspects of software engineering. that's not what you asked); probably most of it was the fault of people ignoring other aspects of software engineering.

Sorry, I stated my question wrong. Of course there have been development failures -- but most of them did not use good software engineering Sorry, I stated my question wrong. Of course there have been development failures -- but most of them did not use good software engineering or had other problems. It is a leap to suggest that because projects fail, that formal methods would have saved the day (which is what or had other problems. It is a leap to suggest that because projects fail, that formal methods would have saved the day (which is what the original message implied). the original message implied).

<<< As long as the costs are great and the arguments are only anecdotal, ... <<< As long as the costs are great and the arguments are only anecdotal, ... >>> Cheap shot: such papers as I have seen published by people using FMs in i ndustry have reported that the use of FMs (short of full >>> Cheap shot: such papers as I have seen published by people using FMs in i ndustry have reported that the use of FMs (short of full proof) does not cost mor e than other methods, even if all the savings in maintenance are ignored. See, f or example, Anthony Hall's proof) does not cost mor e than other methods, even if all the savings in maintenance are ignored. See, f or example, Anthony Hall's IEEE Software paper "Seven Myths of Formal Method s". IEEE Software paper "Seven Myths of Formal Method s".

It was not meant as a cheap shot. Simply a statement that business people want evidence before investing a lot of money. I've read Halls' It was not meant as a cheap shot. Simply a statement that business people want evidence before investing a lot of money. I've read Halls' paper, and I did not agree with his argument. Retraining employess and changing tools in any way is an enormous investment for paper, and I did not agree with his argument. Retraining employess and changing tools in any way is an enormous investment for industry. Most of the successful uses of formal methods in industry have involved a university professor and/or Ph.D. students (like industry. Most of the successful uses of formal methods in industry have involved a university professor and/or Ph.D. students (like CICS, for example). The question is whether, after the Ph.D. leaves, does the company still use formal methods? Does the IBM Hursley CICS, for example). The question is whether, after the Ph.D. leaves, does the company still use formal methods? Does the IBM Hursley group now use Z on all their software projects? After Dave Garlan left Textronics, I was told they never used formal methods again. group now use Z on all their software projects? After Dave Garlan left Textronics, I was told they never used formal methods again. Formal methods will never be widely accepted until a large number of practitioners know and use it, not simply university people and a Formal methods will never be widely accepted until a large number of practitioners know and use it, not simply university people and a few consulting firms such as Martyn's. Without realistic comparative data, I don't think a large number of people are going to buy. few consulting firms such as Martyn's. Without realistic comparative data, I don't think a large number of people are going to buy.

I'm not trying to be a naysayer here -- my research involves formal methods. I'm just suggesting that it has been 30 years now and we do not I'm not trying to be a naysayer here -- my research involves formal methods. I'm just suggesting that it has been 30 years now and we do not see a lot of formal methods use in industry. If that is to change, then there needs to be a realistic assessment of why and an attempt see a lot of formal methods use in industry. If that is to change, then there needs to be a realistic assessment of why and an attempt made to change those factors. I just don't think more sales talk without hard evidence will do it. made to change those factors. I just don't think more sales talk without hard evidence will do it.

Nancy Nancy

http://www.cs.york.ac.uk/hise/safety-critical-archive/2002/0171.html

The specification language does count. There is a concept called "semantic distance" in the human-interface literature. The closer the specification notation matches the reviewers' mental models of the system, the more likely they will be able to understand and find errors in the specification. It is not by coincidence that the most popular use of "formal methods" in industry involves specification languages that are simply control-block diagrams that are then automatically translated into code.

http://www.cs.york.ac.uk/hise/safety-critical-archive/2002/0135.html