Analyzing Tabular Requirements Specifications

Using Infinite State Model Checking

Tevfik Bultan

University of California, Santa Barbara

Constance Heitmeyer

Naval Research Laboratory

Outline

• SCR Toolset• Action Language Verifier• SCR to Action Language• Two Examples• Experiments

HISTORY OFSCR SCR APPROACH

1978: Heninger,Parnas+ publish A-7/SCR requirements documentTabular notation Events and conditionsMode classes and terms

1980s-early 1990s: SCR applied to a wide range of systemsTelephone networks (AT&T Bell Labs)Submarine communications (NRL) Control software for nuclear plants (Ontario Hydro)Avionics software (Grumman)

Early 1990s: Development of Four Variable Model and CoREParnas+ introduce and apply Four Variable ModelSoftw. Productivity Consortium develops CoRE method (based on SCR)Lockheed applies CoRE and SCR tables to C-130J flight program

1994-present: NRL develops formal SCR model and tools

SCR SCR SSoftware C Cost R Reduction

SPECIFY THE SYSTEMPRECISELY

Use a TABULARTABULARnotation with anexplicit formal semantics to specify therequired behavior

APPLY“CONSISTENCY

CHECKING”

Automatically check spec forsyntax/type errors,missing cases,nondeterminism,circular defs, etc.

SIMULATETHE

SYSTEM BEHAVIOR

Symbolicallyexecute the system basedon the (executable) req. specs

SCRSCR GOAL: MAKE ‘FORMALMETHODS’ PRACTICAL

As we move down the chain, we increaseassurance in the spec

INCREASING EFFORT,

INCREASED

EXPERTISE

VERIFYSPECS USING

THEOREM PROVING

VERIFYSPECS USING

MODEL CHECKING

Checkcriticalapplicationproperties

• Usable, scalable tabular notation• Integrated set of robust software tools

– light-weight tools whose use does not require math. sophistication/thm proving

– heavy-duty tools (e.g., theorem prover)

SCR REQUIREMENTS TOOLSETSCR REQUIREMENTS TOOLSET

THEOREM PROVER(TAME)

INVARIANTGENERATOR

MORE MORE ANALYSIS ANALYSIS TOOLSTOOLS

SCR BASICSCR BASICTOOLSTOOLS

PROPERTYCHECKER (Salsa)

CONSISTENCY

CHECKER

DEPENDENCYGRAPH BROWSER

SPECIFICATIONEDITOR

MODEL CHECKER

SIMULATOR

• Designed to help a developer specify and analyze requirements

• most mature tools• installed at 200+

organizations in industry, govt., and academia

• TAME - interface to PVS designed to prove properties of state machine models

TEST CASE GENERATOR

SOURCE CODE

GENERATOR

Research Prototypes Research Prototypes

Produces optimized C & Java codeProduces optimized C & Java code

requirements spec

modes

events

mon vars

cont vars

conditionsterms

Action Language Tool Set

Action Action LanguageLanguage

ParserParser

Action Action LanguageLanguageVerifierVerifier

Omega Omega LibraryLibrary

CUDDCUDDPackagePackage MONAMONA

Composite Symbolic LibraryComposite Symbolic Library

PresburgerPresburgerArithmeticArithmeticManipulatorManipulator

BDDBDDManipulatorManipulator

AutomataAutomataManipulatorManipulator

Action Action LanguageLanguageSpecificationSpecification VerifiedVerified

Counter Counter exampleexample

• A symbolic CTL model checker

• Supports infinite state model checking

– unbounded integer variables, Presburger arithmetic

• Composite symbolic representation:

– Polyhedra and automata representation for Presburger arithmetic, BDDs for Boolean and bounded variables, shape graphs for heap variables

• Uses conservative approximations for fixpoints

– widening and truncated fixpoints

• Supports parameterized verification

– parameterized constants

– parameterized modules

Composite Symbolic Library

CUDD Library OMEGA Library

Symbolic+intersect()

+union()+complement()+isSatisfiable()+isSubset()+pre()+post()

CompSym

–representation: list of comAtom

+intersect()+ union() • • •

BoolSym

–representation: BDD

+intersect()+union() • • •

IntSym

–representation: Polyhedra

+intersect()+union() • • •

compAtom

–atom: *Symbolic

Composite Symbolic Library, Extended

OMEGA Library

Symbolic+union()

+isSatisfiable()+isSubset()+forwardImage()

CompSym

–representation: list of comAtom

+ union() • • •

compAtom

–atom: *Symbolic

IntSymAuto

–representation: automaton

+union() • • •

IntSym

–representation: list of Polyhedra

+union() • • •

CUDD Library

BoolSym

–representation: BDD

+union() • • •

MONA

IntBoolSymAuto

–representation: automaton

+union() • • •

Composite Symbolic Library, Further Extended

CUDD Library OMEGA Library

Symbolic+union()

+isSatisfiable()+isSubset()+forwardImage()

CompSym

–representation: list of comAtom

+ union() • • •

BoolSym

–representation: BDD

+union() • • •

compAtom

–atom: *Symbolic

HeapSym

–representation: list of ShapeGraph

+union() • • •

IntSym

–representation: list of Polyhedra

+union() • • •

ShapeGraph

–atom: *Symbolic

SCR to Action Language

• SCR: Tabular specifications – Mode transition tables– Condition tables– Event tables

• Events

– @T(c) = c c’

• In Action Language: !c and c’

– @T(c) WHEN d = c c’ d

• In Action Language: !c and c’ and d

SCR to Action Language

• Each row in an SCR table can be translated to a formula on current and next-state variables

• The transition relation of a table is defined by the disjunction of the formulas that correspond to its rows

• The transition relation of the whole system is defined by the conjunction of the transition relations of its tables

• The initial expression in Action Language is used to define the initial states

• The restrict expression in Action Language is used to restrict the domains of the variables according to their type definitions

SCR to Action Language module main() enumerated CruiseCtrl {Off,

Inactive, Cruise, Override}; boolean IgnOn; enumerated Throttle { accel,

maintain, decel, off }; initial: CruiseMode=Off and !

IgnOn and Throttle=off; r1: CruiseCtrl=Off and !IgnOn and IgnOn’ and CruiseCtrl’=Inactive; r2: CruiseCtrl=Inactive and IgnOn and !IgnOn’ and CruiseCtrl=Off; CruiseCtrl: r1 | r2 ...; Throttle: ... main: CruiseCtrl & Throttle ...; spec: invariant ...endmodule

CruiseCtrlCruiseCtrl

Source Mode

Event Destination Mode

Off @T(IgnOn) Inactive

Inactive @F(IgnOn) Off

… … …

ThrottleThrottle

Mode Condition …

Cruise … …

… … …

Throttle accel …

Two Example SCR Specifications

Cruise Control System (CCS)– Monitored variables: IgnOn, Speed, Brake, Lever, EngRunning

– Mode class: CruiseCtrl– Term: DesiredSpd– Controlled variable: Throttle

• The transition relation in Action Language: – 26 Boolean and 8 integer variables

Safety Injection System (SIS)– Monitored Variables: Block, Reset, WaterPres– Mode Class: Pressure– Term: Overridden– Controlled Variable: SafetyInjection

• The transition relation in Action Language: – 14 Boolean and 2 integer variables

Verification of Invariants

• An invariant of CCS:

Brake Throttle=off– Action Language Verifier checks the corresponding CTL property:

AG(Brake => Throttle= off)

The fixpoint converges in two iterations, takes 0.76 seconds.

• Another invariant of CCS:

CruiseCtrl = Off IgnOn– Action Language Verifier checks the corresponding CTL property:

AG(CruiseCtrl = Off IgnOn)The fixpoint converges in one iteration, takes 0.32 seconds.

Transition Invariants

• In SCR some properties are specified as transition invariants:

EngRunning EngRunning’ Throttle’=off

• In the Action Language translation – We create a new Boolean variable for each transition invariant:

(!EngRunning and EngRunning’ => Throttle’=off) <=> b– We initialize this new boolean variable to true– Then check the property AG(b)

• For this transition invariant the Action Language Verifier converges in two iterations and takes 0.36 seconds.

Consistency Checking with ALV

• Disjointness check in SCR: – SCR tables should not assign more than one value to a dependant

variable in any system state– In each SCR table, for each mode, every pair of conditions should be

disjoint

• Disjointness check with Action Language Verifier:

EX( d=vd m=vm ) AX( m=vm d=vd)

We set the initial condition to true, and then check the above CTL property

With ALV disjointness check takes 8.07 seconds for CCS and 2.07 seconds for SIS

mM mM

Consistency Checking with ALV

• Coverage check in SCR:– Ensures that each SCR table defines a total function– The coverage check for a condition table reduces to checking that

the disjunction of the set of conditions in each row of the table is equivalent to true

• Coverage check in Action Language Verifier:– We create an auxiliary boolean variable for each variable defined by

a condition table– This boolean variable is set to true initially and it is set to false shen

no condition in the condition table evaluates to true– We set the initial condition to true and then we check that the

property AX(b) holds

• With ALV consistency check takes 0.06 seconds for CCS and 0.02 seconds for SIS

Use of Multiple Symbolic Representations

• Support for multiple symbolic representations in the Composite Symbolic Library is helpful when analyzing SCR specifications

• Automata vs. Polyhedra representations for integers

– Disjointness property for CCS• Polyhedra: 8.07 seconds, 6.6 Mbytes• Automata: 50.78 seconds, 2.7 Mbytes

– Disjointness property for SIS• Polyhedra: 2.07 seconds, 257.8 Mbytes• Automata: 1.13 seconds, 2.2 Mbytes

• It is hard to figure out the relative efficiency of these two representations statically

Explicit vs. Symbolic Model Checking

• For specifications with large number of initial and reachable states explicit state model checkers such as Spin can run out of memory

• For symbolic model checkers the important factor for performance is the size of the symbolic representation not the size of the state space

• In one version of the SIS specification that was verified with Spin the initial value of the WaterPres was specified as:

WaterPres = 14• We modified this initial condition to:

0 WaterPres 900• Interestingly, this more general initial condition does not influence

ALV’s verification performance – i.e., it does not effect the size of the symbolic representation

Finite vs. Infinite State Model Checking

• In a finite state version of the SIS specification the WaterPres variable is restricted to the following domain:

0 WaterPres 2000• We changed this to: 0 WaterPres • Among the 4 properties of SIS, 2 of them (S1 and S3) converge with

exact fixpoint computations for the infinite state case• For the other two properties exact fixpoint computation does not

converge, but we can still verify (or falsify) them using approximate fixpoint computations (using widening and truncated fixpoints)

• For the property S4 for the finite case the fixpoint computation takes:– 12.82 seconds, 11.9 Mbytes and 1,102 iterations

• For the property S4 for the infinite case the approximate fixpoint computation takes:– 0.77 seconds, 1.8 Mbytes and 11 iterations– And the result of the approximate fixpoint computation is strong

enough to prove the property

Parameterized Verification

• In addition to variables with infinite domains, ALV can also handle parameterized constants

• In the SIS specification, two constants determine the boundaries of the modes for the Pressure mode class

Low = 900 and Permit = 1000• We created a version of SIS in which Low and Permit are

paramterized constants that satisfy the following constraint:

0 Low < Permit • ALV able to verify the properties of the SIS specification on this

parameterized version– The counter-example for the property S2 in the original version is a

path that consists of 887 states– The counter-example for the property S2 in the parameterized

version is a path that consists of only 2 states

Abstraction and Symbolic Representation

Dependency analysis• SCR toolset has a dependency analyzer which is able to automatically

removes variables that do not influence a property• Using dependency analysis together with ALV improves the efficiency

of verification with ALV by reducing the number of variables in the specification (which reduces the size of the symbolic representation)

Data abstraction• SCR toolset also supports a data abstraction technique to deal with

large domains such as integers– The data variable is converted to an enumerated variable where

each value of the enumerated variable corresponds to a range of values in the concrete data domain

– This type of abstraction can be used instead of infinite state model checking in cases where the abstraction is strong enough to verify the property

Invariant Generation

• SCR toolset has an invariant generation tool which constructs invariants from the SCR tables

• ALV supports approximate reachability computations which also result with an invariant

• Shortcomings of using ALV’s forward fixpoint comptuation for generating invariants:– The quality of the invariant generated by ALV is sensitive to the

initial condition specification. If the initial condition is over-specified the generated invariant may be too restrictive.

– The invariant formula generated by a forward fixpoint computation may be a huge formula which may be hard to read and understand. SCR invariant construction algorithms generate invariants in certain forms which are easy to read and understand.

Model Checking vs. Theorem Proving

• In the experiments we conducted, verification with ALV seem to be more efficient than automated theorem proving using TAME

• However, TAME is capable of verifying a larger class of systems than ALV, so this performance improvement is mainly due to the fact that ALV is a more specialized verification tool.

• The user interaction in ALV is limited to choosing from a set options (the type of symbolic representation, the type of approximation, etc.), whereas TAME can be used interactively as a semi-automated proof tool.

• In a lot of cases where ALV is able to verify a property automatically TAME also automatically proves the property.

• ALV can generate counter-example behaviors for the properties which are violated.

• Although TAME does not generate counter-examples if the proof fails it generates a set of dead ends and problem transitions which correspond to a characterization of all the ways that a property can be violated.

Conclusions

• ALV would be a useful addition to the SCR toolset especially for verification of infinite state and/or parameterized specifications

• Using ALV for consistency checking could be an overkill since SCR toolset already supports efficient consistency checking tools which are specialized for this problem

• Two examples we analyzed are not very big. For large specifications infinite state model checking techniques maybe too expensive– We need to conduct experiments on larger systems

• Usability of ALV is not as good as the SCR toolset. SCR toolset generates easy to understand feedback for software developers.