mdse-unit4b

Upload: mahedi-hasan

Post on 06-Apr-2018

214 views

Category:

Documents


0 download

TRANSCRIPT

  • 8/3/2019 MDSE-unit4b

    1/73

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    Software Verification and Validation

    J.C. Campos

    Department of Informatics/CCTCUniversity of Minho

    Model Driven Software EngineeringUnit 4 (lecture 2/2)

  • 8/3/2019 MDSE-unit4b

    2/73

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    Motivation

    Model-based Software Engineering (MBSE)

    ... by focusing on models that consolidate engineersunderstanding of a family of software products, engineers can

    create reusable assets that satisfy a wide variety of uses andeasily analyze changes in existing software to quickly compose

    or synthesize new solutions for subsequent products.

    http://www.sei.cmu.edu/mbse/is.html

    What is the assumption?

    That we can (easily and quickly) understand the models

  • 8/3/2019 MDSE-unit4b

    3/73

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    Understanding models...

    We need to be able to reason about (analyse) our models!

  • 8/3/2019 MDSE-unit4b

    4/73

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    Verification and Validation (V&V)

    Validation are we building the right systems?The software should fulfil requirements

    Verification are we building the system right?The software should conform to specification

    Main goals of V&V:Find defects in the system/designEstablish some level of confidence that system is/will be fitfor purpose

    A word of cautionTypically V&V cannot be used to establish completeabsence of defects

    Appropriate level of confidence depends on systemspurpose, user expectations, and marketing environment

  • 8/3/2019 MDSE-unit4b

    5/73

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    V&V Techniques

    Technical Reviewse.g., walk-throughs, inspections and audits

    Software TestingLevels of testing: e.g., unit testing, integration testing,

    system testing, regression testingTechniques: e.g., functional testing, structural testing,stress analysis, failure analysis

    Formal verificationmodel checking, theorem proving

    Simulation and Prototypingbegs the question of the correctness of thesimulation/prototype

    Requirements Tracingmaking sure all requirements are accounted for

  • 8/3/2019 MDSE-unit4b

    6/73

    I t d ti F l V&V t h i SMV d l h k A SE ti V&V IVY V&V t l f IS

  • 8/3/2019 MDSE-unit4b

    7/73

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    Formal V&V techniques

    Main advantagesMathematical rigour in the proof of properties of a model (orconsistency of an implementation against a model)Degree of confidence in results is absolute (assuming no

    mistakes are made!)Main limitations

    Need for a formal specification of requirementsTypically not applicable to non-functional requirements(performance, usability, reliability) (but we are working on

    it!)Two basic techniques

    Deductive methods (i.e. theorem proving)Algorithmic methods (i.e. model checking)

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY a V&V tool for IS

  • 8/3/2019 MDSE-unit4b

    8/73

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    Theorem Proving

    A deductive approach to verification

    Proofs are performed in the traditional mathematical style

    Some formal deductive system is used (axioms +deduction rules)

    Proofs progress by transforming (rewriting) a set ofpremises into a desired conclusion (using the axioms andthe deduction rules)

    Each proof step (i.e. each transformation of the premises)must be formally justified

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY a V&V tool for IS

  • 8/3/2019 MDSE-unit4b

    9/73

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    Proofs

    A small exampleConsider a logical language with sort S, and operators a, b, c : S and: S S Bool.Consider also a deductive system with:

    Axioms Deduction Rules(1) x,y,z:S x y y z x z(2) b c

    P,P QQ

    The proof that if a b then a c, can be carried out in the following way:

    1. a b premise2. a b, b c Axiom 23. a b, b c,a b b c a c Axiom 1 with x = a, y = b, z = c4. a c deduction rule and 3.

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

  • 8/3/2019 MDSE-unit4b

    10/73

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    A larger proof

    Sequent .3

    Sequent.2

    Sequent .4.1

    &Expand

    Skolemise

    Simplify

    Simplify&

    Simplify

    (grind)

    &

    (inst -4 "0" "length(l!1)-1")

    (prop) Simplify

    (grind)

    Expand

    (grind)

    Simplify Simplify

    (prop)

    Simplify

    (grind) (grind)

    (grind)

    (use "append_null" ("l" "l!1"))

    Simplify

    Simplify(grind)

    (prop)

    (case "l!1=null")

    (grind)

    Simplify

    (grind)

    (prop)

    (use "nth_append"...)

    Simplify

    Simplify

    (inst -3 "0" "length(l!1)-1")

    Simplify

    (prop)Simplify

    (use "length_append"...)

    Simplify

    (grind)

    Simplify

    (grind)

    (grind)

    Simplify

    (grind)

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

  • 8/3/2019 MDSE-unit4b

    11/73

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY a V&V tool for IS

    An example sequent

    Sequent .3

    [ 1] append(l, null) = l

    [ 2] r = null[ 3] pr(nth(l, length(l) 1)) pr(m)

    {4} (i,j : {k : int|k 0 k < length(queue(initial

    ))}) :i j pr(nth(l, i)) pr(nth(l,j))

    {5} current(initial) = value(car(l)){6} pr(car(l)) > pr(m){7} queue(initial) = l

    [ 8] queue(final) = append(l, cons(m, null))[ 9] current(final) = value(car(append(l, cons(m, null))))

    {1} l = null{2} l = null[3] get(current((final))) = m

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

  • 8/3/2019 MDSE-unit4b

    12/73

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY a V&V tool for IS

    Theorem proving advantages and drawbacks

    Deductive approaches enable the use of logics with greatexpressive power

    Detailed models can be writtenProofs are very detailed and can become unwieldy

    Theorem provers try to automate as much as possible ofthe different types of proof steps

    Learning curve can be steepEasier to write the models than to perform the proofs!

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

  • 8/3/2019 MDSE-unit4b

    13/73

    t oduct o o a & tec ques S ode c ec e S pe spect e o & a & too o S

    A few theorem provers

    PVS (http://pvs.csl.sri.com/)typed higher-order logic theorem proverintegration between theorem proving and CTL modelchecking

    HOL (http://hol.sourceforge.net/)

    higher order theorem proverintegration with SAT and BDD engines

    COQ (http://coq.inria.fr/)proof assistant for the Calculus of Inductive Constructions

    Larch Prover (http://nms.lcs.mit.edu/Larch/)theorem proving system for multisorted first-order logicdoes not attempt to fully automate the proofs

    Isabelle (http://isabelle.in.tum.de/)a generic proof assistant

    accepts a variety of formal calculi

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

  • 8/3/2019 MDSE-unit4b

    14/73

    q p p

    Model checking

    Proposed as an alternative to the use of theorem proversin concurrent program verification [CES86]

    Basic premise: a finite state machine model of a systemcan be subject to exhaustive analysis of its entire state

    space to determine what properties hold of the systemsbehaviour

    Properties are expressed in some temporal logic thatallows reasoning over the execution paths of the systemType of analysis

    Universality some condition is always true (an invariant!)Inevitability some condition will hold at some stage, nomatter what the behaviour of the system might bePossibility some condition might eventually hold at somestage, depending on the behaviour of the system

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

  • 8/3/2019 MDSE-unit4b

    15/73

    q p p

    Model checking advantages and drawbacks

    The analysis is fully automated (as opposed to theoremprovers reliance on user skills)

    The validity of a property is always decidable (as opposed

    to theorem provers undecidability problems)Some systems might not be specifiable by a finite statemachineState explosion problem might make verification infeasible

    time requirementsmemory requirements

    User skills needed to get the model right for verification

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

  • 8/3/2019 MDSE-unit4b

    16/73

    Addressing the state explosion problem

    State explosion still the major problem in model checkingExplicit state model checking

    each state in the model is explicitly represented

    state explosion problem not addressedSymbolic model checking [BCM90]automata represented by propositional formulacovered in previous session

    Bounded model checking (SAT solvers)

    paths explored up to some upper boundgood for shallow bugsnot so good for verification

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

  • 8/3/2019 MDSE-unit4b

    17/73

    Addressing the state explosion problem (more)

    On-the-fly model checkingautomata representation constructed as needed duringverification

    Partial order reduction

    reduces state space by exploring commutativity ofconcurrently executed transitions resulting in same statewhen executed in different ordersgood for asynchronous systems

    Abstraction

    Reducing the state space by eliminating unnecessary detailfrom the modelUse of non-determinismElimination of irrelevant behaviour

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

  • 8/3/2019 MDSE-unit4b

    18/73

    Abstraction

    New model should capture all relevant aspects of theoriginalStrongly preserving vs. weakly preserving abstractions

    [Dam96]false negatives a problem is reported that does not occurin the original model

    False positives inadequate abstractions will originatewrong conclusions

    a specification is reported true that does not hold in theoriginal model failure to detect a potential problem!

    Skill of modeller a major factor in using abstractions

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

  • 8/3/2019 MDSE-unit4b

    19/73

    Useful abstractions

    Some abstractions developed for GUI modelling, but that canbe generalised

    grouped fields attributes treated as a single entity, can besubstituted by a single attribute

    attribute elimination attributes that behave as constants,can be eliminated by hard-coding them into the model

    independent subsystems when a particular feature ofthe system does not affect the property under analysis, itcan be removed

    type reduction on many occasions it is possible toreduce types to a subset of meaningful values

    [Cam99, DCH97]

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

  • 8/3/2019 MDSE-unit4b

    20/73

    Beyond discrete event systems

    Model checking tools were initially developed to reasonabout discrete state systems

    Traditional model checking enables reasoning aboutrelative ordering of statesConsider modelling a mouse double click

    Real-time model checking (e.g. UPPAAL)

    Consider modelling component failure

    Stochastic model checking (e.g. PRISM)

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

  • 8/3/2019 MDSE-unit4b

    21/73

    Real-time model checking

    Models expressed as timed automataDiscrete time

    Time progressesin discrete stepsCan be modelled through tickscorresponding to clockcyclesFor synchronous systems which proceed in lock-step

    Continuous timeTime progression is continuousClock ranges are positive real values (infinite state space!)Events can happen at any point in timeClocks can advance by any positive real valueFor systems where real-valued delays are needed

    More complex than discrete models (smaller problemstackled)

  • 8/3/2019 MDSE-unit4b

    22/73

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

  • 8/3/2019 MDSE-unit4b

    23/73

    A few model checking tools

    NuSMV (http://nusmv.irst.itc.it/)BDD based symbolic model checker (CTL and LTL), SATsolver

    SPIN (http://www.spinroot.com/)on-the-fly model checker (LTL)

    UPPAAL (http://www.uppaal.com/)real-time model checker

    PRISM (http://www.prismmodelchecker.org/)probabilistic symbolic model checking

    HyTech(http://embedded.eecs.berkeley.edu/research/hytech/)

    Symbolic model checker for hybrid systems (discrete +continuous components)

    JavaPathFinder (http://javapathfinder.sourceforge.net/)

    explicit state software model checker for Java bytecode

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

  • 8/3/2019 MDSE-unit4b

    24/73

    The SMV input language

    Allows description of completely synchronous toasynchronous systems

    Modularized and hierarchical descriptions

    Finite data types: Boolean and enumerated

    Parallel-assignment syntax

    Non-determinism

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

  • 8/3/2019 MDSE-unit4b

    25/73

    A SMV module

    MODULE main

    VAR

    request: boolean;

    proc: {ready, busy};

    ASSIGN

    init(proc) := ready;

    next(proc) :=

    case

    proc=ready & request: busy;

    1: {ready, busy};

    esac;

    SPEC AG(request -> AF (proc = busy))

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

  • 8/3/2019 MDSE-unit4b

    26/73

    Variables

    VAR

    request: boolean;

    proc: {ready, busy};

    State is an assignment of values to a set of state variables

    Type of a variable boolean, scalar, user defined module,

    or array.

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

  • 8/3/2019 MDSE-unit4b

    27/73

    Transition relation (explicit)

    ASSIGN

    init(proc) := ready;

    next(proc) :=

    case

    proc=ready & request: busy;1: {ready, busy};

    esac;

    Assignment to initial state: init(proc) := ready;

    Assignment to next state (transition relation): next(proc):= ...

    SMV is a parallel assignment language

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

  • 8/3/2019 MDSE-unit4b

    28/73

    Case expressions

    case

    state=ready & request: busy;

    1: {ready, busy};

    esac;

    Guards are evaluated sequentially.

    The first guard evaluating to true determines the resulting

    valueIf no guard evaluated to true, result is 1

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

  • 8/3/2019 MDSE-unit4b

    29/73

    Logical expressions syntax

    state=ready & request

    logical not: !logical and: &

    logical or: |

    logical implication: >

    logical equivalence:

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

  • 8/3/2019 MDSE-unit4b

    30/73

    Another SMV example

    MODULE counter_cell(carry_in)VAR value : boolean;

    ASSIGN

    init(value) := 0;

    next(value) := value + carry_in mod 2;

    DEFINE carry_out := value & carry_in;

    FAIRNESS running

    MODULE main

    VARbit0 : counter_cell(1);

    bit1 : counter_cell(bit0.carry_out);

    bit2 : counter_cell(bit1.carry_out);

    SPEC AG AF bit2.carry_out

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

  • 8/3/2019 MDSE-unit4b

    31/73

    Defines

    DEFINE carry_out := value & carry_in;

    a macro definitioneach occurrence of carry_out is replaced by value &carry_in

    no extra BDD variable is generated for carry_out

    the BDD for value & carry_in becomes part of eachexpression using carry_out

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

  • 8/3/2019 MDSE-unit4b

    32/73

    Fairness constraints

    FAIRNESS running

    The module will run infinitely often

    General form: FAIRNESS ctl_formulaEach fairness constraint must be true infinitely often

    Model checker only explores paths satisfying fairnessconstraint

    If there are no fair pathsAll existential formulas are falseAll universal formulas are true

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

  • 8/3/2019 MDSE-unit4b

    33/73

    Modules

    VAR

    bit0 : counter_cell(1);

    bit1 : counter_cell(bit0.carry_out);

    bit2 : counter_cell(bit1.carry_out);

    Modules can be instantiated many times

    Each instantiation creates a copy of the local variables

    Each model has a module mainParameters are passed by reference.

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

  • 8/3/2019 MDSE-unit4b

    34/73

    Module composition

    Module are composed hierarchicallySynchronous compositionall assignments executed in parallel and synchronously

    Asynchronous compositiona step of the composition is a step by exactly one process

    variables not assigned in that process are left unchanged

    Asynchronous composition exampleMODULE main

    VAR

    gate1: process inverter(gate3.output);

    gate2: process inverter(gate1.output);gate3: process inverter(gate2.output);

    SPEC (AG AF gate1.output) & (AG AF !gate1.output)

    MODULE inverter(input)

    VAR output: boolean;

    ASSIGN

    init(output) := 0;

    next(output) := !input;

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

  • 8/3/2019 MDSE-unit4b

    35/73

    Transition relation (implicit)

    The ASSIGN approach can sometimes be too restrictive

    It is possible to define the transition relation axiomatically

    Three types of axioms:TRANS - axioms restricting the transition relationINIT - axioms defining initial statesINVAR - axioms defining invariants for the states of model

    It becomes possible to write unsound models!

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

  • 8/3/2019 MDSE-unit4b

    36/73

    Explicit vs. Implicit

    Explicit versionMODULE mainVAR request: boolean;

    proc: {ready, busy};

    ASSIGN

    init(proc) := ready;

    next(proc) :=

    case

    proc=ready & request: busy;

    1: {ready, busy};

    esac;

    SPEC AG(request -> AF (proc = busy))

    Implicit versionMODULE main

    VAR request: boolean;

    proc: {ready, busy};

    INIT proc = ready;

    TRANS (proc=ready & request) -> next(proc)=busy

    SPEC AG(request -> AF (proc = busy))

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    A f i i i V V

  • 8/3/2019 MDSE-unit4b

    37/73

    A software engineering perspective on V&V

    Software model checking has raised increasing interest

    Having the tools available is only half the story...We must

    Identify relevant issues for analysisBuild the modelExpress the properties/specificationsAnalyse the results

    The questionHow do we support these activities?

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    Id if i i

  • 8/3/2019 MDSE-unit4b

    38/73

    Identifying issues

    Depends on application area

    Depends on specific application

    Determines the type of technique that can be used

    For behavioural issues: model checkingGeneric desirable properties of the system will lead tospecific specifications to be checkedE.g., interactive systems:

    Previsibility does some specific user action always cause

    the same effect?Consistency do similar buttons behave in a similarfashion?...

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    B ildi d l

  • 8/3/2019 MDSE-unit4b

    39/73

    Building models

    Chosen technique/tool determines type of model

    Tool determines modelling language, or does it?

    Language imposes restrictions on what can be modelledApplication domain also conditions modelling approach

    Different domains already use their own modellinglanguages

    Can we use them?

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    A t t d P d ti S t IEC 60848 SFC

  • 8/3/2019 MDSE-unit4b

    40/73

    Automated Production Systems IEC 60848 SFC

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    A tomated Prod ction S stems some iss es

  • 8/3/2019 MDSE-unit4b

    41/73

    Automated Production Systems some issues

    Discrete (the program) + continuous(the plant) system

    Need to model the program + the plantNeed to abstract relevant physicalquantities in the plant (pressure,volume, temperature, ...)

    Need to consider control loop in theprogram not all states in the modelrelevant

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    Interactive Systems analysis of UI designs

  • 8/3/2019 MDSE-unit4b

    42/73

    Interactive Systems analysis of UI designs

    A lot of effort goes into GUI development

    It is also a hard to maintain aspect of the system

    Focus is on the users(?)Analysis can span from early models (very abstract) toimplementation

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    HCI models

  • 8/3/2019 MDSE-unit4b

    43/73

    HCI models

    A model of the interaction A model of system behaviour

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    Interactive Systems some issues

  • 8/3/2019 MDSE-unit4b

    44/73

    Interactive Systems some issues

    Ultimate goal is usability, a non functional requirement!This is a major issue and the use of formal verification in UIdesign is still open to debate however, in safety criticalareas, interest is growing

    How to consider users in the analysis?Can we build a user model?!

    Notion of action relevant (not present in original SMV)What about continuous interaction?

    Dynamic aspects of the systems relevant in most GUIsHard issue to deal with!

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    Expressing properties

  • 8/3/2019 MDSE-unit4b

    45/73

    Expressing properties

    Properties are expressed in some temporal logic (CTL,LTL)

    Observer automata can also be used

    Expressing a requirements in temporal logic is not alwaystrivial [DAC99]:

    Between the time an elevator is called at a floor

    and the time it opens its doors at the floor, the

    elevator can arrive at that floor at most twice.

    Encoding in LTL:A((callF open)

    ((atflooropen) U(open((atflooropen) U

    (open((atflooropen) U(open((atflooropen) U

    (open(atfloor U open))))))))))

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    Property specification patterns

  • 8/3/2019 MDSE-unit4b

    46/73

    Property specification patterns

    A property specification pattern [DAC99]:Described the essential structure of some aspect

    of a systems behaviour and provides expressions

    of this behaviour in a range of common

    formalisms.Patterns can be used to capture expert knowledge andhelp the process of verification:

    By helping write correct propertiesBy directing the analysis to relevant issues in the domain

    [DAC99] propose a generic set of patterns

    [Campos et al] propose a set of patterns for AutomatedProduction Systems

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    Dwyer et als patterns

  • 8/3/2019 MDSE-unit4b

    47/73

    Dwyer et al s patterns

    Occurrence patterns about occurrence of a givenevent/state

    AbsenceUniversalityExistenceBounded Existence

    Order patterns about relative order in which multipleevents/states occur

    PrecedenceResponseChain PrecedenceChain Responce

    http://patterns.projects.cis.ksu.edu/

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    Campos et als patterns

    http://patterns.projects.cis.ksu.edu/http://patterns.projects.cis.ksu.edu/
  • 8/3/2019 MDSE-unit4b

    48/73

    Campos et al s patterns

    Over 70 property specifications were analysedProperties were then aggregated into six classes:1 Possibility AG EF P, meaning P is always possible2 Fairness G F P, meaning P occur infinitely often3 Absence G (stable P) , meaning P does not occur

    (in stable states)4 Response G (P F(stable Q)), meaning after P, Qwill happen (in a stable state)

    5 Universality G P, meaning P occurs in every state6 Precedence A[P W S], meaning S always precedes P

    Four of the classes are also in [DwyerAC99] collectionHowever, the formulations do not coincide.Special variables used to restrict the analysis to stablestates

    Two of the classes are new

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    Possibility Pattern

  • 8/3/2019 MDSE-unit4b

    49/73

    Possibility Pattern

    Intent

    To express that some event or state is always possible throughout the execution of the system. Note that it does notrequire that the state or event actually happens in a specific execution of the model, only that it is possible that it will.

    Basic formulation

    CTL: AG EF P

    P is always possible.

    Stable formulation

    CTL: AG EF (stableP)

    P is always possible in a stable state.

    Examples

    [Ros03] uses this pattern to express the absence of dead code. The author writes a family of propertiesAG EF(etat = prei) where etat is a variable that captures the current state of the system and prei are thepossible execution steps. Hence, what each properties says is that a particular execution step is alwayspossible.

    [BHLL00] use this pattern to express that two events (s4 and sfc_top_s3.s8) have to synchroniserepeatedly: AG EF(s4 sfc_top_s3.s8) In fact, this is a mislead application of the pattern since provingthe property does not guarantee that the events ever synchronise in any specific execution of the model(see intent above and the next pattern).

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    Fairness Pattern

  • 8/3/2019 MDSE-unit4b

    50/73

    Fairness Pattern

    Intent

    To express that some event or state is repeatedly possible throughout the execution of the system. Unlike thepossibility pattern, this does require that the state or event actually happens in any specific execution of the model.

    Basic formulation

    LTL: G F P

    P happens repeatedly.

    Stable formulation

    LTL: G F (stableP)

    P happens repeatedly in stable states.

    Examples

    [Ros03] uses this pattern to express dead lock freedom: G F fdc, where fdcrepresents the end of theprocessing cycle. The objective is to expresses that the processing cycle repeatedly ends. Note however,that a behaviour which satisfies the above property is that where the system does not leave the fdccondition. Hence, the verification should be complemented with the analysis of the fairness of other steps.

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    Absence Pattern

  • 8/3/2019 MDSE-unit4b

    51/73

    Absence Pattern

    Intent

    To express that some event or state is not present throughout the execution of the system.

    Basic formulation

    CTL: AG (P)

    LTL: G (P)

    P is never possible.

    Stable formulation

    CTL: AG(stableP)

    LTL: G (stableP)

    P is never possible in a stable state.

    Examples

    [YTH01] use this pattern repeatedly to express both that a tank should not become empty and that it shouldnot overflow: AG(Lev = 0) AG(Lev = 6), where Levrepresents the Level of the tank.

    [MF01] use this pattern to express the following functional requirement: While the pressure is above 6.1bar, motor 1 should not be turned on and motor 2 should not be turned on., resulting inAG(rdy_plc (i1) (o1 o2)), where rdy_plcplays the role of our stablevariable, i1 represents thepressure condition, and o1 and o2are the states of the engines.

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    Response Pattern

  • 8/3/2019 MDSE-unit4b

    52/73

    Response Pattern

    Intent

    To express that some event or state will always lead (immediately, or at some point in the future) to another event orstate.

    Basic formulation

    CTL (base case): AG (P AF Q)

    LTL (base case): G (P F Q)

    CTL (immediate): AG (P AX Q)LTL (immediate): G (P X Q)

    P always leads to Q.

    Stable formulation

    CTL (base case): AG (P

    AF (stable

    Q))LTL (base case): G (P F (stableQ))

    CTL (immediate): AG (P A[stableU (stableQ)])

    LTL (immediate): G (P (stableU (stableQ)))

    P always lead to Q at some future stable state (or at the next stable state, in the immediate case).Note that, depending on the specific analysis being performed, we might wish to state that P must also beconsidered in a stable state only (Pstable).

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    Universality Pattern

  • 8/3/2019 MDSE-unit4b

    53/73

    y

    Intent

    To express that some event or state occurs in every state of the execution of the system. This pattern is in effect theopposite of the absence pattern.

    Basic formulation

    CTL: AG P

    LTL: G P

    P is always true.

    Stable formulation

    CTL: AG (stable P)

    LTL: G (stable P)

    P is true in all stable states.

    Examples

    [BHLL00] use this pattern to express that two events always synchronise: AG(s4 sfc_top_s3.s8), wheres4and sfc_top_s3.s8are the two event that should always be synchronised.

    [YSCK01] use this pattern to express that the temperature of a reactor always stays inside a desirablerange: AG(reactor.TREA > 0 reactor.TREA < 6)

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    Precedence Pattern

  • 8/3/2019 MDSE-unit4b

    54/73

    Intent

    To express that some event or state must occur before some other event or state. Conceptually this pattern is theopposite of the response pattern.The pattern uses the weak until operator. Because this operator is not usually supported by verification tools, weprovide alternative formulations with the more standard operators.

    Basic formulation

    CTL: A[ Q W P]

    CTL (alternative): E[P U (QP)]

    LTL: G Q W P

    LTL (alternative): F(Q) (Q U (P Q))

    P always precedes Q.

    Stable formulationCTL: A[ (stableQ) W (stableP]

    CTL (alternative): E[(stableP) U (stableQP)]

    LTL: G (stableQ) W (stableP)

    LTL (alternative): F(stableQ) ((stableQ) U (stableP Q))

    P always precedes Q in stable states.

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    Understanding results

  • 8/3/2019 MDSE-unit4b

    55/73

    g

    Model checkers attempt to providecounter examples (traces)

    These traces can be seen aspotential problem scenarios

    Problem might be real or a false

    negativeIt becomes necessary to analyse thetrace against the concrete system

    Traces can be tens of states long,

    with only some of the states beingrelevant

    Not much work in this area

    Some tools provide simulationcapabilities

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    Motivation for IVY

  • 8/3/2019 MDSE-unit4b

    56/73

    We start from a Software Engineering perspectiveThe device

    We acknowledge the need to consider usersWe are interested in formal/rigorous modelling andanalysis (and development?)

    We believe formal methods can help detect potentialusability problems

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    Our approach

  • 8/3/2019 MDSE-unit4b

    57/73

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    Our approach Context

  • 8/3/2019 MDSE-unit4b

    58/73

    Interaction with devices is frequently dependent on contextSystem results are dependent on relation between systemstate and environment (why not abstraction?)User actions are interpreted through implicit assumptionsabout context (ubicomp)

    How can we model context in the analysis of interactivesystems?We develop a separable model of context:

    Help clarify context related assumptions in the analysis ofthe device.

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    Our approach Users

  • 8/3/2019 MDSE-unit4b

    59/73

    Including users in the analysis: develop a user model?Can be hard workSee the HUM project:http://www.dcs.qmul.ac.uk/research/imc/hum/

    We favour a more lightweight approach:Derive user assumptions during the verification process

    Cognitive plausibility of trace is assessed

    False positives are eliminated by assuming the user will

    not perform the problem behaviourA separable model of user assumptions is included in theanalysis

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    The IVY workbench architecture

    http://www.dcs.qmul.ac.uk/research/imc/hum/http://www.dcs.qmul.ac.uk/research/imc/hum/
  • 8/3/2019 MDSE-unit4b

    60/73

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    The IVY workbench Model Editor

  • 8/3/2019 MDSE-unit4b

    61/73

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    The IVY workbench Properties Editor

  • 8/3/2019 MDSE-unit4b

    62/73

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    The IVY workbench Traces Analyser

  • 8/3/2019 MDSE-unit4b

    63/73

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    The IVY workbench Traces Analyser (markers)

  • 8/3/2019 MDSE-unit4b

    64/73

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    The MAL interactor language

  • 8/3/2019 MDSE-unit4b

    65/73

    Interactor: an object equipped with a rendering relationciteDukeH:93Attributes for stateActions for eventsMAL axioms for behaviour

    Modal Action Logic: a deontic modal logic with actionsthe usual propositional operatorsa modal operator [_]_ [start]state=ona reference event the initial state [] []state=offa deontic operator for permission per(start) state=off

    a deontic operator for obligation state=on obl(stop)Modality annotations to model the rendering relation

    Interactors can be aggregated/extended

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    An example the MCP panel

  • 8/3/2019 MDSE-unit4b

    66/73

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    An example context

  • 8/3/2019 MDSE-unit4b

    67/73

    Behaviour of the MCP depends on its context:

    ALT mode is activated near the target altitude.The context of the MCP will be affected by what it does:Different pitch mode will affect differently the aircraftsposition and velocities.

    We need to model this contextualinformation/interdependency:

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    Analysis (I)

  • 8/3/2019 MDSE-unit4b

    68/73

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    User assumptions

  • 8/3/2019 MDSE-unit4b

    69/73

    There is a limit to how much we can encode directly in theproperties.

    It becomes difficult to identify all assumptions being made.

    We will create a separate model for the user assumptions:interactor user

    aggregates

    MCP via ui

    axioms

    # user does not toggle the altitude capture off

    per(ui.toggleALT) -> !ui.ALT

    # user makes rational choices regarding the climb rate

    per(ui.crDial.set(-1)) -> (!ui.ALT | ui.plane.altitude>ui.ALTDial.needle)per(ui.crDial.set(0)) -> (!ui.ALT | ui.plane.altitude=ui.ALTDial.needle)

    per(ui.crDial.set(1)) -> (!ui.ALT | ui.plane.altitude

  • 8/3/2019 MDSE-unit4b

    70/73

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    Wrap up

  • 8/3/2019 MDSE-unit4b

    71/73

    V&V of software is a active research areaModel checking is attracting a lot of attentionResearch happens at two levels:

    Foundations and algorithmsApplications of the techniques

    An example of the second style of work was brieflyintroduced

    [email protected]

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    References

  • 8/3/2019 MDSE-unit4b

    72/73

    J. R. Burch, E. M. Clarke, and K. L. McMillan.

    Symbolic model checking: 1020 states and beyond.In Proceedings of the Fifth Annual IEEE Symposium on Logic In Computer Science, pages 428439. IEEEComputer Society Press, June 1990.

    S. Bornot, R. Huuck, B. Lukoschus, and Y. Lakhnech.

    Verification of sequential function charts using SMV.In International Conference on Parallel and Distributed Processing Techniques and Applications (PDPTA2000), volume V, pages 29872993. CSREA Press, June 2000.

    Jos C. Campos.Automated Deduction and Usability Reasoning.DPhil thesis, Department of Computer Science, University of York, 1999. Available as Technical ReportYCST 2000/9.

    E.M. Clarke, E.A. Emerson, and A.P. Sistla.

    Automatic verification of finite-state concurrent systems using temporal logic specifications.ACM Transactions on Programming Languages and Systems, 8(2):244263, 1986.

    M.B. Dwyer, G.S. Avrunin, and J.C. Corbett.Patterns in property specification for finite-state verification.In B. Boehm, D. Garlan, and J. Kramer, editors, 21st Intern. Conf. on Software Engineering (ICSE99), pages411420. IEEE Computer Society Press, 1999.

    D. Dams.

    Abstract Interpretation and Partition Refinement for Model Checking.PhD thesis, Eindhoven University of Technology, July 1996.

    Introduction Formal V&V techniques SMV model checker A SE perspective on V&V IVY - a V&V tool for IS

    References

  • 8/3/2019 MDSE-unit4b

    73/73

    Matthew B. Dwyer, Vicki Carr, and Laura Hines.

    Model checking graphical user interfaces using abstractions.In Mehdi Jazayeri and Helmut Schauer, editors, Software Engineering ESEC/FSE 97, number 1301 inLecture Notes in Computer Science, pages 244261. Springer, 1997.

    T. Mertke and G. Frey.

    Formal verification of PLC-programs generated from signal interpreted petri nets.In Proceedings of the 2001 IEEE Systems, Man, and Cybernetics Conference, pages 27002705. IEEE,

    2001.

    O. Rossi.

    Validation formelle de programmes ladder pour automates programmables industriels.PhD thesis, cole Normale Suprieure de Cachan, France, June 2003.

    S.H. Yang, O. Stursberg, P.W.H. Chung, and S. Kowalewski.

    Automatic safety analysis of computer-controlled plants.Computers and Chemical Engineering 25, 25:913922, 2001.

    S.H. Yang, L.S. Tan, and C.H. He.

    Automatic verification of safety interlock systems for industrial processes.Journal of Loss Prevention in the Process Industries, 14:379386, 2001.