verification case studies with objectcheck

Verification Case Studies with ObjectCheck

Fei Xie

(Joint work with James C. Browne, Robert P. Kurshan, and Vladimir Levin)

Presentation at Microsoft Research, April 10, 2003

2

Presentation Outline

• Overview and Architecture of ObjectCheck

• Modeling and Verification of a TinyOS Modeling and Verification of a TinyOS Run-time Image with ObjectCheckRun-time Image with ObjectCheck

• More Case StudiesMore Case Studies

• Summary and Future WorkSummary and Future Work

• Work Built on ObjectCheck

3

ObjectCheck Overview

• An integrated development, validation, and mode-checking environment for software system designs;– System designs are specified in xUML;

– xUML is an executable subset of UML;

• Developed in conjunction with – FormalCheck (Robert P. Kurshan, et. al.);

– SDLCheck (Vladimir Levin and Husnu Yenigun);

– Goal: Software/hardware co-design and co-verification;• Sub-goal: Model checking of software system designs in xUML.

4

Executable UML (xUML)

• Has well-defined Execution Semantics;

• Utilizes UML Action Semantics recently adopted by OMG;

• Can be compiled to procedural codes;

• Tools provided by:– Project Technologies;– Kennedy Carter;– Hyperformix (SES);– …

5

Architecture and Workflow of ObjectCheck

Property Specification Interface xUML IDE Error Visualizer

xUML-to-S/R Translator Error Report Generator

COSPAN Model Checker

S/R ModelS/R Query

Error Report

Error Track

Designer

xUML ModelProperty

6





• Summary and Future Work Summary and Future Work


7

Case Study on TinyOS

• A run-time image of TinyOS, a component-based operating system for networked sensors;

• xUML models for TinyOS components have been constructed from their C source codes to enable:– Model-driven development on design level;– Software/hardware co-design and co-verification;

• Two properties are to be checked:– Repeated transmission on physical network;– No consecutive 0’s in any sequence-number sequence.

8

More on TinyOS

• An OS for networked sensors that have– Limited computation ability and energy supply;

– High concurrency requirements;

– Diversities in designs and usages;

– High reliability requirement;

• Is component-based– Run-time images are specialized for different sensors;

– Only necessary components are selected and composed.

• More information -- http://webs.cs.berkeley.edu/tos

http://webs.cs.berkeley.edu/tos








9

Step-by-Step Demonstration

Designer


Error ReportxUML ModelProperty


Error TrackS/R ModelS/R Query


18


Designer






20


Designer






24


Designer






32


Designer






35


Designer






43

Statistics from Model Checking “Repeated Output” Property

• Four model checking runs with different combinations of reduction algorithms – Start at the same time on the same host;– The host is a SUN with 8 CPUs and 2GB memory.

SPOR SMC Memory Usage Time Usage

Off Off 596M 19370S

Off On 80M 1384S

On Off 596M 17438S

On On 102M 1379S

Conclusion: SMC helps and SPOR doesn’t.

44







45

Model Checking of NASA robot Controller

• A typical control-intensive embedded system;• Conducted by Natasha Shyrigina using ObjectCheck;

– 37 properties were checked.– 22 properties were successfully checked.– 6 bugs were found.

consists of

defines position of

specifies

is a part of

R15

GlobalRepresentation

. number_of_joints

. type of search

. abort_var….

R1

R11

Follows to thespecified

R2

R5

R4

Kinematics

InverseKinematics

OSCAR Library

R7

R6

Trajectory (TJ)* TJ_ID. position. final_position. EE_ID(R3)

TrajectoryPoint (TP)*TP_ID. TJ_ID (R4). point

R3

R9

R10

has

can havedifferent

can have anumber of

Is A

is a component of

requires

OSCAR Libraries* OS_ID. TC_ID (R10). status* link-ID

R8

ForwardKinematics

Arm (A)* Arm_ID. arm_status

is configured by

End-Effector (EF)* EE_ID. Current position. Limit. status. end_position. ee_reference

JointConfiguration (JC)*JC_ID. Joint_ID(R7). direction. trial_angle. type. NoinSet. TS_counter. Status

3

Joint (J)* Joint ID. current_angle. limit_Max.. limit_Min. EE_ID (R2). reference. Arm_ID (R1) . DT_ID (R11). JointStatus

is interfaced with

TrialConfiguration (TC)*TC_ID. DT_ID(R8). configuration. validSolutions. SS_ID(R13). status. LockConfiguraion. TP_ID(R5). JC_ID(R6)

3

Is evaluated byDecision Tree

Joint motion iscontrolled byDecision Tree

R13Is a componentof Search Space

R14

Checker (Ch)* Ch_ID. counter. recovery_status. abort_var

is checked by

R16

R15

Is A

R19

Performance Criterion (PC)

* PC ID . average value . status . FC_ID (R16) . scale . PC_name .TS_ID (R15)

Kinetic energy distortion Criterion

* PC ID (R19) *Criteria name .criterion value

System Compliance Criterion

* PC ID (R19) *Criteria name .criterion value

Inertia Criterion * PC ID (R19) *Criteria name .criterion value

Geometric Criterion * PC ID (R19) *Criteria name .criterion value

Constraint Criterion * PC ID (R19) * Criteria name . Error . criterion value

R17

Performance Monitoring

Decision Making

DecisionTree (DT) *DT_ID . optimal_solution . status

3 R12

R13

R18

has

has

has

creates

Search Space (SS) * SS_ID . SS_size . DT ID (R12)

FactorialSearchSpace (FSS)

* FSS_ID . SS_ID (R17) . condition

9

SimpleSearchSpace(SSS) * SSS_ID . SS_ID (R18) . condition

9

Fused Criterion (FC) * FC ID . PI . TS_ID (R14)

Consists of a number of trial configurations

Evaluates a trial configuration

R11 Controls motion of each joint

R8

Is used for evaluation of a trial configuration

NASA Robot Controller

Properties to be Model checked

48

Model Checking of Online Ticket Sale System

• A typical commercial transaction system;

• Presented at FASE 2002;

• Focus: Integrated state space reduction.

49

An Online Ticket Sale System (Class Diagram)

50

An Online Ticket Sale System (A State Model)

51

Some Verification Statistics of Online Ticket Sale System

• Verification of a liveness property– After an agent is assigned to a customer,

eventually the agent will be released.

• Statistics related to state space reductionsSPOR SMC Memory Usage Time Usage

Off Off Out of Memory -

Off On 113.73M 44736.S

On Off 17.3M 6668.3S

On On 74.0M 1450.3S

52







53

Summary and Future Work

• ObjectCheck– Integrates industrial software development environments

and model checkers with research tools;– Provides comprehensive automation for development,

validation, and model checking of xUML models;– Has enabled verification of non-trivial software system

designs modeled in xUML.

• Future work is focused on – State space reduction capability of ObjectCheck;– Hardware/software co-design and co-verification.

54

Related Work

• Most closely related work– UML Model Checking toolset from University of

Michigan;– vUML tool from Åbo Akademi University;

• There is also related work on model checking of statecharts with different semantics.

55

Additional Information

• http: //www.cs.utexas.edu/users/ObjectCheck• Selected publications:

– Fei Xie and James C. Browne. Verified Systems by Composition from Verified Components. Submitted for review.

– Fei Xie, James C. Browne, and Robert P. Kurshan. Translation-based Compositional Reasoning for Software Systems. Submitted for review.

– Fei Xie and James C. Browne. Integrated State Space Reduction for Model Checking Executable Object-oriented Software System Designs. In Proc. of FASE 2002.

– Fei Xie, Vladimir Levin, and James C. Browne. ObjectCheck: A Model Checking Tool for Executable Object-oriented Software System Designs. In Proc. of FASE 2002.

– Fei Xie, Vladimir Levin, and James C. Browne. Model Checking for an Executable Subset of UML. In Proc. of ASE, 2001.

56







57

Work Built on ObjectCheck

• An integrated state space reduction framework;

• Integration of model checking into component-based development of software.

58

Integrated State Space Reduction Framework

xUML-to-S/R Translation

S/R Model S/R Level Query

Model Checking with COSPAN

Success Report / Error Track

xUML Model xUML Level Query

ReducedxUML Model

Reduced xUML Level Query

User-Driven State Space Reduction

Verification Task

Verification Subtasks

Basic Model Checking Process

DecompositionAbstractionSymmetry Reduction

Symbolic VerificationLocalization Reduction

Partial Order Reduction

59

Reduction Steps for Checking P0

Customers, DispatcherAgents, Ticket Sever

Step 1: Symmetry Reduction

Step 2: Decomposition

Step 3: Symmetry Reduction

Step 4: Decomposition

Step 5: Case SplittingStep 6: Symmetry Reduction

P0

Customers, DispatcherAgents, Ticket Sever

P1

Customers Dispatcher Agents, Ticket ServerP21 , P22 P31 , P32

P33 , P23

Agents, Ticket ServerP41 , P42

P43 , P44

Ticket Server AgentsP41 , P42

P43 , P44

P5

Ticket ServerP6 AgentP41 , P42

P43 , P44

60

Evaluation of User-driven State Space Reduction

• Directly model checking P0 on OTSS – Two customer instances and two agent instances;

– SPOR and SMC are both applied.

– Memory usage: 152.79M

– Time usage: 16273.7S

• Memory and time usages for discharging subtasks at the leaf nodes of the reduction tree.

P21 P22 P41 P42 P43 P44 P6

Memory 0.30M 0.95M 0.28M 0.29M 0.28M 0.29M 0.35M

Time 0.02S 1.81S 0.01S 0.04S 0.01S 0.04S 0.63S

61

Integration of Model Checking into Component-based Development

• Temporal properties of a component are – Established with assumptions on the environment of the

component; – Verified under these assumptions and then packaged with

the component.

• Selecting a component for reuse considers not only its functionality but also its temporal properties.

• Properties of a composed component are verified by reusing verified properties of its sub-components and applying compositional reasoning.

62

Sensor Component

63

Properties of Sensor ComponentProperties:

Repeatedly (Output);After (Output) Never (Output) UntilAfter (OP_Ack);After (Done) Eventually (Done_Ack);Never (Done_Ack) UntilAfter (Done);After (Done_Ack) Never (Done_Ack) UntilAfter(Done);

Assumptions: After (Output) Eventually (OP_Ack);Never (OP_Ack) UntilAfter (Output);After (OP_Ack) Never (OP_Ack) UntilAfter (Output);After (Done) Never (Done) UntilAfter (Done_Ack);Repeatedly (C_Intr);After (C_Intr) Never (C_Intr + A_Intr + S_Schd) UntilAfter (C_Ret);After (ADC.Pending) Eventually (A_Intr);After (A_Intr) Never (C_Intr + A_Intr + S_Schd) UntilAfter (A_Ret);After (STQ.Empty = FALSE) Eventually (S_Schd);After (S_Schd) Never (C_Intr + A_Intr + S_Schd) UntilAfter (S_Ret);

verification case studies with objectcheck

Documents

modeldriven development

tinyos runtime image

tinyos components

future work work

verification case studies

objectcheck fei xiejoint

coverificationtwo properties

sequencenumber sequence