on verifying atl transformations using `off-the-shelf' smt solvers

On verifying ATL transformations using‘off-the-shelf’ SMT solvers

Fabian Buttner1, Marina Egea2, Jordi Cabot1

1AtlanMod, INRIA / Ecole de Mines de Nantes

2ATOS Research, Madrid

MODELS 2012

Our ApproachMotivation

Model transformations should be correct

This should be verified automatically

Our approachAddresses partial correctness of ATLtransformations w.r.t. OCLpre-/postconditionsTranslates this into a first-order logicproblemEmploys SMT solvers to check itPuts no upper bounds on the modelWorks well for our case studies

c© AtlanMod – [email protected] 2/19


Model transformations should be correctThis should be verified automatically





Our approachAddresses partial correctness of ATLtransformations w.r.t. OCLpre-/postconditions

Translates this into a first-order logicproblemEmploys SMT solvers to check itPuts no upper bounds on the modelWorks well for our case studies




Our approachAddresses partial correctness of ATLtransformations w.r.t. OCLpre-/postconditionsTranslates this into a first-order logicproblem

Employs SMT solvers to check itPuts no upper bounds on the modelWorks well for our case studies




Our approachAddresses partial correctness of ATLtransformations w.r.t. OCLpre-/postconditionsTranslates this into a first-order logicproblemEmploys SMT solvers to check it

Puts no upper bounds on the modelWorks well for our case studies




Our approachAddresses partial correctness of ATLtransformations w.r.t. OCLpre-/postconditionsTranslates this into a first-order logicproblemEmploys SMT solvers to check itPuts no upper bounds on the model

Works well for our case studies


Outline

1 Transformation correctness

2 Deriving a FOL specification3 Checking it using SMT solvers4 Conclusions


Outline

1 Transformation correctness2 Deriving a FOL specification

3 Checking it using SMT solvers4 Conclusions


Outline

1 Transformation correctness2 Deriving a FOL specification3 Checking it using SMT solvers

4 Conclusions


Outline

1 Transformation correctness2 Deriving a FOL specification3 Checking it using SMT solvers4 Conclusions


Outline



Two MetamodelsER metamodel

RelshipEnd

{xor}

ends

2..*

name : StringisKey : Boolean

ERAttribute

Relship

name : String

entities relships*

ERSchema1 1

*0..1 0..1

*

attrsattrs

*

*

1

Entity

name : Stringtype

1 name : String

-- CONSTRAINTS (PRECONDITIONS)

-- unique schema names

context ERSchema inv:

ERSchema.allInstances()->forall(s1,s2|

s1<>s2 implies s1.name<>s2.name)

-- entity names are unique in schema

-- relship names are unique in schema

-- disjoint entity and relship names

-- attr names are unique in entity

-- attr names are unique in relship

-- entities have a key

REL metamodel

RELAttribute

1

1..*

relations1

1..*

attrs

RELSchema

name : String

Relation

name : StringisKey : Boolean

-- CONSTRAINTS (POSTCONDITIONS)

-- relations have a key

context Relation inv:

self.attrs->exists(a | a.isKey)

-- schema names are unique

-- relation names are unique in schema

-- attribute names unique in relation


An ATL TransformationER

REL

ER2REL

module ER2REL; create OUT : REL from IN : ER;

rule S2S { from s : ER!ERSchema

to t : REL!RELSchema (name<-s.name) }

rule E2R { from s : ER!Entity

to t : REL!Relation (name<-s.name,

schema<-s.schema) }

-- not shown: R2R, EA2A, RA2A, RA2AK ...

Execution semantics of ATL:(i) Match source patterns; (ii) create target objects; (iii) bind properties



REL

ER2REL






schema<-s.schema) }


Execution semantics of ATL:(i) Match source patterns

; (ii) create target objects; (iii) bind properties



REL

ER2REL






schema<-s.schema) }


Execution semantics of ATL:(i) Match source patterns; (ii) create target objects

; (iii) bind properties



REL

ER2REL



to t : REL!RELSchema ( name <- s.name ) }


to t : REL!Relation ( name<-s.name ,

schema<-s.schema ) }


Execution semantics of ATL:(i) Match source patterns; (ii) create target objects; (iii) bind properties


Checking CorrectnessWe are interested in Hoare-style partial correctness ofT : MI → MF with respect to pre- and postconditions.

Our approach:Systematically derive a FOL specification (a set ofassertions) from T : MI → MF and its pre- andpostconditions that is logically valid iff the transformation iscorrect w.r.t. these conditionsCheck validity using an SMT solver (Z3, Yices)

Remarks:We consider core subsets of OCL and declarative ATLThe proofs we get are valid without model boundsThe problem is undecidable in general



Our approach:Systematically derive a FOL specification (a set ofassertions) from T : MI → MF and its pre- andpostconditions that is logically valid iff the transformation iscorrect w.r.t. these conditions

Check validity using an SMT solver (Z3, Yices)Remarks:

We consider core subsets of OCL and declarative ATLThe proofs we get are valid without model boundsThe problem is undecidable in general




Remarks:We consider core subsets of OCL and declarative ATL

The proofs we get are valid without model boundsThe problem is undecidable in general




Remarks:We consider core subsets of OCL and declarative ATLThe proofs we get are valid without model bounds

The problem is undecidable in general


Outline



FOL Semantics of MMs

Encode metamodels and constraints as a specification infirst-order logic (based on [Clavel et al., 2009])

Classes, associations, attributes generatepredicates and functions over object identifiersConstraints generate assertionsInterpretations of the FOL specification correspond to validinstances of the metamodel


Translation of ERRelshipEnd

Relshipname : String

ends

2..*

Predicates: Relship( ),RelshipEnd( ),ends( , )

Functions: name( )

Assertion:

∀x . Relship(x) → ∃y , z . RelshipEnd(y) ∧ ends(x , y)∧RelshipEnd(z) ∧ ends(x , z) ∧ y 6= z

context ERSchema inv:

ERSchema.allInstances()->forall(s1,s2|

s1<>s2 implies s1.name<>s2.name)

Assertion:

∀x , y . ERSchema(x) ∧ ERSchema(y) ∧ x 6= y →name(x) 6= name(y)


FOL Semantics of ATL

Encode transformation rules in first-order logicMatched rules become functions over object identifiersThe semantics of the rules is translated into assertionsInterpretations of the FOL specification correspond to validtraces of the transformation


Translation of ER2RELrule S2S { from s : ER!ERSchema

to t : REL!RELSchema (name <- s.name)}


to t : REL!Relation (name<-s.name, schema<-s.schema)}

rule R2R { from s : ER!Relship









Matching semantics

∀e . Entity(e) → ∃t . Relation(t) ∧ E2R(e) = t

∀e . Relship(rh) → ∃t . Relation(t) ∧ R2R(rh) = t








Creation semantics

∀t . Relation(t) →((∃e . Entity(e) ∧ E2R(e) = t) ∨(∃rh . Relship(rh) ∧ R2R(rh) = t)

)





to t : REL!Relation ( name<-s.name , schema<-s.schema)}



Simple property bindings

∀e, t . Entity(e) ∧ Relation(t) ∧ E2R(e) = t →name(e) = name(t)





to t : REL!Relation (name<-s.name, schema<-s.schema )}



Resolved property bindings

∀e, t . (Entity(e) ∧ Relation(t) ∧ E2R(e) = t) →(∀p . ERSchema(p) ∧ erschema(e,p) →∃s . RELSchema(s) ∧ relschema(t , s) ∧ resolve1(p, s)) ∧

(∀s . RELSchema(s) ∧ relschema(t , s) →∃p . ERSchema(p) ∧ erschema(e,p) ∧ resolve1(p, s))

resolve1(x , y) =def .

(ERSchema(x) ∧ RELSchema(y) ∧ S2S(x) = y) ∨(Entity(x) ∧ Relation(y) ∧ E2R(x) = y) ∨(Relship(x) ∧ Relation(y) ∧ R2R(x) = y)








(and some more details...)

The paper describes the complete translationTwo complete examples are available onlinehttp://www.emn.fr/z-info/atlanmod/index.php/MODELS_2012_SMT


http://www.emn.fr/z-info/atlanmod/index.php/MODELS_2012_SMT

Verification

DefinitionLet T = {r1, . . . , rn} be an ATL model transformation [. . . ].Then, T is correct with respect to preconditions {ς1 . . . ςl} andpostconditions {τ1, . . . , τw} if and only if, upon termination of T ,for every τi , i = 1, . . . ,w , the following formula always holds: l∧

j=1

ocl2fol(ςj)

∧

n∧j=1

atl2fol(rj)

⇒ ocl2fol(τi)


Verification

DefinitionLet T = {r1, . . . , rn} be an ATL model transformation [. . . ].Then, T is correct with respect to preconditions {ς1 . . . ςl} andpostconditions {τ1, . . . , τw} if and only if, upon termination of T ,for every τi , i = 1, . . . ,w , the following formula is unsatisfiable l∧

j=1

ocl2fol(ςj)

∧

n∧j=1

atl2fol(rj)

∧¬(ocl2fol(τi))


Outline



SMT solvers

Automatic Boolean SAT solving + TheoriesUninterpreted functionsArithmetic. . .

Support for quantifiers (incomplete procedures)> 15 implementations (SMT-COMP)Standardized language and libraries (SMT-LIB)


Employing the solver

Feeding our first-order logic specification to the SMT solver:Both Z3 and Yices can be used in the verification

The specification file mirrors our formalization one-to-oneSolving

Z3 solves our examples fully automatically usingModel-based Quantifier InstantiationYices sometimes requires Lemmas


Employing the solver

Feeding our first-order logic specification to the SMT solver:Both Z3 and Yices can be used in the verificationThe specification file mirrors our formalization one-to-one

SolvingZ3 solves our examples fully automatically usingModel-based Quantifier InstantiationYices sometimes requires Lemmas


Some ObservationsProofs found automatically for ER2REL by Z3 and Yices∗

Preconditions Postcondition Unsat core(total = 69)

pre1 (ERSchema.name) post1 (RELSchema.name) 4E::schema[1..1], RS::schema[1..1] R::schema[1..1] 9pre2, pre3, pre4, post2 (Relation.name) 16E::schema[1..1], RS::schema[1..1]RSE:relship[1..1] RA::relation[1..1] 11pre4, RSE::type[1..1], RS::ends[2..*] post4 (RELAttr.isKey) 14. . . . . . . . .

For all our examples:Implied constraints can be found fast and automaticallyUnsat cores are nice to understand the implicationsCounter examples are much harder and often time outBounded-model search seems more useful here(c.f. [Troya et al., JOT, 2011], [Buttner et al., ICFEM, 2012])





For all our examples:Implied constraints can be found fast and automatically

Unsat cores are nice to understand the implicationsCounter examples are much harder and often time outBounded-model search seems more useful here(c.f. [Troya et al., JOT, 2011], [Buttner et al., ICFEM, 2012])





For all our examples:Implied constraints can be found fast and automaticallyUnsat cores are nice to understand the implications

Counter examples are much harder and often time outBounded-model search seems more useful here(c.f. [Troya et al., JOT, 2011], [Buttner et al., ICFEM, 2012])





For all our examples:Implied constraints can be found fast and automaticallyUnsat cores are nice to understand the implicationsCounter examples are much harder and often time out

Bounded-model search seems more useful here(c.f. [Troya et al., JOT, 2011], [Buttner et al., ICFEM, 2012])





For all our examples:Implied constraints can be found fast and automaticallyUnsat cores are nice to understand the implicationsCounter examples are much harder and often time outBounded-model search seems more useful here(c.f. [Troya et al., JOT, 2011], [Buttner et al., ICFEM, 2012])


Outline



ConclusionsSummary:

Partial correctness of declarative ATL transformations w.r.t.pre- and postconditions can be nicely formulated in FOL

Modern SMT solvers seem to be quite good toautomatically find proofs in this setting (but beware:incomplete procedure)The approach complements bounded search approachesfor counter example finding

Future work:Provide improved toolingExtend the supported sets of ATL and OCLVerify larger case studiesProofs of executabilityIdentify decidable fragments of ATL transformations


ConclusionsSummary:

Partial correctness of declarative ATL transformations w.r.t.pre- and postconditions can be nicely formulated in FOLModern SMT solvers seem to be quite good toautomatically find proofs in this setting (but beware:incomplete procedure)

The approach complements bounded search approachesfor counter example finding



ConclusionsSummary:

Partial correctness of declarative ATL transformations w.r.t.pre- and postconditions can be nicely formulated in FOLModern SMT solvers seem to be quite good toautomatically find proofs in this setting (but beware:incomplete procedure)The approach complements bounded search approachesfor counter example finding



ConclusionsSummary:


Future work:Provide improved tooling

Extend the supported sets of ATL and OCLVerify larger case studiesProofs of executabilityIdentify decidable fragments of ATL transformations


ConclusionsSummary:


Future work:Provide improved toolingExtend the supported sets of ATL and OCL

Verify larger case studiesProofs of executabilityIdentify decidable fragments of ATL transformations


ConclusionsSummary:


Future work:Provide improved toolingExtend the supported sets of ATL and OCLVerify larger case studies

Proofs of executabilityIdentify decidable fragments of ATL transformations


ConclusionsSummary:


Future work:Provide improved toolingExtend the supported sets of ATL and OCLVerify larger case studiesProofs of executability

Identify decidable fragments of ATL transformations


ConclusionsSummary:




on verifying atl transformations using `off-the-shelf' smt solvers

Technology

itc atlanmod

correctc atlanmod

booleanc atlanmod

modelc atlanmod

smt solversc atlanmod

atl transformations

veried automaticallyc

fol specicationc atlanmod