Unbounded Data Model Verification Using SMT Solvers

An Integrated Data Model Verifier with Property TemplatesJaideep Nijjar Ivan Bocic Tevfik Bultan{jaideepnijjar, bo, bultan}@cs.ucsb.edu

University of California, Santa BarbaraDepartment of Computer ScienceVerification Lab

FormaliSE 20131Web Software EverywhereCommerce, entertainment, social interaction

We will rely on web apps more in the future

Web apps + cloud will make desktop apps obsolete

2Model-View-Controller PatternDBBenefits of the MVC pattern:Separation of concerns ModularityAbstractionMVC pattern has become the standard way to structure web applicationsRuby on RailsZend for PHPCakePHPStruts for JavaDjango for Python

ViewModelControllerThe user makes a request via his browser.The web server receives the request and forwards it to the appropriate controller.Controllers receive user input and initiate a response by making calls on model objects.Models talk to the database, store and validate data, perform the business logic Views are then generated by the controller; the UI --- HTML, CSS, XML, Javascript.The controller returns the view to the server. The server forms a proper HTTP response and sends it to the user.

MVC has become the standard!

Become popular for the many benefits you get from using it:

Our idea: exploit these design principles when doing modeling and verification of MVC web appsE.g. input validation checking requires focusing just on the Controller code because that is where the user input comes in3A Data Model Verification ApproachMVC Design PrinciplesAutomatic ExtractionAdd data model propertiesBounded and Unbounded4iDaVer: Integrated Data Model Verifierformal data model + property + technique(expressed using Templates)Active Records

Property Failed + CounterexampleProperty VerifiedUnknownModel Extraction

Choice of techniqueVerificationBounded (Alloy)Unbounded (SMT Solver)Unbounded (FOL Thm Prover)Properties PresentingiDaVer, our tool for automated and integrated data model verification!For Ruby on Rails applications, whose data model files are called Active RecordsInput: AR files, properties to check, and choice of verification technique

Property templates: to make tool even more user-friendly --Templates are language neutral, thus makes expressing properties about the data model easier.

Three backend solvers: Alloy for bounded verification, first-order logic theorem prover for unbounded verification, and SMT solver, also for unbounded verification.

Tool outputs either the property verified, failed with counterexample (for assertion properties) or unknown (for unbounded techniques only)5OutlineMotivationOverview of ApproachRails Data ModelsBasic RelationsOptions to Extend RelationsFormalization of SemanticsProperty TemplatesVerification TechniquesCase StudyConclusions and Future Work6A Rails Data Model Exampleclass User < ActiveRecord::Basehas_and_belongs_to_many :roleshas_one :profile, :dependent => :destroyhas_many :photos, :through => :profileendclass Role < ActiveRecord::Basehas_and_belongs_to_many :usersendclass Profile < ActiveRecord::Base belongs_to :user has_many :photos, :dependent => :destroy has_many :videos, :dependent => :destroy, :conditions => "format='mp4'"endclass Tag < ActiveRecord::Basebelongs_to :taggable, :polymorphic => trueendclass Video < ActiveRecord::Base belongs_to :profile has_many :tags, :as => :taggableendclass Photo < ActiveRecord::Base ...Role*0..11UserProfile*1Video*1Taggable*Tag1*1Photo*1format=.mp4A sample Active Records specification for a social networking application where users have profiles which store their photo and video files. The photos and videos can be tagged by users, and users can have different roles.

(Note: body of Photo class same as Video class)7Rails Data ModelsData model verification: Analyzing the relationships between data objectsSpecified in Rails using association declarations inside the ActiveRecord filesThe basic relationshipsOne-to-oneOne-to-manyMany-to-manyExtensions to the basic relationships using Options:through, :conditions, :polymorphic, :dependent8The Three Basic Relationships in RailsOne-to-One

.One-to-Many

class User < ActiveRecord::Base has_one :profileend.class Profile < ActiveRecord::Base belongs_to :userendclass Profile < ActiveRecord::Base has_many :videosend.class Video < ActiveRecord::Base belongs_to :profileendUserProfile1ProfileVideo*11Relationships are expressed by adding a pair of declarations in the corresponding Rails models of the related objects. Rails automatically matches the relation name with the appropriate class by the name given

Inheritance relation in Rails the notation ChildClass < ParentClass ActiveRecord::Base class contains all the database-connection functionality9The Three Basic Relationships in RailsMany-to-Manyclass User < ActiveRecord::Base has_and_belongs_to_many :usersend

class Role < ActiveRecord::Base has_and_belongs_to_many :rolesendUserRole**10Options to Extend the Basic Relationships:through OptionTo express transitive relations:conditions OptionTo relate a subset of objects to another class:polymorphic OptionTo express polymorphic relationships:dependent OptionOn delete, this option expresses whether to delete the associated objects or notFor the sake of time, I will only be focusing on :through and :dependent option for the duration of the talk.11The :through Optionclass User < ActiveRecord::Basehas_one :profilehas_many :photos, :through => :profileendclass Profile < ActiveRecord::Base belongs_to :user has_many :photosendclass Photo < ActiveRecord::Base belongs_to :profileendProfileUserPhoto**0..1111:through option can be set on either the has_one or has_many declaration. This is example of has_many.

Dashed line is actually the join of the other two. So for ease of access of book editions through Author class rather than going through the book class.12The :dependent Option:delete directly delete the associated objects without looking at its dependencies:destroy first checks whether the associated objects themselves have associations with the :dependent option setclass User < ActiveRecord::Base has_one :profile, :dependent => :destroyend

class Profile < ActiveRecord::Base belongs_to :user has_many :photos, :dependent => :destroyend PhotoProfileUser*110..1The User class has the :dependent option set on its :profilerelation. Thus, when a User object is deleted, the associated profile object will be deleted. Further, since the :dependent option is set to :destroy, any relation with the :dependent option set in the Profile class will also have its objects deleted. In this scenario, the delete also gets propagated to the photos associated with that deleted Profile object. If instead of :destroy the :dependent option in User was set to :delete, all related Profile objects would be deleted, but not any Photoss associated with the Profile. We can see how this may lead to dangling references if this option is not set correctly.

Note that options can be combined, like using the :dependent option with the :conditions option.13Formalizing Rails SemanticsS: The sets and relations of the data model (data model schema)e.g. { Photo, Profile, Role, Tag, Video, User} and the relations between themC: Constraints on the relationsCardinality constraints, transitive relations, conditional relations, polymorphic relations

D: Dependency constraints Express conditions on two consecutive instances of a relation such that deletion of an object from the first instance leads to the other instanceFormal data model: M = We can construct a formal data model representing the objects and their relationships in a RoR application.We define a formal data model to be a 3-tuple where

S is the data model schema, identifying the sets and relations of the data model, C is a set of relational constraints and D is a set of dependency constraints. The relational constraints in C express all the constraints on the relations.Ex) of the :conditions option, which limits the relation to those objects that meet a certain criteria. In this example, Video objects are only related to a Profile object if their format field is mp4. The formalization of this constraint defines a set of objects (oV ) that is a subset of the Video objects (oV ) (corresponding to Video objects with format field mp4") and restricts the relation between the Profile and Video objects (rPV ) to that subset.

The dependency constraints in D express conditions on two consecutive instances of a relation such that deletion of an object from one of them leads to the other instance by deletion of possibly more objects (based on the :dependent option). 14OutlineMotivationOverview of ApproachRails Data ModelsProperty TemplatesVerification TechniquesCase StudyConclusions and Future Work15Property TemplatesUser-friendly and makes it easy to express propertiesManually writing properties is a time-consuming and error-prone processRequires familiarity with input modeling language of solver

Templates are language-neutralDo not require familiarity with SMT-LIB, Spass and Alloy languages, and understanding of output specificationsMake it easy to rerun the tool and switch the verification technique, by not requiring the user to rewrite the property

Eight property templates available for the most common data model propertiesTemplates make using tool more user-friendly by making it easy to express properties.Language-neutral => easier to switch between unbounded and bounded, such as when unbounded returns unknownWe created prop templates for eight of the most common data model properties we desired to check in our previous experiments.16Property TemplatesalwaysRelated[classA, classB]To check that objects from classA are always related to objects of classBsomeMultipleRelated[classA, classB] To check that it is possible for objects of classA to be related to more than one object of classBsomeUnrelated[classA, classB]To check that it is possible for an object of classA to not be related to any objects from classBtransitive[classA, classB, classC]To check that the relation between classA and classC is the composition of the relations between classA and classB, and classB and classCProperty TemplatesnoDangling[classA, classB]To check that when an object of classA is deleted, there are no objects from classB that are left with a dangling reference to the deleted objectdeletePropagates[classA, classB]To check that when an object of classA is deleted, related objects in classB are also deletednoDeletePropagation[classA, classB]To check that when an object of classA is deleted, related objects in classB are not deletednoOrphans[classA, classB]To check that deleting an object from classA does not cause related objects in classB to be orphaned An orphan is an object that is related to no other object

OutlineMotivationOverview of ApproachRails Data ModelsProperty TemplatesVerification TechniquesBounded Verification with AlloyUnbounded Verification with SMT SolverUnbounded Verification with FOL Theorem ProverCase StudyConclusions and Future Work19iDaVerSMTSolverinstance or unsat or unknownformulaSMT-LIB EncoderResults InterpreterUnbounded VerificationAlloyAnalyzerinstance or unsat formulaAlloy EncoderResults InterpreterBounded Verificationformal data model + propertyActive Records

Model Extraction

Choice of techniqueProperties Theorem Proverproof found or completion foundor timeoutformulaFOL EncoderResults InterpreterProperty Failed + CounterexampleProperty VerifiedUnknownDetailed architecture of iDaVer.Verification component: - Use an appropriate formal modeling language to express the formal data model (Alloy, FOL and SMT-LIB) - Use a backend solver to perform the verification (Alloy Analyzer, Spass, SMT solver) - Interpret results solvers and output themIll be going into detail about our verification approaches next. Starting with bounded verification.20Bounded VerificationUnbounded VerificationAlloyAnalyzerinstance or unsat formulaAlloy EncoderResults InterpreterBounded Verificationformal data model + propertyActive Records

Model Extraction

Choice of techniqueProperties SMTSolverinstance or unsat or unknownformulaSMT-LIB EncoderResults InterpreterTheorem Proverproof found or completion foundor timeoutformulaFOL EncoderResults InterpreterProperty Failed + CounterexampleProperty VerifiedUnknownNext: Bounded verification21Alloy LanguageA declarative specification language for object modelingBased on first-order logicSet-based representation of objectsDefines sets of objects using signatures (sigs)Defines relations using fields inside the signaturesAdd additional constraints about the model as factsWell-suited for formally specifying data modelsCan add assertions to specify properties about the specificationAlloy Formal language

Language includes set operators(union, intersection, difference, etc), boolean operators (and or, not, etc), and quantifiers

Cuz language is based on sets/relations, makes alloy a good choice for formally specifying data models22Alloy AnalyzerAutomated verification of Alloy specifications is undecidable if the domains are not bounded

To ensure decidability, Alloy Analyzer restricts the domains to a finite scopeUser-specified A finite bound on the sizes of the domains

SAT-based bounded verificationAlloy Analyzer translates the Alloy verification query to a Boolean logic formula satisfiability query Then invokes an off-the-shelf SAT-solverNext step, after obtaining the data model expressed in Alloy, is to add properties we wish to verify.Use assertions to specify properties.

Reason(motivation) for expressing data models in Alloy is to use the model checker called Alloy Analyzer

Bound: on the sizes of the sets of objects

((* small scope hypothesis: a high proportion(most bugs that exist in a model will show up during exhaustive checking within a small scope))

23Sample Translation to Alloyclass User < ActiveRecord::Base has_one :profileend

class Profile < ActiveRecord::Base belongs_to :userendsig Profile {}sig User {}one sig State {profiles: set Profile,users: set User,relation: Profile lone -> one User}The keyword sig is used in Alloy to dene a set of objects.Thus, a sig is created for each class in the input Rails datamodel. In this example, a sig is declared for the Proleand User classes. We also create a State sig, which we useto dene the state of a data model instance. Since we onlyneed to instantiate exactly one State object when checkingproperties, we prepend the sig declaration with a multiplicityof one. The State sig contains elds to hold the set of allobjects and related object pairs. In this example, the Statesig contains three elds. The rst is named profiles andis a binary relation between State and Prole objects. Theeld uses the multiplicity operator set, meaning zero ormore. In other words, the state of a data model instance maycontain zero or more Prole objects. The State sig containsa similar eld for User objects. Finally, the one to one relation between Prole and User objects is translated asanother eld in the State sig. Named relation, it is denedto be a mapping between Prole and User objects. It uses themultiplicity operators to constrain the mappingbetween Proles and Users to be one-to-one.IDAVER does this translation automatically.24Bounded Verification of Data ModelsAutomatically translate the formal data model extracted from the Active Records and the property to AlloyUser may specify a bound or use the defaultUse Alloy Analyzer to perform bounded verificationPossible outputs:Assertion holds within the given boundA counterexample proving falsified assertionsFormal data model + Property+ BoundAlloy Encoderinstance or unsat formulaAlloyAnalyzerBounded VerificationResults InterpreterProperty Fails + CounterexampleProperty Verifies (within bound)To summarize our approach to bounded verification

Formalization of Active Records input to verification

Bound: the maximum number of objects of each type to instantiate (default 5)25Unbounded with SMT SolverUnbounded VerificationAlloyAnalyzerinstance or unsat formulaAlloy EncoderResults InterpreterBounded Verificationformal data model + propertyActive Records

Model Extraction

Choice of techniqueProperties Theorem Proverproof found or completion foundor timeoutformulaFOL EncoderResults InterpreterProperty Failed + CounterexampleProperty VerifiedUnknownSMTSolverinstance or unsat or unknownformulaSMT-LIB EncoderResults InterpreterNext: Unbounded verification with SMT Solvers26Satisfiability Modulo Theories (SMT)SMT-solvers automatically check the satisfiability of first-order formulas with respect to a set of theoriesTypical theories include: Linear Arithmetic, Arrays, Bit Vectors, Equality with Uninterpreted FunctionsOnly implicit universal quantificationTheory of Equality with Uninterpreted Functions.Language: Variables, Constants, Uninterpreted function symbols, Predicate =, and Boolean connectivesUninterpreted functions have no properties except their signature and functional consistency: a = b => F(a) = F(b)Example formula: F(x) = F(G(y)) H(x,y) = 1Area of work for finding decision procedures for FO formulas wrt some logical theory T

We need a logic that can express objects and relations between them. EUF is one such theory.Ex) x,y are variables, F,G,H are UF symbols, 1 is a constant

Two terms are equal => function will return same value when they plugged in

(Satisfiability problem for EUF is NP-complete)27SMT SolversThere are many SMT solvers out there that support popular theoriesHowever many of them are not suitable for us we need support for quantified formulas to handle the property templatesMicrosofts Z3 supports quantified expressions in the theory of uninterpreted functionsUses heuristics for eliminating quantifiers in formulasMay return unknown during satisfiability checkMost SMT solvers do not handle quantifier at all. Because introduces undecidability.Those that do dont handle them well.

We found Microsofts Z3 not only handles quantifiers, but quite well for our purposes.

(An SMT solver make use of procedures specific to the theories they handle, such as linear arithmetic, in combination with the brute force power of a SAT solver. At a high level, proceed by iteratively by replacing the sub-expressions in a formula like(x + y < 10 || x > 9) && (y + z < 5)with propositional variables, to give something like(A || B) && (C). At this point we can apply a SAT solver to look for a solution. If none exists then we need not bother with analyzing the abstracted expressions, as the formula is unsatisfiable. If the SAT solver finds a satisfiable solution we then restore the original expressions and pass the conjunction of this off to the core decision procedure which can deal with the semantics of the given theory. This process then proceeds in the standard DPLL method of iteration, involving finding UNSAT cores and backtracking.)28SMT-LIBDefines a standard input language for SMT SolversDefines theories and logics in which formula can be writtenLisp-like format: Specifications are a series of s-expressionsDeclare types using declare-sort command(declare-sort User)Declare uninterpreted functions using declare-fun command(declare-fun foo (Domain) Range)Quantifier commands: (forall )and (exists )Add constraints and properties using assert commandCheck satisfiability by using (check-sat) commandAn initiative that(Provides standard rigorous descriptions of background theories in which formula can be written)

Among all its commands and operators,

(many-sorted rst-order logic)29Sample Translation to SMT-LIBclass User < ActiveRecord::Base has_one :profileend

class Profile < ActiveRecord::Base belongs_to :userend(declare-sort User)(declare-sort Profile)(declare-fun relation (Profile) User)(assert (forall ((p1 Profile)(p2 Profile))(=> (not (= p1 p2)) (not (= (relation p1) (relation p2) )))))Types in SMT-LIB are declared using the declare-sortcommand. We use this command to declare types for Userand Prole. The relation is translated as an uninterpreted function. Uninterpreted functions are created in SMT-LIB using the declare-fun command. Weuse this command to declare an uninterpreted function namedrelation whose domain is Prole and range is User. Sincefunctions can map multiple elements in the domain to the sameelement in the range, and we instead desire a one-to-one relationrelation, we constrain the function to be one-to-one to obtainthe desired semantics. This constraint is expressed using theassert command, as shown above. 30Unbounded Verification using SMT SolversSMTSolverinstance or unsat or unknownformulaSMT-LIB EncoderResults InterpreterUnbounded VerificationAutomatically translate formal data model and property into the theory of uninterpreted functions with quantification (SMT-LIB)Use the SMT solver Z3 to perform satisfiability checkFor failing assertion properties, our tool interprets outputs and forms a counterexampleUnknowns (and timeouts) possible since the theory is undecidableFormal data model + PropertyProperty Failed + CounterexampleProperty VerifiedUnknownPossible outputs: -prop verified (for unbounded domains this time) -prop failed, with counterex, which our tool extracts from the SMT solver -unknown

Pros/Cons:Unbounded: no bound for verified assertionsBounded: bound, but no unknowns/timeouts31Unbounded VerificationUnbounded VerificationAlloyAnalyzerinstance or unsat formulaAlloy EncoderResults InterpreterBounded Verificationformal data model + propertyActive Records

Model Extraction

Choice of techniqueProperties SMTSolverinstance or unsat or unknownformulaSMT-LIB EncoderResults InterpreterProperty Failed + CounterexampleProperty VerifiedUnknownTheorem Proverproof found or completion foundor timeoutformulaFOL EncoderResults InterpreterNext: Unbounded verification with FOL Theorem Prover32FOL Theorem ProversRails data models and properties are expressible in first-order logic (FOL) with equality and quantifiersNote that this is an undecidable theoryThere are automated theorem provers for first-order logicThey use search strategies to find proofs However due to undecidability of the FOL they cannot always give a definite answerWe use the FOL theorem prover, SpassModeling Active Records using FOLModel object types by declaring a unary predicateReturns true if an object is a member of that classModel relations between data objects using binary predicatesReturns true if the two objects are relatedAxioms are used to express constraints on the data model, e.g.Specifying cardinality, dependency, and transitivity constraints on relationsSpecifying that predicates denoting classes not related by inheritance are mutually exclusive Conjectures are used to model the property to be checked

To verify the property holds on the data model, send the following formula to the theorem prover: axioms => conjecture 34Sample Translation to Spasslist_of_symbols. sorts[Profile, User]. predicates[(relation, 2)]. end_of_list.list_of_formulae(axioms). formula(forall([Profile(a)], not(User(a)))). formula(forall([User(a)], not(Profile(a)))). formula(forall([a, b], implies( relation(a, b), and(Profile(a), User(b))))). formula(forall([a, b1, b2], implies( and(relation(b1,a), relation(b2,a)), equal(b1,b2)))). formula(forall([a, b1, b2], implies( and(relation(a,b1), relation(a,b2)), equal(b1,b2)))). formula(forall( [Profile(a)], exists([b], relation(a, b)) )).end_of_list.Class types are translated into unary predicates (line 2); those denoting classes not related by inheritance are specied to be mutually exclusive (lines 6 and 7). Each relation is translated into a binary predicate (line 3) that can only be true if both elements satisfy their corresponding class type predicate (line 8).The cardinality of relations is translated into additional axiom formulas. For example, the constraint that each User object is related to at most one Prole object is translated into the formula on line 9. For the constraint that each Prole object is associated with exactly one User object, we add two formulas: one similar to that on line 9 and one that says each Prole is related to at lease one User (lines 10 and 11, respectively).35Unbounded Verification of Data Models using Theorem ProversTheorem ProverformulaFOLEncoderResults InterpreterUnbounded VerificationAutomatically translate formal data model and property into Spasss input language (FOL)Use the theorem prover Spass to see if formula provableInterpret results to determine whether property holdsDoes not produce counterexamples since theorem provers are not designed to do soIf Spass times out (due to undecidability of theory), iDaVer returns UnknownFormal data model + PropertyProperty FailedProperty VerifiedUnknownproof found or completion foundor timeoutPossible outputs: -prop verified (for unbounded domains this time) -prop failed no counter example! (in contrast to other two solvers) -unknown (when Spass timesout)

36OutlineMotivationOverview of ApproachRails Data ModelsProperty TemplatesVerification TechniquesCase StudyConclusions and Future Work37Case StudyLovdByLess, a social networking applicationLOC: 3787

iDaver input:Path of the directory containing the Active Record filesName of the file containing the properties to check (Expressed using property templates!)Verification technique

Number Active Record files: 13Performed Case study on an open-source Ruby on Rails application called LovdByLess38Case StudyCheck: alwaysRelated[Photo, Profile]Solver: Spass

+ Unbounded verification No sample instance May report unknown or timeout

Output of iDaVer:-the property being verified-the technique + solverThe number of variables and clauses in the Spass specificationTime to perform verification (i.e. running spec through Spass)Total time (i.e. to translate AR files to Spass +verif time)Verif result (Property holds in this case)39Case StudyCheck: someUnrelated[ForumTopic, ForumPost]Solver: Z3

+ Unbounded verification + Sample instance May report unknown or timeout

Instance output in language-neutral set and tuple form.

Downside to unbounded verification is that it can return unknown or timeout.40Case StudyCheck: deletePropagates[Profile, Photo]Solver: Alloy

41Case StudyCheck: deletePropagates[Profile, Photo]Solver: Alloy

+ Counterexample data model instance+ Always returns a result (for small domains) Bounded Slower

In this case, the failing property points to a data model error. Comments in code indicate when a User is deleted, her Profile and all associated items, such as Photos, should be deleted. However, data model does not uphold this property.

(Currently not manifesting as application error since UI does not allow User delete.)

Always returns a result a plus since the unbounded techniques may return unknown or timeout42Summary of Technique Pros and ConsAlloySMT SolversTheorem ProversUnbounded VerificationNoYesYesProduces CounterexampleYesYesNoGeneratesUnknownNoYesYesSpeedSlowFastFast**In particular, Spass was slightly faster than Z3 in our experiments and timed out less frequentlyOutlineMotivationOverview of ApproachRails Data ModelsProperty TemplatesVerification TechniquesCase StudyConclusions and Future Work44Conclusions and Future WorkIt is possible to extract formal specifications from MVC-based data models and analyze themWe were able to find data model errors in real-world applications using some of these techniques (ISSTA11, ASE12)Integration of multiple automated verification tools makes overall approach more flexibleProperty templates simplify property specification We have some recent work on automated property inference (ISSTA13)analyzing actions that update the data store (submitted) Main goal: Verifiable data model specification!

Questions?46Related WorkVerification of Web Applications [Krishnamurti et al, Springer 2006 ] focuses on correct handling of the control flow given the unique characteristics of web apps Works such as [Hall et al, ASE 2010] and [Han et al, MoDELS 2007] use state machines to formally model navigation behaviorIn contrast to these works, we focus on analyzing the data modelFormal Modeling of Web ApplicationsWebAlloy [Chang, 2009]: user specifies the data model and access control policies; implementation automatically synthesizedWebML [Ceri et al, Computer Networks 2000]: a modeling language developed specifically for modeling web applications; no verificationIn contrast, we perform model extraction (reverse engineering)

Focus has largely been on **navigation** modeling and verification47Related WorkVerification of Ruby on Rails applicationsRubicon [Near et al, FSE 2012] verifies the Controller whereas we verify the Data ModelRequires manual specification of application behavior, whereas we verify manually written propertiesLimited to bounded verificationData Model Verification using Alloy[Cunha and Pacheco, SEFM 2009] maps relational database schemas to Alloy; not automated[Wang et al, ASWEC 2006] translates ORA-SS specifications to Alloy, and uses the Analyzer to produces instances of the data model to show consistency[Borbar et al, Trends 2005] uses Alloy to discover bugs in browser and business logic interactions* Rubicon requires specification files describing expected behavior for each Controller - Also uses Alloy Analyzer

ORA-SS = Object-Relationship-Attribute model for Semi-Structured data

Last two works limited to bounded verification

... A different class of bugs than the data model related bugs we focus on48Related WorkUnbounded Verification of Alloy Specifications using SMT Solvers[Ghazi et al, FM 2011], approach not implementedMore challenging domain since Alloy language contains constructs such as transitive closures Specification and Analysis of Conceptual Data Models[Smaragdakis et al ASE 2009, McGill et al ISSTA 2011] use Object Role Modeling to express data model and constraintsFocus is on checking consistency and producing test cases efficientlyUsing Patterns to Facilitate Formal Property SpecicationFirst proposed for temporal logic properties [Dwyer et al ICSE 1999]The templates we present are not temporal and are specic to data model analysis 1. ...such as transitive closures which do not appear in the data models we extract

2. ORM is a popular conceptual modeling language. Goals of these works: - Consistency: checking whether any instances of a data model exist - Automatic generation of test data that respect constraints expressed in ORM

Our focus is on extracting the data model from an existing application and performing verification (that desirable properties hold.) as opposed to test case generation from a model (reverse engineering vs forward engineering)49