october 8, 2015climepage 1 constraining software evolution steven p. reiss brown university

April 21, 2023 CLIME

Constraining Software Evolution

Steven P. ReissBrown University


Software is Multidimensional

Architecture

SpecificationsDesign

Test Cases

User InterfaceConventions

HistoryDocumentation

CodeModule Structure

Component Structure

DBMS SchemaPatterns


Software Artifacts

• Dimensions are reflected in artifacts– Typically files, sometimes databases– Some artifacts are “paper only”

• Generally one-to-one– But some artifacts have multiple dims– And some dims are spread over multiple

artifacts

• But it is the artifacts that are edited


Evolution is One-dimensional

• Artifacts evolve separately– Thus dimensions evolve independently– Different views become inconsistent– Design, documentation, etc. ignored– Software itself becomes inconsistent

• Some dimensions don’t evolve– Some dimensions don’t have artifacts– These are totally ignored

• Result is poor, unmaintainable software


What is Needed

• CONSISTENT EVOLUTION – OF ALL DIMENSIONS OF SOFTWARE– OVER THE LIFE OF THE SOFTWARE

• This is our goal


What We Propose

• A tool to ensure artifacts consistency– Initially as the software is developed– During maintenance as the software evolves

• The CLIME system


Outline of Talk

• Requirements for consistent evolution• Constraints as a mechanism• Implementing constraints in CLIME• On-going and future work


Requirements for the Tool• Work with existing tools & artifacts• Handle a wide range of dimensions• Be extensible• Be bidirectional• Support partial & incomplete checking• Be able to locate points of inconsistency• Have low overhead and be unintrusive• Handle static and dynamic properties• Work incrementally


Insight Yields Methodology• Artifacts are constraints on artifacts• Design

– Constraints on the code– Constraints on the code’s behavior

• Specifications– Constraints on the solution

• In general– Software artifacts can be kept consistent by

viewing each as constraints on other artifacts and checking these constraints.


Basic Approach

SoftwareArtifacts

Constraints

ConsistencyChecker

InconsistencyReport


Problems with this Approach

• Determining what artifacts are involved• How to extract the information needed

for constraints from the artifacts• How to define constraints from this

information• How to check constraints • What to report to the developer• Making it incremental and complete


Related Work: XLinkIt• Constraining XML documents

– Constraints are XLink/XPath queries– Documents can be dynamically generated

• Has been applied to software engineering– Small number of dimensions– By dynamically generating from java code– Comparing to XML representation of UML

• Problems– NOT incremental, complete, automatic, compatible,

powerful enough, practical at large scales


What are the Artifacts

• Need to define the domain– Essentially the project being worked on– This concept already exists in environments

• Need to identify artifacts– These are typically stored as files now– Identify artifacts using file types

• We handle this using a project manager– Or from existing programming environment


Project Manager: CLIP

• Project determines the set of artifacts• Provides set of files to rest of system

– Based on directory list (include + exclude)– Based on patterns– Based on artifact file types

• Also specifies environment for files• Can be automatically generated

– From existing environments (Eclipse, …)

• Front end also available


Project Interface


Information From Artifacts• First we need to understand the artifacts

– Artifacts alone are too varied and complex• We need specific information

– To define the constraints– Different constraints need different info

• Solution: abstract artifact information– Simpler than considering whole artifact– Tools to extract the needed information

• This must be done incrementally• Store the result in a database


Artifact AbstractionSoftwareArtifacts

Abstractor Abstractor Abstractor Abstractor

Update Information

Database Manager

Database

Update Manager


Artifact Abstraction

• Update Manager– Detects what has changed– Runs appropriate abstractor(s) for each

artifact

• Update Information– Commands to delete sets of tuples (patterns)– Commands to add tuples

• Database Manager– Relational DBMS (JDBC/Postgresql)


Current Abstractors

• Source– Symbol table information (Modified Jikes)

• Refs, defs, types, hierarchy, scopes, …– Semantic information (Using Soot)– Documentation information (CLAD: Doclet)

• UML (Rose or any XMI-based tool)– Class & message interaction diagrams

• CVS History– File version and revision history


Test Case Abstraction• Desired information

– What are the test cases– Test coverage of each test case

• CITE: Test coverage tool– Extension of junit testing program– Use jikesbt to instrument code

• Modified junit that uses cite– Produces coverage information when tests

are run as well as the set of test cases– Line, branch, routine, call coverage


Cite Coverage<TESTRUN DATE='1034344045851' PROJECT='onsets' USER='spr' NAME='spr.onsets.TestBitSet.testFirst'

SIGNATURE='()V' STATUS='PASS' RUNID='17921'> <METHOD NAME='spr.onsets.OnsetBitSet.check' CALLED='35'> <BLOCK INDEX='18429' START='54' END='54' COUNT='35'> <BRANCH TOBLOCK='18431' COUNT='33' /> <BRANCH TOBLOCK='18430' COUNT='2' /> </BLOCK> <BLOCK INDEX='18430' START='54' END='54' COUNT='2'> <BRANCH TOBLOCK='18432' COUNT='2' /> </BLOCK> <BLOCK INDEX='18431' START='54' END='54' COUNT='33'> <BRANCH TOBLOCK='18432' COUNT='33' /> </BLOCK> <BLOCK INDEX='18432' START='54' END='54' COUNT='35' /> </METHOD> <METHOD NAME='spr.onsets.OnsetBitSet.firstBit' CALLED='3'> <CALLS NAME='spr.onsets.OnsetBitSet.check' COUNT='35' /> <BLOCK INDEX='18500' START='134' END='134' COUNT='3'> <BRANCH TOBLOCK='18504' COUNT='3' /> </BLOCK> <BLOCK INDEX='18501' START='135' END='135' COUNT='35'> <BRANCH TOBLOCK='18503' COUNT='33' /> <BRANCH TOBLOCK='18502' COUNT='2' /> </BLOCK> <BLOCK INDEX='18502' START='135' END='135' COUNT='2' /> <BLOCK INDEX='18503' START='134' END='134' COUNT='33'> <BRANCH TOBLOCK='18504' COUNT='33' /> </BLOCK> <BLOCK INDEX='18504' START='134' END='134' COUNT='36'> <BRANCH TOBLOCK='18501' COUNT='35' /> <BRANCH TOBLOCK='18505' COUNT='1' /> </BLOCK> <BLOCK INDEX='18505' START='138' END='138' COUNT='1' /> </METHOD></TESTRUN>


Database Manager: CLIDE• Maintains the database based on

commands– First it finds all tuples that match remove

patterns using queries– Matches new tuples to those to remove– Adds any really new tuples to the database – Updates any modified tuples– Removes tuples that are really deleted

• Generates an update file– What was added, deleted, modified– Based on UIDs


CLIDE: UIDs

• Unique Ids are used to represent tuples– Each relevant tuple has one

• Database manager maintains these– If tuple is only modified, it keeps its UID– Based on set of key fields

• UIDs from abstractor => global UIDs– Provide pointers in the tuples

• These provide low-level handles to the actual data


CLIDE: Table Definition?xml version="1.0"?><!DOCTYPE CLIME:TABLES SYSTEM "file:/pro/clime/xml/schema/climetable.dtd" >

<CLIME:TABLES xmlns:CLIME='file:/pro/clime/xml/schema/clime'>

<CLIME:ENUM NAME="HistoryType"> <CLIME:CONST NAME="NONE" VALUE="0" /> <CLIME:CONST NAME="CVS" VALUE="1" /> </CLIME:ENUM>

<CLIME:TABLE NAME="HistoryFile" > <CLIME:FIELD NAME="Id" TYPE="Id" KEY="1" /> <CLIME:FIELD NAME="FileName" TYPE="String" /> <CLIME:FIELD NAME="HistoryType" TYPE="HistoryType" /> <CLIME:FIELD NAME="RawFile" TYPE="String" /> <CLIME:FIELD NAME="CurrentVersion" TYPE="String" /> <CLIME:MATCH IDFIELD="Id" FIELDS="FileName RawFile" /> </CLIME:TABLE>

<CLIME:TABLE NAME="HistoryVersion"> <CLIME:FIELD NAME="Id" TYPE="Id" KEY="1" /> <CLIME:FIELD NAME="ForFile" TYPE="Id" SOURCE="HistoryFile" /> <CLIME:FIELD NAME="FileName" TYPE="String" /> <CLIME:FIELD NAME="Version" TYPE="String" /> <CLIME:FIELD NAME="Author" TYPE="String" /> <CLIME:FIELD NAME="HasDesc" TYPE="Boolean" /> <CLIME:MATCH IDFIELD="Id" FIELDS="ForFile Version" /> </CLIME:TABLE>

</CLIME:TABLES>


CLIDE: Transitive Closure

• Transitive closure is needed in software– Symbols available in scope– Subclass relationship– Control/data flow; type inference; …– TC is not a relational operation

• CLIDE provides transitive relations– Automatically built and maintained– Update is done incrementally– Can be queried against directly


Defining Constraints

• Defined in terms of artifact abstractions• Multiple types of constraints are possible• Most constraints can be defined as

relationships among the abstractions– Examples– This allows us to define metaconstraints

• Issue: maintaining traceability– Source of the constraint; Point of violation– This leads to constraint instances


Using Metaconstraints

Metaconstraints

ArtifactDatabase

ConstraintFinder

Constraint Instances


Metaconstraints

• Predicate equations (first order logic)

• S is the base relation determines when x is applicable specifies what must be true

– For this constraint to hold– Typically starts with for all or exists

• These are general logic equations– With the full power of SQL available

)()()( xxSx


Example<CLIME:CONSTRAINT TYPE="QUERY" NAME="uml_source_class_correspondence" > <CLIME:EXPR OP="FORALL" VAR="x" TABLE="UmlClass"> <CLIME:EXPR OP="AND">

<CLIME:EXPR OP="NOTNULL"> <CLIME:EXPR OP="FIELD" FIELD="Name" VAR="x" /> </CLIME:EXPR> <CLIME:EXPR OP="EQL">

<CLIME:EXPR OP="FIELD" FIELD="ClassType" VAR="x" /> <CLIME:EXPR OP="INT" VALUE="0" />

</CLIME:EXPR> </CLIME:EXPR> <CLIME:EXPR OP="EXISTS" VAR="y" TABLE="SrcDefinition">

<CLIME:EXPR OP="AND"> <CLIME:EXPR OP="EQL"> <CLIME:EXPR OP="FIELD" FIELD="SymbolType" VAR="y" /> <CLIME:EXPR OP="INT" VALUE="3" />

</CLIME:EXPR> <CLIME:EXPR OP="EQL"> <CLIME:EXPR OP="FIELD" FIELD="Name" VAR="x" /> <CLIME:EXPR OP="FIELD" FIELD="Name" VAR="y" />

</CLIME:EXPR> </CLIME:EXPR> </CLIME:EXPR> </CLIME:EXPR></CLIME:CONSTRAINT>


Current Metaconstraints

• UML Class Diagrams Source– Every class has a corresponding source class– Interface matches a source interface– Class generalization matches hierarchy– Interface generalization matches hierarchy– UML operations correspond to methods– UML attributes correspond to fields– UML associations reflected in source



• Source UML Class Diagrams– Public classes appear in UML model– Public interfaces appear in UML model– Generalizations among public classes– Generalizations using interfaces– Public methods appear in UML model– Associations using fields appear in UML

• Test Cases Source– Source covered by last test not changed



• Documentation => Source– Parameters, See Also, Throws correspond to

source

• Source => Documentation– Public classes and interfaces are documented


Generic Constraints

• Some constraints don’t have a source– Style constraints (e.g. naming conventions)– Usage constraints (e.g. language rules)

• Some constraints have no artifact– Design patterns

• Database manager allows explicit tuples– Generic (style constraints)– Project-specific (design patterns)

• Constraints defined from these tuples


Current Defined Constraints

• Style Constraints– Class naming conventions– Method naming conventions– Field naming conventions– Local variable naming conventions– Constant naming conventions

• Pattern Constraints– Façade patterns


Current Defined Constraints

• Language usage constraints– All parameters not starting with _ are used– All fields are read at least once– All fields are written at least once– All methods are called at least once– Fields are either private or protected– Any routine returning type X must return a

new instance• Note this is specialized by the particular project


Maintaining Constraints• Metaconstraints

– Need to be translated into actual constraints– One for each matching item in the database

• Several tasks need to be done– Look for new constraints to be added– Check new constraints for validity– Check prior constraints for applicability– Check prior constraints for validity– Remove dead constraints

• This has to be done incrementally


Constraint Instances

• Basis for checking, updating, reporting• An instance contains

– The meta constraint– The database object that is its source (UID)– Database objects that validate/invalidate it

• Exists : The particular item that was found• For All : Any items that did not match

– Any tables that affect the constraint– Current status of the instance (valid/invalid)

• Stored in the database


CLIC: Generating Constraints

Metaconstraint

Constraint Manager

Test Query

{UID,true|false}

Constraint Instance

Item Query

Assoc UIDsAssoc Tables


Test Query

• XML metaconstraint mapped to SQL• Returns a set of UIDs and status

– SELECT x.uid,true FROM … WHERE <holds> UNION SELECT x.uid,false FROM … WHERE <not-holds>

• Restricted to UIDs that change– WHERE x.uid IN ...

• Used for new constraints• Used for checking existing constraints


Item Queries

• Find UIDs for a constraint instance– Of those items that made is succeed or fail

• XML metaconstraint mapped to SQL– Multiple queries generated for each instance

• One for each level of nesting• Using information from prior level

– Used to determine when to recheck constraint

• Table associations determined directly– For cases which are NOT-based


Showing Constraints: CLICUI

• Constraint instances– Have enough information for traceability

• UIDs of constraint mapped to database object• Database object has descriptive information

– Have enough information for understanding• UIDs of exceptions or associated objects• Again can be mapped to database objects

• What’s left is to report the results– In a meaningful way– Integrated with an environment


Constraint User Interface


CLICUI

• User interface for browsing constraints– By type, by status, by source

• Constraints kept up to date automatically• Details obtained from database

– Relation-based user output– Location information from relation

• Location information to the environment– “Goto” in the editor – Tell the environment to display something


Eclipse-based Interface


CLIME Experiences

• CLIME used for several projects– On a game program (UML, source, tests, …)– On itself (43,500 LOC)– On SOOT (200KLOC), Eclipse (1.2MLOC)

• CLIME on itself– Generates about 2300 constraints– Unobtrusively used– Bugs found through style and usage

constraints


CLIME: Summary

• Integrating evolving software is the next step in development environments

• This can be done by constructing and using constraints among the artifacts

• A wide variety of artifacts and constraints can be managed cheaply and effectively


CLIME: Summary

• The constraint mechanism works with existing tools and artifacts

• It extends to a variety of dimensions• It is extensible• It supports associating inconsistencies

with artifacts• It has low overhead and is unobtrusive


Current Research

• Additional artifacts and constraints– Documentation, language usage,

• Artifacts requiring complex constraints– UML interaction diagrams

• Behavioral specifications and constraints• User-friendly front ends


Complex Constraints

• Queries can’t check all constraints– Language constraints (CCEL/Parasoft Jtest)

require the abstract syntax tree– Dynamic constraints require static/dynamic

analysis

• The associated analysis is best done outside of queries


Artifact AnalysisSoftwareArtifacts

Abstractor

Update Information

Update Manager

Program Analysis

Result Files


Program Analyzers

• Abstract syntax tree queries– CCEL-type queries against a standard AST– Evaluated dynamically inside Jikes– Query results output in XML

• Static analysis queries– Finite-state specification of desired behavior– Static check if that behavior

• Must occur, can occur, must not occur

• Dynamic checking


UML Interaction Diagrams


UML Interaction Constraints

• What does it mean to check these– Statically using program analysis– Dynamically using trace information

• Build an FSA from interaction diagram– Check this against control flow– Check this against trace

• Difficult to do in a SQL query– But often doable through static analysis


Behavior Constraints

• How a class or library should be used– This is part of the specification– Should be part of evolution

• Ensuring new software meets specifications• Checking effects of changes to existing software

• Example– Always return new instance– Iterators in Java

• Initialize – { hasNext() – next() }


Checking Behavior

• Can sometimes be done statically– Through program analysis– Can detect most simple instances– Can isolate potential violations

• Can also be done dynamically– By adding checking code (aspects)– By looking at trace data


Defining Behavior

• Specify a set of program events– Calls, returns, variable sets, etc.– Can be parameterized (e.g. by object)

• Define extended finite state machines– Trigger event that creates machine instance– Machines have associated variables– Actions can set and test variables– Events cause transitions – Illegal transitions trigger errors


Behavior Constraint<CHECKS> <CHECK META="iterator_check" NAME="iterator_check"> <CLASSES> <USE ID="C1" CLASS="java.util.Iterator"/> <USE ID=“C2” CLASS=“java.util.Colleciton” /> </CLASSES> <IGNORE> <CLASS NAME="java." /> <CLASS NAME="javax." /> </IGNORE> <EVENTS> <EVENT ID="E1" NAME="NewIterator" TYPE="CALLRETURN" VALUE="C1" METHOD="iterator" TRIGGER="1" CLASS="C2" /> <EVENT ID="E2" NAME="hasNext" TYPE="CALLRETURN" OBJECT="C1" METHOD="hasNext" /> <EVENT ID="E3" NAME="next" TYPE="CALLRETURN" OBJECT="C1" METHOD="next" /> <EVENT ID="E4" NAME="hasMore" TYPE="CALLRETURN" OBJECT="C1" METHOD="hasMoreElements" /> <EVENT ID="E5" NAME="nextElement" TYPE="CALLRETURN" OBJECT="C1" METHOD="nextElement" /> </EVENTS> <AUTOMATON START="S1" TRIGGER="NEW">

<STATE ID="S1" ACCEPT="1" NAME="Start" > <ON EVENT="E1" GOTO="S2" />

</STATE> <STATE ID="S2" ACCEPT="1" NAME="IteratorAllocated" > <ON EVENT="E2" GOTO="S3" /> <ON EVENT="E3" GOTO="S4" /> <ON EVENT="E4" GOTO="S3" /> <ON EVENT="E5" GOTO="S4" /> </STATE> <STATE ID="S3" ACCEPT="1" NAME="hasNextCalled" > <ON EVENT="E3" GOTO="S2" /> <ON EVENT="E5" GOTO="S2" /> </STATE> <STATE ID="S4" ERROR="1" NAME="nextCalled“/ >

</AUTOMATON> </CHECK></CHECKS>


Behavior Checker: CLAC

• Takes FSM based definition• Does dynamic checking of application

– Currently using trace files– Easily modified to used class file patching

• Used to test iterators & tokenizers in several systems– Needed to make FSM more complex– Found one problem with our code this way


Future Research

• Show applicability to a wide variety of artifacts on real software systems

• Handling behavior specifications efficiently and statically where possible

• Extending this to handle component and net-based software– Appropriate representations and constraints

• Validate the approach on a large scale


Future Research

• Extending the concept from detecting dependencies to fixing them– Automatic updates are sometimes possible

• Configuration management using constraints

• New design tools based on constraints


Acknowledgements

• NSF (ACI9982266, CCR9988141, CCR9702188)

• Manos Renieris, Shriram Krishnamurthy• Christina Kennedy, Tom Wooldridge,

Brock Pytlik, Daniel Licata, Curran Nachbar, Tudor Antoniu


Questions / Comments

october 8, 2015climepage 1 constraining software evolution steven p. reiss brown university

Documents

artifacts consistencyinitially

databasessome artifacts

files nowidentify artifacts

separatelythus dimensions

dimensions of softwareover

artifactstypically files

existing tools artifactshandle

project manageror