software practices lab learning from the past gail murphy university of british columbia davor joint...

Software Practices Lab

Learning from the past

Gail MurphyGail Murphy

University of British ColumbiaUniversity of British Columbia

Joint work with Martin Robillard and DavorDavor Čubranić

© Copyright 2003, G. Murphy, D. Cubranic, and M. Robillard.Permission is granted to use this presentation, in whole or in part for educational purposes, provided that every slide used is used in its entirety, with no changes, deletions or additions; and that the

copyright notice is preserved on every slide used.

WCRE 2003 © 2003, G. Murphy M. Robillard, and D. Cubranic.

All rights reserved.

Roadmap

The past 10 years

Hipikat and FEAT

The next 10 years?



The past 10 years

• Relatively informed outsider• Not intended to be comprehensive• Focus on some trends

Roadmap



10 years ago

• Challenges to the Field of Reverse Engineering by Selfridge, Waters, and Chikofsky– Avoid artificially contrived

data– Focus on economic

impact– Facilitate communication



Since 1993

• Increased use of bigger, more realistic systems (often open-source)– e.g., LEDA library target (95kLOC), “Recovering Code

to Documentation Links in OO Systems”– e.g., Linux (1.5MLOC),

“Linux as a Case Study: Its Extracted Architecture”

• Helps avoid artificially contrived data and helps facilitate communication (and comparison)



Two “trends”

1. Focus on information analyzed from the current source code

2. Little discussion of costs to developer/maintainer



Consider 2002 proceedings

• 32 papers, 15 of which are tool-oriented

Costs to developer

1 of the 15 papers mentions cost in abstract

Information from source

All 15 papers Time

Req.

Source

Mail



Why reverse engineer?

• To do some present task more effectively

…

Bug fix

Feature addition orenhancement

Reengineering

Retargeting

Faster

Cheaper

“Better”



Trends to opportunities


Consider a wide range of artifacts from the past

Requirements

Design

Source

Bugs

Mail

HipikatTime



Trends to opportunities

2. Little discussion of costs to developer

Ensure techniques are low-cost and/or Increase longevity of result

Requirements

Design

Source

Bugs

Mail

FEATTime

Hipikat



Roadmap

The past 10 years

Hipikat and FEAT

Goal: To help developers perform some present task more effectivelyProperties: Use archived artifacts



Hipikat: The idea

• The best information about a software system is in the heads of the developers who work on the system

• Treat all past system artifacts as a collective group memory about the system

• Recommend artifacts from this collective memory as a developer performs a task

Requirements

Design

Source

Bugs

Mail

HipikatTime



(Initial) Target: Virtual teams

• Distributed across locations and time

• Often little or no face-to-face contact

• Collaborate electronically:– Source code versioning– Archives of electronic communication (e.g.,

newsgroups, emails, documents)– Issue tracking system (e.g., Bugzilla)



Information schema

similar toreply toMessage

postsworks on

writes

about

writesPerson

implements

Change/Bug

Filerevision

similar to

documents

Document



Information schema: Metadata links

Message

Document similar to

documents

similar toreply to

writes

postsworks on

writes

implements

about

Person

Change/Bug

Filerevision

“From:”, “Assigned to:” etc.



Information schema: Heuristics

Message

Document similar to

documents

similar toreply to

writes

postsworks on

writes

implements

about

Person

Change/Bug

Filerevision

Text pattern andtemporal activity

matching



Information schema: IR techniques

Message

Document similar to

documents

similar toreply to

writes

postsworks on

writes

implements

about

Person

Change/Bug

Filerevision

Document vectorsimilarity



Recommending artifacts


postsworks on

writes

about

writesPerson

implements

Change/Bug

Filerevision

similar to

documents

Document

Change/Bug

Filerevision



Recommending artifacts


postsworks on

writes

about

writesPerson

implements

Change/Bug

Filerevision

similar to

documents

Document

Change/Bug

FileRevision

Message



Hipikat: The current tool

• Instantiated for eclipse.org• Runs as an Eclipse plug-in

Eclipse developer withHipikat plug-in

Hipikat Server

eclipse.org

QueryDailyupdates

eclipse.org

• Intense development– Average of 107 file revisions checked in daily– No weekday with < 21 new bugs entered– No Wed. with < 203 comments posted on bugs

• Active developer lists– Typically over 100 articles/day on newsgroups

• Artifact database size– 200,000+ file revisions– 40,000+ bugs– 50,000+ articles



Accessing Hipikat



Accessing Hipikat

Query Hipikat

Bug viewerPackagenavigator

Query Hipikat

Or from the Editor, Console, Resource view, etc.



Hipikat recommendations



Hipikat results: Experimental view



An example

• A project newcomer is about to start working on an enhancement request

• Look at similar enhancements and bug fixes that were done in thepast, and see how they were implemented– starting point for learning the API– code to reuse– project idioms and implementation patterns



Description of the change



Hipikat recommendations



Investigate suggestions



New recommendations



Learn from the fix



Another example



Does it work?

• We performed a quasi-experiment using two tasks from the Eclipse archives– “easy” task: display breakpoint properties in hover– “difficult” task: improve user interaction with versioning

8 newcomershad Hipikat

4 “experts”did not have Hipikat



Does it work?

0 hours

“Easy” task

2.5 hours

All participants solved the task

75%

25%

75% of newcomers handled special casescorrectly compared to only 25% of experts

“Difficult” task

0 hours 2.5 hours

1 expert and 4 newcomers didnot complete all cases of the task

75%50% 75% of experts

met basic req.compared to 50% of newcomers



Reverse engineering?

• Hipikat complements reverse engineering techniques– May reduce times when

reverse engineering is needed

– Can serve as a delivery mechanism of previously reverse engineered knowledge

Requirements

Design

Source

Bugs

Mail

HipikatTime



Roadmap

The past 10 years

Hipikat and FEAT

Goal: To help developers perform some present task more effectivelyProperties: Low cost to developer andabstractions created can be long-lived

FEAT: The idea

• Not all “concern” code can be modularized

• Support a developer in finding and describing latent concerns as part of a modification task

• Enables systematic change, more reasoning, etc.

public class Ant extends Task { private File dir = null; private String antFile = null; private String target = null; private String output = null; Vector properties=new Vector(); Project p1; public void init() { p1 = new Project(); p1.setJavaVersionProperty(); p1.addTaskDefinition("property", (Class)project.getTaskDefinitions().get("property")); } private void reinit() { init(); for (int i=0; i<properties.size(); i++) { Property p = (Property) properties.elementAt(i); Property newP = (Property) p1.createTask("property"); newP.setName(p.getName()); if (p.getValue() != null) { newP.setValue(p.getValue()); } if (p.getFile() != null) { newP.setFile(p.getFile()); } if (p.getResource() != null) { newP.setResource(p.getResource()); } properties.setElementAt(newP, i); } } private void initializeProject() { Vector listeners = project.getBuildListeners(); for (int i = 0; i < listeners.size(); i++) { p1.addBuildListener((BuildListener)listeners.elementAt(i)); } if (output != null) { try { PrintStream out = new PrintStream(new FileOutputStream(output)); DefaultLogger logger = new DefaultLogger(); logger.setMessageOutputLevel(Project.MSG_INFO); logger.setOutputPrintStream(out); logger.setErrorPrintStream(out); p1.addBuildListener(logger); } catch( IOException ex ) { log( "Ant: Can't set output to " + output ); } } Hashtable taskdefs = project.getTaskDefinitions(); Enumeration et = taskdefs.keys(); while (et.hasMoreElements()) { String taskName = (String) et.nextElement(); Class taskClass = (Class) taskdefs.get(taskName); p1.addTaskDefinition(taskName, taskClass); } Hashtable typedefs = project.getDataTypeDefinitions(); Enumeration e = typedefs.keys(); while (e.hasMoreElements()) { String typeName = (String) e.nextElement(); Class typeClass = (Class) typedefs.get(typeName); p1.addDataTypeDefinition(typeName, typeClass); } // set user-define properties Hashtable prop1 = project.getProperties(); e = prop1.keys(); while (e.hasMoreElements()) { String arg = (String) e.nextElement(); String value = (String) prop1.get(arg); p1.setProperty(arg, value); } } /** * Do the execution. */ public void execute() throws BuildException { try { if (p1 == null) { reinit(); } if(dir == null) dir = project.getBaseDir(); initializeProject(); p1.setBaseDir(dir); p1.setUserProperty("basedir" , dir.getAbsolutePath()); // Override with local-defined properties Enumeration e = properties.elements(); while (e.hasMoreElements()) { Property p=(Property) e.nextElement(); p.execute(); } if (antFile == null) antFile = "build.xml"; File file = new File(antFile); if (!file.isAbsolute()) { antFile = (new File(dir, antFile)).getAbsolutePath(); file = (new File(antFile)) ; if( ! file.isFile() ) { throw new BuildException("Build file " + file + " not found."); } } p1.setUserProperty( "ant.file" , antFile ); ProjectHelper.configureProject(p1, new File(antFile)); if (target == null) { target = p1.getDefaultTarget(); } // Are we trying to call the target in which we are defined? if (p1.getBaseDir().equals(project.getBaseDir()) && p1.getProperty("ant.file").equals(project.getProperty("ant.file")) && target.equals(this.getOwningTarget().getName())) { throw new BuildException("ant task calling its own parent target"); } p1.executeTarget(target); } finally { // help the gc p1 = null; } } public void setDir(File d) { this.dir = d; } public void setAntfile(String s) { this.antFile = s; } public void setTarget(String s) { this.target = s; } public void setOutput(String s) { this.output = s; } public Property createProperty() { if (p1 == null) { reinit(); } Property p=(Property)p1.createTask("property"); p.setUserProperty(true); properties.addElement( p ); return p; }}

public class Ant extends Task { /** the basedir where is executed the build file */ private File dir = null; /** the build.xml file (can be absolute) in this case dir will be ignored */ private String antFile = null; /** the target to call if any */ private String target = null; /** the output */ private String output = null; /** should we inherit properties from the parent ? */ private boolean inheritAll = true; /** should we inherit references from the parent ? */ private boolean inheritRefs = false; /** the properties to pass to the new project */ private Vector properties = new Vector(); /** the references to pass to the new project */ private Vector references = new Vector(); /** the temporary project created to run the build file */ private Project newProject; /** * If true, inherit all properties from parent Project * If false, inherit only userProperties and those defined * inside the ant call itself */ public void setInheritAll(boolean value) { inheritAll = value; } /** * If true, inherit all references from parent Project * If false, inherit only those defined * inside the ant call itself */ public void setInheritRefs(boolean value) { inheritRefs = value; } public void init() { newProject = new Project(); newProject.setJavaVersionProperty(); newProject.addTaskDefinition("property", (Class)project.getTaskDefinitions().get("property")); } private void reinit() { init(); final int count = properties.size(); for (int i = 0; i < count; i++) { Property p = (Property) properties.elementAt(i); Property newP = (Property) newProject.createTask("property"); newP.setName(p.getName()); if (p.getValue() != null) { newP.setValue(p.getValue()); } if (p.getFile() != null) { newP.setFile(p.getFile()); } if (p.getResource() != null) { newP.setResource(p.getResource()); } properties.setElementAt(newP, i); } } private void initializeProject() { Vector listeners = project.getBuildListeners (); final int count = listeners.size(); for (int i = 0; i < count; i++) { newProject.addBuildListener((BuildListener)listeners.elementAt(i)); } if (output != null) { try { PrintStream out = new PrintStream(new FileOutputStream(output)); DefaultLogger logger = new DefaultLogger(); logger.setMessageOutputLevel(Project.MSG_INFO); logger.setOutputPrintStream(out); logger.setErrorPrintStream(out); newProject.addBuildListener(logger); } catch( IOException ex ) { log( "Ant: Can't set output to " + output ); } } Hashtable taskdefs = project.getTaskDefinitions(); Enumeration et = taskdefs.keys(); while (et.hasMoreElements()) { String taskName = (String) et.nextElement(); if (taskName.equals("property")) { // we have already added this taskdef in #init continue; } Class taskClass = (Class) taskdefs.get(taskName); newProject.addTaskDefinition(taskName, taskClass); } Hashtable typedefs = project.getDataTypeDefinitions(); Enumeration e = typedefs.keys(); while (e.hasMoreElements()) { String typeName = (String) e.nextElement(); Class typeClass = (Class) typedefs.get(typeName); newProject.addDataTypeDefinition(typeName, typeClass); } // set user-defined or all properties from calling project Hashtable prop1; if (inheritAll) { prop1 = project.getProperties(); } else { prop1 = project.getUserProperties(); // set Java built-in properties separately, // b/c we won't inherit them. newProject.setSystemProperties(); } e = prop1.keys(); while (e.hasMoreElements()) { String arg = (String) e.nextElement(); if ("basedir".equals(arg) || "ant.file".equals(arg)) { // basedir and ant.file get special treatment in execute() continue; } String value = (String) prop1.get(arg); if (inheritAll){ newProject.setProperty(arg, value); } else { newProject.setUserProperty(arg, value); } } } protected void handleOutput(String line) { if (newProject != null) { newProject.demuxOutput(line, false); } else { super.handleOutput(line); } } protected void handleErrorOutput(String line) { if (newProject != null) { newProject.demuxOutput(line, true); } else { super.handleErrorOutput(line); } } /** * Do the execution. */ public void execute() throws BuildException { try { if (newProject == null) { reinit(); } if ( (dir == null) && (inheritAll) ) { dir = project.getBaseDir(); } initializeProject(); if (dir != null) { newProject.setBaseDir(dir); newProject.setUserProperty("basedir" , dir.getAbsolutePath()); } else { dir = project.getBaseDir(); } overrideProperties(); if (antFile == null) { antFile = "build.xml"; } File file = FileUtils.newFileUtils().resolveFile(dir, antFile); antFile = file.getAbsolutePath(); log("calling target "+(target!=null?target:"[default]") + " in build file "+ antFile.toString(), Project.MSG_VERBOSE); newProject.setUserProperty( "ant.file" , antFile ); ProjectHelper.configureProject(newProject, new File(antFile)); if (target == null) { target = newProject.getDefaultTarget(); } addReferences(); // Are we trying to call the target in which we are defined? if (newProject.getBaseDir().equals(project.getBaseDir()) && newProject.getProperty("ant.file").equals(project.getProperty("ant.file")) && getOwningTarget() != null && target.equals(this.getOwningTarget().getName())) { throw new BuildException("ant task calling its own parent target"); } newProject.executeTarget(target); } finally { // help the gc newProject = null; } } /** * Override the properties in the new project with the one * explicitly defined as nested elements here. */ private void overrideProperties() throws BuildException { Enumeration e = properties.elements(); while (e.hasMoreElements()) { Property p = (Property) e.nextElement(); p.setProject(newProject); p.execute(); } } /** * Add the references explicitly defined as nested elements to the * new project. Also copy over all references that don't override * existing references in the new project if inheritall has been * requested. */ private void addReferences() throws BuildException { Hashtable thisReferences = (Hashtable) project.getReferences().clone(); Hashtable newReferences = newProject.getReferences(); Enumeration e; if (references.size() > 0) { for(e = references.elements(); e.hasMoreElements();) { Reference ref = (Reference)e.nextElement(); String refid = ref.getRefId(); if (refid == null) { throw new BuildException("the refid attribute is required for reference elements"); } if (!thisReferences.containsKey(refid)) { log("Parent project doesn't contain any reference '" + refid + "'", Project.MSG_WARN); continue; } thisReferences.remove(refid); String toRefid = ref.getToRefid(); if (toRefid == null) { toRefid = refid; } copyReference(refid, toRefid); } } // Now add all references that are not defined in the // subproject, if inheritRefs is true if (inheritRefs) { for(e = thisReferences.keys(); e.hasMoreElements();) { String key = (String)e.nextElement(); if (newReferences.containsKey(key)) { continue; } copyReference(key, key); } } } /** * Try to clone and reconfigure the object referenced by oldkey in * the parent project and add it to the new project with the key * newkey. * * If we cannot clone it, copy the referenced object itself and * keep our fingers crossed. */ private void copyReference(String oldKey, String newKey) { Object orig = project.getReference(oldKey); Class c = orig.getClass(); Object copy = orig; try { Method cloneM = c.getMethod("clone", new Class[0]); if (cloneM != null) { copy = cloneM.invoke(orig, new Object[0]); } } catch (Exception e) { // not Clonable } if (copy instanceof ProjectComponent) { ((ProjectComponent) copy).setProject(newProject); } else { try { Method setProjectM = c.getMethod( "setProject", new Class[] {Project.class}); if(setProjectM != null) { setProjectM.invoke(copy, new Object[] {newProject}); } } catch (NoSuchMethodException e) { // ignore this if the class being referenced does not have // a set project method. } catch(Exception e2) { String msg = "Error setting new project instance for reference with id " + oldKey; throw new BuildException(msg, e2, location); } } newProject.addReference(newKey, copy); } /** * ... */ public void setDir(File d) { this.dir = d; } /** * set the build file, it can be either absolute or relative. * If it is absolute, <tt>dir</tt> will be ignored, if it is * relative it will be resolved relative to <tt>dir</tt>. */ public void setAntfile(String s) { // @note: it is a string and not a file to handle relative/absolute // otherwise a relative file will be resolved based on the current // basedir. this.antFile = s; } /** * set the target to execute. If none is defined it will * execute the default target of the build file */ public void setTarget(String s) { this.target = s; } public void setOutput(String s) { this.output = s; } /** create a property to pass to the new project as a 'user property' */ public Property createProperty() { if (newProject == null) { reinit(); } Property p = new Property(true); p.setProject(newProject); p.setTaskName("property"); properties.addElement( p ); return p; } /** * create a reference element that identifies a data type that * should be carried over to the new project. */ public void addReference(Reference r) { references.addElement(r); } /** * Helper class that implements the nested <reference> * element of <ant> and <antcall>. */ public static class Reference extends org.apache.tools.ant.types.Reference { public Reference() {super();} private String targetid=null; public void setToRefid(String targetid) { this.targetid=targetid; } public String getToRefid() { return targetid; } }}

public class Ant extends Task { /** the basedir where is executed the build file */ private File dir = null; /** * the build.xml file (can be absolute) in this case dir will be * ignored */ private String antFile = null; /** the target to call if any */ private String target = null; /** the output */ private String output = null; /** should we inherit properties from the parent ? */ private boolean inheritAll = true; /** should we inherit references from the parent ? */ private boolean inheritRefs = false; /** the properties to pass to the new project */ private Vector properties = new Vector(); /** the references to pass to the new project */ private Vector references = new Vector(); /** the temporary project created to run the build file */ private Project newProject; /** The stream to which output is to be written. */ private PrintStream out = null; /** * If true, pass all properties to the new Ant project. * Defaults to true. */ public void setInheritAll(boolean value) { inheritAll = value; } /** * If true, pass all references to the new Ant project. * Defaults to false. */ public void setInheritRefs(boolean value) { inheritRefs = value; } /** * Creates a Project instance for the project to call. */ public void init() { newProject = new Project(); newProject.setJavaVersionProperty(); newProject.addTaskDefinition("property", (Class) project.getTaskDefinitions() .get("property")); } /** * Called in execute or createProperty if newProject is null. * * This can happen if the same instance of this task is run * twice as newProject is set to null at the end of execute (to * save memory and help the GC). * * Sets all properties that have been defined as nested * property elements. */ private void reinit() { init(); final int count = properties.size(); for (int i = 0; i < count; i++) { Property p = (Property) properties.elementAt(i); Property newP = (Property) newProject.createTask("property"); newP.setName(p.getName()); if (p.getValue() != null) { newP.setValue(p.getValue()); } if (p.getFile() != null) { newP.setFile(p.getFile()); } if (p.getResource() != null) { newP.setResource(p.getResource()); } if (p.getPrefix() != null) { newP.setPrefix(p.getPrefix()); } if (p.getRefid() != null) { newP.setRefid(p.getRefid()); } if (p.getEnvironment() != null) { newP.setEnvironment(p.getEnvironment()); } if (p.getClasspath() != null) { newP.setClasspath(p.getClasspath()); } properties.setElementAt(newP, i); } } /** * Attaches the build listeners of the current project to the new * project, configures a possible logfile, transfers task and * data-type definitions, transfers properties (either all or just * the ones specified as user properties to the current project, * depending on inheritall), transfers the input handler. */ private void initializeProject() { newProject.setInputHandler(getProject().getInputHandler()); Vector listeners = project.getBuildListeners(); final int count = listeners.size(); for (int i = 0; i < count; i++) { newProject.addBuildListener((BuildListener) listeners.elementAt(i)); } if (output != null) { File outfile = null; if (dir != null) { outfile = FileUtils.newFileUtils().resolveFile(dir, output); } else { outfile = getProject().resolveFile(output); } try { out = new PrintStream(new FileOutputStream(outfile)); DefaultLogger logger = new DefaultLogger(); logger.setMessageOutputLevel(Project.MSG_INFO); logger.setOutputPrintStream(out); logger.setErrorPrintStream(out); newProject.addBuildListener(logger); } catch (IOException ex) { log("Ant: Can't set output to " + output); } } Hashtable taskdefs = project.getTaskDefinitions(); Enumeration et = taskdefs.keys(); while (et.hasMoreElements()) { String taskName = (String) et.nextElement(); if (taskName.equals("property")) { // we have already added this taskdef in #init continue; } Class taskClass = (Class) taskdefs.get(taskName); newProject.addTaskDefinition(taskName, taskClass); } Hashtable typedefs = project.getDataTypeDefinitions(); Enumeration e = typedefs.keys(); while (e.hasMoreElements()) { String typeName = (String) e.nextElement(); Class typeClass = (Class) typedefs.get(typeName); newProject.addDataTypeDefinition(typeName, typeClass); } // set user-defined properties getProject().copyUserProperties(newProject); if (!inheritAll) { // set Java built-in properties separately, // b/c we won't inherit them. newProject.setSystemProperties(); } else { // set all properties from calling project Hashtable props = getProject().getProperties(); e = props.keys(); while (e.hasMoreElements()) { String arg = e.nextElement().toString(); if ("basedir".equals(arg) || "ant.file".equals(arg)) { // basedir and ant.file get special treatment in execute() continue; } String value = props.get(arg).toString(); // don't re-set user properties, avoid the warning message if (newProject.getProperty(arg) == null){ // no user property newProject.setNewProperty(arg, value); } } } } /** * Pass output sent to System.out to the new project. * * @since Ant 1.5 */ protected void handleOutput(String line) { if (newProject != null) { newProject.demuxOutput(line, false); } else { super.handleOutput(line); } } /** * Pass output sent to System.err to the new project. * * @since Ant 1.5 */ protected void handleErrorOutput(String line) { if (newProject != null) { newProject.demuxOutput(line, true); } else { super.handleErrorOutput(line); } } /** * Do the execution. */ public void execute() throws BuildException { File savedDir = dir; String savedAntFile = antFile; String savedTarget = target; try { if (newProject == null) { reinit(); } if ((dir == null) && (inheritAll)) { dir = project.getBaseDir(); } initializeProject(); if (dir != null) { newProject.setBaseDir(dir); if (savedDir != null) { // has been set explicitly newProject.setInheritedProperty("basedir" , dir.getAbsolutePath()); } } else { dir = project.getBaseDir(); } overrideProperties(); if (antFile == null) { antFile = "build.xml"; } File file = FileUtils.newFileUtils().resolveFile(dir, antFile); antFile = file.getAbsolutePath(); log("calling target " + (target != null ? target : "[default]") + " in build file " + antFile.toString(), Project.MSG_VERBOSE); newProject.setUserProperty("ant.file" , antFile); ProjectHelper.configureProject(newProject, new File(antFile)); if (target == null) { target = newProject.getDefaultTarget(); } addReferences(); // Are we trying to call the target in which we are defined? if (newProject.getBaseDir().equals(project.getBaseDir()) && newProject.getProperty("ant.file").equals(project.getProperty("ant.file")) && getOwningTarget() != null && target.equals(this.getOwningTarget().getName())) { throw new BuildException("ant task calling its own parent " + "target"); } newProject.executeTarget(target); } finally { // help the gc newProject = null; if (output != null && out != null) { try { out.close(); } catch (final Exception e) { //ignore } } dir = savedDir; antFile = savedAntFile; target = savedTarget; } } /** * Override the properties in the new project with the one * explicitly defined as nested elements here. */ private void overrideProperties() throws BuildException { Enumeration e = properties.elements(); while (e.hasMoreElements()) { Property p = (Property) e.nextElement(); p.setProject(newProject); p.execute(); } getProject().copyInheritedProperties(newProject); } /** * Add the references explicitly defined as nested elements to the * new project. Also copy over all references that don't override * existing references in the new project if inheritrefs has been * requested. */ private void addReferences() throws BuildException { Hashtable thisReferences = (Hashtable) project.getReferences().clone(); Hashtable newReferences = newProject.getReferences(); Enumeration e; if (references.size() > 0) { for (e = references.elements(); e.hasMoreElements();) { Reference ref = (Reference) e.nextElement(); String refid = ref.getRefId(); if (refid == null) { throw new BuildException("the refid attribute is required" + " for reference elements"); } if (!thisReferences.containsKey(refid)) { log("Parent project doesn't contain any reference '" + refid + "'", Project.MSG_WARN); continue; } thisReferences.remove(refid); String toRefid = ref.getToRefid(); if (toRefid == null) { toRefid = refid; } copyReference(refid, toRefid); } } // Now add all references that are not defined in the // subproject, if inheritRefs is true if (inheritRefs) { for (e = thisReferences.keys(); e.hasMoreElements();) { String key = (String) e.nextElement(); if (newReferences.containsKey(key)) { continue; } copyReference(key, key); } } } /** * Try to clone and reconfigure the object referenced by oldkey in * the parent project and add it to the new project with the key * newkey. * * If we cannot clone it, copy the referenced object itself and * keep our fingers crossed. */ private void copyReference(String oldKey, String newKey) { Object orig = project.getReference(oldKey); Class c = orig.getClass(); Object copy = orig; try { Method cloneM = c.getMethod("clone", new Class[0]); if (cloneM != null) { copy = cloneM.invoke(orig, new Object[0]); } } catch (Exception e) { // not Clonable }

if (copy instanceof ProjectComponent) { ((ProjectComponent) copy).setProject(newProject); } else { try { Method setProjectM = c.getMethod("setProject", new Class[] {Project.class}); if (setProjectM != null) { setProjectM.invoke(copy, new Object[] {newProject}); } } catch (NoSuchMethodException e) { // ignore this if the class being referenced does not have // a set project method. } catch (Exception e2) { String msg = "Error setting new project instance for " + "reference with id " + oldKey; throw new BuildException(msg, e2, location); } } newProject.addReference(newKey, copy); } /** * The directory to use as a base directory for the new Ant project. * Defaults to the current project's basedir, unless inheritall * has been set to false, in which case it doesn't have a default * value. This will override the basedir setting of the called project. */ public void setDir(File d) { this.dir = d; } /** * The build file to use. * Defaults to "build.xml". This file is expected to be a filename relative * to the dir attribute given. */ public void setAntfile(String s) { // @note: it is a string and not a file to handle relative/absolute // otherwise a relative file will be resolved based on the current // basedir. this.antFile = s; } /** * The target of the new Ant project to execute. * Defaults to the new project's default target. */ public void setTarget(String s) { this.target = s; } /** * Filename to write the output to. * This is relative to the value of the dir attribute * if it has been set or to the base directory of the * current project otherwise. */ public void setOutput(String s) { this.output = s; } /** * Property to pass to the new project. * The property is passed as a 'user property' */ public Property createProperty() { if (newProject == null) { reinit(); } Property p = new Property(true, getProject()); p.setProject(newProject); p.setTaskName("property"); properties.addElement(p); return p; } /** * Reference element identifying a data type to carry * over to the new project. */ public void addReference(Reference r) { references.addElement(r); } /** * Helper class that implements the nested <reference> * element of <ant> and <antcall>. */ public static class Reference extends org.apache.tools.ant.types.Reference { /** Creates a reference to be configured by Ant */ public Reference() { super(); } private String targetid = null; /** * Set the id that this reference to be stored under in the * new project. * * @param targetid the id under which this reference will be passed to * the new project */ public void setToRefid(String targetid) { this.targetid = targetid; } /** * Get the id under which this reference will be stored in the new * project * * @return the id of the reference in the new project. */ public String getToRefid() { return targetid; } }}

Version 1 Version 2 Version 3



An example: JHotDrawTask: Disable meaningless menu entries for selections



JHotDraw: Commands available

if (currentMenu instanceof CommandMenu) ((CommandMenu)currentMenu).checkEnabled();

if (currentMenu instanceof CommandMenu) ((CommandMenu)currentMenu).checkEnabled();



JHotDraw: Figure attributes

class AttributeFigure { setAttribute(String,Object);}

class AttributeFigure { setAttribute(String,Object);}



JHotDraw: Two concerns for task

Commands

Figures &attributes

Concern: high-level concept in the mind of a developer that has a corresponding mapping in source code

Concern: high-level concept in the mind of a developer that has a corresponding mapping in source code



The underlying idea



With automatic mapping to the relevant source code

The Concern Graph Idea

Support for program queries

• Capture the essence of the concern code (as a Concern Graph)

• Support mapping of concern back to the code

• Support program queries



FEAT



Does it work?

Study (kLOC) Useful Low-cost

AVID (13)

jEdit (65)

Jex (57)

Redback (>100)

Involved external subjects

• Can CGs adequately capture relevant code?

• How does a CG help?

• Why is the behavior of developers using CG more systematic?

• Can CGs adequately capture relevant code?

• How does a CG help?

• Why is the behavior of developers using CG more systematic?

• Can developers build CGs without difficulty during program investigation?

• Does the technique scale?

• Can developers build CGs without difficulty during program investigation?

• Does the technique scale?



Two properties

1. Low-cost to developer to create concern descriptions

• Product of a developer’s navigation through code

• Automatic detection algorithm

2. Concern descriptions can be long-lived• Can detect and repair inconsistencies



The Concern Graph structure

Concern Graph

Concern*

Concern*Fragments*

Domain: (A.method1())Domain: (A.method1())

Relation: (called by)Relation: (called by)

Range: (ALL)Range: (ALL)

Projection: (B.caller1(), C.caller2())Projection: (B.caller1(), C.caller2())

Intent

Mapping



FEAT and inconsistency management

• Create a Concern Graph(s) for the disabling menu entry task in Version 5.3 of JHotDraw

• Import the Concern Graph into Version 5.4 of JHotDraw for a similar task

• FEAT will compare the Mappings of each Intended fragment

• Developer can examine any inconsistencies and in some cases, the tool can help repair any differences



FEAT and inconsistency management

Call is present in source code but not in Concern Graph

Repair

In this case, adds relationshipinto Concern Graph



FEAT and Hipikat

• Concern Graphs are cheap to create and can be meaningful on subsequent versions of the system

• Hipikat can be a way to inform developers of relevant Concern Graphs

Requirements

Design

Source

Bugs

Mail

HipikatTimeFEAT



FEAT and Hipikat

AttributeMenuGreying Concern Graph Code Involved



Roadmap

The past 10 years

Hipikat and FEAT

The next 10 years?



Lessons learned from Hipikat & FEAT

• Context matters– For the present– For the past

• Fluid integration of past and present information matters

when the goal is….

to help a developer perform a present task more effectively



The next 10 years: Context

Developer’s current context

Historical artifact’s context

Fixing bug #?Debugging?New feature?public class BoxHandleKit {

/** * Fills the given collection */static public void addCornerHandles(Figure f, List handles) {handles.add(southEast(f));handles.add(southWest(f));handles.add(northEast(f));handles.add(northWest(f));}

Exploring old revisionsDetermining relevancy of design rationale

User modelling?

Situational cognition?



The next 10 years: Fluid integration

Can we (subtly) notify the user of relevant information from the past?Can we make it easy to share information that is generated?



Summary: Trends to opportunities


Consider a wide range of artifacts from the past

Requirements

Design

Source

Bugs

Mail

HipikatTime

Hipikat



Summary: Trends to opportunities

2. Little discussion of costs to developer

Reduce cost of using technique and/or Increase longevity of result

Requirements

Design

Source

Bugs

Mail

FEATTime



Summary

• Current focus is in the present

• Potential in reaching into artifacts from the past

• Challenge is to determine how to make items of (some) lasting value accessible in the future

Requirements

Design

Source

Bugs

Mail

Present System



Learning from the past

For more information on Hipikat or FEAT:

www.cs.ubc.ca/labs/splHipikat is joint work with Davor Čubranić, Kellogg Booth and Janice Singer (NRC). Kaili Vesik, Derek Shimozawa and Shawn Minto have contributed to its implementation.

FEAT is joint work with Martin Robillard. Jason Xu has contributed to its implementation.

Software Practices Lab

software practices lab learning from the past gail murphy university of british columbia davor joint...

Documents

source mail slide

communication slide

developermaintainer

comparison slide

trends roadmap slide

martin robillard

cheaper better slide

years hipikat