Experiments on Design Pattern Discovery

Jing Dong and Yajing ZhaoDepartment of Computer ScienceThe University of Texas at Dallas{ jdong, yxz045100}@utdallas.edu

March 27, 2007 The University of Texas at Dallas 2

Outline

Introduction Related Work Our Approach Experiment Data Result and Discussion Precision and Recall Benchmark Conclusion and Future Work

March 27, 2007 The University of Texas at Dallas 3

Introduction

Difficult to understand large computer-based systems

No original architecture and design Patterns help on understanding systems Patterns embed future change of systems Need of design pattern discovery since

patterns are lost in source code

March 27, 2007 The University of Texas at Dallas 4

Introduction (Cont’d)

DP-Miner Experiment data: System source code

JUnit JEdit JHotDraw Java.AWT

Such experiments lead to benchmarks

March 27, 2007 The University of Texas at Dallas 5

Related Work

Au

thors

Tools

Ab

stract Factory

Ad

apter

Brid

ge

Bu

ilder

Ch

ain of

Resp

onsib

ility

Com

man

d

Com

posite

Decorator

Facad

e

Factory M

ethod

Flyw

eight

Ob

server

Prototyp

e

Proxy

Sin

gleton

Strategy

Tem

plate M

ethod

Visitor

Tsantalis 2006 X X X X X X X X X X X

Antoniol 1998 X X X X X

Heuzeroth 2003 X X X

Niere 2002 X X X

Blewitt 2001 X X X X

Balanyi 2003 X X X X X X X X X X X X X

Shi 2006 X X X X X X X X X X X X X X X

Gueheneuc 2006 X X X X X X X X X X X X X

March 27, 2007 The University of Texas at Dallas 6

Our Approach – DP-Miner

Matrix and Weight Product of Prime Numbers Structural, Behavioral, and Semantic Analysis

March 27, 2007 The University of Texas at Dallas 7

Experiment Data

JUnit: A regression test framework that helps developers to implement unit tests in Java.

JEdit: A mature and easy-to-use text editor. JHotDraw: A two dimensional graphics

frameworks for technical and structured drawing editors written in Java.

Java.AWT: A library for developing graphical user interfaces for Java programs.

March 27, 2007 The University of Texas at Dallas 8

Why Those Four Systems as Data? At the time they were developed, the idea of

design patterns is already mature and widely applied.

Other works on design pattern discovery use one or more of these systems to evaluate their approaches, which allows us to compare and evaluate our experiment results.

March 27, 2007 The University of Texas at Dallas 9

Experiment Data

Systems Version Class # File #

Java.AWT JDK1.4.2 570 345

JUnit 3.8.2 126 93

JEdit 4.2 1001 394

JHotDraw 6.0 beta 1 530 484

System information Version Number of Classes Number of Files

March 27, 2007 The University of Texas at Dallas 10

Experiment Result

Systems Adapter Bridge Strategy Composite

Java.AWT 21 65 76 3

JUnit 3 6 6 3

JEdit 17 24 24 0

JHotDraw 4 58 64 0

March 27, 2007 The University of Texas at Dallas 11

Experiment Result – Java.AWTResults for Each Analysis Phase

SystemsJava.AWT

Structural Analysis

Behavioral Analysis

Semantic Analysis

Final Result

Adapter 57 21 N/A 21

Bridge 100 76 65 65

Strategy 100 76 76 76

Composite 92 3 N/A 3

March 27, 2007 The University of Texas at Dallas 12

Experiment Result – JUnitResults for Each Analysis Phase

SystemsJUnit

Structural Analysis

Behavioral Analysis

Semantic Analysis

FinalResult

Adapter 15 3 N/A 3

Bridge 6 6 6 6

Strategy 6 6 6 6

Composite 9 3 N/A 3

March 27, 2007 The University of Texas at Dallas 13

Experiment Result – JEditResults for Each Analysis Phase

SystemsJEdit

Structural Analysis

Behavioral Analysis

Semantic Analysis

Final Result

Adapter 80 17 N/A 17

Bridge 33 24 24 24

Strategy 33 24 24 24

Composite 0 0 N/A 0

March 27, 2007 The University of Texas at Dallas 14

Experiment Result – JHotDrawResults for Each Analysis Phase

SystemsJHotDraw

Structural Analysis

Behavioral Analysis

Semantic Analysis

FinalResult

Adapter 27 4 N/A 4

Bridge 74 64 58 58

Strategy 74 64 64 64

Composite 0 0 N/A 0

March 27, 2007 The University of Texas at Dallas 15

Observations Behavioral analysis phase only deals with a small

number of classes, since it’s based on the result of structural analysis phase.

Significant reduction of the candidates between structural and behavioral analysis of Adapter and Composite pattern than those of Bridge and Strategy patterns, since the former have more behavioral characteristics than the latter do.

Result of structural and behavioral analysis of the Bridge and Strategy patterns are the same, since they have same structural and behavioral characteristics and only differ in their intents.

March 27, 2007 The University of Texas at Dallas 16

Recovery Precisions for JHotDraw

JHotDraw TP FP Precision

Adapter 4 0 100%

Bridge 53 5 91.38%

Strategy 58 6 90.63%

Composite 0 0 100%

Manually checked the results generated by our tool and see whether they are real pattern instances.

TP: True Positive FP: False Positive

March 27, 2007 The University of Texas at Dallas 17

Instances Missed Instances manually found but missed by DP-Miner Recall: 89.23%

CONTEXT STRATEGY

[1] LineConnection Connector

[2] ChangeConnectionHandle Connector

[3] ConnectionTool Connector

[4] PolygonHandle Locator

[5] LocatorHandle Locator

[6] LocatorConnector Locator

[7] SelectionTool DrawingView

March 27, 2007 The University of Texas at Dallas 18

Reasons of Discrepancy Flexibility of design pattern

Composite pattern: Collapse both the Component and Composite classes into a single class or not.

Adapter pattern: Full matching all three roles, i.e. Target, Adapter, and Adaptee, or partial matching of only Adapter and Adaptee.

Object-oriented programming languages provide special language constructs that greatly simplify the implementation of a design pattern. E.g., Java provides LinkedList, ArrayList, HashMap, and Hashtable, which make the implementation of aggregate elements in patterns easy, but the detection harder.

March 27, 2007 The University of Texas at Dallas 19

Benchmark Correct number of pattern instances and their locations

are generally not available. Lacking benchmarks is the main impediment

Hard to evaluate and compare design pattern discovery techniques.

Hard to judge whether an approach discovers all instances and whether the discovered ones are correct.

To calculate the precision and recall of a pattern matching result, it’s essential to know The number of correct pattern instances The locations of correct pattern instances

JHotDraw pattern benchmark

March 27, 2007 The University of Texas at Dallas 20

Conclusion We did a series of experiments on design pattern

discovery from open-source systems using DP-Miner Patterns concerned: Adapter, Bridge, Strategy,

Composite Experiment data: Java.AWT, JUnit, JEdit, JHotDraw We compared our experiment results with others and

found several discrepancies. We analyzed the issues and discussed possible

reasons. We argue for benchmarks for design pattern discovery.

March 27, 2007 The University of Texas at Dallas 21

Future Work

Improve our tool. Experiment on other systems Manually check results of other systems and

perfect the benchmark Research on other object-oriented languages,

such as C++.

March 27, 2007 The University of Texas at Dallas 22

Thank You!

March 27, 2007 The University of Texas at Dallas 23

Questions?

March 27, 2007 The University of Texas at Dallas 24

Overall Architecture of Our Approach

Structural Analysis Semantic Analysis Behavioral Analysis

design pattern

instances

Eliminates false positives

Extract structural information

Matrix TypeWeight

Structural constraints

Apply

Behavioral constraints

Instances from structural analysis

Apply

Instances from behavioral analysis

XMI file

Java Code

Scans Check for naming conventions for participating classes

System

Design Patternmatch

Source CodeIntermediate

Representation

March 27, 2007 The University of Texas at Dallas 25

Structural Analysis – MatrixButton

(from awt)

Canvas

base : String = "canvas"nameCounter : int = 0serialVersionUID : long

Canvas()Canvas()constructComponentName()addNotify()paint()update()postsOldMouseEvents()createBufferStrategy()createBufferStrategy()getBufferStrategy()getAccessibleContext()

(from awt)

Checkbox(from awt)

Choice(from awt)

Component(from awt)

Container(from awt)

parent

component[]

Label(from awt)

Scrollbar(from awt)

TextComponent(from awt)

March 27, 2007 The University of Texas at Dallas 26

Button

Canvas

Checkbox

Choice

Com

ponent

Container

Label

Scrollbar

TextC

omponent

Button 1 1 1 1 7 1 1 1 1

Canvas 1 1 1 1 7 1 1 1 1

Checkbox 1 1 1 1 7 1 1 1 1

Choice 1 1 1 1 7 1 1 1 1

Component 1 1 1 1 1 5 1 1 1

Container 1 1 1 1 35 1 1 1 1

Label 1 1 1 1 7 1 1 1 1

Scrollbar 1 1 1 1 7 1 1 1 1

TextComponent 1 1 1 1 7 1 1 1 1

March 27, 2007 The University of Texas at Dallas 27

Button

Canvas

Checkbox

Choice

Com

ponent

Container

Label

Scrollbar

TextC

omponent

Button 1 1 1 1 7 1 1 1 1

Canvas 1 1 1 1 7 1 1 1 1

Checkbox 1 1 1 1 7 1 1 1 1

Choice 1 1 1 1 7 1 1 1 1

Component 1 1 1 1 1 5 1 1 1

Container 1 1 1 1 35 1 1 1 1

Label 1 1 1 1 7 1 1 1 1

Scrollbar 1 1 1 1 7 1 1 1 1

TextComponent 1 1 1 1 7 1 1 1 1

Leaf Component Composite

Leaf 1 7 1

Component 1 1 1

Composite 1 35 1