Generating Software Documentation in Use Case Maps

from Filtered Execution Traces

Edna Braun, Daniel Amyot, Timothy LethbridgeUniversity of Ottawa, Canada

damyot@uottawa.ca

SDL 2015, Berlin, October 14

SDL 2015 – p. 2

Motivation

• Maintenance and evolution of large legacy software

• User-friendly, automated software comprehension

• Focus on the elimination of utilities from long execution traces, with documentation and visualization based on Use Case Maps

SDL 2015 – p. 3

Program Comprehension

• Static Analysis– Based on code structure– Recovery of packages, classes, components,

attributes, associations and other relationships… • Dynamic Analysis

– Based on traces and/or other behavioral information– Recovery of invocation sequences, scenarios, state

machines… • Hybrid Approaches

– Combine (parts of) static and dynamic analyses

SDL 2015 – p. 4

Context

• For most software applications, dynamic analysis based on execution traces is difficult– Very, very lengthy traces, even for simple features

• Several approaches for component filtering and for trace compression exist– E.g., filtering out simple getters and setters in Java– But traces are still large– Need to use many approaches in practice

• Different visualizations exist– MSC / UML sequence diagrams, (business)

processes…

SDL 2015 – p. 5

Trace

Exam

ple

(Extr

act

)main 0

Entered: cruise/umple/UmpleConsoleMain.optParse 1

Entered: joptsimple/OptionParser.<init> 2

Entered: joptsimple/BuiltinHelpFormatter.<init> 3

Exited: joptsimple/BuiltinHelpFormatter.<init>

Entered: joptsimple/internal/AbbreviationMap.<init> 3

Exited: joptsimple/internal/AbbreviationMap.<init>

Entered: joptsimple/OptionParserState.moreOptions 3

Entered: joptsimple/OptionParserState$2.<init> 4

Entered: joptsimple/OptionParserState.<init> 5

Exited: joptsimple/OptionParserState.<init>

Exited: joptsimple/OptionParserState$2.<init>

Exited: joptsimple/OptionParserState.moreOptions

Exited: joptsimple/OptionParser.<init>

Entered: joptsimple/OptionParser.acceptsAll 2

Entered: joptsimple/ParserRules.<clinit> 3

Exited: joptsimple/ParserRules.<clinit>

Entered: joptsimple/ParserRules.ensureLegalOptions 3

Entered: joptsimple/ParserRules.ensureLegalOption 4

SDL 2015 – p. 6

Problem and Approach

• Can we detect and filter out utilities in software in order to shorten traces to be visualized and used for program comprehension, while preserving the traces’ meaning?

• Exploration of hybrid approaches– Different algorithms for utility detection, based on

static and dynamic information– Use of execution traces for dynamic analysis and

recovery of high-level behaviour– Visualization with Use Case Maps– Experiments with OO programs in Java

SDL 2015 – p. 7

Process Overview

Application +

Source Code GenerateLong

Trace

FilterShort

Trace

Visualize

UCMAdjust Filter

and Visualize

HelpComprehend

UCM

SDL 2015 – p. 8

Definitions

• A utility is any element of a program designed for the convenience of the designer and implementer and intended to be accessed from multiple places within a certain scope of the program

• Utilityhood is a metric defined as the extent to which a particular method can be considered a utility. • Calculated with dynamic and static variables• When the utilityhood is high, remove from the trace• Non-utility methods help describe features and

business logic

SDL 2015 – p. 9

TracetoUCMTool

SDL 2015 – p. 10

SDL 2015 – p. 11

Static Metrics

• Collected with Metrics 1.3.6– Method Lines of Code (MLOC)– Nested Block Depth (NBD)– McCabe Cyclomatic Complexity (McCabe)

SDL 2015 – p. 12

Dynamic Metrics

• Collected with Eclipse TPTP tracer and a Java profiler– UniqueFanout (array): Unique set of methods that

this method called. The length of the array is the fan-out.

– UniqueFanin (array), Fan-in, and Fan-out– NumberOfTimesCalledBy (array)– NumberOfTimesMethodsCalled (array)– TotalSegmentPresence– Depth– Method BaseTime, CumulativeCpuTime,

CpuUsageAverage– …

SDL 2015 – p. 13

SDL 2015 – p. 14

Utilityhood Metrics

• Combine basic static/dynamic metrics collected from software code and execution traces

• Combinations based on preliminary experiments and on related work– Tested on 18 simple Java open source systems– Kept the most promising combinations for further

experiments• U(r): utilityhood for method r used in a trace• RelativeMethodSize(r): size of a method r in relation

to other methods in the system• N: number of unique methods in the execution trace

SDL 2015 – p. 15

SDL 2015 – p. 16

SDL 2015 – p. 17

Use Case Maps

• Part of the User Requirements Notation (Rec. Z.151)• Combine structure with scenario-oriented behaviour• High level of abstraction, above that of messages

SDL 2015 – p. 18

Trace element

UCM element Symbol

Package Shown as a rectangle with thick border.

Class Shown as a rectangle in another rectangle (Package).

Beginning / End of trace

Start point (circle) / End point (bar) (also used as connectors for linking sub-scenarios to the parent stub)

Method Method name (shown as a X on a path)

Block of x or more instructions in the same class/object

Stub (diamond) with the name of the first instruction that is not a constructor. This stub contains a plug-in (another sub-map) showing the sub-sequence with one responsibility per instruction.The number of instructions that are placed in a block is set in the TraceToUCM tool.

Repeated method

Method name (repetition count between curly brackets) method {2}

Order of trace items

Direction of scenario flow (shown as an arrow head). We use the order of trace items in execution traces to dictate the direction of the scenario flow.

… …IN1 OUT1

… …

… …

… …

TraceUCM Mapping [Hamou-Lhadj et al., 2005]

SDL 2015 – p. 19

Example: UCEd Tool

• Use Case Editor, version 1.6.2 (Somé, 2007)• 700 Java classes, 3284 methods, 33,004 lines of code• Feature explored: “Open a project”• Trace size: 19,543 method invocations

SDL 2015 – p. 20

SDL 2015 – p. 21

Example UCM

• Algorithm 4• 40 top methods preserved• Needs another iteration, with

better parameters...

SDL 2015 – p. 22

• Same initial trace• Algorithm 4• 20 top methods

preserved• Use of UCM stubs

for hierarchical decomposition

SDL 2015 – p. 23

Validation with Experts

Software Classes Methods LOC

UCEd 700 3284 33,004

Umple 430 4457 79,459

jUCMNav 851 7406 93,919

TraceToUCM 35 96 3441

SDL 2015 – p. 24

Results of Utility Selection Algorithms

“Are the methods selected by this algorithm correct?”

Algorithms 2 (Hamou-Lhadj) and 4 (Braun) ahead of the pack

SDL 2015 – p. 25

“Assuming the correct methods had been selected, would you use UCM diagrams to document the behavior of a feature in your design documents?”

Results of Visualization

SDL 2015 – p. 26

Documentation Experiment

• Traced our own TraceToUCM tool during the analysis of the HotDraw software application

• 12 grad students and 4 professional developers• Showed UCMs of traces; some were very (too) short• “Pick the top three UCM diagrams that you fell gives

enough information to describe what the program does”• Newer algorithms scored best, but…

Subjects Alg 1 Alg 2 Alg 3 Alg 4 Alg 5 Alg 6 Alg 7 Alg 8

S1 2 3 1S2 3 2 1S3 3 2 1S4 1 3 1S5 3 2 1S6 3 2 1S7 1 2 3S8 2 1 3S9 3 2 1S10 2 3 1S11 1 1 3 (4)S12 (4) 2 2 1

S13 1 1 3 3

S14 1 3 1 3 3 (6)S15 3 2 1

S16 (4) (4) 2 2 1

Ranking Alg 1 Alg 2 Alg 3 Alg 4 Alg 5 Alg 6 Alg 7 Alg 8Small & simple

First 3 1 1 3 1 0 2 0 9Second 0 0 3 1 1 3 1 5 0Third 0 3 4 0 3 2 1 4 0Count 3 4 8 4 5 5 4 9 9

SDL 2015 – p. 27

Related Work and Threats to Validity

• Tons of them! See paper and Braun’s thesis

SDL 2015 – p. 28

Conclusions

• TraceToUCM: an environment where different algorithms targeting the filtering of utilities (exploiting dynamic, static, and CPU information) can be implemented for comparison and evaluation

• Visualization of traces with UCM (and indirectly MSC)• Comparison of 8 utilityhood functions, 4 based on

existing work and 4 new ones– Usually better results with some new functions– But: still much room for improvement

SDL 2015 – p. 29

Future Work

• Re-run study with other traces, software, parameter configurations and OO languages, with more subjects

• Improve UCM generation with better auto-layout and multiple layers of stubs

• Investigate whether utilities change from one feature to the next

• Extend the study to handle multi-threaded applications