How does context affect the distribution of software maintainability metrics?
Feng Zhang, Audris Mockus, Ying Zou, Foutse Khomh,
and Ahmed E. Hassan
2
Software Metrics
Numerous Software
3
Various Usage of Software Metrics
4
Contexts !
Motivation
5
In Software Engineering Area?
6
What are the Contexts of Software?
Age (AG)
Number of Changes (NC)
Life Span (LS)
Number of Downloads (ND)
Application Domain (AD)
Programming Language (PL)
7
39 Software Maintainability Metrics
Complexity (14 metrics) Abstraction (5 metrics)
Coupling (8 metrics)
Cohesion (4 metrics)
Encapsulation (4 metrics)
Documentation (4 metrics)
8
Data Collection
56,833
824
9
Data Cleaning
618
506
478
390
320
824
10
31
26
23
29
49
19
16
41
29
14
13
7
7
9
7
Build Tools
Code Generators
Communications
Framework
Games / Entertainmaint
Internet
Network
Software Development
System Administrator
Build & CodeGen
Comm & Internet
Comm & Network
Games & Internet
Internet & SW Dev
SW Dev & Sys Admin
57
85
18
146
14
C
C++
C#
Java
Pascal
Data Description
320 Software Systems
11
Research Questions
12
Separately
RQ1. Analysis Methods
13
RQ1. Analysis Methods (cont’)
For example
C Java Pascal C++ C#
Metric 1
Metric 1
Metric 1
Metric 1
Metric 1
Metric n
Metric n
Metric n
Metric n
Metric n
Kruskal Wallis test
Kruskal Wallis test
14
Complexity (8/14 metrics) Abstraction (1/5 metrics)
Coupling (5/8 metrics)
Cohesion (2/4 metrics)
Encapsulation (1/4 metrics)
Documentation (3/4 metrics)
YES!! the Contexts Matter!
51 % of metrics are impacted by all Six
Contexts
15
and Among the Six Contexts …
at least 72 % of metrics are impacted by a Single Context
16
Does it mean ALL six contexts
should be considered all the time?
17
Research Question 2
18
RQ2. Analysis Methods
19
RQ2. Analysis Methods (cont’)
C Java Pascal C++ C#
Metric i
Metric i
Mann-Whitney U test
Metric i
Metric i
Mann-Whitney U test
Metric i
Metric i
Mann-Whitney U test
Metric i
Metric i
Mann-Whitney U test
Metric i
Metric i
Mann-Whitney U test
Metric i
Mann-Whitney U test Metric i
20
RQ2. Analysis Methods (cont’)
0.147 0.330 0.474 Cliff’s delta 14.7% 33.0% 47.4% % of non-overlap
Small Medium Large Cohen’s standard 0.20 0.50 0.80 Cohen’s d
21
RQ2. Findings for
each Category of Metrics
22
Metric AD PL AG LS NC ND
TLOC - - - - √ -
TNF - - - - √ -
TNC √ √ - - √ -
TNM √ √ - - √ -
TNS - - - - - -
CLOC - - - - - -
NOM - - - - - -
NIM - - - - - -
NIV - - - - - -
WMC - - - - - -
NMP - - - - - -
CC - - - - - -
NPATH - - - - - -
MNL - - - - - -
Contexts Impacting ‘Complexity’
AD: Application Domain PL : Programming Language NC: Number of Changes
23
Metric AD PL AG LS NC ND
CF - - - - √ -
CBO √ √ - - - -
ICP - - - - - -
MPC - - - - - -
RFC √ √ - - - -
NMI - √ - - - -
FANIN - - - - - -
FANOUT - - - - - -
Contexts Impacting ‘Coupling’
AD: Application Domain PL : Programming Language NC: Number of Changes
24
Metric AD PL AG LS NC ND
LCOM √ - - - - -
TCC - - - - - -
LCC - - - - - -
ICH - - - - - -
Contexts Impacting ‘Cohesion’
AD: Application Domain
25
Metric AD PL AG LS NC ND
NACI - √ - - - -
MIF - √ - - - -
IFANIN √ √ - - - -
NOC - - - - - -
DIT √ - - - - -
Contexts Impacting ‘Abstraction’
AD: Application Domain PL : Programming Language
26
Metric AD PL AG LS NC ND
RPA √ - - - - -
RPM - - - - - -
RSA - - - - - -
RSM √ - - - - -
Contexts Impacting ‘Encapsulation’
AD: Application Domain
27
Metric AD PL AG LS NC ND
CLC - - - - - -
RCCC √ √ - - - -
CLM - - - - - -
RCCM - - - - - -
Contexts Impacting ‘Documentation’
AD: Application Domain PL : Programming Language
28
Summary of RQ2 Findings
Metric Category AD PL AG LS NC ND
Complexity √ √ - - √ - Coupling √ √ - - √ -
Cohesion √ - - - - - Abstraction √ √ - - - -
Encapsulation √ - - - - - Documentation √ √ - - - -
AD: Application Domain PL : Programming Language NC: Number of Changes
29
Metric Category Context Groups
Complexity
AD (2) (Framework); and others
PL (3) (C); (Pascal); and others
NC (3) (Low NC;) (moderate NC); and (high NC)
Coupling
AD (3) (Communication, Network); (Build Tools, Code Generators;) and others
PL (3) (Pascal;) (Java;) and others
NC (3) (Low NC); (moderate NC); and (high NC)
Cohesion AD (2) (Communication, Network); and others
Abstraction
AD (4) (Communication, Network); (Games); (Build Tools, Code Generators);
and others
PL (3) (Java;) (C++); and others
Encapsulation AD (3) (Build Tools); (Communication, Network); and others
Documentation AD (2) (Build Tools, Code Generators); and others
PL (2) (Java); and others
Guidelines for Benchmarking Maintainability Metrics
30