The Goal Question Metric Approach Victor R. Basili,

Gianluigi Caldiera

and H. Dieter Rombach

Presenter:

Jose Calvo-Villagran

jcalvovi@uwaterloo.ca

Overview

Motivations

GQM

Definition

Process

Examples

Extensions

Conclusions

Motivations [1/2]

Measurement

Planning

Quality Control

Refinements

Progress

The case of Amisoft [2]

CPI/SPI

Volatility

Client incidents

Motivations [2/2]

Effective measurement must be [1]:

Focused on specific goals

Applied to all cycles of production

Embedded into an specific context

Top down (?)

Quantitative model

GQM: Definition [1/2]

Measurement system

Set of rules for interpretation of measurement data

Three levels

Conceptual Level [Goal]

Objects

Operation Level [Question]

Assessment

Quantitative Level [Metric]

Objective/Subjective

4A GQM model is a hierarchical structure (Figure 1) starting with a goal (specifying

purpose of measurement, object to be measured, issue to be measured, and viewpoint

from which the measure is taken). The goal is refined into several questions, such as the

one in the example, that usually break down the issue into its major components. Each

question is then refined into metrics, some of them objective such as the one in the

example, some of them subjective. The same metric can be used in order to answer

different questions under the same goal. Several GQM models can also have questions and

metrics in common, making sure that, when the measure is actually taken, the different

viewpoints are taken into account correctly (i.e., the metric might have different values

when taken from different viewpoints).

In order to give an example of application of the Goal/Question/Metric approach, let's

suppose we want to improve the timeliness of change request processing during the

maintenance phase of the life cycle of a system. The resulting goal will specify a purpose

(improve), a process (change request processing), a viewpoint (project manager), and a

quality issue (timeliness). This goal can be refined to a series of questions, about, for

instance, turn-around time and resources used. These questions can be answered by

metrics comparing specific turn-around times with the average ones. The complete

Goal/Question/Metric Model is shown in Figure 2.

Figure 2

Goal Purpose Improve

Issue the timeliness of

Object (process) change request processing

Viewpoint from the project manager's viewpoint

Question What is the current change request processing

speed?

Metrics Average cycle time

Standard deviation

% cases outside of the upper limit

Question Is the performance of the process improving?

Metrics Current average cycle time

Baseline average cycle time100*

Subjective rating of manager's satisfaction

3. THE GOAL QUESTION METRIC PROCESS

GQM: Definition [2/2]

GQM: Process

Identify goals

Policies/Strategies

Description of process/product

Model of the organization

Derive questions

How to characterize the object?

How to characterize the attributes?

How to evaluate the characteristics?

Specify metrics

Amount and quality of data

Maturity of objects

Learning process

GQM: Remarks

Simple, intuitive approach for specifying metrics

Metrics can be reused by several questions

Four phases: [3]

Planning

Definition

Collection

Interpretation

Goals include different viewpoints (stakeholders)

Subjective metrics

Examples [1/2]

[4]

Examples [2/2]

The purpose is to characterize the effect of pair programming on programmer effort and program quality from the point of view of software managers in the context of a small web-development company [5].

*Side notes

Applicable on mature organizations:

Employee involvement (resistance)

Clear goals

Gradual/Slow changes

Does GQM evolve over time? How fast?

Does GQM apply to more agile environments?

Why GQMs main focus is SE?

Simplicity: Double edged sword

Relation between goals

How do GQMs goals align with organizations goal?

Extensions

GQM+Strategies [6]

Conclusions

Top-down approach for definition and collection of metrics

3 levels & 4 phases

Applied to all phases of software development

Requires organizational maturity

Bibliography

[1] Victor Basili, Gianluigi Caldiera, Dieter Rombach. 1994. The Goal Question Metric Approach.

[2] Romain Robbes, Rene Vidal, Maria Cecilia Bastarrica. 2013. Are software analytics efforts worthwhile for small companies? The case of Amisoft. IEEE

[3] Irene Eusgeld, Feliz Freiling, Ralf Reussner. 2008. Dependability Metrics: Advanced Lectures.

[4] D.Ernst, F. Houdek. GQM Method Application. Taken on 27-Jan-2013. http://www-ivs.cs.uni-magdeburg.de/sw-eng/us/java/GQM/link3.shtml

[5] http://en.wikipedia.org/wiki/GQM

[6] Victor Basili, et. al. 2010. Linking software development and business strategy through measurement. IEEE