michel chaudron universiteit leiden - tu/ejohanl/educ/2ii45/2010/tue... · michel chaudron...

Assessing Architecture QualityAssessing Architecture QualityAssessing Architecture QualityAssessing Architecture Quality

Michel ChaudronMichel ChaudronUniversiteit Leiden

With slides from Lethbridge and Laganiere

Introducing ….� Michel Chaudron

� Ph.d. Leiden (Computer Science)

� 2 years at IT company (RWS & MinDef)

� 10 years at TU Eindhoven (SAN)

� 2 years in Leiden � 2 years in Leiden

program director M.Sc. ICT & Business

� Research Interests

� Software Architecture / Enterprise Architecture:

� Quality of Architecture, UML-metrics, Impact on Productivity/Agility, Use of Architecture in Offshoring

2MRV ChaudronSheet 2

Leiden


Agenda

� What is Quality

� Requirements & Quality

� Software Metrics

� Metrics for Architecture Quality

4

MRV Chaudron

Sheet 4

� Metrics for Architecture Quality

� Measure quality of architecture/design

� Feel free to raise questions during the lecture (raise your hand)

What is quality?

5

Now for software

Perspectives on quality

� Manufacturing-based (conformance to specs)

� Product-based (based on attributes of the software)

� User-based (“fitness for use”)

©2008 John Wiley & Sons Ltd.www.wileyeurope.com/college/van vliet 6

� Transcendent

(“I really like this program”)

� Value-based (balancing time and cost vs profits)

What is Quality of Software?

7

MRV Chaudron

Sheet 7

What is Quality of Software?

� Absence of defects?

� program does not crash

� computes correct output Formal

Methods

8

MRV Chaudron

Sheet 8

� We cannot establish the absence of defects, only their presence.

� We can count the number of defects we find after X hours of testing

ISO 9216 Quality Model

9

MRV Chaudron

Sheet 9

Quality ModelsExisting models

Boehm McCall

ISO 9126 Dromey

…

� Decomposition of characteristics� Bottom level: metrics

� Differences in � Relations between

characteristics

� Vocabulary

Boehm’s Quality Model

McCalls Quality Factors and Criteria

11

MRV Chaudron

Sheet 11

Boehm’s Quality tree

12

MRV Chaudron

Sheet 12

� Do it yourself: Goal-Question-Metric

SW-CMMMIL-Q -9858

Trillium Baldrige

IEEE Stds. 730,828829, 830,1012,1016

1028,1058,1063ISO 15504*(SPICE)

People CMM

IPD-

SDCCR

SCE

NATO AQAP1,4,9

BS

MIL-STD-498

DOD-STD-2167A

DOD-STD -7935A

SDCE

EIA/IEEE

MIL-STD-1679

EQA

CMMI*

PSP

SA-CMM

DOD-STD-2168

FAA-iCMM

SW-CMM

Frameworks for Software Quality

13

MRV Chaudron

Sheet 13Also see www.software.org/quagmire

TrilliumIPD-CMM*

DODIPPD

SECAMAF IPD Guide

BS5750

MIL-STD-499B*

ISO/IEC12207

IEEE1220

ISO 10011

SE-CMMSECM*(EIA/IS 731)

EIA/IS632

ISO 9000Series

EIA/IEEEJ-STD-016

IEEE/EIA12207

EIA 632*

IEEE 1074

TickITSSE-CMM

ISO 15288*

* Not yet released

Q9000

quag14d: 5 June 1998

iCMM

DO-178B

Courtesy Sarah Sheard, SPC

Software Quality Attributeshttp://satc.gsfc.nasa.gov/support/STC_APR96/qualtiy/stc_qual.html

14

MRV Chaudron

Sheet 14

Feasibility study

The waterfall model with QA

User Requirements

System Design

Analysis

QA

QA

QA

QA = Quality Assurance

Corrections

15

MRV Chaudron

Sheet 15

System Design

Coding

Operation

Testing

Program Design

QA

QA

QA

Corrections

Corrections

Corrections

CorrectionsProject Management- Planning and control- Risk Management- People Management

Problems with standards / QA� They may not be seen as relevant

and up-to-date by software engineers.

� They often involve too much bureaucratic form filling.

� If they are unsupported by software� If they are unsupported by softwaretools, tedious manual work is often involved to maintain the documentation associated with the standards.

Some more examples of *ilitiesAccessibility, Administrability, Understandability, Generality,

Operability, Simplicity, Mobility, Nomadicity, Portability,

Accuracy, Efficiency, Footprint, Responsiveness, Scalability,

Schedulability, Timeliness, CPU utilization, Latency,

Throughput, Concurrency, Flexibility, Changeability,

Evolvability, Extensibility, Modifiability, Tailorability,

17

MRV Chaudron

Sheet 17

Evolvability, Extensibility, Modifiability, Tailorability,

Upgradeability, Expandability, Consistency, Adaptability,

Composability, Interoperability, Openness, Integrability,

Accountability, Completeness, Conciseness, Correctness,

Testability, Traceability, Coherence, Analyzability, Modularity,

Reusability, Configurability, Distributeability, Availability,

Confidentiality, Integrity, Maintainability, Reliability, Safety,

Security, Affordability, Serviceablility, …

Design is a balancing act

Resource use

Performance

Reliability

Timeliness

SystemSystemSystemSystem

Security

… …

…


Resource useCPU, Mem, Netw

Essential system engineering problem:

� a plurality of contradictory goals

� a plurality of means (tactics, technology, process)

each of which provides a varying degree of help or hindrance in

achieving a given goal

Engineering is constrained in resources

QUALITY

QUANTITY

OF FUNCTIONALITY

‘SIZE’SCHEDULE /

TIME

MRV Chaudron

Sheet 19

‘SIZE’ TIME

These factors are closely related to each other

Faster time to market � reduce quality or quantity

Increase features � allow more time or reduce quality

Improve quality � allow more time or reduce quantity

Why Care about Quality during Design?

As a project progresses,more and more workdepends on earlier decisions.

Cost of Defect Repair

Cost of repair increases exponentially

MRV Chaudron

Sheet 20

Defects should be eliminated as soon as possible after their introduction

Economic Model for Cost of Quality

From: H. Krasner, Cost of Quality, 1998

Perfection is not economical

Requirements and Quality

A system is of good quality when it meets its requirements


Why Software Projects Fail

MRV Chaudron

Sheet 23

Related to

Requirements

Engineering

Related to

Requirements

Engineering

Essentially lack of defining goals!

Requirements on Requirements

SSSS SpecificSpecificSpecificSpecific

To-the-point, precise

MMMM MeasurableMeasurableMeasurableMeasurable

Quantifiable and verifiable

AAAA AcceptableAcceptableAcceptableAcceptable (to the stakeholders)

MRV Chaudron

Sheet 24

AAAA AcceptableAcceptableAcceptableAcceptable (to the stakeholders)

Accessible, understandable (for the user)

Achievable (technically/planning/economically)

RRRR RealisticRealisticRealisticRealistic

Deducible to the real business drivers

TTTT TestableTestableTestableTestable

Quality of Requirements

� Complete ?

� Consistent ?

� Fixed ?

� Prioritized ?� Prioritized ?

� Traceable ?

� Managed change

� versioning?


Software Metrics

� Metrics and Models in Software Quality Metrics and Models in Software Quality Metrics and Models in Software Quality Metrics and Models in Software Quality

EngineeringEngineeringEngineeringEngineering (2nd edition)

Stephen H. Kan

Addison Wesley, 2002

26

MRV Chaudron

Sheet 26

� Software Metrics Software Metrics Software Metrics Software Metrics A Rigorous & Practical ApproachA Rigorous & Practical ApproachA Rigorous & Practical ApproachA Rigorous & Practical Approach

Norman E. Fenton & Shari Lawrence Pfleeger, 2nd ed. International Thomson Computer Press, 1997

If you cannot measure it, then it is not scienceIf you cannot measure it, then it is not scienceIf you cannot measure it, then it is not scienceIf you cannot measure it, then it is not science

In physical science the first essential step in the direction of learning any subject is to find principles of numerical reckoning and practicable methods for measuring some quality connected with it. I often say that when you can measure what you are speaking about, and express it in numbers, you know something about it;

but when you cannot measure it, when you cannot express it in numbers, your knowledge cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind; it may be the beginning of knowledge, but you have scarcely in your thoughts advanced to the state of Science, whatever the matter may be.

— Sir William Thompson, Lord Kelvin (1824Sir William Thompson, Lord Kelvin (1824Sir William Thompson, Lord Kelvin (1824Sir William Thompson, Lord Kelvin (1824----1907) 1907) 1907) 1907) From 'Electrical Units of Measurement', a lecture delivered at the Institution of Civil Engineers, London (3 May 1883), Popular Lectures and Addresses (1889), Vol. 1, 73. Quoted in American Association for the Advancement of Science, Science (Jan-Jun 1892), 19191919, 127.

Why measure?

Gilb’s principle of fuzzy targets:

Projects without clear goals

will not achieve goals clearly

Tom DeMarco

You can neither predict nor control

what you cannot measure

• Measure:Measure:Measure:Measure: A quantitative indication of the extent, amount, dimension, capacity or size of some attribute of a product or process.• A single data point (e.g. number of defects from a single

review)

• Measurement:Measurement:Measurement:Measurement: The act of determining a measure

Measurement, Metrics, IndicatorsMeasurement, Metrics, IndicatorsMeasurement, Metrics, IndicatorsMeasurement, Metrics, Indicators

maat

meten

29

MRV Chaudron

Sheet 29

• Measurement:Measurement:Measurement:Measurement: The act of determining a measure

• Metric:Metric:Metric:Metric: A measure of the degree to which a system, component or process possesses a given attribute.• Metrics relate measures (e.g. Average number of defects

found in reviews)

• Relate data points to each other

• Indicator:Indicator:Indicator:Indicator: A metric or series of metrics that provide insight into a process, project or product.

meting

Why Measure?

� Understanding

� Controlling

� Comparing

Predicting

Improvement

� Predicting

� Metrics are

�objective

�(often) automatically collectable

Motivation for Metrics� Estimate the cost & schedule of projects

Bidding

� Evaluate the productivity impacts of tools and techniques

Establish productivity trends over time� Establish productivity trends over time

� Monitor/Improve software quality

� Forecast future staffing needs

� Anticipate and reduce future maintenance needs

Metrics Domains� processprocessprocessprocess

� duration or effort of tasks,

� no. of changes in requirements

� resourcesresourcesresourcesresources� no. of staff working on a task;

� staff overturn� staff overturn

� staff experience/skills

� productproductproductproduct� requirements document

� architecture document

� design document

� implementation (code, libraries)

• size (lines of code)

• complexity

• functionality

Project MetricsProject MetricsProject MetricsProject Metrics

• Effort/time per SE task

• Defects detected per review hour

• Scheduled vs. actual milestone dates

• Changes (number) and their characteristics

Distribution of effort on SE tasks

33

MRV Chaudron

Sheet 33

• Distribution of effort on SE tasks

Example Process MetricDistribution of effort over different activities in development

Product MetricsProduct MetricsProduct MetricsProduct Metrics

• focus on the quality of deliverables

• measures of analysis model

• complexity of the design

• internal algorithmic complexity

architectural complexity

35

MRV Chaudron

Sheet 35

• architectural complexity

• data flow complexity

• code measures (e.g., Halstead)

• measures of process effectiveness

• e.g., defect removal efficiency

Algorithmic Complexity

• McCabe’s Cyclomatic Complexity

is based on number of paths through a flow graph (explained later in testing lectures)

• Relationship between complexity and defects

36

MRV Chaudron

Sheet 36

• Relationship between complexity and defects and maintainability (tf. time to repair defects)

Size and Complexity Metrics II

� McCabe’s cyclomatic number

�Measures Complexity of a module

�Heuristic: should be <10G is control flowgraph

e edges and n nodes

37

MRV Chaudron

Sheet 37

e edges and n nodes

V(G) = e – n + 2

(number of linearly independent paths in G)

Here: V(G) = 12 – 10 + 2 = 4

More simply, d is number of decision nodes

V(G) = d + 1

Quality Metrics

What: Testability, extensibility, maintainability, error-proneness, …

How:� Coupling, Cohesion

� Complexity

38

MRV Chaudron

Sheet 38

� Complexity

� Inheritance metrics

Design Principles

� Simplicity

� Separation of Concerns

� Information Hiding

� Modularity� Modularity

� Keep things that belong together in a single place

How to assess?


Separation of Concerns

� Separate What from How

� The interface of a component exposes what

function it can perform, not how.function it can perform, not how.

� The ‘how’ is the information-hiding ‘secret’

40MRV Chaudron

Sheet 40

Dependency: Coupling

Coupling is the degree

of interdependence between modules

Heuristic: minimize coupling between modules

high coupling low coupling

What is a dependency?

� Component A requires B for it to work�Functional coupling

� A change in module B requires change in module A

Run-time

42

MRV Chaudron

Sheet 42

A change in module B requires change in module A

� Implementation coupling

�Typically requires: re-testing A & B

Development-time

� There is coupling between two classes AAAA and BBBB if:

�AAAA has an attribute that refers to (is of type) BBBB.

�AAAA calls on services of an object BBBB. �AAAA calls on services of an object BBBB.

�AAAA has a method which references BBBB

(via return type or parameter).

�AAAA is a subclass of (or implements) class BBBB.

43Chapter 9: Architecting and designing software

This is not an exhaustive definition

Dependency: Cohesion

Cohesion is concerned with the interactions

within a module

Heuristic: Keep things together that

belong together.

High cohesion within a

module is good

low cohesionhigh cohesion

Coupling and CohesionCoupling and CohesionCoupling and CohesionCoupling and Cohesion

� CouplingCouplingCouplingCoupling is the degree of interaction between modules.

� CohesionCohesionCohesionCohesion is a measure of the coherence of a module amongst the pieces of that module.module amongst the pieces of that module.

� You want high cohesion and low coupling


� Modulariteit = Isolatie

� wat binnen zit, moet binnen blijven


Spaghetti – absence of structure


An edge running from the node aaaa to the node bbbbrepresents a call of function bbbb from function aaaa

Belang van modulariteit

� Software qualities

�Correctheid, robuustheid, betrouwbaarheid e.d

⇒ goede requirements specification

Onderhoudbaarheid, uitbreidbaarheid, �Onderhoudbaarheid, uitbreidbaarheid, herbruikbaarheid en interoperability

⇒ goede software architectuur

� Software architectuur

�Modulaire structuur (modules en relaties)

Benefits of Low Coupling/Dependencies1. Modules are easier to replace

2. fewer interconnections between modules reduce

time needed for understandingunderstandingunderstandingunderstanding the modules and

interactions

3. fewer interconnections between modules reduce the 3. fewer interconnections between modules reduce the

chance that changeschangeschangeschanges in one module cause problemsproblemsproblemsproblems

in other modules, which enhances reusability

4. fewer interconnections between modules reduce the

chance that a fault in one module will cause a failurefailurefailurefailure

in other modules, which enhances robustness

49Chapter 9: Architecting and designing software

Page-Jones, M. 1980. The Practical Guide to Structured Systems Design. New York, Yourdon Press, 1980.

. x x x x x

x . x x x x x x x x x

Drive x x . x x x

System x x x . x x x x x x x x

x x . x

x x x x . x x x

x x x . x x

x x x . x x x x

x x x . x x x x x

Main x x x . x x x

Board x x x x x x x x . x x x x x

x x x x x . x x

x x x x x x . x x x

x x x . x

Design Structure Matrix Map of a Laptop Computer

Dependencies between subsystems

Relates to

architecture

conformance

50Chapter 9: Architecting and designing software©

Lethbridge/L

x x x . x

x x x . x x x

x x x x . x x x x

LCD x x x . x x

Screen x x x x . x x x

x x x x x x x . x x x

x x x . x

x x x x . x x x x

x x x . x x x x

x x x x x . x x x

Packaging x x x x . x x

x x x x x . x x

x x x x . x x

x x x x x .

x x x x x .

Graphics controller on Main Board or not?

If yes, screen specifications change;

If no, CPU must process more; adopt different interrupt protocols

Design Structure Matrix Map of a Modular System

. x x x x

x . x x

Design x . x x Design Rules Task Group

Rules x x . x

x x x .

x . x x x

x x . x x x

Drive x x x x . x

System x x x x x . x x Hidden Modules

x x x . x many Task groupsx x x x .

x . x x

x x x x x . x x

x x . x x x x

Main x x x x x . x x

Board x x x x x x x . x x

x x x x x . x


Lethbridge/L

x x x x x . x

x x x x x x . xx x x x x .

x x . x x x

x x x . x x x

LCD x x x . x

Screen x x x x x . x x

x x x x x . xx x x x x x .

x x . x x x x

x x x . x x x x

x x x . x x x

Pack- x x x x x x . x x

aging x x x . x x

x x x x x . x x

x x x x x .x x x x x x .

x x x x x x . x x x xSystem x x x x x x x x . x x System

Testing x x x x x x x x x . x x x Integration

& Integ- x x x x x x x x x x x x and Testing

ration x x x x x x x x . x Task Groupx x x x x x x x x x x .

DSM of Mozilla before and after redesign

number

of files


Lethbridge/L

Formerly Mozilla was the commercial Netscape Navigator, then released into open source.

From: Exploring the Structure of Complex Software Designs: An Empirical Study of Open Source and Proprietary Code, Alan MacCormack, John Rusnak, Carliss Baldwin, Harvard Business School, draft October 1st 2005

1.3 dependencies per KSLOC2.4 dependencies per KSLOC

Architecture / Design Metrics

� Weighted Methods per Class (WMC)

� Depth of Inheritance Tree (DIT)

� Number of Children (NOC)

� Coupling between Objects (CBO)� Coupling between Objects (CBO)

� Response for a Class (RFC)

� Lack of Cohesion in Methods (LCOM)

Metrics Suite by Chidamber & Kemerer (1993)

Object-oriented metrics

Object-oriented

metric

Description

Depth of inheritance

tree

This represents the number of discrete levels in the inheritance tree where sub-

classes inherit attributes and operations (methods) from super-classes. The

deeper the inheritance tree, the more complex the design. Many di fferent object

classes may have to be understood to understand the object classes at the leaves

of the tree.

Method fan-in/fan-

out

This is directly related to fan-in and fan-out as described above and means

essentially the same thing. However, it may be appropriate to make a

distinction between calls from other methods within the object and calls from

external methods.

Weighted methods

per class

This is the number of methods that are included in a class weighted by the

complexity of each method. Therefore, a simple method may have a complexity

of 1 and a large and complex method a much higher value. The larger the value

for this metric, the more complex the object class. Complex objects are more

likely to be more difficult to understand. They may not be logically cohesive so

cannot be reused effectively as super-classes in an inheritance tree.

Number of

overriding

operations

This is the number of operations in a super-class that are over-ridden in a sub-

class. A high value for this metric indicates that the super-class used may not be

an appropriate parent for the sub-class.

Example: Before Refactoring

CouplingDynamic Coupling

Method Calls

Saat 7 10 25

55

MRV Chaudron

Sheet 55

Saat 7 10 25

Stat.-Filter 0 3 0

Stat.-Calculator 0 3 0

Analyser 0 4 0

DB-Checker 0 2 0

DB-Filler 0 5 0

DB-Creator 0 2 0

Parser 0 3 0

Example: After Refactoring

CouplingDynamic Coupling

Method Calls

Saat 2 2 2

56

MRV Chaudron

Sheet 56

Stat.-Filter 1 2 1

Stat.-Calculator 0 2 0

Analyser 3 3 12

DB-Checker 0 1 0

DB-Filler 1 4 10

DB-Creator 0 1 0

Parser 0 1 0

Façade pattern

Façade

Clientclasses

57

MRV Chaudron

Sheet 57

Note the effect on the fan-in/coupling of the component

Distribution of Coupling ?

nu

mb

er o

f cl

ass

es

coupling

nu

mb

er o

f cl

ass

es

System of 100+ classes

Distribution of Coupling

150

200

250

num

ber

of

cla

sses

59

MRV Chaudron

Sheet 59

0

50

100

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49

coupling

num

ber

of

cla

sses

System of about 140 classes

Selective QA: Use thresholds� Heuristic (e.g. DIT < 7)

� Mean + 2 * StDev

� Distribution dependent

140

160

Number of Methods per Class

60

MRV Chaudron

Sheet 60

0

20

40

60

80

100

120

1 8 15 22 29 36 43 50 57 64 71

Façade pattern

Façade

Clientclasses

61

MRV Chaudron

Sheet 61

Note the effect on the fan-in/coupling of the component

Complexity: Fan-in & Fan-out

Fan-in = no. of ingoing dependencies

Fan-out = no. of outgoing dependencies

Heuristic: a high fan-in/fan-out indicates a high complexity

MetricView: Graphical Tool forDeveloper Feedback

http://www.win.tue.nl/empanada/metricview/

MRV Chaudron

Sheet 63

UML model

Visualization of model + metricsQuality Metrics/Rules• Completeness• Consistency• Conventions

Analysis ToolAnalysis ToolAnalysis ToolAnalysis Tool

MetricView video http://www.win.tue.nl/empanada/metricview/

http://www.youtube.com/watch?v=G3HJ_QR9EG4

Relation between level of detail in

Sequence diagrams and defects in implementation

Strengths and Weaknesses of Metrics

StrenghtsStrenghtsStrenghtsStrenghts� Objective

� Incremental

� Fast results (if automated)

� Require little effort (if automated)Require little effort (if automated)

WeaknessesWeaknessesWeaknessesWeaknesses� Interpretation is not straigtforward

� Rather than black-white: use as indicator for weak spots

Summary

� Get stakeholders involvement in the

definition & maintenance of requirements

� Get feedback early and often

� Establish your schedule, cost, quality

prioritiespriorities

� Know & apply your design principles

� Quality Metrics can be applied early and

cost-effectively

“Not everything that is important can be measured, and not everything that can be measured is important.“

Albert Einstein

Software Architecture Design Process

FunctionalRequirementsFunctionalRequirements

Extra-FunctionalRequirementsExtra-FunctionalRequirements

DomainRequirementsDomainRequirements

UserRequirementsUserRequirements

Select Select

MRV Chaudron

Sheet 69

RequirementsRequirements RequirementsRequirements

Group Functionalityin subsystemsGroup Functionalityin subsystems

Design approach forrealizing extra-functional quality properties

Design approach forrealizing extra-functional quality properties

Select •Architectural Style•Reference Architecture•Architecture Tactics

Select •Architectural Style•Reference Architecture•Architecture Tactics

SynthesizeSynthesize

Analyze Analyze

refine

Architecture Design Principles

� Dependencies direct in the direction of stability

A

70MRV Chaudron

Sheet 70

B

B is less likely to change than A

«layer»

Business Layer

«layer»

Common Elements«layer»

Presentation and Dialogue Layer

«subsystem»P

«subsystem»D

«subsystem»M

«subsystem»F

«subsystem»C

«subsystem»M

«subsystem»Client / Browser

«subsystem»E

«subsystem»

«subsystem»S

«subsystem»Client Authentication

71

MRV Chaudron

Sheet 71

«layer»

Persistence Layer

«subsystem»P

«subsystem»M

«subsystem»F

«subsystem»D

«subsystem»C

«subsystem»M

«subsystem»P«subsystem»

M

«subsystem»F

«subsystem»D«subsystem»

C

«subsystem»M

«subsystem»Apache

«subsystem»RC

«subsystem»JR

«subsystem»PL

«subsystem»Data Security

Generic Design Principles

� Keep things that belong together at a single place

e.g. in OO: data and e.g. in OO: data and

the operations on that data

� Don’t replicate functionality


Architecture Design Principles

� No Circular Dependencies


Callers depend on callee, not vice versa

� Separation of Concerns ( = Decomposition)

� Zaken die niet bij elkaar horen moeten in verschillende eenheden (componenten /

procedures / .. ) worden behandeld

Generic Design Principles


procedures / .. ) worden behandeld

Example Separation of Concern PrincipleTelecom Domain:

Separate the encoding/decoding of a message from the handling of a message, so

handle

encode/decode

75

MRV Chaudron

Sheet 75

message, so

� decode1 ; decode2 ; decode3 ;

action1 ; action2

And not

� decode1 ; action1 ; decode2 ;

action2 ; decode3

decode

handle &encode/decode

Aspect Orientation

Design & maintain concerns in isolation

Automatically construct implementation

by ‘weaving’ concerns

76

MRV Chaudron

Sheet 76

What is Modularity?

� We can “see it” via a � Design Structure Matrix (DSM) Map

michel chaudron universiteit leiden - tu/ejohanl/educ/2ii45/2010/tue... · michel chaudron...

Documents