Software Quality CS5391 Spring 2003 1

Software Quality :

The Elusive Target

Zhenyu Wu

Kitchenham B. & Pfleeger S. L.

January, 1996 IEEE Software

Software Quality CS5391 Spring 2003 2

Introduction

What do you really mean by software quality (SQ)?

Is your definition adequate?

How to evaluate your product’s SQ ?

How to measure SQ?

How to achieve SQ?

Questions:

Software Quality CS5391 Spring 2003 3

Meaning of SQ

A good definition should let us measure quality in a meaningful way

Measurements let us know if techniques improve quality how process quality affects product quality how quality methods affect product quality during

use Is the investment in SQ methods profitable?

Software Quality CS5391 Spring 2003 4

Meaning of SQ…

Depends on how you approach software quality

Different groups involved have different views Product-based

Process-based

People believe quality is important and can be improved

Software Quality CS5391 Spring 2003 5

Views of SQ

SQ can be perceived in various domains (philosophy, economics, marketing and operations management)

SQ is complex and multifaceted concept and can be described from five different perspectives Transcendental view

User view

Manufacturing view

Product view

Value-based view

Software Quality CS5391 Spring 2003 6

Transcendental View

See quality as something that can be recognized but not defined

Quality as something toward which we strive as an ideal, but may never implement completely

Abstract sense

Software Quality CS5391 Spring 2003 7

User View

More concrete, grounded in the product characteristics that meet user’s needsEvaluate product in a task context can be personalized viewAsses product behavior with respect to operational profilesRelated to product usability and reliability

Software Quality CS5391 Spring 2003 8

Manufacturing ViewFocus on product quality during production and after deliveryConcerns whether or not the product was constructed “right the first time”Can lead to quality assessment independent of the product Adapted by ISO 9001 and CMM, advocates conformance to process rather than to specification

Software Quality CS5391 Spring 2003 9

Product View

Consider the product’s inherent characteristics, looks inside

Assumption : measuring and controlling internal product properties will result in improved external product behavior

More research needed to conform this assumption

Software Quality CS5391 Spring 2003 10

Value-Based View

Different views can be held by different groups involved in software development user’s requirement for a useful product conflict

with manufacturer’s goal of minimizing rework consider the trade-offs between cost and quality manage conflict design to cost compare with potential benefits

This view tries to bring in compromise between different groups and views

Software Quality CS5391 Spring 2003 11

Measuring QualityWe need to measure quality so we can: Establish baselines Predict likely quality Monitor improvement

Product attributes contribute to user satisfaction are a mixture of: product’s functions product’s nonfunctional qualities constraints

ISO definition of quality: the totality of characteristics of an entity that bear on its ability to satifsfy stated and implied needs.

Software Quality CS5391 Spring 2003 12

Measuring user's view

Reliability: how long the product functions properly between failures Usability: including ease of installation, learning and useTom Gilb’s technique: characteristics can be measured directly Example: Learning time = average elapsed time (in hours) for a

typical user to achieve stated level of competence

Software Quality CS5391 Spring 2003 13

Measuring user's view…

This technique can be generialized to any quality feature

The quality concept is broken down into component parts until each can be stated in terms of directly measurable attributes

Software Quality CS5391 Spring 2003 14

Measuring the Manufacturer's View

Defects countCount the number of known defects

recorded during development and useCan compare modules, products or

projects (with same way and same time)Relationship between defects count and

operational failures is unclearCan indicate test efficiency and identify

process improvement areas

Software Quality CS5391 Spring 2003 15

Measuring the Manufacturer's View…

Rework costs Any additional effort required to find and fix

problems after documents and code are formally signed-off as part of configuration management

Count debugging effort during integration and testing

Not count end-phase verification and validation Pre-release rework: measure manufacturing

efficient and process improvement Post-release rework: measure of deliverd quality

Software Quality CS5391 Spring 2003 16

Modeling Quality

Several models have been built to understand and measure quality

Old ModelsMcCall’s model ISO 9126

New modelDromey’s model

Software Quality CS5391 Spring 2003 17

McCall's Model

Defines product qualities as hierarchy of factors, criteria and metrics:Quality factor represents behavior

characteristic of the systemQuality criterion is an attribute of a quality

factor that is related to software production and design

Quality metric is a measure that captures some aspect of a quality criterion.

Software Quality CS5391 Spring 2003 18

McCall's quality Model…

The metrics have to be answered with ‘yes’ or ‘no’ answer

The final quality is assessed by dividing the ‘yes’ answers by the total number of questions chosen to describe the quality

19CS5391 Spring 2003Software Quality

McCall’s Model…

Software Quality CS5391 Spring 2003 20

ISO 9126

Six characteristics, completely hierarchicalQuality charactertistics are divided into sub-characteristics which can be measured using metrics Recommends measuring the char-acteristics directly, but does not indicate clearly how to do it

Software Quality CS5391 Spring 2003 21

Quality Characteristic

Definition

Functionality A set of attributes that bear on the existence of a set of functions and their specified propertise. The functions are those that satisfy stated or implied needs.

Relibity A set of attributes that bear on the capability of software to maintain its performance level under stated conditions for a stated period of time.

Usablity A set of attributes that bear on the effort needed for use and on the individual assessment of such use by a stated or implied set of users.

Software Quality CS5391 Spring 2003 22

Efficiency

A set of attributes that bear on the relationship between the software’s performance and the amount of resources used under stated conditions.

Maintainability A set of attributes that bear on the effect needed to make specified modifications (which may include correction, improvements, or adaptations of software to environmental changes and changes in the requirements and functonal specifications).

Portability A set of attributes that bear on the ability of software to be transferred from one environment to another (this includes the organizational, hardware or software environment).

Software Quality CS5391 Spring 2003 23

ISO 9126…

Reliebility

Functionality

Usability

SuitabilityAccurecy

InteroperabilitySecuity

MaterityFault TeleranceRecoverability

UnderstandabilityLearnabilityOperability

Characteristics Sub-characteristics

Software Quality CS5391 Spring 2003 24

ISO 9126…

Maintainability

Effiency

Portability

Time behaviorResource behavior

AnalyzabilityChangeability

StabilityTestability

AdaptabilityInstallabilityConformanceReplaceability

Characteristics Sub-characteristics

Software Quality CS5391 Spring 2003 25

Main Differences between Mccall’s and ISO 9126 model

Mccall’swith different quality frame

quality factor

not hierarchical

ISO 9126

work and terminology

quality characteristic

and subcharacteristics

completely hierarchical

Software Quality CS5391 Spring 2003 26

Model Problems

Lack rationale for determining which factor should be included in the quality definition

Lack rationale for deciding which criteria relate to a particular factor

The measure of metrics can be too subjective

Does not describe how lower level metrics are composed into an overall assessment of higher level quality characteristics

Software Quality CS5391 Spring 2003 27

Dromey's models(new model)

Higher level quality attributes cannot be measured or built directly

Higher level quality can be achieved by building components which exhibit a consistent, harmonious and complete set of product properties that result in manifestations of quality attributes

Can allow us to verify models

Software Quality CS5391 Spring 2003 28

Business Value of Quality

Businesses invest valuable money for obtaining good software, we have to pay them back in good We should be careful in assessing and providing software with good qualityHow much "less than perfect" software are the businesses willing to accept should be determined

Software Quality CS5391 Spring 2003 29

Conclusion

Quality is a complex concept; It means different things to different people

You must define aspects of quality in which you are interested

Quality should be defined in a measurable way, so that it can be understood and verified

Quality must be related to business goals