software quality and management 5. process oriented approaches towards software quality. spice model...

Software quality and management

5. Process oriented approaches towards software quality.

SPICE model / ISO 15504 standard.

Dr. Balla Katalin

Dr. Balla Katalin Software quality and management - 5.

2

Contents

Continuous maturity models– What is SPICE? – History of SPICE – Elements of SPICE model– Dimensions of SPICE model

• Process dimension– Software life cycle processes: ISO 12207– V-model

• Maturity dimension

– SPICE audit


3

Maturity models Staged models

Look to the entire organisation In their wiev there is one process with different aspects, characteristics Deal with managerial and technical processes, the technology used, the

organisation itself …

Continuous modelsContinuous models• Define maturity levels for certain processes (not for the Define maturity levels for certain processes (not for the

entire organisation), according to different characteristics entire organisation), according to different characteristics • The user of the model can decide about the process for The user of the model can decide about the process for

which maturity will be checked which maturity will be checked „Combined”, integrated models

• Combine the two approaches, making use of the most useful elements


4

What is SPICE?

SSoftware PProcess I Improvement and CCapability dEEtermination

http://www.sei.cmu.edu/iso-15504/ http://www.sqi.gu.edu.au/spice/

An international collaborative effort to develop a standard It has been underway (unofficially) since 1990 and officially

since June of 1993 The (prospective) standard is intended to, among other

things, establish a common framework – for expressing the process capability ratings resulting from a 15504-

conformant assessment and– to provide a migration path for existing assessment models and

methods wishing to become 15504-conformant.


5

History of SPICE

Approximately 20 countries are actively contributing resources to this effort. The effort is being managed by a working group ISO/IEC JTC1/SC7/WG10 of the JTC1 subcommittee responsible for software engineering standards, ISO/IEC JTC1/SC7.

JTC1 ISO/IEC JTC1 is a joint ISO and IEC technical committee which deals with information technology.


6

History of SPICE Interest in developing an international

standard on software process assessment was sparked by an investigative study sponsored by the U.K. Ministry of Defense (MOD) into methods for assessing the development capability of software suppliers.


7

History of SPICE The study identified and reviewed two dozen existing methods already

in use and put forth these findings: – There is a general need to supplement reliance of software procurers on ISO

9001.

– There is wide support for a software assessment scheme which is in the public domain, widely recognised, and preferably backed by an international standard.

– Some organisations using or developing their own assessment schemes have registered interest in supporting a public domain, standardised scheme, in preference to their own schemes.

– An initiative to develop such a scheme would be directed toward continuous process and quality improvement matched to business needs. The initiative would be focused around an international standard on process management which would itself provide a framework for a capability assessment scheme supporting both self-improvement for software suppliers and capability determination as a means of evaluating contract risk.


8

The SPICE project January 1993: standard development has been approved June 1993: SPICE project was started SPICE goal and scope:

– The SPICE (Software Process Improvement Capability dEtermination) project is an ancillary effort staffed primarily by volunteers from around the world. The SPICE project has three goals:

• Assist the standardization effort in its preparatory stage to develop initial working drafts.

• Undertake user trials in order to gain early experience data that will form a basis for revision of the published technical reports prior to review as full international standards.

• Create market awareness and take-up of the evolving standard.


9

Actual state of SPICE

All parts of the 15504 document set have been approved for publication as Technical Reports; 15504-5 will lag behind the other 8 parts by approximately 6 months. Availability can be determined by contacting the authorized standards agent in one’s country (ANSI for the United States) or ISO. The work of WG10 will now focus on progressing the document set to full international standard status; this work will likely take approximately 24 months.

SPICE Trials Around the World– Phase 2 trials have now closed and phase 3 is being planned.

– U.S. SPICE Trials

– There have been no additional trials reported in the United States since the first one was reported in early 1998.

Expression of interest :


10

ISO 15504 elements ISO/IEC TR 15504-1:1998

Information technology -- Software process assessment -- Part 1: Concepts and introductory guide

ISO/IEC 15504-2:2003Information technology -- Process assessment -- Part 2: Performing an assessment

ISO/IEC 15504-2:2003/Cor 1:2004 ISO/IEC 15504-3:2004

Information technology -- Process assessment -- Part 3: Guidance on performing an assessment

ISO/IEC 15504-4:2004Information technology -- Process assessment -- Part 4: Guidance on use for process improvement and process capability determination

ISO/IEC TR 15504-5:1999Information technology -- Software Process Assessment -- Part 5: An assessment model and indicator guidance

ISO/IEC TR 15504-9:1998Information technology -- Software process assessment -- Part 9: Vocabulary


11

What is SPICE/ ISO 15504 good for?

It is a framework related to processes and process maturity, summarising the experience of small, medium and large companies worldwide

A framework for– Understanding strengths and weaknesses of processes– SPI and measurement of SPI– Helping the customers understand the maturity of their

providers Helps in harmonising process auditing models


12

Elements of SPICE model

Concentrates on individual processes– Input and output

– Goal / scope of the process

– Activities of the process

– Roles and responsibilities

– Checkpoints

– Process measures

– Performance requirements

– Competencies needed

Process


13

SPICE process management Are the activities connected to individual

processes performed (at least informally)? Is the process properly planned? Are the activities executed conform to the plan? Is process execution controlled? Is process execution monitored, are corrective

actions being taken? Is the process standardised? Is the process understood quantitatively also? Is the process continuously improved?


14

SPICE maturity levels

0

1

2

3

5. Optimising

4 . Predictable

3. Established

2. Managed

1. Performed

0. Not performed

4

5


15

Process maturity

Processes Process maturity levels

0. Not performed

1. Performed

2. Managed

3. Established

4. Predictable

5. Optimising

•Primary •Customer related•Engineering

•Supporting•Managerial•Organisational


16

Dimensions of SPICE model

aremappedagainst

ORG

Category

Process

CUS ENG MANSUP

Life Cycle Processes

Process Dimension Capability Dimension

Level Name Attributes5 Optimizing process

Process change attribute

4 Predictable ProcessProcess measurement attribute

3 Established ProcessProcess definition attribute

2 Managed ProcessPerformance management attribute

1 Performed Process

0 Incomplete ProcessProcess perfomance attribute

Work Product management attribute

Process resource attribute

Process control attribute

Continuous improvement attribute


17

Process dimension

Processes = Software life cycle processes !!!


18

Process dimension:Software life cycle processes ISO /IEC 12207: Information Technology - Software

life cycle processes– The international standard ISO 12207 is a globally accepted standard for

software lifecycle processes. Though not suited for the direct application in a concrete project, it offers a frame that national standards or corresponding process details to be integrated in order to achieve a standard that can be used in an actual case. Furthermore, the ISO 12207 standard includes definitions that can be applied as a basis for a common

terminology, even in national standards. „This is the Process for Building Software In the 21st

Century! „http://www.12207.com/


19

Process dimension:Software life cycle processes

ISO/IEC 12207:1995Information technology -- Software life cycle processes

ISO/IEC 12207:1995/Amd 1:2002

ISO/IEC TR 15271:1998Information technology -- Guide for ISO/IEC 12207 (Software Life Cycle Processes)

ISO/IEC TR 16326:1999Software engineering -- Guide for the application of ISO/IEC 12207 to project management


20

Process dimension:Software life cycle processes

According to ISO 12207, the following processes are possible to execute during software life cycle :


21

Process dimension:V-model

/Myers, 1979/ The Art of Software Testing – The testing cycle has been structured to model the development cycle.

/Boehm, 1979/ Guidelines for Verifying and Validating Software Requirements and Design Specifications – "V-chart" which shows the context of verification and validation

activities throughout the software lifecycle /VM 1997/ V-Model 97, Lifecycle Process Model

– Lifecycle Process Model -Developing Standard for IT Systems of the Federal Republic of Germany. General Directive No. 250. June 9

– V-model: Regulations setting up all activities, products, and their logical interdependencies during the development and maintenance/modification of systems, realizing the system tasks predominantly by using IT, within the scope o the federal administration.

/Sommerville, 1999/ Software Engineering – V&V Process: is a whole life-cycle process. V&V must be applied at each stage in the

software process.


22


Boehm, 1979/ Guidelines for Verifying and Validating Software Requirements and Design Specifications


23


V&V Process: is a whole life-cycle process. V&V must be applied at each stage in the software process. Has two principal objectives:- The discovery of defects in a system.- The assessment of whether or not the system is usable in an operational situation.

(Sommerville, 1999/ )


24

Process dimension:Conformity between ISO 12207 and V-model

Development Process

Activities according to ISO 12207

Activities according to the V-Model

Process Implementation

PM1.3 - Generation of Project-Specific V-Model PM1.5 - Generation of Preliminary Plan PM1.4 - Toolset Management PM4 - Detailed Planning

System Requirements AnalysisSD1.2 - Description of Application System

SD1.5 - User-Level System Structure QA4 - Product Assessment

System Architectural Design

SD2.1 - Technical System Design SD2.4 - Allocation of User Requirements SD2.5 - Interface Description SD2.3 - Investigation of Feasibility QA4 - Product Assessment

Software Requirements AnalysisSD3 - SW/HW Requirements Analysis

QA4 - Product Assessment PM6 - Phase Review

Software Architectural Design

SD4.1 - SW Architecture Design SD4.2 - Design of Internal and External SW Interfaces QA2.1 - Definition of Assessment Methods and Criteria QA4 - Product Assessment PM6 - Phase Review


25

Process dimension:Conformity between ISO 12207 and V-model

• Software Detailed Design SD5.1 - Description of SW Component/Module/DatabaseQA1.2 - Generation the Assessment PlanQA2.1 - Definition of Assessment Methods and Criteria QA4 - Product Assessment PM6 - Phase Review

• Software Coding and Testing SD6.1 - Coding of SW Modules SD6.2 - Realization of Database QA2.3 - Definition of Test Cases QA2.4 - Generation of Assessment Procedure SD6.3 - Self-Assesment of the SW Module/Database QA4 - Product Assessment

• Software Integration SD4.3 - Specification of SW Integration SD7 - SW Integration QA2.3 - Definition of Test Cases QA2.4 - Generation of Assessment Procedure QA4 - Product Assessment PM6 - Phase Review

• Software Qualification Testing QA4 - Product Assessment CM2 - Product and Configuration Management CM4.4 - Data Backup PM6 - Phase Review

• System Integration SD8 - System Integration QA2.3 - Definition of Test Cases QA2.4 - Generation of Assessment Procedure QA4 - Product Assessment

• System Qualification Testing QA4 - Product Assessment CM2 - Product and Configuration Management CM4.4 - Data Backup PM6 - Phase Review

• Software Installation SD9 - Transition to Utilization • Software Acceptance Support SD9 - Transition to Utilization

PM6 - Phase Review


26

Maturity dimension: process maturity

„Generic practices” are used to determine process maturity. If a process is on a certain level of maturity, the generic practices associated to that maturity level should be there.

If a process is on a certain maturity level, it must satisfy certain goals and must produce certain deliverables.


27

Maturity dimension: Characteristics of SPICE maturity levels

Level 1. (performed):– The process is performed, it has no specific

characteristics.


28


Level 2. (managed): 2.1. Performance management

Resource needs definition

Process efficiency planning

Implementing the defined activities

Managing execution of the activities

2.2. Managing work product attribute Defining the requirements relative to integrity and quality

Definition of the activities needed

Configuration management of work products

Quality management of work products


29


Level 3. (established) 3.1. Process definition

Standard process definition Standard process customisation Institutionalisation of standard process Feedback to standard process

3.2. Resources connected to the process Definition of human resource competencies Defining infrastructural requirements of the process Ensuring the human resources having the required competencies Ensuring proper infrastructure


30


Level 4. (predictable) 4.1.: Process measurement

Defining process goals and associated metrics Ensuring appropriate resources and infrastructure Gathering defined measurement data Monitoring if process goals are reached

4.2. Process control Defining analysis and control techniques Ensuring appropriate resources and infrastructure Analysing existing measurement data Identifying deviances and taking the appropriate corrective

action


31


Level 5. (optimising): 5.1. Process change

Identification and approval of changes needed in the standard process Ensuring resources needed for institutionalisation Institutionalising the approved change Analysing efficiency of the change introduced

5.2. Continuous improvement Identifying improvement opportunities Defining institutionalisation strategy Institutionalising changes in the customised procedure Analysing efficiency of the change introduced


32

SPICE audit Working method

– Choosing the processes– Questionnaires– Interviews– Report– Process improvement plan– Registration in database

Assessment results: maturity profile for the chosen processes.


33

SPICE audit process


34

Components of SPICE documents set


35

SPICE audit

5. Optimising

4 . Predictable

3. Established

2. Managed

1. Performed

0. Not performed

ProcessesMaturity levels

P1 P2 P3


36

SPICE - profile

Process

Requirement management

Supplier management

Requirement analysis

Planning

Coding

Testing

Process maturity

A1 2.1 2.2 3.2. ..5.2


37

SPICE assessment

SPI plan is an important result of the assessment


38

SPICE assessment - example

Level 1.: performed process N P W F Remarks

1.1. Process performance

1.1.1. Basic practices needed for the process are in place.


39

SPICE assessment - exampleLevel 2.: Managed process N P W F Remarks

2.1. Performance management

2.1.1. Identifying resource needs

2.1.2. Performance planning

2.1.3. Implementing the defined activities

2.1.4. Managing the execution


40

SPICE assessment

Evaluation of the results:– Base practice existence based algorithm / or– goal based algorithm– Weighting based on most important drivers


41

Goal based algorithm

Algorithm for calculating the maturity level : The attributes „fully”, „widely”, „partially” and „not existing” have

fixed values T – 1, W – 0.666, P – 0.333, N – 0 -this is valid on level 1, on other

levels we have other values !!! According to experience / averages, the results are the following:

from „quarter” level

0.0000 1 +0.00

0.1950 2 +0.25

0.3950 3 +0.50

0.5950 4 +0.75

0.7950 Next level + 1


42

Goal based algorithm: example Process: configuration management

Goals Evaluation Connected value

All project members are using the appropriate version of the products F 1

The elements have their Checked in / out state F 1

It is possible at any time to produce a complete release or a test configuration

W 0,666

Description of configuration elements is constantly updated W 0,666

Possibility for configuration state report exists N 0

Changes can be followed W 0,666

Configuration elements are complete, consistent and correct P 0,333

Configuration elements are fully transparent, and the degree in which they satisfy the requirements can be followed

P 0,333

from „quarter” level0.0000 1 +0.000.1950 2 +0.250.3950 3 +0.500.5950 4 +0.750.7950 Next level + 1

Average: 0,583

Result: 0,50


43

Taking into account basic „drivers” - example

Process Business drivers Effect Maturitylevel

Time dependence

Productquality

Servicequality

Costs

Wei

ght 30 30 20 20

Project management 3 1 2 3 2,2 2

Quality assurance 1 3 3 2 2,2 1

Configuration management

2 2 2 1 1,8 1

Risk management 2 2 2 2 2,0 1

Subcontractor management

0 2 0 1 0,8 2

Testing 1 3 1 1 1,8 1

Integration 1 2 2 1 1,5 1

Importance: 1-3 effect= time dependence x 30 + product qual x 30 + process qual x 20 + cost x 20 100


44

Choosing development direction

01

2

3

4

5

1 2 3Effect

Mat

urit

y le

vel

Areas for improvement


45

SPICE assessment results

The algorithms are always the basis only. Final decision is to be taken by the auditors,

taking into account all circumstances.


46

What we talked about...

Product

Process

Resourc

e

Definition

Quality attribute

Metric

software quality and management 5. process oriented approaches towards software quality. spice model...

Documents

software process assessment

software assessment

software process improvement

process management

continuous process

quality improvement

capability assessment

software engineering