sdm chapter6 se01 - wordpress.com · copyright © 2004, infosys technologies ltd ... environmental...

1

Software Quality

2

Copyright © 2004, Infosys Technologies Ltd

2 ER/CORP/CRS/SE01/003 Version No: 2.0

Software Quality

What is Quality?

Roger Pressman, a noted software engineering author and consultant, defines software quality as:

Conformance to explicitly stated functional and performance requirements, explicitly documented development standards, and implicit characteristics that are expected of all professionally developed software

The International Organization for Standardization (ISO) defines quality asThe totality of features and characteristics of a product or service that bears on its ability to satisfy stated or

implied needs. - (ISO 8402, 1986)

Introduction

What is quality?

Roger Pressman, a noted software engineering author and consultant, defines software quality as:

Conformance to explicitly stated functional and performance requirements, explicitly documented development standards, and implicit characteristics that are expected of all professionally developed software

The International Organization for Standardization (ISO) defines quality as

The totality of features and characteristics of a product or service that bears on its ability to satisfy stated or implied needs. - (ISO 8402, 1986)

Note carefully that these definitions are not used to express a degree of excellence in a comparative sense; it merely identifies a product’s quality with a subset of its attributes (which usually includes its price). The word 'implied' is also important: a decade ago, the emphasis was on products, and quality meant a product’s ability to conform to specifications, i.e., stated needs. In the last several years, the word quality has been used increasingly to encompass features and characteristics that have a bearing on an organization’s many stakeholders, including community, suppliers, shareholders, employees and management. Quality now includes measures of work life, workplace diversity, environmental conditions and competitiveness.

A shorter definition is that a high-quality software system is one that’s delivered to the users on time, costs no more than was projected, and, most importantly, works properly. Working properly implies that the software must be as nearly bug-free as possible.

3



What is Quality?

Q: What is the Quality of a radio?

What is the Quality of a radio?

Throw this question to the house and wait for responses. You will get a lot of technical attributes like portability, clarity of sound, bandwidth etc. List them on the board.

Typically, you will not get someone saying “cost.” So make a note of this and add cost to the list. The rationale is simply that cost is one of the attributes that have a bearing on the desirability!

Point out that this set of attributes is a vector, so that the “quality of a radio” is nothing but a vector. So it is meaningless to talk of high quality or low quality just as it is meaningless to talk of a large vector or a small one.

Each customer segment likes one particular kind of vector. A customer buying a portable radio for carrying it for picnic will want a vector with a large component of portability but a low component of cost and fidelity so that he can listen to cricket commentary while roaming.

Finally, then, because quality is a vector, you can compare two vectors only if they are the same on some attributes but one is better on every other attribute. Bettering a product this way is what it means to “improve quality”

4



What is Quality? – (Contd..)

• Please refer to Notes page for more explanation on previous slide

It is worth mulling over what it means to "improve" on a product’s quality. Is a pocket transistor radio costing Rs. 200 of lower quality than a high-fidelity monster costing Rs. 20,000? Or is it the other way round? To a large extent, the answer depends on how your existing and potential customers are defined, since it is they who mostly determine stated and implied needs. The ISO uses the term grade to indicate a planned difference in requirements, or, if not planned, a recognized difference. Thus, for example, the pocket radio and the hi-fi monster would belong to different grades since they are targeted at customers with distinct sets of requirements. We can, in general, only be sure that if each attribute of one radio is at least as good as the corresponding attribute in the other, then one of them can unambiguously be deemed the higher quality product. In turn, the process which makes the better product can be deemed to be the better process.

5



Need for Improvement

• Competitive Pressures• Customer Satisfaction• Management of Change

Why improve on the quality of a product or process?

Competitive Pressures: Today’s business is extremely competitive, and the software industry is no exception. Companies must continuously make and sustain improvements in cost and quality to remain in business.

Customer Satisfaction: Acquiring a new customer is far more expensive than retaining a current one. Further, few dissatisfied customers complain; the vast majorities simply take their business elsewhere.

The Management of Change: In an industry like software where new tools and methods appear to arrive at a faster rate than it takes to train staff in their use, it is especially important that organizations fundamentally change their management styles and their workforce’s attitudes so as to effect significant improvements.

6



Attributes of Quality

• To ‘improve’ the Quality of software we first need to measure Quality• To measure Quality we need to understand measure ‘What’ and ‘How’• Software Quality Factors are the ‘What’ part of the measure.• Software Quality Matrix are the ‘How’ part of the measure

Attributes of Software Quality

In order to ‘improve’ the quality of software first we should be able to measure the quality of software. To measure quality we need to understand the factors that affect quality. These factors should be sufficiently objective, rather than subjective and therefore measurable to help correctly evaluate the quality of the software. The collection of such measurable quality factors is called the Quality Matrix.

Although there are many schools of thought with different set of factors we will discuss here McCall & Cavano’s quality factors because McCall's work has been first steps towards development of Software Quality Metrics. We will also discuss quality Factors defined by ISO 9126 as they are the current standards.

7



Cost Of Quality (COQ)

• Why COQ?� “COQ is the quantification of the benefits of Improved Quality.”� In simple terms low COQ is higher benefits for same quality.

• What is COQ?� Quality costs are the costs associated with preventing, finding and correcting defective work.

• Components of COQ� Prevention Costs: These are costs of activities specifically designed to prevent poor quality� Appraisal Costs: These are Costs of activities designed to find quality problems, such as code inspections

and any type of testing� Failure Costs: Costs that result from poor quality, such as the cost of fixing bugs, wasted efforts, cost of

dealing with customer complaints, etc.

Cost of QualityThe motivation for improvement has been helped considerably by quantifying the benefits of improved quality into the language of money. This measurement is known as the cost of quality, and there are a number of ways in which it can be used:•To quantify the costs of chronic waste; software that fails to be delivered falls in this category. It is estimated that fifteen percent of software that is written is never used.•To quantify the benefits of using preventive techniques to avoid problems rather than using appraisal methods to correct them.

It is often tempting to conclude, at the end of a cost-of-quality estimation exercise, that there is no point in improving quality because the associated costs are greater than the benefits from the improvement. This conclusion is often arrived at without properly estimating the future benefits from quality improvement. Further, cost of quality figures have an important limitation: they typically ignore the intangible but very real loss of goodwill from a customer.

There are two well-accepted and documented ways of recording your quality costs which are described in the British Standard BS 6143: Guide to the economics of quality. These are the Process Analysis model and the Prevention, Appraisal, Failure (PAF) model. In the process analysis model, all the activities of the firm are first viewed as well-defined processes; the cost of quality is then deduced in terms of the cost of conformance or non-conformance to these processes, without making judgments about the efficiency or relevance of the processes themselves. In the PAF model, as the name suggests, costs are split into the three abovementioned categories. The following figure shows the behavior of these three cost components. As the figure shows, there is usually an optimum level of quality at any given point of time.

8



Cost Of Quality Contd. …

Degree of quality

cost

Prevention costs

Failure costs

Appraisal costs

Total Cost of Quality

Prevention Costs: These are costs of activities specifically designed to prevent poor quality (coding errors, design errors, bad documentation, etc). E.g. Staff Training, requirement analysis, accurate internal documentation, etc. These costs increase with increased degree of quality.

2. Appraisal Costs: These are Costs of activities designed to find quality problems, such as code inspections and any type of testing. E.g. Design Review, Code Inspection, White box testing, Black box testing, training testers, Beta testing, test automation, usability testing, etc. Appraisal costs are initially low because there is practically no appraisal - you just make and market come what may. When you realize the importance of quality, you institute testing practices and so incur some costs. So at that stage, increase in quality implies some increase in costs. When you improve quality even more, you are relying on prevention rather than appraisal, so that the further increase in quality is due to increased investment in prevention and reduced dependence on appraisal.

3. Failure Costs: Costs that result from poor quality, such as the cost of fixing bugs and cost of dealing with customer complaints. Internal failure costs are failure costs that arises before your company supplies the product to customer, e.g. bug fixing. External failure costs are failure costs that arises after your company supplies the product to customer, e.g. production problems, investigation of customer complaints, refunds, penalties and recalls, etc. Obviously these costs reduce with increase in degree of quality.

4. Total Cost of Quality = Prevention Costs+ Appraisal Costs + Internal failure Costs + External failure Costs.

9



International Quality Standards

• ISO: International Organization for Standardization• CMM: Capability & Maturity Model• CMMI: Capability & Maturity Model - Integrated• TQM: Total Quality Management• Six Sigma• Malcolm Baldrige National Quality Award

10



International Quality Standard: ISO

• ISO 9000 & ISO 14000 are Generic Management Systems Standards• ISO Defines Quality System as:

“The organizational structure, responsibilities, procedures, processes and resources needed to implement quality management.”

-(ISO 8402, 1991)

• Benefits of ISO include� Improved operations� Improved margins� External recognition� Market expansion

• ISO 9000 is about Processes not Products.• ISO 14000 is about Environmental Management.

ISO

ISO 9000 and ISO 14000 are Generic Management System Standards. Generic means that the same standards can be applied to any organization large or small, whatever its product and whether it is a private enterprise, a public administration or a government department.

•Benefits of ISO include:

•Improved operations

•Improved margins

•External recognition

•Market expansion

To be able to understand ISO 9000 we need to know what a quality system is. ISO defines this as:

The organizational structure, responsibilities, procedures, processes and resources needed to implement quality management.

(ISO 8402, 1991)

The ISO 9000 is a series of five international standards on quality management and assurance. The series consists of ISO 9000, ISO 9001, ISO 9002, ISO 9003 and ISO 9004.

11



The ISO 9000 Series

• ISO 9000, Quality management and quality assurance standards - Guidelines for selection and use is a road map for implementing ISO 9001 - ISO 9003.

• ISO 9001, Quality systems - Model for quality assurance in design/development, production, installation and servicing.

• ISO 9002, Quality systems - Model for quality assurance in production and installation.

• ISO 9003, Quality systems - Model for quality assurance in final inspection and test.

• ISO 9004, Quality management and quality system elements - Guidelines and other related documents describe recommended quality actions or guidelines.

ISO 9000, Quality management and quality assurance standards - Guidelines for selection and use is a road map for implementing ISO 9001 - ISO 9003.ISO 9001, Quality systems - Model for quality assurance in design/development, production, installation and servicing is used by companies to certify their quality systems throughout their product-development cycle.ISO 9002, Quality systems - Model for quality assurance in production and installation may be used by a company whose products have already been marketed, tested, improved and approved.ISO 9003, Quality systems - Model for quality assurance in final inspection and test is forcompanies in which comprehensive quality systems may not be important or necessary, such as commodity suppliers.ISO 9004, Quality management and quality system elements - Guidelines and other related documents describe recommended quality actions or guidelines.

The ISO 9001, 9002 & 9003 are contractual documents which describe compliance in “shall”language. The ISO 9004 uses “should” language.

12



The ISO 9000 Series –(contd..)


It is important to know that ISO 9000 registration is for quality systems, not products. The assumption is that if quality systems exist, internal processes are controlled. This, however, does not warrant another assumption one may be tempted to make: that if these processes are controlled, they are capable and are improving, and that the products from these processes conform to customer requirements. In other words, having registered quality systems can be far removed from guaranteeing high product quality or satisfying customers. Thus, it is perfectly possible for a firm manufacturing concrete life jackets or paper raincoats to get its quality systems certified under ISO 9002, covering production and installation, although the products would not serve any perceived customer need.

This, however, is not a major deficiency in the registration procedure: any processes in the quality system for design and development would have to include processes for recording and addressing customer feedback. If the targeted customers found the product not satisfying their requirements, their feedback would ultimately drive this firm to examine the suitability of making concrete life jackets or paper raincoats.

13



International Quality Standards: CMM

• What is CMM?It defines how software organizations mature or improve in their ability to develop software.

• CMM is software specific TQM model.• Developed by Software Engineering Institute (SEI) of Carnegie Mellon University (CMU) in 1993 (CMM Ver.

1.1).• Defines 5 levels of process maturity for a software organization• Each level consists of a grouping of homogeneous Key Process Areas (KPAs)• The relationship is such that the most benefit from a KPA can only be achieved when KPAs from lower levels

are in place.• The KPAs in each level enhance the effectiveness of the previous maturity level KPAs while strengthening the

foundation for further process maturity growth.• CMM Model enables an organization to reduce project risk and increase productivity and quality.

CMM

As ISO is generic management system standards CMM standards are specific to software or IT industry. CMM model is based on Total Quality Management or TQM. There are lots of definitions for CMM. One good one is this:

It defines how software organizations mature or improve in their ability to develop software.

The model was developed by Software Engineering Institute (SEI) of Carnegie Mellon University in late 80s. This model provides a structured view in a five-layer model of increasingly sophisticated practices for those working in software. Each level is focused on achieving certain objective and each level (except Level 1) has certain Key Process Areas or KPAs associated with it. Each level addresses levels of maturity exhibited by the project. The first version of CMM came in early 90s. Following level describes the various levels of CMM.

14



CMM Contd. …

Project RiskProject RiskProject RiskProject Risk

Productivity and QualityProductivity and QualityProductivity and QualityProductivity and Quality

1 Initial

2 Repeatable

3 Defined

4 Managed

5 Optimizing

Heroes

ProjectProjectManagementManagement

EngineeringEngineeringProcessProcess

Product and Product and Process QualityProcess Quality

Change ManagementChange ManagementDefect PreventionDefect Prevention

Disciplined

Standard,,Consistent

Predictable

ContinuouslyImproving

The orange triangle in the diagrams is the Project Risk, as the organization achieves higher level of maturity the project risks reduces and productivity increases, the green triangle is productivity. The yellow boxes are the five levels of CMM.

Level 1 Initial: There are no processes defined and success is person dependent. The process capability of this level, therefore, is defined as ‘Heroes’.

Level 2 Repeatable: The focus area of this level is ‘Project Management’. Project standards are defined and followed. Projects functionality, scheduled and costs are tracked. Projects history data is available and used. Risks can be identified at early stages of the projects. Projects processes are therefore under control. The process capability of the organization at level two is disciplined.

Level 3 Defined: The focus area of this level is ‘Engineering Process’. At this level the processes are defined and standardized at organization level. Projects tailor these standard processes to suit their need. The process capability of the organization at this level is standard & consistent.

Level 4 Managed: The focus area of this level is ‘Product & Process Quality’. At this level quantitative quality goals are set for software products and processes at organization level. Organization–wide “Process Capability Data” is maintained deriving data from completed projects. Projects use this data as a benchmark or quality goal. The process capability of the organization at this level is predictable.

Level 5 Optimizing: The focus area of this level is ‘Continuous Improvement’. Organization has means to identify weakness and strengthen the process proactively, with a goal of preventing the occurrence of defect. The process capability achieved at this level is continuous improvement.

15



ISO vs. CMM

Not mandatoryInternal Audits are mandatory6

CMM is staged (It has different levels of process maturity)ISO is continuous 5

After CMM assessment, there are no such checks. It is up to the organization to use it for internal process improvements

ISO Certification is followed by Surveillance audit once in 6 months4

CMM is specific to software processesISO focuses on the entire organization’s processes3

CMM is a model. It doesn’t mandate the practicesISO is a standard. It tells what needs to be done, in an organization.2

CMM is very specific to software industry.ISO is a generic standard. It is applicable for all industries.1

CMMISOSr.No

ISO vs. CMM

When applied to software industry what are the differences between the two internationally acclaimed quality standards, ISO and CMM?

Though both ISO & CMM are based on the TQM model, both talk about Continuous Improvement (CMM stresses more on this), both stress on “Say what you do. Do what you say”, there are some obvious differences.

16



International Quality Standard: CMMI

• CMMI is CMM-Integrated.• Multiple CMM models, like CMM for Software Development, CMM for Product Development, etc

were integrated by SEI in 1998-99 to form a new model called CMMI.• Structure of CMM & CMMI is same, both have 5 levels of maturity.• Difference lies in the KPAs for each level.

� Level 1: Both CMM & CMMI has no KPA at this level,� Level 2: CMM has 6 KPA while CMMI has 7 KPA� Level 3: CMM has 7 KPA while CMMI has 13 KPA� Level 4: Both CMM & CMMI has 2 KPA each.� Level 5: CMM has 3 KPA while CMMI has 2 KPA

• In all CMM has 18 KPA while CMMI has 24 KPA.

17



International Quality Standard: TQM

• Definition of TQM“A management approach to an organization centered on quality, based on the participation of

all its members and aiming at long-term success through customer satisfaction, and benefits to the members of the organization and to society.”

-(ISO 8402, 1991)

• Principles of TQM� Quality is an organization wide process.� Quality is what the customer says it is. � Quality is an ethic. � Quality is the most cost-effective, least capital-intensive route to productivity.

Total Quality Management (TQM)Now as we know that ISO and CMM are based on TQM, let’s see what TQM is. In the last ten years there has been a rapid acceleration in defining different approaches to the production of quality products that can all be categorized loosely as Total Quality Management (TQM). The term “Total Quality Management” was probably first adopted by the US Naval Air Systems Command in 1985, to describe the Japanese style management approach to continuous improvement. The ISO definition of TQM isA management approach to an organization centered on quality, based on the participation of all its members and aiming at long-term success through customer satisfaction, and benefits to the members of the organization and to society.(ISO 8402, 1991)Some of the canons that constitute this management approach are:Quality is an organization wide process. It is not a specialist function, a department, nor awareness or testing program alone.Quality is what the customer says it is. No one can compress in a market research statistic or defect rate the extent of buyer frustration or delight.Quality is an ethic. The pursuit of excellence, deep recognition that what you are doing is right, is the strongest human emotional motivator in an organization. Cold metrics alone are not enough.Quality is the most cost-effective, least capital-intensive route to productivity. The productivity concept, which was thus far confined to more, is enhanced to include the quality aspect of good.TQM, in being an approach, goes beyond mere quality management, which refers to the activities of the overall management function that determine the quality policy, objectives, and responsibilities, and implement them by various means.The software industry is still relatively young; as such, the application of TQM to software is still very much in its infancy. The industry is attempting to reach in 30 years a stage of maturity that has taken some industries over 200 years to achieve. This is illustrated in the following slide Ages of Quality.

18



Three Steps to TQM

• First� Create a strategic objective that commits the entire company to total quality and customer satisfaction as

the primary function of business.

• Second� Determine what customers want.

• Third� Design the products the way the customer wants to use them.� Build quality into every product.� Deliver on promises made to the customer.� Ensure that customer orders are filled correctly and on time.� Determine the correct price that gives the customers value.

Although experts differ in their views on how best to implement TQM, there are a number of essential common elements of any TQM system. A prerequisite to the implementation of TQM is the establishment of effective planning, measurement and control mechanisms. The other common elements include continuous improvement, an objective definition of quality, commitment, teamwork, and, of course, training and education.

The three steps to implement TQM are:FIRSTCreate a strategic objective that commits the entire company to total quality and customer satisfaction as the primary function of business.SECONDDetermine what customers want.THIRDDesign the products the way the customer wants to use them.Build quality into every product.Deliver on promises made to the customer.Ensure that customer orders are filled correctly and on time.Determine the correct price that gives the customers value.

19



International Quality Standards: Six Sigma

• The Six key concepts of Six Sigma are:1. Critical to Quality: Attributes most important to the customer.2. Defect: Failing to deliver what the customer wants.3. Process Capability: What your process can deliver.4. Variation: What the customer sees and feels5. Stable Operation: Ensuring consistent, predictable processes to improve what

the customer sees and feels.6. Design: Designing to meet customer needs and process capability

Six Sigma

The word 'Sigma' is a statistical term that measures how far a given process deviates from perfection. The significance of Six Sigma is that if you can measure how many defects you have in a process, you can systematically figure out how to eliminate them and get as close to 'zero defects' as possible.

The Six Key concepts of Six Sigma are:1. Critical to Quality: Attributes most important to the customer

Understanding Customer's needs and expectations by employing six approaches to communicating with customer

Measure business performance against dynamic customer requirements and respond to changing market place conditions

Quality function deployment (QFD) and failure modes and effects analysis (FMEA) can help identify critical to quality characteristics.

2. Defect: Failing to deliver what the customer wantsReducing the defect rateDetermining the defect costMistake Proofing techniques eliminate the sources of errors and ensure that a

process is free of defects.3. Process Capability: What your process can deliver

Understanding Process Capability principles and calculating Process Capability are integral to staying competitive and meeting customer requirements.

20



International Quality Standards: Six Sigma – (Contd..)


4. Variation: What the customer sees and feels

Two sources of variation : Common cause and special cause

Multi-Vari Analysis offers a means of reducing large numbers of unrelated causes of variation to a family of related causes.

Reducing common-cause variation so that the distribution has a very small standard deviation

5. Stable Operations: Ensuring consistent, predictable processes to improve what the customer sees and feels.

Operational Excellence methodology for identifying the right projects, using the right people to lead projects, right tools and roadmap

Turning quality into a management system

Improving cycle time to process applications for long term disability benefits

6. Design: Designing to meet customer needs and process capability.

Follow a four phase process to achieve design. Identify, Design, Optimize and Validate

Design helps eliminate designed-in quality problems that account for 70-80% of defects

Link Six Sigma with QS 9000.

21



Malcolm Baldrige National Quality Awards

6%Effectiveness of integrating quality requirements into business plansStrategic quality planning

7%Effectiveness of information collection and analysis for quality improvement and planningInformation and analysis

14%10%

Effectiveness of systems and processes for assuring the quality of al operationsSenior executives’ success in creating and sustaining a quality culture

Management of process quality leadership

15%Success of efforts to develop and realize the full potential of the work force for qualityHuman resource development and

management

18%Results in quality achievement and quality improvement, demonstrated through quantitative

measuresQuality and operational results

30%Effectiveness of systems to determine customer requirements and demonstrated success in

meeting themCustomer focus on satisfaction

Relative WeightingDescriptionBaldrige Award Criteria

Malcolm Baldrige National Quality Award

One response to the inroads Japanese and European competitors made in both U.S. and international markets during the 1980s was the creation of the Malcolm Baldrige National Quality Award. Widespread concerns that the quality levels of imported products exceeded American-made products led Congress to establish this prize in 1987 as part of a national quality improvement effort. Named after the late secretary of commerce, the Baldrige Award is the highest level of national recognition for quality that a U.S. company can receive. Its purposes are to promote quality awareness, to recognize quality achievements of U.S. companies, and to publicize successful quality strategies.

The table in the slide lists the criteria for winning the Baldrige Award

22



Summary

• In this chapter we looked at:� What is Software Quality

� What are the attributes and factors of Software Quality� The Cost of Quality

� Parameters of Quality Matrix � International Quality Standards like

� ISO

� CMM � CMMI � TQM

� Six Sigma & � Malcolm Baldrige National Quality Award.

23



References

• Pressman, R.S (1992)., Software Engineering: A practitioner’s approach., McGraw-Hill, Singapore, 3rd Edition.

• Gillies, AC & Smith, P (1994)., Managing Software Engineering: CASE studies and solutions., Chapman & Hall, London.

• PriDE• CSQA Program @ Infosys

24



Thank You!

sdm chapter6 se01 - wordpress.com · copyright © 2004, infosys technologies ltd ... environmental...

Documents