1

Chapter 8Chapter 8Software Quality Software Quality

AssuranceAssurance

2

Software Quality Software Quality AssuranceAssurance

Software Software requirementsrequirements

Specified standards – Specified standards – development criteriadevelopment criteria

Implicit requirementsImplicit requirements

3

Cost of Cost of QualityQuality

Prevention costsPrevention costs•Quality planningQuality planning•Formal technical reviewsFormal technical reviews•Test equipmentTest equipment•Training Training 

Appraisal costsAppraisal costs• In-process and interprocess inspectionIn-process and interprocess inspection•Equipment calibration and Equipment calibration and

maintenancemaintenance•Testing Testing 

4

Failure costsFailure costs• Internal failure costsInternal failure costs

Failure mode analysis Failure mode analysis RepairRepairReworkRework

• External failure costsExternal failure costsComplaint resolutionComplaint resolutionProduct return and replacementProduct return and replacementHelp line supportHelp line supportWarranty workWarranty work

5

ResponsibiResponsibilitylity

Project managersProject managers Software Software

engineersengineers SQA teamSQA team SalespeopleSalespeople CustomersCustomers

6

Quality Quality ManagementManagement

&&Software Software

DevelopmentDevelopment

7

SQA SQA Activities Activities PreparesPrepares an SQA plan for a project to the an SQA plan for a project to the

software project team.software project team. ParticipatesParticipates in the development of the in the development of the

project’s software process description.project’s software process description. ReviewsReviews software engineering activities to software engineering activities to

verify compliance with the defined software verify compliance with the defined software process.process.

AuditsAudits designated software work products to designated software work products to verify compliance with those defined as part verify compliance with those defined as part of the software process.of the software process.

Ensures Ensures that deviations in software work and that deviations in software work and work products are documented and handled work products are documented and handled according to a documented procedure.according to a documented procedure.

RecordsRecords any noncompliance and any noncompliance and reportsreports to to senior management.senior management.

8

Software Software ReviewReview

9

WhWhyy

To error is humanTo error is human Large classes of errors escape the Large classes of errors escape the

originator mare easily than they originator mare easily than they escape anyone else.escape anyone else.

Defect amplificationDefect amplification

10

Why SQA Activities Why SQA Activities Pay Off?Pay Off?

cost to findcost to findand fix a defectand fix a defect

100100

1010

loglogscalescale

11

Req.Req.DesignDesign

codecodetesttestsystemsystem

testtestfieldfielduseuse

0.750.751.001.001.501.50 3.003.0010.0010.00

60.00-100.0060.00-100.00

11

Defect Defect AmplificationAmplification

12

13

14

Formal Technical Formal Technical ReviewsReviews(FTR) (FTR)

Objectives of FTR:Objectives of FTR:• to uncover errors to uncover errors • to verify requirementsto verify requirements• to ensure predefined to ensure predefined

standardsstandards• to achieve uniform mannerto achieve uniform manner• to make more manageableto make more manageable

15

The PlayersThe Players

reviewreviewleaderleader

producerproducer

recorderrecorder reviewerreviewer

16

Conducting the Conducting the ReviewReviewbe prepared—evaluate be prepared—evaluate

product before the reviewproduct before the review

review the product, not review the product, not the producerthe producer

keep your tone mildkeep your tone mild

stick to the review schedulestick to the review schedule

raise issues, don't resolve themraise issues, don't resolve them

record and report all review resultsrecord and report all review results

1.1.

2.2.

3.3.

4.4.

5.5.

6.6.

17

Review Summary Review Summary ReportReport

What was reviewed?What was reviewed? Who reviewed it?Who reviewed it? What were the findings and What were the findings and

conclusions?conclusions?

18

Review Review Guidelines Guidelines

Review the Review the productproduct, not the producer., not the producer. Set an Set an agendaagenda and maintain it. and maintain it. Limit Limit debatedebate and and rebuttalrebuttal.. Enunciate/specify Enunciate/specify problem areasproblem areas, but don't , but don't

attempt to solve every problem noted.attempt to solve every problem noted. Take Take written noteswritten notes.. Limit the Limit the number of participantsnumber of participants and insist and insist

upon upon advance preparationadvance preparation.. Develop a Develop a checklistchecklist for each product that is for each product that is

likely to be reviewed.likely to be reviewed. Allocate Allocate resourcesresources and and schedule timeschedule time for FTRs. for FTRs. Conduct meaningful Conduct meaningful trainingtraining for all reviewers. for all reviewers. Review your Review your early reviewsearly reviews..

19

Statistical Statistical SQASQA

ProductProduct& Process& Process

measurement

... an understanding of how to improve quality ...

• • collect and categorizecollect and categorize information on all defectsinformation on all defects

• • find the causes of the defectsfind the causes of the defects

• • move to provide fixesmove to provide fixes

• • using Pareto principleusing Pareto principle

20

ExamplExamplee

21

CauseCausess Incomplete or erroneous specification (IES)Incomplete or erroneous specification (IES)

Misinterpretation of customer communication Misinterpretation of customer communication (MCC)(MCC)

Intentional deviation from specifications (IDS)Intentional deviation from specifications (IDS) Violation of programming standards (VPS)Violation of programming standards (VPS) Error in data representation (EDR)Error in data representation (EDR) Inconsistent component interface (ICI)Inconsistent component interface (ICI) Error in design logic (EDL)Error in design logic (EDL) Incomplete or erroneous testing (IET)Incomplete or erroneous testing (IET) Inaccurate or incomplete documentation (IID)Inaccurate or incomplete documentation (IID) Error in programming language translation of Error in programming language translation of

design (PLT)design (PLT) Ambiguous or inconsistent human/computer Ambiguous or inconsistent human/computer

interface (HCI)interface (HCI) Miscellaneous (MIS)Miscellaneous (MIS)

22

Phase & Error Phase & Error IndexIndex Phase IndexPhase Index

PIPIii = w = wss(S(Sii/E/Eii) + w) + wmm(M(Mii/E/Eii) + ) + wwtt(T(Tii/E/Eii))

Error IndexError Index EI = EI = (i x PI(i x PIii) / PS) / PS

• EEii = total number of errors uncovered during = total number of errors uncovered during the ith stepthe ith step• SSi i = the number of serious errors= the number of serious errors• MMi i = the number of moderate errors= the number of moderate errors• TTi i = the number of minor errors= the number of minor errors• wwss, w, wmm, w, wtt = = weighting factors for serious, weighting factors for serious, moderate, and trivial errorsmoderate, and trivial errors• PSPS = size of the product = size of the product

23

Software Reliability & Software Reliability & Availability Availability

Reliability Reliability

MTBF = MTTF + MTTRMTBF = MTTF + MTTRMTBF: Mean Time between failureMTBF: Mean Time between failureMTTF: Mean Time To FailureMTTF: Mean Time To FailureMTTR: Mean Time To RepairMTTR: Mean Time To Repair

  Availability Availability

Availability =[MTTF/(MTTF + MTTR)] x Availability =[MTTF/(MTTF + MTTR)] x 100%100%

24

SQA SQA planplan

The plan identifiesThe plan identifies• EvaluationsEvaluations to be performed to be performed •AuditsAudits and and reviewsreviews to be performed to be performed•StandardsStandards that are applicable to the that are applicable to the

projectproject•ProceduresProcedures for error reporting and for error reporting and

trackingtracking•DocumentsDocuments to be produced by the to be produced by the

SQA groupSQA group•Amount of feedbackAmount of feedback provided to the provided to the

software project teamsoftware project team

25

IEEE SQA IEEE SQA Plan Plan

26

The ISO 9000 Quality The ISO 9000 Quality StandardsStandards

20 requirements:20 requirements:•management responsibilitymanagement responsibility•quality systemquality system• contract reviewcontract review•design controldesign control•documents and data controldocuments and data control•product identificationproduct identification•……