software quality assurance. recap what is sqa? what is quality? different terminologies cost of...

23
Software Quality Assurance

Upload: marjorie-logan

Post on 22-Dec-2015

232 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Software Quality Assurance. Recap What is SQA? What is quality? Different Terminologies Cost of correcting an error Elements of SQA

Software Quality Assurance

Page 2: Software Quality Assurance. Recap What is SQA? What is quality? Different Terminologies Cost of correcting an error Elements of SQA

Recap

• What is SQA?• What is quality?• Different Terminologies• Cost of correcting an error• Elements of SQA

Page 3: Software Quality Assurance. Recap What is SQA? What is quality? Different Terminologies Cost of correcting an error Elements of SQA

SQA Goals, Attributes and Metrics

GoalsRequirement quality

Design quality

Attributes• Ambiguity• Completeness• Understandability• Volatility

• Traceability

• Model clarity

• Architectural integrity• Component

completeness• Interface complexity• Patterns

Metric• Number of ambiguous modifiers (e.g.,

many, large, human-friendly)• Number of TBAs, TBDs• Number of sections/subsections

• Number of changes per requirement• Time (by activity) when change is

requested• Number of requirements not traceable to

design/code• Number of UML models• Number of descriptive pages per model• Number of UML errors

• Existence of architectural model• Number of components that trace to

architectural model• Complexity of procedural design• Layout appropriateness• Number of patterns used

Page 4: Software Quality Assurance. Recap What is SQA? What is quality? Different Terminologies Cost of correcting an error Elements of SQA

SQA Goals, Attributes and Metrics

GoalsCode quality

QC effectiveness

Attributes• Complexity• Maintainability• Understandability

• Reusability• Documentation

• Resource allocation• Completion rate• Review effectiveness• Testing effectiveness

Metric• Cyclomatic complexity

• Design factors• Percent internal comments• Variable naming conventions• Percent reused components

• Readability index

• Staff hour percentage per activity

• Actual vs. budgeted completion time

• Review metrics

• Number of errors found and criticality• Effort required to correct an error• Origin of error

Page 5: Software Quality Assurance. Recap What is SQA? What is quality? Different Terminologies Cost of correcting an error Elements of SQA

SQA plan

• Management section– describes the place of SQA in the structure of the organization

• Documentation section– describes each work product produced as part of the software process

• Standards, practices, and conventions section– lists all applicable standards/practices applied during the software process and any

metrics to be collected as part of the software engineering work• Reviews and audits section

– provides an overview of the approach used in the reviews and audits to be conducted during the project

• Test section– references the test plan and procedure document and defines test record keeping

requirements• Problem reporting and corrective action section

– defines procedures for reporting, tracking, and resolving errors or defects, identifies organizational responsibilities for these activities

• Other– tools, SQA methods, change control, record keeping, training, and risk management

Page 6: Software Quality Assurance. Recap What is SQA? What is quality? Different Terminologies Cost of correcting an error Elements of SQA

Statistical SQA

• Information about software defects is collected and categorized • An attempt is made to trace each defect to its underlying cause• Isolate the vital few causes of the major source of all errors• Then move to correct the problems that have caused the defects

Page 7: Software Quality Assurance. Recap What is SQA? What is quality? Different Terminologies Cost of correcting an error Elements of SQA

Statistical SQA – Categories of errors

• Incomplete or erroneous specification (IES)• Misinterpretation of customer comm (MCC)• Intentional deviation from specification (IDS)• Violation of programming standards (VPS)• Error in data representation (EDR)• Inconsistent module interface (IMI)• Error in design logic (EDL)• Incomplete or erroneous testing (IET)• Inaccurate or incomplete documentation (IID)• Error in programming lang. Translation (PLT)• Ambiguous or inconsistent human-computer interface (HCI)• Miscellaneous (MIS)• Most often IES, MCC and EDR are the vital few causes for majority of

errors.

Page 8: Software Quality Assurance. Recap What is SQA? What is quality? Different Terminologies Cost of correcting an error Elements of SQA

Identifying the vital few

Page 9: Software Quality Assurance. Recap What is SQA? What is quality? Different Terminologies Cost of correcting an error Elements of SQA

Statistical SQA

Page 10: Software Quality Assurance. Recap What is SQA? What is quality? Different Terminologies Cost of correcting an error Elements of SQA

Example

Page 11: Software Quality Assurance. Recap What is SQA? What is quality? Different Terminologies Cost of correcting an error Elements of SQA

Statistical SQA – Six Sigma

• Most widely used strategy for statistical SQA• Three core steps

– Define customer requirements, deliverables and project goals via well-defined methods of customer communication

– Measure the existing process and its output to determine quality– Analyze defect metrics and determine the vital few causes

• If an existing software process is in place, but improvement is required six sigma suggests– Improve the process by eliminating the root causes of defects– Control the process to ensure that future work does not reintroduce the cases of defects

• If an organization is developing a software process, the core steps are augmented– Design the process to (1) avoid the root causes of defects and (2) to meet customer

requirements– Verify that the process model will, in fact, avoid defects and meet customer

requirements

Page 12: Software Quality Assurance. Recap What is SQA? What is quality? Different Terminologies Cost of correcting an error Elements of SQA

Reviews To uncover errors/defects

• To uncover errors in function, logic, or implementation for any representation of the software

• To verify that software meets its requirements• To ensure that software representation meets predefined

standards• To achieve software development in a uniform manner• To make projects more manageable

Page 13: Software Quality Assurance. Recap What is SQA? What is quality? Different Terminologies Cost of correcting an error Elements of SQA

Review Roles

• Presenter (designer/producer).• Coordinator (not person who hires/fires).• Recorder

– records events of meeting– builds paper trail

• Reviewers – maintenance oracle– standards bearer– user representative– others

Page 14: Software Quality Assurance. Recap What is SQA? What is quality? Different Terminologies Cost of correcting an error Elements of SQA

Formal Technical Reviews

• Involves 3 to 5 people (including reviewers)• Advance preparation (no more than 2 hours per person) required• Duration of review meeting should be less than 2 hours• Focus of review is on a discrete work product• Review leader organizes the review meeting at the producer's request.• Reviewers ask questions that enable the producer to discover his or her

own error (the product is under review not the producer) • Producer of the work product walks the reviewers through the product• Recorder writes down any significant issues raised during the review• Reviewers decide to accept or reject the work product and whether to

require additional reviews of product or not.

Page 15: Software Quality Assurance. Recap What is SQA? What is quality? Different Terminologies Cost of correcting an error Elements of SQA

Formality and Timing

• Formal review presentations– resemble conference presentations.

• Informal presentations– less detailed, but equally correct.

• Early– tend to be informal– may not have enough information

• Later– tend to be more formal– Feedback may come too late to avoid rework

Page 16: Software Quality Assurance. Recap What is SQA? What is quality? Different Terminologies Cost of correcting an error Elements of SQA

Formality and Timing

• Analysis is complete.• Design is complete.• After first compilation.• After first test run.• After all test runs.• Any time you complete an activity that produce a complete work product.

Page 17: Software Quality Assurance. Recap What is SQA? What is quality? Different Terminologies Cost of correcting an error Elements of SQA

Why do peer reviews?

• To improve quality.• Catches 80% of all errors if done properly.• Catches both coding errors and design errors.• Enforce the spirit of any organization standards.• Training and insurance.

Page 18: Software Quality Assurance. Recap What is SQA? What is quality? Different Terminologies Cost of correcting an error Elements of SQA

Review Guidelines..

• Review the product, not producer

• Set an agenda and maintain it• Limit the debate • Enunciate problem areas, not

to solve every problem noted• Take written notes• Allocate resources and time

schedule for FTR’s• Use standards to avoid style

disagreements.• Let the coordinator run the

meeting and maintain order.

• Limit the number of participants and insist upon advance preparation

• Develop a checklist for each work product to be reviewed

• Training for all reviewer’s• Reviewing earlier reviews

Keep it short (< 30 minutes).• Don’t schedule two in a row.• Don’t review product

fragments.

Page 19: Software Quality Assurance. Recap What is SQA? What is quality? Different Terminologies Cost of correcting an error Elements of SQA

Effectiveness of review Defect Amplification and Removal

Errors passed through

Amplified errors 1:x

Newly identified errors

Percent efficiency for error detection

Errors from previous steps Errors passed to

next step

Development step

Defects Detection

• Used to illustrate the generation and detection of errors during design and code generation

Page 20: Software Quality Assurance. Recap What is SQA? What is quality? Different Terminologies Cost of correcting an error Elements of SQA

Effectiveness of review Defect Amplification and Removal

No reviews

With reviews

Page 21: Software Quality Assurance. Recap What is SQA? What is quality? Different Terminologies Cost of correcting an error Elements of SQA

Effectiveness of review Defect Amplification and Removal

Page 22: Software Quality Assurance. Recap What is SQA? What is quality? Different Terminologies Cost of correcting an error Elements of SQA

Review metrics and their use

• Many metrics can be defined for technical reviews• The following can be calculated for each review conducted:

– Preparation effort (Ep)

– Assessment effort (Ea)

– Rework effort (Er)– Work product size (WPS)– Minor errors found (Errminor)

– Major errors found (Errmajor)

Page 23: Software Quality Assurance. Recap What is SQA? What is quality? Different Terminologies Cost of correcting an error Elements of SQA

Summary

• SQA goal, attributes and metrics• SQA plan• Formal Technical Review (FTR)• Statistical SQA– Six Sigma– Identifying vital few causes– Review efficiency