nosqaa requirements inspection

Requirements Inspection

Leland Smith

2

What is an Inspection

Inspections are a disciplined engineering practice for detecting and correcting defects in software artifacts, and preventing their leakage into field operations.

Focused review to identify issues Defined roles and steps Requires training and preparation Adheres to time contracts Manages and reports metrics Formal documented process

3

Why Inspections?

By far, THE most effective way to reduce defect detection cost reduce rework improve product quality reduce cycle time enhance competency

Cumulative industry evidence is beyond reasonable dispute

4

Michael Fagan initiated inspections at IBM in1976 - pioneered their early adoption and later evolution

… finds 60-90% of all defects and provides feedback to programmers that enables them to avoid injecting defects in future design and coding work - Fagan

IEEE has standardized inspections A Key Process Area (kpa) of the CMM (Capability

Maturity Model) The hardest single part of building a software

system is deciding precisely what to build. – Fred Brooks

The single most common reason software engineers inject defects has been and is still poorly written requirements. - Steve Rakitin

Industry Experience

5

Requirements – PNC Bank began conducting requirement inspections in February 2006

Architecture / Design- work in progress Code Test Plans / Test Cases – PNC Bank began

conducting test inspections on a limited scale in November 2007

Inspection Types

6

FIND defects

in Requirements not Fix defects (or to provide brilliant insights into

how the author should have done it!) essential to efficiency and effectiveness

Requirements Inspection Goal

7

Impact without Requirements Inspection

Poor Requirements

Incomplete

Unclear

Incorrect

Application Design and Programming

Errors

Rework Due to Poor

Requirements

Test Case / Script Design and

Programming Errors

Rework Due to Poor

Requirements

Rework Due to Poor

Requirements

Impact to Customer Due to

Poor Requirements

Rework Due to Poor

Requirements

Impact to Bank $$$ due to Poor Requirements

Rework Due to Poor

Requirements

Analysis Phase Design / Build Phase

Test PhaseDeploy / Run

Phase

8

Target of Requirements Inspection

Functional Requirements Use Case (High Priority)

DescriptionPre and post conditions, actorsHappy path and alternate pathsInclusions and exclusions

Non-functional Requirements Business rules modifying functional requirements Constraints imposed upon the system

9

Requirements Inspection Overview

Requirements Inspection

Remediated Requirements

ChecklistsInspector Training

Defect Log

Use Cases and

Supplementary Reqs.

10

Requirements Inspection Process

1. Planning2. Individual Review3. Team Review4. Defect and

Investigation Resolution

5. Validate & Exit

11

Requirements Inspection Team Roles

Moderator

Leads the team through the process AND keeps team focused; manages inspection time contract; ensures work product entrance criteria is met; records defects and items to investigate; distributes inspection materials; tracks metrics

Author

Creator/maintainer of the work products being inspected; has a vested interest in ensuring all defects are found

12

Requirements Inspection Team Roles

ReaderRepresentative from the development team; during team meeting will paraphrase each statement in their own words to demonstrate sufficient understanding to carry out next stage of design.

Tester May be one or more persons responsible for test planning and/or testing; will consider how they would test the product - which paths, conditions, data, states, expected results, etc - and express as questions in team review.

13

Requirements Inspection Steps

1. Planning - Moderator conducts value-based assessment to engage; Moderator, Project Lead, and Author select persons for Reader and Tester roles; schedule training; schedule inspection

2. Individual Review - Author provides Moderator requirements and supporting documentation prior to team review. Moderator distributes needs and features, glossary, requirements, and diagrams. Inspectors independently review requirements and identify anything not perfectly clear, anything which seems wrong, or anything they disagree with (substance, not style)

3. Team Review - Walk through requirements; Moderator records defects, keeps track of time spent for individual and team review

14

Requirements Inspection Steps

4. Defect & Investigation Resolution - Author performs requirement remediation based upon defects recorded in defect log and updates log with appropriate responses (accept, reject, or defer). At completion of remediation, Author sends copy of defect log to Moderator with total remediation hours spent.

5. Validate & Exit - Author reviews defect log with Moderator and Project Lead; verifies no secondary defects exist; discusses rejected defects; determines if further inspections are needed.

15

Management’s Role

Do not participate in the Inspection Participate in planning and exit meetings Kept apprised of progress by Moderator Encouraged to attend Inspection training

16

Requirements Inspection Performance Indicators

Minimum of 4 Inspectors Requirements coverage runs 60 requirements per

hour (average) Maximum 2 hours for Inspection Team Review

National City experience has shown that effectiveness declines sharply after 2 hours

Typically, you hold two meetings to cover 240 requirements2 mtgs x (2 hr/mtg x 60 req/hr)

Preparation – if participants are not prepared, postpone Inspection and inform management

Effective inspections are HARD WORK!

17

Requirement Defects Found

0

50

100

150

200

250

300

350

400

450

500

Fre

qu

en

cy o

f D

efe

cts

0.0%

20.0%

40.0%

60.0%

80.0%

100.0%

120.0%

Cu

mu

lati

ve P

erc

en

tag

e

18

Author’s Response

0

200

400

600

800

1000

1200

Fre

qu

en

cy o

f A

cti

on

Taken

0.00%

20.00%

40.00%

60.00%

80.00%

100.00%

120.00%

Cu

mu

lati

ve %

19

Competency Improvement

0.00%

5.00%

10.00%

15.00%

20.00%

25.00%

30.00%

35.00%

Pe

rce

nt

of

Re

qu

ire

me

nts

Incomplete

Unclear

Unverifiable

Incorrect

Irrelevant

Investigations

Inconsistent

Infeasible

Defect type as percent of requirements

20

Requirements Inspection Process Assets

Inspector training deck Inspection checklist Moderator training deck Database – collect/manage and report metrics Inspection group mailbox Template letters for

Engagement request Denial Various meetings

Periodic meetings with Moderators

21

Summary

Process implementation High return on investment Low risk Short wait for payback of initial investment

Defects cost more the longer they exist

22