make a difference with evaluations
TRANSCRIPT
Make a difference with Evaluations
Daniël van der Gaag & Ewald Roodenrijs
Program
What are evaluations?
Why evaluations?
How to make a difference?
—Evaluations strategy
—Reading Techniques
Definition
Business Case
+ QA
Design
+ QA
Realisation
+ QATest
+ QA
Closing
+ QA
Time savedImpl.
+ QA
Focus of evaluations
Expected quality
Impl.
PhaseTest
Phase
Realisation
Phase
Design
Phase
Business
Case Phase
Closing
Phase
Quality gain by
preventive
measures
Quality gain by
detective
measures
Test & Repair
Quality gain by
corrective
measures
Moments of quality control
Test & Repair
Focus Evaluations
Reductie doorlooptijd 10%-30% [Gilb, Fagan]
Reductie ontwikkelkosten 15%-25% [Gilb, IBM, Fagan]
The difference when outsourcing
Impl.
PhaseTest
Phase
Realisation
Phase
Design
Phase
Business
Case Phase
Closing
Phase
?
Strategy of evaluations
Project/process
risks
Product risks
Functie
Document
Level 0 L2 / CMC L2 / MIS
SRS review review review
STD review review review
SDD review review review
Example: Strategie of evaluations
Function
Document type
Level 0 L2 / CMC L2 / MISRisk class
Software
Requirements Report
Intensive Medium Light A
Software Test
Description
Medium Light Intake B
Software Design
Description
Light Intake Intake C
Risc class A B C
Example
Evaluation class Time # part. # rounds Experience Combi
Intensive 12 m/p 4 4 Senior Inspection +
Desk check
Medium 8 m/p 2 3 Medior Desk check
Light 4 u 1 2 Junior Review
Intake 1 u 1 1 Junior Format check
Improve the evaluations proces
— Using a basis of evaluations increases the efficiency
— A reading technique helps in effectively judging(reading) of document
Possible reading techniques:
—Ad-hoc Reading
—Checklist-Based Reading (CBR)
—Perspective-Based Reading (PBR)
—Usage-Based Reading (UBR)
Ad-hoc Reading
The normal reading of documentation
Goal: finding errors
Unstructured
Simplest way of evaluations
Focus on everything!
Checklist-Based Reading
Based on the use of a predefined list of questions - a checklist
— Checklists are based on a number of specific questions with the objective of the profiler
trainees during the test process
More structure and support versus ad-hoc reading
Most accepted method of evaluations
— Because of simple design and repeatability
— Example: Checklist master test plan TMap NEXT®
Perspective-Based Reading
Starting point or Perspective becomes predominate
Each reviewer has a different perspective
— Use your own perspective (e.g. Test, Design, User)
— Own requirements = Own perspective
Reviewer assesses his own perspective of the document
58% of the defects is found
Defects are found in approx. 56 minutes
PBR
Questions/ scenarios
Perspective
Focus
B1 B2 B3 B4 B5 B6 B7
T1 X X X
T2 X X X X
T3 X X X
B1 B2 B3 B4 B5 B6 B7
T1 X X
T2 X X
T3 X X X
Ad-hoc reading
Perspective-Based Reading
Usage-Based Reading
Focus 'reading difficulty' on finding the most critical errors in the document
Goal: Finding the worst mistakes
— Read, read the document, use cases and requirements
— Errors found, starting with the "follow" the first case uses the document and continue with
the following ...
UBR
Lees- vs. Testontwerptechnieken
Reading techniques are for evaluations what test design techniques are for testing
Processing in the evaluations strategy
Assign reading techniques
Evaluation
method
Light coverage Medium
coverage
Intensive
coverage
Inspection CBR UBR PBR/CBR
Review Ad-hoc CBR CBR
PBR
Walkthrough Ad-hoc CBR
Ad-hoc
UBR
Desk Check CBR CBR PBR
Intake Ad-hoc CBR CBR
Attribute/
Document type
Risk Class Evaluation method Technique
Functionality
Functional Design A Inspectie PBR/CBR
Technical Design C Desk Check UBR
Definition Studie A Intake PBR
Usability
Working instructions B Walkthrough UBR/CBR
Maintainability
Control document A Inspectie UBR
Installation manual C Review CBR
Effective evaluations
Proper use of evaluation methods
Proper planning for evaluations
Use reading techniques
Risk per document type and/or system
component strategy
The end