![Page 1: SPIN 23 February 2006 Norman Fenton Agena Ltd and Queen Mary University of London](https://reader036.vdocuments.mx/reader036/viewer/2022062410/568156f1550346895dc495c3/html5/thumbnails/1.jpg)
Slide 1
SPIN23 February 2006
Norman Fenton
Agena Ltd and
Queen Mary University of London
Improved Software Defect Prediction
![Page 2: SPIN 23 February 2006 Norman Fenton Agena Ltd and Queen Mary University of London](https://reader036.vdocuments.mx/reader036/viewer/2022062410/568156f1550346895dc495c3/html5/thumbnails/2.jpg)
Slide 2
delivery
Pre-release testing Post-release operation
Using fault data to predict reliability
![Page 3: SPIN 23 February 2006 Norman Fenton Agena Ltd and Queen Mary University of London](https://reader036.vdocuments.mx/reader036/viewer/2022062410/568156f1550346895dc495c3/html5/thumbnails/3.jpg)
Slide 3
Pre-release vs post-release faults?
0
0
Post-release faults
10
20
30
40 80 120 160
Pre-release faults
?
![Page 4: SPIN 23 February 2006 Norman Fenton Agena Ltd and Queen Mary University of London](https://reader036.vdocuments.mx/reader036/viewer/2022062410/568156f1550346895dc495c3/html5/thumbnails/4.jpg)
Slide 4
Pre-release vs post-release faults: actual
Post-release faults
0
10
20
30
0 40 80 120 160
Pre-release faults
![Page 5: SPIN 23 February 2006 Norman Fenton Agena Ltd and Queen Mary University of London](https://reader036.vdocuments.mx/reader036/viewer/2022062410/568156f1550346895dc495c3/html5/thumbnails/5.jpg)
Slide 5
Software metrics….?
![Page 6: SPIN 23 February 2006 Norman Fenton Agena Ltd and Queen Mary University of London](https://reader036.vdocuments.mx/reader036/viewer/2022062410/568156f1550346895dc495c3/html5/thumbnails/6.jpg)
Slide 6
Regression models….?
![Page 7: SPIN 23 February 2006 Norman Fenton Agena Ltd and Queen Mary University of London](https://reader036.vdocuments.mx/reader036/viewer/2022062410/568156f1550346895dc495c3/html5/thumbnails/7.jpg)
Slide 7
Solution: causal models (risk maps)
![Page 8: SPIN 23 February 2006 Norman Fenton Agena Ltd and Queen Mary University of London](https://reader036.vdocuments.mx/reader036/viewer/2022062410/568156f1550346895dc495c3/html5/thumbnails/8.jpg)
Slide 8
What’s special about this approach?
• Structured
• Visual
• Robust
![Page 9: SPIN 23 February 2006 Norman Fenton Agena Ltd and Queen Mary University of London](https://reader036.vdocuments.mx/reader036/viewer/2022062410/568156f1550346895dc495c3/html5/thumbnails/9.jpg)
Slide 9
The specific problem for Philips
time
Defectsfound
predicted
actual
![Page 10: SPIN 23 February 2006 Norman Fenton Agena Ltd and Queen Mary University of London](https://reader036.vdocuments.mx/reader036/viewer/2022062410/568156f1550346895dc495c3/html5/thumbnails/10.jpg)
Slide 10
Background to work with Philips
Fixed Risk Map (AID)
2000
MODIST
Agena Risk
2001 20032002 2004
Reliability models
![Page 11: SPIN 23 February 2006 Norman Fenton Agena Ltd and Queen Mary University of London](https://reader036.vdocuments.mx/reader036/viewer/2022062410/568156f1550346895dc495c3/html5/thumbnails/11.jpg)
Slide 11
Projects in the trial
MM&C (9)28%
TV (16)50%
DVD (7)22%
(actual number of projects shown in brackets)
![Page 12: SPIN 23 February 2006 Norman Fenton Agena Ltd and Queen Mary University of London](https://reader036.vdocuments.mx/reader036/viewer/2022062410/568156f1550346895dc495c3/html5/thumbnails/12.jpg)
Slide 12
Factors used at Philips
• Existing code base…
• Complexity and management of new requirements ...
• Development, testing, rework, process …
• Overall project management …
![Page 13: SPIN 23 February 2006 Norman Fenton Agena Ltd and Queen Mary University of London](https://reader036.vdocuments.mx/reader036/viewer/2022062410/568156f1550346895dc495c3/html5/thumbnails/13.jpg)
Slide 13
Actual versus predicted defects
0
200
400
600
800
1000
1200
1400
1600
1800
2000
0 500 1000 1500 2000 2500
Actual defects
pre
dic
ted
def
ects
Correlationcoefficient
95%
![Page 14: SPIN 23 February 2006 Norman Fenton Agena Ltd and Queen Mary University of London](https://reader036.vdocuments.mx/reader036/viewer/2022062410/568156f1550346895dc495c3/html5/thumbnails/14.jpg)
Slide 14
Validation Summary (Philips’ words)
“Bayesian Network approach is innovative and one of the most promising techniques for predicting defects in software projects”
“Our evaluation showed excellent results for project sizes between 10 and 90 KLOC”
“Initially, projects outside this range were not within the scope of the default model so predictions were less accurate, but accuracy was significantly improved after standard model tailoring”
“AgenaRisk is a valuable tool as is”
![Page 15: SPIN 23 February 2006 Norman Fenton Agena Ltd and Queen Mary University of London](https://reader036.vdocuments.mx/reader036/viewer/2022062410/568156f1550346895dc495c3/html5/thumbnails/15.jpg)
Slide 15
Specific benefits
• Accurate predictions of defects at different phases
• Know when to stop testing
• Identify where to allocate testing
• Minimise the cost of rework
• Highlight areas for improvement
• Use out of the box now if you have no data
• Approach fully customisable
• Models arbitrary project life-cycles
![Page 16: SPIN 23 February 2006 Norman Fenton Agena Ltd and Queen Mary University of London](https://reader036.vdocuments.mx/reader036/viewer/2022062410/568156f1550346895dc495c3/html5/thumbnails/16.jpg)
Slide 16
Summary
• Risk maps – the way forward
• Validation results excellent
![Page 17: SPIN 23 February 2006 Norman Fenton Agena Ltd and Queen Mary University of London](https://reader036.vdocuments.mx/reader036/viewer/2022062410/568156f1550346895dc495c3/html5/thumbnails/17.jpg)
Slide 17
…And
You can use the technology NOW
www.agenarisk.com