e software quality - university college londoncrest.cs.ucl.ac.uk/cow/15/slides/stefanwagner.pdf ·...
TRANSCRIPT
![Page 1: e Software Quality - University College Londoncrest.cs.ucl.ac.uk/cow/15/slides/StefanWagner.pdf · NASA1 NASA2 NASA3 NASA4 Tomcat 70,0 12,1 24,8 21,7 6,0 6,0 36,1 19,4 19,2 15,9 Average](https://reader033.vdocuments.mx/reader033/viewer/2022053023/605784fa57fbe2645f03c95b/html5/thumbnails/1.jpg)
www.uni-stuttgart.de
PredictionSoftware Quality
Stefan Wagner
The 15th CREST Open Workshop25 October 2011
London, UK
![Page 2: e Software Quality - University College Londoncrest.cs.ucl.ac.uk/cow/15/slides/StefanWagner.pdf · NASA1 NASA2 NASA3 NASA4 Tomcat 70,0 12,1 24,8 21,7 6,0 6,0 36,1 19,4 19,2 15,9 Average](https://reader033.vdocuments.mx/reader033/viewer/2022053023/605784fa57fbe2645f03c95b/html5/thumbnails/2.jpg)
"Quality is a complex and multi-faceted concept... it is also the source of great confusion."
–David A. Garvin
![Page 3: e Software Quality - University College Londoncrest.cs.ucl.ac.uk/cow/15/slides/StefanWagner.pdf · NASA1 NASA2 NASA3 NASA4 Tomcat 70,0 12,1 24,8 21,7 6,0 6,0 36,1 19,4 19,2 15,9 Average](https://reader033.vdocuments.mx/reader033/viewer/2022053023/605784fa57fbe2645f03c95b/html5/thumbnails/3.jpg)
I know it when I see it
![Page 4: e Software Quality - University College Londoncrest.cs.ucl.ac.uk/cow/15/slides/StefanWagner.pdf · NASA1 NASA2 NASA3 NASA4 Tomcat 70,0 12,1 24,8 21,7 6,0 6,0 36,1 19,4 19,2 15,9 Average](https://reader033.vdocuments.mx/reader033/viewer/2022053023/605784fa57fbe2645f03c95b/html5/thumbnails/4.jpg)
Quality Model
Functionality
Reliability
Performance
Usability
Portability
Maintainability
ISO 9126
![Page 5: e Software Quality - University College Londoncrest.cs.ucl.ac.uk/cow/15/slides/StefanWagner.pdf · NASA1 NASA2 NASA3 NASA4 Tomcat 70,0 12,1 24,8 21,7 6,0 6,0 36,1 19,4 19,2 15,9 Average](https://reader033.vdocuments.mx/reader033/viewer/2022053023/605784fa57fbe2645f03c95b/html5/thumbnails/5.jpg)
"By the time you figure out you have a quality problem it is probably too late to fix it."
–John S. Reel
![Page 6: e Software Quality - University College Londoncrest.cs.ucl.ac.uk/cow/15/slides/StefanWagner.pdf · NASA1 NASA2 NASA3 NASA4 Tomcat 70,0 12,1 24,8 21,7 6,0 6,0 36,1 19,4 19,2 15,9 Average](https://reader033.vdocuments.mx/reader033/viewer/2022053023/605784fa57fbe2645f03c95b/html5/thumbnails/6.jpg)
Development
Quality assurance
Re
vie
w
Te
st
An
alysi
s
Softwarequality control
Continuous
Deissenboeck, Wagner et al., IEEE Software, 2008
Evaluation
Specification
Qualitymodel
Qualityrequirements
Change requests
![Page 7: e Software Quality - University College Londoncrest.cs.ucl.ac.uk/cow/15/slides/StefanWagner.pdf · NASA1 NASA2 NASA3 NASA4 Tomcat 70,0 12,1 24,8 21,7 6,0 6,0 36,1 19,4 19,2 15,9 Average](https://reader033.vdocuments.mx/reader033/viewer/2022053023/605784fa57fbe2645f03c95b/html5/thumbnails/7.jpg)
Analysis TestModification
MaintenancePr
oduc
t
Comment
Function
Module
Activity
-based
quality m
odel
Deissenboeck, Wagner et al., ICSM'07
Activities
Entities
Impacts
![Page 8: e Software Quality - University College Londoncrest.cs.ucl.ac.uk/cow/15/slides/StefanWagner.pdf · NASA1 NASA2 NASA3 NASA4 Tomcat 70,0 12,1 24,8 21,7 6,0 6,0 36,1 19,4 19,2 15,9 Average](https://reader033.vdocuments.mx/reader033/viewer/2022053023/605784fa57fbe2645f03c95b/html5/thumbnails/8.jpg)
How can we assess and predictquality?
![Page 9: e Software Quality - University College Londoncrest.cs.ucl.ac.uk/cow/15/slides/StefanWagner.pdf · NASA1 NASA2 NASA3 NASA4 Tomcat 70,0 12,1 24,8 21,7 6,0 6,0 36,1 19,4 19,2 15,9 Average](https://reader033.vdocuments.mx/reader033/viewer/2022053023/605784fa57fbe2645f03c95b/html5/thumbnails/9.jpg)
Prediction research concentrates on bug prediction...
![Page 10: e Software Quality - University College Londoncrest.cs.ucl.ac.uk/cow/15/slides/StefanWagner.pdf · NASA1 NASA2 NASA3 NASA4 Tomcat 70,0 12,1 24,8 21,7 6,0 6,0 36,1 19,4 19,2 15,9 Average](https://reader033.vdocuments.mx/reader033/viewer/2022053023/605784fa57fbe2645f03c95b/html5/thumbnails/10.jpg)
1. Scoring approach
![Page 11: e Software Quality - University College Londoncrest.cs.ucl.ac.uk/cow/15/slides/StefanWagner.pdf · NASA1 NASA2 NASA3 NASA4 Tomcat 70,0 12,1 24,8 21,7 6,0 6,0 36,1 19,4 19,2 15,9 Average](https://reader033.vdocuments.mx/reader033/viewer/2022053023/605784fa57fbe2645f03c95b/html5/thumbnails/11.jpg)
The Benchmark for Software Quality
![Page 12: e Software Quality - University College Londoncrest.cs.ucl.ac.uk/cow/15/slides/StefanWagner.pdf · NASA1 NASA2 NASA3 NASA4 Tomcat 70,0 12,1 24,8 21,7 6,0 6,0 36,1 19,4 19,2 15,9 Average](https://reader033.vdocuments.mx/reader033/viewer/2022053023/605784fa57fbe2645f03c95b/html5/thumbnails/12.jpg)
Project partners
![Page 13: e Software Quality - University College Londoncrest.cs.ucl.ac.uk/cow/15/slides/StefanWagner.pdf · NASA1 NASA2 NASA3 NASA4 Tomcat 70,0 12,1 24,8 21,7 6,0 6,0 36,1 19,4 19,2 15,9 Average](https://reader033.vdocuments.mx/reader033/viewer/2022053023/605784fa57fbe2645f03c95b/html5/thumbnails/13.jpg)
Format string injection
Variable manipulation
Embedding scripts
AttackPr
oduc
t
Cookie
Dynamic Web page
Static field
![Page 14: e Software Quality - University College Londoncrest.cs.ucl.ac.uk/cow/15/slides/StefanWagner.pdf · NASA1 NASA2 NASA3 NASA4 Tomcat 70,0 12,1 24,8 21,7 6,0 6,0 36,1 19,4 19,2 15,9 Average](https://reader033.vdocuments.mx/reader033/viewer/2022053023/605784fa57fbe2645f03c95b/html5/thumbnails/14.jpg)
Format string injection
Variable manipulation
Embedding scripts
Attack
CookieDynamic Web
page Static field
Sanitation Sanitation Immutability
![Page 15: e Software Quality - University College Londoncrest.cs.ucl.ac.uk/cow/15/slides/StefanWagner.pdf · NASA1 NASA2 NASA3 NASA4 Tomcat 70,0 12,1 24,8 21,7 6,0 6,0 36,1 19,4 19,2 15,9 Average](https://reader033.vdocuments.mx/reader033/viewer/2022053023/605784fa57fbe2645f03c95b/html5/thumbnails/15.jpg)
Format string injection
Variable manipulation
Embedding scripts
Attack
CookieDynamic Web
page Static field
Sanitation Sanitation Immutability
HTTP cookie formed from untrusted input
![Page 16: e Software Quality - University College Londoncrest.cs.ucl.ac.uk/cow/15/slides/StefanWagner.pdf · NASA1 NASA2 NASA3 NASA4 Tomcat 70,0 12,1 24,8 21,7 6,0 6,0 36,1 19,4 19,2 15,9 Average](https://reader033.vdocuments.mx/reader033/viewer/2022053023/605784fa57fbe2645f03c95b/html5/thumbnails/16.jpg)
Quality Impact Evaluation Specification Language (QIESL)
result = distributeRatio(100, %%Missing destructor%% /%%#Classes%%);
• Java-based syntax• Access to factors and measures• Helper functions• Aggregation, evaluation, calibration
![Page 17: e Software Quality - University College Londoncrest.cs.ucl.ac.uk/cow/15/slides/StefanWagner.pdf · NASA1 NASA2 NASA3 NASA4 Tomcat 70,0 12,1 24,8 21,7 6,0 6,0 36,1 19,4 19,2 15,9 Average](https://reader033.vdocuments.mx/reader033/viewer/2022053023/605784fa57fbe2645f03c95b/html5/thumbnails/17.jpg)
Format string injection
Variable manipulation
Embedding scripts
Attack
CookieDynamic Web
page Static field
Sanitation Sanitation Immutability
HTTP cookie formed from untrusted input
QIESL:result = distributeRatio(
100, %%HTTP cookie formed from untrusted input%% /%%Cookie creations%%);
QIESL:result = 100 - %%Sanitation@Cookie%%;
![Page 18: e Software Quality - University College Londoncrest.cs.ucl.ac.uk/cow/15/slides/StefanWagner.pdf · NASA1 NASA2 NASA3 NASA4 Tomcat 70,0 12,1 24,8 21,7 6,0 6,0 36,1 19,4 19,2 15,9 Average](https://reader033.vdocuments.mx/reader033/viewer/2022053023/605784fa57fbe2645f03c95b/html5/thumbnails/18.jpg)
Format string injection
Variable manipulation
Embedding scripts
Attack
CookieDynamic Web
page Static field
Sanitation Sanitation Immutability
HTTP cookie formed from untrusted input 5 findings
75 points
25 points
![Page 19: e Software Quality - University College Londoncrest.cs.ucl.ac.uk/cow/15/slides/StefanWagner.pdf · NASA1 NASA2 NASA3 NASA4 Tomcat 70,0 12,1 24,8 21,7 6,0 6,0 36,1 19,4 19,2 15,9 Average](https://reader033.vdocuments.mx/reader033/viewer/2022053023/605784fa57fbe2645f03c95b/html5/thumbnails/19.jpg)
Validation on OSS projects
JabRef
TV-Browser
RSSOwl
Log4J
Checkstyle
Rankingfrom model
Rankingfrom experts
Best
Worst JabRef
TV-BrowserRSSOwl
Log4J
Checkstyle
![Page 20: e Software Quality - University College Londoncrest.cs.ucl.ac.uk/cow/15/slides/StefanWagner.pdf · NASA1 NASA2 NASA3 NASA4 Tomcat 70,0 12,1 24,8 21,7 6,0 6,0 36,1 19,4 19,2 15,9 Average](https://reader033.vdocuments.mx/reader033/viewer/2022053023/605784fa57fbe2645f03c95b/html5/thumbnails/20.jpg)
2. Bayesian net
![Page 21: e Software Quality - University College Londoncrest.cs.ucl.ac.uk/cow/15/slides/StefanWagner.pdf · NASA1 NASA2 NASA3 NASA4 Tomcat 70,0 12,1 24,8 21,7 6,0 6,0 36,1 19,4 19,2 15,9 Average](https://reader033.vdocuments.mx/reader033/viewer/2022053023/605784fa57fbe2645f03c95b/html5/thumbnails/21.jpg)
Bayesian net example
# fieldfailures
Low = 0.6High = 0.4
Low HighSmall 0.7 0.1Med 0.2 0.2Large 0.1 0.7
Test effortCode
complexity
Node Probability Table
![Page 22: e Software Quality - University College Londoncrest.cs.ucl.ac.uk/cow/15/slides/StefanWagner.pdf · NASA1 NASA2 NASA3 NASA4 Tomcat 70,0 12,1 24,8 21,7 6,0 6,0 36,1 19,4 19,2 15,9 Average](https://reader033.vdocuments.mx/reader033/viewer/2022053023/605784fa57fbe2645f03c95b/html5/thumbnails/22.jpg)
Format string injection
Variable manipulation
Embedding scripts
AttackPr
oduc
t
Cookie
Dynamic Web page
Static field
![Page 23: e Software Quality - University College Londoncrest.cs.ucl.ac.uk/cow/15/slides/StefanWagner.pdf · NASA1 NASA2 NASA3 NASA4 Tomcat 70,0 12,1 24,8 21,7 6,0 6,0 36,1 19,4 19,2 15,9 Average](https://reader033.vdocuments.mx/reader033/viewer/2022053023/605784fa57fbe2645f03c95b/html5/thumbnails/23.jpg)
Attack
Injection
Format string injection
Resource manipulation
Embedding scripts
Variable Manipulation
Sanitation of dynamic web page
Locality of field
Sanitation of cookie
Vulnerability
density
COS density
DWS density
FDL density
![Page 24: e Software Quality - University College Londoncrest.cs.ucl.ac.uk/cow/15/slides/StefanWagner.pdf · NASA1 NASA2 NASA3 NASA4 Tomcat 70,0 12,1 24,8 21,7 6,0 6,0 36,1 19,4 19,2 15,9 Average](https://reader033.vdocuments.mx/reader033/viewer/2022053023/605784fa57fbe2645f03c95b/html5/thumbnails/24.jpg)
Distribution
Measurement
AgenaRisk: http://www.agenarisk.com
![Page 25: e Software Quality - University College Londoncrest.cs.ucl.ac.uk/cow/15/slides/StefanWagner.pdf · NASA1 NASA2 NASA3 NASA4 Tomcat 70,0 12,1 24,8 21,7 6,0 6,0 36,1 19,4 19,2 15,9 Average](https://reader033.vdocuments.mx/reader033/viewer/2022053023/605784fa57fbe2645f03c95b/html5/thumbnails/25.jpg)
Validation
Goals: gather experiences test predictive validity
Maintainability Security
Wagner, Information and Software Technology, 2010
Tomcat7 – 43 KLOC3 – 6 years
300 KLOC2.5 years
![Page 26: e Software Quality - University College Londoncrest.cs.ucl.ac.uk/cow/15/slides/StefanWagner.pdf · NASA1 NASA2 NASA3 NASA4 Tomcat 70,0 12,1 24,8 21,7 6,0 6,0 36,1 19,4 19,2 15,9 Average](https://reader033.vdocuments.mx/reader033/viewer/2022053023/605784fa57fbe2645f03c95b/html5/thumbnails/26.jpg)
PredictiveValidity
NASA1 NASA2 NASA3 NASA4 Tomcat
70,0
12,1
24,821,7
6,0 6,0
36,1
19,219,415,9
Average change effort in person hours
PredictionObservation
Vulnerabilities per MLOC
![Page 27: e Software Quality - University College Londoncrest.cs.ucl.ac.uk/cow/15/slides/StefanWagner.pdf · NASA1 NASA2 NASA3 NASA4 Tomcat 70,0 12,1 24,8 21,7 6,0 6,0 36,1 19,4 19,2 15,9 Average](https://reader033.vdocuments.mx/reader033/viewer/2022053023/605784fa57fbe2645f03c95b/html5/thumbnails/27.jpg)
Conclusions
• Attempts to assess and predict a broader notion of quality
• Simple scoring approach• Bayesian net approach• Problems
– Missing measures for quality attributes– Missing independent quality assessments for comparisons– Missing data– Aggregation and weighting