a framework for adoption of machine learning in industry for software defect prediction
Post on 15-Aug-2015
116 Views
Preview:
TRANSCRIPT
Rakesh Rana1, Miroslaw Staron1, Jörgen Hansson1, Martin Nilsson2, Wilhelm Meding3
1Computer Science & Engineering, Chalmers | University of Gothenburg, Sweden
2Volvo Car Group, Gothenburg, Sweden
3Ericsson, Gothenburg, Sweden
rakesh.rana@gu.se
Software Defect Prediction (SDP) methods
Image 1: https://www.reliablesoft.net/how-to-become-an-expert-in-your-niche-even-if-you-are-not/
Image 2: Fenton, Norman, et al. "Predicting software defects in varying development lifecycles using Bayesian nets." Information and Software Technology 49.1 (2007): 32-43.
Image 3: Kan, Stephen H. Metrics and models in software quality engineering. Addison-Wesley Longman Publishing Co., Inc., 2002.
Image 4: http://www.codeodor.com/index.cfm/2009/11/12/Its-Not-Your-Fault-Your-Software-Sucks/3058
SDP: Methods based on Machine Learning
• Decision Trees (DTs)
• Support Vector Machines (SVMs)
• Artificial Neural Networks (ANNs)
• Bayesian Belief Networks (BNNs)
Image 1: http://www.webpages.uidaho.edu/veg_measure/Modules/Lessons/Module%202(Sampling)/2_3_Accuracy_and_bias.htm
Image 2: http://www.business2community.com/marketing/3-easy-keyword-research-tips-for-inbound-marketing-success-0215660#!bKz0G0
Image 3: http://dpss.co.riverside.ca.us/childrens-services-division/adoption-information/foster-adoptive-parent
Image 4: http://www.haitian-truth.org/treatys-tighter-adoption-rules-kick-in-for-haiti/
Objective
“How can we use the technology acceptance and
adoption models for developing framework for ML
adoption in industry and how to adapt it for software
defect prediction?”
Davis Jr, F.D., 1986. A technology acceptance model for empirically testing new end-user information systems: Theory and results. Massachusetts Institute of Technology.
Research Process
Framework for Adoption of ML Techniques in Industry
Framework for Adoption of ML Techniques in Industry
H1: Higher levels of perceived benefits of adopting ML techniques will strongly
(and positively) affect the likelihood of their adoption.
H2: Higher levels of perceived barriers of adopting ML techniques will
strongly (and negatively) affect the likelihood of their adoption.
…
ML characteristics that affects its acceptance for SDP
Organizational characteristics that affects its
acceptance for defect prediction task
ML Adoption Framework: How it can be used?
Attribute Tool A Tool B
Predictive Accuracy 85% 82%
Auto data acquisition Yes Yes
Report generation Yes, web basedYes, multiple
format
Can handle multiple projects Yes Yes
Generate causal mapsYes, Non-
InteractiveYes, Interactive
Running time (typical project) 30min 40min
Cost of license (tool)$ 20000/
license$ 35000/ license
Maintenance cost (estimate) $ 7000 pa $ 9000 pa
…
To compare between two new tools/services
AttributeExisting
Method
New ML based
technique
Predictive Accuracy Good Very Good
Auto data acquisition Yes Yes
Report generationYes, word
documentYes, web based
Can handle multiple projects No Yes
Generate causal maps No Yes
Running time (typical
project)15min 30min
Cost of license (tool) None $ 20000/ license
Maintenance cost (estimate) $ 2000 pa $ 7000 pa
…
To evaluate new technique for SDP
ML Adoption Framework: Conclusions
Large and constantly growing amount of data is available
ML based techniques have high potential to aid companies in SDP efforts
ML adoption framework help increase our understanding of factors and
attributes relevant for industrial practitioners
ML adoption framework will be useful for
Companies,
Researchers, and
Tool vendors
ML Adoption Framework: Initial Industrial Validation
“The adoption of machine learning techniques for software
defect prediction: An initial industrial validation”
In Proceedings of “Joint Conference on Knowledge-Based Software Engineering”
top related