machine learning in software testing
TRANSCRIPT
MACHINE LEARNING IN SOFTWARE TESTING
Mithun Kumar S R
IDENTIFY THE MOVIE
a machine can actually learn if we communicate with it
MACHINE LEARNING
Machine Learning is the study of computer algorithms that improve automatically through experience
- Tom Mitchell
Traditional Programming
Computer
Data
Program
Output
Computer
Data
Machine Learning
Output
Program
HOW THIS WORKS
Training
Data
Test Data
Learning
Machine
Analyzed
data for
prediction
SOFTWARE TEST LIFE CYCLE
Pre-execution
• Test planning
• Code Review
• Test case management
Execution
• Automated run
• Defect analysis
Post-execution
• Debugging
• Regression suite update
SOFTWARE TESTING
Critical task in Software development process
Overspend in time and resources
Automation limited to test execution
SUPERVISED LEARNING
http://www.astroml.org/sklearn_tutorial/general_concepts.html
UNSUPERVISED LEARNING
http://www.astroml.org/sklearn_tutorial/general_concepts.html
SOFTWARE TEST LIFE CYCLE
Pre-execution
• Test planning
• Code Review
• Test case management
Execution
• Automated run
• Defect analysis
Post-execution
• Debugging
• Regression suite update
SOFTWARE TEST ACTIVITIES AND ML
Software defect prediction
Test Planning
Test case management
Debugging
BAYESIAN ALGORITHM FOR SOFTWARE DEFECT PREDICTION
CLASSIFICATION
https://alliance.seas.upenn.edu/~cis520/wiki/index.php?n=Lectures.Classification
NAÏVE BAYES ALGO
Branch Count LOC Defective
5 15 No
3 5 No
9 20 No
15 40 Yes
16 35 Yes
Branch Count = 16 LOC = 39
C = No -> 0.000000912
C = Yes -> 0.0181Leandru Minku: Automated Software Defect Prediction Using Machine Learning
LINEAR REGRESSION – DEFECT DENSITY
http://openclassroom.stanford.edu/MainFolder/DocumentPage.php?course=MachineLearning&doc=exercises/ex2/ex2.html
LOC
Defe
ct D
ensity
TEST PLANNING
Database formation
Data collection
Classification of software
Analyzing the results
Test Cost prediction
Thomas J. Cheatham, Jungsoon P. Yoo, and Nancy J. Wahl. Software testing: a machine learning experiment.
Complexity
Cost
MELBA – MACHINE LEARNING BASED REFINEMENT OF BLACKBOX TEST SPECIFICATION
Lionel C. Briand. Novel applications of machine learning in software testing. Quality Software, International Conference on, 0:3–10,
2008.
AREAS OF APPLICATION
Machine Learning-based Software Testing: Towards a Classification Framework Mahdi Noorian1, Ebrahim Bagheri1,2, and Wheichang Du1
CHALLENGES
Past data availability
Predictable pattern
STEPS FORWARD
Black Box techniques
Finding the right patterns
Algorithm analysis for different types of test activity
Crowdsourcing
