@vinaya1980 @hrishikarekar

TESTING IN THE AGE OF MACHINE LEARNING

March 2016

https://www.youtube.com/watch?v=DCgHsxISE0Q

W H E R E A R E W E H E A D E D

httpsweforum.org/agenda/2017/01/worried-about-ai-taking-your-job-its-already-happening-in-japan?utm_content=buffera14af&utm_medium=social&utm_source=twitter.com&utm_campaign=buffer

THE WORLD OF ARTIFICIAL INTELLIGENCE

NLP

Neural Networks

MACHINE LEARNING

DEEP LEARNING

…..

THE FUTURE IS ALREADY HERE

Google RankBrain

Assistants

HOW DOES IT WORK

Supervised

• Labelled data• Given new

data, predict outcome

• Classification

Unsupervised

• No labels• Find hidden

structures• Clustering

Reinforcement

• Decision process

• Actions are rewarded or punished

• Learns to optimize rewards

http://scikit-learn.org/

LEARNING FROM DATA

SUPERVISED LEARNING - CLASSIFCATION

http://www.mikedeff.in/MLIntro.PNG

INTO THE REALM OF PROBABILITIES

Y = f ( x ) Y ≈ f ( x )

What is scrum ?

{ "Prediction": { "details": { "Algorithm": "SGD", "PredictiveModelType": "MULTICLASS" }, "predictedLabel": "definition", "predictedScores": { "advantages": 0.0001860455668065697, "characteristics": 0.00006915141420904547, "compare": 0.00017757616296876222, "definition": 0.9970965385437012, "disadvantages": 0.0000534967657586094, } }}

Can you tell me about scrum ?

{ "Prediction": { "details": { "Algorithm": "SGD", "PredictiveModelType": "MULTICLASS" }, "predictedLabel": "definition", "predictedScores": { "advantages": 0.01977257989346981, "characteristics": 0.022757112979888916, "compare": 0.008386141620576382, "definition": 0.21092116832733154, "disadvantages": 0.04002799838781357 } }}

TOLERANCE LEVELS

Y ≈ f ( x )

Know the probability that is within acceptable limits

EVALUATE WITH DIFFERENT MODELS

Evaluate against a set of algorithms to iterate towards a model that’s closest representation and for further tuning

https://s3.amazonaws.com/MLMastery/MachineLearningAlgorithms.png?__s=h4reg8jqwyg4sz3bzdqf

EVALUATION – DATA SET APPROACHES

Random split• 70% train, 30% test

K-fold cross validation

Split into 3 datasets• #1 Train on 1 and 2, test on 3• #2 Train on 2 and 3, test on 1• #3 Train on 1 and 3, test on 2

Never use the same dataset for training and evaluating

ITERATIVE – LEARNING PROCESS

Be prepared to throw the model and start again

MODEL ACCURACY - CONFUSION MATRIX

Strive for better models, not 100% accurate.

OVERFITTING

MODEL IS AS GOOD AS THE TRAINING DATA

If all of the algorithms perform poorly,• it maybe worth considering if there is a lack of learning

structure in the data set

• some transformation needed to make the structure more learnable • remove unnecessary noise - stop words are typically

removed because they cause unnecessary noise)

SUMMARY

Machine Learning applications demand a shift in testing approach

• Use objective acceptance levels to evaluate the application

• Express test outcomes in statistical terms

• Have a high level understanding of the underlying working of the application

@vinaya1980 @hrishikarekar

THANK YOU