machine learning 101

Machine Learning 101Setu ChokshiCommunity Technology Update 2016

Machine Learning 101

This is an introductory session and we are here to learnFeel free to ask questions at any time

Why?Discover reason behind success, failureUnderstand customers, productsPlan futureExperiment meaningfullyImprove performance

Run on analytics

Data science

Scientific method of reasoning applied to data-driven decisions

Hypothesis, experiments, facts, logical reasoning+ data engineering.

Data wrangling (munging), retrieval

+ storage

Data mining & machine learning

Statistics

Big data

Data scienc

e

Machine learning ≣ data mining

Exploresdata

Finds patterns

Predicts (scoring)

Means strictly equivalent to. Yes.

Tools

Tools & salaries

Chart from "2016 Data Science Salary Survey"

Examples of Machine Learning

How does machine learning help?There are only 5 questions that machine learning can help answer

Source: Data Science For Beginners - 5 Questions Data Science Answers by Brandon Rohrer

1. Is this A or B?

Is this A or B?Classification Algorithms

2. Is this Weird?

Is this Weid?Anomaly detection algorithms

3. How much? How many?How many?How much?

Regression algorithms

4. How is this organized?How is this organized?

Clustering algorithms

5. What should I do now?What should I do

now?

Reinforcement learning algorithms

Machine learning process and algorithms

How does it workAlgorithm

Your data

Computer

Your answer

=

=

=

=

Recipe

Ingredients

Blender

Smoothie

1. Define & initialise a model2. Train model (process cases)3. Validate model

…by scoring (making predictions) a test data set and evaluating the results

4. Use it: Explore or Deploy…visualise and study…deploy as a (web) service

5. Update and revalidate

How?

Cheat Sheet

http://download.microsoft.com/download/A/6/1/A613E11E-8F9C-424A-B99D-65344785C288/microsoft-machine-learning-algorithm-cheat-sheet-v6.pdf

Classification

21

ClassificationThe classification model can be implemented in several ways• Decision trees• Rules• Mathematical formulae

Lets build some intuition for decision treeshttp://www.r2d3.us/visual-intro-to-machine-learning-part-1/

Support Vector MachineDraw a line/plane to separate the variables

Solution….

…now on to Neural Networks

http://playground.tensorflow.org/

Anomaly Detection

27

Anomaly DetectionThe different types of anomaly detection schemes• Statistical based• Distance based• Model based

28

Lets build some intuition

Second Attempt

®

99.9%-ile

30

Exampleshttp://anomalydetection-aml.azurewebsites.net/

Is my algorithm any good?

34

Accuracy is not enough

35

2 metrics to rememberFALSE NEGATIVE TRUE NEGATIVE

TRUE POSITIVE

FALSEPOSITIVE

Relevant Elements

How many selected items are relevant = PRECISION

How many relevant items are Selected = RECALLSELECTED ELEMENTS

4 Step process to mastery

Step 1: Take a courseMachine Learning: Andrew NgMining Massive Datasets: Leskovec, Rajaraman, UllmanDeep learning at Oxford 2015 http://bit.ly/2ccQmnXNeural Networks for Machine Learning: G Hinton

Step 2: Get a book

Deep Learning: http://www.deeplearningbook.org/

Step 3: Do a project on KaggleBinary Classification: Titanic: Machine Learning from DisasterMulti-Class Classification: Forest Cover Type PredictionRegression with temporal component: Bike Sharing DemandBinary Classification with text data: Random Acts of PizzaSentiment Analysis: Sentiment Analysis on Movie ReviewsAudio/Video: Challenges in Representation Learning: Facial Expression Recognition ChallengeImage: The Marinexplore and Cornell University Whale Detection Challenge

Step 4: Keep yourself upto datePodcasts: Linear Digressions & Talking MachinesCheat sheets (Python/R/ML etc): http://bit.ly/2ccOQlu

Other resources:Interesting iPython Notebooks: http://bit.ly/2ccPEGZLearn Data Science http://learnds.com/A Few Useful Things to Know about Machine Learning http://bit.ly/2ccQNi5

Keep LearningYou can reach me on

Email: setu.chokshi at gmail Twitter: @setuc

Best Speaker Award from Soyoung Lee

Thank you!