building data driven applications with machine learning
TRANSCRIPT
![Page 1: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/1.jpg)
Building a Data Driven Product with Machine Learning
Kadriye Doğan, Yalçın Yenigün
26.01.2017
![Page 2: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/2.jpg)
Agenda
![Page 3: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/3.jpg)
Agenda1. What is Data-Driven Product?
a) Introductionb) Examples
2. Machine Learninga) Term Definitionsb) A Visual Examplec) Supervised Learningd) Unsupervised Learninge) Cross Validationf) Feature Extraction
3. Case Study
![Page 4: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/4.jpg)
What isData Driven Product?
![Page 5: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/5.jpg)
Data Driven Product• Data driven is the future!!!• It’s the ‘right’ way of doing things!!!..etc.
• What is “data-driven” ??• Is Facebook a data-driven product??• Is Uber a data-driven product??
• We can say that “all” of these are data-driven products• All of them works with data.• But they are really data-driven products??
![Page 6: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/6.jpg)
Data Driven Product• Experimentation:
• Data-Driven: Making design decisions based on behavioral evidence from users.
• Example: Picking a green button for your website because conversion metrics are significantly improved over the purple button
![Page 7: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/7.jpg)
Data Driven Product• Machine Learning : Building systems that
learn from behavioral data generated by users
• Examples:• Recommendation• Personalized Ranking• People-you-may-know• Products-you-may-like
![Page 8: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/8.jpg)
Data Driven Product• Databases or APIs
• They just use the data• To them their system is also data-driven.• But they are NOT data-driven.• They don’t use behavioral data generated
by users.
![Page 9: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/9.jpg)
Examples• A mobile app that gives information about public transport around you.
• Pulls data from transport operator or APIs, merges and gives you.• Nothing really data-driven.
• Data-driven version of this app:• Learn what part of the transport network relevant to you.• Predict when cycling is better when walking is better.• Predict waiting times.• Predict delays of transports.
![Page 10: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/10.jpg)
Examples• A website that provides blogging services to users
• Write posts, subscribe other posts.. etc.
• Data-driven version of this blog:• Recommend who to follow based on your previous likes• Auto-tag your content to allow people quickly find it• Create relevance-sorted feed of posts.
![Page 11: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/11.jpg)
Machine Learning
![Page 12: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/12.jpg)
Term Definitions• Machine Learning: “Field of study that gives computers
the ability to learn without being explicitly programmed” Arthur Samuel
• Arthur Samuel: A pioneer in the field of computer gaming and artificial intelligence. He coined the term "machine learning" in 1959.
• Feature: In machine learning and pattern recognition, a feature is individual measurable property of a phenomenon being observed.
![Page 13: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/13.jpg)
Term Definitions• Data Sampling: Data sampling is
a statistical analysis technique used to select, manipulate and analyze a representative subset of data points in order to identify patterns in the larger data set being examined.
![Page 14: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/14.jpg)
Term Definitions• Training Set: A training set is a set of data used to discover
potentially predictive relationships.
• ML Model: You can use the ML model to get predictions on new data for which you do not know the target.
• Cross Validation: A model validation technique for assessing how the results of a statistical analysis will generalize to an independent data set.
![Page 15: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/15.jpg)
Term Definitions
![Page 16: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/16.jpg)
Confusion Matrix
![Page 17: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/17.jpg)
Confusion Matrix• Accuracy: Ratio of correctly predicted observations.
(TP + TN) / (TP + TN + FP + FN)
• Precision: Ratio of correct positive observations. TP / (TP + FP)
• Recall: Ratio of correctly predicted positive events.
TP / (TP + FN)
![Page 18: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/18.jpg)
Visual Example
![Page 20: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/20.jpg)
Supervised Learning
![Page 21: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/21.jpg)
Supervised Learning• Input data is called training data and has
a known label or result such as spam/not-spam or a stock price at a time.
• Example problems are classification and regression.
• Example algorithms include Logistic Regression and the Back Propagation Neural Network.
![Page 22: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/22.jpg)
Supervised Learning Example
![Page 23: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/23.jpg)
Supervised Learning Example
![Page 24: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/24.jpg)
Supervised Learning• Supervised Learning: Right
answers given
• Regression: Predict continuous valued output
• Classification: Discrete valued output
![Page 25: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/25.jpg)
Supervised Learning – Classification Example
![Page 26: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/26.jpg)
Supervised Learning – Classification Example
![Page 27: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/27.jpg)
Supervised Learning – Classification Example
http://localhost:8888/notebooks/dev/workspaces/iyzico/scipy_2015_sklearn_tutorial/notebooks/02.1%20Supervised%20Learning%20-%20Classification.ipynb
![Page 28: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/28.jpg)
Linear Regression with One Variable
![Page 29: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/29.jpg)
Linear Regression with One Variable
![Page 30: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/30.jpg)
Cost Function
![Page 31: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/31.jpg)
Cost Function
![Page 32: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/32.jpg)
Cost Function
![Page 33: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/33.jpg)
Supervised Learning – Regression Example
http://localhost:8888/notebooks/dev/workspaces/iyzico/scipy_2015_sklearn_tutorial/notebooks/02.2%20Supervised%20Learning%20-%20Regression.ipynb
![Page 34: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/34.jpg)
Unsupervised Learning
![Page 35: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/35.jpg)
Unsupervised Learning• Input data is not labeled and does not
have a known result.
• Example problems are clustering, dimensionality reduction and association rule learning.
• Example algorithms include: the Apriori algorithm and k-Means.
![Page 36: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/36.jpg)
Unsupervised Learning Examples
![Page 37: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/37.jpg)
Unsupervised Learning – Transformation Example
http://localhost:8888/notebooks/dev/workspaces/iyzico/scipy_2015_sklearn_tutorial/notebooks/02.3%20Unsupervised%20Learning%20-%20Transformations%20and%20Dimensionality%20Reduction.ipynb
![Page 38: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/38.jpg)
Unsupervised Learning – Clustering Example
http://localhost:8888/notebooks/dev/workspaces/iyzico/scipy_2015_sklearn_tutorial/notebooks/02.4%20Unsupervised%20Learning%20-%20Clustering.ipynb
![Page 39: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/39.jpg)
Cross Validation
![Page 40: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/40.jpg)
Cross Validation• A model validation technique
for assessing how the results of a statistical analysis will generalize to an independent data set.
![Page 41: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/41.jpg)
Cross Validation Example
http://localhost:8888/notebooks/dev/workspaces/iyzico/scipy_2015_sklearn_tutorial/notebooks/04.1%20Cross%20Validation.ipynb
![Page 42: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/42.jpg)
Feature Extraction
![Page 43: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/43.jpg)
Feature Extraction• Feature extraction starts from an initial set of measured data and builds
derived values (features) intended to be informative and non-redundant.
• Feature extraction involves reducing the amount of resources required to describe a large set of data.
![Page 44: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/44.jpg)
Text Feature Extraction Example
http://localhost:8888/notebooks/dev/workspaces/iyzico/scipy_2015_sklearn_tutorial/notebooks/03.4%20Methods%20-%20Text%20Feature%20Extraction.ipynb
![Page 45: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/45.jpg)
Case Study
![Page 46: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/46.jpg)
Handwriting Digits
http://localhost:8888/notebooks/dev/workspaces/iyzico/scipy_2015_sklearn_tutorial/notebooks/03.1%20Case%20Study%20-%20Supervised%20Classification%20of%20Handwritten%20Digits.ipynb
![Page 47: Building Data Driven Applications with Machine Learning](https://reader036.vdocuments.mx/reader036/viewer/2022081520/589a60591a28abc3438b5c9f/html5/thumbnails/47.jpg)
thanks26.01.2017