![Page 1: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/1.jpg)
Better than Deep Learning:Gradient Boosting Machines (GBM)
Szilárd Pafka, PhDChief Scientist, Epoch (USA)
½ Day Workshop, Budapest Data Forum ConferenceJune 2018
![Page 2: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/2.jpg)
![Page 3: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/3.jpg)
At a Glance...
ML: sup.L: y = f(x) “learn” f from data (y, X)training, testing/prediction, algos (LR,DT,NN…), optimization, overfitting, regularization...
GBM: ensemble of decision trees
GBM libs: R/Python
![Page 4: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/4.jpg)
![Page 5: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/5.jpg)
other than GBMs
![Page 6: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/6.jpg)
![Page 7: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/7.jpg)
Disclaimer:
✔ I understand this is an intermediate/advanced workshop
Prerequisites:
basic ML conceptsR/Python experience
![Page 8: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/8.jpg)
Schedule:
1. Intro talk (slides)
2. Demo main features (me running code)
3. Hands-on (you install/run code)
![Page 9: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/9.jpg)
![Page 10: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/10.jpg)
![Page 11: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/11.jpg)
Student Intros / Goals
![Page 12: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/12.jpg)
Disclaimer:
I am not representing my employer (Epoch) in this talk
I cannot confirm nor deny if Epoch is using any of the methods, tools, results etc. mentioned in this talk
![Page 13: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/13.jpg)
Source: Andrew Ng
![Page 14: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/14.jpg)
Source: Andrew Ng
![Page 15: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/15.jpg)
Source: Andrew Ng
![Page 16: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/16.jpg)
![Page 17: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/17.jpg)
![Page 18: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/18.jpg)
![Page 19: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/19.jpg)
![Page 20: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/20.jpg)
Source: https://twitter.com/iamdevloper/
![Page 21: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/21.jpg)
![Page 22: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/22.jpg)
...
![Page 23: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/23.jpg)
![Page 24: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/24.jpg)
![Page 25: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/25.jpg)
http://www.cs.cornell.edu/~alexn/papers/empirical.icml06.pdf
http://lowrank.net/nikos/pubs/empirical.pdf
![Page 26: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/26.jpg)
http://www.cs.cornell.edu/~alexn/papers/empirical.icml06.pdf
http://lowrank.net/nikos/pubs/empirical.pdf
![Page 27: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/27.jpg)
![Page 28: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/28.jpg)
![Page 29: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/29.jpg)
![Page 30: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/30.jpg)
![Page 31: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/31.jpg)
structured/tabular data: GBM (or RF)very small data: LRvery large sparse data: LR with SGD (+L1/L2)images/videos, speech: DL
![Page 32: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/32.jpg)
structured/tabular data: GBM (or RF)very small data: LRvery large sparse data: LR with SGD (+L1/L2)images/videos, speech: DL
it depends
![Page 33: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/33.jpg)
structured/tabular data: GBM (or RF)very small data: LRvery large sparse data: LR with SGD (+L1/L2)images/videos, speech: DL
it depends / try them all
![Page 34: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/34.jpg)
structured/tabular data: GBM (or RF)very small data: LRvery large sparse data: LR with SGD (+L1/L2)images/videos, speech: DL
it depends / try them all / hyperparam tuning
![Page 35: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/35.jpg)
structured/tabular data: GBM (or RF)very small data: LRvery large sparse data: LR with SGD (+L1/L2)images/videos, speech: DL
it depends / try them all / hyperparam tuning / ensembles
![Page 36: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/36.jpg)
structured/tabular data: GBM (or RF)very small data: LRvery large sparse data: LR with SGD (+L1/L2)images/videos, speech: DL
it depends / try them all / hyperparam tuning / ensemblesfeature engineering
![Page 37: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/37.jpg)
structured/tabular data: GBM (or RF)very small data: LRvery large sparse data: LR with SGD (+L1/L2)images/videos, speech: DL
it depends / try them all / hyperparam tuning / ensemblesfeature engineering / other goals e.g. interpretability
![Page 38: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/38.jpg)
structured/tabular data: GBM (or RF)very small data: LRvery large sparse data: LR with SGD (+L1/L2)images/videos, speech: DL
it depends / try them all / hyperparam tuning / ensemblesfeature engineering / other goals e.g. interpretability
the title of this talk was misguided
![Page 39: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/39.jpg)
structured/tabular data: GBM (or RF)very small data: LRvery large sparse data: LR with SGD (+L1/L2)images/videos, speech: DL
it depends / try them all / hyperparam tuning / ensemblesfeature engineering / other goals e.g. interpretability
the title of this talk was misguidedbut so is recently almost every use of the term AI
![Page 40: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/40.jpg)
Source: Hastie etal, ESL 2ed
![Page 41: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/41.jpg)
Source: Hastie etal, ESL 2ed
![Page 42: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/42.jpg)
Source: Hastie etal, ESL 2ed
![Page 43: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/43.jpg)
Source: Hastie etal, ESL 2ed
![Page 44: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/44.jpg)
![Page 45: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/45.jpg)
I usually use other people’s code [...] I can find open source code for what I want to do, and my time is much better spent doing research and feature engineering -- Owen Zhanghttp://blog.kaggle.com/2015/06/22/profiling-top-kagglers-owen-zhang-currently-1-in-the-world/
![Page 46: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/46.jpg)
![Page 47: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/47.jpg)
![Page 48: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/48.jpg)
![Page 49: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/49.jpg)
![Page 50: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/50.jpg)
![Page 51: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/51.jpg)
![Page 52: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/52.jpg)
![Page 53: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/53.jpg)
![Page 54: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/54.jpg)
![Page 55: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/55.jpg)
![Page 56: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/56.jpg)
10x
![Page 57: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/57.jpg)
![Page 58: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/58.jpg)
![Page 59: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/59.jpg)
10x
![Page 60: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/60.jpg)
![Page 61: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/61.jpg)
![Page 62: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/62.jpg)
![Page 63: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/63.jpg)
![Page 64: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/64.jpg)
![Page 65: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/65.jpg)
![Page 66: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/66.jpg)
![Page 67: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/67.jpg)
![Page 68: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/68.jpg)
![Page 69: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/69.jpg)
![Page 70: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/70.jpg)
![Page 71: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/71.jpg)
![Page 72: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/72.jpg)
![Page 73: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/73.jpg)
![Page 74: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/74.jpg)
![Page 75: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/75.jpg)
![Page 76: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/76.jpg)
http://www.jmlr.org/papers/volume13/bergstra12a/bergstra12a.pdf
![Page 77: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/77.jpg)
http://www.argmin.net/2016/06/20/hypertuning/
![Page 78: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/78.jpg)
![Page 79: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/79.jpg)
![Page 80: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/80.jpg)
![Page 81: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/81.jpg)
ML training:
lots of CPU coreslots of RAM
limited time
![Page 82: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/82.jpg)
ML training:
lots of CPU coreslots of RAM
limited time
![Page 83: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/83.jpg)
![Page 84: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/84.jpg)
![Page 85: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/85.jpg)
![Page 86: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/86.jpg)
“people that know what they’re doing just use open source [...] the same open source tools that the MLaaS services offer” - Bradford Cross
![Page 87: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/87.jpg)
![Page 88: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/88.jpg)
![Page 89: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/89.jpg)
![Page 90: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/90.jpg)
![Page 91: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/91.jpg)
no-one is using this crap
![Page 92: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/92.jpg)
![Page 93: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/93.jpg)
![Page 94: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/94.jpg)
![Page 95: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/95.jpg)
More:
![Page 96: Better than Deep Learning: Gradient Boosting Machines (GBM)biconsulting.hu/letoltes/2018budapestdata/pafka... · structured/tabular data: GBM (or RF) very small data: LR very large](https://reader034.vdocuments.mx/reader034/viewer/2022042411/5f28de44311ca407a11cf04f/html5/thumbnails/96.jpg)