introduction to deep learning with...
TRANSCRIPT
![Page 1: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/1.jpg)
Introduction to Deep Learning with Python
Samar HaiderUniversity of Engineering and Technology, Lahore
PyCon Pakistan16/12/2017
![Page 2: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/2.jpg)
Artificial intelligence, machine learning & deep learning
![Page 3: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/3.jpg)
The AI universe
Samar Haider PyCon Pakistan 2017 3
Goodfellow et. al., “Deep Learning.” MIT Press (2016)
![Page 4: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/4.jpg)
The AI universe
“What's the difference between AI and ML?”
“It's AI when you're raising money, it's ML when you're trying to hire people.”
Samar Haider PyCon Pakistan 2017 4
![Page 5: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/5.jpg)
Machine learning
Samar Haider PyCon Pakistan 2017 5
Data
![Page 6: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/6.jpg)
Machine learning
Samar Haider PyCon Pakistan 2017 6
Learning Algorithm
Data
![Page 7: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/7.jpg)
Machine learning
Samar Haider PyCon Pakistan 2017 7
Hypothesis
Learning Algorithm
Data
![Page 8: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/8.jpg)
Machine learning
Samar Haider PyCon Pakistan 2017 8
HypothesisInput Prediction
Data
Learning Algorithm
![Page 9: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/9.jpg)
Machine learning
Samar Haider PyCon Pakistan 2017 9
Engineer features Learn mapping
Data Features Labels
![Page 10: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/10.jpg)
Deep learning
Samar Haider PyCon Pakistan 2017 10
Learn features Learn mapping
Data Features Labels
![Page 11: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/11.jpg)
The rise of deep learning
Samar Haider PyCon Pakistan 2017 11
![Page 12: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/12.jpg)
The rise of deep learning
Samar Haider PyCon Pakistan 2017 12
Goodfellow et. al., “Deep Learning.” MIT Press (2016)
![Page 13: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/13.jpg)
The rise of deep learning
Samar Haider PyCon Pakistan 2017 13
Dean, “Large-Scale Deep Learning for Intelligent Computer Systems.” WSDM (2016)
![Page 14: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/14.jpg)
The rise of deep learning
1. Bigger datasets
2. More computational power
3. Improvements in algorithms (due to 1 and 2)
Samar Haider PyCon Pakistan 2017 14
![Page 15: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/15.jpg)
The rise of deep learning
“If big data is the new oil, deep learning is the new internal combustion engine.”
– Yann LeCun
(Director, Facebook AI Research)
Samar Haider PyCon Pakistan 2017 15
![Page 16: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/16.jpg)
The rise of deep learning
“AI is the new electricity: Just as electricity transformed almost everything 100 years ago, today I actually have a hard time thinking of an industry that I don’t think AI will transform in the next several years.”
– Andrew Ng
(Founder, deeplearning.ai)
Samar Haider PyCon Pakistan 2017 16
![Page 17: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/17.jpg)
Neural networks
![Page 18: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/18.jpg)
Biological neuron
Samar Haider PyCon Pakistan 2017 18
![Page 19: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/19.jpg)
Artificial neuron
Samar Haider PyCon Pakistan 2017 19
![Page 20: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/20.jpg)
Biological vs artificial neuron
Samar Haider PyCon Pakistan 2017 20
![Page 21: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/21.jpg)
Activation functions
Samar Haider PyCon Pakistan 2017 21
Sze et. al., “Efficient Processing of Deep Neural Networks: A Tutorial and Survey.” arXiv (2017)
![Page 22: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/22.jpg)
Activation functions
Samar Haider PyCon Pakistan 2017 22
LeCun et. al., “Deep Learning.” Nature (2015)
![Page 23: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/23.jpg)
A shallow neural network
Samar Haider PyCon Pakistan 2017 23
![Page 24: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/24.jpg)
A deep neural network
Samar Haider PyCon Pakistan 2017 24
![Page 25: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/25.jpg)
What we want
Samar Haider PyCon Pakistan 2017 25
Good features/representations
Correct predictions
![Page 26: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/26.jpg)
What we want
Samar Haider PyCon Pakistan 2017 26
Goodfellow et. al., “Deep Learning.” MIT Press (2016)
![Page 27: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/27.jpg)
The need for depth
Samar Haider PyCon Pakistan 2017 27
Goodfellow et. al., “Deep Learning.” MIT Press (2016)
![Page 28: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/28.jpg)
… and even more depth
Samar Haider PyCon Pakistan 2017 28
Szegedy et. al., “Going Deeper with Convolutions.” CVPR (2015)
![Page 29: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/29.jpg)
Specialized architectures
Vision: Convolutional Neural Networks
Samar Haider PyCon Pakistan 2017 29
![Page 30: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/30.jpg)
Specialized architectures
Language: Recurrent Neural Networks
Samar Haider PyCon Pakistan 2017 30
![Page 31: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/31.jpg)
Deep learning
![Page 32: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/32.jpg)
The learning process
1. Pick a training example
2. Make a prediction for it
3. Compare your prediction with the truth (= error)
4. Modify your weights in order to minimize this error
5. Repeat until convergence
Samar Haider PyCon Pakistan 2017 32
![Page 33: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/33.jpg)
Minimizing the error
Samar Haider PyCon Pakistan 2017 33
![Page 34: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/34.jpg)
Gradient descent
Samar Haider PyCon Pakistan 2017 34
![Page 35: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/35.jpg)
Gradient descent in higher dimensions
Samar Haider PyCon Pakistan 2017 35
![Page 36: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/36.jpg)
Learning multiple layers
Samar Haider PyCon Pakistan 2017 36
![Page 37: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/37.jpg)
Forward pass
Samar Haider PyCon Pakistan 2017 37
![Page 38: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/38.jpg)
Backward propagation
Samar Haider PyCon Pakistan 2017 38
![Page 39: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/39.jpg)
Backward propagation
Samar Haider PyCon Pakistan 2017 39
![Page 40: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/40.jpg)
Backward propagation
Samar Haider PyCon Pakistan 2017 40
![Page 41: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/41.jpg)
Backward propagation
Samar Haider PyCon Pakistan 2017 41
![Page 42: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/42.jpg)
Backpropagation algorithm in full
Samar Haider PyCon Pakistan 2017 42
LeCun et. al., “Deep Learning.” Nature (2015)
![Page 43: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/43.jpg)
Applications
![Page 44: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/44.jpg)
Object Detection
Samar Haider PyCon Pakistan 2017 44
Ren et. al., “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks.” NIPS (2015)
![Page 45: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/45.jpg)
Scene segmentation
Samar Haider PyCon Pakistan 2017 45
Badrinarayanan et. al., “SegNet: A Deep Convolutional Encoder-Decoder Architecture…” PAMI (2016)
![Page 46: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/46.jpg)
Super resolution
Samar Haider PyCon Pakistan 2017 46
Dahl et. al., “Pixel Recursive Super Resolution.” arXiv (2017)
![Page 47: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/47.jpg)
Style transfer
Samar Haider PyCon Pakistan 2017 47
Gatys et. al., “A Neural Algorithm of Artistic Style.” arXiv (2015)
![Page 48: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/48.jpg)
Image translation
Samar Haider PyCon Pakistan 2017 48
Liu et. al., “Unsupervised Image-to-Image Translation Networks.” NIPS (2017)
![Page 49: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/49.jpg)
Image generation
Samar Haider PyCon Pakistan 2017 49
Karras et. al., “Progressive Growing of GANs for Improved Quality, Stability, and Variation.” arXiv (2017)
![Page 50: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/50.jpg)
Image generation
Samar Haider PyCon Pakistan 2017 50
Karras et. al., “Progressive Growing of GANs for Improved Quality, Stability, and Variation.” arXiv (2017)
![Page 51: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/51.jpg)
Learning word representations
Samar Haider PyCon Pakistan 2017 51
Mikolov et. al., “Efficient Estimation of Word Representations in Vector Space.” arXiv (2013)
![Page 52: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/52.jpg)
Learning sentiment representations
Samar Haider PyCon Pakistan 2017 52
Radford et. al., “Learning to Generate Reviews and Discovering Sentiment.” arXiv (2017)
![Page 53: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/53.jpg)
Image captioning
Samar Haider PyCon Pakistan 2017 53
Vinyals et. al., “Show and Tell: A Neural Image Caption Generator.” CVPR (2015)
![Page 54: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/54.jpg)
Visual question answering
Samar Haider PyCon Pakistan 2017 54
Yang et. al., “Stacked Attention Networks for Image Question Answering.” CVPR (2016)
![Page 55: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/55.jpg)
Playing games
Samar Haider PyCon Pakistan 2017 55
Silver et. al., “Mastering the Game of Go with Deep Neural Networks and Tree Search.” Nature (2016)
![Page 56: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/56.jpg)
Building better neural networks
Samar Haider PyCon Pakistan 2017 56
Zoph et. al., “Neural Architecture Search with Reinforcement Learning.” ICLR (2017)
![Page 57: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/57.jpg)
Building better software
Samar Haider PyCon Pakistan 2017 57
Kraska et. al., “The Case for Learned Index Structures.” arXiv (2017)
![Page 58: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/58.jpg)
What you need to get started with deep learning
![Page 59: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/59.jpg)
These, pretty much
Samar Haider PyCon Pakistan 2017 59
![Page 60: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/60.jpg)
… plus a handful of other stuff
Samar Haider PyCon Pakistan 2017 60
https://medium.com/towards-data-science/building-your-own-deep-learning-box-47b918aea1eb
![Page 61: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/61.jpg)
Building a deep learning rig
Samar Haider PyCon Pakistan 2017 61
https://pcpartpicker.com/list/FRp8XH
![Page 62: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/62.jpg)
An alternative
Samar Haider PyCon Pakistan 2017 62
![Page 63: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/63.jpg)
Amazon Web Services
Samar Haider PyCon Pakistan 2017 63
![Page 64: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/64.jpg)
Amazon Web Services
Samar Haider PyCon Pakistan 2017 64
![Page 65: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/65.jpg)
Amazon Web Services
Samar Haider PyCon Pakistan 2017 65
![Page 66: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/66.jpg)
Deep learning software ecosystem
Samar Haider PyCon Pakistan 2017 66
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
https://twitter.com/fchollet/status/915366704401719296
![Page 67: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/67.jpg)
A typical beginner stack
Samar Haider PyCon Pakistan 2017 67
![Page 68: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/68.jpg)
A typical beginner stack
Samar Haider PyCon Pakistan 2017 68
![Page 69: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/69.jpg)
A typical beginner stack
Samar Haider PyCon Pakistan 2017 69
![Page 70: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/70.jpg)
A typical beginner stack
Samar Haider PyCon Pakistan 2017 70
![Page 71: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/71.jpg)
Deep learning in a day
1. Create an AWS account
2. Launch an EC2 instance
3. SSH into your instance
4. Launch a Jupyter Notebook
5. ???
6. Profit!
Samar Haider PyCon Pakistan 2017 71
![Page 72: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/72.jpg)
Samar Haider PyCon Pakistan 2017 72
![Page 73: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/73.jpg)
Samar Haider PyCon Pakistan 2017 73
![Page 74: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/74.jpg)
Samar Haider PyCon Pakistan 2017 74
![Page 75: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/75.jpg)
Samar Haider PyCon Pakistan 2017 75
![Page 76: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/76.jpg)
Samar Haider PyCon Pakistan 2017 76
![Page 77: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/77.jpg)
Samar Haider PyCon Pakistan 2017 77
![Page 78: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/78.jpg)
Samar Haider PyCon Pakistan 2017 78
![Page 80: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/80.jpg)
Samar Haider PyCon Pakistan 2017 80
![Page 81: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/81.jpg)
![Page 82: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/82.jpg)
![Page 83: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/83.jpg)
Samar Haider PyCon Pakistan 2017 83
![Page 84: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/84.jpg)
Where to learn more
![Page 85: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/85.jpg)
Courses
fast.aiby Jeremy Howard
Samar Haider PyCon Pakistan 2017 85
![Page 86: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/86.jpg)
Courses
Samar Haider PyCon Pakistan 2017 86
deeplearning.aiby Andrew Ng
![Page 87: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/87.jpg)
Books
Samar Haider PyCon Pakistan 2017 87
Goodfellow
Bengio
& Courville
![Page 88: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/88.jpg)
Books
Samar Haider PyCon Pakistan 2017 88
Michael Nielsen
![Page 89: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/89.jpg)
Papers
Samar Haider PyCon Pakistan 2017 89
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton, “Deep Learning.” Nature (2015)
Most Cited Deep Learning Papers
https://github.com/terryum/awesome-deep-learning-papers
Deep Learning Papers Reading Roadmap
https://github.com/songrotek/Deep-Learning-Papers-Reading-Roadmap
![Page 90: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/90.jpg)
Demos
Samar Haider PyCon Pakistan 2017 90
TensorFlow Playground
https://playground.tensorflow.org/
ConvNetJS
https://cs.stanford.edu/people/karpathy/convnetjs/
Quick, Draw!
https://quickdraw.withgoogle.com/
![Page 91: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/91.jpg)
“Software is eating the world, but AI is going to eat software.”
– Jensen Huang
(CEO, Nvidia)
Samar Haider PyCon Pakistan 2017 91
![Page 92: Introduction to Deep Learning with Pythonpycon.pk/static/media/uploads/presentations/samar_haider...The rise of deep learning 1. Bigger datasets 2. More computational power 3. Improvements](https://reader033.vdocuments.mx/reader033/viewer/2022042409/5f25b631f6a7383289201fe4/html5/thumbnails/92.jpg)
Thank you
@samarhdr