deep learning for mere mortals - devoxx belgium 2015
TRANSCRIPT
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Shallow Dive into Deep Learning
Sam Bessalah - @samklr
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
“ This talk is filled with buzzwords and terms, that might make you sound too nerdy or pedantic at dinner. ”
Use with caution.
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
In the news, lately ...
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Skype Translator Demo https://www.youtube.com/watch?v=eu9kMIeS0wQ
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Google DeepMind's Deep Q-learning playing Atari
https://www.youtube.com/watch?v=V1eYniJ0Rnk
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Deep Learning?
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Multiple Definitions :
- A set of algorithms that try to model high-level abstractions in data by using architectures composed of multiple non linear transformations.
- A set of machine learning algorithms that automatically learn features hierarchies.
- Usually built using Neural Networks.
- Representation Learning: Automatically learning good representations of the data for your classifier, i.e learn good features
- Deep Learning : Learning multiple levels of representation with
complex layers architectures.
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Typical Machine Learning Workflow
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Typical Machine Learning Workflow
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
A Neural Networks Primer
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Loose imitation of the brain neurons
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Loose imitation of the brain neurons
@YourTwitterHandle#Devoxx #YourTag @samklr#Devoxx #deepLearning
@nivdul
Perceptron
@YourTwitterHandle#Devoxx #YourTag @samklr#Devoxx #deepLearning
@nivdul
Perceptron
Output Error computed with E, and Network error
minimized with a stochastic gradient descent.
@YourTwitterHandle#Devoxx #YourTag @samklr#Devoxx #deepLearning
Stochastic gradient descent and Backpropagation
- The optimal value for each weight where at error achieves a global minimum.
- Backpropagation : Compute error in the output, then propagate it back through the network, to update the weights during the training phase.
@YourTwitterHandle#Devoxx #YourTag @samklr#Devoxx #deepLearning
@YourTwitterHandle#Devoxx #YourTag @samklr#Devoxx #deepLearning
@nivdulMulti-layer Perceptron
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Problem with Large Networks
- Vanishing Gradient : More layers kills the back propagation. As information is passed back; gradient value startts to vanish and become smaller compared to weights.
- Overfitting : Algorithm fits too closely the training data, but will fail miserably on real examples
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Problem with Large Networks
- Vanishing Gradient : More layers kills the back propagation. As information is passed back; gradient value startts to vanish and become smaller compared to weights.
- Overfitting : Algorithm fits too closely the training data, but will fail miserably on real examples
This led to another AI
winter
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Why does work suddenly ?
- A lot more labeled data- More compute power CPU and GPU- Clever new idea on how to train deep
architectures
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Convolution Neural Networks
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Convolution Neural Networks
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Convolution Neural Networks
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Recurrent Neural Networks (RNN)
- Learn from arbitrary sequential inputs, by using their internal states
- Cannot look far back (back propagation limited)- Long Short Term Memory Networks help solving this- Good for NLP, hand writing, etc
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Recurrent Neural Networks (RNN)
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Deep Visual-Semantic Alignments for Generating Image Descriptions CNN + RNN
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Other Techniques
- Auto Encoders
- Restricted Boltzmann Machine (RBM)
- Deep Belief
- Hierarchical temporal memory (HTM)
- LSTM
- More and more ...
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Frameworks and Libraries
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
H20.ai Deep Learning
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
DeepLearning4j
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #deepLearning
Getting Started
- Coursera course on Neural Networks from G. Hinton
- Deeplearning4j.org
- Deeplearning.net
@YourTwitterHandle#Devoxx #YourTag @samklr#Devoxx #deepLearning
Deep Dream
@YourTwitterHandle#Devoxx #YourTag @samklr#Devoxx #deepLearning
CONCLUSION