Teaching Recurrent Neural Networks
using Tensorflow Rajiv Shah
twitter: rajcs4 [email protected] www.rajivshah.com
Teaching Recurrent Neural Networks using Tensorflow
• How they are used
• Theory - http://playground.tensorflow.org/
• Learn a sine wave - http://tinyurl.com/raj-sine
• Learn to add - http://tinyurl.com/raj-add
• Learn handwriting - https://github.com/hardmaru/write-rnn-tensorflow
Theory
Recurrent Neural Networks: Theory
Unrolling an RNN
RNN
LSTM
Examples of RNNs
Little more theory . . . but fun
Tensorflow Playground• http://playground.tensorflow.org/
Sine Wave
High Level Architecture
• Function to get batches
• Determined type of RNN cells and initialize them
• Decide on cost function and hyper parameters (dropout, learning rate, optimizer, batch size, sequence length)
• Run session
Go to the python notebook
Friendly Tensorflow - Using R / Shiny
• Wrapped calls to tensorflow using the rPython language
• Shiny provided a nice visual interface and the ability to quickly change different sorts of inputs / models
• https://github.com/rajshah4/tensorflow_shiny
Adding
Teaching Kids to Add
Teaching to Add(array([[7], [4], [1], [8], [0], [3], [1], [5], [6], [9], [2], [0], [0], [0], [0]]), 46.0)
Go to the python notebook
Learning Handwriting
Links
• http://www.cs.toronto.edu/~graves/handwriting.html
• https://github.com/hardmaru/write-rnn-tensorflow
Take Aways
• RNNs are sequential learners and state of the art
• Understand why deep learning is powerful
• See how a RNN learns to model a sine wave, add, learn handwriting . . .
Teaching Recurrent Neural Networks
using Tensorflow Rajiv Shah
twitter: rajcs4 [email protected] www.rajivshah.com
