# Teaching Recurrent Neural Networks using Tensorflow (May 2016)

Post on 21-Apr-2017

2.096 views

Embed Size (px)

TRANSCRIPT

Teaching Recurrent Neural Networks

using TensorflowRajiv Shah

twitter: rajcs4 rshah@pobox.com www.rajivshah.com

mailto:rshah@pobox.comhttp://www.rajivshah.com

Teaching Recurrent Neural Networks using Tensorflow

How they are used

Theory - http://playground.tensorflow.org/

Learn a sine wave - http://tinyurl.com/raj-sine

Learn to add - http://tinyurl.com/raj-add

Learn handwriting - https://github.com/hardmaru/write-rnn-tensorflow

https://github.com/hardmaru/write-rnn-tensorflow

Theory

Recurrent Neural Networks: Theory

Unrolling an RNN

RNN

LSTM

Examples of RNNs

Little more theory . . . but fun

Tensorflow Playground http://playground.tensorflow.org/

Sine Wave

High Level Architecture

Function to get batches

Determined type of RNN cells and initialize them

Decide on cost function and hyper parameters (dropout, learning rate, optimizer, batch size, sequence length)

Run session

Go to the python notebook

Friendly Tensorflow - Using R / Shiny

Wrapped calls to tensorflow using the rPython language

Shiny provided a nice visual interface and the ability to quickly change different sorts of inputs / models

https://github.com/rajshah4/tensorflow_shiny

https://github.com/rajshah4/tensorflow_shiny

Adding

Teaching Kids to Add

Teaching to Add(array([[7], [4], [1], [8], [0], [3], [1], [5], [6], [9], [2], [0], [0], [0], [0]]), 46.0)

Go to the python notebook

Learning Handwriting

Links

http://www.cs.toronto.edu/~graves/handwriting.html

https://github.com/hardmaru/write-rnn-tensorflow

https://github.com/hardmaru/write-rnn-tensorflow

Take Aways

RNNs are sequential learners and state of the art

Understand why deep learning is powerful

See how a RNN learns to model a sine wave, add, learn handwriting . . .

Teaching Recurrent Neural Networks

using TensorflowRajiv Shah

twitter: rajcs4 rshah@pobox.com www.rajivshah.com

mailto:rshah@pobox.comhttp://www.rajivshah.com

Recommended