Teaching Recurrent Neural Networks using Tensorflow (May 2016)

Download Teaching Recurrent Neural Networks using Tensorflow (May 2016)

Post on 21-Apr-2017

2.096 views

Category:

Data & Analytics

9 download

Embed Size (px)

TRANSCRIPT

  • Teaching Recurrent Neural Networks

    using TensorflowRajiv Shah

    twitter: rajcs4 rshah@pobox.com www.rajivshah.com

    mailto:rshah@pobox.comhttp://www.rajivshah.com

  • Teaching Recurrent Neural Networks using Tensorflow

    How they are used

    Theory - http://playground.tensorflow.org/

    Learn a sine wave - http://tinyurl.com/raj-sine

    Learn to add - http://tinyurl.com/raj-add

    Learn handwriting - https://github.com/hardmaru/write-rnn-tensorflow

    https://github.com/hardmaru/write-rnn-tensorflow

  • Theory

  • Recurrent Neural Networks: Theory

  • Unrolling an RNN

  • RNN

  • LSTM

  • Examples of RNNs

  • Little more theory . . . but fun

  • Tensorflow Playground http://playground.tensorflow.org/

  • Sine Wave

  • High Level Architecture

    Function to get batches

    Determined type of RNN cells and initialize them

    Decide on cost function and hyper parameters (dropout, learning rate, optimizer, batch size, sequence length)

    Run session

  • Go to the python notebook

  • Friendly Tensorflow - Using R / Shiny

    Wrapped calls to tensorflow using the rPython language

    Shiny provided a nice visual interface and the ability to quickly change different sorts of inputs / models

    https://github.com/rajshah4/tensorflow_shiny

    https://github.com/rajshah4/tensorflow_shiny

  • Adding

  • Teaching Kids to Add

  • Teaching to Add(array([[7], [4], [1], [8], [0], [3], [1], [5], [6], [9], [2], [0], [0], [0], [0]]), 46.0)

  • Go to the python notebook

  • Learning Handwriting

  • Links

    http://www.cs.toronto.edu/~graves/handwriting.html

    https://github.com/hardmaru/write-rnn-tensorflow

    https://github.com/hardmaru/write-rnn-tensorflow

  • Take Aways

    RNNs are sequential learners and state of the art

    Understand why deep learning is powerful

    See how a RNN learns to model a sine wave, add, learn handwriting . . .

  • Teaching Recurrent Neural Networks

    using TensorflowRajiv Shah

    twitter: rajcs4 rshah@pobox.com www.rajivshah.com

    mailto:rshah@pobox.comhttp://www.rajivshah.com

Recommended

View more >