Introduction to Recurrent Neural Networks with Application to Sentiment Analysis

by Rajarshee Mitra, Research Engineer (NLP) , Artifacia(@rajarshee_mitra)

November 19, 2016

AI Meet|

Agenda

1. What is AI ? 2. What is NLU ? Why is it hard ?3. Introduction to Neural Networks.4. Introduction to Recurrent Neural Networks (RNN).5. Application of RNN models.6. Variants of RNN.7. Sequence to Sequence Learning.8. Sentiment Analysis - an application.9. Food for Thought.

AI Meet|

What is AI

AI is the ability of software to mimic human brain and perform human-like abilities such as understanding emotions and meanings from text, handling ambiguities, recognizing objects etc.

AI Meet|

What is NLU? Why is it hard ?

Natural Language Understanding is the ability to process, understand and generate human languages (to create some action or intent).

● Language contains ambiguities.“I am looking at the elephant in white pyajamas”

● Modification of a single word (insertion, deletion) changes the meaning of the whole sentence.

● Context plays a serious role in language understanding.

AI Meet|

Neural Network - An introduction

AI Meet|

Neural Network - An introduction

INPUT OUTPUT TARGET

am He, running I, going

looked I, am I, at

sofa The, is The, is

Neural Language Modelling - skip gram model

AI Meet|

Neural Network - An introduction

Loss Functions:

1. Absolute Difference2. Root Mean Square3. Cross Entropy or Log Loss

AI Meet|

Recurrent Neural Nets

AI Meet|

Application of RNN Models

1. Sentiment Analysis2. Language Modelling3. Translation4. Conversational Agents5. Language Generation6. Image Captioning7. Text Summarization

AI Meet|

Some Interesting Variants of Neural Network

1. Long Short Term Memory Networks

2. Gated Recurrent Unit.

3. End to End networks - Sequence to Sequence Learning

4. Memory Networks.

AI Meet|

Sequence to Sequence Learning

AI Meet|

Sentiment Analysis - An ApplicationThe process of computationally identifying and categorizing opinions expressed in a piece of text, especially in order to determine whether the writer's attitude towards a particular topic, product, etc. is positive, negative, or neutral.

AI Meet|

Sentiment Analysis - An Application

1. We can treat the last output vector as our predicted sentiment.

2. We calculate loss between our output and target vector which contains the actual sentiment.

3. We update our model accordingly to minimize the loss.

4. A successfully learnt model will automatically predict sentiments of unseen sentences.

AI Meet|

Food for Thought

As we are more approaching towards linking concepts of neuroscience with mathematical concepts of Deep Learning, I imagine a system which might have following components :

1. A processor or generator - An RNN that process sentence word by word or generates sentence in the same way.

2. A memory that will facilitate read / write operations.3. A some variant of feed forward neural network that will act

between the processor and the memory and will determine which neurons to activate, what to read and write.

AI Meet|

Further Reading

1. http://karpathy.github.io/2015/05/21/rnn-effectiveness/

2. http://colah.github.io/posts/2015-08-Understanding-LSTMs/

3. http://papers.nips.cc/paper/5346-sequence-to-sequence-learning-with-neural-networks.pdf

From Our Blog:- http://research.artifacia.com/learn-deep-learning-the-hard-way

THANK YOUJoin

meetup.com/Artifacia-AI-Meet/