recurrent neural networks


Post on 19-Jan-2016




0 download

Embed Size (px)


RECURRENT NEURAL NETWORKS. OUTLINE. Why use recurrent networks Typical architectures A taste of backprop. Simple Recurrent Network. Robot sensory prediction with SRN. Short Term Memory (STM) & Simple T-maze problems. - PowerPoint PPT Presentation



  • OUTLINEWhy use recurrent networksTypical architecturesA taste of backprop

  • Simple Recurrent Network

  • Robot sensory prediction with SRN

  • Short Term Memory (STM) & Simple T-maze problemsWhat types of neural nets have already been used for similar tasks and why? What are the inputs and the outputs in an artificial neural net used for such a task?

    Delayed response tasks are a standard way of investigating short-term memory (STM). The agent is typically assumed to remember in some way the necessary information about the stimulus (for example the side on which a stimulus appeared) during the delay period.

    Studies upto now are far away from suggesting a specific model of how animals solve delayed response tasks (Ziemke). The solution seems to lie in an interdisciplinary approach.

    Figure The two situations in the simple T-maze environment. Adapted from Ulbricht (1996).

  • Sensors of the Khepara Robotas Artificial Neural Network InputsSensors of the Khepara robot used in this study:

    I1, I2....I6: infrared sensors;G: ground (zone) sensor.L1,L2: Light sensors.

  • Repeated T-maze

  • Multiple T-maze

  • Ziemkes Food/Poison Maze (1999)

  • Recurrent Connections and SequencesA sequence is a succession of patterns that relate to the same object. For example, letters that make up a word or words that make up a sentence. Sequences can vary in length. This is a challenge. How many inputs should there be for varying length inputs?Several feed-forward alternatives: shift registers, tapped delay lines, etc

  • The simple recurrent networkJordan network has connections that feed back from the output to the input layer and also some input layer units feed back to themselves. Useful for tasks that are dependent on a sequence of a successive states.The network can be trained by backpropogation.The network has a form of short-term memory.Simple recurrent network (SRN) has a similar form of short-term memory.

  • Figure 5.6

  • Elman Network (SRN). The number of context units is the same as the number of hidden units

  • Modified Elman Network (Adapted from Rylatt and Czarnecki (2000))

    In addition predict sensory inputs.

  • Short-term memory in SRNThe context units remember the previous internal state.Thus, the hidden units have the task of mapping both an external input and also the previous internal state to some desired output.

  • An SRN can predict the next item in a sequence from the current and preceding input.In an SRN the hidden units are fed back into the input layer.That is: The hidden units are copied into input units. Input units supply weighted sums into hidden units in the next time step. ********************The hidden units represent an internal reduced representation of the data in the sequence that precedes the current input.

  • The reduced representation provides context which is essential for certain tasks.

    As an example SRN can learn to solve XOR problem.

    Input 101000011110101.. Bit 3 is XOR of bit 1 and 2, bit 6 is XOR of 4 and 4, and so on

    An SRN with 1 input, two context, two hidden and one output unit was trained on a sequence of 3000 bits.input: 1 0 1 0 0 0 0 1 1 1 1 0 1 0 1. . .output: 0 1 0 0 0 0 1 1 1 1 0 1 0 1 ?. . .

  • ESCN (Extended Sequential Cascaded Networks) & T-mazesTypically used networks for STM tasks are first-order or higher-order feedback recurrent networks.

    In first-order recurrent networks, certain neuron activation values are fed back into the network, typically in the next time step such as to remember agents goal, which part of the environment agent is currently in etc.

    In the case of higher-order feedback, its typically connection weights (and/or bias weights) that are modulated dynamically. Examples are sequential cascaded networks (SCN) or Extended(SCN).

  • ESCNMore specifically, given an input vector ij(t), j = 1n, state unit vector s(t) and output vectoro(t) are usually calculated as follows by the function network:

    Where f is the logistic activation function, and W0(t) and WS(t), together referred to as function network weights, are two-dimensional connection weight matrices dynamically computed in every time step by the context network.

    W0(t) = {wa(t), wb(t), .wn(t) } // ONE FOR EACH OUTPUT UNITWs(t) = {wsa(t), wsb(t), .wsn(t) } // ONE FOR EACH STATE UNIT

  • ESCNTheres an additional decision unit in ESCN. Given the state vector sk(t), k = 1m, the decision unit activation d(t) and the function network weight matrices W0(t) and WS(t), are dynamically computed in every time step t as follows by the ESCNs context network:

    Where f is the logistic activation function, Vd is a one-dimensional connection weight matrix, mapping current state to decision unit activation, and V0 and VS are two-dimensional connection weight matrices mapping the current internal state s(t) to the next time steps function network weights, if the decision unit is active, i.e. has an activation value of at least 0.5.

  • ESCN Example Behaviour (Ziemke 1999)

  • BP: Unfolding Network ExampleA 3-2-2 fully connected network has recurrent connections where a hidden unit connects to itself and to every other output unit. Sketch the network and its equivalent feedforward network with no recurrent connections, for a single time step. The single time step in this example means that an input pattern will have passed between two hidden units and two output units.

  • Figure 5.4

  • Figure 5.5

  • A recurrent backpropogation networkA backpropogation network need not be strictly feedforward and can have recurrent connections. A unit can feed to itself, to units in the same or lower levels.A recurrent connection feeds back activation that will affect the output from the network during subsequent iterations.For every recurrent network there is a feedforward network with identical behavior.

  • Figure A recurrent backpropogation network (Connections from output to input layer)

  • An expanded version of the network in the previous figure.

  • Backprop Through TimeDuring training a pattern is presented to the network and a feedforward pass is made. Each network copy corresponds to a time step. The weight changes calculated for each network copy are summed before individual weights are adapted.The set of weights for each copy(time step) always remain the same.

  • Sequence Completion ExampleRumelhart et. al. (1986) performed experiments with recurrent connections. Train a network to perform sequence completion.A sequence consisted of 6 characters: two letters from the set {A,B,C,D,E} and 4 being numbers from the integer set {1,2,3}.For example, A can denote 12 and C can denote 13.The complete sequence prefixed by AC would be AC1213.The complete sequence prefixed by AA would be AA1212.

  • Rumelhart networkThe network has:5 input units, one for each letter.3 output units, one for each number.30 hidden units.

    Each hidden unit is connected to itself and every other hidden unit.

  • Training StepsT1: Input unit corresponding to the first letter is 1 and other inputs are off.T2: Input unit corresponding to the second letter is 1 and other inputs are off.T3: The target output is the third character in the sequence.T4: The target output is the fourth character in the sequence.T5: The target output is the fifth character in the sequence.T6: The target output is the sixth character in the sequence.

    Output generation at different time steps.

  • Rumelhart Network ResultsAll hidden and output unit activations started at 0.2Network is trained on 20 sequences.Given first two characters, the rest of the characters were completed in five test cases.The completion could be achieved even if there were delays between the presentation of the first two letters.

  • Could the study of recurrent networks lead to studying other capabilites of memory?Yes, counting.

  • An example: Counting to three.

  • Evolving Virtual CreaturesKarl Sims (1994)ANN ControllersBoth body and mind evolved


View more >