PSY105 Neural Networks 5/5

5. “Function – Computation - Mechanism”

Lecture 1 recap

• We can describe patterns at one level of description that emerge due to rules followed at a lower level of description.

• Neural network modellers hope that we can understand behaviour by creating models of networks of artificial neurons.

Lecture 2 recap

• Simple model neurons– Transmit a signal of (or between) 0 and 1– Receive information from other neurons– Weight this information

• Can be used to perform any computation

Lecture 3 recap

• Classical conditioning is a simple form of learning which can be understood as an increase in the weight (‘associative strength’) between two stimuli (one of which is associated with an ‘unconditioned response’)

Lecture 4 recap

• A Hebb Rule for weight change between two neurons is:– Δ weight = activity 1 x activity 2 x learning rate constant

• In order to use this rule to associate two stimuli which are separated in time we need neuron activity associated with stimuli to persist in time.– This can be implemented as an ‘eligibility trace’

• I present you with a robot which uses a simple neural network to acquire classically conditioned responses. It can, for example, learn to associate a warning stimulus with an upcoming wall and hence turn around before it reaches the wall. Describe an experiment which you would do to test the details of how the robot learns. Say what you would do, and what aspect(s) of the robot's learning the results would inform you of, and why.

The problem of continuous time

Stimulus 1

Stimulus 2

Traces

Stimulus 1

Stimulus 2

Consequences of this implementation

• Intensity of CS stimulus• Duration of CS stimulus• Intensity of UCS stimulus• Duration of UCS stimulus• Separation in time of CS and UCS• The order in which the CS and UCS occur – (cf. Rescola-Wagner discrete time model)

We have designed an information processing system that learns associations

Trials

Asso

ciati

on

Sutton, R.S., Barto, A.G. (1990). Time-derivative models of pavlovian reinforcement. In Learning and Computational Neuroscience: Foundations of Adaptive Networks, M. Gabriel and J. Moore, Eds., pp. 497-537. MIT Press.

Without continued pairing

Trials

Asso

ciati

on

Stop

pai

ring

Without continued pairing -> extinction

Trials

Asso

ciati

on

Stop

pai

ring

Analysis of information processing systems

• Function (‘computational level’)• Computation (‘algorithmic level’)• Mechanism (‘implementational level’)

Marr, David (1982) Vision. San Francisco: W.H. Freeman

Marrian analysis of classical condtioning

• Function: learn to predict events based on past experience

• Computation: Stimuli evoke ‘eligibility traces’. Hebb Rule governs changes in weights [+ other additional assumptions which are always needed when you try and make a computational recipe]

• Mechanism: At least one response neuron, one unconditioned stimulus neuron and one neuron for each conditioned stimulus

Kim, J. J., & Thompson, R. F. (1997). Cerebellar circuits and synaptic mechanisms involved in classical eyeblink conditioning. Trends in Neurosciences, 20(4), 177-181.

Marrian analysis: a simple example

• Function

• Computation

• Mechanism

Theory < - > Experiments

Synthesis < - > Analysis

Our classical conditioning networks

Stimuli

Responses

CS1 UCSCS2

S-S link

Internal representation of the conditioned stimulus

The lab : using Marrian analysis to make predictions

Function

• What is the purpose of learning for an animal?– Does our model behave in a sensible (‘adaptive’)

way when it follows our rule?– Is the rule sufficient to explain animal learning?

• Test: think of a way you would want the model/robot to behave, test if it does

Computation• Intensity of CS stimulus• Duration of CS stimulus• Intensity of UCS stimulus• Duration of UCS stimulus• Separation in time of CS and UCS• The order in which the CS and UCS occur – (cf. Rescola-Wagner discrete time model)

• The learning rate • The rate of decay of the trace• The frequency of pairing

Mechanism

Stimuli

Responses

CS1 UCSCS2

S-S link

• What is your prediction? What will you do to the rule or the environment?

• How will you know if it has been confirmed or falsified?

ψπΩ