the reward factor in the control of action: a neurophysiological theory johan lauwereyns victoria...

The Reward Factor in the Control of Action:

A Neurophysiological Theory

Johan LauwereynsVictoria University of Wellington,

New Zealand

• How does the brain incorporate reward value in the control of action?

• How does the brain incorporate reward value in the control of action?

• Studied in monkeys using saccadic eye movement tasks

with asymmetrical reward schedule

Biased Saccade Task (BST)

Biased Saccade Task (BST)

Biased Saccade Task (BST)

Biased Saccade Task (BST)

Biased Saccade Task (BST)

Biased Saccade Task (BST)

Biased Saccade Task (BST)

Target position =

unpredictable

Biased Saccade Task (BST)

Reward association = known

Biased Saccade Task (BST)

Biased Saccade Task (BST)

Biased Saccade Task (BST)

Biased Saccade Task (BST)

No escape!

Biased Saccade Task (BST)

Biased Saccade Task (BST)

Biased Saccade Task (BST)

Saccade-related brain areas (macaque monkey)

FEF: frontal eye fieldSEF: supplementary eye fieldLIP: area LIP of parietal cortexCD: caudate nucleusSNr: substantia nigra pars reticulataSC: superior colliculusClbm: cerebellumSG: brainstem saccade generators

DA neuron responds to Reward & Reward Predictor

SignalNoise

Neuronal activity

Noise

Noise

Signal +

Signal +

Improved discrimination

Improved discrimination:Synergistic, multiplicative

• Sensory properties

• Non-linearly enhanced by reward

Response = Input * Reward Gain

Prefrontal cortex, parietal cortex

Superior colliculus

General increase

General increase:Prospective, additive

• Bias in anticipatory activity

• Linearly enhances sensory activity

Response = Input + Reward Bias

Prefrontal cortex, basal ganglia

Superior colliculus

Improved discrimination & General increase

Combination of both mechanisms

• Seen in all areas• Loops between FC, BG and SC• But most common in Superior Colliculus

Response = (Input * Reward Gain) + Reward Bias

On toward the oculomotor plant

Dopamine

DopamineExcitation

DopamineExcitation

Disinhibition

Synergistic,multiplicative

DopamineExcitation

Disinhibition

Prospective, additive

Synergistic,multiplicative

DopamineExcitation

Disinhibition