Slide 1

AbstractIn this presentation, I will rehearse the free-energy formulation of action and perception, with a special focus on the representation of uncertainty: The free-energy principle is based upon the notion that both action and perception are trying to minimise the surprise (prediction error) associated with sensory input. In this scheme, perception is the process of optimising sensory predictions by adjusting internal brain states and connections; while action is regarded as an adaptive sampling of sensory input to ensure it conforms to perceptual predictions (this is known as active inference). Both action and perception rest on an optimum representation of uncertainty, which corresponds to the precision of prediction error. Neurobiologically, this may be encoded by the postsynaptic gain of prediction error units. I hope to illustrate the plausibility of this framework using simple simulations of cued, sequential, movements. Crucially, the predictions driving movements are based upon a hierarchical generative model that infers the context in which movements are made. This means that we can temporarily confuse agents by changing the context (order) in which cues are presented. These simulations provide a (Bayes-optimal) simulation of contextual uncertainty and set-switching that can be characterised in terms of behaviour and electrophysiological responses. Interestingly, one can lesion the encoding of precision (postsynaptic gain) to produce pathological behaviours that are reminiscent of those seen in Parkinson's disease. I will use this as a toy example of how information theoretic approaches to uncertainty may help understand action selection and set-switching.Brain Meeting at the CHUVFree energy and affordanceKarl Friston

or precision and uncertainty, ansen seale (2005)

Objects are always imagined as being present in the field of vision as would have to be there in order to produce the same impression on the nervous mechanism - Hermann Ludwig Ferdinand von Helmholtz

Thomas BayesGeoffrey HintonRichard FeynmanFrom the Helmholtz machine to the Bayesian brain and self-organizationHermann Haken

Richard Gregory

Overview

Ensemble dynamicsEntropy and equilibriaFree-energy and surprise

The free-energy principlePerception and generative modelsHierarchies and predictive coding

PerceptionBirdsong and categorizationSimulated lesions and perceptual uncertainty

ActionCued reaching and affordance Simulated lesions and behavioral uncertainty

temperatureWhat is the difference between a snowflake and a bird?

Phase-boundary

a bird can move (to avoid surprises)

4

What is the difference between snowfall and a flock of birds?Ensemble dynamics, clumping and swarming

birds (biological agents) stay in the same place They resist the second law of thermodynamics, which says that their entropy should increaseThis means biological agents self-organize to minimise surprise. In other words, to ensure they occupy a limited number of states (cf homeostasis).

But what is the entropy?

entropy is just average surpriseLow surprise (we are usually here)High surprise (I am never here)

But there is a small problem agents cannot measure their surpriseBut they can measure their free-energy, which is always bigger than surprise

This means agents should minimize their free-energy. So what is free-energy??

What is free-energy?free-energy is basically prediction error

where small errors mean low surprisesensations predictions= prediction error

Free-energy is a function of sensations and a proposal density over hidden causes

and can be evaluated, given a generative model (Gibbs Energy) or likelihood and prior:

So what models might the brain use?

Action

External states in the worldInternal states of the agent (m)Sensations

More formally,

Backward(modulatory)Forward(driving)lateral

Hierarchal models in the brain

Adjust hypothesessensory inputBackward connections return predictionsby hierarchical message passing in the brain

prediction

Forward connections convey feedbackSo how do prediction errors change predictions?Prediction errorsPredictions

Backward predictionsForward prediction errorSynaptic activity and message-passing

David MumfordMore formally,cf., Predictive coding or Kalman-Bucy filtering

13Forward prediction errorBackward predictions

Low level macrocolumnhigh level macrocolumn

Superficial pyramidal cellsDeep pyramidal cells

Cortical layers

IIIIII

IV

V

VI

Dendritic spine

Presynaptic terminalsExcitatory (AMPA) receptorsModulatory (D1) receptorsInhibitory (GABAA) receptors13

Summary

Biological agents resist the second law of thermodynamics

They must minimize their average surprise (entropy)

They minimize surprise by suppressing prediction error (free-energy)

Prediction error can be reduced by changing predictions (perception)

Prediction error can be reduced by changing sensations (action)

Perception entails recurrent message passing in the brain to optimise predictions

Predictions depend upon the precision of prediction errorsOverview

Ensemble dynamicsEntropy and equilibriaFree-energy and surprise

The free-energy principlePerception and generative modelsHierarchies and predictive coding

PerceptionBirdsong and categorizationSimulated lesions and perceptual uncertainty

ActionCued reaching and affordance Simulated lesions and behavioral uncertainty

Making bird songs with Lorenz attractorsSyrinxVocal centre

time (sec)FrequencySonogram0.511.5

causal stateshidden states

102030405060-505101520prediction and error102030405060-505101520hidden statesBackward predictionsForward prediction error102030405060-10-505101520causal statesPredictive coding and message passing

stimulus0.20.40.60.82000250030003500400045005000time (seconds)

Perceptual categorization

Frequency (Hz)Song a

time (seconds)Song b

Song c

Hierarchical (itinerant) birdsong: sequences of sequencesSyrinxNeuronal hierarchy

Time (sec)Frequency (KHz)sonogram0.511.5

Frequency (Hz)perceptFrequency (Hz)no top-down messagestime (seconds)Frequency (Hz)no lateral messages0.511.5-40-200204060LFP (micro-volts)LFP-60-40-200204060LFP (micro-volts)LFP0500100015002000-60-40-200204060peristimulus time (ms)LFP (micro-volts)LFPSimulated lesions and false inference

no structural priorsno dynamical priors

Overview

Ensemble dynamicsEntropy and equilibriaFree-energy and surprise

The free-energy principlePerception and generative modelsHierarchies and predictive coding

PerceptionBirdsong and categorizationSimulated lesions and perceptual uncertainty

ActionCued reaching and affordance Simulated lesions and behavioral uncertainty

predictionFrom reflexes to action

action

dorsal horndorsal rootventral rootventral horn

22

Proprioceptive forward modelMotor commandsEasy inverse problemProprioceptive cuesVisual cuesVisual forward modelExteroceptionClassical reflex arc ProprioceptionSensorimotor contingencies and schema

23

Autonomous behaviour and prior beliefs:Lotka-Volterra dynamics: winnerless competition

Misha Rabinovich

Superior colliculus salienceParietal cortexfinger locationMotor cortexjoint positionsPremotor cortexaffordancePrefrontal cortexchanges in setStriatumset selectionMotoneurones

Active inference

Anatol Feldman

2526

Model and real worldJoint positions

Finger location

Target salienceActionChanges in joint positions

and affordanceSet switching

Hidden causes

Extrinsic locationsAction selection26

SN/VTAMesocortical DA projectionsNigrostriatal DA projectionsSuperior colliculusMesorhombencephalic pathwayParietal cortexMotor cortexPremotor cortexPrefrontal cortexStriatumMotoneuronesDopamine and precision

2700.511.5hidden causes20406080100120-0.5time00.511.5hidden causes20406080100120-0.5time00.511.5hidden causestime20406080100120-0.500.511.5hidden causes20406080100120-0.5time00.511.5hidden causestime20406080100120-0.500.511.5hidden causes20406080100120-0.5time00.511.5hidden causes20406080100120-0.5time00.511.5hidden causestime20406080100120-0.5-2-101

-2-10122-2-101

-2-10122-2-101

-2-10122-2-101

-2-10122-2-101

-2-10122-2-101

-2-10122-2-101

-2-10122-2-101

-2-10122

Uncertainty and perseveration -2-101-2-10122-2-101-2-10122-2-101-2-10122-2-101-2-10122

salience123456789300350400450500550cue onset (sec)millisecondsreaction times123456789300320340360380400420440cue onset (sec)millisecondsreaction times123456789300320340360380400420cue onset (sec)millisecondsreaction timesproprioceptionaffordanceLow DAHigh DALow DAHigh DAHigh DALow DA

Motor cortexPremotor cortexSuperior colliculus

Motor cortexPremotor cortexSuperior colliculus

SN/VTAMotor cortexPremotor cortexSuperior colliculusSN/VTASN/VTA

SN/VTAMotor cortexPremotor cortexSuperior colliculus

SN/VTAMotor cortexPremotor cortexSuperior colliculusUncertainly, delusions and confusion perseverationconfusion

Thank you

And thanks to collaborators:

Rick AdamsSven BestmannHarriet BrownJean DaunizeauLee HarrisonStefan KiebelJames KilnerJrmie MattoutKlaas Stephan

And colleagues:

Peter DayanJrn DiedrichsenPaul VerschureFlorentin Wrgtter

And many others