computational models of cognitive control (i)

Computational models of cognitive control (I)

Matthew BotvinickPrinceton Neuroscience Institute andDepartment of Psychology, Princeton University

Atkinson & Shiffrin, 1968


Structural elements

Short-term store

Sensory register

Long-term store


Structural elements

Short-term store

Sensory register

Long-term store

Control elements

Search / RetrievalTransfer to/from STS

Which register?Forward into sts?

SearchRehearsal

Baddeley, 1986/2007

Baddeley, 1986/2007

“Slave systems”

Shiffrin & Schneider, 1977

Norman & Shallice, 1986

Contention scheduling system

Norman & Shallice, 1986

Supervisory attentional system (SAS)

Contention scheduling system

< < < > < < <

-- Controlled (task-guided) attention: “attention for action” (Stroop)

-- Ignoring or inhibiting task-irrelevant stims/responses (Go/No-Go)

-- Manipulating information in working memory (N-Back)

-- Switching between tasks (Wisconsin Card Sort)

-- Planning / scheduling (Tower of London)

-- Navigating through extended, hierarchically structured tasks

“Executive/Cognitive Control”

AR

ZN

KZ

Q







GETTING WITH (AND STAYING WITH) THE PROGRAM


GETTING WITH (AND STAYING WITH) THE PROGRAM

Encoding / Formulation Maintenance Projection Updating

Task Context

Cohen, McClelland & Dunbar, 1990

< < < > < < <

Cohen, Servan-Schreiber & McClelland, AJP, 1992.

White & Wise, Exp Br Res, 1999

(See also: Assad, Rainer & Miller, 2000; Bunge, 2004; Hoshi, Shima & Tanji, 1998; Johnston & Everling, 2006; Wallis, Anderson & Miller, 2001; White, 1999…)

From Curtis & D’Esposito, TICS, 2003, after Funahashi et al., J. Neurophysiol,1989.

Questions…

-- What about manipulation in WM, etc? -- dynamics (switching, sequences)-- what controls control? (Homunculus)

Intermission

“A controlled process is a temporary sequenceof nodes activated under control of, andthrough attention by, the subject. Becauseactive attention by the subject is required,only one such sequence at a time may becontrolled without interference.”

Shiffrin & Schneider, 1977

Ridderinkhof et al., Science, 2004 (Based on Picard & Strick, Curr. Op. Biol., 2001)

Response override

Response override Underdetermined responding


Error commission


Error commission

Conflict

GREEN

GREEN

< < < > < < <

> > > > > > >

Botvinick, et al. (1999) Nature.

< < < > < < <

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Thompson-Schill et al., PNAS, 1997

High constraint: APPLE

Low constraint: BALL

Carter, Braver, Barch, Botvinick, Noll & Cohen, Science, 1998

Barch, et al. Cerebral Cortex, 2001 / Botvinick, Carter & Cohen, TICS, 2004

Ridderinkhoff et al., Science, 2004

Botvinick, et al. (2001) Psychological Review.

Carter, Braver, Barch, Botvinick, Noll & Cohen, Science, 1998

Gratton et al., JEPG, 1992

Low controlHigh conflict

High controlLow conflict

-0.05

0

0.05

0.1

0.15

0.2

scan 1 scan 2 scan 3 scan 4 scan 5

Scan within trial

MR

sig

na

l c

ha

ng

e (

%)

CcI

iI

Botvinick, et al. (1999) Nature.


Tzelgov, et al. (1992) Memory & Cognition.

High controlLow conflict


Carter, MacDonald, Botvinick et al. (2000) PNAS.


Barch, Braver, Sabb & Noll, JCN, 2000

Underdeterminedresponding

ResponseOverride

Yeung, Botvinick & Cohen, Psychological Review, 2004

Stuphorn, Taylor & Schall, Nature, 2000

Ito et al., Science, 2000ACC

SEF

Nakamura, Roesch & Olson, J. Neurophys. 2005

Curtis et al., Cereb. Ctx., 2005

Davis et al., J. Neurosci. 2005

Neutral Incongruent

44%

Control

Conflict

Gratton et al., JEPG, 1992

Tzelgov, et al. (1992) Memory & Cognition.

Mayr, Awh & Laurey, Nature Neuroscience, 2003

Ullsperger & Botvinick, PB&R, 2005 Kerns, et al. (2004) Science.

See also:Freitas, Bahar, Yang, and Banai, Psychological Science, 2007 Notebaert, Gevers, Verbruggen, & Liefooghe, Psychonomic Bulletin & Review, 2006

Freitas, Bahar, Yang, and Banai, Psychological Science, 2007

Kerns, et al. (2004) Science.

DiPellegrino, Ciaramelli & Ladavas, J. Cog. Neuro., 2007

Monitoring of action outcomes -- especially outcomes considered aversive or signaling reduction in reward

Gehring & Willoughby, Science, 2002Luu et al., Psychol. Sci., 2004 Niewenhuis et al., Cerebral Cortex, 2004Bush et al., PNAS, 2002Holroyd & Coles, Psychol. Rev., 2002

Use of outcome information to guide action selection

Matsumoto, et al. Science, 2003Bush, et al., PNAS, 2002Holroyd & Coles, Psychol. Rev. 2002Hadland, et al., J. Neurophysiol., 2003Kennerley, et al., Nature Neurosci., 2006

Action selection based on cost-benefit analysis

Rushworth, et al., TICS, 2004

Cost-benefit analysis might take effort into account

Walton, et al., J. Neurosci., 2003

Johansen & Fields, Nature Neuroscience, 2004

Glu antagonist Glu agonist(kynurenic acid) (homocysteic acid)

Jackson, Frost & Moghaddam, J. Neurochem., 2001

Toward an integrative account

Conflict is an outcome of action / strategy selection



Conflict is aversive (registers as a cost)




Conflict informs subsequent decision making




Conflict avoidance

Conflict informs subsequent decision making


task cue

strategy

stimulus task cue

strategyresponse

stimulus task cue

strategyresponse

conflict

Law of least mental effort:

All else being equal, actions will be chosen so as to minimize the demand for cognitive control (indexed by processing conflict).

Law of least effort (Hull): All else being equal, actions will be chosen so as to minimize the amount of work performed.

Macleod, Hunt & Mathews, Journal of Verbal Learning and Verbal Behavior, 1978

STAR ABOVE CROSS

“One basis for strategy selection: minimization of cognitive workload.”

-- Reichle, Carpenter & Just, Cog. Psychol., 2003.

3

90% switch 10% switch

Botvinick, CABN, 2007

Card

Card

Botvinick, CABN, 2007

Anticipatory skin conductance responses

Bechara, Damasio, Damasio, & Lee, Journal of Neuroscience, 1999.

0

0.02

0.04

0.06

0.08

0.1

0.12

Performance Decision

SCR (area)

High demand

Low demand

Botvinick & Rosen, Psych Res, in press

0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

Performance Decision

SCR (area)

High demand

Low demand


SCR ACC (Nagai, Critchley, Featherstone, Trimble, & Dolan, 2004)

ACC damage loss of effort- and IGT-related SCR (Naccache et al., 2005))

8

9

3

Deciding your pay…

X

6

1

9

3

4


$

2000 ms

2000 ms

2000-8000 ms

2000 ms

2000-8000 ms

2

4

8

9

3


X

6

1

9

3

4


$

2000 ms

2000 ms

2000-8000 ms

2000 ms

2000-8000 ms

X $

Botvinick, Huffstetler & McGuire, in press

Botvinick, Huffstetler & McGuire, in press

-0.4

-0.2

0.2

0.4

-0.2 0.2

ACC EffectD

LP

FC

Eff

ect

-0.4

-0.2

0.2

0.4

-0.2 0.2

ACC EffectD

LP

FC

Eff

ect

-0.4

-0.2

0.2

0.4

-0.2 0.2

ACC EffectD

LP

FC

Eff

ect

-0.4

-0.2

0.2

0.4

-0.2 0.2

ACC EffectD

LP

FC

Eff

ect

Conclusions

Conflict can be viewed as an index of the demand for control

The occurrence of conflict appears to be detected in the brain

Conflict detection appears to impact cognitive control

Conflict may also register as a cost

Tasks / strategies may be chosen so as to minimize conflict

Where does this put us?


Chipping away at the homunculus



But this is just about modulating task representations




How are task representations selected in the first place?





How are they sequenced?





How are they sequenced?

Dynamics (decision-making) and Learning

computational models of cognitive control (i)

Documents

tasks wisconsin card

action stroop

working memory n

shiffrin schneider

servanschreiber mcclelland

mcclelland dunbar