laboratory for computational cognitive neuroscience ...soma.mcmaster.ca/papers/cds2008/ashby -...

Laboratory for Computational Cognitive NeuroscienceUniversity of California, Santa Barbara

1) category learningA) ExperimentsB) TheoryB) Theory

2) categorization automaticityA) C t ti l i d lA) Computational neuroscience modelB) Tests of the model

A B

Except with many exemplars per categoryn

A B

ntat

ion

Ori

enO

Bar Width

A B

Categorization rule is

ntat

ion

easy to describe

Ori

en

ABar WidthA

Effective learning requires:

• no distractions

• active and effortful processing of feedback

h d i i f f db k i i i lBut the nature and timing of feedback is not critical

(cluster learning is possible)

A

B

Does this mammogram show a tumor?

i e is it in the categoryi.e., is it in the category “tumor” or the category

“nontumor”?nontumor ?

Tumor!

A

Categorization rule is enta

tion

gdifficult to describeO

rie

BBar Width

Effective learning requires:

• consistent feedback immediately after responsey p

• consistent mapping from category to response location

i f db k i• no active feedback processing

(no evidence that cluster learning is possible)

Rule Based Information IntegrationRule-Based Information-Integration

Bar Width Bar WidthBar WidthBar Width

Is the information-integration task inherently more difficult?

(Ashby, Alfonso-Reese, Turken, & Waldron, Psychological Review, 1998)

• explicit, logical-reasoning systemp g g y-- quickly learns explicit rules

• procedural- or habit-learning system-- slowly learns similarity-based rules

• simultaneously active in all tasks (at least initially)

Romo, Merchant et al.

Low Speed Cell High Speed Cell

Merchant et al. (1997, J. of Neurophysiology)

• logical reasoning system ki d i i• uses working memory and executive attention

• prefrontal cortex, anterior cingulate, head of the d t l th l ti l l di l caudate nucleus, thalamo-cortical loops, medial

temporal lobe structures

• Working memory & attentional switching component FROST (Ashby Ell Valentin & Casale component – FROST (Ashby, Ell, Valentin, & Casale, 2005, J. of Cognitive Neuroscience)

ACC = Anterior CingulatePFC = Lateral Prefrontal CortexMDN = Medial Dorsal Nucleus of the ThalamusGP = Globus PallidusCD = Head of the Caudate NucleusCD = Head of the Caudate NucleusVTA = Ventral Tegmental AreaSN = Substantia Nigra pars compactaHC = Hippocampus

Excitatory projection

19

Excitatory projectionInhibitory projectionDopamine projection

The Striatal Pattern Classifier (Ashby & Waldron, 1999)

PremotorA

VisualAreas

suaAreas

• Information-integration category learning should• Information-integration category learning shouldbe sensitive to feedback delay

• Rule-based category learning should not beiti t f db k d lsensitive to feedback delay

Maddox, Ashby, & Bohil (2003, JEP:LM&C)

Maddox, Ashby, & Bohil (2003, JEP:LM&C)

• Results identical with 2.5 and 10 sec delaysy

• RB results replicated at 4 increased levels of RB results replicated at 4 increased levels of difficulty

• Replication with a rule-based task that uses a conjunction rule?a conjunction rule?

(Note: Rule-based discriminability higher)

Rule-Based Information-IntegrationDimensional-4 (Dim4) Non-Dimensional-4 (Non-Dim4)

on on

Ori

enta

tio

Ori

enta

tio

Frequency Frequency

Maddox & Ing (2005, JEP:LM&C)

1.00

0.80

orre

ct *

0.60

ortio

n C

o

DelayImm

0.40Prop

o

0.20Dim4 Non-Dim4

Rule-Based Information-Integration

Maddox & Ing (2005, JEP:LM&C)

Feedback delay interferes with Feedback delay interferes with information-integration category learning, but not with rule-based category learningbut not with rule-based category learning.

Asaad, Rainer, & Miller, 2000; Hoshi, Shima, & Tanji, 1998; Merchant, Z i H d S li & R 1997 R M h t R i C

Single-cell recording studies

Zainos, Hernadez, Salinas, & Romo, 1997; Romo, Merchant, Ruiz, Crespo, & Zainos, 1995; White & Wise, 1999

Animal lesion experimentsAnimal lesion experiments Eacott & Gaffan, 1991; Gaffan & Eacott, 1995; Gaffan & Harrison, 1987; McDonald & White, 1993, 1994; Packard, Hirsch, & White, 1989; Packard & McGaugh 1992; Roberts & Wallis 2000McGaugh, 1992; Roberts & Wallis, 2000

Neuropsychological patient studiesA hb N bl Fil t W ld & Ell 2003 B & M d 1988 C lAshby, Noble, Filoteo, Waldron, & Ell, 2003; Brown & Marsden, 1988; Cools et al., 1984; Downes et al., 1989; Filoteo, Maddox, & Davis, 2001a, 2001b; Filoteo, Maddox, Ing, Zizak, & Song, in press; Filoteo, Maddox, Salmon, & Song 2005; Janowsky Shimamura Kritchevsky & Squire 1989; KnowltonSong, 2005; Janowsky, Shimamura, Kritchevsky, & Squire, 1989; Knowlton, Mangels, & Squire, 1996; Leng & Parkin, 1988; Snowden et al., 2001

Konishi et al., 1999; Lombardi et al., 1999; Nomura et al., in press; P ld k t l 2001 R t l 1997 R A d G b B k

Neuroimaging experiments

Poldrack, et al., 2001; Rao et al., 1997; Rogers, Andrews, Grasby, Brooks, & Robbins, 2000; Seger & Cincotta, 2002; Volz et al., 1997

Traditional cognitive behavioral experimentsTraditional cognitive behavioral experiments Ashby & Ell, 2002; Ashby, Ell, & Waldron, 2003; Ashby, Maddox, & Bohil, 2002; Ashby, Queller, & Berretty, 1999; Ashby, Waldron, Lee, & Berkman, 2001; Maddox Ashby & Bohil 2003; Maddox Ashby Ing & Pickering2001; Maddox, Ashby, & Bohil, 2003; Maddox, Ashby, Ing, & Pickering, 2004; Maddox, Bohil, & Ing, in press; Waldron & Ashby, 2001; Zeithamova& Maddox, in press

"As I write, my mind is not preoccupied with how my fingers form the letters; my attention is fixed simply on the thought the words express. But there was a time when the formation of the letters, as each one was written would have occupied my whole one was written, would have occupied my whole attention.”

Sir Charles Sherrington (1906)

“It has been widely held that although memory traces are at first formed in the cerebral cortex they are finally are at first formed in the cerebral cortex, they are finally reduced or transferred by long practice to subcortical levels” (p. 466)

Karl Lashley (1950) In search of the engram.

“Routine, automatic, or overlearned behavioral sequences however complex do not engage the PFC sequences, however complex, do not engage the PFC and may be entirely organized in subcortical structures” (p. 323) (p )

Joaquin Fuster (2001). The prefrontal cortex – an update.

Category CategorizationLearning Expertise

Patients with Basal Ganglia Dysfunction

(Parkinson’s disease, Impaired Unimpaired

Huntington’s disease)

P ti t ith t i Unimpaired if Patients with certain visual cortex lesions

(category specific agnosia)

pstimuli are perceived Impaired

(category-specific agnosia) normally?

n

A

tatio

nO

rien

B Wid h

O

BBar Width

A

AB

A

Ashby Ennis & Spiering (2007 Psych Review)Ashby, Ennis, & Spiering (2007, Psych Review)

Excitatory projection (glutamate)

Inhibitory projection (GABA)

Dopamine projectionDopamine projection

Ashby, Ennis, & Spiering (2007, Psych Review)

Izhikevich (2003, IEEE Trans. on Neural Networks)

( )[ ]bIuvvvvkvC trest +−−−=

&

& ))((( )[ ]uvvbau rest −−=&

If v ≥ vpeak then reset v to v = cpeakand reset u to u + d

C = 50, vrest = -80, vt = -25, a = .01,b = -20, vpeak = 40, c = -55, d = 150

)(dS ⎤⎡

Activation in striatal unit J at time t, denoted SJ(t) equals

[ ] [ ] ( )[ ],)(1)()()()(1)()()(, tStStStStStStInw

dttdS

JJSbaseJSMSJK

KJKJ −+−−−−⎥

⎦

⎤⎢⎣

⎡= ∑ εσγβ

where IK(t) is the input from visual cortical unit K at time t, and wK,iJ(n) is the strength of the synapse between cortical

i K d i di i ll J d ( ) i hiunit K and spine i on medium spiny cell J, and ε(t) is white noise.

Globus Pallidus: [ ]baseJGJJGJ GtGtGtS

dtdG

−−−= )()()()( βα [ ]baseJGJJGdt)()()( β

Thalamus: ),()()()( tTtTtGdt

tdTJTJJT

J βα −−=

P t A

[ ] [ ] [ ],)(1)()()()()(1)()()()(, tEtEtEtEtEtEtInvtT

dttdE

JJEbaseJEKEJKK

JKJEJ −+−−−−⎥

⎦

⎤⎢⎣

⎡+= ∑ εσγβα

Premotor Area:

dt K ⎦⎣

Excitatory projection (glutamate)

Inhibitory projection (GABA)

Dopamine projectionDopamine projection

Hebbian learning

reinforcement

learning

reinforcement learning

vK,J(n) = strength of synapse between visual cortical cell K and premotor cell J on trial n

presynaptic postsynaptic activation

Global Feedback Algorithm

LTPpresynaptic activation

postsynaptic activation (above NMDA threshold)

[ ] [ ])(1)()()()1( ,,, nvtEtInvnv JKNMDAJkvJKJK −−+=+ +θα

[ ] )()()( , tvtEtI JKJNMDAkv+−− θβ

LTD postsynaptic activation p y p(below NMDA threshold)

CorticalCortical--StriatalStriatal Learning Learning (3 factor)(3 factor)(reinforcement learning (reinforcement learning –– also a global learning algorithm)also a global learning algorithm)

LTP activation above NMDA threshold

dopamine above baseline (Correct Response)

(reinforcement learning (reinforcement learning also a global learning algorithm)also a global learning algorithm)

)()1( nwnw iJKiJK =+

NMDA threshold (Correct Response)

[ ] [ ] [ ][ ] [ ]

)(1)()()( ,, nwDnDtrtS iJKbaseNMDAiJKJw++ −−−+ θα

)()1( ,, nwnw iJKiJK +

[ ] [ ][ ] ),()(

)()()()(

,,

,,

nwtr

nwnDDtrtS

iJKiJKNMDAw

iJKbaseNMDAiJKJw

+

++

−−

−−−

θγ

θβ

activation below

dopamine below baseline (error)

LTD

activation below NMDA threshold activation above

NMDA threshold

Obtained Reward Predicted RewardIncreases with:

Obtained Reward – Predicted Rewardwhere obtained reward on trial n equals

⎪⎨

⎧+= receivedisfeedbacknoif

received isfeedback correct if0 1

R⎪⎩

⎨− received isfeedback error if

receivedisfeedback noif 1 0nR

∑−

=

−=1

1

)(n on trial Reward Predicted andn

ii

in ReC θ

=1i

Bayer & Glimcher (2005 Neuron) Dopamine Release in SPEEDBayer & Glimcher (2005, Neuron) Dopamine Release in SPEED

Obtained Reward – Predicted RewardObtained Reward Predicted Reward

1

0.8

0 4

0.6

opam

ine

0.2

0.4Do

01 101 201 301 401 501 601

Trial

n

A

tatio

nO

rien

B Wid h

O

BBar Width

1

0.9

ect With Striatal

0.7

0.8

tion

Cor

re

With StriatalNoise

Without

0.6

Prop

ort Without

StriatalNoise

0.4

0.5

1 2 3 4 5 6 7 8 9Block

Romo, Merchant et al.

1

nses Monkeys

SPEED

Res

pon SPEED

0.5

of L

ow Low High

port

ion

012 14 16 18 20 22 24 26 28 30

Prop

Cycles per SecondSpeed (mm/s)

Low Speed Cell High Speed Cell

Merchant et al. (1997)

Low Speed Cells High Speed Cells

Romo et al., 1997

(1997 J of Neuroscience)(1997 J of Neuroscience)(1997, J. of Neuroscience)(1997, J. of Neuroscience)

Lever press to tone

70 trials/day

18 days18 days

Striatal Response

Carelli et al. (1997, Journal of Neuroscience)

Carelli et al.

(i.e., RT > 10 s)

F d ll t d d

(2005, J. of Neuroscience)(2005, J. of Neuroscience)

Food pellet dropped into compartment

Minimum 30 s between trials

28 trials/day

17 days17 days

Injected with dopamine (D1)

antagonist

Munsell Color Patches – 3 Subjects – 1800 Trials

ABs A A

A ABghtn

ess

A

A

AB

B

Brig

B B B

Saturation

Model Performance

90

100ec

t

70

80

ent C

orre

50

60Per

ce

5 20 35 50 65 80 95 110

125

140

Block (N)( )

SPEEDNosofsky & Palmeri (1997)

Participant 1

2000

SPEEDy ( )

1400

1600

1800

2000

pons

e Ti

me

800

1000

1200

5 0 5 0 5 0 5 0 5 0

Mea

n R

es

5 20 35 50 65 80 95110 125 140

Block (N)

•fMRI

• Model automaticity development in:

-- neuropsychological populations

bj t d i fl f d-- subjects under influence of drugs

•Automaticity in rule-based tasks•Automaticity in rule-based tasks

• Two category learning systems

• Explicit, logical reasoning system-- Uses working memory & executive attention-- Frontal cortex

• Procedural learning system• Procedural learning system-- Striatum

• Learning systems train long-term cortical representations

Collaborators:Learning

Leola Alfonso Reese Michael Beran Robert Cook Leola Alfonso-Reese, Michael Beran, Robert Cook, Shawn Ell, Vince Filoteo, Todd Maddox, Alan

Pickering, David Smith, And Turken, Elliott Waldron, gmany others

AutomaticityAutomaticityJohn Ennis, Brian Spiering

Funding:Public Health Services Grant MH3760 2Public Health Services Grant MH3760-2