interactions between frontal cortex and basal ganglia in...

Cognitive Affective amp Behavioral Neuroscience2001 1 (2) 137-160

It is almost universally accepted that the prefrontalcortex (PFC) plays a critical role in working memoryeven though there is little agreement about exactly whatworking memory is or how else the prefrontal cortex con-tributes to cognition Furthermore it has long been knownthat the basal ganglia interact closely with the frontalcortex (eg Alexander DeLong amp Strick 1986) andthat damage to the basal ganglia can produce many of thesame cognitive impairments as damage to the frontal cor-tex (eg L L Brown Schneider amp Lidsky 1997 R GBrown amp Marsden 1990 Middleton amp Strick 2000b)This close relationship raises many questions regardingthe cognitive role of the basal ganglia and how it can bedifferentiated from that of the frontal cortex itself Arethe basal ganglia and frontal cortex just two undifferen-tiated pieces of a larger system Do the basal ganglia andthe frontal cortex perform essentially the same function butoperate on different domains of informationprocessingAre the basal ganglia an evolutionary predecessor to thefrontal cortex with the frontal cortex performing a moresophisticated version of the same function

We attempt to answer these kinds of questions by pre-senting a mechanistic theory and an implemented compu-

tational model of the contributions of the prefrontal cor-tex and basal ganglia to working memory We find that thesomewhat Byzantine nature of the anatomical loops con-necting the frontal cortex and the basal ganglia make goodcomputational sense in terms of a well-defined characteri-zation of working memory function Specifically we arguethat working memory requires rapid updating and robustmaintenance as achieved by a selective gating mechanism(Braver amp Cohen 2000 Cohen Braver amp OrsquoReilly 1996OrsquoReilly Braver amp Cohen 1999 OrsquoReilly amp Munakata2000) Furthermore although the frontal cortex and thebasal ganglia are mutually interdependent in our modelwe can nevertheless provide a precise division of labor be-tween these systems On this basis we can make a num-ber of specific predictions regarding the differential ef-fects of frontal versus basal ganglia damage on a varietyof cognitive tasks

We begin with a brief overview of working memoryhighlighting what we believe are the critical functionaldemands of working memory that the biological substratesof the frontal cortex and basal ganglia must subserve Weshow that these functional demands can be met by a se-lective gating mechanism which can trigger the updatingof some elements in working memory while others arerobustly maintained Building on existing biologicallybased ideas about the role of the basal ganglia in workingmemory (eg Beiser amp Houk 1998 Dominey 1995)we show that the basal ganglia are well suited for pro-viding this selective gating mechanism We then presenta neural network model that instantiates our ideas andperforms a working memory task that requires a selectivegating mechanism We also show that this network can ac-count for the role of the basal ganglia in sequencing tasks

137 Copyright 2001 Psychonomic Society Inc

Versions of this material provided partial fulfillment of masterrsquos the-ses for MJF and BL This work was supported by NIH Program Pro-ject Grant MH47566 NSF Grant IBN-9873492 and ONR GrantN00014-00-1-0246 We thank David Noelle Todd Braver Clay Hol-royd Jonathan Cohen Seth Herd and Marie Banich for helpful com-ments and discussion Correspondence concerning this article shouldbe addressed to R C OrsquoReilly Department of Psychology Universityof Colorado 345 UCB Boulder CO 80309 (e-mail oreillypsychcoloradoedu)

Interactions between frontal cortexand basal ganglia in working memory

A computational model

MICHAEL J FRANK BRYAN LOUGHRY and RANDALL C OrsquoREILLYUniversity of Colorado Boulder Colorado

The frontal cortex and the basal ganglia interact via a relatively well understood and elaborate sys-tem of interconnections In the context of motor function these interconnections can be understoodas disinhibiting or ldquoreleasing the brakesrdquo on frontal motor action plans The basal ganglia detect ap-propriate contexts for performing motor actions and enable the frontal cortex to execute such actionsat the appropriate time We build on this idea in the domain of working memory through the use of com-putational neural network models of this circuit In our model the frontal cortex exhibits robust ac-tive maintenance whereas the basal ganglia contribute a selective dynamic gating function that en-ables frontal memory representations to be rapidly updated in a task-relevant manner We apply themodel to a novel version of the continuous performance task that requires subroutine-like selectiveworking memory updating and compare and contrast our model with other existing models and theo-ries of frontal-cortexndashbasal-ganglia interactions

138 FRANK LOUGHRY AND OrsquoREILLY

We conclude by discussing the relationship between thismodel and other existing models of the basal-gangliandashfrontal-cortex system

WORKING MEMORY

Working memory can be defined as an active systemfor temporarily storing and manipulating informationneeded for the execution of complex cognitive tasks (Bad-deley 1986) For example this kind of memory is clearlyimportant for performing mental arithmetic (eg multi-plying 42 3 17)mdashone must maintain subsets of the prob-lem (eg 7 3 2) and store partial products (eg 14)while maintaining the original problem as well (eg 42and 17 see eg Tsung amp Cottrell 1993) It is also usefulin problem solving (maintaining and updating goals andsubgoals imagined consequences of actions etc) lan-guage comprehension (keeping track of many levels ofdiscourse using prior interpretations to correctly interpretsubsequent passages etc) and many other cognitive ac-tivities (see Miyake amp Shah 1999 for a recent survey)

From a neural perspective one can identify workingmemory with the maintenance and updating of informa-tion encoded in the active firing of neurons (activation-based memory see eg Fuster 1989 Goldman-Rakic1987) It has long been known that the PFC exhibits thiskind of sustained active firing over delays (eg FunahashiBruce amp Goldman-Rakic 1989 Fuster amp Alexander1971 Kubota amp Niki 1971 Miller Erickson amp Desi-mone 1996 Miyashita amp Chang 1988) Such findingssupport the idea that the PFC is important for active main-tenance of information in working memory

The properties of this activation-based memory can beunderstood by contrasting them with more long-term kindsof memories that are stored in the synaptic connections be-tween neurons (weight-based memory Cohen et al 1996Munakata 1998 OrsquoReilly et al 1999 OrsquoReilly amp Mu-nakata 2000) Activation-based memories have a num-ber of advantages relative to weight-based memories Forexample activation-based memories can be rapidly up-dated just by changing the activation state of a set of neu-rons In contrast changing weights requires structuralchanges in neural connectivity which can be much slowerAlso information maintained in an active state is directlyaccessible to other parts of the brain (ie as a constantpropagation of activation signals to all connected neu-rons) whereas synaptic changes only directly affect theneuron on the receiving end of the connection and thenonly when the sending neuron is activated In more fa-miliar terms activation-based memories are like sendinga message via broadcast radio signal whereas weight-based memories are like sending a letter in the mail

These mechanistic properties of activation-basedmemories coincide well with oft-discussed characteristicsof information maintained in working memory Specifi-cally working memory is used for processing because itcan be rapidly updated to reflect the ongoing products anddemands of processing and it is generally consciously ac-cessible and can be described in a verbal protocol (eg

Miyake amp Shah 1999) Furthermore the active nature ofworking memory provides a natural mechanism for cog-nitive control (also known as task-based attention) wheretop-down activation can influence processing elsewhereto achieve task-relevant objectives (Cohen Dunbar ampMcClelland 1990 Cohen amp OrsquoReilly 1996 OrsquoReillyet al 1999) Thus working memory and cognitive con-trol can be seen as two different manifestations of the sameunderlying mechanism of actively maintained informa-tion Of course these manifestations function in differ-ent ways in different tasks and thus are not the samepsychological construct but both can be subserved by acommon mechanism

However with these advantages of activation-basedmemories there are also concomitant disadvantages Forexample because these memories do not involve struc-tural changes they are transient and therefore do not pro-vide a suitable basis for long-term memories Also be-cause information is encoded by the activation states ofneurons the capacity of these memories scales as a func-tion of the number of neurons whereas the capacity ofweight-based memories scales as a function of the numberof synaptic connections which is much larger

Because of this fundamental tradeoff between activation-and weight-based memory mechanisms it makes sensethat the brain would have evolved two different special-ized systems to obtain the best of both types of memoryThis is particularly true if there are specific mechanisticspecializations that are needed to make each type of mem-ory work better There has been considerable discussionalong these lines of the ways in which the neural struc-ture of the hippocampus is optimized for subserving aparticular kind of weight-based memory (eg McClel-land McNaughton amp OrsquoReilly 1995 OrsquoReilly amp Mc-Clelland 1994 OrsquoReilly amp Rudy 2000 2001) Simi-larly this paper represents the further development of aline of thinking about the ways in which the frontal cor-tex is specialized to subserve activation-based memory(Braver amp Cohen 2000 Cohen et al 1996 OrsquoReilly et al1999) In the next section we will introduce a specificworking memory task that exemplif ies the functionalspecializations needed to support effective activation-based memories and we then proceed to explore how thebiology of the frontal-cortexndashbasal-ganglia system is spe-cialized to achieve these functions

Working Memory Functional DemandsThe AndashX version of the continuous performance task

(CPTndashAX) is a standard working memory task that hasbeen extensively studied in humans (Braver amp Cohen2000 Cohen et al 1997) The subject is presented withsequential letter stimuli (A X B Y) and is asked to detectthe specific sequence of an A followed by an X by push-ing the right button All other combinations (AndashY BndashXBndashY) should be responded to with a left button push Thistask requires a relatively simple form of working memorywhere the prior stimulus must be maintained over a delayuntil the next stimulus appears so that one can discrim-inate the target from the nontarget sequences We have

FRONTAL CORTEX AND BASAL GANGLIA INTERACTIONS IN WM 139

devised an extension of this task that places somewhatmore demands on the working memory system In thisextension which we call the 1ndash2ndashAX task (Figure 1) thetarget sequence varies depending on prior task demandstimuli (a 1 or a 2) Specifically if the subject last saw a1 the target sequence is AndashX However if the subjectlast saw a 2 the target sequence is BndashY1 Thus the task de-mand stimuli define an outer loop of active maintenance(maintenance of task demands) within which there can bea number of inner loops of active maintenance for theAndashX level sequences

The full 1ndash2ndashAX task places three critical functionaldemands on the working memory system

1 Rapid updating As each stimulus comes in it mustbe rapidly encoded in working memory (eg one-trialupdating which is not easily achieved in weight-basedmemory)

2 Robust maintenance The task demand stimuli (1 or2) in the outer loop must be maintained in the face of in-terference from ongoing processing of inner loop stim-uli and irrelevant distractors

3 Selective updating Only some elements of workingmemory should be updated at any given time while oth-ers are maintained For example in the inner loop As andXs (etc) should be updated while the task demand stim-ulus (1 or 2) is maintained

One can obtain some important theoretical leverageby noting that the first two of these functional demandsare directly in conflict with each other when viewed interms of standard neural processing mechanisms (Braveramp Cohen 2000 Cohen et al 1996 OrsquoReilly et al 1999OrsquoReilly amp Munakata 2000) Specifically rapid updat-ing can be achieved by making the connections betweenstimulus input and working memory representationsstrong but this directly impairs robust maintenancesince such strong connections would allow stimuli to in-

terfere with ongoing maintenance This conflict can be re-solved by using an active gating mechanism (Cohen et al1996 Hochreiter amp Schmidhuber 1997)

GatingAn active gating mechanism dynamically regulates

the influence of incoming stimuli on the working mem-ory system (Figure 2) When the gate is open stimulusinformation is allowed to flow strongly into the workingmemory system thereby achieving rapid updating Whenthe gate is closed stimulus information does not stronglyinfluence working memory thereby allowing robustmaintenance in the face of ongoing processing The com-putational power of such a gating mechanism has beendemonstrated in the LSTM model of Hochreiter andSchmidhuber (1997) which is based on error backprop-agation mechanisms and has not been related to brainfunction and in more biologically based models by Braverand Cohen (2000) and OrsquoReilly and Munakata (2000)

These existing biologically based models provide thepoint of departure for the present model These modelswere based on the idea that the neuromodulator dopaminecan perform the gating function by transiently strength-ening the efficacy of other cortical inputs to the frontalcortex Thus when dopamine release is phasically ele-vated as has been shown in a number of neural record-ings (eg Schultz Apicella amp Ljungberg 1993) work-ing memory can be updated Furthermore these modelsincorporate the intriguing idea that the same factors thatdrive dopamine spikes for learning (eg MontagueDayan amp Sejnowski 1996) should also be appropriate fordriving working memory updating Specifically work-ing memory should be updated whenever a stimulus trig-gers an enhanced prediction of future reward Howeveran important limitation of these models comes from thefact that dopamine release is relatively global large areas

1

L

A X

L R

B

L

X

L L

B

L2 R

Ytime

outer loopinner loops

Figure 1 The 1ndash2ndashAX version of the continuous performancetask Stimuli are presented one at a time in a sequence and thesubject must respond by pressing the right key (R) to the targetsequence otherwise a left key is pressed If the subject last sawa 1 the target sequence is an A followed by an X If a 2 was lastseen then the target is a B followed by a Y Distractor stimuli (eg3 C Z) may be presented at any point in a sequence and are tobe ignored Shown is an example sequence of stimuli and the cor-rect responses emphasizing the inner- and outer-loop nature ofthe memory demands (maintaining the task stimuli [1 or 2] is anouter loop whereas maintaining the prior stimulus of a sequenceis an inner loop)

SensoryInput

WorkingMemory 4920054

8675309Jenny Igot your

Myphone is4920054

4920054

Gating open closed

a) Update b) Maintain

Figure 2 Illustration of active gating When the gate is opensensory input can rapidly update working memory (eg allowingone to store a phone number) but when it is closed it cannotthereby preventing other distracting information (eg an irrele-vant phone number) from interfering with the maintenance ofpreviously stored information

492-0054 492-0054

492-0054

867-5309


of the PFC would therefore receive the same gating sig-nal In short a dopamine-based gating mechanism doesnot support the selectiveupdating functional demand listedabove where some working memory representations areupdated as others are being robustly maintained There-fore the present model explores the possibility that thebasal ganglia can provide this selective gating mechanismas will be described next

THE BASAL GANGLIA ASA SELECTIVE GATING MECHANISM

Our model is based directly on a few critical featuresof the basal-gangliandashfrontal-cortex system which we re-view here Figures 3 and 4 show schematic diagrams of

the relevant circuitry At the largest scale one can see anumber of parallel loops from the frontal cortex to thestriatum (also called the neostriatum consisting of thecaudate nucleus putamen and nucleus accumbens) to theglobus pallidus internal segment (GPi) or substantia nigrapars reticulata (SNr) and then on to the thalamus finallyprojecting back up in the frontal cortex (Alexander et al1986) The GPi and SNr circuits are functionally analo-gous (although they have different subcortical targets) sowe consider them as one functional entity Both the frontalcortex and the striatum also receive inputs from variousareas of the posteriorsensory cortex There are also otherpathways within the basal ganglia involving the externalsegment of the globus pallidus and the subthalamic nu-cleus that we see as having a role in learning but are not re-quired for the basic gating operation of the network theseother circuits only project through the GPiSNr to affectfrontal function

The critical aspect of this circuit for gating is that thestriatal projections to the GPiSNr and from the GPiSNrto the thalamus are inhibitory Furthermore the GPiSNrneurons are tonically active meaning that in the absenceof any other activity the thalamic neurons are inhibitedby constant firing of GPiSNr neurons Therefore whenthe striatal neurons fire they serve to disinhibit the thal-amic neurons (Chevalier amp Deniau 1990 Deniau ampChevalier 1985) As was emphasized by Chevalier andDeniau (and was suggested earlier by others NeafseyHull amp Buchwald 1978 Schneider 1987) this disinhi-bition produces a gating function (this is literally the termthey used) It enables other functions to take place butdoes not directly cause them to occur as a direct excita-tory connection would Chevalier and Deniau review arange of findings from the motor control domain show-ing that the activation of striatal neurons enables but doesnot directly cause subsequent motor movements

In short one can think of the overall influence of thebasal ganglia on the frontal cortex as ldquoreleasing thebrakesrdquo for motor actions and other functions Put anotherway the basal ganglia are important for initiating motormovements but not for determining the detailed proper-ties of these movements (eg Bullock amp Grossberg 1988Chevalier amp Deniau 1990 Hikosaka 1989 Passingham1993) Clearly this disinhibitory gating in the motor do-main could easily be extended to gating in the workingmemory domain Indeed this suggestion was made byChevalier and Deniau in generalizing their ideas fromthe motor domain to the cognitive one Subsequently sev-eral theories and computational models have includedvariations of this idea (Alexander Crutcher amp DeLong1990 Beiser amp Houk 1998 Dominey 1995 Dominey ampArbib 1992 Gelfand Gullapalli Johnson Raye amp Hen-derson 1997 Goldman-Rakic amp Friedman 1991 Houkamp Wise 1995) Thus we find a striking convergence be-tween the functionally motivated gating ideas we pre-sented earlier and similar ideas developed more from abottom-up consideration of the biological properties ofthe basal-gangliandashfrontal-cortex system

Figure 3 Schematic diagram of the major structures of thebasal ganglia and their connectivity with the frontal cortex in therat (A) and human (B) GP globus pallidus GPi GP internal seg-ment GPe GP external segment SNr substantia nigra parsreticulata EP entopeduncular nucleus STN subthalamic nu-cleus Numbers indicate total numbers of neurons within eachstructure Note that the EP in rodents is generally considered homologous to the GPi in primates and the GP in rodents is ho-mologous to the GPe in primates From ldquoBasal Ganglia Struc-ture and Computationsrdquo by J Wickens 1997 Network Compu-tation in Neural Systems 8 p R79 Copyright 1997 by IOPPublishing Ltd Reprinted with permission


Specifically in the context of the working memoryfunctions of the frontal cortex our model is based on theidea that the basal ganglia are important for initiating thestorage of new memories In other words the disinhibitionof the thalamocortical loops by the basal ganglia resultsin the opening of the gate into working memory result-ing in rapid updating In the absence of striatal firing thisgate remains closed and the frontal cortex maintains ex-isting information Critically the basal ganglia can pro-vide a selective gating mechanism because of the manyparallel loops Although the original neuroanatomicalstudies suggested that there are around five such loops(Alexander et al 1986) it is likely that the anatomy cansupport many more subloops within these larger scaleloops (eg Beiser amp Houk 1998) meaning that relativelyfine-grained selective control of working memory is pos-sible We will discuss this in greater detail later

To summarize at least at this general level it appearsthat the basal ganglia can provide exactly the kind of se-lective gating mechanism that our functional analysis ofworking memory requires Our detailed hypotheses re-garding the selective gating mechanisms of this systemare specified in the following sections

Details of Active Maintenanceand the Gating Mechanism

We begin with a discussion of the mechanisms of ac-tive maintenance in the frontal cortex which then con-strain the operation of the gating mechanism provided bythe basal ganglia

Perhaps the most obvious means of achieving thekinds of actively maintained neural firing observed inPFC neurons using basic neural mechanisms is to have re-

current excitation among frontal neurons resulting in at-tractor states that persist over time (eg Braver amp Cohen2000 Dehaene amp Changeux 1989 Moody Wise di Pel-legrino amp Zipser 1998 OrsquoReilly amp Munakata 2000Seung 1998 Zipser Kehoe Littlewort amp Fuster 1993)With this kind of mechanism active maintenance isachieved because active neurons will provide further acti-vation to themselves perpetuating an activity state Mostof the extant theoriesmodels of the basal ganglia role inworking memory employ a variation of this type of main-tenance where the recurrent connections are betweenfrontal neurons and the thalamus and back (Alexanderet al 1990 Beiser amp Houk 1998 Dominey 1995 Dom-iney amp Arbib 1992 Gelfand et al 1997 Goldman-Rakic amp Friedman 1991 Hikosaka 1989 Houk ampWise1995 Taylor amp Taylor 2000) This form of recurrence isparticularly convenient for enabling the basal ganglia toregulate the working memory circuits since thalamicdisinhibition would directly facilitate the flow of excita-tion through the thalamocortical loops

However it is unclear whether there are suff icientnumbers of thalamic neurons relative to frontal neuronsto support the full space of maintainable frontal represen-tations When a given thalamic neuron sends activation tothe frontal cortex to support maintenance its connectivitywould have to uniquely support one particular representa-tion or part thereof otherwise the specificity of the main-tained information would be lost Therefore the numberof thalamic neurons would have to be on the same orderas that of the frontal neurons unless frontal representationsare massively redundant Recurrent connectivity within thefrontal cortex itself avoids this problem Furthermore weare not aware of any definitive evidence suggesting that

Frontal Cortex

Striatum

reverberatory loops

inhibition

PosteriorCortex

GPiSNr

Thalamus

posterior anterior

Figure 4 The basal ganglia (striatum globus pallidus and thalamus) are inter-connected with the frontal cortex through a series of parallel loops Excitatory con-nections are in solid lines and inhibitory ones are in dashed lines The frontal cor-tex projects excitatory connections to the striatum which then projects inhibitionto the globus pallidus internal segment (GPi) or the substantia nigra pars reticu-lata (SNr) which again project inhibition to nuclei in the thalamus which are rec-iprocally interconnected with the frontal cortex Because GPi SNr neurons aretonically active they are constantly inhibiting the thalamus except when the stria-tum fires and disinhibits the thalamus This disinhibition provides a modulatoryor gating-like function


these loops are indeed critical for active maintenance (egshowing that frontal active maintenance is eliminatedwith selective thalamic lesions which is presumably afeasible experiment) Another issue with thalamocorti-cally mediated recurrent loops is that they would gener-ally require persistent disinhibition in the thalamus dur-ing the entire maintenance period (although see Beiser ampHouk 1998 for a way of avoiding this constraint) Forthese reasons we are inclined to think in terms of intra-cortical recurrent connectivity for supporting frontalmaintenance

Although it is intuitively appealing the recurrence-based mechanism has some important limitations stem-ming from the fact that information maintenance is en-tirely dependent on the instantaneous activation state ofthe network For example it does not allow for the frontalcortexrsquos exhibiting a transient stimulus-driven activationstate and then returning to maintaining some previouslyencoded informationmdashthe set of neurons that are mostactive at any given point in time will receive the strongestexcitatory recurrent feedback and will therefore be whatis maintained If a transient stimulus activates frontalneurons above the level of previously maintained infor-mation this stimulus transient will displace the prior in-formation as what is maintained

This survival-of-the-most-active characteristic is oftenviolated in recordings of prefrontal cortex neurons Forexample Miller et al (1996) observed that frontal neu-rons will tend to be activated transiently when irrelevantstimuli are presented while monkeys are maintainingother task-relevant stimuli During these stimulus tran-sients the neural firing for the maintained stimulus canbe weaker than that for the irrelevant stimulus After theirrelevant stimuli disappear the frontal activation revertsto maintaining the task-relevant stimuli We interpretthis data as strongly suggesting that frontal neurons havesome kind of intrinsic maintenance capabilities2 Thismeans that individual frontal neurons have some kind ofintracellular ldquoswitchrdquo that when activated provides theseneurons with extra excitatory input that enhances theircapacity to maintain signals in the absence of externalinput Thus this extra excitation enables maintainingneurons to recover their activation state after a stimulustransient After the actual stimulus ceases to support itsfrontal representation the neurons with intrinsic excita-tion will dominate

There are a number of possible mechanisms that couldsupport a switchable intrinsic maintenance capacity forfrontal neurons (eg Dilmore Gutkin amp Ermentrout1999 Durstewitz Seamans amp Sejnowski 2000b Fel-lous Wang amp Lisman 1998 Gorelova amp Yang 2000Lewis amp OrsquoDonnell 2000 Wang 1999) For exampleLewis and OrsquoDonnell report clear evidence that at leastin an anesthetized preparation prefrontal neurons exhibitbistabilitymdashthey have up and down states In the up stateneurons have a higher resting potential and can easilyfire spikes In the down state the resting potential is morenegative and it is more difficult to fire spikes A number

of different possible mechanisms are discussed by Lewisand OrsquoDonnell that can produce these effects includingselective activation of excitatory ion channels in the upstate (eg Ca2+ or Na+) or selective activation of inhibitoryK+ ion channels in the down state

Other mechanisms that involve intracellular switchingbut depend more on synaptic input have also been pro-posed These mechanisms take advantage of the proper-ties of the NMDA receptor which is activated both bysynaptic input and by postsynaptic neuron depolarizationand produces excitation through Ca2+ ions (DurstewitzKelc amp Gunturkun 1999 Durstewitz Seamans amp Sej-nowski 2000a Fellous et al 1998 Wang 1999) In themodel by Wang and colleagues a switchable bistabilityemerges as a result of interactions between NMDA chan-nels and the balance of excitatory and inhibitory inputsIn the model by Durstewitz and colleagues dopaminemodulates NMDA channels and inhibition to stabilize aset of active neurons and prevent interference from otherneurons (via the inhibition) Consistent with these modelswe think that recurrent excitation plays an importantmaintenance role in addition to a switchable intrinsicmaintenance capacity As we will discuss below recur-rent excitation can provide a ldquodefaultrdquo maintenance func-tion and it is also important for magnifying and sustain-ing the effects of the intrinsic maintenance currents

There are many complexities and unresolved issues withthese maintenance mechanisms For example althoughdopamine clearly plays an important role in some of thesemechanisms it is not clear whether tonic levels present inawake animals would be sufficient to enable these mech-anisms or whether phasic bursts of dopamine would berequired This can have implications for the gating mech-anism as we will discuss later Despite the tentative na-ture of the empirical evidence there are enough compu-tational advantages to a switchable intrinsic maintenancecapacity (as combined with a more conventional form ofrecurrent excitation) to compel us to use such a mecha-nism in our model Furthermore we think the neurophys-iological finding that working memory neurons recovertheir memory-based firing even after representing tran-sient stimuli (as was reviewed above) makes a compellingempirical case for the presence of such mechanisms

There are two primary computational advantages to aswitchable intrinsic maintenance capacity The first isthat it imparts a significant degree of robustness on activemaintenance as has been documented in several models(eg Durstewitz et al 2000a Fellous et al 1998) Thisrobustness stems from the fact that intrinsic signals are notdependent on network dynamics whereas spurious strongactivations can hijack recurrent maintenance mecha-nisms Second these intrinsic maintenance mechanismsby allowing the frontal cortex to represent both transientstimuli and maintained stimuli avoid an important catch-22 problem that arises in bootstrapping learning over de-lays (OrsquoReilly amp Munakata 2000 Figure 5) Brieflylearning that it is useful to maintain a stimulus can occuronly after that stimulus has been maintained in frontal rep-


resentations meaning that the gating mechanism mustlearn what to maintain on the basis of frontal represen-tations However if these frontal representations only re-flect stimuli that have already been gated in for mainte-nance the gating mechanism will not be able to detect thisstimulus as something to gate in until it is already gatedinto the frontal cortex However if the frontal representa-tions always reflect current stimuli as well as maintainedinformation this problem does not occur

Dynamic gating in the context of an intracellularmaintenance switch mechanism amounts to the activationand deactivation of this switch Neurons that participatein the maintenance should have the switch turned on andthose that do not should have the switch turned off Thiscontrasts with other gating models developed in the con-text of recurrent activation-based maintenance which re-quired gating to modulate the strength of input weights intothe frontal cortex (eg Braver amp Cohen 2000 OrsquoReillyamp Munakata 2000) or the strength of the thalamocorti-cal recurrent loops (eg Beiser amp Houk 1998 Dominey1995 Gelfand et al 1997) Therefore we propose thatthe disinhibition of the thalamocortical loops by the basalganglia results in the modulation of the intracellularswitch Specifically we suggest that the activation of theLayer 4 frontal neurons that receive the excitatory pro-jection from the thalamus (or the equivalent cell types inLayer 3 in motor areas of the frontal cortexmdashwe will justuse the Layer 4 notation for convenience) is responsiblefor modulating intracellular ion channels on the neuronsin other layers (which could be in either Layers 2ndash3 or 5ndash6) that are ultimately responsible for maintaining theworking memory representations

In our model we further specify that the intracellularswitch is activated when a neuron is receiving strong ex-

citatory input from other areas (eg stimulus input) inaddition to the Layer 4 input and it is deactivated if theLayer 4 input does not coincide with other strong excita-tory input Otherwise the switch just stays in its previousstate (and is by default off ) This mechanism works wellin practice for appropriately updating working memoryrepresentations and could be implemented through theoperation of NMDA channels that require a conjunctionof postsynaptic depolarization and synaptic input (neu-rotransmitter release) Alternatively such NMDA chan-nels could also activate other excitatory ion channels viasecond messengers or other voltage-gated channels coulddirectly mediate the effect so we are at present unsure asto the exact biological mechanisms necessary to imple-ment such a rule Nevertheless the overall behavior of theion channels is well specified and could be tested with ap-propriate experiments

Finally more conventional recurrent excitation-basedmaintenance is important in our model for establishing aldquodefaultrdquo propensity of the frontal cortex to maintain in-formation Thus if nothing else has been specif icallygated on in a region of the frontal cortex (ie if no otherneurons have a specific competitive advantage owing tointracellular maintenance currents) the recurrent con-nectivity will tend to maintain representations over timeanyway However any new stimulus information willeasily displace this kind of maintained information andit cannot compete with information that has been specif-ically gated on This default maintenance capacity is im-portant for ldquospeculativerdquo trial-and-error maintenance ofinformation the only way for a learning mechanism todiscover whether it is important to maintain something isif it actually does maintain it and then it turns out to beimportant Therefore having a default bias to maintain is

SensoryInput

WorkingMemory A

A

learning

notrecog-nized

a) learning from random gating

b) does not transfer to later trials

gatenotopened

GatingControl

Figure 5 Illustration of the catch-22 problem that occurs when the gatingmechanism learns on the basis of maintained working memory representationsand those representations can become activated only after the gating mecha-nism fires for a given stimulus (A) Learning about a stimulus A presented ear-lier and maintained in the frontal cortex which is based on initially random ex-ploratory gating signals will be between the maintained representations andthe gating controller (B) When this stimulus is later presented it will not acti-vate the working memory representations until the gate is opened but the gatehas only learned about this stimulus from the same working memory repre-sentations which are not activated

A B


useful However this default maintenance bias is overrid-den by the active gating mechanism allowing learning tohave full control over what is ultimately maintained

To summarize in our model active maintenance oper-ates according to the following set of principles

1 Stimuli generally activate their correspondingfrontal representations when they are presented

2 Robust maintenance occurs only for those stimulithat trigger the intracellular maintenance switch (as a re-sult of the conjunction of external excitation from othercortical areas and Layer 4 activation resulting from basal-ganglia-mediated disinhibition of the thalamocorticalloops)

3 When other stimuli are being maintained those rep-resentations that did not have the intracellular switch ac-tivated will decay quickly following stimulus offset

4 However if nothing else is being maintained recur-rent excitation is sufficient to maintain a stimulus untilother stimuli are presented This ldquodefaultrdquo maintenanceis important for learning by trial and error what it is rel-evant to maintain

Additional Anatomical ConstraintsIn this section we will discuss the implications of a few

important anatomical properties of the basal-gangliandashfrontal-cortex system First we will consider the conse-quences of the relative sizes of different regions in thebasal-ganglia frontal cortex pathway Next we will ex-amine evidence that can inform the number of differentseparately gatable frontal areas Finally we will discussthe level of convergence and divergence of the loops

A strong constraint on understanding basal gangliafunction comes from the fact that the GPi and SNr havea relatively small number of neuronsmdashthere are approx-imately 111 million neurons in the human striatum (Foxamp Rafols 1976) whereas there are only 160000 in theGPi (Lange Thorner amp Hopf 1976) and a similar num-ber in the SNr This means that whatever information isencoded by striatal neurons must be vastly compressed oreliminated on its way up to the frontal cortex This con-straint coincides nicely with the gating hypothesis Thebasal ganglia do not need to convey detailed content in-formation to the frontal cortex instead they simply needto tell different regions of the frontal cortex when to up-date As we noted in the context of motor control damageto the basal ganglia appears to affect initiation but not thedetails of execution of motor movementsmdashpresumablynot that many neurons are needed to encode this gatingor initiation information

Given this dramatic bottleneck in the GPiSNr onemight wonder why there are so many striatal neurons inthe first place We think this is also sensible under thegating proposal In order for only task-relevant stimuli toget updated (or an action initiated) via striatal firing theseneurons need to fire only for a very specific conjunctionof environmental stimuli and internal context represen-tations (as conveyed through descending projections fromthe frontal cortex) This context specificity of striatal fir-

ing has been established empirically (eg Schultz Api-cella Romo amp Scarnati 1995) and is an important partof many extant theoriesmodels (eg Amos 2000 Beiseramp Houk 1998 Berns amp Sejnowski 1996 Houk amp Wise1995 Jackson amp Houghton 1995 Wickens 1993 Wick-ens Kotter amp Alexander 1995) Thus many striatal neu-rons are required to encode all of the different specificconjunctions that can be relevant Without such conjunc-tive specificity there would be a risk that striatal neuronswould fire for inappropriate subsets of stimuli For exam-ple the 1 and 2 stimuli should be maintained separatelyfrom the other stimuli in the 1ndash2ndashAX task but this is notlikely to be true of other tasks Therefore striatal neuronsshould encode the conjunction of the stimulus (1 or 2) to-gether with some representation of the 1ndash2ndashAX task con-text from the frontal cortex If the striatum instead em-ployed a smaller number of neurons that just respondedto stimuli without regard to task context (or other simi-lar kinds of conjunctions) confusions between the manydifferent implications of a given stimulus would resultNote that by focusing on conjunctivity in the striatum wedo not mean to imply that there is no conjunctivity in thefrontal representations as well (eg S C Rao Rainer ampMiller 1997 Watanabe 1992) frontal conjunctive rep-resentations can be useful for maintaining appropriatelycontextualized information

Another constraint to consider concerns the numberof different subregions of the frontal cortex for which thebasal ganglia can plausibly provide separate gating con-trol Although it is impossible to determine any preciseestimates of this figure even the very crude estimates weprovide here are informative in suggesting that gating oc-curs at a relatively fine-grained level Fine-grained gatingis important for mitigating conflicts where two repre-sentations require separate gating control and yet fallwithin one gating region An upper limit estimate is pro-vided by the number of neurons in the GPiSNr which isroughly 320000 in the human as was noted previouslyThis suggests that the gating signal operates on a regionof frontal neurons instead of individually controllingspecific neurons (and assuming that the thalamic areasprojecting to the frontal cortex are similarly sized arguesagainst the notion that the thalamocortical loops them-selves can maintain detailed patterns of activity)

An interesting possible candidate for the regions ofthe frontal cortex that are independently controlled by thebasal ganglia are distinctive anatomical structures con-sisting of interconnected groups of neurons called stripes(Levitt Lewis Yoshioka amp Lund 1993 Pucak LevittLund amp Lewis 1996) Each stripe appears to be isolatedfrom the immediately adjacent tissue but interconnectedwith other more distal stripes forming a cluster of inter-connected stripes Furthermore it appears that connectiv-ity between the PFC and the thalamus exhibits a similaralthough not identical kind of discontinuous stripelikestructuring (Erickson amp Lewis 2000) Therefore itwould be plausible that each stripe or cluster of stripesconstitutes a separately controlled group of neurons


each stripe can be separately updated by the basal gan-glia system Given that each stripe is roughly 02ndash04 32ndash 4 mm in size (ie 04ndash16 mm2 in area) one can makea rough computation that the human frontal cortex (hav-ing roughly one fourth of the approximately 140000 mm2

surface area of the entire cortex Douglas amp Martin1990) could have over 20000 such stripes (assumingthat the stripes found in monkeys also exist in humanswith similar properties) If the thalamic connectivity werewith stripe clusters and not individual stripes this fig-ure would be reduced by a factor of around five In eithercase given the size of the GPi and SNr there would besome degree of redundancy in the per stripe gating sig-nal at the GPiSNr level Also note that the 20000 (or4000 for stripe clusters) figure is for the entire frontalcortex so the proportion located in the PFC (and thus in-volved in working memory function) would be smallerFurther evidence consistent with the existence of suchstripelike structures comes from the finding of isocodingmicrocolumns of neighboring neurons that all encoderoughly the same information (eg having similar direc-tional coding in a spatial delayed response task S G RaoWilliams amp Goldman-Rakic 1999)

The precise nature of the inputs and outputs of theloops through the basal ganglia can have implications forthe operation of the gating mechanism From a compu-tational perspective it would be useful to control eachstripe by using a range of different input signals from thesensory and frontal cortex (ie broad convergence of in-puts) to make the gating appropriately context specificIn addition it is important to have input from the currentstate of the stripe that is being controlled since this wouldaffect whether this stripe should be updated or not Thisimplies closed loops going through the same frontal re-gion Data consistent with both of these connectivity pat-terns has been presented (see Graybiel amp Kimura 1995and Middleton amp Strick 2000a for reviews) Althoughsome have taken mutually exclusive positions on thesetwo patterns of connectivity and the facts are a matter ofconsiderable debate both patterns are mutually compat-ible from the perspective of our model Furthermoreeven if it turns out that the cortical projections to the stria-tum are relatively focused context sensitivity in gatingcan be achieved via context-sensitive frontal input repre-sentations In other words the context sensitivity of gat-ing could come either from focused context-sensitive in-puts to the striatum or from broad sensory inputs that areintegrated by the striatum itself One particularly intrigu-ing suggestion is that the convergence of inputs fromother frontal areas may be arranged in a hierarchical fash-ion providing a means for more anterior frontal areas(which may represent higher level more abstract taskgoalinformation) to appropriately contextualize more poste-rior areas (eg supplementary and primary motor areasGobbel 1997) This hierarchical structure is reflected inFigure 4

There are two aspects of the basal ganglia connectiv-ity that we have not yet integrated into our model and

thus stand as challenges for future work First the basalganglia circuits through the thalamus also project to pos-terior cortex areas (the inferior temporal and parietalcortex) in addition to the frontal cortex (eg Middletonamp Strick 2000a) It is thus possible that gating occurs inthese areas as well but it is not clear that they are essen-tial for robust working memory function (eg Constan-tinidis amp Steinmetz 1996 Miller et al 1996) Thereforewe are not sure what functional role these connectionsplay Second striatal neurons receive a substantial pro-jection from the same thalamic areas that they disinhibit(eg McFarland amp Haber 2000) This projection hasbeen largely ignored in computational and theoreticalmodels of the basal ganglia but could have important im-plications For example such a projection would quicklyinform striatal neurons about exactly which frontal re-gions were actually updated and could thus provide use-ful constraints on the allocation of subsequent updatingto unused regions

To summarize anatomical constraints are consistentwith the selective gating hypothesis by suggesting thatthe basal ganglia interacts with a large number of distinctregions of the frontal cortex We hypothesize that thesedistinct stripe structures constitute separately gated col-lections of frontal neurons extending the parallel loopsconcept of Alexander et al (1986) to a much finer grainedlevel (see also Beiser amp Houk 1998) Thus it is possibleto maintain some information in one set of stripes whileselectively updating other stripes

Learning and the Role of DopamineImplicit in our gating model is that the basal ganglia

somehow know when it is appropriate to update workingmemory representations To avoid some kind of ho-munculus in our model we posit that learning is essentialfor shaping the striatal firing in response to task de-mands This dovetails nicely with the widely acknowl-edged role that the basal ganglia and the neuromodula-tor dopamine play in reinforcement learning (eg Barto1995 Houk Adams amp Barto 1995 Schultz Dayan ampMontague 1997 Schultz Romo et al 1995) Howeverour work on integrating learning mechanisms with thebasal ganglia selective gating model is still in progressTherefore our current model presented in this paper useshand-wired representations (ie ones causing the stria-tum to fire only for task-relevant stimuli) to demonstratethe basic gating capacity of the overall system

In addition to shaping striatal neurons to fire at theright time through stimulus-specific phasic firing do-pamine may also play an important role in regulating theoverall excitability of striatal neurons in a tonic mannerThe gating model places strong demands on these ex-citability parameters because striatal neurons need to begenerally silent while still being capable of firing whenthe appropriate stimulus and contextual inputs are presentThis general silence which is a well-known property ofstriatal neurons (eg Schultz Apicella et al 1995) canbe accomplished by having a relatively high effective


threshold for firing (either because the threshold itself ishigh or because they experience more inhibitory currentsthat offset excitation) However if this effective thresholdis too high striatal neurons will not be able to fire whenthe correct circumstances arise Therefore it is likelythat the brain has developed specialized mechanisms forregulating these thresholds The effects of Parkinsonrsquosdisease which results from a tonic loss of dopamine in-nervation of the basal ganglia together with neurophys-iological data showing dopaminergic modulation of dif-ferent states of excitability in striatal neurons (egGobbel 1995 Surmeier amp Kitai 1999 C J Wilson 1993)all suggest that dopamine plays an important part in thisregulatory mechanism

The Motor-ControlndashWorking-MemoryContinuum

We have emphasized that our view of the basal gangliainteractions with the frontal cortex builds on existingideas regarding these interactions in the context of motorcontrol Specifically both the initiation of a motor act andthe updating of working memory representations requirestriatal firing to disinhibit or gate frontal cortex represen-tations Although we have discussed motor control andworking memory as two separable functions it is proba-bly more useful to think in terms of a continuum betweencognitive working memory and motor control functionsFor example one can think of the neurons in premotor orsupplementary motor areas as maintaining a motor con-trol plan that guides a sequence of basic motor move-ments (eg Shima amp Tanji 1998 Wise 1985) This planwould need to be maintained over the duration of the se-quence and can thus be considered a working memory rep-resentation Thus the line between working memory andmotor control is fuzzy indeed this ambiguity providesuseful insight as to why both motor control and workingmemory are colocalized within the frontal cortex

Summary The Division of LaborBetween Frontal Cortex and Basal Ganglia

Before describing our model in detail and by way ofsummary we return to the fundamental question posed atthe outset of this paper What is the nature of the divisionof labor between the frontal cortex and the basal gangliaIn light of all the foregoing information we offer the fol-lowing concise summary of this division of labor Thefrontal cortex uses continuously firing activations to en-code information over time in working memory (or on ashorter time scale to execute motor actions) and the basalganglia fires only at very select times to trigger the updat-ing of working memory states (or initiate motor actions) inthe frontal cortex

Furthermore we can speculate as to why it wouldmake sense for the brain to have developed this division oflabor in the first place Specifically one can see that theuse of continuously firing activation states to encode in-formation is at odds with the need to f ire only at very spe-cific times Therefore the brain may have separated these

two systems to develop specialized mechanisms support-ing each For example striatal neurons must have a rela-tively high effective threshold for firing and it can bedifficult to regulate such a threshold to ensure that firinghappens when appropriate and not when it is not appro-priate The dopaminergic neuromodulation of these neu-rons and its control by descending projections from thestriatum may be important specializations in this regardFinally we do not mean to claim that all striatal neuronsfire only in a punctate manner others exhibit sustaineddelay period activations (eg Alexander 1987 SchultzRomo et al 1995) We think that these reflect sustainedfrontal activations not an intrinsic maintenance capabil-ity of striatal neurons themselves Similarly punctate fir-ing in cortical neurons especially in motor output areascould be a reflection of gating signals from the basalganglia

THE 1ndash2ndashAX MODEL

We have implemented the ideas outlined above in acomputational model of the 1ndash2ndashAX task This modeldemonstrates how the basal ganglia can provide a selec-tive gating mechanism by showing that the outer-loopinformation of the task demand stimuli (1 or 2) can be ro-bustly maintained while the inner-loop information (AB etc) is rapidly updated Furthermore we show that ir-relevant distractor stimuli are ignored by the model eventhough they transiently activate their frontal representa-tions In addition the model demonstrates that the samemechanisms that drive working memory updating alsodrive the motor responses in the model

The Mechanics of the ModelThe model is shown in Figure 6 The units in the

model operate according to a simple point neuron func-tion using rate-coded output activations as implementedin the Leabra framework (OrsquoReilly 1998 OrsquoReilly ampMunakata 2000) There are simulated excitatory synap-tic input channels and inhibitory input is computedthrough a simple approximation to the effects of inhibitoryinterneurons There is also a constant leak current that rep-resents the effects of K+ channels that are always openand the maintenance frontal neurons have a switchableexcitatory ion channel that is off by default (See the Ap-pendix for the details and equations) The modelrsquos repre-sentations were predetermined but the specific weightswere trained using the standard Leabra error-driven andassociative (Hebbian) learning mechanisms to achievetarget activations for every step in the sequence In a morerealistic model the representations would not be prede-termined but rather would develop as a function of thelearning mechanisms this shortcut was simply used as aconvenient way of achieving a desired set of representa-tions to test the basic sufficiency of our ideas about thegating mechanism

For simplicity every layer in the model has been or-ganized into three different stripes where a stripe corre-


sponds to an individually updatable region of the frontalcortex as was discussed previously The rightmost stripein each layer represents the outer-loop task demand infor-mation (1 or 2) The middle stripe represents informationmaintained at the inner-loop sequence level (A or B)The leftmost stripe represents stimuli that actually trig-ger an action response (X or Y) To clarify and simplifythe motor aspects of the task we have a response only atthe end of an inner-loop sequence (ie after an X or Y)instead of responding L for all the preceding stimuli Allthese other responses should be relatively automaticwhereas the response after the X or the Y requires takinginto account all the information maintained in workingmemory so it is really the task-critical motor response

We describe the specific layers of the model (whichmatch those shown in Figure 4 as was discussed previ-ously) in the course of tracing a given trial of input Firsta stimulus is presented (activated) in the input layer Everystimulus automatically activates its corresponding frontalrepresentation located in the PFC_Maint layer of themodel This layer represents cortical Layers 2ndash3 and 5ndash6

(without further distinguishing these layers although it ispossible there are divisions of labor between them) and iswhere stimulus information is represented and main-tained The other frontal layer is PFC_Gate which repre-sents the gating action of cortical Layer 4 (we will returnto it in a moment)

If the input stimulus has been recognized as importantfor task performance as a result of as-yet-unimplementedlearning experiences (which are represented in the modelthrough hand-set enhanced weight values) it will acti-vate a corresponding unit in the striatum layer This ac-tivation of the high-threshold striatal unit is the criticalstep in initiating the cascade of events that leads to main-taining stimuli in working memory via a process of ldquore-leasing the brakesrdquo or disinhibiting the thalamic loopsthrough the frontal cortex Note that these striatal unitsin the model encode conjunctions of maintained informa-tion in the frontal cortex (1 or 2 in this case) and incom-ing stimulus information (A B X or Y) Although notcomputationally essential for this one task these con-junctions reflect our theorizing that striatal neurons need to

Figure 6 Working memory model with basal-ganglia-mediated selective gating mechanism The network structure isanalogous to Figure 4 but with the prefrontal cortex (PFC) has been subdivided into maintenance (PFC_Maint) andgating (PFC_Gate) layers Three hierarchically organized stripes of the PFC and basal ganglia are represented as thethree columns of units within each layer each stripe is capable of being independently updated The rightmost task(Tsk) stripe encodes task-level information (ie 1 or 2) The middle sequence (seq) encodes sequence-level informationwithin a task (ie A or B) The leftmost action (act) stripe encodes action-level information (ie responding to the X orY stimulus and actually producing the left or right output in PFC) Non-task-relevant inputs (eg 3 C Z) are also pre-sented and the model ignores themmdashthat is they are not maintained


encode conjunctions in a high-threshold manner to avoidtask-inappropriate stimulus activation The frontal rep-resentations are also necessarily conjunctive in their de-tection of the combination of stimuli that trigger a responseaction the stimulus maintenance representations couldalso be more conjunctive as well even though it is notstrictly necessary for this one task

Once a striatal unit fires it inhibits the globus pallidusunit in its corresponding stripe which has to this pointbeen tonically active and inhibiting the corresponding thal-amus unit Note the compression of the signal from thestriatum to the globus pallidus as was discussed aboveThe disinhibition of the thalamic unit opens up the re-current loop that flows from the PFC_Maint units to thethalamus and back up to the PFC_Gate layer Note thatthe disinhibited thalamic unit will only get activated ifthere is also descending activation from PFC_Maint unitsAlthough this is always the case in our model it wouldnot be true if a basal ganglia stripe got activated (disin-hibited) that did not correspond to an area of frontal ac-tivation this property may be important for synchronizingfrontal and basal ganglia representations during learning

The effect of thalamic firing is to provide general ac-tivation to an entire stripe of units in the PFC_Gate layerThese frontal units cannot fire without this extra thala-mic activation but they also require excitation from unitsin the PFC_Maint layer which are responsible for select-ing the specific gate unit to activate Although this is con-figured as a simple one-to-one mapping between main-tenance and gating frontal units in the model the realsystem could perform important kinds of learning hereto fine-tune the gating mechanism Finally the activationof the gating unit controls the switchable excitatory ionchannels in the frontal maintenance units For those main-tenance units within a stripe that receive both input fromthe current input stimulus and the gating activation theexcitatory ion channels are opened Maintenance unitsthat get only the gating activation but not stimulus inputhave their ion channels closed This mechanism providesa means of updating working memory by resetting pre-viously active units that are no longer receiving stimulusinput while providing sustained excitatory support forunits that do have stimulus input

An Example SequenceFigure 7 shows an example sequence of 2ndashBndashCndashY as

processed by the model The first stimulus presented isthe task contextmdashin this case it is Task 2 the BndashY de-tection task Because the striatum detects this stimulus asbeing task relevant (via the 2 striatal unit) it inhibits thetask globus pallidus unit which then disinhibits the cor-responding thalamus unit This disinhibition enables thethalamus to then become excited via descending projec-tions from the frontal cortex The thalamic activationthen excites the PFC_Gate unit that also receives activa-tion from the PFC_Maint layer resulting in the activationof the excitatory ion channel for the 2 frontal unit in thePFC_Maint layer

Next the B input activates the 2B conjunctive striatalunit which detects the combination of the 2 task main-tained in the frontal cortex and the B stimulus input Thisresults in the firing of the sequence stripe and maintenanceof the B stimulus encoding in the frontal cortex Note thatthe 2 has been maintained as the B stimulus was beingprocessed and encoded into active memory owing to thefact that these items were represented in different stripesin the frontal cortex This demonstrates the principle ofselective gating which is central to our model

The next stimulus is a C distractor stimulus this is notdetected as important for the task by the striatum (ie allstriatal units remain subthreshold) and is thus not gatedinto robust active maintenance (via the intrinsic ion chan-nels) Note that despite this lack of gating the C repre-sentation is still activated in the PFC_Maint frontal cortexlayer as long as the stimulus is present However whenthe next stimulus comes in (the Y in this case) the C acti-vation decays quickly away

Finally the Y stimulus is important because it triggersan action The 2Y striatal unit enables firing of the R2unit in the PFC layers this is a conjunctive unit that de-tects the conjunction of all the relevant working memoryand input stimuli (2ndashBndashY in this case) for triggering onekind of R output response (the other R conjunction wouldbe a 1ndashAndashX) This conjunctive unit then activates the basicR motor response in a manner consistent with observedfrontal recordings (eg Hoshi Shima amp Tanji 2000)Thus the same basal-ganglia-mediated disinhibitory func-tion supports both working memory updating and motorresponse initiation in this model

Although it is not represented in this example the modelwill maintain the 2 task signal over many inner-loop se-quences (until a different task input is presented) becausethe inner-loop updating is selective and therefore doesnot interfere with maintenance of the outer-loop task in-formation

SummaryTo summarize the model illustrates how the frontal

cortex can maintain information for contextualizing motorresponses in a task-appropriate fashion while the basalganglia trigger the updating of these frontal representa-tions and the initiation of motor responses

DISCUSSION

We have presented a theoretical framework and an im-plemented neural network model for understanding howthe frontal cortex and the basal ganglia interact in provid-ing the mechanisms necessary for selective working mem-ory updating and robust maintenance We have addressedthe following central questions in this paper

1 What are the specific functional demands of work-ing memory

2 What is the overall division of labor between thefrontal cortex and the basal ganglia in meeting these func-tional demands


3 What kinds of specialized mechanisms are presentin the frontal cortex to support its contributions to work-ing memory

4 What aspects of the complex basal ganglia circuitryare essential for providing its functionality

Our answers to these questions are as follows1 Working memory requires robust maintenance (in the

face of ongoing processing other distractor stimuli andother sources of interference) but also rapid selective up-dating where some working memory representations canbe quickly updated while others are robustly maintained

2 The frontal cortex provides maintenance mecha-nisms whereas the basal ganglia provide selective gatingmechanisms that can independently switch the mainte-

nance mechanisms on or off in relatively small regions ofthe frontal cortex

3 Frontal cortex neurons have intrinsic maintenancecapabilities via persistent excitatory ion channels thatgive maintained activation patterns the ability to persistwithout stimulus input This allows frontal neurons to always encode stimulus inputs while only maintainingselected stimuli which is otherwise difficult using onlyrecurrent excitatory attractor mechanisms Recurrentconnections play an additional maintenance role and areimportant for trial-and-error learning about what is im-portant to maintain

4 The disinhibitory nature of the basal ganglia effecton the frontal cortex is important for achieving a modu-

Figure 7 An example sequence in the model (2ndashBndashCndashY) (A) Task Context 2 is presented The striatum detects this stimulus as rel-evant and disinhibits the task stripe of the thalamus allowing PFC_Gate to become active causing the task number to be maintainedin PFC_Maint (B) The next stimulus is B which the striatum detects in conjunction with Task Context 2 (from the PFC) via the 2Bunit The sequence stripe of the thalamus is then disinhibited and B is gated into PFC_Maint while Task Context 2 remains activeowing to persistent ionic currents This demonstrates selective gating (C) A distractor stimulus C is presented and because the stria-tum has not built up relevant associations to this stimulus all units are subthreshold The thalamus remains inhibited by the tonicallyactive globus pallidus and C is not maintained in the PFC (D) Stimulus Y is presented and the striatum detects the conjunction ofit and the task context via the 2Y unit The thalamus action level stripe is disinhibited which activates conjunctive units in the frontalcortex (R2) that detect combinations of maintained and input stimuli (2ndashBndashY ) These frontal units then activate the R response in theprimary motor area (M1)


latory or gating-like effect Striatal neurons must have ahigh effective threshold and selective conjunctive rep-resentations (combining maintained frontal goaltask in-formation with incoming stimuli) to fire only under spe-cific conditions when updating is required Although thisconjunctivity requires large numbers of neurons the stri-atal signal is collapsed down into a small number of globuspallidus neurons consistent with the idea that the basalganglia is important for determining when to do some-thing but not the details of what to do The organizationof this basal ganglia circuitry into a large number of par-allel subcircuits possibly aligned with the stripe structuresof the frontal cortex is essential for achieving a selectivegating signal that allows some representations to be up-dated while others are maintained

In the remaining sections we discuss a range of is-sues including the following a comparison between ourmodel and other theories and models in the literature theunique predictions made by this model and more gener-ally how the model relates to existing literature on thecognitive effects of basal ganglia damage and the limi-tations of our model and future directions for our work

OTHER THEORIES AND MODELSOF FRONTAL-CORTEXmdashBASAL-GANGLIA

FUNCTION

We begin with an overview of general theories of theroles of the frontal cortex and basal ganglia system froma neuropsychological perspective and then review arange of more specific computational theoriesmodels Wethen contrast the present model with the earlier dopamine-based gating mechanisms

General TheoriesOur discussion is based on a comprehensive review of

the literature on both the frontal cortex and the basalganglia and on their relationship by Wise Murray andGerfen (1996) They summarize the primary theories offrontal-cortexndashbasal-ganglia function according to fourcategories attentional set shifting working memory re-sponse learning and supervisory attention We cover thesetheories in turn and then address some further issues

Attentional set shifting The attentional-set-shiftingtheory is supported in part by deficits observed from bothfrontal and basal ganglia damagemdashfor example patientsperform normally on two individual tasks separately butwhen required to switch dynamically between the twothey make significantly more errors than do normals(eg R G Brown amp Marsden 1990 Owen et al 1993)This is exactly the kind of situation in which our modelwould predict deficits resulting from basal ganglia dam-age indeed the 1ndash2ndashAX task was specifically designedto have a task-switching outer loop because we think thisspecifically taps the basal ganglia contribution

Furthermore we and others have argued extensivelythat the basic mechanism of working memory function is

integral to most of the cognitive functions attributed to thefrontal cortex system (eg Cohen et al 1996 Munakata1998 OrsquoReilly et al 1999 OrsquoReilly amp Munakata 2000)For example the robust maintenance capacity of thekinds of working memory mechanisms we have devel-oped are necessary to maintain activations that focus at-tention in other parts of the brain on specific aspects ofa task and to maintain goals and other task-relevant pro-cessing information In short one can view our model asproviding a specific mechanistic implementation of theattentional-set-shifting idea (among other things)

This general account of how our model could addressattentional-set-shifting data is bolstered by specif icmodeling work using our earlier dopamine-based gatingmechanism to simulate the monkey frontal lesion data of Dias Robbins and Roberts (1997 OrsquoReilly NoelleBraver amp Cohen 2001) The dopamine-based gatingmechanism was capable of inducing task switching infrontal representations so that damage to the frontal cor-tex resulted in slowed task switching Moreover we wereable to account for the dissociation between dorsal andorbital frontal lesions observed by Dias et al in terms oflevel of abstractness of frontal representations instead ofinvoking entirely different kinds of processing for theseareas Thus we demonstrated that an entirely working-memory-based model augmented with a dynamic gat-ing mechanism and some assumptions about the organi-zation of frontal representations could account for datathat were originally interpreted in very different func-tional terms (ie attentional task shifting and overcom-ing previous associations of rewards)

Working memory It is clear that our account is con-sistent with the working memory theory but aside froma few papers showing the effects of caudate damage onworking memory function (Butters amp Rosvold 1968Divac Rosvold amp Szwaracbart 1967 Goldman amp Ros-vold 1972) not much theorizing from a broad neuro-psychological perspective has focused on the specif icrole of the basal ganglia in working memory Thus wehope that the present work will help to rekindle interestin this idea

Response learning This theory is closely associatedwith the ideas of Passingham (1993) who argued that thefrontal cortex is more important for learning (specificallylearning about appropriate actions to take in specific cir-cumstances) than for working memory Certainly thereis ample evidence that the basal ganglia are important forreinforcement-based learning (eg Barto 1995 Houket al 1995 Schultz et al 1997 Schultz Romo et al1995) and we think that learning is essential for avoidingthe (often implicit) invocation of a homunculus in theo-rizing about executive function and frontal control How-ever we view learning within the context of the workingmemory framework In this framework frontal learningis about what information to maintain in an active stateover time and how to update it in response to task demandsThis learning should ensure that active representations


have the appropriate impact on overall task performanceboth by retaining useful information and by focusing at-tention on task-relevant information

Supervisory attention The supervisory attention the-ory of Norman and Shallice (Norman amp Shallice 1986Shallice 1988) is essentially that the supervisory atten-tion system (SAS) controls action by modulating the op-eration of the contention scheduling (CS) system whichprovides relatively automatic input output response map-pings As is reviewed in Wise et al (1996) other re-searchers (but not Shallice) have associated the SASwith the frontal cortex and the CS system with the basalganglia However this mapping is inconsistent with theinability of the basal ganglia to directly produce motoroutput or other functions without frontal involvement in-stead as we and Wise et al argue the basal ganglia shouldbe viewed as modulating the frontal cortex which is theopposite of the SASCS framework

Behavior-guiding rule learning Wise et al (1996)proposed their own overarching theory of the frontal-cortexndashbasal-ganglia system which is closely related toother ideas (Fuster 1989 Owen et al 1993 Passingham1993) They propose that the frontal cortex is importantfor learning new behavior-guiding rules (which amountto sensoryndashmotor mappings in the simple case) whereasthe basal ganglia modulate the application of existingrules as a function of current behavioral context and re-inforcement factors Thus as in our model they think ofthe basal ganglia as having a modulatory interaction withthe frontal cortex Furthermore they emphasize that thefrontal-cortexbasal-ganglia system should not be viewedas a simple motor control system but rather should becharacterized as enabling flexible dynamic behavior thatcoordinates sensory and motor processing However theydo not provide any more specif ic biologically basedmechanisms for how this function could be carried outand how in general the frontal cortex and basal gangliaprovide this extra flexibility We consider our theory andmodel as an initial step toward developing a mechanisticframework that is generally consistent with these overallideas

Process versus content The ideas above can all becharacterized as describing different types of processing(switching maintaining modulating attention learn-ing) In contrast theories of posterior cortex tend tofocus on content such as representing object identity(ldquowhatrdquo) in the ventral visual pathway versus spatial lo-cation (ldquowhererdquo) or vision-for-action in the dorsal path-way Can we also provide a content-based explanation ofwhat is represented in the frontal-cortexbasal-gangliasystem In areas of the frontal cortex more directly asso-ciated with motor control the content is obviously aboutmuscles and sequences or plans of motor movements Inthe monkey prefrontal cortex attempts to incorporate thewhat where distinctions from the posterior cortex (egF A W Wilson Scalaidhe amp Goldman-Rakic 1993) havebeen questioned (S C Rao et al 1997) and the humanneuroimaging data is also controversial (eg Nystrom

et al 2000) The original working memory theory of Bad-deley (1986) posited a content-based distinction betweenvisuospatial and verbal information which has met withsome support from neuroimaging studies showing aleftright (verbalvisuospatial) organization of frontal cor-tex (Smith amp Jonides 1997) although this is also not al-ways consistently found (eg Nystrom et al 2000)

As was mentioned previously we have recently pro-posed that it might be more useful to think of the organi-zation of frontal content in terms of level of abstractness(OrsquoReilly et al 2001) Furthermore we suggest that thefrontal cortex represents a wide variety of content that isalso encoded in the posterior cortex (but this content canbe robustly maintained only in the frontal cortex) to-gether with sensoryndashmotor mappings plans and goalsthat may be uniquely represented in the frontal cortex(OrsquoReilly et al 1999) Thus more posterior and inferiorareas have more concrete specif ic representationswhereas more anterior and dorsal areas have more ab-stract representations In the domain of motor control itis known that more anterior areas contain progressivelymore abstract representations of plans and sequencesmdasha similar progression can occur with more stimulus-basedrepresentations as well This notion f its well with theideas of Fuster (1989) who suggested that the frontal cor-tex sits at the top of a hierarchy of sensoryndashmotor map-ping pathways where it is responsible for bridging thelongest temporal gaps This hierarchy can continue withinthe frontal cortex with more anterior areas concernedwith ever longer time scales and more abstract plans andconcepts (see Christoff amp Gabrieli 2000 and KoechlinBasso Pietrini Panzer amp Grafman 1999 for recent evi-dence consistent with this idea) The notion that the frontalcortex represents behavior-guiding rules (Wise et al1996) can be similarly fit within this overall paradigmby assuming that such rules are organized according todifferent levels of abstraction as well Furthermore theframework of Petrides (1994) which involves a distinctionbetween simple maintenance versus more complex pro-cessing can be recast as differences in levels of abstractionof the underlying representations (ie simple maintenanceinvolves concrete representations of stimuli whereas pro-cessing involves more abstract representations of goalsplans and mappings)

Computational TheoriesModelsPerhaps the dominant theme of extant computational

models of the basal ganglia is that they support decisionmaking andor action selection (eg Amos 2000 Beiseramp Houk 1998 Berns amp Sejnowski 1996 Houk amp Wise1995 Jackson amp Houghton 1995 Kropotov amp Etlinger1999 Wickens 1993 Wickens et al 1995 see BeiserHua amp Houk 1997 and Wickens 1997 for recent re-views) These selection models reflect a convergence be-tween the overall idea that the basal ganglia are somehowimportant for linking stimuli with motor responses andthe biological fact that striatal neurons are inhibitory andshould therefore inhibit each other to produce selection


effects Thus the basal ganglia could be important forselecting the best linkage between the current stimulus +context and a motor response using inhibitory competi-tion so that only the best match will ldquowinrdquo However allof these theories suffer from the finding that striatal neu-rons do not appear to inhibit each other (Jaeger Kita ampWilson 1994) One possible way of retaining this overallselection model is to have the inhibition work indirectlythrough dopamine modulation via the indirect pathwayconnections with the subthalamic nucleus as was pro-posed by Berns and Sejnowski (1996) This model maybe able to resolve some inconsistencies between the slicestudy that did not find evidence of lateral inhibition(Jaeger et al 1994) and in vitro studies that do find suchevidence (see Wickens 1997 for a discussion of the rele-vant data) the slice preparation does not retain this largerscale indirect pathway circuitry which may be providingthe inhibition This model is also generally consistent withthe finding that dopamine appears to be important for es-tablishing an independence of neural firing in the basalganglia that is eliminated with damage to the dopaminesystem (Bergman et al 1998)

At least some of the selection models also discuss thedisinhibitory role of the basal ganglia and suggest that theend result is the initiation of motor actions or workingmemory updating (eg Beiser amp Houk 1998 Dominey1995) The selection idea has also been applied in thecognitive domain by simulating performance on the Wis-consin card sorting task (WCST Amos 2000) In thismodel the striatal units act as match detectors betweenthe target cards and the current stimulus and are modu-lated by frontal attentional signals When an appropriatematch is detected a corresponding thalamic neuron isdisinhibited and this is taken as the networkrsquos responseAlthough this model does not capture the modulatory na-ture of the basal gangliarsquos impact on the frontal cortex anddoes not speak directly to the involvement of the basalganglia in working memory it nevertheless provides aninteresting demonstration of normal and impaired cogni-tive performance on the WCST task using the selectionframework

Our model is generally consistent with these selectionmodels insofar as we view the striatum as important fordetecting specific conditions for initiating actions or up-dating working memory As we emphasized earlier thisdetection process must take into account contextual (egprior actions goals task instructions) information main-tained in the frontal cortex to determine whether a givenstimulus is task relevant and if so which region of thefrontal cortex should be updated Thus the basal gangliaunder our model can be said to be performing the selec-tion process of initiating an appropriate response to agiven stimulus (or not)

Perhaps the closest model to our own is that of Beiserand Houk (1998) which is itself related to that of Dominey(1995) and is based on the theoretical ideas set forth byHouk and Wise (1995) This model has basal ganglia dis-

inhibition resulting in the activation of recurrent cortico-thalamic working memory loops to maintain items in astimulus sequence Their maintenance mechanism in-volves a recurrent bistability in the cortico-thalamic loopswhere a phasic disinhibition of the thalamus can switchthe loop from the inactive to the active state dependingon a calcium channel rebound current They also mentionthat the indirect path through the subthalamic nucleuscould potentially deactivate these loops but do not imple-ment this in the model They apply this model to a simplesequence-encoding task involving three stimuli (A B C)presented in all possible orders They show that for someparameter values the network can spontaneously (with-out learning) encode these sequences using unique acti-vation patterns

There are a number of important differences betweenour model and the Beiser and Houk (1998) model Firstas we discussed earlier their use of recurrent loops foractive maintenance incurs some diff iculties that areavoided by the intracellular maintenance mechanismsemployed in our model For example they explicitly sep-arate the frontal neurons that encode stimulus inputs andthose that maintain information which means that theirnetwork would suffer from the catch-22 problem men-tioned previously if they were to try to implement alearning mechanism for gating information into workingmemory Furthermore this separation constrains them topostulate direct thalamic activation resulting from striataldisinhibition (via the calcium rebound current) becausethe frontal neurons that project descending connectionsto the thalamus are not otherwise activated by stimuli Incontrast our model has the thalamus being activated bydescending frontal projections as is consistent with avail-able data showing that disinhibition alone is insufficientto activate the thalamus (Chevalier amp Deniau 1990)

Perhaps the most important difference is that theirmodel does not actually implement a gating mechanismbecause they do not deal with distractor stimuli and itseems clear that their model would necessarily activate aworking memory representation for each incoming stim-ulus The hallmark of a true gating mechanism is that itselectively updates only for task-relevant stimuli as de-fined by the current context This is the reason that ourmodel deals with a task domain that requires multiplehierarchical levels of maintenance and gating whereastheir sequencing task requires only maintenance of the im-mediately prior stimulus so that their model can succeedby always updating Furthermore their model has no pro-visions or apparent need for a learning mechanismwhereas this is a central if presently incompletely imple-mented aspect of our model

To demonstrate that our model can also explain the roleof the basal ganglia in sequencing tasks we applied ourarchitecture to a motor sequencing task that has beenshown in monkeys to depend on the basal ganglia (Mat-sumoto Hanakawa Maki Graybiel amp Kimura 1999) Inthis task two different sequences are trainedmdasheither 1 2


3 or 1 3 2mdashwhere the numbers represent locations oflights in a display Monkeys are trained to press buttonsin the positions of these lights The key property of thesesequences is that after the second step in the sequencethe third step is completely predictable Thus it should beresponded to faster which is the case in intact monkeysbut not in monkeys with basal ganglia impairments Weshowed that the network can learn to predict the third stepin the sequence by encoding in the striatum a conjunctionbetween the prior step and the onset of the third stimulusand therefore produce an output more rapidly The sametarget representation kind of learning as that used in the1ndash2ndashAX model was used to ldquotrainrdquo this network As aresult of this learningrsquos producing stronger representa-tions the network became better able to produce this pre-diction on the third step This enabled the network to re-produce the basic finding from the monkey studies afaster reaction time to the third step in the sequence (Fig-ure 8) We anticipate being able to provide a more com-putationally satisfying sequencing model when we de-velop the learning aspect of our model in a more realisticfashion

Finally there are a number of other computationalmodels that focus mainly on the PFC without a detailedconsideration of the role of the basal ganglia (Dehaeneamp Changeux 1989 1991 Guigon Dorizzi Burnod ampSchultz 1995 Moody et al 1998 Seung 1998 Tanakaamp Okada 1999 Zipser 1991) Most of these modelslack a true gating mechanism even though some have aldquogaterdquo input that is additive and not modulatory as re-quired by a true gating mechanism (eg Moody et al1998) We argue that such models will suffer from the in-ability to dynamically shift between rapid updating androbust maintenance For example the Moody et al modelrequired 10 million training trials to acquire a simple de-

layed matching-to-sample task with distractors we arguethat this was because the network had to learn a delicatebalance between updating and maintenance via additivenetwork weights This is consistent with computationalcomparisons of gated versus nongated memory models(Hochreiter amp Schmidhuber 1997) Other prefrontal-based models that were discussed earlier incorporate in-trinsic bistability as in our model (Durstewitz et al 1999Durstewitz et al 2000a Fellous et al 1998 Wang 1999)but these models lack explicit gating circuitry as imple-mented by the basal ganglia in our model

To summarize we see the primary contributions of thepresent work as linking the functionalcomputationallevel analysis of working memory function in terms of aselective gating mechanism with the underlying capaci-ties of the basal-gangliandashfrontal-cortex system Althoughthere are existing models that share many properties withour own our emphasis on the gating function is novelWe have also provided a set of specific ideas motivatedagain by functionalcomputational considerations aboutactive maintenance in terms of persistent ionic channelsand how these could be modulated by the basal ganglia

Relationship to theDopamine-Based Gating Models

As was noted above the present model was developedin the context of existing dopamine-based gating modelsof frontal cortex (Braver amp Cohen 2000 Cohen et al1996 OrsquoReilly et al 1999 OrsquoReilly amp Munakata 2000)The primary difference between these models at thefunctional level is that the basal ganglia allow for selec-tive updating whereas dopamine is a relatively globalneuromodulator that would result in updating large re-gions of the frontal cortex at the same time In tasks thatdo not require this selective updating however we think

Figure 8 Settling time in the sequential network for the second and third steps inthe sequence The third step is faster owing to the ability of the network to predict itbetter

30-

35-

40-

45-

50-

55-

60-

0 1 2 3 4 5 6 7 8 9 10

Set

tling

Cyc

les

Training Trials

2nd step

3rd step


that the two models would behave in a similar fashionoverall We will test this idea explicitly after we have de-veloped the learning mechanism for the basal gangliamodel by replicating earlier studies that have used thedopamine-based model

Despite having a high level of overall functional sim-ilarity these two models clearly make very different pre-dictions regarding the role of dopamine in working mem-ory Perhaps the most important difference is that thedopamine-based gating mechanism is based on a coinci-dence between the need to gate information into workingmemory and differences in level of expected rewardSpecifically dopamine bursts are known to occur for un-expected rewards and critically for stimuli that havebeen previously predictive of future rewards (eg Mon-tague et al 1996 Schultz et al 1993) Because it willby definition be rewarding to maintain stimuli that needto be maintained for successful task performance itmakes sense that dopamine bursts should occur for suchstimuli (and computational models demonstrate that thisis not a circular argument even though it may sound likeone Braver amp Cohen 2000 OrsquoReilly amp Munakata 2000)However this coincidence between reward predictionand the need to gate into working memory may not al-ways hold up In particular it seems likely that after a taskbecomes well learned rewards will no longer be unex-pected especially for intermediate steps in a chain ofworking memory updates (eg as required for mentalarithmetic) The basal ganglia gating mechanism canavoid this problem because in this model dopamine isonly thought to play a role in learning after expertise isachieved striatal neurons can be triggered directly fromstimuli and context without any facilitory boost fromdopamine being required

It remains possible that both dopamine and the basalganglia work together to trigger gating For examplebroad dopamine-based gating may be important duringinitial phases of learning a task and then the basal gan-glia play a dominant role for more well learned tasks Onepiece of data consistent with such a scenario is the find-ing that in anesthetized animals dopamine can shift pre-frontal neurons between two intrinsic bistable states (Lewisamp OrsquoDonnell 2000) However this finding has not beenreplicated in awake animals so it is possible that normaltonic dopamine levels are sufficient to allow other activa-tion signals (eg from Layer 4 activation driven by thal-amic disinhibition) to switch bistable modes (as hypoth-esized in the present model) In short further empiricalwork needs to be done to resolve these issues

Unique Predictions and Behavioral DataIn addition to incorporating a wide range of known

properties of the frontal cortex and basal ganglia systemour model makes a number of novel predictions at a rangeof different levels At a basic biological level the modelincorporates a few features that remain somewhat specu-lative at this point and therefore constitute clear predic-tions of the model that could be tested using a variety ofelectrophysiological methods

1 Frontal neurons have some kind of intrinsic main-tenance capacitymdashfor example excitatory ion channelsthat persist on the order of seconds Note that subsequentpredictions suggest that these currents will be activatedonly under very specific conditions making them poten-tially somewhat difficult to find empirically

2 Disinhibition of thalamic neurons should be a dom-inant factor in enabling the activation of correspondingLayer 4 frontal neurons

3 Coactivation of Layer 4 neurons and other synapticinputs into neurons in Layers 2ndash3 or 5ndash6 should lead tothe activation of intrinsic maintenance currents Activa-tion of Layer 4 without other synaptic input should resetthe intrinsic currents

4 Frontal neurons within a stripe (eg within a shortdistance of each other as in the isocoding columns ofS G Rao et al 1999) should all exhibit the same timecourse of updating and maintenance For example if oneneuron shows evidence of being updated others nearbyshould as well Note that this does not mean that theseneurons should necessarily encode identical informationdifferent subsets of neurons within a stripe can be acti-vated in different tasks or situations However the com-mon gating signal should in general induce a greater levelof commonality to neurons within a stripe than between

At the behavioral level the model has the potential tomake detailed predictions regarding the different effectsof lesions of each component along the circuit betweenthe frontal cortex and the basal ganglia At the most basiclevel because these systems are mutually dependent ourmodel predicts that damage anywhere within the circuitwill result in overall impairments (see L L Brown et al1997 R G Brown amp Marsden 1990 and Middleton ampStrick 2000b for reviews of relevant data supportingthis idea) For the more detailed predictions we can onlymake qualitative predictions because we have not di-rectly modeled many behavioral tasks we plan to use thelearning-based version of our model (currently under de-velopment) to simulate a wide range of frontal tasks andto make more detailed predictions The following aresome suggestions of how damage to different parts of themodel should differ in their behavioral consequences

1 Selective damage to the basal ganglia (sparing thefrontal cortex) should generally be more evident withmore complex working memory tasks that require selec-tive gating of information in the face of ongoing process-ing andor other distracting information Basic motorplans sequences and other kinds of frontal knowledgeshould remain intact This suggestion is consistent withdata reviewed in R G Brown and Marsden (1990) sug-gesting that Parkinsonrsquos patients show deficits most reli-ably when they have to maintain internal state informationto perform tasks (ie working memory) For exampleParkinsonrsquos patients were selectively impaired on a Strooptask without external cues available but not when thesecues were available (R G Brown amp Marsden 1988)However Parkinsonrsquos patients can also have reduced do-pamine levels in the frontal cortex so it is difficult to drawtoo many strong conclusions regarding selective basal gan-


glia effects from this population Other evidence comesfrom neuroimaging studies that have found enhanced GPiactivation in normals for a difficult planning task (Towerof London) and working memory tasks but not in Parkin-sonrsquos patients (Owen Doyon Dagher Sadikot amp Evans1998) Another interesting case with stroke-induced se-lective striatal damage and selective planning and workingmemory deficits was reported by Robbins et al (1995)They specif ically interpreted this case as reflecting adeficit in the updating of strategies and working mem-ory which is consistent with our model

2 Selective damage to the GPi will cause the frontalloops to be constantly disinhibited (which is presumablywhy pallidotomies are beneficial for enabling Parkinsonrsquospatients to move more freely) This should cause differ-ent kinds of behavioral errors as compared with the ef-fects of striatal damage which would prevent the loopsfrom becoming disinhibited For example constant disin-hibition via GPi damage should result in excessive work-ing memory updating according to our model whereasstriatal damage should result in an inability to selectivelyupdate at the appropriate time Thus GPi patients mightappear scattered impulsive and motorically hyperactiveas in Huntingtonrsquos syndrome Tourettersquos syndrome andpeople with attention-deficit hyperactivity disorder (allof which are known to involve the basal ganglia but havenot been specifically linked with the GPi) In contrastpatients with striatal damage should exhibit both physicalakinesia and ldquopsychic akinesiardquo (R G Brown amp Mars-den 1990)mdashthe inability to initiate both actions andthoughts Either updating too frequently or not enoughwill cause errors on many tasks (eg selective GPi dam-age has been described as impairing performance on arange of ldquofrontal-likerdquo tasks Dujardin KrystkowiakDefebvre Blond amp Destee 2000 Trepanier Saint-CyrLozano amp Lang 1998) but it should be possible to de-termine which of these problems is at work by analyzingthe patterns of errors across trials

3 Tasks that require multiple levels of working mem-ory (eg the outer and inner loops of the 12ndashAX task)should activate different stripes in the frontal cortex ascompared with those that require only one level of work-ing memory Although it is entirely possible that thesestripe-level differences would not be resolvable with pres-ent neuroimaging techniques there is in fact some ev-idence consistent with this prediction For example anf MRI study has shown that activation is present in theanterior PFC specifically when ldquomultitaskingrdquo is required(Christoff amp Gabrieli 2000 Koechlin et al 1999) Othermore direct studies testing this prediction in the 12ndashAXare also currently underway and preliminary data supportour prediction (Jonathan D Cohen personal communi-cation February 1 2001) One other possible experi-mental paradigm for exploring the modelrsquos predictionswould be through the P300 component in event related po-tential studies which has been suggested to reflect contextupdating (Donchin amp Coles 1988) and should be closelyrelated to working memory updating (see also Kropotovamp Etlinger 1999)

Limitations of the Model and Future DirectionsThe primary limitation of our model as it stands now

is in the lack of an implemented learning mechanism forshaping the basal ganglia gating mechanism so that itfires appropriately for task-relevant stimuli In previouswork we and our colleagues have developed such learn-ing models on the basis of the reinforcement learningparadigm (Braver amp Cohen 2000 Cohen et al 1996OrsquoReilly et al 1999 OrsquoReilly et al 2001) There is abun-dant motivation for thinking that the basal ganglia are in-timately involved in this kind of learning via their in-fluence over the dopaminergic neurons of the substantianigra pars compacta and the ventral tegmental area (egBarto 1995 Houk et al 1995 Schultz et al 1997 SchultzRomo et al 1995)

The diff iculty of extending this previous learningwork to the present model comes from two factors Firstwhereas in previous models we used a fairly abstract im-plementation of the dopaminergic system we are at-tempting to make the new model faithful to the underly-ing biology of the basal ganglia system about whichmuch is known Second the selective nature of basal gan-glia gating requires a mechanism capable of learning toallocate representations across the different separatelycontrollable working memory stripes In contrast the ear-lier dopamine-based gating model had to contend onlywith one global gating signal We are making progressaddressing these issues in ongoing modeling work

CONCLUSION

This research has demonstrated that computationalmodels are useful for helping to understand how complexfeatures of the underlying biology can give rise to aspectsof cognitive function Such models are particularly im-portant when trying to understand how a number of dif-ferent specialized brain areas (eg the frontal cortex andthe basal ganglia) interact to perform one overall function(eg working memory) We have found in the presentwork a useful synergy between the functional demands ofa selective gating mechanism in working memory andthe detailed biological properties of the basal gangliaThis convergence across multiple levels of analysis is im-portant for building confidence in the resulting theory

REFERENCES

Alexander G E (1987) Selective neuronal discharge in monkeyputamen reflects intended direction of planned limb movements Ex-perimental Brain Research 67 623-634

Alexander G E Crutcher M amp DeLong M (1990) Basal ganglia-thalamocortical circuits Parallel substrates for motor oculomotorldquoprefrontalrdquo and ldquolimbicrdquo functions In H Uylings C Van EdenJ De Bruin M Corner amp M Feenstra (Eds) The prefrontal cortexIts structure function and pathology (pp 119-146) Amsterdam El-sevier

Alexander G E DeLong M R amp Strick P L (1986) Parallel or-ganization of functionally segregated circuits linking basal gangliaand cortex Annual Review of Neuroscience 9 357-381

Amos A (2000) A computational model of information processing inthe frontal cortex and basal ganglia Journal of Cognitive Neuro-science 12 505-519


Baddeley A D (1986) Working memory New York Oxford Univer-sity Press

Barto A G (1995) Adaptive critics and the basal ganglia In J CHouk J L Davis amp D G Beiser (Eds) Models of information pro-cessing in the basal ganglia (pp 215-232) Cambridge MA MITPress

Beiser D G amp Houk J C (1998) Model of cortical-basal ganglionicprocessing Encoding the serial order of sensory events Journal ofNeurophysiology 79 3168-3188

Beiser D G Hua S E amp Houk J C (1997) Network models of thebasal ganglia Current Opinion in Neurobiolog y 7 185-190

Bergman H Feingold A Nini A Raz A Slovin H Abeles Mamp Vaadia E (1998) Physiological aspects of information process-ing in the basal ganglia of normal and parkinsonian primates Trendsin Neurosciences 21 32-38

Berns G S amp Sejnowski T J (1996) How the basal ganglia makedecisions In A Damasio H Damasio amp Y Christen (Eds) Neuro-biology of decision-making (pp 101-113) Berlin Springer-Verlag

Braver T S amp Cohen J D (2000) On the control of control Therole of dopamine in regulating prefrontal function and working mem-ory In S Monsell amp J Driver (Eds) Attention and performanceXVIII Control of cognitive processes (pp 713-737) CambridgeMA MIT Press

Brown L L Schneider J S amp Lidsky T I (1997) Sensory andcognitive functions of the basal ganglia Current Opinion in Neuro-biology 7 157-163

Brown R G amp Marsden C D (1988) Internal versus external cuesand the control of attention in Parkinsonrsquos disease Brain 111 323-345

Brown R G amp Marsden C D (1990) Cognitive function in Parkin-sonrsquos disease From description to theory Trends in Neurosciences13 21-29

Bullock D amp Grossberg S (1988) Neural dynamics of plannedarm movements Emergent invariants and speedndashaccuracy propertiesduring trajectory formation Psychological Review 95 49-90

Butters N amp Rosvold H E (1968) The effect of caudate and sep-tal nuclei lesions on resistance to extinction and delayed-alternationperformance in monkeys Journal of Comparative Physiological Psy-chology 65 397-403

Chevalier G amp Deniau J M (1990) Disinhibition as a basic pro-cess in the expression of striatal functions Trends in Neurosciences13 277-280

Christoff K amp Gabrieli J D E (2000) The frontopolar cortex andhuman cognition Evidence for a rostrocaudal hierarchical organiza-tion within the human prefrontal cortex Psychobiology 28 168-186

Cohen J D Braver T S amp OrsquoReilly R C (1996) A computationalapproach to prefrontal cortex cognitive control and schizophreniaRecent developments and current challenges Philosophical Transac-tions of the Royal Society of London Series B 351 1515-1527

Cohen J D Dunbar K amp McClelland J L (1990) On the con-trol of automatic processes A parallel distributed processing modelof the Stroop effect Psychological Review 97 332-361

Cohen J D amp OrsquoReilly R C (1996) A preliminary theory of theinteractions between prefrontal cortex and hippocampus that con-tribute to planning and prospective memory In M BrandimonteG O Einstein amp M A McDaniel (Eds) Prospective memory The-ory and applications (pp 267-296) Mahwah NJ Erlbaum

Cohen J D Perlstein W M Braver T S Nystrom L E NollD C Jonides J amp Smith E E (1997) Temporal dynamics ofbrain activity during a working memory task Nature 386 604-608

Constantinidis C amp Steinmetz M A (1996) Neuronal activity inposterior parietal area 7a during the delay periods of a spatial mem-ory task Journal of Neurophysiolog y 76 1352-1355

Dehaene S amp Changeux J P (1989) A simple model of prefrontalcortex function in delayed-response tasks Journal of Cognitive Neu-roscience 1 244-261

Dehaene S amp Changeux J P (1991) The Wisconsin Card SortingTest Theoretical analysis and modeling in a neuronal network Cere-bral Cortex 1 62-79

Deniau J M amp Chevalier G (1985) Disinhibition as a basic pro-cess in the expression of striatal functions II The striato-nigral in-fluence on thalamocortical cells of the ventromedial thalamic nu-cleus Brain Research 334 227-233

Dias R Robbins T W amp Roberts A C (1997) Dissociable formsof inhibitory control within prefrontal cortex with an analog of theWisconsin Card Sort Test Restriction to novel situations and inde-pendence from ldquoon-linerdquo processing Journal of Neuroscience 179285-9297

Dilmore J G Gutkin B G amp Ermentrout G B (1999) Effectsof dopaminergic modulation of persistent sodium currents on the ex-citability of prefrontal cortical neurons A computational study Neu-rocomputing 26 104-116

Divac I Rosvold H E amp Szwaracbart M K (1967) Behavioraleffects of selective ablation of the caudate nucleus Journal of Com-parative Physiological Psychology 63 184-190

Dominey P F (1995) Complex sensoryndashmotor sequence learningbased on recurrent state representation and reinforcement learningBiological Cybernetics 73 265-274

Dominey P F amp Arbib M A (1992) Cortico-subcortical model forgeneration of spatially accurate sequential saccades Cerebral Cortex2 153-175

Donchin E amp Coles M G (1988) Is the P300 component a mani-festation of context updating Behavioral amp Brain Sciences 11 357-427

Douglas R J amp Martin K A C (1990) Neocortex In G M Shep-herd (Ed) The synaptic organization of the brain (pp 389-438) Ox-ford Oxford University Press

Dujardin K Krystkowiak P Defebvre L Blond S amp Destee A(2000) A case of severe dysexecutive syndrome consecutive to chronicbilateral pallidal stimulation Neuropsychologia 38 1305-1315

Durstewitz D Kelc M amp Gunturkun O (1999) A neurocom-putational theory of the dopaminergic modulation of working mem-ory functions Journal of Neuroscience 19 2807-2822

Durstewitz D Seamans J K amp Sejnowski T J (2000a) Dopamine-mediated stabilization of delay-period activity in a network model ofprefrontal cortex Journal of Neurophysiolog y 83 1733-1750

Durstewitz D Seamans J K amp Sejnowski T J (2000b) Neuro-computational models of working memory Nature Neuroscience 3(Suppl) 1184-1191

Erickson S L amp Lewis D A (2000) Prefrontal cortical inputs tomonkey mediodorsal thalamus Society for Neuroscience Abstracts(p 461) San Diego Society for Neuroscience

Fellous J M Wang X J amp Lisman J E (1998) A role for NMDA-receptor channels in working memory Nature Neuroscience 1 273-275

Fox C A amp Rafols J A (1976) The striatal efferents in the globuspallidus and in the substantia nigra In M D Yahr (Ed) The basalganglia (pp 37-55) New York Raven

Funahashi S Bruce C J amp Goldman-Rakic P S (1989) Mne-monic coding of visual space in the monkeyrsquos dorsolateral prefrontalcortex Journal of Neurophysiolog y 61 331-349

Fuster J M (1989) The prefrontal cortex Anatomy physiology andneuropsychology of the frontal lobe (3rd ed) New York Lippincott-Raven

Fuster J M amp Alexander G E (1971) Neuron activity related toshort-term memory Science 173 652-654

Gelfand J Gullapalli V Johnson M Raye C amp Henderson J(1997) The dynamics of prefrontal cortico-thalamo-basal ganglionicloops and short term memory interference phenomena In Proceed-ings of the 19th Annual Conference of the Cognitive Science Society(pp 253-258) Mahwah NJ Erlbaum

Gobbel J R (1995) A biophysically-based model of the neostriatumas a dynamically reconfigurable network In M Boden amp L-ENiklasson (Eds) Proceedings of the Second Swedish Conference onConnectionism Hillsdale NJ Erlbaum

Gobbel J R (1997) The role of the neostriatum in the execution of ac-tion sequences Unpublished doctoral dissertation University of Cal-ifornia San Diego

Goldman P S amp Rosvold H E (1972) The effects of selective cau-date lesions in infant and juvenile Rhesus monkeys Brain Research43 53-66

Goldman-Rakic P S (1987) Circuitry of primate prefrontal cortexand regulation of behavior by representational memory In F Plum ampV Mountcastle (Eds) Handbook of physiology The nervous system(Vol 5 pp 373-417) Bethesda MD American Physiological Society


Goldman-Rakic P S amp Friedman H R (1991) The circuitry ofworking memory revealed by anatomy and metabolic imaging InH S Levin H M Eisenberg amp A L Benton (Eds) Frontal lobe func-tion and dysfunction (pp 72-91) New York Oxford University Press

Gorelova N A amp Yang C R (2000) Dopamine D1D5 receptoractivation modulates a persistent sodium current in rat prefrontal cor-tical neurons in vitro Journal of Neurophysiolog y 84 75-87

Graybiel A M amp Kimura M (1995) Adaptive neural networks inthe basal ganglia In J C Houk J L Davis amp D G Beiser (Eds)Models of information processing in the basal ganglia (pp 103-116) Cambridge MA MIT Press

Guigon E Dorizzi B Burnod Y amp Schultz W (1995) Neuralcorrelates of learning in the prefrontal cortex of the monkey A pre-dictive model Cerebral Cortex 2 135-147

Hikosaka O (1989) Role of basal ganglia in initiation of voluntarymovements In M A Arbib amp S Amari (Eds) Dynamic interactionsin neural networks Models and data (pp 153-167) Berlin Springer-Verlag

Hochreiter S amp Schmidhuber J (1997) Long short-term memoryNeural Computation 9 1735-1780

Hoshi E Shima K amp Tanji J (2000) Neuronal activity in the pri-mate prefrontal cortex in the process of motor selection based on twobehavioral rules Journal of Neurophysiolog y 83 2355-2373

Houk J C Adams J L amp Barto A G (1995) A model of how thebasal ganglia generate and use neural signals that predict reinforce-ment In J C Houk J L Davis amp D G Beiser (Eds) Models of in-formation processing in the basal ganglia (pp 233-248) CambridgeMA MIT Press

Houk J C amp Wise S P (1995) Distributed modular architectureslinking basal ganglia cerebellum and cerebral cortex Their role inplanning and controlling action Cerebral Cortex 5 95-110

Jackson S amp Houghton G (1995) Sensorimotor selection and thebasal ganglia A neural network model In J C Houk J L Davis ampD G Beiser (Eds) Models of information processing in the basalganglia (pp 337-368) Cambridge MA MIT Press

Jaeger D Kita H amp Wilson C J (1994) Surround inhibitionamong projection neurons is weak or nonexistent in the rat neostria-tum Journal of Neurophysiolog y 72 2555-2558

Koechlin E Basso G Pietrini P Panzer S amp Grafman J (1999)The role of the anterior prefrontal cortex in human cognition Nature399 148-151

Kropotov J D amp Etlinger S C (1999) Selection of actions in thebasal ganglia-thalamocoritcal circuits Review and model Interna-tional Journal of Psychophysiolog y 31 197-217

Kubota K amp Niki H (1971) Prefrontal cortical unit activity and de-layed alternation performance in monkeys Journal of Neurophysiol -ogy 34 337-347

Lange H Thorner G amp Hopf A (1976) Morphometric-statisticalstructure analysis of human striatum pallidum and nucleus subthal-amicus III Nucleus subthalamicus Journal fuumlr Hirnforschung 1731-41

Levitt J B Lewis D A Yoshioka T amp Lund J S (1993) Topog-raphy of pyramidal neuron intrinsic connections in macaque monkeyprefrontal cortex (areas 9 amp 46) Journal of Comparative Neurology338 360-376

Lewis B L amp OrsquoDonnell P (2000) Ventral tegmental area afferentsto the prefrontal cortex maintain membrane potential ldquouprdquo states inpyramidal neurons via D1 dopamine receptors Cerebral Cortex 101168-1175

Matsumoto N Hanakawa T Maki S Graybiel A M ampKimura M (1999) Role of nigrostriatal dopamine system in learn-ing to perform sequential motor tasks in a predictive manner Jour-nal of Neurophysiolog y 82 978-998

McClelland J L McNaughton B L amp OrsquoReilly R C (1995)Why there are complementary learning systems in the hippocampusand neocortex Insights from the successes and failures of connection-ist models of learning and memory Psychological Review 102 419-457

McFarland N R amp Haber S N (2000) Convergent inputs fromthalamic motor nuclei and frontal cortical areas to the dorsal striatumin the primate Journal of Neuroscience 20 3798-3813

Middleton F A amp Strick P L (2000a) Basal ganglia and cerebel-

lar loops Motor and cognitive circuits Brain Research Reviews 31236-250

Middleton F A amp Strick P L (2000b) Basal ganglia output andcognition Evidence from anatomical behavioral and clinical stud-ies Brain amp Cognition 42 183-200

Miller E K Erickson C A amp Desimone R (1996) Neural mech-anisms of visual working memory in prefontal cortex of the macaqueJournal of Neuroscience 16 5154-5167

Miyake A amp Shah P (Eds) (1999) Models of working memoryMechanisms of active maintenance and executive control New YorkCambridge University Press

Miyashita Y amp Chang H S (1988) Neuronal correlate of pictorialshort-term memory in the primate temporal cortex Nature 331 68-70

Montague P R Dayan P amp Sejnowski T J (1996) A frameworkfor mesencephalic dopamine systems based on predictive Hebbianlearning Journal of Neuroscience 16 1936-1947

Moody S L Wise S P di Pellegrino G amp Zipser D (1998) Amodel that accounts for activity in primate frontal cortex during a de-layed matching-to-sample task Journal of Neuroscience 18 399-410

Munakata Y (1998) Infant perseveration and implications for objectpermanence theories A PDP model of the A-not-B task Develop-mental Science 1 161-184

Neafsey E J Hull C D amp Buchwald N A (1978) Preparationfor movement in the cat I Unit activity in the cerebral cortex Elec-troencephalography amp Clinical Neurophysiolog y 44 714-723

Norman D amp Shallice T (1986) Attention to action Willed andautomatic control of behavior In R Davidson G Schwartz ampD Shapiro (Eds) Consciousness and self-regulation Advances inresearch and theory (Vol 4 pp 1-18) New York Plenum

Nystrom L E Braver T S Sabb F W Delgado M R Noll D Camp Cohen J D (2000) Working memory for letters shapes and lo-cations fMRI evidence against stimulus-based regional organizationin human prefrontal cortex NeuroImage 11 424-446

OrsquoReilly R C (1998) Six principles for biologically-based computa-tional models of cortical cognition Trends in Cognitive Sciences 2455-462

OrsquoReilly R C Braver T S amp Cohen J D (1999) A biologicallybased computational model of working memory In A Miyake ampP Shah (Eds) Models of working memory Mechanisms of activemaintenance and executive control (pp 375-411) New York Cam-bridge University Press

OrsquoReilly R C amp McClelland J L (1994) Hippocampal conjunc-tive encoding storage and recall Avoiding a tradeoff Hippocam-pus 4 661-682

OrsquoReilly R C amp Munakata Y (2000) Computational explorationsin cognitive neuroscience Understanding the mind by simulating thebrain Cambridge MA MIT Press

OrsquoReilly R C Noelle D Braver T S amp Cohen J D (2001)Prefrontal cortex and dynamic categorization tasks Representationa lorganization and neuromodulatory control Manuscript submitted forpublication

OrsquoReilly R C amp Rudy J W (2000) Computational principles oflearning in the neocortex and hippocampus Hippocampus 10 389-397

OrsquoReilly R C amp Rudy J W (2001) Conjunctive representations inlearning and memory Principles of cortical and hippocampal func-tion Psychological Review 108 311-345

Owen A M Doyon J Dagher A Sadikot A amp Evans A C(1998) Abnormal basal ganglia outflow in Parkinsonrsquos disease iden-tified with PET Implications for higher cortical functions Brain 121949-965

Owen A M Roberts A C Hodges J R Summers B A PolkeyC E amp Robbins T W (1993) Contrasting mechanisms of impairedattentional set-shifting in patients with frontal lobe damage or Parkin-sonrsquos disease Brain 116 1159-1175

Passingham R E (1993) The frontal lobes and voluntary action Ox-ford Oxford University Press

Petrides M (1994) Frontal lobes and working memory Evidencefrom investigations of the effects of cortical excisions in nonhumanprimates In F Boller amp J Grafman (Eds) Handbook of neuropsy-chology (Vol 9 pp 59-82) Amsterdam Elsevier

Pucak M L Levitt J B Lund J S amp Lewis D A (1996) Pat-


terns of intrinsic and associational circuitry in monkey prefrontal cor-tex Journal of Comparative Neurology 376 614-630

Rao S C Rainer G amp Miller E K (1997 May) Integration of what and where in the primate prefrontal cortex Science 276 821-824

Rao S G Williams G V amp Goldman-Rakic P S (1999) Isodi-rectional tuning of adjacent interneurons and pyramidal cells duringworking memory Evidence for microcolumnar organization in PFCJournal of Neurophysiolog y 81 1903-1916

Robbins T W Shallice T Burgess P W James M Rogers R DWarburton E amp Wise R S J (1995) Selective impairments inself-ordered working memory in a patient with a unilateral striatal le-sion Neurocase 1 217-230

Schneider J S (1987) Basal ganglia-motor influences Role of sen-sory gating In J S Schneider amp T I Lidsky (Eds) Basal gangliaand behavior Sensory aspects of motor functioning (pp 103-121) Toronto Hans Huber

Schultz W Apicella P amp Ljungberg T (1993) Responses ofmonkey dopamine neurons to reward and conditioned stimuli duringsuccessive steps of learning a delayed response task Journal of Neu-roscience 13 900-913

Schultz W Apicella P Romo R amp Scarnati E (1995) Context-dependent activity in primate striatum reflecting past and future be-havioral events In J C Houk J L Davis amp D G Beiser (Eds)Models of information processing in the basal ganglia (pp 11-28)Cambridge MA MIT Press

Schultz W Dayan P amp Montague P R (1997 March) A neuralsubstrate of prediction and reward Science 275 1593-1599

Schultz W Romo R Ljungberg T Mirenowicz J Holler-man J R amp Dickinson A (1995) Reward-related signals carriedby dopamine neurons In J C Houk J L Davis amp D G Beiser(Eds) Models of information processing in the basal ganglia(pp 233-248) Cambridge MA MIT Press

Seung H S (1998) Continuous attractors and oculomotor controlNeural Networks 11 1253-1258

Shallice T (1988) From neuropsychology to mental structure NewYork Cambridge University Press

Shima K amp Tanji J (1998) Both supplementary and presupplemen-tary motor areas are crucial for the temporal organization of multiplemovements Journal of Neurophysiolog y 80 3247-3260

Smith E E amp Jonides J (1997) Working memory A view from neu-roimaging Cognitive Psychology 33 5-42

Surmeier D J amp Kitai S T (1999) D1 and D2 modulation ofsodium and potassium currents in rat neostriatal neurons Progress inBrain Research 99 309-324

Tanaka S amp Okada S (1999) Functional prefrontal cortical cir-cuitry for visuospatial working memory formation A computationalmodel Neurocomputing 26 891-899

Taylor J G amp Taylor N R (2000) Analysis of recurrent cortico-basal ganglia-thalamic loops for working memory Biological Cy-bernetics 82 415-432

Trepanier L L Saint-Cyr J A Lozano A M amp Lang A E(1998) Neuropsychological consequences of posteroventral pallido-tomy for the treatment of Parkinsonrsquos disease Neurology 51 207-215

Tsung F-S amp Cottrell G W (1993) Learning simple arithmeticprocedures Connection Science 5 37-58

Wang X-J (1999) Synaptic basis of cortical persistent activity Theimportance of NMDA receptors to working memory Journal of Neu-roscience 19 9587-9603

Watanabe M (1992) Frontal units of the monkey coding the asso-ciative significance of visual and auditory stimuli ExperimentalBrain Research 89 233-247

Wickens J [R] (1993) A theory of the striatum Oxford PergamonPress

Wickens J [R] (1997) Basal ganglia Structure and computations Network Computation in Neural Systems 8 R77-R109

Wickens J R Kotter R amp Alexander M E (1995) Effects oflocal connectivity on striatal function Simulation and analysis of amodel Synapse 20 281-298

Wilson C J (1993) The generation of natural firing patterns in neos-triatal neurons In G W Arbuthnott amp P C Emson (Eds) Chemicalsignalling in the basal ganglia (Progress in Brain Research Vol 99pp 277-297) Amsterdam Elsevier

Wilson F A W Scalaidhe S P O amp Goldman-Rakic P S(1993) Dissociation of object and spatial processing domains in pri-mate prefrontal cortex Science 260 1955-1957

Wise S P (1985) The primate premotor cortex Past present andpreparatory Annual Review of Neuroscience 8 1-19

Wise S P Murray E A amp Gerfen C R (1996) The frontal cortex-basal ganglia system in primates Critical Reviews in Neurobiolog y10 317-356

Zipser D (1991) Recurrent network model of the neural mechanismof short-term active memory Neural Computation 3 179-193

Zipser D Kehoe B Littlewort G amp Fuster J (1993) A spik-ing network model of short-term active memory Journal of Neuro-science 13 3406-3420

NOTES

1 Other variations in target sequences for the two subtasks are pos-sible and are being explored empirically

2 Although it is still possible that other frontal areas were reallymaintaining the signal during the intervening stimulus activations thisexplanation becomes less appealing as this phenomenon is consistentlyobserved across many different frontal areas

APPENDIXImplementational Details

The model is implemented using a subset of the Leabra framework (OrsquoReilly 1998 OrsquoReilly amp Munakata2000) The two relevant properties of this framework for the present model are (1) the use of a point neuronactivation function and (2) the k-Winners-Take-All (kWTA) inhibition function that models the effects of in-hibitory neurons These two properties are described in detail below In addition the gating equations formodulating the intracellular maintenance ion currents in the PFC are described

Point Neuron Activation FunctionLeabra uses a point neuron activation function that models the electrophysiological properties of real neu-

rons while simplifying their geometry to a single point This function is nearly as simple computationally asthe standard sigmoidal activation function but the more biologically based implementation makes it consid-erably easier to model inhibitory competition as will be described below Furthermore use of this functionenables cognitive models to be more easily related to more physiologically detailed simulations thereby fa-cilitating bridge building between biology and cognition

The membrane potential Vm is updated as a function of ionic conductances g with reversal (driving) poten-tials E as follows

(1)dV t

dtg t g E V tc

cc c m

m ( )( ) ( ) = -[ ]aringt


APPENDIX (Continued)

with four channels (c) corresponding to the following e excitatory input l leak current i inhibitory inputand h for a hysteresis channel that reflects the action of a switchable persistent excitatory input this h chan-nel is used for the intracellular maintenance mechanism described below Following electrophysiological con-vention the overall conductance is decomposed into a time-varying component gc(t) computed as a functionof the dynamic state of the network and a constant gc that controls the relative influence of the different con-ductances

The excitatory net input conductance ge(t) or h j is computed as the proportion of open excitatory channelsas a function of sending activations times the weight values

(2)

The inhibitory conductance is computed via the kWTA function described in the next section and leak is aconstant

Activation communicated to other cells (yj ) is a thresholded (Q) sigmoidal function of the membrane po-tential with gain parameter c

(3)

where [x]+ is a threshold function that returns 0 if x 0 and x if x 0 Note that if it returns 0 we assumeyj (t) = 0 to avoid dividing by 0 As it is this function has a very sharp threshold which interferes with gradedlearning mechanisms (eg gradient descent) To produce a less discontinuous deterministic function with asofter threshold the function is convolved with a Gaussian noise kernel which reflects the intrinsic process-ing noise of biological neurons

(4)

where x represents the [Vm(t) 2 Q]+ value and yj (x) is the noise-convolved activation for that value In the

simulation this function is implemented by using a numerical lookup table since an analytical solution is notpossible

k-Winners-Take-All InhibitionLeabra uses a kWTA function to achieve sparse distributed representations with two different versions hav-

ing different levels of flexibility around the k out of n active units constraint Both versions compute a uni-form level of inhibitory current for all units in the layer as follows

(5)

where 0 q 1 is a parameter for setting the inhibition between the upper bound of gQk and the lower bound

of g Qk + 1 These boundary inhibition values are computed as a function of the level of inhibition necessary to

keep a unit right at threshold

(6)

where ge is the excitatory net input without the bias weight contribution this allows the bias weights to over-

ride the kWTA constraintIn the basic version of the kWTA function which is relatively rigid about the kWTA constraint gQ

k and gQ

k + 1 are set to the threshold inhibition value for the kth and k + 1th most excited units respectively Thus theinhibition is placed exactly to allow k units to be above threshold and the remainder below threshold For thisversion the q parameter is almost always 25 allowing the kth unit to be sufficiently above the inhibitorythreshold

In the average-based kWTA version gQk is the average gQ

t value for the top k most excited units and gQk + 1

is the average of gQi for the remaining n 2 k units This version allows for more flexibility in the actual number

of units active depending on the nature of the activation distribution in the layer and the value of the q param-eter (which is typically between 5 and 7 depending on the level of sparseness in the layer with a standard de-fault value of 6)

Activation dynamics similar to those produced by the kWTA function have been shown to result from sim-ulated inhibitory interneurons that project both feedforward and feedback inhibition (OrsquoReilly amp Munakata2000) Thus although the kWTA function is somewhat biologically implausible in its implementation (egrequiring global information about activation states and using sorting mechanisms) it provides a computa-tionally effective approximation to biologically plausible inhibitory dynamics

gg g E g g E

Ete e e l l l

i

Q Q QQ

= - + --

( ) ( )

g g q g gi k k k= + -( )+ +1 1Q Q Q

y x e y z x dzjx

j ( )( ) ( ) = -

-yenyen -ograve 1

2

2 22

pss

y t

V t

j

m

( )

( )

=

+-[ ]

aelig

egraveccedilccedil

ouml

oslashdividedivide

+

1

1 1

g Q

h j e i i j i i ji

g t x w n x w= pound sup3 aring( ) 1



Intracellular Ion Currents for PFC MaintenanceThe gating function for switching on maintenance was implemented as follows If any unit in the PFC Gat-

ing layer has activation that exceeds the maintenance threshold the corresponding unit in the PFC Mainte-nance layer has its intracellular excitatory current (gh) set to the value of the sending unitrsquos (in PFC_Gate) ac-tivation times the amount of excitatory input being received from the sensory input layer

(7)

where xi is the sending activation h j is the net input from the sensory input and Qm is the maintenance thresholdIf the gh conductance is nonzero it contributes a positive excitatory influence on the unitrsquos membrane potential

(Manuscript received November 22 2000revision accepted for publication May 3 2001)

gx x

hi j i=

gtigraveiacuteicirc

h if

otherwisemQ

0


We conclude by discussing the relationship between thismodel and other existing models of the basal-gangliandashfrontal-cortex system

WORKING MEMORY

Working memory can be defined as an active systemfor temporarily storing and manipulating informationneeded for the execution of complex cognitive tasks (Bad-deley 1986) For example this kind of memory is clearlyimportant for performing mental arithmetic (eg multi-plying 42 3 17)mdashone must maintain subsets of the prob-lem (eg 7 3 2) and store partial products (eg 14)while maintaining the original problem as well (eg 42and 17 see eg Tsung amp Cottrell 1993) It is also usefulin problem solving (maintaining and updating goals andsubgoals imagined consequences of actions etc) lan-guage comprehension (keeping track of many levels ofdiscourse using prior interpretations to correctly interpretsubsequent passages etc) and many other cognitive ac-tivities (see Miyake amp Shah 1999 for a recent survey)

From a neural perspective one can identify workingmemory with the maintenance and updating of informa-tion encoded in the active firing of neurons (activation-based memory see eg Fuster 1989 Goldman-Rakic1987) It has long been known that the PFC exhibits thiskind of sustained active firing over delays (eg FunahashiBruce amp Goldman-Rakic 1989 Fuster amp Alexander1971 Kubota amp Niki 1971 Miller Erickson amp Desi-mone 1996 Miyashita amp Chang 1988) Such findingssupport the idea that the PFC is important for active main-tenance of information in working memory

The properties of this activation-based memory can beunderstood by contrasting them with more long-term kindsof memories that are stored in the synaptic connections be-tween neurons (weight-based memory Cohen et al 1996Munakata 1998 OrsquoReilly et al 1999 OrsquoReilly amp Mu-nakata 2000) Activation-based memories have a num-ber of advantages relative to weight-based memories Forexample activation-based memories can be rapidly up-dated just by changing the activation state of a set of neu-rons In contrast changing weights requires structuralchanges in neural connectivity which can be much slowerAlso information maintained in an active state is directlyaccessible to other parts of the brain (ie as a constantpropagation of activation signals to all connected neu-rons) whereas synaptic changes only directly affect theneuron on the receiving end of the connection and thenonly when the sending neuron is activated In more fa-miliar terms activation-based memories are like sendinga message via broadcast radio signal whereas weight-based memories are like sending a letter in the mail

These mechanistic properties of activation-basedmemories coincide well with oft-discussed characteristicsof information maintained in working memory Specifi-cally working memory is used for processing because itcan be rapidly updated to reflect the ongoing products anddemands of processing and it is generally consciously ac-cessible and can be described in a verbal protocol (eg

Miyake amp Shah 1999) Furthermore the active nature ofworking memory provides a natural mechanism for cog-nitive control (also known as task-based attention) wheretop-down activation can influence processing elsewhereto achieve task-relevant objectives (Cohen Dunbar ampMcClelland 1990 Cohen amp OrsquoReilly 1996 OrsquoReillyet al 1999) Thus working memory and cognitive con-trol can be seen as two different manifestations of the sameunderlying mechanism of actively maintained informa-tion Of course these manifestations function in differ-ent ways in different tasks and thus are not the samepsychological construct but both can be subserved by acommon mechanism

However with these advantages of activation-basedmemories there are also concomitant disadvantages Forexample because these memories do not involve struc-tural changes they are transient and therefore do not pro-vide a suitable basis for long-term memories Also be-cause information is encoded by the activation states ofneurons the capacity of these memories scales as a func-tion of the number of neurons whereas the capacity ofweight-based memories scales as a function of the numberof synaptic connections which is much larger

Because of this fundamental tradeoff between activation-and weight-based memory mechanisms it makes sensethat the brain would have evolved two different special-ized systems to obtain the best of both types of memoryThis is particularly true if there are specific mechanisticspecializations that are needed to make each type of mem-ory work better There has been considerable discussionalong these lines of the ways in which the neural struc-ture of the hippocampus is optimized for subserving aparticular kind of weight-based memory (eg McClel-land McNaughton amp OrsquoReilly 1995 OrsquoReilly amp Mc-Clelland 1994 OrsquoReilly amp Rudy 2000 2001) Simi-larly this paper represents the further development of aline of thinking about the ways in which the frontal cor-tex is specialized to subserve activation-based memory(Braver amp Cohen 2000 Cohen et al 1996 OrsquoReilly et al1999) In the next section we will introduce a specificworking memory task that exemplif ies the functionalspecializations needed to support effective activation-based memories and we then proceed to explore how thebiology of the frontal-cortexndashbasal-ganglia system is spe-cialized to achieve these functions

Working Memory Functional DemandsThe AndashX version of the continuous performance task

(CPTndashAX) is a standard working memory task that hasbeen extensively studied in humans (Braver amp Cohen2000 Cohen et al 1997) The subject is presented withsequential letter stimuli (A X B Y) and is asked to detectthe specific sequence of an A followed by an X by push-ing the right button All other combinations (AndashY BndashXBndashY) should be responded to with a left button push Thistask requires a relatively simple form of working memorywhere the prior stimulus must be maintained over a delayuntil the next stimulus appears so that one can discrim-inate the target from the nontarget sequences We have












1

L

A X

L R

B

L

X

L L

B

L2 R

Ytime



SensoryInput



Myphone is4920054

4920054

Gating open closed



492-0054 492-0054

492-0054

867-5309

















Frontal Cortex

Striatum

reverberatory loops

inhibition

PosteriorCortex

GPiSNr

Thalamus

posterior anterior

















SensoryInput

WorkingMemory A

A

learning

notrecog-nized



gatenotopened

GatingControl


A B
























































DISCUSSION


















FUNCTION




































30-

35-

40-

45-

50-

55-

60-

0 1 2 3 4 5 6 7 8 9 10

Set

tling

Cyc

les

Training Trials

2nd step

3rd step




















CONCLUSION


REFERENCES





























































































































NOTES








(1)dV t

dtg t g E V tc

cc c m

m ( )( ) ( ) = -[ ]aringt





(2)



(3)


(4)





(5)




(6)



k and gQ







gg g E g g E

Ete e e l l l

i

Q Q QQ

= - + --

( ) ( )

g g q g gi k k k= + -( )+ +1 1Q Q Q

y x e y z x dzjx

j ( )( ) ( ) = -

-yenyen -ograve 1

2

2 22

pss

y t

V t

j

m

( )

( )

=

+-[ ]

aelig

egraveccedilccedil

ouml

oslashdividedivide

+

1

1 1

g Q

h j e i i j i i ji






(7)



gx x

hi j i=

gtigraveiacuteicirc

h if

otherwisemQ

0












1

L

A X

L R

B

L

X

L L

B

L2 R

Ytime



SensoryInput



Myphone is4920054

4920054

Gating open closed



492-0054 492-0054

492-0054

867-5309

















Frontal Cortex

Striatum

reverberatory loops

inhibition

PosteriorCortex

GPiSNr

Thalamus

posterior anterior

















SensoryInput

WorkingMemory A

A

learning

notrecog-nized



gatenotopened

GatingControl


A B
























































DISCUSSION


















FUNCTION




































30-

35-

40-

45-

50-

55-

60-

0 1 2 3 4 5 6 7 8 9 10

Set

tling

Cyc

les

Training Trials

2nd step

3rd step




















CONCLUSION


REFERENCES





























































































































NOTES








(1)dV t

dtg t g E V tc

cc c m

m ( )( ) ( ) = -[ ]aringt





(2)



(3)


(4)





(5)




(6)



k and gQ







gg g E g g E

Ete e e l l l

i

Q Q QQ

= - + --

( ) ( )

g g q g gi k k k= + -( )+ +1 1Q Q Q

y x e y z x dzjx

j ( )( ) ( ) = -

-yenyen -ograve 1

2

2 22

pss

y t

V t

j

m

( )

( )

=

+-[ ]

aelig

egraveccedilccedil

ouml

oslashdividedivide

+

1

1 1

g Q

h j e i i j i i ji






(7)



gx x

hi j i=

gtigraveiacuteicirc

h if

otherwisemQ

0

















Frontal Cortex

Striatum

reverberatory loops

inhibition

PosteriorCortex

GPiSNr

Thalamus

posterior anterior

















SensoryInput

WorkingMemory A

A

learning

notrecog-nized



gatenotopened

GatingControl


A B
























































DISCUSSION


















FUNCTION




































30-

35-

40-

45-

50-

55-

60-

0 1 2 3 4 5 6 7 8 9 10

Set

tling

Cyc

les

Training Trials

2nd step

3rd step




















CONCLUSION


REFERENCES





























































































































NOTES








(1)dV t

dtg t g E V tc

cc c m

m ( )( ) ( ) = -[ ]aringt





(2)



(3)


(4)





(5)




(6)



k and gQ







gg g E g g E

Ete e e l l l

i

Q Q QQ

= - + --

( ) ( )

g g q g gi k k k= + -( )+ +1 1Q Q Q

y x e y z x dzjx

j ( )( ) ( ) = -

-yenyen -ograve 1

2

2 22

pss

y t

V t

j

m

( )

( )

=

+-[ ]

aelig

egraveccedilccedil

ouml

oslashdividedivide

+

1

1 1

g Q

h j e i i j i i ji






(7)



gx x

hi j i=

gtigraveiacuteicirc

h if

otherwisemQ

0
















SensoryInput

WorkingMemory A

A

learning

notrecog-nized



gatenotopened

GatingControl


A B
























































DISCUSSION


















FUNCTION




































30-

35-

40-

45-

50-

55-

60-

0 1 2 3 4 5 6 7 8 9 10

Set

tling

Cyc

les

Training Trials

2nd step

3rd step




















CONCLUSION


REFERENCES





























































































































NOTES








(1)dV t

dtg t g E V tc

cc c m

m ( )( ) ( ) = -[ ]aringt





(2)



(3)


(4)





(5)




(6)



k and gQ







gg g E g g E

Ete e e l l l

i

Q Q QQ

= - + --

( ) ( )

g g q g gi k k k= + -( )+ +1 1Q Q Q

y x e y z x dzjx

j ( )( ) ( ) = -

-yenyen -ograve 1

2

2 22

pss

y t

V t

j

m

( )

( )

=

+-[ ]

aelig

egraveccedilccedil

ouml

oslashdividedivide

+

1

1 1

g Q

h j e i i j i i ji






(7)



gx x

hi j i=

gtigraveiacuteicirc

h if

otherwisemQ

0
























































DISCUSSION


















FUNCTION




































30-

35-

40-

45-

50-

55-

60-

0 1 2 3 4 5 6 7 8 9 10

Set

tling

Cyc

les

Training Trials

2nd step

3rd step




















CONCLUSION


REFERENCES





























































































































NOTES








(1)dV t

dtg t g E V tc

cc c m

m ( )( ) ( ) = -[ ]aringt





(2)



(3)


(4)





(5)




(6)



k and gQ







gg g E g g E

Ete e e l l l

i

Q Q QQ

= - + --

( ) ( )

g g q g gi k k k= + -( )+ +1 1Q Q Q

y x e y z x dzjx

j ( )( ) ( ) = -

-yenyen -ograve 1

2

2 22

pss

y t

V t

j

m

( )

( )

=

+-[ ]

aelig

egraveccedilccedil

ouml

oslashdividedivide

+

1

1 1

g Q

h j e i i j i i ji






(7)



gx x

hi j i=

gtigraveiacuteicirc

h if

otherwisemQ

0















FUNCTION




































30-

35-

40-

45-

50-

55-

60-

0 1 2 3 4 5 6 7 8 9 10

Set

tling

Cyc

les

Training Trials

2nd step

3rd step




















CONCLUSION


REFERENCES





























































































































NOTES








(1)dV t

dtg t g E V tc

cc c m

m ( )( ) ( ) = -[ ]aringt





(2)



(3)


(4)





(5)




(6)



k and gQ







gg g E g g E

Ete e e l l l

i

Q Q QQ

= - + --

( ) ( )

g g q g gi k k k= + -( )+ +1 1Q Q Q

y x e y z x dzjx

j ( )( ) ( ) = -

-yenyen -ograve 1

2

2 22

pss

y t

V t

j

m

( )

( )

=

+-[ ]

aelig

egraveccedilccedil

ouml

oslashdividedivide

+

1

1 1

g Q

h j e i i j i i ji






(7)



gx x

hi j i=

gtigraveiacuteicirc

h if

otherwisemQ

0



























30-

35-

40-

45-

50-

55-

60-

0 1 2 3 4 5 6 7 8 9 10

Set

tling

Cyc

les

Training Trials

2nd step

3rd step




















CONCLUSION


REFERENCES





























































































































NOTES








(1)dV t

dtg t g E V tc

cc c m

m ( )( ) ( ) = -[ ]aringt





(2)



(3)


(4)





(5)




(6)



k and gQ







gg g E g g E

Ete e e l l l

i

Q Q QQ

= - + --

( ) ( )

g g q g gi k k k= + -( )+ +1 1Q Q Q

y x e y z x dzjx

j ( )( ) ( ) = -

-yenyen -ograve 1

2

2 22

pss

y t

V t

j

m

( )

( )

=

+-[ ]

aelig

egraveccedilccedil

ouml

oslashdividedivide

+

1

1 1

g Q

h j e i i j i i ji






(7)



gx x

hi j i=

gtigraveiacuteicirc

h if

otherwisemQ

0




















CONCLUSION


REFERENCES





























































































































NOTES








(1)dV t

dtg t g E V tc

cc c m

m ( )( ) ( ) = -[ ]aringt





(2)



(3)


(4)





(5)




(6)



k and gQ







gg g E g g E

Ete e e l l l

i

Q Q QQ

= - + --

( ) ( )

g g q g gi k k k= + -( )+ +1 1Q Q Q

y x e y z x dzjx

j ( )( ) ( ) = -

-yenyen -ograve 1

2

2 22

pss

y t

V t

j

m

( )

( )

=

+-[ ]

aelig

egraveccedilccedil

ouml

oslashdividedivide

+

1

1 1

g Q

h j e i i j i i ji






(7)



gx x

hi j i=

gtigraveiacuteicirc

h if

otherwisemQ

0

























































































































NOTES








(1)dV t

dtg t g E V tc

cc c m

m ( )( ) ( ) = -[ ]aringt





(2)



(3)


(4)





(5)




(6)



k and gQ







gg g E g g E

Ete e e l l l

i

Q Q QQ

= - + --

( ) ( )

g g q g gi k k k= + -( )+ +1 1Q Q Q

y x e y z x dzjx

j ( )( ) ( ) = -

-yenyen -ograve 1

2

2 22

pss

y t

V t

j

m

( )

( )

=

+-[ ]

aelig

egraveccedilccedil

ouml

oslashdividedivide

+

1

1 1

g Q

h j e i i j i i ji






(7)



gx x

hi j i=

gtigraveiacuteicirc

h if

otherwisemQ

0





(2)



(3)


(4)





(5)




(6)



k and gQ







gg g E g g E

Ete e e l l l

i

Q Q QQ

= - + --

( ) ( )

g g q g gi k k k= + -( )+ +1 1Q Q Q

y x e y z x dzjx

j ( )( ) ( ) = -

-yenyen -ograve 1

2

2 22

pss

y t

V t

j

m

( )

( )

=

+-[ ]

aelig

egraveccedilccedil

ouml

oslashdividedivide

+

1

1 1

g Q

h j e i i j i i ji






(7)



gx x

hi j i=

gtigraveiacuteicirc

h if

otherwisemQ

0





(7)



gx x

hi j i=

gtigraveiacuteicirc

h if

otherwisemQ

0

interactions between frontal cortex and basal ganglia in...

Documents