lth,f Pit

romus oeviebytheli. Tethgle

behavior. We conclude that multiple processes and pathways contribute to active vision in the primate

rld thaerns ofal repron witbraintention wor

brain areas responsible for generating an internal image of theworld. This image is itself shifted so as to stay in alignment withthe new visual information that will arrive following the eye move-ment. This simple proposal seems to suggest that the entire imageshould be shifted when the eyes move.

tended visual stimuli; the brain circuits that produce remapping;evidence for remapping in humans; and the neural basis of activevision.

2. Parietal mechanisms of attention

2.1. Activity in lateral intraparietal area neurons

Neurons in the lateral intraparietal area (LIP) are active undermany conditions. When studied in a standard memory-guided

* Corresponding author. Fax: +1 301 402 0511.E-mail addresses: bermanr@nei.nih.gov (R. Berman), ccolby@cnbc.cmu.edu

(C. Colby).

Vision Research 49 (2009) 12331248

Contents lists availab

Vision Re

.e lsevier .com/locate /v isres1 Fax: +1 412 268 5060.percept of a stable visual world.Why does the world appear to stay still when we move our

eyes? Attempts to understand our experience of spatial constancyhave a long history. More than a century ago, Helmholtz (Helm-holtz, 1866) hypothesized that the effort of will involved in mak-ing an eye movement simultaneously adjusts our perception totake that eye movement into account. He proposed that when amotor command is issued to shift the eyes in a given direction, acopy of that command, known as a corollary discharge, is sent to

changes in our visual environment (ORegan and Noe, 2001;Simons & Rensink, 2005). We dont see what we dont attend toand so for many objects there is no spatial constancy problem.For those things that we do attend to, updating portions of theinternal image may be part of the solution. What does this meanin neural terms? In the following sections we describe a neuralmechanism for spatial updating or remapping, and how it de-pends on attention. We begin with attentional effects at the singleneuron level in parietal cortex and then discuss remapping of at-1. Spatial constancy and attention

Vision is an active process. The wodirect impression derived from pattstead, the brain constructs an internbased on sensory input in conjunctimotor signals. Sensory activity in thetion and memory and even by the inon how attention and motor intentio0042-6989/$ - see front matter 2008 Elsevier Ltd. Adoi:10.1016/j.visres.2008.06.017brain. 2008 Elsevier Ltd. All rights reserved.

t we see is not simply alight on the retina. In-esentation of the worldh central cognitive andis modulated by atten-n to act. We focus herek together to create the

But does the whole image really move? When we begin to thinkabout brain mechanisms of spatial constancy, it seems improbablethat every detail of a visual image could be updated with every eyemovement. It is more likely that multiple mechanisms contributeto solving the problem of spatial constancy. This is where attentionis critical. Much of the problem of spatial constancy may be solvedby simply not attending to things. We dont see most of what hap-pens around us; extensive research on the phenomenon of changeblindness has shown that we are generally oblivious to unattendedMacaqueImaging

well as subcortical structures. This neural circuitry is distinguished by signicant redundancy and plas-ticity, suggesting that the updating of salient stimuli is fundamental for spatial stability and visuospatialAttention and active vision

Rebecca Berman a,*, Carol Colby b,1

a Laboratory of Sensorimotor Research, National Eye Institute, National Institutes of HeabDepartment of Neuroscience and Center for the Neural Basis of Cognition, University o

a r t i c l e i n f o

Article history:Received 10 October 2007Received in revised form 11 June 2008

Keywords:UpdatingParietalNeuron

a b s t r a c t

Visual perception results fmotor signals. Here we focvisual cortical areas. We rselective attention but alsoand human that elucidatesentations of salient stimusely intertwined, and togaccomplished not by a sin

journal homepage: wwwll rights reserved.Room 2A50, 49 Convent Drive, Bethesda, MD 20892, USAtsburgh, 115 Mellon Institute, 4400 Fifth Ave, Pittsburgh, PA 15213, USA

the interaction of incoming sensory signals and top down cognitive andn the representation of attended locations in parietal cortex and in earlierw evidence that these spatial representations are modulated not only bythe intention to move the eyes. We describe recent experiments in monkeymechanisms and circuitry involved in updating, or remapping, the repre-wo central ideas emerge. First, selective attention and remapping are clo-er contribute to the percept of spatial stability. Second, remapping isarea but by the participation of parietal, frontal and extrastriate cortex asle at ScienceDirect

search

saccade task (Hikosaka & Wurtz, 1983), neural activity can occurthroughout the trial (Barash, Bracewell, Fogassi, Gnadt, & Ander-sen, 1991a; Colby, Duhamel, & Goldberg, 1996). The task requiresthe monkey to maintain xation while a stimulus is presented inthe receptive eld. After the stimulus has been presented, the mon-key continues to xate during a delay period. At the end of the de-lay period, the xation point is extinguished. This serves as a cue tothe monkey to make a saccade to the location where the stimuluspreviously appeared. The monkey is typically required to maintainxation at this new location for a brief period and then a reward isgiven. At the time of the eye movement, there is no stimulus on thescreen so the monkey must use a memory trace of the stimuluslocation to perform the correct saccade. Neural activity in this taskis typically measured in at least four main epochs: a baseline epoch(after xation has begun and before the stimulus appears); a visualepoch (after the stimulus appears); a saccade epoch (around thetime of the eye movement); and a delay period epoch encompass-ing the interval between the end of the visual stimulus and theoccurrence of the cue that tells the monkey to make an eye move-ment. In the following sections we describe the effects of attentionon neural activity in three of these task epochs.

2.2. Attentional modulation in anticipation of a stimulus

The baseline ring rate of neurons in area LIP increases whenthe animal is working in a task in which it expects that a behavior-ally-relevant stimulus will appear. Two tasks have been used to

rons in area LIP increase their baseline level of activity in this task(Colby et al., 1996). Even before the stimulus appears, the cell resmore in the memory-guided task (Fig. 1B) than it does when theanimal is simply xating without the expectation of seeing anattention-demanding stimulus (Fig. 1A). This increase in baselineactivity is observed in the population of LIP neurons (Fig. 1C) andreects increased attention directed toward the spatial locationwhere the animal anticipates that an important stimulus willappear.

Anticipatory attentional modulation during the baseline epochis also found in a second task in which the monkey attends tothe stimulus in the receptive eld without ever generating aneye movement to it. In this peripheral attention task, the monkeyinitiates the trial by grasping a lever, which causes the xationpoint to be illuminated. The animal must maintain xation as longas the central xation point is present. The stimulus appears in thereceptive eld as usual but now the animal has to attend to it with-out looking at it. When the stimulus dims slightly, the animal hasto release the lever. In this task, as in the memory-guided saccadetask, baseline activity in the xation epoch preceding stimulusappearance is increased. In both tasks, baseline neural activity in-creases specically when the animal expects a stimulus relevantfor behavior to appear in the receptive eld (Colby et al., 1996).The increase is not a change in the visual response itself but ratherin the neurons readiness to respond. Similar increases in baseline,pre-stimulus activity are present in ventral stream areas includingareas V2 and V4 (Luck, Chelazzi, Hillyard, & Desimone, 1997).

chie

uid

pleganlus a

1234 R. Berman, C. Colby / Vision Research 49 (2009) 12331248demonstrate this attentional modulation of baseline activity (Colbyet al., 1996). In the memory-guided saccade task, described above,a delay is imposed between the time that a stimulus appears andthe time that the animal is allowed to look at it. During the delay,the animal has to remember where the stimulus appeared. We in-fer that the animal has to attend to the stimulus. In addition, whentrials to a single location are presented consecutively, the animalcan anticipate at the beginning of each trial where a behavior-ally-relevant stimulus is about to appear. As shown in Fig. 1, neu-

VH

FPStim

VH

FPStim

FP

RF

FP

Fixation achieved Fixation a

Fixation Memory-g

Fig. 1. Attentional modulation of baseline activity in area LIP. The activity of an examsaccade task (B). Rasters and histograms are aligned on the time when the monkey belines from a sample trial, above the rasters, show the earliest time at which the stimu

stimulus in receptive eld (RF). In the xation trials, the neuron does not respond until tbegins to build up before the stimulus appears. In the population (C), baseline activity waxation task (x axis). From Colby et al. (1996).Attentional modulation of baseline activity has also been observedin human frontal, parietal, and extrastriate visual cortex (Kastner,Pinsk, De Weerd, Desimone, & Ungerleider, 1999; McMains, Fehd,Emmanouil, & Kastner, 2007).

2.3. Attentional enhancement of visual responses

The visual response itself can also be modulated by attention.LIP neurons respond more strongly to the onset of an attended

RF

ved Baseline activity in fixation task

Base

line

activ

ity in

mem

ory-

guid

ed s

acca

de ta

sk

ed saccade

LIP neuron is shown during performance of a xation task (A) and a memory-guidedto xate the central xation point, even before the appearance of the stimulus. Timeppeared. V: vertical eye position, H: horizontal eye position, FP: xation point, Stim:

he stimulus appears. In memory-guided saccade trials, by contrast, baseline activitys greater on average for the memory-guided saccade task (y axis) compared with the

thus contribute to maintaining the spatial alignment between theexternal world and its internal representation.

Resestimulus than they do to the same stimulus when the animal issimply xating and is free to ignore the stimulus. This augmentedvisual response occurs in both the memory-guided saccade taskand in the peripheral attention task (Bushnell, Goldberg, & Robin-son, 1981; Colby et al., 1996). As described above, the monkeymaintains central xation throughout the peripheral attention taskand never makes an eye movement, either during the trial or dur-ing the intertrial interval. Nonetheless, the stimulus in this task isrelevant for the animals behavior because it must respond to aslight dimming of the stimulus. The visual response to the onsetof the stimulus in the peripheral attention task is larger than theresponse to the same stimulus in the context of a simple xationtask. The increase in the number of spikes is comparable to theenhancement observed in the memory-guided saccade task (Colbyet al., 1996). The interpretation is that in both cases the responseenhancement reects an increased level of attention to behavior-ally-relevant stimuli. Both types of enhancement occur togetherin LIP neurons that have only visual responses and no saccade re-lated activity. Attentional enhancement of visual responses is inde-pendent of motor planning. The role of area LIP in covert shifts ofattention has been conrmed by reversible inactivation studies(Wardak, Olivier, & Duhamel, 2004) and studies of neural activityin additional tasks that require covert attention (Balan & Gottlieb,2006; Bisley & Goldberg, 2003, 2006).

Attention modulates visual activity in many cortical areas,including dorsal stream areas V3A (Nakamura & Colby, 2000) andMT (Seidemann & Newsome, 1999; Treue & Martinez Trujillo,1999; Treue & Maunsell, 1996); see (Treue, 2003) for review).The impact of attention on visual responses has been studiedextensively in ventral stream area V4 (Armstrong & Moore, 2007;Connor, Preddie, Gallant, & Van Essen, 1997; Fries, Reynolds, Rorie,& Desimone, 2001; Hayden & Gallant, 2005; McAdams & Maunsell,2000; Mitchell, Sundberg, & Reynolds, 2007; Moore & Armstrong,2003; Moran & Desimone, 1985; Reynolds & Desimone, 2003; Rey-nolds, Pasternak, & Desimone, 2000). Attention can also modulatevisual responses at still earlier stages of processing, including V1and the lateral geniculate nucleus (Casagrande, Sary, Royal, & Ruiz,2005; McAdams & Reid, 2005; McAlonan, Cavanaugh, & Wurtz,2007; Motter, 1993). Imaging studies likewise have shown thatattention modulates visual responsivity in the human brain, atboth earlier and later stages of the visual hierarchy (Liu, Larsson,& Carrasco, 2007; McMains et al., 2007; OConnor, Fukui, Pinsk &Kastner, 2002). In sum, attention acts upon sensory signals at manylevels to construct a selective representation of visual space.

2.4. Attention and delay period activity

Many LIP neurons are active during the delay period betweenthe appearance of a visual stimulus and the generation of a saccad-ic response. Delay period activity in LIP is strongly modulated bythe kind of motor response (saccade or reach) that the monkeymight make to a stimulus (Cui & Andersen, 2007; Dickinson, Cal-ton, & Snyder, 2003; Lawrence & Snyder, 2006). This delay periodactivity before an eye movement has been variously interpretedas working memory (Gnadt & Andersen, 1988), expectation orevaluation of reward (Dorris & Glimcher, 2004; Platt & Glimcher,1999; Sugrue, Corrado, & Newsome, 2004), motor planning(Mazzoni, Bracewell, Barash, & Andersen, 1996), accumulation ofsensory evidence (Huk & Shadlen, 2005, Palmer, Huk & Shadlen,2005) and decision making (Shadlen & Newsome, 2001). Many ofthe factors considered in these studies are consistent with anattentional interpretation as well. As the monkey plans a saccadetoward a particular location, or remembers a location, or assesses

R. Berman, C. Colby / Visionthe probability that a saccade to a location will yield a reward, itis also attending to that location. A parsimonious explanation ofdelay period activity is that it reects attention to a spatial location3.1. Remapping requires the combination of visual and motor signals

What produces this remapped response? The hypothesis is thata corollary discharge of the eye movement triggers a transfer ofinformation. Neurons that initially represent the stimulated, at-tended location transfer their activity to another population ofneurons, those which will represent that location after the saccade.There are two reasons to think that the triggering signal must be acorollary discharge rather than based on proprioception. First, at(Colby & Goldberg, 1999; Maunsell, 2004). Activity in area LIP hasbeen interpreted as a salience map, representing locations that arerelevant for behavior (Goldberg, Bisley, Powell, & Gottlieb, 2006;Gottlieb, 2007; Gottlieb, Kusunoki, & Goldberg, 2005). The ongoingactivity during the delay period can be characterized as a memorytrace of an attended location. This memory trace can only be usefulfor guiding eye movements if it is maintained in an eye-centeredrepresentation. The question we are concerned with here is whathappens to the memory trace of an attended location when theeyes move?

3. Remapping of attended visual stimuli

We studied the fate of stimulus memory traces by developing asimple paradigm called the single step task (Duhamel, Colby, &Goldberg, 1992a). The idea was to see if we could better under-stand, at the single neuron level, the process of adjusting the align-ment between the new retinal image and the corticalrepresentation of a previous stimulus. As shown in Fig. 2, we rstrecord the neurons response to a visual stimulus presented inthe receptive eld while the animal xates (Fig. 2A). This responsein the xation task is the ordinary, expected visual response. Thesingle step task reveals an unexpected response that indicates adynamic updating of the internal visual representation. In the sin-gle step task (Fig. 2B), the monkey xates while a stimulus is pre-sented briey (50 ms) at a screen location that is well outside theneurons receptive eld. A subsequent eye movement to a new x-ation point brings the receptive eld onto the location of theashed stimulus. The neuron responds following this eye move-ment even though the stimulus has been extinguished well beforethe onset of the saccade. We refer to this as a stimulus trace (ormemory trace) response because the neuron is responding to astimulus that is no longer there.

Most neurons in LIP are activated in this single step remappingtask. Two control tasks demonstrate that this activity is a responseto the updated memory trace of the stimulus. Neurons are not ac-tive when the stimulus is presented without any subsequent sac-cade (Fig. 2C). Likewise, they are not active when the monkeymakes the same saccade without any stimulus having appeared(Fig. 2D). In the single step task, LIP neurons respond as thoughthere were an actual stimulus in the receptive eld. Activity inthe single step task can only be a response to the updated memorytrace of the stimulus: there is no physical stimulus in the receptiveeld at time one, before the saccade, and likewise no stimulus inthe receptive eld at time two, after the saccade. This responseto the memory trace of a previous stimulus indicates that LIP neu-rons participate in updating an internal representation of space.We call this process remapping to emphasize that visual infor-mation is being shifted from the coordinates of the initial eye posi-tion to the coordinates of the next eye position. Remapping could

arch 49 (2009) 12331248 1235the single neuron level, remapping can occur before the saccadebegins, before there can be any proprioceptive signal. Second,behavioral experiments have tested whether proprioception is

Rese1236 R. Berman, C. Colby / Visionnecessary for performance on a task thought to require remapping,the double step task. These experiments show that performanceon the task can be accurate even after all proprioceptive input fromthe eyes has been removed (Guthrie, Porter, & Sparks, 1983). Theconclusion is that information about the intention to make a sac-cade evokes a transfer of visual information.

In order to test whether this transfer of visual information de-pends on corollary discharge signals, we devised a task in whichwe could look for remapping in the absence of an eye movement.We asked whether a shift of attention alone would be as effectiveas an eye movement in evoking a remapped visual response. Inother words, is a covert shift of attention alone sufcient to remapthe internal representation of a stimulus? We tested this proposi-

Fig. 2. Remapping in area LIP. In a simple xation condition (A), the neuronresponds to the stimulus in its receptive eld. In the single step task (B), thestimulus is ashed outside the receptive eld for 50 ms, as indicated by the timeline. The stimulus has disappeared before the eyes move to the second xationpoint, FP2. The LIP neuron res after the saccade brings the receptive eld onto thepreviously stimulated location. In the stimulus-alone control (C), the stimulus ispresented outside the receptive eld and no saccade is made; this does not drive theneuron. In the saccade-alone control (D), the same eye movement is made but doesnot drive the neuron in the absence of the stimulus. Adapted from Duhamel et al.(1992a).tion in a variant of the peripheral attention task. As in the standardsingle step task, a stimulus was presented outside the receptiveeld of the neuron under study. Simultaneously, a new xationpoint appeared on the screen. When the animal simply shiftedits attention to the new location (in order to detect a slight dim-ming) without generating an eye movement there was no responsefrom the neuron. In contrast, when the animal made a saccade tothis target in the standard single step task, a remapped visual re-sponse was evoked (Colby, 1996).

The failure to trigger remapping with attention alone is critical,because it provides further evidence that the function of remap-ping is to maintain an accurate alignment between the visualworld and its internal representation. With an attentional shiftalone, nothing moves on the retina and there is no need to remapthe internal representation. Indeed, it would be counterproductiveto do so because such a shift would introduce a mismatch betweenthe external world and the internal image of it. When a saccade isabout to occur, however, parietal cortex can make use of the infor-mation about the intended eye movement to anticipate the retinalconsequences of that saccade and update the stored representationof object locations.

Having described the basics of attentional processes and remap-ping in area LIP, we turn now to studies of the nature of the re-mapped signal and temporal and spatial aspects of remapping.

3.2. What remaps?

Saccade generation is the nal trigger for remapping, yet atten-tion seems to be a necessary prerequisite. Studies by Gottlieb andcolleagues, described below, indicate that images are remappedonly if they are rst attended (Gottlieb, Kusunoki, & Goldberg,1998). Attention can be directed toward an object or locationthrough either bottom-up or top-down mechanisms (see (Moore,2006) for review). Sudden stimulus onsets automatically attractattention, and as such, are an example of bottom-up mechanismsof attention (Jonides & Yantis, 1988; Yantis & Jonides, 1984). Bycontrast, top-down mechanisms can be invoked when a stimulusis relevant for voluntary behavior. Gottlieb et al. studied the effectsof both bottom-up and top-down attention on remapping re-sponses in area LIP. In the standard remapping tasks describedabove, the to-be-updated stimulus is never a target for the animalsbehavior but it does always have a sudden onset. Gottlieb and col-leagues rst asked whether this sudden onset was important forremapping. They used a stable array task in which there were nosudden stimulus onsets. They found that LIP neurons did not re-map when the receptive eld was brought onto a stable, continu-ously visible stimulus embedded in a stable, continuously visiblearray. In other words, unattended stimuli were not remapped. If,however, the to-be-updated stimulus was briey ashed withinthe stable array, making it salient, LIP neurons did show remappingactivity. This observation indicates that attention, here broughtabout by bottom-up mechanisms, is a prerequisite for remapping.

Attention can also be controlled through top-down mecha-nisms. Gottlieb and colleagues went on to ask whether thesetop-down mechanisms also inuenced remapping in LIP. Theyagain used a stable array task with no sudden stimulus onsets. Inthis case, however, the animal was cued to make a sequence of sac-cades that would bring a stimulus into the receptive eld of a neu-ron. They found that a neuron would remap the stimulus locationonly when it had been selected as the target and entered the recep-tive eld as the result of a saccade. When an unattended stimulusentered the receptive elda stimulus that was not used to guidethe saccadethere was no remapping. From this set of experi-

arch 49 (2009) 12331248ments, they concluded that remapping depends on the salienceof a stimulus, regardless of whether that stimulus has been madesalient by bottom-up or top-down mechanisms. Their ndings

reinforce the notion that remapping is an attentional phenomenon.The necessity of attention indicates an economy of processing: thebrain updates only what it needs to in order to maintain a stableimage.

3.3. When does remapping occur?

Remapping depends on the conjunction of an attended stimulusand the intention to move the eyes. Beyond that, what have welearned about the timing of remapping? The intention to movethe eyes of course precedes the actual eye movement. The transferof visual signals that is triggered by the corollary discharge can intheory occur at any time after the decision to move the eyes. One ofthe most compelling benets of remapping is that visual responsesto a stimulus can occur before the normal visual latency of the neu-ron. These predictive signals are available well before reafferent vi-sual signals are available. In other words, LIP neurons do not needto wait for signals to arrive from lower visual areas every time theeyes move to a new position. Instead, the remapped response takesadvantage of a corollary discharge signals to yield a representationthat is aligned with the anticipated new position of the eyes.Remapping is considered predictive if the response, when alignedon the saccadic eye movement, is earlier than the expected visuallatency observed in a xation task. For some predictive neurons,the location of the receptive eld shifts even before the eyes beginto move. These presaccadic responses are a subset of predictive re-

before the saccade begins (bottom panel, Time 1). This observationtells us that remapping is a rapid process and led us to ask aboutthe temporal limits of remapping.

What is the earliest that remapping can occur? In area LIP, wecompared ordinary visual response latencies in a xation task toresponse latencies in the single step task (Duhamel et al., 1992a).The standard way of measuring latency in the single step task isto align activity on the start of the eye movement. For these data,however, we also aligned single step activity on the appearance ofthe stimulus for direct comparison. The most rapid remapping wasobserved in a neuron with a normal, visual response latency of70 ms (Fig. 4). Its latency of response in the single step task was83 ms when aligned on stimulus appearance (note that this is a re-sponse to a stimulus that is outside the receptive eld). This is only13 ms longer than the cells normal visual latency. It means that insome cases, the whole circuit can accomplish its task within just afew milliseconds. This observation helps us think about what thecircuit might be. There is time for only a few synapses so the circuitshould ideally be local and short.

3.4. How long do memory traces last?

An intriguing question about the temporal limits of remappingconcerns how long memory traces are maintained. The longest thathas been systematically tested is up to one second: the stimulus isashed for 50 ms and the new xation point appears 1 s later

s pr

R. Berman, C. Colby / Vision Research 49 (2009) 12331248 1237sponses. Many neurons in area LIP exhibit predictive responses andthus anticipate the visual consequences of the eye movement. In arecent investigation of remapping in area LIP, we found that abouttwo-thirds of neurons exhibited predictive responses (Heiser,Berman, Saunders, & Colby, 2005). These predictive responsesmay be particularly useful for maintaining spatial constancy. Thetiming of remapping has been examined in detail in area LIP(Kusunoki & Goldberg, 2003) and in extrastriate area V3A(Nakamura & Colby, 2002). The V3A neuron illustrated in Fig. 3 isstrongly activated by a stimulus ashed at the new receptive eldand has a presaccadic remapped response, more than 100 ms

Fig. 3. Timing of remapping in V3A. Response of an example V3A neuron to a stimulu

four different timings. Time lines (top panel) show trial events. Data are aligned on stimuthe new xation point. The neuron begins to respond at the new, future RF at Time 1, lonold, original RF at both Time 1 and Time 2, indicating a dual responsivity that encompa(Duhamel et al., 1992a). In some neurons the remapped responseafter this delay was just as strong as when there was no delay. Gi-ven that remapping is essentially an attentional phenomenon, itmay be that the memory trace remains active until some otherevent captures attention. This possibility remains to be tested.

3.5. Is remapping universal across directions?

Our daily experience tells us that spatial constancy is main-tained no matter what size or direction of eye movement we make.We asked whether this universal insouciance was present at the

esented in the old receptive eld (old RF, middle panel) or new RF (bottom panel) at

lus onset. Inverted triangles show the mean time of the beginning of the saccade tog before the eye movement begins. The neurons also responds simultaneously at thesses the old and new RF locations. From Nakamura and Colby (2002).

theshothisr dimapthDu

1238 R. Berman, C. Colby / Vision Resesingle neuron level. For saccades of different sizes (10 vs. 20degrees) we found no difference in the strength of the remappedsignal (Nakamura & Colby, 2002).

For saccades in different directions the answer was more com-plicated (Heiser & Colby, 2006). We asked whether neurons in LIPremap when the eyes move in the four canonical directions. At thesingle neuron level, some cells actually do remap in all four direc-tions. For most neurons, however, remapping is detectable only forsome directions and the strength of remapping varies with saccadedirection. Overall, neurons that exhibit remapping in multipledirections have more robust remapped responses. We also foundthat the prevalence of remapping is independent of receptive eld

Fig. 4. Timing of remapping and truncation in area LIP. The rst panel (A) showscontinues to respond even after the stimulus has disappeared. The second panel (B)brings the receptive eld onto a location where a stimulus is present. Note that inilluminated. The single step data are aligned on stimulus appearance (B, left panel) foare also aligned on the beginning of the saccade (B, right panel), which shows that reactivity of the neuron when an eye movement moves the receptive eld away fromcontrast to the slow decay of activity after stimulus disappearance in panel A. Fromlocation. Remapping was as common in neurons with peripheralreceptive elds as in those with central receptive elds. Finally,we found that remapping is independent of the response proper-ties exhibited in the memory-guided saccade task: visual andvisuomovement cells in LIP are equally likely to exhibit remapping.

Despite the variability in single neuron responses, we found atthe population level that all eye movement directions are equally

Fig. 5. Remapping in LIP is independent of saccade direction. A polar plot shows thedistribution of preferred remapping directions in area LIP. Across the population,preferred directions are distributed throughout the visual eld. From Heiser andColby (2006).good for remapping (Fig. 5). This observation tells us that behaviorshould be equally good in all directions. Behavioral experimentssuggest that this is true. For example, accuracy on the double steptask does not differ for horizontal vs. vertical initial saccades(Ram-Tsur, Caspi, Gordon, & Zivotofsky, 2005). For the LIP popula-tion as a whole, we found no difference in the latency of remappedresponses for different directions. We were particularly interestedin whether neurons might respond differently when the stimulustrace had to be remapped within or across hemields. We foundthat LIP neurons respond with similar time course and magnitudefor either situation. We conclude that LIP neurons can access infor-mation from throughout the visual eld and spatial updating is

visual response of an example LIP neuron while the monkey xates. This neuronws the activity of the neuron during the single step task, when a leftward saccadeversion of the single step task, the stimulus in the future receptive eld remainsrect comparison with panel A, and show a rapid remapped response. The same dataping in this neuron starts before the eyes begin to move. The third panel (C) showse stimulated location. This eye movement causes a sharp truncation of activity, inhamel et al. (1992a).

arch 49 (2009) 12331248independent of saccade direction.

3.6. Do receptive elds shift or expand?

When a stimulus trace is remapped, what happens to the recep-tive eld of the neuron?

There are multiple possibilities. One is that the receptive eldmay make a discrete shift: sensitivity to visual stimuli could shiftentirely from the original receptive eld location to the anticipatednew location or future eld. Another possibility is that the recep-tive eld expands like a slinky to encompass both the original andfuture eld location. We looked at these possibilities by placingstimuli at one of two locations, the current receptive eld or the fu-ture receptive eld (Nakamura & Colby, 2002). We found that V3Acells fall along a continuum. Some appear to have a discrete shift ofthe receptive eld around the time of an intended saccade. Theybecome less responsive at the old location while simultaneouslybecoming much more responsive at the future location of thereceptive eld. Other neurons appear to undergo a momentaryexpansion of the receptive eld immediately before a saccade.These neurons responded to a stimulus at the future receptive eldeven when it was presented long before the saccade (Fig. 3). Thisdual responsiveness suggests that the receptive eld has expandedto encompass both locations.

Neurons in area LIP and in the frontal eye elds also exhibit thisdual responsiveness (Kusunoki &Goldberg, 2003; Sommer&Wurtz,2006). Recent behavioral experiments (Jeffries, Kusunoki, Bisley,Cohen, & Goldberg, 2007) suggest that this dual responsiveness

may relate to mislocalizations that occur around the time of the eyemovement, such as those originally observed by Matin and Pearce(Matin & Pearce, 1965). A critical experiment by Sommer andWurtz(2006) recently added to our understanding of how eyemovementsaffect the structure of receptive elds in the frontal eye eld. Theyfound that even thoughaneuronmay respond to stimuli at two loca-tions, the receptive eld does not expand indiscriminately betweenthe two points. Specically, frontal eye eld neurons respond at theoriginal and future receptive elds just before a saccade, but do notrespond to a stimulus placed in between. This nding indicates thatthe receptiveeld expansion is not simplya continuous spreadbut isgovernedby the coordinates of the impending eyemovement: as theeyes move from point A to point B, the neurons sensitivity changesat the corresponding original and future eld locations. This partic-ular aspect of remappinghasnot yet been studied inother areas such

The effects of remapping are also evident when a saccade shiftsa stimulus out of the receptive eld. For the LIP neuron shown in

essential for guring out the neural mechanism that produces it. Asa rst step toward elucidating the mechanism, we asked where inthe brain remapping takes place. Since the initial discovery ofremapping in area LIP, many brain regions have been found to haveneurons that remap stimulus traces. These include oculomotorareas, such as the frontal eye elds and the superior colliculus.Approximately 60% of visually-responsive neurons in the frontaleye elds exhibit remapping (Umeno & Goldberg, 2001), as do30% of neurons in the intermediate layers of the superior colliculus(Walker, FitzGibbon, & Goldberg, 1994).

We asked whether remapping is limited to these oculomotorand quasi-oculomotor areas, or whether it can be observed in areasthought to be primarily visual in function. We expected that ifremapping actually has something to do with the perception ofspatial constancy we might be able to detect it in regions that

R. Berman, C. Colby / Vision Research 49 (2009) 12331248 1239Fig. 4, two conditions are shown that are equivalent in terms ofwhat happens on the retina. In panel A, the stimulus in the recep-tive eld is turned off. The neuron continues to re for some time.In panel C, the monkey makes an eye movement that moves thereceptive eld away from the stimulus. In both cases, the stimulushas been removed from the receptive eld. But the effect on thisparietal neuron is very different. Neural activity is abruptly trun-cated by the eye movement. This truncation tells us that an activeprocess has shut down the cells ongoing response. Truncation isthe necessary obverse of remapping. It demonstrates that corollarydischarge signals are used not only to increase sensitivity at theanticipated future receptive eld but also to decrease sensitivityat the original receptive eld. Truncation is another means bywhich the alignment between the internal representation of an at-tended stimulus and incoming visual signals is maintained.

4. Brain circuits that produce remapping

4.1. Where does remapping take place?

The above observations on remapping imply that receptiveelds in cerebral cortex must be dynamic and not at all like simplelabeled lines. Discovering the circuitry that produces remapping is

% o

f neu

rons

with

re

mapp

ed re

spon

ses

V1 V2 V3 V3A

Monkey

Post-saccadic remapping

Pre-saccadic remappingas LIP,V3A,or theSC. It is notyet clearhowthesephysiological obser-vations relate to psychophysical ndings (Melcher, 2007). The rela-tionship between receptive eld changes and perisaccadicperception remains an important question for future experiments.

3.7. TruncationFig. 6. Remapping in early visual areas. In both monkey (A) and human (B), there is an inIn monkey, the proportion of neurons with presaccadic remapping (hatched shading) alsMerriam et al. (2007).we think of as being more purely visual in function. We began inarea V3A with our standard remapping task (Nakamura & Colby,2002). We found that more than half of V3A neurons (52%) re-sponded in the single step task. Remapped responses in area V3Aare as robust as those in area LIP, and V3A neurons were not activein the control conditions. We went on to ask whether neurons ateven earlier stages of the visual system hierarchy would remapstimulus traces (Fig. 6A). Using the same tasks and conditions,we tested neurons in areas V3, V2, and V1 (striate cortex). Wefound that many neurons in area V3 respond in the single step taskbut that the proportion drops off rapidly in V2. In striate cortex,only 1 neuron out of 64 tested showed evidence of remapping.Two other trends are clear. First, the mean latency of the remappedresponse relative to saccade onset was much longer at lower levels.Second, in relation to the rst trend, the proportion of neurons thatremap predictively decreased markedly at lower levels of the hier-archy (Fig. 6A, hatched shading). All of these ndings suggest thatearlier stages of the visual system are connectionally or computa-tionally further from the source of the central signal that drivesremapping. The next question for future research is whetherremapping is in fact an entirely top-down process, in which thecomputation is carried out in LIP (or elsewhere), or whether it pro-ceeds in parallel at multiple levels of the visual system. We con-clude that remapping is not limited to neurons in oculomotorand closely allied brain regions but rather is present in dorsalstream visual areas as well.

4.2. Converging visual attention and motor signals

What are the neural circuits that generate remapping? In thissection we consider the signals used to construct a dynamic, up-dated representation of space, and their neural underpinnings. Aswe have described, remapping must involve the convergence of

% o

f vox

els

with

re

mapp

ed re

spon

ses

hV4V1 V2 V3 V3A

Humancrease in the proportion of remapped responses as one ascends the visual hierarchy.o increases at higher levels of the hierarchy. From Nakamura and Colby (2002) and

Reseat least two kinds of signals. The rst is a visual signal, which indi-cates the location of a salient, attended stimulus. The second is amotor signal: a corollary discharge of the impending eye move-ment must be used to update the visual representation.

Where in the brain do these visual and corollary dischargesignals converge? Area LIP is a likely candidate. Studies in humanand monkey have demonstrated that parietal cortex is essentialfor behaviors that require spatial updating. One well-establishedbehavioral measure of updating is the double step task (Hallett &Lightstone, 1976). This task allows us to measure the ability tokeep track of a spatial location when the eyes move. In the doublestep task, the subject must make eye movements to two succes-sively ashed targets, T1 and T2 (Fig. 7). The key feature is thatthe second target (T2) disappears before the eyes leave the xationpoint (FP). If the subject generates the sequence based only on theretinal location of the second target, the second saccade will beincorrect. For accurate performance of the sequence, the locationof T2 must be updated in conjunction with the rst saccade. Bothhumans and monkeys are able perform the double step taskaccurately (Baizer & Bender, 1989; Gnadt & Andersen, 1988; Gold-berg & Bruce, 1990; Hallett & Lightstone, 1976; Mays & Sparks,1980; Medendorp, Goltz, & Vilis, 2006; Ray, Schall, & Murthy,2004).

Neuropsychological studies have shown that accurate updatingin the double step task depends on parietal cortex (Duhamel, Gold-berg, Fitzgibbon, Sirigu, & Grafman, 1992b; Heide, Blankenburg,Zimmermann, & Kompf, 1995; Heide & Kompf, 1998). Patients withdamage to parietal cortex are capable of generating slow doublestep sequences when both saccades are visually guided. In thiscase, the trajectory of the second saccade can be computed accu-rately using retinal information. When the targets are ashed in ra-pid succession, however, the second saccade is not visually-guidedbut must be based on the memory trace of the second target. Itsaccurate computation relies on spatial updating of this memorytrace, to account for the intervening saccade to the rst target. Inthis rapid version of the task, parietal patients fail to completethe double step sequence accurately. Impairment is not due to asimple motor decit, because the patients can perform the slow,visually-guided version of the task. Rather, parietal damage is asso-ciated with a specic failure to update the memory trace of a visualstimulus on the basis of corollary discharge. Patients with damageto the frontal eye elds, by contrast, show more generalizedimpairments on the double step task, with decits on the slow,visually-guided version as well as the rapid version of the doublestep task (Heide et al., 1995).

These results suggest that the parietal and frontal lobes mayhave distinct contributions to spatial updating, and that intactparietal function is critical for remapping. Consistent with theseobservations from lesion studies, a recent investigation has shownthat double step performance is disrupted by transmagnetic stim-ulation over parietal cortex in normal humans (Morris, Chambers,& Mattingley, 2007). The importance of parietal cortex for perfor-mance of the double step task has also been demonstrated in mon-keys. Inactivation of area LIP causes decreased accuracy andincreased latencies for the second eye movement in the task (Li& Andersen, 2001). It difcult to draw precise comparison betweenmonkey inactivation studies and human patient data, as the corre-spondence between macaque area LIP and subregions of humanparietal cortex is an area of active research (Konen & Kastner,2008; Schluppeck, Curtis, Glimcher, & Heeger, 2006; Schluppeck,Glimcher, & Heeger, 2005; Sereno, Pitzalis, & Martinez, 2001;Swisher, Halko, Merabet, McMains, & Somers, 2007). Even if thehomologue for area LIP were known in human, parietal lesions

1240 R. Berman, C. Colby / Visionare unlikely to be conned to such restricted subregions. Nonethe-less, these ndings in humans and monkeys point toward parietalcortex as an essential site for the convergence of corollary dis-charge information and the visual representation of salientlocations.

The anatomical connectivity and physiological properties ofarea LIP make it well suited to act as an central node in the networkthat constructs updated visual representations. LIP is a higher-or-der association area in the dorsal visual stream, with input frommultiple lower-level visual areas. As described above, visual activ-ity in LIP is strongly modulated by spatial attention. LIP thereforeseems ideally suited to supply the visual information necessaryfor creating a remapped representation: it encodes the locationof attended visual stimuli in eye-centered coordinates. The nextquestion iswhat is the origin of the corollary discharge signalsthat interact with these visual representations?

4.3. Corollary discharge pathways

What are the pathways for the transmission of corollary dis-charge signals? Recent research has identied a pathway for cor-ollary discharge related to eye movements (see (Sommer &Wurtz, 2008), for review). The pathway they investigated origi-nates in the intermediate layers of the superior colliculus (SC).Here, presaccadic burst neurons send eye movement commandsdown to the brainstem nuclei that drive saccades. These neuronssimultaneously send a copy of these commands, the corollarydischarge, up to cortex via the thalamus. These corollary dis-charge signals are relayed to the frontal eye eld (FEF) throughthe mediodorsal thalamus (Sommer & Wurtz, 2002). Disruptionof this pathway interferes with updating. Inactivation of themediodorsal thalamus causes a selective decit in accuracy andprecision of the second saccade in the double step task. Thisbehavioral nding tells us that the pathway transmits a corollarydischarge signal for accurate spatial updating. In an elegantstudy, Sommer and Wurtz showed that the disruption of thispathway also affects neural activity in FEF associated withremapping (Sommer & Wurtz, 2006). Specically, inactivationof the mediodorsal thalamus reduces the strength of the re-mapped visual activity in FEF. These behavioral and physiologicaldata indicate that the pathway from SC, via the mediodorsalthalamus, transmits corollary discharge signals to the cerebralcortex. Area LIP could easily receive corollary discharge signalsfrom the FEF, as the two areas are strongly interconnected (Bul-lier, Schall, & Morel, 1996; Chafee & Goldman-Rakic, 1998, 2000;Petrides & Pandya, 1984; Schall, Morel, King, & Bullier, 1995;Stanton, Bruce, & Goldberg, 1995). Area LIP may also receive cor-ollary discharge signals from the SC via the pulvinar, a pathwaythat has not yet been physiologically investigated (Clower, West,Lynch, & Strick, 2001). There are likely to be multiple routes bywhich corollary discharge information can inuence visual repre-sentations. This is suggested by the fact that inactivation of themediodorsal thalamus causes only a partial decit in double stepperformance (Sommer & Wurtz, 2002). As will be discussed be-low, there is behavioral and physiological evidence that corollarydischarge signals can be transmitted through more than onepathway.

4.4. Are direct cortical links needed for remapping?

The idea that multiple pathways underlie remapping is empha-sized by a set of experiments in lesioned animals. The goal of theseexperiments was to determine whether direct cortico-cortical linksare necessary for remapping. The premise is that remapping in-volves communication between neurons in area LIP that encodea salient visual location before the eyes move, and those that will

arch 49 (2009) 12331248encode the location after the eyes move (Quaia, Optican, & Gold-berg, 1998). For example, we can consider a case in which a salientstimulus has appeared 10 degrees to the right of the center of gaze.

-20 -10 0 10 200

10

20

-20 -10 0 10 20

Monkey EM Monkey CH

Verti

cal e

ye p

os. (d

eg)

First session

T1 T1

T2 T2T2 T2 T2T2

FPFirst ten trials

EM

CH

Within Central Across CentralAcross Within

0

10

20

C

Verti

cal e

ye p

os. (d

eg)

Horizontal eye position (deg) Horizontal eye position (deg)-20 -10 0 10 20-20 -10 0 10 20

Final sessionMonkey EM Monkey CH

D

E

A BAcross-hemifield updating Within-hemifield updating

Fig. 7. Impairment and recovery of updating behavior when direct cortical links are disrupted. Top panels show the double step conditions used to test updating behavior insplit brain monkeys. In the across-hemield condition (A), the second target (T2) appears in the right visual eld when the eyes are at central xation and therefore is initiallyrepresented by neurons in the left hemisphere (black T2). When the eyes reach the rst target T1, the memory trace of T2 is now located in the left visual eld and encoded byneurons in the right hemisphere (gray T20). Updating in this condition must involve a transfer of visual information between cortical hemispheres. In the within-hemieldcondition (B), T2 is in the right visual eld both at FP and at T1; updating therefore involves communication within the same hemisphere. Behavioral testing revealed aninitial impairment on across-hemield sequences (C). The inset shows the six randomly interleaved sequences that were tested: trained central sequences (black) and novelwithin (green) and across (red) sequences. Eye traces (B) show double step performance in the rst ten trials of the rst testing session for monkeys EM (top) and CH(bottom). Dots indicate the locations of FP, T1 and T2; scale bar represents 10. Summary data from the rst session (D) show the second-saccade endpoints, which indicate apersistent impairment for monkey EM in both visual elds but a rapid improvement for monkey CH in the left but not the right eld. Summary data from the nal session oftesting (E) show that both monkey ultimately achieved successful performance on across-hemield sequences. Adapted from Berman et al. (2005). (For interpretation of thereferences to color in this gure legend, the reader is referred to the web version of this paper.)

R. Berman, C. Colby / Vision Research 49 (2009) 12331248 1241

ReseThe stimulus location is initially represented by LIP neurons in theleft hemisphere, with retinotopic receptive elds at +10 degrees.After an eye movement of 20 degrees to the right, the same spatiallocation in the world will be represented by LIP neurons in theright hemisphere, with retinotopic receptive elds at 10 degrees.When the command is given to move the eyes, a corollary dis-charge could initiate a transfer of information from the beforecells in the left hemisphere to the after cells in the righthemisphere.

This model suggests that direct communication between thebefore and after cells is important for updating. We hypothe-sized that this requires direct cortico-cortical links. In the examplejust described, we have an opportunity to examine the role of cor-tico-cortical links. In this example, the stimulus representation isupdated across visual hemields, and therefore the memory tracemust be transferred from one cerebral hemisphere to the other.The cortical connections that would allow for this transfer areaccessible to experimental manipulation: the forebrain commis-sures, comprised of the corpus callosum and anterior commissure,are the ber pathways that link the cerebral hemispheres. We rea-soned that if direct cortico-cortical links are necessary for spatialupdating, then we should observe a profound decit in across-hemield updating following transection of these commissures.

We tested this hypothesis by assessing the behavioral and neu-ral correlates of spatial updating in monkeys whose forebrain com-missures were surgically transected (Colby, Berman, Heiser, &Saunders, 2005; Heiser et al., 2005). We tested spatial behaviorusing two conditions double step task. In the across-hemieldupdating condition, as described, the representation of the secondtarget, T2, had to be updated from one visual hemield to the other(Fig. 7A). In the within-hemield updating condition, T2 had to beupdated from one location to another, within the same visualhemield (Fig. 7B). We tested the monkeys performance on theacross-hemield and within-hemield sequences after trainingon a central sequence that did not require across-hemieldupdating (inset, panel C). We then simultaneously introduced theacross-hemield and within sequences. These two types of se-quences were matched for saccade amplitude, counterbalancedfor direction, and were equally new to the monkey, so that any dif-ferences in performance could be attributed to the type of updatingrequired. This approach allowed a within-subject comparison ofupdating within and across hemispheres. We predicted that inthe split-brain monkeys, spatial updating would be severely dis-rupted if not abolished in the across-hemield condition, but unaf-fected in the within condition.

Initial testing conrmed this prediction: the split-brain mon-keys rst exposures to the within and across-hemield conditionsrevealed a striking and selective impairment for sequences that re-quired updating across visual hemields. Eye traces from the uppereld show this initial double step decit (Fig. 7C). Traces from thecentral conditions show that the monkeys were very accurate inthe execution of these trained sequences. Likewise, the monkeyswere able to perform the within conditions with considerableaccuracy, despite the fact that these sequences were entirely novel.In contrast, both monkeys made inaccurate movements on earlyacross-hemield sequences. On these trials, the trajectory of thesecond saccade deviated only slightly from a straight vertical sac-cade. Control experiments in an intact monkey demonstrated thatthe across-hemield impairments were selective to the absence ofthe forebrain commissures. These initial behavioral data, summa-rized in Fig. 7D, are consistent with the prediction that perfor-mance on across-hemield sequences would be impaired whendirect cortical connections were disrupted. To our surprise, how-

1242 R. Berman, C. Colby / Visionever, the across-hemield decit was not permanent. This tellsus that direct cortical links are not necessarily required forremapping.4.5. Multiple pathways for the transfer of visual signals

We found that the split-brain monkeys behavior recovered,very rapidly for some sequences although quite slowly for others.The essential result is that both monkeys were ultimately success-ful in performing double step sequences that required updating ofthe visual representations from one hemield to the other (Fig. 7E).Successful performance depended on experience with individualsequences. When we introduced new spatial arrangements of thetarget, we found that the across-hemield impairment could bere-instated. Given continued experience, we found that perfor-mance again improved (Berman, Heiser, Dunn, Saunders, & Colby,2007; Berman, Heiser, Saunders, & Colby, 2005). These behavioraldata tell us that the route for updating across hemields is not justcortex-to-cortex, via the forebrain commissures. The conclusion isthat subcortical pathways must contribute to the recoveredbehavior.

If subcortical pathways are important for the recovery of across-hemield performance, what is happening in cortical areas? Wewere particularly curious to know about the activity of LIP neuronsin the split-brain monkeys. Are LIP neurons still in the loop? Weasked whether LIP neurons are active when stimuli are remappedacross hemields. When we recorded in LIP, many neurons showedacross-hemield remapping. An example neuron is shown in Fig. 8.In the within-hemield condition of the single step task (panel A),this neuron exhibited a burst of activity that began even before theeye movement started. In the across-hemield condition, we mighthave expected that the neuron would fail to show any updatingactivity, but instead the neuron red briskly, with a burst of activ-ity after the eye movement began (panel B). This demonstrates thatLIP neurons are still in the loop; they remain part of the circuitryfor across-hemield updating in the absence of the forebrain com-missures. This example neuron illustrates two additional ndingsabout across-hemield remapping in the split-brain monkeys.First, across-hemield activity is diminished relative to within-hemield activity. Second, across-hemield activity begins laterthan within-hemield activity; for this cell, presaccadic activitywas observed only for the within-hemield condition. These twodifferences were signicant in the population of LIP neurons fromthe split-brain animals. By contrast, no such differences werefound in the intact animal (Heiser et al., 2005). These observationsindicate that across-hemield remapping, while present in thesplit-brain animal, is less robust in the absence of direct interhemi-spheric links.

Our behavioral and physiological ndings lead to two majorconclusions. First, the forebrain commissures are important for ra-pid, robust updating of visual representations across hemields.Second, these commissures are not strictly necessary. Alternatepathways can come online. There must be some neural reorganiza-tion, and our behavioral data suggest that this reorganization isexperience-dependent: recovery does not generalize to any newdouble step sequence but seems to emerge as the monkey hasexperience with specic sequences. This aspect of our ndingsraises the possibility that, following the loss of the forebrain com-missure pathway, the circuit for updating may need to recalibratethe relationships among neurons that represent visual locationsbefore and after saccades. The conclusion is that the system forspatial updating is redundant and plastic, much like the systemfor saccade generation. Direct cortical links are the usual pathwayfor remapping of memory traces but not the only pathway.

4.6. Multiple pathways for the transfer of corollary discharge signals

arch 49 (2009) 12331248The initial impairment for across-hemield conditions suggeststhat the forebrain commissures indeed serve as the primaryroute for communicating visual signals between the cortical

ReseSingleStep

Within Across

em71a.2

Stimulus control

A

C

E

B

D

F

R. Berman, C. Colby / Visionhemispheres at the time of an eye movement. Are these same com-missures also the primary route for transmitting information aboutthe impending eye movement, in order to initiate visuospatialupdating? In another set of experiments, we investigated the roleof cortico-cortical links in the transfer of corollary discharge sig-nals that initiate spatial updating (Colby et al., 1996). We testedthe performance of the split-brain monkeys on two new types ofdouble step sequences. These sequences differed only in whetherthe corollary discharge signal stayed within one hemisphere orhad to travel across hemispheres. Our expectation was that thesplit-brain monkeys would be impaired selectively on the se-quences that required an interhemispheric transmission of the cor-ollary discharge signal. Contrary to our expectation, we found thatthe split-brain monkeys could readily perform both types of doublestep sequences, regardless of whether the corollary dischargeinformation traveled within or across hemispheres. The immediacyof successful performance indicates that corollary discharge signalsare typically relayed to visual areas without reliance on the fore-brain commissures. This nding is consistent with recent physio-logical studies of the corollary discharge pathway from SC to FEF.

ulus has high contrast. Extrastriate and parietal areas should also

20 sp/s

Saccade control

G H

200 ms

Fig. 8. Existence of across-hemield remapping in the split-brain monkey. Activityof a single LIP neuron in the split-brain monkey exhibits remapping both within andacross hemields. Top panels (A and B) show the spatial conguration for the singlestep task, determined by the location of the neurons receptive eld (circle). In thewithin condition (A), a vertical saccade brings the receptive eld onto a previouslystimulated location in the same hemield. In the across condition (B), a horizontalsaccade brings the receptive eld onto the stimulated location, which was in the leftvisual eld at FP1 but in the right visual eld at FP2. The neuron remapped thememory trace in both the within-hemield (C) and across-hemield conditions (D)of the single step task. In the within-hemield condition, remapping begins evenbefore the beginning of the eye movement, with an initial burst of activity, followedby a brief inhibition after the saccade and a subsequent second burst. In the across-hemield activity, only the later burst of activity is present. The correspondingcontrol conditions show that activity was minimal when the stimulus appearedalone (E and F) and when the saccade was generated with no stimulus present (Gand H). Rasters and histograms are aligned on the beginning of the saccade. FromHeiser et al. (2005).become active because the sudden onset and high-frequency ick-er of the stimulus make it salient. Second, following the eye move-ment, the memory trace of the stimulus should activate visualareas ipsilateral to the stimulus (panel B, red trace). We use theterm remapped response to describe this ipsilateral activation in or-der to emphasize that it is not driven by direct visual stimulation.Third, this remapped response should be larger than the responseselicited by either an ipsilateral stimulus alone or an ipsiversive sac-In addition to predominant ipsilateral connections, e.g., from rightSC to right FEF (Sommer &Wurtz, 2002), there are also connectionsthat cross, e.g., from right SC to left FEF (Crapse & Sommer, 2006).Similarly, anatomical studies of the pathway from SC to LIP suggestthe presence of a crossed connection in addition to prominent ipsi-lateral connections (Clower et al., 2001). These data reinforce theidea that there are multiple pathways by which corollary dischargesignals can interact with sensory and attentional signals.

5. Remapping in humans

5.1. Remapping in human parietal cortex

Several lines of evidence suggest that humans and monkeys usethe same updating mechanisms for generating stable percepts.Behavioral studies have demonstrated that both species have sim-ilar abilities in eye movement tasks that require updating (Baizer &Bender, 1989; Dassonville, Schlag, & Schlag-Rey, 1992). As dis-cussed earlier, data from neuropsychological and inactivation stud-ies suggest that the parietal lobe is crucial for these abilities in bothspecies (Duhamel et al., 1992b; Heide et al., 1995; Li & Andersen,2001; Morris et al., 2007). These ndings imply that remappingalso occurs in the human parietal cortex. We hypothesized thatwe could visualize remapping in humans using functional mag-netic resonance imaging (fMRI) (Merriam, Genovese, & Colby,2003). In this section, we present functional imaging evidence thatthe human parietal cortex is involved in remapping.

In order to demonstrate remapping physiologically in humans,we designed an imaging experiment that followed the task param-eters of the monkey experiments as closely as possible. The updat-ing task used in the imaging experiment is shown in Fig. 9 (panelA). Each trial began when the subject looked at one of two xationpoints on the screen. On some trials, a stimulus appeared at thecenter of the screen, far from the location of gaze. The stimulusickered at high contrast in an otherwise dark visual environment,making it a highly salient stimulus. As in the single-unit experi-ments, the stimulus was totally irrelevant to performance of thetask. After 2 s of stimulus presentation, the stimulus disappearedand a tone cued the subject to make a saccade to the other xationpoint. There was no stimulus present at the time of the saccade soonly a memory trace of the stimulus could be remapped. The sec-ond xation point was positioned so that the eye movement to itbrought the stimulus location into the opposite visual eld (Fig.9A, second panel). In the example shown, gaze was initially direc-ted to the right xation point, and the stimulus appeared in the leftvisual eld. Concurrent with stimulus offset, a tone instructed thesubjects to move their eyes to the left xation point. As a result ofthis eye movement, the stimulus location was then in the right vi-sual eld.

The predictions from this experiment are straightforward. First,the stimulus should activate visually responsive cortical areas inthe contralateral hemisphere (panel B, blue trace). Low level visualareas, such as V1 and V2, should become active because the stim-

arch 49 (2009) 12331248 1243cade alone. Both ipsilateral stimuli and ipsiversive saccades mayinduce minimal activation; receptive elds in extrastriate and pari-etal cortices can be large and sometimes encroach a few degrees

Fig. 9. Remapping in human parietal cortex. Panels in the top left (A) show the sequence of events in the single step task adapted for imaging. At the beginning of the trial, thesubject xates a right crosshair while a stimulus appears in the left visual eld (LVF) and is on the screen for 2 s. We expected the stimulus to activate occipital and parietalcortex in the contralateral, right hemisphere (blue circle). The stimulus disappears and simultaneously a tone cues the subject to make a leftward saccade to the othercrosshair. This saccade brings the location of the now-extinguished stimulus (dotted circle) into the right visual eld. We expected that remapping of the stimulus tracewould cause activation to shift from the right to the left hemisphere (hatched red circle). The predicted time course (B) of activation for this condition. Shaded region indicatesthe time that the stimulus is on and the vertical line at 2 s indicates the time of the auditory cue. Activation in the right parietal cortex was expected to follow a standardhemodynamic time course (blue trace); this represents visually-driven contralateral activation. Activation in the left hemisphere, due to the remapped stimulus trace, wasexpected to occur with a similar time course but shifted by 2 s because the saccade cue occurred 2 s after stimulus appearance (red trace). We also expected the remappedresponse to be lower in amplitude than the visually-driven response. Activation in a single subject (C) for a stimulus that is remapped from the left to the right visual eld.Blue outline indicates the parietal region of interest. The stimulus elicits visually-driven activation in the contralateral (right) hemisphere occipital and parietal areas.Activation was also observed in the ipsilateral (left) parietal lobe, indicating that the visual representation was remapped in conjunction with the saccade. Time course ofactivation (D) evoked by the visual and remapped stimuli from parietal cortex in each hemisphere. Time courses are an average of 72 trials. The remapped response (red line)occurs later and is smaller than the visual response (blue line). BOLD-image raster plots of the responses from the same hemispheres for 72 successive trials, for the visual (E)and remapped (F) responses. On the y axis, each row represents a trial, and on the x axis, percent signal change is represented in pseudocolor plotted over time. Adapted fromMerriam et al. (2003). (For interpretation of the references to color in this gure legend, the reader is referred to the web version of this paper.)

1244 R. Berman, C. Colby / Vision Research 49 (2009) 12331248

that are currently of importance for the organism (Huang, Treis-

Reseinto the ipsilateral side of space (Barash, Bracewell, Fogassi, Gnadt,& Andersen, 1991b; BenHamed, Duhamel, Bremmer, & Graf, 2001).There may also be some activation associated with the saccade be-cause of retinal stimulation during the movement itself. Thesesources of activity should nonetheless be smaller than the re-mapped response. Fourth, remapped responses should have a char-acteristic shape and time course that distinguish them from visualresponses. Specically, remapped responses should occur later intime than visual responses, because the cue that triggers the eyemovement occurs after the stimulus has been on the screen fortwo seconds. Remapped responses should also be lower in ampli-tude than visual responses; at the single neuron level, responsesto memory traces are about half as large as responses to actualstimuli.

These four predictions were conrmed in our imaging data ofhuman parietal cortex (Merriam et al., 2003). Our main result isillustrated in Fig. 9. A left visual eld stimulus strongly activatedthe region of interest in right parietal cortex (outlined in blue inFig. 9C). This contralateral activation in the right hemisphere re-ects visual activity that is directly driven by the stimulus. We alsoobserved activation in the ipsilateral parietal lobe (Fig. 9C, lefthemisphere). We interpret this ipsilateral activation as a responseto the remapped trace of the stimulus. The time course of this ipsi-lateral activation indicates that it is indeed remapping (Fig. 9D, leftpanel). The remapped activation in left parietal cortex (red trace)occurs later than the visual activation in this hemisphere (bluetrace) and the amplitude is reduced, as predicted. We found a sim-ilar pattern of responses in right parietal cortex (Fig. 9D, right pa-nel). Control conditions demonstrated that activation observed inthe remapping task cannot be accounted for by responses to theipsilateral stimulus alone or responses to the ipsiversive saccade.These data show that remapped responses are detectable in humanparietal cortex.

5.2. Remapping in human extrastriate and striate cortex

Remapping in monkeys, as described above, is not limited toparietal cortex. Remapping has been observed in the frontal eyeeld (Umeno & Goldberg, 1997, 2001), the superior colliculus(Walker et al., 1994), and extrastriate visual cortex (Nakamura &Colby, 2002). Neurons in all these areas have spatially selective vi-sual and perisaccadic responses and are modulated by spatialattention (Bruce & Goldberg, 1985; Ignashchenkova, Dicke, Haar-meier, & Thier, 2004; Murthy, Ray, Shorter, Priddy, Schall &Thompson, 2007; Schafer & Moore, 2007; Thompson, Biscoe, &Sato, 2005; Wurtz & Goldberg, 1971); for reviews see (Krauzlis,2005; Schall, 2002; Treue, 2003). If remapping contributes to per-ceptual constancy, remapping should not be limited to brain re-gions with attentional and oculomotor functions. Rather, updatedspatial information should reach visual areas that are involved invisual perception. We found remapping in early visual areas ofmonkeys, and so we next asked whether the same is true in hu-mans (Merriam, Genovese, & Colby, 2007). Two lines of evidencesupported our hypothesis that remapping occurs in human earlyvisual cortex. First, psychophysical studies have demonstrated thatupdated visual signals are required to integrate information aboutstimulus features across saccades (Hayhoe, Lachter, & Feldman,1991; Melcher, 2005, 2007; Melcher & Morrone, 2003; Prime, Nie-meier, & Crawford, 2006). Second, several human fMRI studieshave demonstrated strong top-down effects throughout occipitalcortex. Multiple visual areas are activated in tasks that involve spa-tial attention (Brefczynski & DeYoe, 1999; Buracas & Boynton,2007; Gandhi, Heeger, & Boynton, 1999; Kastner et al., 1999;

R. Berman, C. Colby / VisionMcMains & Somers, 2004; McMains et al., 2007; Noesselt et al.,2002; Ress, Backus, & Heeger, 2000; Silver, Ress, & Heeger, 2007;Tootell et al., 1998; Yantis et al., 2002). Many of these areas are alsoman, & Pashler, 2007). These observations return us to the closelink between attention and updating. Psychophysical work on inte-gration of information across saccades indicates that rather littleinformation is maintained from one xation to the next (Hayhoeet al., 1991; Henderson & Hollingworth, 2003; Irwin, 1991; Irwin& Andrews, 1996; Irwin & Zelinsky, 2002; Lachter & Hayhoe,1995). Perceptual stability may result not from fusing completeimages acquired in separate glances but from integrating informa-tion about only a few selected objects or potential targets. Further-more, the spatial reference frame in which this limited informationis integrated may reect the demands of the specic task. Psycho-physical results indicate that eye-centered (updated retinotopic),object-centered, and other spatial reference frames are used tointegrate transsaccadic information (Karn, Moller, & Hayhoe,1997; Melcher, 2007; Melcher & Morrone, 2003; Pelz & Hayhoe,1995). They also suggest that target selection and transsaccadicintegration may take place at relatively early stages of the visualsystem (Hikosaka, Miyauchi, & Shimojo, 1993; Shimojo, Tanaka,Hikosaka, & Miyauchi, 1996).

We conclude that visual processing is dynamically modulated,even in early visual areas, by extraretinal signals such as attentionand corollary discharge information. These ndings emphasize theimportance of studying visual responsivity in the context of behav-modulated by oculomotor signals (DeSouza, Dukelow, & Vilis,2002; Sylvester, Haynes, & Rees, 2005; Sylvester & Rees, 2006; Val-lines & Greenlee, 2006). These fMRI studies indicate that visual cor-tex has access to the corollary discharge signals needed forremapping.

We conducted a new fMRI experiment to test whether re-mapped visual signals are present in human extrastriate visual cor-tex. In the parietal experiment, the imaged area was restricted toparietal cortex in order to maximize detectability of remapped re-sponses; the new experiment focused instead on extrastriate re-gions. As in the parietal experiment, the task was designed sothat activation related to the memory trace of the stimulus wouldbe remapped from one hemisphere to the other when the eyesmoved. We predicted that the hemisphere that was initially ipsilat-eral to the stimulus would become active around the time of theeye movement, reecting a remapped response. We found strongevidence for remapping in striate cortex and in each extrastriatevisual area examined (Merriam et al., 2007). Further, we found thatremapping was more robust in higher-order extrastriate areas (Fig.6B). This nding parallels our observations from macaque striateand extrastriate cortex (Fig. 6A). Our results indicate that remap-ping is present in visual areas that are directly involved in visualperception.

6. The neural basis of active vision

In the dorsal stream of macaque cortex, we found that there isrobust remapping in single neurons in areas V3A, V3, and V2(Nakamura & Colby, 2002). Likewise, in humans, remapping isnot limited to parietal cortex (Medendorp, Goltz, Vilis, & Crawford,2003; Merriam et al., 2003), but is also observed in early visual cor-tex (Merriam et al., 2007). These ndings are signicant becausethey tell us that extrastriate cortex is not simply engaged in pas-sive elaboration of retinal signals. Rather, an active process is guid-ing the acquisition and maintenance of stimulus representations inextrastriate cortex (Maunsell, 1995). The function of such activeprocesses in extrastriate cortex may be to narrow the task of stim-ulus representation to only those locations or stimulus features

arch 49 (2009) 12331248 1245ior. The visual response properties of neurons in extrastriate andparietal cortex have commonly been studied during xation or un-der anesthesia. Perception, however, normally takes place in the

Casagrande, V. A., Sary, G., Royal, D., & Ruiz, O. (2005). On the impact of attention

Reseand motor planning on the lateral geniculate nucleus. Progress in Brain Research,149, 1129.

Chafee, M. V., & Goldman-Rakic, P. S. (1998). Matching patterns of activity inprimate prefrontal area 8a and parietal area 7ip neurons during a spatialworking memory task. Journal of Neurophysiology, 79(6), 29192940.

Chafee, M. V., & Goldman-Rakic, P. S. (2000). Inactivation of parietal and prefrontalcortex reveals interdependence of neural activity during memory-guidedsaccades. Journal of Neurophysiology, 83(3), 15501566.

Clower, D. M., West, R. A., Lynch, J. C., & Strick, P. L. (2001). The inferior parietallobule is the target of output from the superior colliculus, hippocampus, andcerebellum. Journal of Neuroscience, 21(16), 62836291.

Colby, C. L. (1996). A neurophysiological distinction between attention andintention. In J. McClelland & T. Inui (Eds.), Attention and performance XVI:context of frequent eye movements. Understanding the dynamicnature of receptive elds around the time of saccades provides sig-nicant information about how we perceive the visual world in thenatural environment. Specically, predictive remapping allowsspatial processing to proceed in advance of a saccade, as if the sac-cade had already taken place. Furthermore, it permits the mainte-nance of spatially accurate representations across saccades. Thismechanism may be useful in constructing a stable percept of thevisual world.

Acknowledgments

This work was supported by National Institutes of HealthGrants EY-12032 and MH-45156, technical support was providedby core Grant EY-08908, and collection of MR images was sup-ported by P41RR-03631.

References

Armstrong, K. M., & Moore, T. (2007). Rapid enhancement of visual cortical responsediscriminability by microstimulation of the frontal eye eld. Proceedings of theNational Academy of Sciences of the United States of America, 104(22),94999504.

Baizer, J. S., & Bender, D. B. (1989). Comparison of saccadic eye movements inhumans and macaques to single-step and double-step target movements. VisionResearch, 29(4), 485495.

Balan, P. F., & Gottlieb, J. (2006). Integration of exogenous input into a dynamicsalience map revealed by perturbing attention. Journal of Neuroscience, 26(36),92399249.

Barash, S., Bracewell, R. M., Fogassi, L., Gnadt, J. W., & Andersen, R. A. (1991a).Saccade-related activity in the lateral intraparietal area I. Temporal properties;comparison with area 7a. Journal of Neurophysiology, 66(3), 10951108.

Barash, S., Bracewell, R. M., Fogassi, L., Gnadt, J. W., & Andersen, R. A. (1991b).Saccade-related activity in the lateral intraparietal area. II. Spatial properties.Journal of Neurophysiology, 66(3), 11091124.

BenHamed, S., Duhamel, J. R., Bremmer, F., & Graf, W. (2001). Representation of thevisual eld in the lateral intraparietal area of macaque monkeys: A quantitativereceptive eld analysis. Experimental Brain Research, 140(2), 127144.

Berman, R. A., Heiser, L. M., Dunn, C. A., Saunders, R. C., & Colby, C. L. (2007).Dynamic circuitry for updating spatial representations. III. From neurons tobehavior.. Journal of Neurophysiology, 98(1), 105121.

Berman, R. A., Heiser, L. M., Saunders, R. C., & Colby, C. L. (2005). Dynamic circuitryfor updating spatial representations. I. Behavioral evidence for interhemispherictransfer in the split-brain macaque.. Journal of Neurophysiology, 94(5),32283248.

Bisley, J. W., & Goldberg, M. E. (2003). Neuronal activity in the lateral intraparietalarea and spatial attention. Science, 299(5603), 8186.

Bisley, J. W., & Goldberg, M. E. (2006). Neural correlates of attention anddistractibility in the lateral intraparietal area. Journal of Neurophysiology,95(3), 16961717.

Brefczynski, J. A., & DeYoe, E. A. (1999). A physiological correlate of the spotlight ofvisual attention. Nature Neuroscience, 2(4), 370374.

Bruce, C. J., & Goldberg, M. E. (1985). Primate frontal eye elds I. Single neuronsdischarging before saccades. Journal of Neurophysiology, 53(3), 603635.

Bullier, J., Schall, J. D., & Morel, A. (1996). Functional streams in occipito-frontalconnections in the monkey. Behavioral Brain Research, 76(1-2), 8997.

Buracas, G. T., & Boynton, G. M. (2007). The effect of spatial attention on contrastresponse functions in human visual cortex. Journal of Neuroscience, 27(1),9397.

Bushnell, M. C., Goldberg, M. E., & Robinson, D. L. (1981). Behavioral enhancementof visual responses in monkey cerebral cortex. I. Modulation in posteriorparietal cortex related to selective visual attention. Journal of Neurophysiology,46(4), 755772.

1246 R. Berman, C. Colby / VisionInformation integration in perception and communication (pp. 157177).Cambridge: M.I.T. Press.Colby, C. L., Berman, R. A., Heiser, L. M., & Saunders, R. C. (2005). Corollary dischargeand spatial updating: When the brain is split, is space still unied? Progress inBrain Research, 149, 187205.

Colby, C. L., Duhamel, J. R., & Goldberg, M. E. (1996). Visual, presaccadic, andcognitive activation of single neurons in monkey lateral intraparietal area.Journal of Neurophysiology, 76(5), 28412852.

Colby, C. L., & Goldberg, M. E. (1999). Space and attention in parietal cortex. AnnualReview of Neuroscience, 22, 319349.

Connor, C. E., Preddie, D. C., Gallant, J. L., & Van Essen, D. C. (1997). Spatial attentioneffects in macaque area V4. Journal of Neuroscience, 17(9), 32013214.

Crapse, T. B., & Sommer, M. A. (2006). Crossed connections in the visuosaccadicsystem: Frontal eye eld neurons orthodromically activated from thecontralateral superior colliculus. Society of Neuroscience Abstracts, 413439.

Cui, H., & Andersen, R. A. (2007). Posterior parietal cortex encodes autonomouslyselected motor plans. Neuron, 56(3), 552559.

Dassonville, P., Schlag, J., & Schlag-Rey, M. (1992). Oculomotor localization relies ona damped representation of saccadic eye displacement in human andnonhuman primates. Visual Neuroscience, 9(3-4), 261269.

DeSouza, J. F., Dukelow, S. P., & Vilis, T. (2002). Eye position signals modulate earlydorsal and ventral visual areas. Cerebral Cortex, 12(9), 991997.

Dickinson, A. R., Calton, J. L., & Snyder, L. H. (2003). Nonspatial saccade-specicactivation in area LIP of monkey parietal cortex. Journal of Neurophysiology,90(4), 24602464.

Dorris, M. C., & Glimcher, P. W. (2004). Activity in posterior parietal cortex iscorrelated with the relative subjective desirability of action. Neuron, 44(2),365378.

Duhamel, J. R., Colby, C. L., & Goldberg, M. E. (1992a). The updating of therepresentation of visual space in parietal cortex by intended eye movements.Science, 255(5040), 9092.

Duhamel, J. R., Goldberg, M. E., Fitzgibbon, E. J., Sirigu, A., & Grafman, J. (1992b).Saccadic dysmetria in a patient with a right frontoparietal lesion. Theimportance of corollary discharge for accurate spatial behaviour. Brain, 115(Pt5), 13871402.

Fries, P., Reynolds, J. H., Rorie, A. E., & Desimone, R. (2001). Modulation of oscillatoryneuronal synchronization by selective visual attention. Science, 291(5508),15601563.

Gandhi, S. P., Heeger, D. J., & Boynton, G. M. (1999). Spatial attention affects brainactivity in human primary visual cortex. Proceedings of the National Academy ofSciences of the United States of America, 96(6), 33143319.

Gnadt, J. W., & Andersen, R. A. (1988). Memory related motor planning activity inposterior parietal cortex of macaque. Experimental Brain Research, 70(1),216220.

Goldberg, M. E., Bisley, J. W., Powell, K. D., & Gottlieb, J. (2006). Saccades, salienceand attention: The role of the lateral intraparietal area in visual behavior.Progress in Brain Research, 155, 157175.

Goldberg, M. E., & Bruce, C. J. (1990). Primate frontal eye elds. III. Maintenance of aspatially accurate saccade signal.. Journal of Neurophysiology, 64(2), 489508.

Gottlieb, J. (2007). From thought to action: The parietal cortex as a bridge betweenperception, action, and cognition. Neuron, 53(1), 916.

Gottlieb, J., Kusunoki, M., & Goldberg, M. E. (2005). Simultaneous representation ofsaccade targets and visual onsets in monkey lateral intraparietal area. CerebralCortex, 15(8), 11981206.

Gottlieb, J. P., Kusunoki, M., & Goldberg, M. E. (1998). The representation of visualsalience in monkey parietal cortex. Nature, 391(6666), 481484.

Guthrie, B. L., Porter, J. D., & Sparks, D. L. (1983). Corollary discharge providesaccurate eye position information to the oculomotor system. Science, 221,11931195.

Hallett, P. E., & Lightstone, A. D. (1976). Saccadic eye movements to ashed targets.Vision Research, 16(1), 107114.

Hayden, B. Y., & Gallant, J. L. (2005). Time course of attention reveals differentmechanisms for spatial and feature-based attention in area V4. Neuron, 47(5),637643.

Hayhoe, M., Lachter, J., & Feldman, J. (1991). Integration of form across saccadic eyemovements. Perception, 20(3), 393402.

Heide, W., Blankenburg, M., Zimmermann, E., & Kompf, D. (1995). Cortical control ofdouble-step saccades: Implications for spatial orientation. Annals of Neurology,38(5), 739748.

Heide, W., & Kompf, D. (1998). Combined decits of saccades and visuo-spatialorientation after cortical lesions. Experimental Brain Research, 123(1-2),164171.

Heiser, L. M., Berman, R. A., Saunders, R. C., & Colby, C. L. (2005). Dynamic circuitryfor updating spatial representations. II. Physiological evidence forinterhemispheric transfer in area LIP of the split-brain macaque.. Journal ofNeurophysiology, 94(5), 32493258.

Heiser, L. M., & Colby, C. L. (2006). Spatial updating in area LIP is independent ofsaccade direction. Journal of Neurophysiology, 95(5), 27512767.

Helmholtz, H. (1866). Treatise on physiological optics. New York: Dover.Henderson, J. M., & Hollingworth, A. (2003). Global transsaccadic change blindness

during scene perception. Psychological Science, 14(5), 493497.Hikosaka, O., Miyauchi, S., & Shimojo, S. (1993). Focal visual attention produces

illusory temporal order and motion sensation. Vision Research, 33(9),12191240.

Hikosaka, O., & Wurtz, R. H. (1983). Visual and oculomotor functions of monkey

arch 49 (2009) 12331248substantia nigra pars reticulata. III. Memory-contingent visual and saccaderesponses.. Journal of Neurophysiology, 49(5), 12681284.

ReseHuang, L., Treisman, A., & Pashler, H. (2007). Characterizing the limits of humanvisual awareness. Science, 317(5839), 823825.

Huk, A. C., & Shadlen, M. N. (2005). Neural activity in macaque parietal cortexreects temporal integration of visual motion signals during perceptualdecision making. Journal of Neuroscience, 25(45), 1042010436.

Ignashchenkova, A., Dicke, P. W., Haarmeier, T., & Thier, P. (2004). Neuron-speciccontribution of the superior colliculus to overt and covert shifts of attention.Nature Neuroscience, 7(1), 5664.

Irwin, D. E. (1991). Information integration across saccadic eye movements.Cognitgive Psychology, 23(3), 420456.

Irwin, D. E., & Andrews, R. V. (1996). Integration and accumulation of informationacross saccadic eye movements. In J. McClelland & T. Inui (Eds.), Attention andperformance XVI: Information integration in perception and communication(pp. 125156). Cambridge, MA: MIT Press.

Irwin, D. E., & Zelinsky, G. J. (2002). Eye movements and scene perception: Memoryfor things observed. Perception & Psychophysics, 64(6), 882895.

Jeffries, S. M., Kusunoki, M., Bisley, J. W., Cohen, I. S., & Goldberg, M. E. (2007).Rhesus monkeys mislocalize saccade targets ashed for 100 ms around the timeof a saccade. Vision Research, 47(14), 19241934.

Jonides, J., & Yantis, S. (1988). Uniqueness of abrupt visual onset in capturingattention. Perception & Psychophysics, 43(4), 346354.

Karn, K. S., Moller, P., & Hayhoe, M. M. (1997). Reference frames in saccadictargeting. Experimental Brain Research, 115(2), 267282.

Kastner, S., Pinsk, M. A., De Weerd, P., Desimone, R., & Ungerleider, L. G. (1999).Increased activity in human visual cortex during directed attention in theabsence of visual stimulation. Neuron, 22(4), 751761.

Konen, C. S., & Kastner, S. (2008). Two hierarchically organized neural systems forobject information in human visual cortex.Nature Neuroscience, 11(2), 224231.

Krauzli