[progress in brain research] plasticity in the adult brain: from genes to neurotherapy volume 138 ||...

M.A.Hofman,G.J.Boer,A.J.G.D.Holtmaat,E.J.W.VanSomeren,J.VerhaagenandD.F.SwaabCEds.) Pmgress in Brain Research, Vol. 138 0 2002 Elsevier Science B.V. All rights reserved

CHAPTER 10

Memory reactivation and consolidation during sleep: from cellular mechanisms to human performance

C.M.A. Pennartzl~*, H.B.M. Uylings ‘, C.A. Barnes 2 and B.L. McNaughton 2

t Graduate School Neurosciences Amsterdam, Netherlands Institute for Brain Research, Meibergdreef 33, 1105 AZ Amsterdam, The Netherlands

2 Arizona Research Laboratories, Department of Neural Systems, Memory and Aging, University of Arizona, Tucson, AZ 85724, USA

“When we have been exposed to an unusual stimulus for many minutes to hours, a nervous process is set up which results in the haunting of conscious- ness by the impression for a long time afterwards. The tactile and muscular feelings of a day of skat- ing or riding, afer long disuse of exercise, will come back to us all through the night. [. . . ] These revivals [. . . ] show that profound rearrangements and slow settlings into a new equilibrium are go- ing on in the neural substance, and they form the transition to that more peculiar and proper phenomenon of memory”.

William James (1890).

Memory consolidation: introductory remarks

William James’ vivid descriptions of postexperiential information processing were articulated at a time when modem insights were emerging into memory deficits resulting from head injury. These culminated in the formulation of the Law of Regression by Ribot (1882). This ‘law’ states that memories acquired long before the trauma are generally better

* Correspondence to: C.M.A. Pennartz, Graduate School Neurosciences Amsterdam, Netherlands Institute for Brain Research, Meibergdreef 33, 1105 AZ, Amsterdam, The Netherlands. Tel.: +31-20-566-5497; Fax: -t-31-20-696- 1006; Errnail: [email protected]

preserved than memories of recent experiences - a phenomenon known as temporally graded retrograde amnesia. The essentials of this phenomenon will only be briefly summarized here because they have been reviewed elsewhere (Squire and Zola- Morgan, 1991; Squire et al., 1993; McClelland et al., 1995). Experiments in which the interval between task learning (e.g. one-trial avoidance) and an amnestic treatment (e.g. electroconvulsive shock, protein synthesis inhibition) was systematically var- ied gave further credence to the ‘consolidation theory’ holding that memories become less modifiable and less vulnerable to disruption over time (e.g. McGaugh, 1966,200O; Squire and Alvarez, 1995).

A critical neuroanatomical substrate of memory was discovered when severe antero- and retrograde amnesia was manifested following bilateral lesions of the hippocampus and adjacent temporal lobe areas in patient HM (Scoville and Milner, 1957). The amnesia comprised memory for events that occurred weeks to months before the lesion, whereas events from a more distant past were remembered well. In some other, more recent cases, however, the amnestic gradient has been reported to comprise 15 years or more (Squire and Alvarez, 1995). When psycholo- gists began to dissect the generic term ‘memory’ into different classes (Cohen and Squire, 1980), lesions of the hippocampal system appeared to disrupt a specific class of memory, viz. declarative memory (i.e.,

144

memory of comprehensive associations between the elements making up events, of a nature that can be explicitly reported about by e.g. verbal description (Squire and Zola-Morgan, 1991)). In contrast, hippocampal lesions did not affect non-declarative or procedural memory. Procedural memory can guide behavior without depending on explicit recall of the memorized contents of the events resulting in this behavior. Lesions of the dorsal striatum and con- nected basal ganglia structures have been shown to result in procedural memory deficits, while leaving declarative memory largely unaffected (Mishkin et al., 1984; Knowlton et al., 1996; White, 1997).

Complementary work in experimental animals has produced results generally consistent with clinical- neuropsychological studies. Studies in primates revealed object recognition deficits for periods up to several weeks before surgery (Zola-Morgan and Squire, 1990). Because explicit reporting is difficult to investigate in rats, researchers have generally focussed in this species on the role the hippocampus plays in contextual/spatial aspects of episodic memory, and indeed these aspects of learning tasks appear to be heavily impaired by hippocampal lesions, inactivations or transmitter receptor blockade (Morris et al., 1986; Selden et al., 1991; Phillips and LeDoux, 1992; Riedel et al., 1999). As compared to primates and humans, hippocampal lesions in rats produce shorter lasting retrograde amnestic gradients for socially transmitted food preference (Winocur, 1990) or contextual fear conditioning, lasting up till several weeks, with a maximum of about 28 days (Kim and Fanselow, 1992). Altogether, these results suggest important inter-species differences in the steepness of the gradient. In line with the lesion effects on spatial memory, O’Keefe and Dostrovsky (1971) discovered that hippocampal pyramidal cells discharged when the rat is at a specific location of its environment (the ‘place field’) and not at others, and initiated an extended search for the mechanisms underlying ‘place cell’ firing and its precise behavioral-cognitive correlates. While there is no common agreement on this latter point, increasing support has been gathered for the notion that area CA1 of rat hippocampus does not literally encode a spatial map of the environment, in the sense that precise geometric relationships between environmental objects are laid down in a neural representation, ac-

cording to a definable, mathematically explicit met- ric. Ensemble firing patterns in area CA1 do seem to be governed by spatiotemporal configurations of cues in the environment, however without an explicit possibility to capture this ‘code’ in a computable allocentric map that the rat would use for spatial navigation (Eichenbaum et al., 1999; McNaughton et al., 2002). Regardless of the precise nature of the hippocampal ‘code’ and its use in hippocampal target areas, it can be safely stated that the hippocampus plays a causally important, but time-restricted role in the formation of long-term memories depending on space and other aspects of context.

Two more lines of research should be mentioned to complete this introduction. First, consolidation theory has been challenged by the notion that memory traces can be ‘reactivated’ and return to a modifiable state after the animal has been exposed to the context or cue that was used for its original training. We will refer to this phenomenon as ‘cue-triggered reactivation’. When animals were briefly exposed to such ‘reminders’ before they enter a task they had been trained on previously, they made fewer errors than control animals placed in a neutral environment before the retention test (Deweer and Sara, 1984; Sara, 2000). More dramatic with respect to the presumed stability of consolidated memories were a se- ries of early papers showing that amnestic treatments can disrupt performance in a retention test when they were applied to well-trained animals that had been exposed to a cue or task-reminder just before treatment. In contrast, classic consolidation theory presumes that such amnestic treatments should be largely ineffective well after an initial consolidation period. For instance, animals that had been fear conditioned to a white-noise stimulus were re-exposed 24 h later to the same conditioned stimulus (CS), given an electroconvulsive shock (ECS) treatment briefly thereafter, and showed a deficit in the subsequent retention test (Misanin et al., 1968). Other treatments disrupting performance in retention tests after cue-triggered reactivation include hypothermia (Mactutus et al., 1979) and protein synthesis inhibition (Judge and Quartet-main, 1982). Task reminders that are effective in reactivating brain states amenable to memory disruption include contextual cues and conditioned as well as unconditioned stimuli. Recently, local inhibition of protein synthesis in

145

the amygdala shortly after CS exposure has been shown to impair behavioral expression of previously acquired conditioned fear memory (Nader et al., 2000), a finding which has rendered the issue of cue- triggered reconsolidation more concrete by implying a specific neuroanatomical substrate in memory reactivation. Nonetheless, researchers in this field remain situated in the midst of a somewhat con- tentious debate about both conceptual and practical issues, rather than having an accomplished, full theory of both consolidation and reconsolidation. This debate addresses questions concerning the longevity of memory disruptions after reactivation and amnestic treatment and whether the relative persistence of the memory deficit depends on the type of learning task used. Furthermore, it has not always been clear whether performance deficits in the retention test are attributable purely to memory loss or to failure to retrieve memorized information and utilize it to or- ganize the behavioral response effectively (Cahill et al., 2001). Finally, it is conceivable that reminders not only recall original memory traces, but also initiate a process of newly organizing the memory trace which thus becomes an ‘update’ rather than a copy of the original version.

The second line of research has addressed the importance of sleep processes for memory consolidation. It is intuitively attractive to think that sleep, as an essentially ‘off-line’ period of the day-night cycle, may allow reprocessing of information that has been acquired during prior waking. Such reprocessing might not only involve consolidation of memorized experiences that had been acquired in rapid succession during previous waking, but also a selection of information especially relevant to the organism’s survival and reproduction, filtering out irrelevant information that need not be stored in an endurable form. First we recall the critical distinction between rapid eye movement (REM) sleep and slow- wave sleep (SWS). REM sleep, also called paradoxical sleep, is associated with dreaming and is characterized by rapid eye movement, atonia of the postural muscles and rapid desynchronization in cortical electroencephalographic (EEG) traces, with dominant activity in the theta frequency band (6-12 Hz in rats). In contrast, SWS, being part of the broader category of non-REM sleep, is recognized by slow waves in the cortical EEG (delta type; 0.5-3 Hz), alternating

with neocortical spindles (lo-15 Hz EEG oscillations lasting 1-2 s). During SWS, the hippocampal EEG is marked by the occurrence of ripple/sharp waves complexes, which are high-frequency (-200 Hz) bursts of pyramidal cell activity with concomitant dendritic depolarization (Buzs&i, 1986; Buzs&i et al., 1992; Ylinen et al., 1995).

REM sleep deprivation has been shown to lead to performance deficits on subsequent retention tests in various tasks both in animals and humans (Fish- bein, 197 1; Pearlman and Greenberg, 1973; Fishbein and Gutwein, 1977; Smith, 1985; Smith and Rose, 1996). While suggestive of a memory deficit, cau- tion has to be applied in interpreting these results in view of possible disruptions of arousal, attention, working memory, planning and other executive deficits, and it has proven difficult to disentangle effects on memory formation from such confounding factors (Siegel, 2001). In a correlative approach, it was shown that periods of intensive learning are followed by an increase in REM sleep (Hennevin and Ham, 1985; Smith and Rose, 1996, 1997; Stick- gold, 1998). Analogous to cue-triggered reactivation tests in the awake state, Ham et al. (1985) presented a CS having an acquired association with a footshock as a reminder during REM sleep and found improved memory performance on a subsequent retention test during wakefulness. The importance of SWS for memory consolidation has been examined less extensively, but recently affirmative evidence was found in a visual discrimination task, both in a correlative (Stickgold et al., 2000a) and an interventional approach (Gais et al., 2000). The causal relevance of SWS and REM sleep for different types of learning will be evaluated in further detail below, but in the next section we will first pay attention to theoretical and electrophysiological studies on ‘replay’ of memorized information during sleep.

Consolidation during sleep: a conceptual framework

The hippocampal formation plays an essential, time- restricted role in declarative/episodic memory. Con- sidering the vast expanse of plastic neural processing elements - be it synapses or other modifiable elements of somatodendritic architectures - found in specialized, distributed neocortical brain areas with

146

functions in sensory, motor and higher-order cognitive processing, it is logical to propose a main role for the neocortex in more final, long-term storage of memories. Lesion studies on the basal ganglia and cerebellum impair procedural but not declarative forms of learning and have thus pointed attention towards the neocortex as a focus of investigation. This tenet of neocortical dominance is widely adhered to and not contradicted by experimental findings, but it should not be forgotten that it is still an assumption at this point, and that an involvement of some subcortical structures such as the thalamus, mammillary bodies and basal forebrain should not be excluded as yet. The hypothesized course of events leading to long-term declarative memory formation would be as follows. In the awake state, a feedforward surge of polysensory activation sequences sweeps across the neocortex during the first stages of a behavioral experience. When this sweep arrives at higher processing stages such as the temporal lobe, parietal and prefrontal cortex, networks for sensorimotor processing, planning and decision-making are activated, and recurrent connections back to the lower-level stages initiate more refined, attentive processing. Meanwhile, the dynamic ensemble representations of experience in these neocortical areas would be propagated, at least in part, towards deep temporal lobe structures, including the entorhinal, perirhinal and parahippocampal cortices. Widespread and re- ciprocal connections between these regions and al- most the entire neocortical mantle have been found (Van Hoesen, 1982; Squire et al., 1989; Squire and Zola-Morgan, 1991; Witter et al., 2000). From these way stations, information flows converge further until arriving in the dentate gyrus of the hippocampus. Following processing throughout the trisynaptic hippocampal circuit - and other, nested circuits within the larger hippocampal formation (Witter et al., 1989) - an ensemble representation specific for the current configuration of objects and events in the organism’s environment would be formed, as discussed above. The hippocampus is presumed to store this representation, at least on a temporary basis, by way of distributed synaptic weight changes in its synaptic matrices (Hebb, 1949; Marr, 1971; McNaughton and Morris, 1987; Treves and Rolls, 1992; McClelland et al., 1995). At the next stage of consolidation, the hippocampus may then ‘re-

generate’ or reactivate this representation either in a stimulus-driven or spontaneous fashion. Next, the hippocampus may orchestrate the coordinated reactivation in its target areas which then leads to the selective potentiation or depression of co&co-cortical connections.

Marr (1971) was the first to propose a role for sleep in memory consolidation within a computationally explicit body of theory. He already recognized the importance of the transience of hippocampal memory and thus, by implication, the need to transfer the results of hippocampal information processing to other brain regions. Sleep, in his view, would be a suitable period for this transfer process, as this would avoid an interference between hippocampal output to the neocortex and the bom- bardment of inputs to neocortex instigated by novel sensory experiences. Whereas Marr hypothesized the hippocampus to provide ‘rehearsal trials’ to the neocortex to support strengthening of representations shared between hippocam$us and neocortex, most of the recent evidence is not consistent with the view that the hippocampus would replay copies, or even compressed versions, of neocortical information patterns. Instead, it is proposed that hippocampal representations, as far as they reach neocortical sites intactly, may provide a contextuospatial ‘stamp’ by which more focal, object-based neocortical representations may be bound together (cf. the ‘index theory’ of Teyler and DiScenna, 1986; McNaughton et al., 2002). The experiments reviewed below provide evidence that it is this type of representation that may be re-expressed or replayed during sleep. By linkage to a hippocampus-dependent spatiotemporal framework a neocortical representation may be integrated with other simultaneously active neocortical representations. This may explain the importance of the hippocampal memory system in episodic memory, which is by definition specific to the time and place of the events an individual experiences throughout life. Although it is evident that this hypothesis must be extensively tested by experiment, computational modeling studies support the possibility of this process (Treves and Rolls, 1994). There is still considerable debate about the precise contents of experiential ‘records’ produced by the hippocampus. Whereas some researchers favor contextuospatial coding, others emphasize their more global nature, incorporat-

147

ing distinct sensory object features and features of the behavioral task next to spatial aspects (Wood et al., 2000).

The seminal works of Marr and Hebb have in- spired a whole generation of theoretical neuroscien- tists to elaborate this framework, and address further, deeper problems concerning consolidation. Two fun- damental questions were addressed by McClelland et al. (1995). First, if the brain already contains a giant information storage device such as the neocortex, why would it need another, much smaller temporary repository such as the hippocampus? Second, why would it take the neocortex such a long time to incorporate new knowledge and become independent from the hippocampus in long-term storage and retrieval of information? Briefly, McClelland et al. developed a hippocampal-neocortical model based on the assumption that initial exposure to an event leads to memory trace formation in both hippocampus and neocortex, but with different strengths: the strength of initial trace storage would be larger for the hippocampus than neocortex. Memory storage in both structures is also subject to decay, with the decay rate being larger for hippocampus than neocortex. Furthermore, hippocampal information is transferred to the neocortex at a specified, constant rate, also regulating the pace at which hippocampal output is incorporated into the overall body of neocortical representations.

Simulations with this model have provided the following answers to the two aforementioned questions. First, despite its vast storage capacities, the neocortex would depend on the hippocampus during memory formation because the hippocampus, as a device for initial memory storage, allows the entire system to avoid interference between storage of new experiences and storage of previously acquired knowledge. Without an intact hippocampus the memorization of novel events would disrupt older representations, or the stability of older representations would prevent proper incorporation of novel event representations. Second, incorporation of novel information may take such a long time because simulations suggest a catastrophic breakdown of the existing knowledge structure if the continuous flux of experiences is crammed into the neocortical system at the high pace characteristic of daily life. A more gradual process of knowledge incor-

poration is needed, which McClelland et al. have labeled ‘interleaving’ of novel representations with the previously acquired knowledge structure. The hippocampus would store essential episodic aspects of novel experiences and feed these pieces of information repeatedly and gradually to the neocortical system, which responds by slowly adjusting its existing synaptic matrices and thus preserving older representations next to new ones. Computationally, sleep would be a period most suitable for ‘replay’ of hippocampal representations and their insertion into neocortical storage processes, as interference with novel sensory inputs can be avoided.

Needless to say, these model-based predictions must be subjected to experimental scrutiny and are uncertain in several respects. For example, the sever- ity of interference effects and the complementary need for gradual interleaving of information regen- erated from an initial storage medium will strongly depend on the sparsity of neocortical representations and the exact learning and ‘forgetting’ rates in the neocortical system. Simulation studies suggest that cortical synaptic matrices with sparsely coded cortical representations can accommodate large amounts of new representations before interference occurs (Nadal et al., 1986; Rolls, 1989). Furthermore, it is not clear whether the model, under a variety of test conditions, may still be hampered by the problem of catastrophic interference, now having been displaced from the neocortex to the hippocampus.

The ever-looming question concerning the need for a hippocampus in the presence of a much larger neocortex can also be approached from a novel, evolutionary-ecological angle. Instead of pointing to a possible deficit of the neocortex in mediating long- term trace formation, one might posit the question which representations are worthy of being stored in the neocortex, based on their value for predicting consequences important for survival and reproduction of the organism. We may hypothesize that the hippocampus forms temporary contextuospatial representations which can only coordinate and reinforce long-term storage in the neocortex if they have been validated as being functionally important to the animal. Thus, processing of hippocampal output by its target areas would be accompanied by a filtering process by which irrelevant information is either dis- carded or stored at a lower strength. In this view,

148

the time limitation of hippocampal storage will not be defined by a basically malfunctioning neocortex slowly absorbing new information, but by the temporal span across which animals can form and hold representations of temporal contingencies between neutral and reinforcing events that may be spaced far apart in time. The gradually evolving independence of the hippocampus in memory formation may thus arise when the time span has elapsed across which the brain was trucking possible consequences of the original novel experience. Cues and situations that appeared to be predictive of reinforcing events must be consolidated by that time, and those that were not can be either forgotten or stored with lower access priority.

Trace conditioning is a form of classical conditioning in which a considerable time interval (usually in the order of 30 s to 1 n-tin) elapses between the CS and the unconditioned stimulus (US). Although hippocampal lesions impair trace conditioning (Meek et al., 1984; Huerta et al., 2000), little attention has been paid to contingencies spanning intervals of several days or longer. Our hypothesis would predict that the hippocampus would be important for forming context-related contingency representations for intervals lasting until 28 days in rats (Rim and Fanselow, 1992; see above), with a larger ‘weight’ attached to more recent experiences. According to this idea, the hippocampus still encodes contextuospatial frameworks within which discrete events are situated, not having a primary role in the assess- ment of reinforcers or their prediction by discrete stimuli. The evaluation of stimuli and actions calls for involvement of reward-encoding and reward- predicting structures in the overall processes mediating long-term memory, and these are likely to include the amygdaloid complex, orbitofrontal and medial prefrontal cortex, ventral tegmental area, ventral striatum and/or anterior cingulate cortex (Nishijo et al., 1988; Pennartz et al., 1994, 2000; Pennartz, 1996,1997; Robbins and Ever&t, 1996; Schoenbaum et al., 1998; Baxter et al., 2000; Gehring and Knight, 2000; McGaugh, 2000; Schultz, 2000). Testing the hypothesis will require elaborate efforts to under- stand how hippocampal and amygdaloid/prefrontal outputs are integrated in order to ‘weigh’ the behavioral relevance of situations for future survival and thus to direct long-term memory storage.

We will next review experimental data arguing in favor of one crucial element of the aforementioned hypotheses, viz. the ‘replay’ of hippocampal records of behavioral experience during off-line periods such as sleep.

Spontaneous reactivation during sleep: unit recordings in hippocampus

Pavlides and Winson (1989) first showed that hippocampal place cells exhibit a retention effect when they are first activated by placement of the animal in the appropriate place field of the environment and then go to sleep. They made a counterbalanced comparison between cells that were activated by forced exposure of the rat to their place field and a control group of ‘non-exposed’ cells (i.e. cells not activated by exposure to their place field). They found that the mean firing rates of the ‘exposed’ cells during postbehavioral periods of slow-wave sleep (SWS) and REM or pre-REM sleep were higher than in the ‘non-exposed’ cells, whereas there was no significant difference during several awake states (i.e., exploration, ‘still alert’ and quiet, awake states). In addition to the mean firing rate, the burst occurrence rate was also enhanced in the exposed cells during sleep. Thus, the level of hippocampal firing in the awake stage significantly affects their activation status during subsequent sleep.

However, this first indication for information reprocessing during sleep remained susceptible to the criticism that the activated place cells may partially retain their firing rates due to an interaction between place field exposure and long-lasting effects of arousal or stress, caused by forcing the rat into a sub- space of the arena. Replay of memories during sleep should be characterized not just by re-expression of enhanced firing rates of individual neurons, but also by reactivation of temporally precise relationships between the firing patterns of many neurons, reflecting their synchronous or sequential activation during preceding behavior. If such temporally specific replay occurs it is unlikely to originate from a persistent arousal effect because this will be expressed as a global, non-specific change in neuronal state, for instance as a gain modulation of the overall firing rates of the neurons. Using the tetrode-array recording technique (Wilson and McNaughton, 1993), Wilson

149

and McNaughton (1994) were able to simultaneously follow ensembles of 49-74 CA1 neurons during a behavioral episode of spatial exploration and a prebehavioral and postbehavioral sleep phase. When firing activity of pairs of neurons within these ensembles was highly correlated in the behavioral state (mostly because their place fields would overlap), this correlation was also enhanced during subsequent sleep, but low during preceding sleep. In contrast, firing patterns that were uncorrelated during behavior remained uncorrelated during subsequent sleep. This temporally specific sleep reactivation across ensembles was spontaneous, in contrast to cue-triggered reactivation, and particularly prominent during SWS, when CA1 cells discharge in ripples/sharp-wave complexes. Indeed, reactivation in the postbehavioral sleep phase was significantly larger during ripples as compared to non-ripple intervals. The latter finding also argues in favor of the possibility that ‘reactivation’ reflects the re-emergence of a representation, rather than reflecting a fragment of a persistent neural representation that keeps on lingering within the hippocampus after the behavioral experience.

Using partial correlation methods, Kudrimoti et al. (1999) showed that a significant fraction of the variance in pairwise firing correlations during postexperiential sleep was accounted for by preceding awake-phase correlations. This statistical approach evaluates reactivation on the basis of all firing correlations within the recorded ensemble, not just the most highly correlated cells. Taking the explained variance as a measure, the reactivation was observed to decay within about 30 min during postexperiential sleep, although incidental pattern recurrence during the next 24 h seems likely. No evidence for REM sleep reactivation was found in this study. Importantly, the presence of sleep was not mandatory to observe reactivation. Instead, the presence of ripple/sharp wave complexes turned out to be an essential condition, and these can also occur during awake immobility and consummatory behavior (Buzsiiki, 1986). Thus, hippocampal ripples present a temporal window of strong reactivation, whereas non-ripple epochs exhibit the phenomenon to a much lesser extent. When rats were subsequently exposed to two different behavioral conditions requiring spatial exploration (a familiar and an unfamiliar configuration for maze running), both conditions were

found to make independent contributions to reactivation during the following SWS. This finding suggests an interleaved replay of multiple memory traces and shows that both novel and familiar situations give rise to reactivation.

Two important elements on hippocampal reactivation have been added to this pioneering work in subsequent studies. First, if the hippocampus stores memory traces that are more than momen- tary ‘snapshots’ of experiences and also incorporate their evolution in time, reactivation processes should be expected to contain information about the temporal order of firing of previously activated cells. This prediction was confirmed first by Skaggs and McNaughton (1996) and later by Nadasdy et al. (1999). When rats were instructed to run along a triangular track, a subset of hippocampal CA1 neurons exhibit partially overlapping place fields. Because of the fixed running direction, many of these place cell pairs were activated in a distinct temporal order, reflected in the temporal asymmetry of their cross-correlation (Fig. 1; Skaggs and McNaughton, 1996). This asymmetry, or temporal bias, was significantly preserved during subsequent sleep, whereas it was absent during the pre-experiential sleep phase. Fragments that may be replayed in this way are of a rather short duration (So-100 ms), and thus ripple-associated reactivation may not be a suitable mechanism to encode long-range temporal relationships. Nadasdy et al. (1999) sought for recurrence of spike sequences during sleep after a wheel running experience with the use of template-matching methods and joint probability maps, controlling for random recurrences by Monte Carlo shuffling of spike trains. They confirmed the replay of spike sequences during postexperiential sleep and extended the previous findings by demonstrating re-expression of temporal structure in more than two cells. Possi- bly, replay is subject to a time-compression (Skaggs and McNaughton, 1996; Nadasdy et al., 1999).

The second major novel element concerns neurophysiological studies investigating the possibility of REM sleep-associated reactivation. Louie and Wil- son (2001) trained rats on a four-trial sequence in a circular track task, resulting in a rather stereotyped running behavior intermitted by reward consumption phases. Subsets of task-activated hippocampal neurons showed a significant resemblance (correla-

150

tion) between the familiar ‘run’ phase of the task as compared to preceding REM sleep in 19 out of 38 episodes, whereas correlations were not significant for the remaining 19 cases. The measure of resemblance was based on computing a template correlation coefficient between two multi-neuron spiking patterns and emphasized pattern similarity across behavioral timescales (seconds to minutes), while ignoring temporal structure in the millisecond range because of an additional Gaussian smoothing oper- ation imposed on individual spike trains. A scaling factor was employed to accommodate the possibility that the speed of REM replay may be different from spike patterns during awake behavior. The statistical

B

Position on track

-400 -200 0 200 400

The (ms)

significance of the template correlation coefficient was assessed by comparison to spike train data that were shuffled in various ways, and the smallest difference between values computed for real and shuffled data was used for further evaluation. No- tably, the rats spent much more time in REM before the behavioral episode than afterwards and it proved difficult to examine the resemblance between spike patterns obtained from behavior versus postexperiential REM sleep. At this point, it is thus difficult to decide whether the significant correlations between prebehavioral REM and behavioral patterns reflect a replay of rather stereotyped experiences from the preceding test day, or a preplay of specific REM spike patterns that persist in the awake state, per- haps as a result of the intrinsic attractor dynamics of the hippocampus, unrelated to mnemonic operations. To control for this possibility, Louie and Wilson examined correlations between prebehavioral REM episodes and a novel track-running behavior instead of the familiar task, and failed to find pattern correlations between these two. Again, however, reactivation during postbehavioral REM could not be demonstrated, which complicates the evaluation of these results.

Poe et al., 2000 (Fig. 2) focussed on the phase specificity of firing patterns relative to theta oscilla-

Fig. 1. Preservation of temporal order in hippocampal spike trains replayed during sleep. Multiple neurons were simultaneously recorded from area CA1 of the rat hippocampus. Rats went through a sequence of task stages consisting of a prebehavioral sleep phase (‘Sleep before’), a behavioral phase (repeated, counterclockwise running along an elevated triangular track with intermittent consumption of food reward ‘On track’) and a postbehavioral sleep phase (‘Sleep after’). Cells 1 and 2 had partially overlapping place fields on the triangular track as shown in the upper graph. The mean firing rate of the two cells is plotted as a function of the rat’s position on the track. The direction of motion is from left to right. In the three bottom graphs, cross-correlation plots (bin size: 10 ms) are shown for cell 1 and 2 during the three task stages. The duration of the sleep periods taken into account was 15 min for each (before and after track running). The temporal bias in the firing relationship of the two cells was quantified as the difference in the light- and dark-shaded regions. Note the absence of clear cross-correlated firing during prebehavioral sleep, contrasting to the strong peak, just left to time lag zero, during postbehavioral sleep. During track-running, cross-correlated firing was subject to a clear theta- modulation (-8 Hz). (From Skaggs and McNaughton, 1996. Reproduced with permission.)

151

tions in hippocampal EEG activity during behavior and REM sleep. They compared place-associated firing of hippocampal neurons during familiar and novel experiences. Surprisingly, cells having a ‘place field’ only on the familiar track showed a phase reversal (180” shift) during subsequent REM sleep, whereas cells with fields on the novel track showed only small phase differences between running and REM episodes. The phase reversal seen in the ‘farnil- iar field’ cells displaced spike bursts from the theta peak phase (during behavior) to the theta trough phase (during REM sleep) or vice versa. It has been shown that firing activity during the peak phase of the theta oscillation is associated with long-term potentiation (LTP) and firing during the trough phase with depotentiation or long-term depression (LTD; Pavlides et al., 1988; Huerta and Lisman, 1995; Holscher et al., 1997). These combined results lead to the hypothesis that synapses involved in representing older, often-repeated experiences may be weak- ened at the expense of reinforcement of synapses engaged by new memories.

Finally, complementary studies on experience- dependent expression of immediate early genes during sleep have been conducted. Ribeiro et al. (1999) found enhanced $268 expression during REM sleep in the neocortex and hippocampus of animals exposed to an enriched environment, whereas non- exposed control animals showed a general downreg- ulation. These results are in line with earlier results on the effects of enriched environment on both sleep and brain development. Working with juvenile rats, Mirmiran et al. (1982) reported an increase of both slow-wave and REM sleep by environmental emich- ment, as well as an increase in the weight of their cerebral cortex. Although this type of study does not offer the means to reveal temporally specific pattern replay of neural activity, these findings are consistent with sleep representing a temporal window of enhanced neuronal plasticity (Jones et al., 2001).

Memory reactivation in non-hippocampal structures

Neocortex

If the hippocampus is capable of replaying recently acquired information, one must ask how this replay affects ‘off-line’ information processing in other

parts of the brain. If the hippocampus engages in a carefully orchestrated dialogue with the neocortex during memory consolidation, one may expect to see a replay also in this latter brain structure. In ensemble recordings with tetrodes placed in hippocampal area CA1 and posterior parietal cortex (PPC, a structure processing visuospatial information), Qin et al. (1997) indeed found evidence for sleep reactivation with preservation of temporal order in both areas, taken individually. Moreover, the correlation patterns between hippocampus and PPC also partially re-emerged during postbehavioral sleep, although temporal asymmetries between cells were not re-expressed. While these results partially support the idea of a hippocampal-neocortical dialogue during off-line processing, the question remains why the temporal order of firing as observed in the hippocampus is not apparent in PPC. One possibility would be that neocortical ensembles ‘drive’ the hippocampus during behavior, while the reverse occurs during sleep (Qin et al., 1997). Strictly speaking, this would predict a reversal in the temporal asymmetry in the cross-correlogram of hippocampal-PPC cell pairs from waking to sleep, which was not observed. This possibility should not be rejected yet, however, because the order in which sequentially acquired memories are replayed is unknown, as are the ways in which temporal records of experience are seg- mented and phased with respect to one another. A second explanation for the lack of observed temporal coordination between hippocampus and PPC is the presence of at least one way station placed in between these structures (Squire and Zola-Morgan, 1991; Eichenbaum, 2000; Witter et al., 2000). In- deed, one potential caveat in all the different versions of hippocampal-neocortical consolidation processes is the anatomical fact that the intermediate stations, such as the subiculum, perirhinal, postrhinal and entorhinal cortex, receive extensive inputs from many other brain structures, including the medial prefrontal, piriform and cingulate cortices. The perirhinal and entorhinal cortices should by no means be regarded as way stations simply mediating ‘through- put’, but as highly integrative centers of multimodal and behaviorally relevant information (Meunier et al., 1993; Suzuki et al., 1993; Liu et al., 2000; Wit- ter et al., 2000). By consequence, the necessity for relatively undisturbed hippocampal-neocortical in-

152

formation transfer through these intermediate stages findings. Amzica et al. (1997) conditioned cats to may require some unknown mechanism regulating generate 20-50 Hz oscillations in the EEG of mo- and segregating spike trafficking. tor or visual cortex (area 4 or 17). They found that

Nonetheless, the results of Qin et al. (1997) sug- conditioning induced an increase in grouped fast os- gested for the first time that experience-dependent cillations that were spatially selective for these areas, reactivation does occur in the neocortex, and ad- and that this local increase was associated with a ditional evidence has been raised to support these much more widespread increase in the synchrony

A Familiar Field Cell B Novel Field Cell

180°

Track

90

“f (Track - REM) 180 0 :

Familiar Field Cells Novel Field CeUs

of these fast oscillations across thalamocortical networks. This enhanced cross-correlation amongst areas was also found during post-training periods of quiet waking and REM as well as non-REM sleep, but not during control or extinction sessions. It remains to be investigated whether the enhanced recurrence of cross-regional oscillation synchrony represents replay of specific memory contents or serves a more global purpose such as implement- ing a brain state that might facilitate post-training processing in a general sense. In a recent study on hippocampal-neocortical communication, Siapas and Wilson (1998) studied the temporal relationship between hippocampal ripple activity and neocortical spindles during slow-wave sleep in rats. On average, hippocampal ripples and associated spiking activity tended to precede neocortical spindles and unit firing in prefrontal cortex. These findings provide initial, tentative support for the concept that hippocampal replay during ripples may bias or select which neocortical neuron assemblies are engaged during ensuing spindle episodes.

Recently, support for reactivation in the human cerebral cortex has come from a PET imaging study by Maquet et al. (2000). They chose a procedural learning paradigm, viz. a visuomotor serial reaction time task, which is unlikely to require an intact hippocampus to be accomplished. Taking into account the earlier results of Karni et al. (1994), implying REM sleep in procedural learning, they focussed their PET imaging strategy on REM sleep episodes.

153

Brain activation patterns were compared between awake and REM states, and REM-associated activation was compared between trained versus task-naive subjects. In a conjunction analysis the overlap was determined between brain regions activated by the task in the awake state and brain regions selectively activated during REM in trained subjects, excluding overlap with brain regions activated in task-naive subjects. Focal regions of reactivation were found predominantly in the neocortex, especially striate cortex (mostly cuneus) and premotor cortex. Re- markably, overlap between the awake training and ‘trained’ REM state was also revealed for the subcortical mesencephalon. As expected, no evidence for hippocampal reactivation was found in this task.

Ventral striatum

There are two strong reasons to investigate the possible occurrence of reactivation in subcortical structures that receive direct hippocampal inputs. First, a demonstration of reactivation in those structures would imply that the hippocampal-neocortical dialogue - when considered from the viewpoint of memory replay - should be extended to include hippocampal-subcortical dialogues. Second, because of the overwhelming interest in the role of the hippocampus in memory consolidation in general, contextuospatial forms of memory have been logically emphasized until now. If any subcortical structures showing reactivation process information

Fig. 2. Phase reversal of spikes with respect to theta rhythm during REM sleep. Rats were instructed to run for food reward on a paired rectangular track. This track was divided into one rectangle to which the rat was familiarized, whereas the other rectangle presented a novel part of the environment. Ensemble firing activity was recorded from area CA1 of the rat hippocampus during track running and subsequent sleep. Simultaneously, electroencephalographic (EEG) traces were recorded from the same area. Hippocampal theta rhythm, observed in the 5-10 Hz band, was prominent during both track running and REM sleep. This figure shows representative examples of two hippocampal cells, simultaneously recorded from one rat. (A-D) Polar plots illustrating the mean percentage of spikes occurring at each phase of the theta rhythm. The cell in A had a place field only on the familiar track whereas the cell in B had its field on the novel track. The shaded area represents the firing phase profile during track running and the area outlined by a thick black line represents tbe profile of the same cell during REM sleep. In C and D, the mean number of spikes was plotted versus theta phase during track running (C) and REM sleep (D). Tbe gray and black outlines denote the novel and familiar groups, respectively. The mean durations of behavior on the familiar (12 min) and novel tracks (14 min) were similar. In the subsequent sleep period, rats spent an average of 2 rnin in REM sleep. For familiar group cells, the firing phase distributions were significantly different for track running versus REM sleep (n = 13, P < 0.001, x2 test). In contrast, no difference was found for novel group cells (n = 19, P = 0.49). (E,F) These diagrams depict the change in peak firing phase from maze running to REM sleep for familiar group cells (E; change: 168 k 6”, mean f SEM; n = 19) and for novel group cells (F, change: 11 f 6”; n = 13), as observed on the day of novel track exposure. Each filled circle represents the difference in peak firing phase between track running and REM sleep for each cell. The length of the arrow represents the relative mean vector. (From Poe et al., 2000. Reproduced with permission.)

154

streams that are distinctly different from place-like information, it may become feasible to generalize conclusions across different types of information do- mains. Therefore we decided to examine whether the ventral striatum, which mainly consists of the nucleus accumbens (ACB), displays reactivation of multi-neuron activity patterns (Pennartz et al., 2001). The ventral striatum receives extensive projections from both the dorsal and ventral parts of the hippocampal formation (area CA1 and subiculum) as well as from the entorhinal cortex (Groenewegen et al., 1987; Pennartz et al., 1994). This structure is thought to integrate emotional, contextual and polysensory information and to direct motivated behavior (Mogenson et al., 1980; Pennartz et al., 1994; Robbins and Ever&, 1996). A role for the nucleus accumbens in spatial learning has been suggested by several studies. Recently, Setlow and McGaugh (1998) showed that intra-accumbens infusion of the D&ass antagonist sulpiride immediately following a water maze training session impaired spatial memory, raising considerable support for this contention.

In our ensemble recording study in ACB (Pen- nartz et al., 2001), rats were trained on a T-maze task involving probabilistic reward distributions at both ends of the T-maze arms and subsequently a 12- tetrode array was implanted and directed to the ACB, with EEG electrodes placed in both the hippocampus and neocortex. The task consisted of: (1) a choice phase, in which the rat started from the T-maze stem and had to choose between the left and right arm for locating the site with the highest reward probability (e.g. 75% at south end, versus 25% at north end); and (2) a shuttle phase, in which the rat had to run from the north to the south arm ends and vice versa for 20-40 times in order to retrieve food reward at these arm ends, with the same probabilities as applied to the choice phase. This behavioral phase was flanked by two sleep episodes, referred to as Sleep 1 (Sl, pre-experiential sleep) and Sleep 2 (S2, postexperiential sleep). In most sessions, ensembles of 15-25 cells could be recorded simultaneously and stably across the three task phases. Changes in firing rate correlated to: (1) encountering and con- suming reward; (2) approach to reward, coupled to the direct anticipation of reward; (3) specific motor behavior, such as turning on the T-maze, or locomo- tion across particular sectors of the maze; and (4)

specific phases of the task, such as intertrial intervals (‘waiting’ phase). Only rarely did we find evidence for strictly place-related activity. Unlike firing patterns in the hippocampus, firing of ACB units that occurred at specific maze locations generally correlated to specific behavioral events or salient stimuli at those locations, not to the mere location itself. In general, neural correlates of reward and approach to reward were predominant, in agreement with earlier studies (Schultz et al., 1992; Williams et al., 1993; Lavoie and Mizumori, 1994; Shibata et al., 2001). Thus, information processed by the ACB car- ries a strong motivational-emotional component and is generally different from place-related correlates as found in the hippocampus. The heterogeneity of information apparent in ACB firing patterns should be emphasized, spanning virtually all phases of an individual trial of the task.

During an initial screening for reactivation phenomena in ACB we considered whether firing rates could be useful measures, as shown by Pavlides and Winson (1989) for hippocampus. However, in agreement with.Callaway and Henriksen (1992) we found a strong modulation of ACB firing rate by sleep/wake state, and even by specific sleep stages such as REM and SWS, making it difficult to de- tect small added effects during sleep as induced by enhanced firing during previous behavior. Therefore we concentrated on the analysis of temporal relationships in firing during Sl, behavior on the T-maze and S2. First, spike trains from all simultaneously recorded ACB neurons were binned into T intervals to obtain, for each cell i, a sequence of spike counts per consecutive bin (fi). The T intervals were taken from one episode (i.e., S 1, behavior or S2) and we did not discriminate between different types of behavior displayed on the maze or between REM versus SWS. Furthermore, we used the last 10 min of S 1, immediately preceding the behavior for analysis, as well as the first 10 min of S2, immediately following the behavior. Thus, differences in pattern similarity between S l-behavior and S2-behavior cannot be ascribed to a mere time-dependent, non- specific drift of firing patterns. Usually a bin size of 50 ms was chosen, although 10, 20 and 100 ms bins gave similar results. Next, temporal correlations in firing patterns of ACB neurons were determined by computing a Pearson’s correlation coefficient (PCC)

155

using the following equation (cf. Kudrimoti et al., 1999):

n=l rij =

1 T 1 T (1)

where rij is the correlation coefficient between neurons i and j, /.~i the mean spike count for cell i and n an index for bin number. When using bin sizes of lo-50 ms, this measure of firing correlations is especially sensitive to near-synchronous firing, not to temporal relationships with fairly long delays or ad- vances which may be detected in cross-correlograms with broad time windows.

Thus, a PCC value was obtained for each cell pair and for each of the three sleep/wake phases of the experiment. All PCC values pertaining to a particular sleep/wake phase were assembled into a single matrix, and thus the similarity of these three (so-called R) matrices could be compared: Sl vs. Maze, Sl vs. S2 and Maze vs. S2 (McNaughton, 1998). The similarity between two matrices with correlation coefficients as entries was determined by computing a novel correlation coefficient, this time defined at the level of two matrices. Thus, the following similarity measures were obtained for the three consecutive episodes: rMaZe,sl, ~2,s~ and r&&2. Reactivation was assessed by testing whether the similarity between S2 and Maze was larger than for Sl and Maze. An important refinement in this approach, utilizing partial correlation methods (Kleinbaum et al., 1998), was to compute the SZMaze similarity while factoring out any preexisting influence from S 1. This was done by determining the explained variance, which signifies how much of the variance in the correlation patterns during S2 can be explained by the pattern of correlations during prior behavior, but correcting for any correlations already present during the pre-experiential sleep phase, S 1. This cor- rection is important because ensemble firing patterns in ACB are characterized by a fair amount of basal correlations that remain constant throughout the entire sleep/wake cycle, including Sl, which would tend to overshadow the recruitment of novel correlations during learning and their reactivation during

subsequent sleep. The explained variance (EV) was computed as follows:

EV = Gla%S2,SI 2 rMaze,S2 - rMaze,SlrS2,Sl

= (2) (1 -&,,s1)(1 -r&1) To be able to assess the statistical significance of

the EV, which always assumes a value between 0 and 1, it was mandatory to design a novel control measure that could be applied to the same animal and preferably to the same experimental session as was used for computing the EV. By definition, reactivation must follow the arrow of time, as pattern recurrence follows, but never precedes a novel behavioral experience. Our design of a proper control measure was based on the idea that the same R matrices and the same matrix correlation coefficients rMxlaze,S 1, rs2,sl and rM=,s2 can be used as for the EV, but in reverse temporal order. Thus, a ‘reverse explained variance’ (REV) was computed simply by switching around S 1 and S2 in Eq. 2:

REV = &ze,S 1 JS2 2

rMaze,Sl - rM’Maze,S2rS2,Sl = (3)

(1 - r&e,s2)(l - rs22.d

If EV and REV are not significantly different, one should accept the null hypothesis that any difference between S2-Maze and Sl-Maze pattern similarity should be attributed to chance. When these measures were applied to eight recording sessions in ACB, however, a significant difference was found (Fig. 3; EV = 9.2f3.3% versus REV = 1.7 f0.6%; P < 0.02 according to Wilcoxon’s matched-pairs signed rank test). Specific association of ACB reactivation with different stages of sleep awaits further investigation.

If the hypothesis is true that the hippocampus acts as an initiator of reactivation and the ACB acts as a ‘follower structure’, we predict that ripple-associated reactivation in the hippocampus precedes, on average, reactivation in ACB. Although this prediction has not yet been tested, the combination of ACB ensemble recordings with hippocampal EEG recordings allowed us to test whether the firing rates of ACB neurons are modulated by the occurrence of

156

15

E

P < 0.02 I I

Normal (forward) Reverse (backward)

(r2Maze,S21S1) (r2Maze,S1 152) Fig. 3. Reactivation of ensemble firing patterns in nucleus accumbens. The explained variance was computed as explained in the main text. Briefly, the normal (forward) explained variance is a measure of reactivation, because it quantifies the pattern similarity between the behavioral phase (‘Maze’) and subsequent sleep (‘S2’), correcting for any preexisting influences as visible during prebehavioral sleep (‘Sl’). With ‘pattern’ we denote the matrix of cross-correlation coefficients of cell pairs in the recorded population. In contrast, the reverse (backward) explained variance is a statistical control measure, which assesses the pattern similarity in the reverse temporal direction. It quantifies the pattern similarity between ‘Maze’ and Sl, correcting for any correlation patterns visible during S2. Both measures are based on the same three matrices of cross-correlation coefficients. If these two measures are not significantly different, then it remains possible that pattern similarity between Maze and S2 is due to chance occurrences. This null hypothesis can be rejected at P cc 0.02 (Wilcoxon’s matched-pairs signed rank test; n = 8 sessions).

hippocampal ripples. Indeed, we found strong excitatory, inhibitory or biphasic excitatory-inhibitory firing rate modulations in close temporal association with hippocampal ripples in a subset (ranging from 20-40s) of ACB neurons. It is not yet known whether this subset accounts for a large part of the reactivation as gauged by our EV and REV measures. Nevertheless, the exciting perspective emerging from these ensemble recording studies is that ripples may not only define narrow time windows in which hippocampal retrieval of specific behavioral

events occurs, but may also represent ‘information vehicles’ traveling from the hippocampal areas CA3 and CA1 and the subiculum, to target structures such as ACB, triggering reactivation at these sites. This working hypothesis is illustrated in Fig. 4.

Finally, after having considered reactivation in hippocampus, neocortex and ventral striatum, one may wonder whether this phenomenon may be generally present throughout the brain during sleep and may not be specific for brain areas involved in learning. Hoffman et al. (2001) presented evidence from monkey neocortex arguing against such a general, non-specific reactivation, because they failed to observe reactivation in the dorsal prefrontal cortex whereas the somatosensory and motor cortex did reactivate.

Different sleep phases, difSerent learning?

The multi-neuron recording studies reviewed above, notable for their high time resolution, offer evidence for a temporally specific recurrence of ensemble firing patterns in hippocampus, neocortex and ventral striatum during resting and sleep. Despite their sophistication and ability to reveal natural modes of information processing, they cannot answer the question on the causal relevance of this phenomenon for memory formation. No tools are currently known to disrupt precisely this phenomenon without af- fecting other physiological processes associated with sleep. In an attempt to bridge the large gap between these studies in rats and those conducted in humans, we will briefly summarize some recent literature on contributions by specific sleep stages to memory consolidation.

Studies aiming to manipulate specific sleep stages have primarily focussed on REM sleep deprivation both in humans and rats. In the past, animal studies have made extensive use of the platform deprivation technique, which exploits the fact that animals lose their muscular tone during REM sleep and thus will slide into the water surrounding the platform (Vimont-Vicary et al., 1966; Fishbein and Gutwein, 1977; Smith, 1985). This approach has been met with equally extensive criticisms because of confounding effects of stress and related motivational and emotional changes, such as anxiety, irritabil- ity and general activity levels (Vertes and Eastman,

Fig. 4. Hypothesis on the role of the hippocampal formation in triggering reactivation in neocortex and ventral striatum. Reactivation is postulated to originate spontaneously within the hippocampal formation. Owing to its recurrent associative connections and burst- generating capacities, area CA3 may constitute the site where ripple-associated reactivation is initiated, although external control is also possible. Following propagation to area CA1 and the subiculum, ripple-associated hippocampal output modulates ensemble activity both in neocortex and accumbens and thereby drives these structures into the appropriate state for spontaneous reactivation a.s well. Abbreviations: AC, anterior commissure; ACB, nucleus accumbens; DG, dentate gyrus; S, subiculum; TH, thalamic complex.

2000; Siegel, 2001). Although REM sleep deprivation studies in humans can be liable to the same critique, the more recent study by Karni et al. (1994) on visual discrimination learning and REM sleep included two important control procedures, making non-specific confounding effects of the deprivation procedure less likely. Whereas performance on this procedural learning task improved after a regular night of sleep, no learning effect occurred after REM sleep deprivation during a comparable sleep interval. In one control procedure, improvement was shown to occur when SWS was disrupted, which would also be expected to cause side effects such as stress. Sec- ondly, performance of a well-established task was

not affected by REM sleep deprivation. However, it remains possible that an interaction between REM sleep and stress caused a learning impairment.

Using a slightly modified version of the perceptual skill learning task of Karni et al. (1994), Stickgold et al. (2OOOa) adopted a correlational, non- intervening approach in humans and found overnight improvement in task performance to be proportional to the amount of SWS in the first quarter of the night and to the amount of REM sleep in the last quarter (Fig. 5A). The product of these two parameters was highly correlated to performance improvement (Fig. 5B). Essentially, these results do not contradict the results of Kami et al. (1994) but rather extend

1.58

1 2 3 4 - 0 50 loo 150 200

Quartile SWS, x REM, (% x %) Fig. 5. Correlations between early-night slow-wave sleep and late-night REM sleep with visual discrimination learning. (A) the Pearson correlation coefficient (r-value) between the relative amount of slow-wave sleep and overnight task improvement is plotted as a function of the consecutive quartiles of the total overnight sleep period (black squares). The same applies to the correlation between the relative amount of REM sleep and overnight task improvement (open circles). Heavy- and light-dashed lines indicate significance levels of 0.05 (r > 0.57) and 0.01 (r Y 0.71), respectively. Overnight task improvement is found to correlate significantly with slow-wave sleep during the first quartile of the night and with REM sleep during the last quartile. (B) if the improvement on the task is plotted as a function of the product of the relative amounts of slow-wave sleep in the first quartile (SWSr) and REM sleep in the fourth quartile (REW), a strong positive correlation is obtained (r = 0.89; P < 0.0001). This suggests a two-step model of memory consolidation during sleep. (From Stickgold, 1998. Reproduced with permission.)

their hypothesis by suggesting a two-step process depending on SWS in the early night and REM sleep in the late night as being critical for memory formation. In a complementary sleep deprivation study in humans, Stickgold et al. (2000b) showed that performance gradually increased across the first four day-night cycles after training, whereas no significant gain in learning was seen when subjects were deprived of sleep the first night after training and were tested following two nights of recovery sleep. Whereas these results control for non-specific consequences of sleep loss immediately before testing, it remains possible that such effects interacted with the consolidation process in the first night after training. While it is difficult to decide whether learning is actively supported by sleep or rather is permitted by the avoidance of stress in post-training stages, the results from Stickgold et al’s correlational study (2000a) would favor a beneficial role for sleep itself.

A study by Gais et al. (2000) on the same visual discrimination task produced results strikingly parallel to those of Stickgold et al. (2000a). In- stead of adopting the method of Karni et al. (1994) to disrupt REM sleep episodes shortly after their onset, which may lead to substantial sleep fragmen- tation and emotional disturbance, Gais et al. (2000)

deprived subjects of much larger blocks of sleep either in the early night (enriched with SWS) or late night (enriched with REM sleep), or across a full night. Effects of circadian timing on consolidation or testing were controlled for. No improvement was seen upon early night sleep deprivation, whereas late night deprivation produced only a minor gain in performance, clearly smaller than the effect of a full night’s sleep. Thus, REM-enriched, late night sleep on its own appeared insufficient for learning. These findings also support a two-step consolidation process during sleep, prompted by SWS and car- ried further by a second stage of late REM sleep, which is only effective if early sleep has occurred. However, the results do not rule out the possibility that early-night REM or late-night SWS may be important despite their lower prominence. It is not clear how the discrepancy concerning the necessity of SWS between the studies of Karni et al. (1994) and Gais et al. (2000) should be explained, although it is possible that the short period (about 30 min) of SWS remaining intact in the study of Kami et al. (1994) may have been sufficient for initiation of consolidation. In rats, SWS- and ripple-associated reactivation is most clear within the first 30 min after a behavioral experience (Kudrimoti et al., 1999).

159

It remains to be examined whether the results on the visual discrimination task can be generalized to others forms of skill learning.

As compared to procedural learning, less is known about the sleep dependence of declarative memory consolidation. Whereas some declarative memory tasks in humans failed to reveal a requirement for sleep, two recent studies argue in favor of such a dependence. Recall on a declarative paired- associate task after early or late night sleep deprivation was compared to recall on a mirror-tracing task. Improvement on the declarative memory task showed dependence on early night sleep, whereas procedural performance improved particularly when late night sleep was intact (Plihal and Born, 1997). In their study from 1999, using a similar experimental design, Plihal and Born found superior recall of spatial memory (in a mental spatial rotation task) after an early night sleep period as compared to the late night, whereas the opposite held for a non-declarative memory task (wordstem priming). Because early and late sleep retention intervals are enriched with SWS and REM sleep, respectively, these results suggest that SWS supports consolidation of declarative memory while REM sleep fa- cilitates non-declarative memory (Plihal and Born, 1999). This suggestion becomes even more tempting when recalling that reactivation in the rat hippocampus is especially associated with SWS and ripple episodes (Kudrimoti et al., 1999). However, slow hippocampal replay of often rehearsed behavioral experiences during REM sleep remains a distinct possibility (Louie and Wilson, 2001). Smith and Rose (1997) compared performance of rats on two versions of the Morris water maze: one (spatial) version with a hidden platform and one (visually guided) version with a visible platform. REM sleep deprivation was found to impair performance in the spatial, hippocampus-dependent version but not the visually guided, striatum-dependent task.

More types of declarative memory will have to be tested before generalizing conclusions can be drawn. Further studies should also take into account that: (1) the dependence on SWS may be restricted to a rather narrow time window during the early phase of sleep, considering the decay of SWS-associated reactivation in the hippocampus; and (2) the parameter critical for consolidation may not be SWS per se, but

rather the repetitive occurrence of ripple episodes, which also occur during quiet wakefulness.

In summary, recent studies in humans and rats support the concept that certain forms of procedural learning depend on a sequential memory processing during early night, SWS-enriched sleep followed by a later, REM-enriched sleep period. Certain forms of spatial and declarative memory show dependence on early sleep periods predominated by SWS and associated with hippocampal reactivation, possibly with additional dependence on subsequent REM sleep (Fig. 6). -

Synthesis: three-phase hypothesis of memory consolidation

Before attempting to integrate the data reviewed above, ranging from human cognition to single- neuron firing, we wish to draw attention to some recent studies on cue-triggered memory reactivation in the awake state. Behavioral evidence supports the concept that exposure of the awake animal to a cue or a context previously associated with a learning task reactivates memories linked to that task and renders these neural representations fragile and modifiable. If spontaneous reactivation during sleep represents a similar kind of recall, it may be assumed that this process activates the same synaptic matrices as during waking, and thereby renders them modifiable as well. As in the awake case, this modifiability may be presumed to allow reinforcement of initially weak representations by repeated replay and updating of older memories to match the current situation more precisely.

Przybyslawski and Sara (1997) first overtrained rats on a radial arm maze task. At a dose which had no effect on overt behavior, pretrial injection of MK-801 disrupted well-trained performance of this task. In line with the concept of memory reconsolidation, they also found a persisting behavioral deficit (i.e. increased number of errors to find the three baited alleys) on a drug-free performance test 24 h after MK-801 injection. The temporal window for observing these amnestic drug effects comprised the first 2 h following the trial, thus indicating the period during which NMDA receptor activity would be required for reconsolidation. Although the injections were given systemically, it is reasonable to assume

160

Macroscopic state Cellular Processes

Phase 1: Awake state Initial storage

Phase 2: SIow-wave sleep: I Fast replay during ripples Short-term consolidation

Phase 3: REM sleep:

I

High ACh activity Old/new memory selection Further consolidation

NMDA-R dependent synaptic plasticity

Muscarinic-R and PLC/PKC mediated effects on synaptic plasticity

Fig. 6. Three-phase hypothesis of memory consolidation. On the left-hand side, macroscopic brain states are represented. On the right-hand side, the corresponding cellular processes are outlined. The first stage of consolidation takes place during the actual behavioral experience in the awake state. Little is known about the cellular processes underlying this initial storage process. The second stage is marked by the prominence of slow-wave sleep and associated hippocampal ripples. This second stage may already begin before the actual onset of slow-wave sleep, because ripples are also found during behavioral immobility and consummatory behavior in the awake state. Fast replay during high-frequency ripples may give rise to NMDA-receptor dependent synaptic weight changes in the hippocampus and its target areas, supporting short-term, consolidation. The hallmark of the third phase is REM sleep, with its prominent muscarinic receptor-dependent theta rhythm. This phase may serve to further strengthen consolidated memory traces, integrate and replay larger segments of behavioral information, and select newer versus older memory traces at the level of hippocampal output. The biochemical cascade activated by muscarinic receptors may support those forms of plasticity subserving these macroscopic processes. The hypothesis is based on data from correlative as well as interventional studies on sleep in humans, data from unit recordings and interventional studies on sleep in rats and in vitro electrophysiological studies on long-term potentiation and other forms of synaptic plasticity.

that the hippocampus was critically affected. Fur- thermore, the exact wake/sleep state of the animal in the posttrial period was not documented in Przy- byslawski and Sara (1997), but very likely involved resting and/or sleeping in the home cage (personal communication). The results with MK-801 contrast to those with intracranially applied b-(noradrenergic) receptor blocker, which caused amnestic effects not earlier than 60 min post-training (Roullet and Sara, 1998).

Combined with the human data and the rat studies on spontaneous reactivation, these findings permit us to outline a three-phase hypothesis of memory consolidation. It should be noted that much of the neurophysiological evidence supporting this hypothesis is derived from studies on declarative/spatial memory and the hypothesis as such is of a heuristic nature. It should also be kept in mind that the hypothesis is based on both human and animal literature, and that there are large differences between different

161

animal species in sleep architecture, dynamics of individual sleep stages and alternations of wake/sleep episodes throughout the day-night cycle. Such differences have not yet been accommodated at this rather crude stage of formulation.

The first phase is composed of early, rapid memory storage during and just after the relevant behavioral experience occurred (McGaugh, 1966; Fig. 6). Surprisingly, little is known about the mechanisms underlying this phase. Experiments by Miller and Springer (1971) suggest that at least temporary representations associated with passive avoidance behavior with a footshock as unconditioned stimulus can be formed within seconds after training. They exposed rats to a single training trial and adminis- tered electroconvulsive shock 10 s after footshock onset. Performance was found intact when tested 15 min later, but not with longer delays in testing. Thus, the initial memory may be formed rapidly, seems relatively resilient to a massively disruptive stimulus but is nonetheless transient in nature. It has proven difficult so far to investigate specific receptor and transduction signaling pathways involved in this acute awake phase, because manipulations by drug infusion often extend beyond the immediate learning phase. Taking this restriction into account, NMDA receptors may mediate some aspects of early hippocampus-dependent memory but not others (Riedel et al., 1999; Steele and Morris, 1999; Zamanillo et al., 1999; Huerta et al., 2000).

The second phase is postulated to span the period of off-line processing during rest/sleep periods directly after training. This period is predominated by SWS and, within the hippocampus, by ripple- associated replay of memory representations. Neo- cortical and striatal target areas are modulated by these ripples and/or the events underlying the triggering of ripples (Battaglia et al., 2001; Pennartz et al., 2001). Possibly, ripples draw these areas into a state of reactivation. The findings of Przybyslawski and Sara (1997) argue for a strong NMDA receptor dependence at this stage. Tentatively, this can be explained by recalling that ripples are rather massive high-frequency (-200 Hz) population discharges, resembling bursts of tetanic stimulation having been shown to induce NMDA-receptor dependent LTP in in vitro studies (Buzs&i et al., 1987; Staubli and Lynch, 1987; Larson and Lynch, 1988). Al-

though hippocampal ensemble reactivation is still observed after NMDA receptor blockade (Ekstrom et al., 2000), we hypothesize that the consequence of ripple-associated reactivation is affected by this treatment. This consequence holds that the synaptic interfaces targeted by those hippocampal population subsets selectively engaged in high-frequency reactivation bursts will be modified to strengthen the associative storage of new memories. From the perspective of area CAl, the main direct targets would include the subiculum, entorhinal and perirhinal cortices, ventral striatum, whereas associative neocortical areas, including cingulate cortex, and the amygdaloid complex may be reconfigured by polysynaptic projections (cf. Gabriel, 1990).

In rats, SWS-associated replay and consolidation decays across about 30-60 min, but is hypothesized to leave behind a reinforced set of cell assemblies that are capable of more prolonged reactivation, amongst others during REM sleep. This constitutes the third phase of consolidation. Possibly, REM replay occurs at a slower time scale than SWS replay and may involve larger time segments of behavioral records (Louie and Wilson, 2001). One of the hall- marks of REM sleep, theta activity, strongly depends on activation of the cholinergic system and may sub- serve two functions: (1) selective strengthening of newly acquired behavioral records at the expense of others, due to shifting of phase relationships between firing patterns and the theta cycle, and concomitant shifts in LTP/LTD induction (Poe et al., 2000); (2) general support for synaptic plasticity by activation of muscarinic receptor signaling pathways involving G-proteins, IP3, phospholipase C, protein kinase C and A (Metherate et al., 1987; Auerbach and Segal, 1994; Kirkwood et al., 1999; Graves et al., 2001).

The hypothesized sequential dependence of consolidation on early ripple-associated processing, prominently but not exclusively occurring during SWS, and later REM sleep, is consistent with the aforementioned correlational and interventional studies on visual discrimination learning in humans (Gais et al., 2000; Stickgold et al., 2000a). These and other studies have indeed prompted similar hypotheses emphasizing sequential consolidation stages (Giuditta et al., 1995; Stickgold et al., 2001) viewed from a more global, psychological angle rather than from a cellular and network perspec-

162

tive. The question whether a sequential scheme also holds for declarative memory must await experimental scrutiny in the future. It should be recalled that SWS and REM sleep can alternate within larger sleep epochs, and it may well be that natural SWS- REM sequences are repeated cyclically during a full night’s sleep. Finally, previous proposals favoring REM sleep as a hippocampal ‘read/storage’ mode and SWS as a ‘send/retrieval’ mode (cf. Buzsaki, 1989) also deserve further scrutiny.

Conclusions and future perspectives

Clearly, the hypothesis advanced above must be regarded as a heuristic working model rather than an established theory. In this review, we have at- tempted to forge links between our knowledge of synaptic plasticity and neuronal population dynamics on the one hand and macroscopic behavioral- cognitive phenomena on the other hand. This cannot be done unless inference and speculation is admitted on some points, and these are precisely the points calling for further investigation. First, the causal relevance of cortical and striatal ensemble reactivation for memory consolidation awaits further study. A difficult problem will be to examine the contribution by isolated brain regions in a time-restricted way, so that distinctions between the contributions of various sleep phases can be made in an anatom- ically specific, stress-free manner. Reactivation in neocortical regions and ventral striatum has been demonstrated, and we know that activity in these structures targeted directly or indirectly by the hippocampus is modulated in close association with hippocampal ripples. Reactivation is not restricted to a hippocampal-neocortical dialogue. Much remains to be learned, however, about the precise modes of information transfer between areas, and how puta- tive hippocampal traces may be disseminated, in a parallel-distributed fashion, to other areas (Treves and Rolls, 1994; McNaughton et al., 2002). An- other gap in our knowledge concerns the possible dependence of REM sleep processing on preceding SW& and the question whether results from perceptual skill tasks can be generalized to declarative memory and motor skills. What are the critical steps during SWS and other early ‘off-line’ processing stages that would be required to make subse-

quent REM sleep effective for further consolidation? Finally, prolonged research efforts will be needed to elucidate the molecular and cellular mechanisms underlying different consolidation phases, not only paying the traditional attention to NMDA receptor- dependent LTP but also to alternative mechanisms.

Acknowledgements

This work was supported by PHS Grant MH46823 to B.L.M., NATO collaborative Grant CRG 972196 and HFSPO Grant RGP 0127 to C.M.A.P. and B.L.M. and the Graduate School Neurosciences Amsterdam. We would like to thank J. Verheul for her comments on this review.

References

Amzica, F., Neckelmann, D. and Steriade, M. (1997) Instrumen- tal conditioning of fast (20- to 50-Hz) oscillations in corti- cothalamic networks. Pmt. Narl. Acad. Sci. USA, 94: 1985- 1989.

Auerbach, J.M. and Segal, M. (1994) A novel cholinergic induction of long-term potentiation in rat hippocampus. J. Neum- physiol., 72: 2034-2040.

Battaglia, F.P., Sutherland, G.R. and McNaughton, B.L. (2001) Widespread modulation of neocortical cell activity during hippocampal sharp waves. Sot. Neurosci. Abstx, 27: 1699.

Baxter, M.G., Parker, A., Lindner, C.C.C., Izquierdo, A.D. and Murray, E.A. (2000) Control of response selection by re- inforcer value requires interaction of amygdala and orbital prefrontal cortex. .I. Neumsci., 20: 4311-4319.

Buzs&i, G. (1986) Hippocampal sharp waves: their origin and significance. Brain Res., 398: 242-252.

Buz&i, G. (1989) Two-stage model of memory trace formation: a role for “noisy” brain states. Neuroscience, 31: 551-570.

Buzsaki, G., Haas, H.L. and Anderson, E.G. (1987) Long-term potentiation induced by physiologically relevant stimulus patterns. Brain Res., 435: 331-333.

Buzsaki, G., Horvath, U.R., Hetke, J. and Wise, K. (1992) High- frequency network oscillation in the hippocampus. Science, 256: 1025-1027.

Cahill, L., McGaugh, J.L. and Weinberger, N.M. (2001) The neurobiology of learning and memory: some reminders to remember. Trends Neurosci., 24: 578-581.

Callaway, C.W. and Hemiksen, S.J. (1992) Neuronal firing in the nucleus accumbens is associated with the level of cortical arousal. Neuroscience, 51: 547-553.

Cohen, N.J. and Squire, L.R. (1980) Preserved learning and retention of pattern analyzing skill in amnesia: dissociation of knowing how and knowing that. Science, 210: 207-209.

Deweer, B. and Sara, S.J. (1984) Background stimuli as a reminder after spontaneous forgetting: role of duration of cuing and cuing-test interval. Anim. Learn. Behav., 12: 238-247.

163

Eichenbaum, H. (2000) A cortical-hippocampal system for declarative memory. Nat. Rev. Neumsci., 1: 41-50.

Eichenbaum, H., Dudchenko, P., Wood, E., Shapiro, M. and Tanila, H. (1999) The hippocampus, memory, and place cells: is it spatial memory or a memory space? Neuron, 23: 209- 226.

Ekstrom, A.D., Meltzer, J.A., McNaughton, B.L. and Barnes, C.A. (2000) Reactivation of recent hippocampal neural ensemble patterns does not require NMDA receptor dependent encoding. Sot. Neurosci. Abstr, 26: 981.

Fishbein, W. (1971) Disruptive effects of rapid eye movement sleep deprivation on long-term memory. Physiol. Behav., 6: 279-282.

Fishbein, W. and Gutwein, B.M. (1977) Paradoxical sleep and memory storage processes. Behan Biol., 19: 425-464.

Gabriel, M. (1990) Functions of anterior and posterior cingulate cortex during avoidance learning in rabbits. In: H.B.M. Uylings, C.G. van Eden, J.P.C. de Bruin, M.A. Comer and M.G.P. Feenstra (Eds.), The Prefrontal Cortex: Its Structure, Function and Pathology. Progr Brain Res., Vol. 85. Elsevier, Amsterdam, pp. 467-485.

Gais, S., Plibal, W., Wagner, U. and Born, J. (2000) Early sleep triggers memory for early visual discrimination skills. Nat. Neumsci., 3: 13351339.

Gehring, W.J. and Knight, R.T. (2000) Prefrontal-cingulate in- teractions in action monitoring. Nat. Neumsci., 3: 516-520.

Giuditta, A., Ambrosini, M.V., Montagnese, P., Mandile, P., Cotugno, M., Zucconi, G.G. and Vescia, S. (1995). Behav. Brain Rex, 69: 157-168.

Graves, L., Pack, A. and Abel, T. (2001) Sleep and memory: a molecular perspective. Trends Neurosci., 24: 237-243.

Groenewegen, H.J., Vermeulen-Van der Zee, te Kortschot, A. and Witter, MI? (1987) Organization of the projection from the subiculum to the ventral striatum in the rat. A study using anterograde transport of Phaseolus vulgaris leucoagglutinin. Neuroscience, 23: 103-120.

Ham, B., Hennevin, E. and Pasques, P (1985) Improvement of learning by cueing during postlearning paradoxical sleep. Behav. Brain Res., 18: 241-250.

Hebb, D.O. (1949) The Organization of Behavior. New York, Wiley.

Hennevin, E. and Ham, B. (1985) Post-learning paradoxical sleep: a critical period when new memory is reactivated? In: B.E. Will, P. Schmitt and J.C. Dahymple-Alford (Eds.), Brain Plasticity, Learning and Memory. Plenum, New York, pp. 193-203.

Hoffman, K.L., McNaughton, B.L., Lipa, P., Ellmore, T.M. and Stengel, K. (2001) Spontaneous reactivation of recent memory traces in macaque neocortical ensembles. Sot. Neurosci. Abstr, 27: 932.

Holscher, C., Anwyl, R. and Rowan, M.J. (1997) Stimulation on the positive phase of hippocampal theta rhythm induces long- term potentiation that can be depotentiated by stimulation on the negative phase in area CA1 in vivo. J. Neurosci., 17: 6470-6477.

Huerta, PT. and Lisman, J.E. (1995) Bidirectional synaptic plas-

ticity induced by a single burst during cholinergic theta oscillation in CA1 in vitro. Neuron, 15: 1053-1063.

Huerta, PT., Sun, L.D., Wilson, M.A. and Tonegawa, S. (2ooO) Formation of temporal memory requires NMDA receptors within CA1 pyramidal neurons. Neuron, 25: 473-480.

James, W. (1890) The Principles of Psychology. Hemy Halt/Dover edition, New York.

Jones, M.W., Errington, M.L., French, PJ., Fine, A., Bliss, T.V.P., Garel, S., Charnay, P., Bozon, B., Laroche, S. and Davis, S. (2001) A requirement for the immediate early gene Zif268 in the expression of late LTP and long-term memories. Nat. Neurosci., 4: 289-296.

Judge, M.E. and Quartermain, D. (1982) Alleviation of anisomycin-induced amnesia by pre-test treatment with lysine- vasopressin. Phannacol. Biochem. Behav., 16: 463-466.

Karni, A., Tanne, D., Rubenstein, B.S., Askenasy, J.J.M. and Sagi, D. (1994) Dependence on REM sleep of overnight improvement of a perceptual skill. Science, 265: 679-682.

Kim, 3.5. and Fanselow, MS. (1992) Modality-specific retrograde amnesia of fear. Science, 256: 675-677.

Kirkwood, A., Rozas, C., Kirkwood, J., Perez, F. and Bear, M.F. (1999) Modulation of long-term synaptic depression in visual cortex by acetylcholine and norepinephrine. .I. Neumsci., 19: 1599-1609.

Kleinbaum, D.G., Kupper, L.L., Muller, K.E. and Nizam, A. (1998) Applied Regression Analysis and Other Multivariable Methods. Duxbury, Pacific Grove.

Knowlton, B.J., Mangels, J.A. and Squire, L.R. (1996) A neo- striatal habit learning system in humans. Science, 273: 1399- 1402.

Kudrimoti, H.S., Barnes, CA. and McNaughton, B.L. (1999) Reactivation of hippocampal cell assemblies: effects of behavioral state, experience, and EEG dynamics. J. Neumsci., 19: 4090-4101.

Larson, J. and Lynch, G. (1988) Role of N-methyl-D-aspartate receptors in the induction of synaptic potentiation by burst stimulation patterned after the hippocampal &rhythm. Brain Res., 441: 111-118.

Lavoie, A.M. and Mizumori, S.J.Y. (1994) Spatial, movement- and reward-sensitive discharge by medial ventral striatum neurons of rats. Brain Res., 638: 157-168.

Liu, Z., Murray, E.A. and Richmond, B.J. (2000) Learning motivational significance of visual cues for reward schedules requires rhinal cortex. Nat. Neumsci., 3: 1307-1315.

Louie, K. and Wilson, M.A. (2001) Temporally structured replay of awake hippocampal ensemble activity during rapid eye movement sleep. Neuron, 29: 145-156.

Mactutus, C.F., Riccio, D.C. and Ferer, J.M. (1979) Retrograde amnesia for old (reactivated) memory: some anomalous char- acteristics. Science, 204: 1319-1320.

Maquet, P., Laureys, S., Peigneux, E, Fuchs, S., Petiau, C., Phillips, C., Aerts, J., Del Fiore, G., Degueldre, C., Meule- mans, T., Luxen, A., Franck, G., Van Der Linden, M., Smith, C. and Cleeremans, A. (2000) Experience-dependent changes in cerebral activation during human REM sleep. Nat. Neu- msci., 3: 1235-1236.

164

Marr, D. (1971) Simple memory: a theory for archicortex. Philos. Trans. R. Sot. Land. B., 262: 23-81.

McClelland, J.L., McNaughton, B.L. and O’Reilly, R.C. (1995) Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and fail- ures of connectionist models of learning and memory. Psychol. Rev., 102: 419-456.

McGaugh, J.L. (1966) Time dependent processes in memory storage. Science, 153: 1351-1358.

McGaugh, J.L. (2000) Memory - a century of consolidation. Science, 287: 248-251.

McNaughton, B.L. (1998) The neurophysiology of reminiscence. Neurobiol. Learn. Mem., 70: 252-267.

McNaughton, B.L. and Morris, R.G.M. (1987) Hippocampal synaptic enhancement and information storage within a distributed memory system. Trends Neurosci., 10: 408-415.

McNaughton, B.L., Barnes, C.A., Battaglia, F.P., Bower, M.R., Cowen, S.L., Ekstrom, A.D., Gerrard, J.L., Hoffman, K.L., Houston, EI?, Karten, Y., Lipa, P., Pennartz, C.M.A. and Sutherland, G.R. (2002) Off-line reprocessing of recent memory and its role in memory consolidation: a progress report. In: P. Maquet, B. Stickgold and C. Smith (Eds.), Sleep and Brain Plasticity. in press.

Meek, W.H., Church, R.M. and Olton, D.S. (1984) Hippocam- pus, time, and memory. Behav. Neurosci., 98: 3-22.

Metherate, R., Tremblay, N. and Dykes, R.W. (1987) Acetyl- choline permits long-term enhancement of neuronal respon- siveness in cat primary somatosensory cortex. Neuroscience, 22: 75-81.

Meunier, M., Bachevalier, J., Mishkin, M. and Murray, E.A. (1993) Effects on visual recognition of combined and separate ablations of the entorhinal and perirhinal cortex in rhesus monkeys. J. Neurosci., 13: 5418-5432.

Miller, R.R. and Springer, A.D. (1971) Temporal course of amnesia in rats after electroconvulsive shock. Physiol. Behav., 6: 229-233.

Mirmiran, M., Van den Dungen, H. and Uylings, H.B.M. (1982) Sleep patterns during rearing under different environmental conditions in juvenile rats. Brain Res., 233: 287-298.

Misanin, J.R., Miller, R.R. and Lewis, D.J. (1968) Retrograde amnesia produced by electroconvulsive shock after reactivation of a consolidated memory trace. Science, 160: 554-555.

Mishkin, M., Malamut, B. and Bachevalier, J. (1984) Memories and habits: two neural systems. In: G. Lynch, J.L. McGaugh and N.M. Weinberger (Eds.), Neurobiology of Leaning and Memory. Guildford Press, New York, pp. 65-17.

Mogenson, G.J., Jones, D.L. and Yrm, C.Y. (1980) From motivation to action: functional interface between the limbic system and the motor system. Progl: NeurobioZ., 14: 69-97.

Morris, R.G.M., Anderson, E., Lynch, G.S. and Bat&y, M. (1986) Selective impairment of learning and blockade of long- term potentiation by an N-methyl-D-aspartate receptor antagonist, AF’5. Nature, 319: 774-776.

Nadal, S., Toulouse, G., Changeux, J.P. and DeHaene, S. (1986) Networks of formal neurons and memory palimpsests. Euro- phys. L.&t., 1: 535-542.

Nadasdy, Z., Hirase, H., Czurk6, A., Csicsvari, J. and Buzs&i,

G. (1999) Replay and time compression of recurring spike sequences in the hippocampus. .J. Neurosci., 19: 9497-9507.

Nader, K., Schafe, G.E. and Le Doux, J.E. (2OBO) Fear memories require protein synthesis in the amygdala for reconsolidation after retrieval. Nature, 406: 722-726.

Nishijo, H., Ono, T. and Nishino, H. (1988) Single neuron responses in amygdala of alert monkey during complex sensory stimulation with affective significance. J. Neurosci., 8: 3570- 3583.

O’Keefe, J. and Dostrovsky, J. (1971) The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat. Brain Res., 34: 171-175.

Pavlides, C. and Wrnson, J. (1989) Influences of hippocampal place cell firing in the awake state on the activity of these cells during subsequent sleep episodes. J. Neurosci., 9: 2907-2918.

Pavlides, C., Greenstein, Y.J., Gmdman, M. and Winson, J. (1988) Long-term potentiation in the dentate gyrus is induced preferentially on the positive phase of theta-rhythm. Brain Res., 439: 383-387.

Pearlman, CA. and Greenberg, R. (1973) Posttrial REM sleep: a critical period for consolidation of shuttlebox avoidance. Anim. Learn. Behav., 1: 49-51.

Pennartz, C.M.A. (1996) The ascending neuromodulatory sys- terns in learning by reinforcement: comparing computational conjectures with experimental findings. Brain Res. Rev, 21: 219-245.

Pennartz, C.M.A. (1997) Reinforcement learning by Hebbian synapses with adaptive thresholds. Neuroscience, 81: 303- 319.

Pemtartz, C.M.A., Groenewegen, H.J. and Lopes da Silva, F.H. (1994) The nucleus accumbens as a complex of functionally distinct neuronal ensembles: an integration of behavioural, electrophysiological and anatomic data. Progr: Neumbiol., 42: 719-761.

Pemmrtz, C.M.A., McNaughton, B.L. and Mulder, A.B. (2000) The glutamate hypothesis of reinforcement learning. In: H.B.M. Uylings, C.G. van Eden, J.P.C. de Bruin, M.G.P. Feenstra and C.M.A. Pennartz (Eds.), Cognition, Emotion and Autonomic Responses: the Integrative Role of the Prefrontal Cortex and Limbic Structures. Progress in Brain Research, Vol. 126. Elsevier, Amsterdam, pp. 231-253.

Pennartz, C.M.A., Geurtsen, A.M.S., Lipa, P, Barnes, C.A. and McNaughton, B.L. (2001) Reactivation of neuronal ensembles in the nucleus accumbens during sleep. Sot. Neurosci. Abstr, Vol. 27, program number 189.6.

Phillips, R.G. and LeDoux, J.E. (1992) Differential contribution of amygdala and hippocampus to cued and contextual fear conditioning. Behav. Neurosci., 106: 274-285.

Plihal, W. and Born, J. (1997) Effects of early and late nocturnal sleep on declarative and procedural memory. J. Cogn. Neurosci. 9: 534-547.

Plihal, W. and Born, J. (1999) Effects of early and late nocturnal sleep on priming and spatial memory. Psychophysiology, 36: 571-582.

Poe, G.R., Nitz, D.A., McNaughton, B.L. and Barnes, C.A. (2000) Experience-dependent phase-reversal of hippocampal neurons firing during REM sleep. Brain Res., 855: 176-180.

165

Przybyslawski, J. and Sara, S.J. (1997) Reconsolidation of memory after its reactivation. Behav. Brain Res., 84: 241-246.

Qin, Y.-L., McNaughton, B.L., Skaggs, W.E. and Barnes, CA. (1997) Memory reprocessing in corticocortical and hippocam- pocortical neuronal ensembles. Philos. Trans. R. Sot. Lond. B., 352: 1525-1533.

Ribeiro, S., Goyal, V., Mello, C.V. and Pavlides, C. (1999) Brain gene expression during REM sleep depends on prior waking experience. Learn. Men., 6: 500-508.

Ribot, T. (1882) Diseases of Memory. Appleton-Century Crofts, New York, NY.

Riedel, G., Micheau, J., Lam, A.G.M., Roloff, E.V.L., Martin, S.J., Bridge, H., de Hoz, L., Poeschel, B., McCulloch, J. and Morris, R.G.M. (1999) Reversible neural inactivation reveals hippocampal participation in several memory processes. Nat. Neurosci., 2: 898-905.

Robbins, T.W. and Eve&t, B.J. (1996) Neurobehavioural mechanisms of reward and motivation. Curr: Opin. Neurobiol., 6: 228-236.

Rolls, E. (1989) The representation and storage of information in neuronal networks in the primate cerebral cortex and hippocampus. In: R. Durbin, C. Miall and G. Mitchison (Eds.), The Computing Neuron. Addison-Wesley, Wokingham.

Roullet, P and Sara, S.J. (1998) Consolidation of memory after its reactivation: involvement of b noradrenergic receptors in the late phase. Neural. Plasticity, 6: 63-68.

Sara, S.J. (2000) Retrieval and reconsolidation: toward a neurobiology of remembering. Learn. Mem., 7: 73-84.

Schoenbaum, G., Chiba, A.A. and Gallagher, M. (1998) Or- bitofrontal cortex and basolateral amygdala encode expected outcomes during learning. Nat. Neutosci., 1: 155-159.

Schultz, W. (2000) Multiple reward signals in the brain. Nut. Rev Neurosci., 1: 199-207.

Schultz, W., Apicella, P, Scarnati, E. and Ljungberg, T. (1992) Neuronal activity in monkey ventral striatum related to the expectation of reward. J. Neurosci., 12: 4595-4610.

Scoville, W.B. and Milner, B. (1957) Loss of recent memory after bilateral hippocampal lesions. J. Neuml. Neurosurg. Psy- chiatry, 20: 11-21.

Selden, N.R.W., Everitt, B.J., Jarrard, L.E. and Robbins, T.W. (199 1) Complementary roles for the amygdala and hippocampus in aversive conditioning to explicit and contextual cues. Neuroscience, 42: 335-350.

Setlow, B. and McGaugh, J.L. (1998) Sulpiride infused into the nucleus accumbens posttraining impairs memory of spatial water maze training. Behav. Neurosci., 112: 603-610.

Shibata, R., Mulder, A.B., Trullier, 0. and Wiener, S.I. (2001) Position sensitivity in phasically discharging nucleus accumbens neurons of rats alternating between tasks requiring complementary types of spatial cues. Neuroscience, 108: 391-411.

Siapas, A.G. and Wilson, M.A. (1998) Coordinated interac- tions between hippocampal ripples and cortical spindles during slow-wave sleep. Neuron, 21: 1123-1128.

Siegel, J.M. (2001) The REM sleep-memory consolidation hypothesis. Science, 294: 1058-1063.

Skaggs, W.E. and McNaughton, B.L. (1996) Replay of neuronal

firing sequences in rat hippocampus during sleep following spatial experience. Science, 271: 1870-1873.

Smith, C. (1985) Sleep states and learning: a review of the animal literature. Neurosci. Biobehav. Rev, 9: 157-168.

Smith, C. and Rose, G.M. (1996) Evidence for a paradoxical sleep window for place learning in the Morris water maze. Physiol. Behav., 59: 93-97.

Smith, C. and Rose, G.M. (1997) Posttraining paradoxical sleep in rats is increased after spatial learning in the Morris water maze. Behav. Neumsci., 111: 1197-1204.

Squire, L.R. and Alvarez, P (1995) Retrograde amnesia and memory consolidation: a neurobiological perspective. Cur,: Opin. Neurobiol., 5: 169-177.

Squire, L.R. and Zola-Morgan, S. (1991) The medial temporal lobe memory system. Science, 253: 1380-1386.

Squire, L.R., Shimamura, A.P. and Amaral, D.G. (1989) Memory and the hippocampus. In: J.H. Byrne and W.O. Berry (Eds.), Neural Models of Plasticity: Experimental and Theoretical Approaches. Academic Press, New York, pp. 208-239.

Squire, L.R., Knowlton, B. and Musen, G. (1993) The structure and organization of memory. Annu. Rev. Psychol., 44: 453- 495.

Staubli, U. and Lynch, G. (1987) Stable hippocampal long-term potentiation elicited by ‘theta’ pattern stimulation. Bruin Res., 435: 227-234.

Steele, R.J. and Morris, R.G.M. (1999) Delay dependent impairment in matching-to-place task with chronic and intrahip- pocampal infusion of the NMDA antagonist D-APS. Hip- pocampus, 9: 118-136.

Stickgold, R. (1998) Sleep: off-line memory processing. Trends Cogn. Neutosci., 2: 484-489.

Stickgold, R., Whidbee, D., Schirmer, B., Patel, V and Hobson, J.A. (2000a) Visual discrimination task improvement: a multi- step process occurring during sleep. J. Cogn. Neurosci., 12: 246-254.

Stickgold, R., James, L.T. and Hobson, J.A. (2000b) Visual discrimination learning requires sleep after training. Nat. Neu- rosci., 3: 1237-1238.

Stickgold, R., Hobson, J.A., Fosse, R. and Fosse, M. (2001) Sleep, learning, and dreams: off-line memory reprocessing. Science, 294: 1052-1057.

Suzuki, W.A., Zola-Morgan, S., Squire, L.R. and Amaral, D.G. (1993) Lesions of the perirhinal and parahippocampal cortices in the monkey produce long-lasting memory impairment in the visual and tactual modalities. J. Neurosci., 13: 2430-245 1.

Teyler, J.T. and DiScenna, P. (1986) The hippocampal memory indexing theory. Behav. Neurosci., 100: 147-154.

Treves, A. and Rolls, E.T. (1992) Computational constraints suggest the need for two distinct input systems to the hippocampal CA3 network. Hippocampus, 2: 189-200.

Treves, A. and Rolls, E.T. (1994) Computational analysis of the role of the hippocampus in memory. Hippocampus, 4: 374- 391.

Van Hoesen, G.W. (1982) The parahippocampal gyms. New observations regarding its cortical connections in the monkey. Trends Neumsci., 5: 345-350.

166

Vertes, R.P. and Eastman, K.E. (2000) The case against memory consolidation in REM sleep. Behav. Brain Sci., 23: 867-876.

Vimont-Vicaty, P., Jouvet-Mounier, D. and Delorme, F. (1966). Electmencephalogr Clin. Neurophysiol., 20: 439-449.

White, N.M. (1997) Mnemonic functions of the basal ganglia. Cum Opin. Neurobiol., 7: 164-169.

Williams, G.V., Rolls, E.T., Leonard, CM. and Stem, C. (1993) Neuronal responses in the ventral striatum of the behaving macaque. Behav. Bruin Res., 55: 243-252.

Wilson, M.A. and McNaughton, B.L. (1993) Dynamics of the hippocampal ensemble code for space. Science, 261: 105% 1058.

Wilson, M.A. and McNaughton, B.L. (1994) Reactivation of hippocampal ensemble memories during sleep. Science, 265: 676-679.

Winocur, G. (1990) Anterograde and retrograde amnesia in rats with dorsal hippocampal or dorsomedial thalamic lesions. Be- hav. Brain Rex, 38: 145-154.

Witter, M.P., Groenewegen, H.J., Lopes da Silva, F.H. and Lohman, A.H.M. (1989) Functional organization of the ex- trinsic and intrinsic circuitry of the parahippocampal region. Pmgr: Neurobiol., 33: 161-253.

Witter, MI?, Naber, P.A., van Haeften, T., Machielsen, W.C., Rombouts, S.A., Barkhof, F., Scheltens, P and Lopes da Silva, F.H. (2000) Cortico-hippocampal communication by way of parallel parahippocampal-subicular pathways. Hippocampus, 10: 398-410.

Wood, E.R., Dudchenko, P.A. and Eichenbaum, H. (2000) The global record of memory in hippocampal neuronal activity. Nature, 397: 613-616.

Ylinen, A., Bragin, A., Nadasdy, Z., Jando, G., Szabd, I., Sik, A. and Buzs&i, G. (1995) Sharp wave-associated high-frequency oscillation (200 Hz) in the intact hippocampus: network and intracellular mechanisms. .I. Neumsci., 15: 30-46.

Zamanillo, D., Sprengel, R., Hvalby, @., Jenssen, V., Bumashev, N., Rozov, A., Kaiser, K.M.M., Koster, H.J., Borchardt, T., Worley, P., Liibke, J., Fritscher, M., Kelly, PH., Sommer, B., Andersen, P., Seeburg, P.H. and Sakmann, B. (1999) Impor- tance of AMPA receptors for hippocampal synaptic plasticity but not for spatial learning. Science, 284: 1805-1811.

Zola-Morgan, S. and Squire, L.R. (1990) The primate hippocampal formation: evidence for a time-limited role in memory storage. Science, 250: 288-290.

[progress in brain research] plasticity in the adult brain: from genes to neurotherapy volume 138 ||...

Documents