Emotional Modulation of Visual Attention

by

Emma Ferneyhough

A dissertation submitted in partial fulfillment

of the requirements for the degree of

Doctor of Philosophy

Department of Psychology

New York University

September, 2011

_________________________________

Elizabeth A. Phelps, PhD

_________________________________

Marisa Carrasco, PhD

All rights reserved

INFORMATION TO ALL USERSThe quality of this reproduction is dependent on the quality of the copy submitted.

In the unlikely event that the author did not send a complete manuscriptand there are missing pages, these will be noted. Also, if material had to be removed,

a note will indicate the deletion.

All rights reserved. This edition of the work is protected againstunauthorized copying under Title 17, United States Code.

ProQuest LLC.789 East Eisenhower Parkway

P.O. Box 1346Ann Arbor, MI 48106 - 1346

UMI 3482879

Copyright 2011 by ProQuest LLC.

UMI Number: 3482879

© Emma Ferneyhough

All Rights Reserved, 2011

Chapter 1 has been published elsewhere (Psychonomic Bulletin and Review

(2010), Volume 17, Issue 4, p. 529-535). Per the publishing agreement with

PBR, the final version has been included in this dissertation. Excerpt from the

publishing agreement (http://www.psychonomic.org/psp/access.html):

The author retains the right to use his/her article for his/her further scientific career by including the final published journal article in other publications such as dissertations and postdoctoral qualifications provided acknowledgement is given to the original source of publication.

“Fear is the mind-killer.”

- Frank Herbert, Dune (1965)

iv

DEDICATION

For my family and friends.

v

ACKNOWLEDGEMENTS

I would like to thank my advisors Liz Phelps and Marisa Carrasco for

their guidance, support, and generosity, and for being possibly the best role

models for women in science that I can think of. I am extremely grateful for

everything I have learned from being a member of both labs.

To my fellow graduate students, thank you for sharing in the trials and

tribulations of academia, for inspiring me, and for making sure I never had to

drink alone.

To all the members of the Phelps and Carrasco labs, thank you for

keeping things in perspective over the years, for giving me your feedback, and

for being there day in and day out.

Lastly I would like to thank my mom for encouraging me to keep going

when thoughts of dropping everything and becoming a potato farmer clouded

my vision. I could not have done it without your love, phone calls, and

continued commitment to my success.

vi

ABSTRACT

Emotion has been shown to improve perception, and to both facilitate

and impair selective visual attention. The conjoint effect of emotion with

attention has been demonstrated across a range of tasks measuring accuracy

and response speed. Of particular interest in this dissertation are the

behavioral and neural correlates of emotion’s cost to visual attention

allocation, and the individual differences across observers that modulate the

magnitude of this effect. Costs of emotion to visual attention are assessed by

measuring decreases in (1) contrast sensitivity, a low-level visual perceptual

ability, and (2) word identification accuracy. Chapters 1 and 2 utilize a visual

psychophysics spatial cuing paradigm in which emotional or neutral face cues

direct attention prior to an orientation discrimination task dependent on

contrast sensitivity. An incongruent spatial relationship of cues and oriented

targets has previously been shown to alter contrast sensitivity. We show that

observer handedness (Chapter 1), trait anxiety and sex of the observer

(Chapter 2) also modulate this effect. Chapter 3 utilizes a variant of the

attentional blink paradigm to investigate the neural correlates of emotion’s cost

to temporal attention. Emotional distracter words disrupt processing of neutral

target words in a rapid serial visual presentation. We show that brain regions

underlying bottom-up emotional responses, such as the amygdala, may help

vii

direct attention to distracters via the orbitofrontal cortex and intraparietal

sulcus during emotional costs. Emotional costs to attention may be worsened

in individuals who engage the dorsolateral prefrontal cortex less (primarily

observers low in attentional control). As opposed to facilitative effects of

emotion to attention, costs are suggested to occur when bottom-up emotional

responses out-compete top-down attentional control mechanisms. Similar

neural circuitry may underlie emotional costs in both spatial and temporal

attention tasks.

viii

TABLE OF CONTENTS

DEDICATION…………………………………………………………………… iv

ACKNOWLEDGEMENTS……………………………………………………... v

ABSTRACT……………………………………………………………………... vi

LIST OF FIGURES…………………………………………………………….. ix

LIST OF TABLES………………………………………………………………. xi

LIST OF APPENDICES……………………………………………………….. xii

LIST OF ABBREVIATIONS…………………………………………………… xiii

INTRODUCTION………………………………………………………………. 1

CHAPTER 1…………………………………………………………………….. 28

CHAPTER 2…………………………………………………………………….. 47

CHAPTER 3…………………………………………………………………….. 86

CONCLUSION…………………………………………………………………. 128

APPENDICES………………………………………………………………….. 165

BIBLIOGRAPHY……………………………………………………………….. 179

ix

LIST OF FIGURES

Chapter 1

Fig. 1 Experiment 1 trial sequence

144

Fig. 2 (A) Contrast sensitivity data for all observers

(B) Contrast sensitivity data split by handedness group

145

Fig. 3 Contrast sensitivity data split by target visual field

146

Fig. 4 Correlation of handedness score with cue validity effect

147

Chapter 2

Fig. 5 Experiment 1 trial sequence

148

Fig. 6 Experiment 1 cueing effects: all observers

149

Fig. 7 Experiment 1 cueing effects: by anxiety and sex

150

Fig. 8 Experiment 2 trial sequence 152

x

Chapter 2 cont’d

Fig. 9 Experiment 2 cueing effects: all observers and by anxiety 153

Chapter 3

Fig. 10 (A) Trial sequence

(B) fMRI trial events

155

Fig. 11 Experiment 1 behavioral results

156

Fig. 12 Experiment 2 behavioral results

(A) Accuracy

(B) Reaction time

157

Fig. 13 IPS results

(A) Left posterior IPS

(B) Right posterior IPS

158

Fig. 14 Right DLPFC results 159

xi

LIST OF TABLES

Chapter 3

Table 1 (A) Talairach coordinates of ROIs

(B) Self-Report results

161

Table 2 Whole-brain contrasts

(A) All > Baseline

(B) Emotion > Neutral

162

Table 3 Whole-brain contrasts

(A) Late > Early

(B) Early Emotion > Early Neutral

Early Neutral > Early Emotion

(C) Late Emotion > Late Neutral

(D) Early Neutral > Late Neutral

Late Neutral > Early Neutral

163

Table 4 Whole-brain contrasts 164

(A) Early Emotion > Late Emotion

Late Emotion > Early Emotion

(B) Early Emotion Correct > Early Emotion Incorrect

Early Emotion Incorrect > Early Emotion Correct

xii

LIST OF APPENDICES

Appendix A

Study instructions for Chapter 1

165

Appendix B

Study instructions for Chapter 2

170

Appendix C

Study instructions for Chapter 3

Experiment stimuli

173

xiii

LIST OF ABBREVIATIONS

AB attentional blink

cpd cycles per degree

CS contrast sensitivity

ACS Attentional Control Scale

EHI Edinburgh Handedness Inventory

PANAS Positive and Negative Affect Scale

STAI State Trait Anxiety Inventory

fMRI functional magnetic resonance imaging

BOLD blood oxygen level dependent

ROI region of interest

rACC rostral anterior cingulate cortex

OFC orbitofrontal cortex

DLPFC dorsolateral prefrontal cortex

VLPFC ventrolateral prefrontal cortex

IPS intraparietal sulcus

1

INTRODUCTION

Our brains are extremely powerful information processors, but due to

limited skull volume they are likewise limited in size, and consequently, in

processing capacity (Lennie, 2003; Marois & Ivanoff, 2005). Both emotion and

attention can be thought of as strategies the human brain uses to selectively

filter the barrage of information our sensory receptors constantly receive. By

doing so, most brain resources can be dedicated to processing information

based upon not only our goals (top-down; e.g., writing a paper), but also upon

the features of our environment that we learn to associate with survival

(bottom-up; e.g., flashing railroad crossing lights). This efficient allocation of

brain resources allows for the fast extraction of meaning and coordination of

action necessary to survive in a dynamic world.

Humans predominantly depend on the visual system for information

gathering. Much of the research that has been conducted on emotion and

attention has thus focused on how they interactively affect visual processing.

Two ways that emotion and attention interact are to (1) improve, and (2) impair

visual processing. Their interaction can affect visual processing of both very

simple stimuli and very complex stimuli, and can occur both in space and at

different points in time. The outcome of their interaction depends on a

combination of qualities of both task-relevant and -irrelevant stimuli in the

2

external world, as well as the internal state (e.g., anxious) and stable

characteristics (e.g., sex) of an observer. These combined qualities, however,

have not been fully explored, leaving many questions unanswered and

debates unresolved within the emotion and attention literature.

A large part of the difficulty in understanding how visual processing is

affected by emotion and attention interactions is the fact that both emotion and

attention are complex, multi-faceted, psychological constructs on their own.

Within each are a number of subdivisions that have been defined by

generations of researchers, and are being further refined as time goes on. For

decades, emotion and cognitive abilities such as attention had largely been

thought of as separate, non-interacting entities. A consequence of this

independence within scientific research communities is that each field of study

has focused on different issues and has progressed at different rates.

Over the last 25 years, however, mounting evidence has increasingly

suggested that emotion and attention do interact, and their interactions can be

quite extensive across multiple experimental domains and cognitive

processes. This evidence usually comes in the form of studies measuring

reaction time or accuracy while performing a cognitive task. Emotion has been

shown to interact with attention resulting in speeded reaction time or increased

accuracy compared to a baseline condition, but in other instances (some of

3

which will be described later in the introduction) this interaction can also result

in slowed reaction time or decreased accuracy.

More recently, emotion and attention have been investigated in terms of

how they affect visual perception of low-level visual features such as contrast

or spatial frequency. While psychophysicists have made great advances over

the last 10 or more years in our understanding of how attention affects

perception, much less is known about how emotion affects perception. In fact,

at the time of this writing there are only 2 papers that have been published in

the last 5 years on emotion’s independent effects on perception (Bocanegra &

Zeelenberg, 2009; Phelps, Ling & Carrasco, 2006).

The three chapters in this dissertation describe studies investigating

how emotion and attention interact to both improve and impair visual

processing, with the first two focusing on perception of the low-level visual

feature of contrast. Given the short history of emotion and perception

research, these studies will therefore significantly contribute to our

understanding of emotion’s effects, as well as emotion and attention’s conjoint

effect, on visual processing. However, an attempt to understand the interaction

of emotion and attention (both in behavior and in the brain), and how this

impacts the way we see our world, necessitates some preliminary definitions.

Selective attention is the mechanism through which limited processing

resources are allocated to some information in the world at the same time as

4

other information is discarded. This system allows us to focus on our current

goals and minimize distractions (Broadbent, 1954; Cherry, 1953). These

benefits of attention on perception are accompanied by costs in the

unselected, or unattended, parts of the visual world (Pestilli & Carrasco, 2005).

Ignoring irrelevant information allows us to dedicate more energy to what is

important. Put simply, we see better when and where we are attending and we

see worse when and where we are not (for reviews on selective attention see

Carrasco, 2006, 2011).

Emotion is the mechanism that marks the importance or value of events

in our lives as they relate to our own interests and goals (Frijda, 1986). It

influences our learning (LeDoux 1996, 2000), memory for events

(Easterbrook, 1959; Brown & Kulik, 1977; Sharot, Delgado & Phelps, 2004),

and decision-making (e.g., Bechara et al, 1997). At one level, it is a

psychophysiological reaction to external stimuli based on our prior

experiences, and at another level it is our cognitive appraisal of events that

trigger emotional reactions. There is great variability in the range of emotions a

person can experience when faced with an external stimulus. For example,

while one person may unconsciously experience physiological signs of fear in

response to seeing a dog on the street, due to a prior negative experience

with animals, another person seeing the same dog may instead interpret

elevated heart rate as happiness or excitement.

5

As will be reviewed in the following paragraphs, emotion can modulate

attention within both spatial and temporal tasks by (1) enhancing its effects on

visual processing at attended, and (2) reducing its effects on visual processing

at unattended, locations in space and points in time, respectively.

Selective Attention and Visual Perception

Attention may be overt (directing gaze) or covert (keeping gaze fixed),

and covert attention may be deployed in a sustained or transient manner

(Nakayama & Mackeben, 1989). Sustained or endogenous attention is goal-

oriented and completely voluntary. Its effects typically peak around 300 ms

after initial attention allocation and can be sustained indefinitely. Transient or

exogenous attention, on the other hand, is stimulus-driven, occurring

automatically when a sudden change in the environment (e.g., movement or

change in contrast) draws attention to that location. The effects of exogenous

attention typically peak around 100-120 ms (Cheal & Lyon, 1991; Müller &

Rabbitt, 1989) after initial attention allocation, and rapidly decays thereafter.

Brain areas important for endogenous deployment of visual attention

include dorsal frontoparietal cortex, specifically dorsolateral prefrontal cortex

(DLPFC), anterior and posterior intraparietal sulcus (IPS), and putative human

homologue of the frontal eye field (FEF). Exogenous deployment of visual

6

attention depends on ventral frontoparietal cortex, specifically temporal

parietal junction (TPJ) and ventral frontal cortex (VFC), and is weighted more

towards the right hemisphere (Corbetta & Shulman, 2002). Many behavioral,

electrophysiological and imaging studies have shown that attending covertly to

a region in space can improve performance on visual discrimination tasks in

that location via feedback from attention-related brain areas to sensory cortex.

This feedback can either increase the gain or narrow tuning of neurons

sensitive to particular target features (for reviews see Carrasco, 2006, 2011).

Exogenous Covert Attention Modulates Contrast Sensitivity

Exogenous covert attention will be of particular importance in Chapters

1 and 2. When most people think of attention, we think of how we voluntarily

pay attention to different aspects of our environment. We move our bodies, our

heads and our eyes to better position ourselves to look at what we want to

see. However, before we can even “tell” our eyes how to move overtly, we

have to know where we want to look. This is where covert visual attention

plays an important role: it helps us select areas of interest in our visual field

without moving our eyes.

In this dissertation, covert attention will be manipulated spatially in three

ways similar to many other spatial cueing studies (e.g., Pestilli & Carrasco,

7

2005; Posner, 1980), using “valid”, “distributed” and “invalid” cues preceding a

visual discrimination task. Valid cues appear in spatial locations that draw

attention to the target or task location, and visual discrimination or detection of

that target is expected to improve. Invalid cues appear in spatial locations that

draw attention away from the target or task location, and visual discrimination

or detection of that target is expected to be impaired, consistent with the idea

that attention is a limited resource (Lennie, 2003; Pestilli & Carrasco, 2005;

Montagna, Pestilli & Carrasco, 2009). Distributed cues appear in all possible

spatial locations, spreading attention equally. In this case the cues give the

same timing information as the valid and invalid cues, but attention does not

select one location for preferential processing. Thus, there is a benefit of

attention with validly cued targets and a cost of attention with invalidly cued

targets, compared to the baseline attention condition (distributed cues). The

effects of exogenous covert attention on contrast sensitivity will be measured

using these three cuing conditions.

All visual stimuli can be composed of a collection of basic features and

contrast, like spatial frequency, motion or color, is one such low-level visual

feature. Contrast sensitivity is our ability to perceive differences between light

and dark parts of a stimulus. From electrophysiology studies in monkeys and

imaging work in humans, we know contrast is processed in primary visual

cortex, V1, which is the earliest cortical area that processes visual information.

8

It is our contrast sensitivity that, at this early stage, determines what we see

and what we do not see. We discriminate objects, for example, based on

differences in contrast that define the edges and boundaries separating an

object from the surrounding environment. Thus, perception of contrast is one

factor that underlies all of what we see, and as we learn more about contrast

sensitivity we likewise learn more about how we see complex visual objects or

scenes in our everyday life.

It has been previously shown that contrast sensitivity increases at

attended, and decreases at unattended, spatial locations (Pestilli & Carrasco,

2005; Pestilli, Viera & Carrasco, 2007), as if the physical contrast of a stimulus

increase or decreased, respectively. These benefits and costs of attention to

contrast sensitivity have also been found to correlate with changes in striate

and extrastriate visual cortical regions (Liu, Pestilli & Carrasco, 2006). To

investigate these changes in contrast sensitivity, observers participate in an

orientation discrimination task in which exogenous attention cues precede a

number of Gabor stimuli (sinusoidal gratings convolved with a Gaussian

envelope). One of these stimuli is a target while the rest are distracters, and

observers must discriminate the tilt of the target Gabor. On each trial, the

Gabor contrast varies systematically from very low to very high in several log

steps. Performance on the orientation discrimination task is expected to

increase with rising stimulus contrast. Thus, on trials with the lowest contrast

9

Gabors, performance is expected to be at chance, while on trials with highest

contrast Gabors, performance is expected to asymptote.

Gabors are artificial stimuli well suited for use in visual psychophysics

experiments where performance is measured as a function of stimulus

intensity. Although we never come across Gabors in our everyday lives

(unless we are participating in a vision experiment), by using a stimulus that

can be composed of single dimensions of basic visual features, we can learn

about the functioning of the visual system in a systematic manner. For

example, a Gabor is composed of a single spatial frequency, a single contrast

level, a single orientation and a single size. In addition, each of these

dimensions can be modified independently of all the other dimensions, making

it an ideal stimulus to investigate how our visual systems process each of

these basic features. To investigate the effects of exogenous covert attention

on contrast sensitivity, the studies described in the first two chapters will keep

spatial frequency, orientation and size constant while changing contrast from

trial to trial. Performance on an orientation discrimination task that depends on

contrast sensitivity is then measured for each contrast.

Psychometric (e.g., Weibull) functions, which relate observer

performance as a function of stimulus intensity, are fitted to the performance

data across the tested contrast levels. Contrast threshold is then calculated

based on the function fit. This is the contrast needed to perform at a given

10

level of accuracy, and it serves as a measure of contrast sensitivity in our

visual system. It provides a quantitative summary of all the data points

collected in a psychophysics experiment, and allows researchers to assess

differences in sensitivity due to condition at any stimulus contrast level.

Relative to averaging performance over all stimulus intensities and comparing

those averages across conditions, calculating contrast threshold provides a

more precise measurement of perceptual ability and provides information

about how this perceptual ability changes as a function of stimulus intensity. In

addition, there is considerable variation in contrast sensitivity across a subject

population. For example, one person may perform correctly 75% of the time

with a stimulus contrast of 5% whereas another person may need 10%

contrast to perform at the same level. By fitting psychometric functions to data

from individual subjects, we are able to capture and account for these

differences.

The psychophysical task and method of analysis described in the

preceding paragraphs are valuable tools that allow researchers to measure

contrast sensitivity at a given performance level at both attended and

unattended locations, within a single individual. We use these tools to

investigate the effects of exogenous attention cues on contrast sensitivity in

Chapters 1 and 2.

11

The Attentional Blink

In addition to investigating spatial attention in Chapters 1 and 2,

Chapter 3 will investigate the temporal limits of attention using the attentional

blink (AB; Raymond, Shapiro & Arnell, 1992) paradigm. Using the AB

paradigm, one can control which stimuli are accessible to awareness by

inserting task-relevant word(s) or picture(s) into a rapid serial visual

presentation (RSVP) stream of distracter stimuli. Each stimulus is on the

screen for a brief period of time (e.g., 90 ms) so that individual stimuli may not

be fully processed before they are replaced. In a typical AB experiment, two

targets T1 and T2 are inserted in a RSVP stream with a variable number of

intervening distracters. Zero intervening distracters is called “Lag 1”, whereas

six intervening distracters is called “Lag 7”. The observer’s task is to identify

both targets. At short lags, there is a robust decrement in performance in

identifying T2 contingent on correct T1 identification. This decrement lasts for

500 ms post-T1. At longer lags, performance on T2 identification recovers.

The AB shows that there is a brief window of time in which attentional

processing is at capacity, such that only one stimulus may be attended and

encoded at a time.

A number of models have been put forward to explain the AB effect (for

a review see Dux & Marois, 2009). One of the more commonly cited is the

two-stage model proposed by Chun and Potter (1995). In the first stage of

12

processing, stimuli are rapidly recognized and monitored for relevant features

according to the task. In the second stage, this information is encoded and

consolidated into working memory. However, the second stage is capacity-

limited so new information cannot gain entry until the encoding of old

information is finished. The AB is thought to be due to T2 items’ inability to

enter the second stage of processing because T1 is still being encoded. If T2

does not enter Stage 2 processing, its representation in Stage 1 rapidly

decays and is overwritten by flanking distracters. The result is as though one

never saw T2 at all – the AB.

While psychophysics specifically measures performance as a function

of the intensity of an elementary visual feature, such as stimulus contrast, the

AB measures one’s ability to identify words (or pictures), which requires

higher-level computation. Although these two tasks may have different

underlying mechanisms, emotionally salient cues can modify the effects of

attention on visibility in both tasks. This suggests that emotion can modulate

attention at multiple levels of analysis, and can be thought of as a general

strategy used by the brain to prioritize information processing. Throughout this

dissertation, we will show that this is the case, both at the lower level of visual

perception (Chapters 1 and 2) and at the higher level of word identification

(Chapter 3).

13

Emotion Modulates Attention and Perception

Emotion-laden stimuli, and inherently neutral stimuli that become

associated with emotions, are preferentially processed in terms of greater

speed and depth. This is especially true of threat-related stimuli that evoke

emotions such as fear and anger, which are negative in valence and high in

arousal. Fast and exhaustive processing of threat is crucial to an organism’s

survival (LeDoux, 1996), and this is manifested in more efficient use of

resources such as attention (e.g., Notebaert, Crombez, Van Damme, De

Houwer & Theeuwes, 2011; Ohman 2009). Indeed, it has been shown that

emotion’s impact on cognition starts early on in the information-processing

stream with perception and attention (for reviews see Compton, 2003; Stanley,

Ferneyhough & Phelps, 2009).

Emotionally salient stimuli have been shown to modulate perception in

two ways. Emotion can (1) enhance contrast sensitivity (Phelps et al., 2006)

and tilt detection of low spatial frequency stimuli (Bocanegra & Zeelenberg,

2009), or (2) impair tilt detection of high spatial frequency stimuli (Bocanegra &

Zeelenberg, 2009). Furthermore, emotional stimuli can modulate spatial

attention by (1) facilitating the benefits of attention on contrast sensitivity

(Phelps et al., 2006), performance or reaction time (e.g., Koster, Crombez,

Van Damme, Verschuere & De Houwer, 2004), or by (2) capturing attention

and drawing it away from goal-relevant tasks with the result of impaired

14

performance and slowed reaction time (e.g., Fox, Russo, Bowles & Dutton,

2001). What has yet to be shown, however, is whether the interaction of

emotion with attention impairs contrast sensitivity at unattended locations. If it

can, then this means the emotional significance of an object in the world can

not only improve our perception of that and nearby objects, but also impair our

perception of the surrounding environment. Importantly, this detrimental effect

may go beyond the impairment of attention due to non-emotional distractions.

This has real-world implications for things such as highway billboard design,

which in recent years has trended towards more dynamic imagery at the cost

of increasingly more inattentive drivers. The possibility that emotion and

attention will conjointly impair contrast sensitivity at unattended locations will

be tested in Chapters 1 and 2 using visual psychophysics.

Emotion can also modulate attention in temporal tasks such as the AB,

to make target stimuli more, or less, visible. Anderson and Phelps (2001) used

this paradigm with neutral and emotional T2 word stimuli and found that the

AB was attenuated with emotionally arousing compared to low arousal, neutral

T2. This suggested that emotional stimuli have preferential access to the

second stage of Chun and Potter’s (2005) two-stage model of attention.

As mentioned previously, the two-stage model of attention states that a

short-duration initial stage first detects target stimuli before a capacity-limited

second stage takes over to provide more in-depth perceptual and memory

15

encoding. In the traditional AB task, T1 will progress through stage 1 and 2,

but while being processed in stage 2, items that appear after T1 (such as T2)

may be lost due to rapid stage 1 decay. However, it may be the case that

emotionally arousing T2s are able to enter stage 2 processing more easily

than neutral T2. This is supported by data showing that neither bilateral nor left

amygdala patients experienced emotional facilitation in the AB, further

suggesting that the facilitation effect is dependent on the left amygdala. Thus,

the amygdala, a medial temporal lobe structure important for emotional

learning, may have a role in modulating perception of emotional stimuli.

In addition to emotion benefiting attentional engagement with emotional

targets, emotion can impair attentional engagement with neutral targets in the

AB. Modified AB paradigms including a single neutral target preceded by a

neutral or emotional distracter have found that target identification is impaired

with task-irrelevant emotional distracters within the AB window (Arnell, Killman

& Fijavs, 2007; Mathewson, Arnell & Mansfield, 2008; Most, Chun, Widders &

Zald, 2005). The effect seems to depend both on the arousal and valence of

the target stimuli with highly arousing negative stimuli producing the largest

effects (Jefferies, Smilek, Eich & Enns, 2008; Keil & Ihssen, 2004; Most,

Smith, Cooter, Levy & Zald, 2007). Top-down attention towards goal-relevant

stimuli appears to be disrupted by a bottom-up, emotionally modulated

attention component. This results in greater attentional resources being

16

diverted towards emotional, yet task-irrelevant, stimuli in the temporal domain,

leaving insufficient resources for task-relevant target processing.

While the amygdala has been shown to have a crucial role in emotion’s

facilitative effect in the AB, the neural correlates of emotion’s detrimental effect

in this modified AB have not been fully explored. Emotional distractions are

harder to ignore than neutral ones, but for many people they may be nearly

impossible to ignore. This inability to filter out emotional distractions could

indicate under recruitment of brain regions that provide top-down attentional

control, which have previously been associated with vulnerability to anxiety

disorder (e.g., Bishop 2007).

There may also be, however, an interaction between goal-oriented and

stimulus-driven systems of selective attention within the context of this

modified AB task. Goal-oriented attention, for example, can help form an

attentional set which effectively sensitizes stimulus-driven attention to task-

relevant items (e.g., particular target words); however, an attentional set can

also be disrupted if stimulus-driven attention is drawn to an irrelevant location

by a highly salient (i.e., emotional) stimulus. Chapter 3 will address the

behavioral and neural costs of emotion to attention in healthy undergraduates

with functional magnetic resonance imaging (fMRI) using a modified AB task.

17

A Note on Fear Face Stimuli

There is a long and well-established history of research on fear in

animals and in humans. Whereas fear is an emotion that consistently elicits

increased autonomic responses such as freezing in rats, and skin

conductance in humans, it has not been established whether other emotions

such as happiness or anger can be studied in animals. Therefore, parallels

between the human experience of these other emotions with animal models

cannot be drawn. Due to the fact that the neural circuitry of fear has been well-

characterized, we use face stimuli with fearful expressions as spatial cues to

manipulate both exogenous attention and emotional salience in the visual

psychophysics experiments in Chapters 1 and 2. We specifically chose face

stimuli over other possible choices for a variety of reasons, ranging from the

theoretical to the practical.

Humans are a social species and depend on multiple modes of non-

verbal communication. Faces and their underlying musculature have evolved

to express a vast array of emotional states, and this meaningful information

can be conveyed in a short amount of time within the limits of exogenous

attention (e.g., Eimer & Holmes, 2002; Phelps et al., 2006). A fearful face

signals to an observer that threat is present in the environment and that they

should be fearful as well. This fear response may serve to mobilize the

observer’s attentional resources in order to both collect more information and

18

act on that information. The sudden appearance of the face in the visual field

may initially direct exogenous attention to its location for the purpose of

collecting more information about the possible threat; for example, it is likely

that the threat is near the person making the fearful expression. After this

initial attentional process the observer can decide how to react, such as

whether to avoid or engage the threat. All of this occurs before the observer is

able to cognitively evaluate the situation (which can take hundreds of

milliseconds). Thus, we associate fearful expressions with the presence of

threat because this is an evolutionarily beneficial trait that has increased the

chances of our species’ survival.

It can be argued that fearful facial expressions provide a relatively weak

fear signal relative to real threats, such as electric shocks, or even relative to

pictures of evolutionarily “prepared” stimuli that could pose a real threat if we

were to actually face them (e.g., snakes, spiders or tigers). Given that the

studies in this dissertation tests the effects of emotion and attention on

perception using psychophysics methodology (requiring large numbers of

trials), it is not practical or safe to induce fear responses in humans using

shock. Fortunately for our research, it has been shown that seeing another

person’s fearful reaction to threat can induce the same reaction in the

observer without having to subject the observer to the same threat. This is

observational fear learning (for a review: Olsson & Phelps, 2007). Since fearful

19

expressions have been shown to reliably induce a fear response (Olsson,

Nearing & Phelps, 2007; Vaughan & Lanzetta, 1980) using a more prepared

stimulus such as a tiger is not necessary.

Face stimuli have other practical benefits, besides being safe to use

over many hundreds or thousands of trials, that have more to do with the

experimental design. To ensure that any effects of the fearful faces are due to

their emotional expression rather than the fact that they are a face, faces with

neutral expressions provide a natural control. Both fearful and neutral

expressions can be made by the same face, which also controls for other low-

level factors such as coloring, size or shape. Contrast and luminance can also

be equated more easily across a set of face stimuli compared to a set of

stimuli composed of animals of more extreme differences in these features.

These low-level visual differences matter in experiments investigating

exogenous attention because visual salience alone can alter how attention is

deployed. It is also unclear what would serve as a good neutral condition in

the case that, for example, a tiger is used as an emotionally salient cue. While

other striped animals could conceivably be used, this choice lacks the

advantage that faces have – fearful and neutral faces have largely the same

low-level visual characteristics and differ only in emotional valence.

Lastly, face recognition has been studied extensively. Faces have been

shown to modulate activity of specific regions in ventral visual cortex

20

(Kanwisher et al., 1997) and the amygdala (Adolphs, Tranel, Damasio &

Damasio, 1994; Morris, Frith, Perrett, Rowland, Young, Calder & Dolan, 1996).

Attention to faces has also been shown to modulate activity in these emotion-

and face-sensitive regions, and regions underlying shifts of attention (Armony

& Dolan, 2002; Pourtois et al., 2006; Vuilleumier et al., 2003, 2004). By using

stimuli that are commonly used in studies of emotion and attention, predictions

can be made more reliably relative to other stimuli that are not as well-studied.

Individual Differences that Modulate Emotion and Attention Interactions

External stimulus characteristics such as emotional significance or

perceptual salience can modify attention’s effects on perception. What is

relatively understudied are the stable internal characteristics, or the current

state, of individuals that can modulate how emotion and attention interact to

change perception. All of the studies described in this dissertation investigate

individual differences, which will be described below. Chapter 1 has a special

focus on handedness, while Chapter 2 focuses on anxiety and sex. In

Chapter 3 we explore effects of both anxiety and attentional control.

There is a growing literature investigating how subclinically anxious

individuals selectively allocate their attention, especially in cases of potential

threat. In theories of attentional control, anxious individuals are influenced

21

more by bottom-up or stimulus-driven attention, due to an impairment of their

goal-directed attentional system (Eysenck, Derakshan, Santos & Calvo, 2007).

Indeed, attentional control has been found to be inversely related to trait

anxiety (Derryberry & Reed, 2002).

An AB study found that low anxious subjects were better able to avoid

the detrimental effects of emotional distracters on the target discrimination

task than high anxious subjects when they exerted attentional control (Most et

al, 2005). Similarly, Fox and other researchers have used spatial cuing

paradigms to demonstrate that individuals with heightened state and trait

anxiety have difficulty disengaging from threatening stimuli such as negative

emotional words, pictures of angry or fearful faces, or fear-conditioned stimuli,

compared to individuals with low trait anxiety (Fox et al., 2001; Fox 2002;

Koster, Crombez, Verschuere & De Houwer, 2006; Mogg & Bradley, 1999;

Smith, Most, Newsome & Zald, 2006; Yiend & Mathews, 2001). The result is

that performance of highly anxious individuals is impaired in experiment

conditions that require a disengagement and shift of spatial attention from

threat-related stimuli, while individuals with a greater degree of attentional

control are more successful at these tasks.

Studies have also shown an interaction of anxiety with sex. High

anxious females, for example, have greater amygdala activation in response

to unattended fear faces compared to high anxious males (Dickie & Armony,

22

2008). This finding may also be related to the findings showing females are

better at recognizing faces and facial affect than males (McClure, 2000;

Thayer & Johnsen, 2000). Better recognition of fearful expressions on faces

could make them more salient, thereby enhancing the emotional significance

these faces have to the observer.

Individual differences in degree of handedness may affect the

interaction of emotion and attention as well. Previous research has

demonstrated the dominance of the right hemisphere in spatial attention in

right-handers (e.g., Mesulam, 1999). There have also been reports of greater

right hemisphere representation for face (Yovel, Tambini & Brandman, 2008)

and emotion processing (Bourne, 2008). In contrast, less is known about left-

hander functional lateralization, other than it is more inconsistent. Greater

inter-subject variability of left-hander behavior in attention and face perception

tasks (e.g., Bourne, 2008; Dronkers & Knight, 1989; Luh, Redl & Levy, 1994)

has been demonstrated, providing evidence that left-handers and right-

handers may differ in regards to cerebral lateralization of these functions.

As previously mentioned, throughout Chapters 1-3 we will investigate

effects of stable individual characteristics and current states of the observer on

the interaction of emotion with attention. We recruit equal numbers of left- and

right-handers in Chapter 1, and males and females with wide-ranging self-

reported anxiety in Chapter 2 and compare the interactive effects of emotion

23

and attention on their contrast sensitivity scores. For our imaging study of

emotional costs in Chapter 3, all participants provide us with anxiety and

attentional control self-report measures after completing our AB task so we

can investigate how error rate and brain activity changes relate to these

personality traits.

Neural Correlates of Emotion’s Effects on Attention and Perception

There are several neural routes through which emotion may exert its

influence on attention and perception. While they are not mutually exclusive,

the evidence cited to form the basis of these routes have been debated in the

literature (e.g., Bach, Talmi, Hurlemann, Patin & Dolan, 2011; Pessoa 2010;

Pessoa & Adolphs, 2010).

Representations of emotion-laden stimuli or other objects in the vicinity

of emotional stimuli may be boosted in retinotopic visual cortex by feedback

from the amygdala. This is supported by ERP and fMRI research that has

shown enhanced striate and extra-striate activity to emotional compared to

neutral stimuli (e.g., Pourtois, Grandjean, Sander & Vuilleumier, 2004;

Schupp, Markus, Weike & Hamm, 2003; Armony & Dolan, 2002), and this

enhancement is inversely correlated with degree of amygdala sclerosis

(Vuilleumier, Richardson, Armoney, Driver & Dolan, 2004). While these

24

studies provide support for the amygdala feedback hypothesis, they cannot

differentiate among the following three routes. (1) The amygdala receives

feedforward projections from temporal visual cortex, and has feedback

projections throughout ventral visual cortex including area V1 (Amaral,

Behniea & Kelly, 2003). (2) The amygdala receives fast, low-resolution

information regarding the significance of visual stimuli from the superior

colliculus and pulvinar thalamus (Morris, deGelder, Weiskrantz & Dolan,

2001), which then feeds back to V1, bypassing cortex (Adolphs, 2002; LeDoux

2002). (3) In addition to direct feedback from the amgydala, V1 may receive

indirect feedback via frontoparietal attention areas.

Ventral prefrontal cortex (VPFC) has reciprocal connections with the

amygdala (Barbas, 2000) and can regulate its activity in response to emotional

stimuli as well as influence responses of the IPS (Kelly, Uddin, Shehzad,

Margulies, Castellanos, Milham & Petrides, 2010; Taylor & Fragopanagos,

2005). IPS can then feed back to visual cortical areas specific to sensory

processing of the stimuli (e.g., Vuilleumier, Schwartz, Verdon, Maravita,

Hutton, Husain & Driver, 2008).

The neural correlates of emotion’s cost may be, in large part, based on

the correlates of emotion’s benefit, however there are a few important

differences unique to costs. Recent imaging work has revealed that bilateral

IPS activity is reduced in response to contralateral targets cued by invalid fear

25

faces (Pourtois, Schwartz, Seghier, Lazeyras & Vuilleumier, 2006). However,

IPS activity has also been shown to increase in trials with either valid or invalid

fear face cues (Armony & Dolan, 2002). This discrepancy may be explained by

the fact that in the first study, target-related IPS activity was assessed,

whereas in the second study, IPS activity was in response to the whole trial

including cue and target. This suggests that the fearful faces lead to increased

transient focusing of attention to their location, and is evidence for the notion

that emotional cues can capture and hold attention producing a cost when

targets are presented in unattended locations. In addition, both studies report

increased orbitofrontal cortex (OFC) activity in response to fear-invalid trials.

OFC may be important in cases where attention has been involuntarily

captured, or in “breaches of expectation”, by an emotional stimulus. OFC may

redirect attention via top-down signals to IPS (Armony & Dolan, 2002; Nobre,

Coull, Frith & Mesulam, 1999; Pourtois et al, 2006). It has also been

hypothesized that the rostral anterior cingulate cortex (rACC), which receives

input from emotion-sensitive ventral striatum, has a role in gating awareness

to potentially threatening stimuli and may regulate amygdala response in order

to resolve affective interference (DeMartino, Kalisch, Rees & Dolan, 2009;

Most, Chun, Johnson & Kiehl, 2006).

Chapter 3 investigates costs of emotion to attention within the context

of a modified attentional blink task. In addition to the amygdala, based on prior

26

research we are interested in the roles of IPS, OFC, rACC and DLPFC,

regions involved in both bottom-up and top-down attention processes.

Benefits and Costs of Emotion to Visual Attention

The following three chapters explore how external stimulus factors,

stable internal characteristics and current states of individual observers can

modulate the way emotion incurs both benefits and costs to visual attention.

Chapter 1 shows that contrast sensitivity is modulated by exogenous attention

cued by pictures of faces, but this effect depends on observer handedness.

We also find that facial expression effects depend on a specific range of

spatial frequencies. Chapter 2 revisits contrast sensitivity and finds that its

modulation by face cues is further influenced by individual differences in trait

anxiety and sex of the observer. Chapter 3 investigates the neural correlates

of emotional costs to attention in the attentional blink paradigm, revealing a

frontoparietal network of brain areas important in this interaction, as well as

specific regions whose activity is modulated by individual differences in anxiety

and attentional control.

The study of how emotion and attention interact at the perceptual level

has important implications for many cognitive processes that occur

downstream. The particular way we remember an event, make a decision, or

27

perform some action, for example, depends on how we see the world around

us. How we see is intimately linked to individual differences in our selective

attention processes, the emotional significance that each object in our

environment has for us, and the interactions among attention, emotion and

perception.

28

CHAPTER 1

Cueing Effects of Faces are Dependent on Handedness and Visual Field

(2010, Psychonomic Bulletin & Review, Vol. 17, Issue 4, p. 529-535)

Emma Ferneyhough1, Damian A. Stanley1, Elizabeth A. Phelps1, 2 & Marisa

Carrasco1, 2

1New York University Psychology Department

2New York University Center for Neural Science

29

Abstract

Faces are unlike other visual objects we encounter, alerting us to

potentially relevant social information. Both face processing and spatial

attention are dominant in the right hemisphere of the human brain, with a

stronger lateralization in right- than left-handers. Here we demonstrate

behavioral evidence for an effect of handedness on performance in tasks

using faces to direct attention. Non-predictive, peripheral cues (faces or dots)

directed exogenous attention to contrast-varying stimuli (Gabor patches) – a

tilted target, a vertical distracter, or both; observers made orientation

discriminations on the target stimuli. Whereas cueing with dots increased

contrast sensitivity in both groups, cueing with faces increased contrast

sensitivity in right- but not left-handers, for whom opposite hemifield effects

resulted in no net increase. Our results reveal that attention modulation by

face cues critically depends on handedness and visual hemifield. These

previously unreported interactions suggest that such lateralized systems may

be functionally connected.

30

Acknowledgements

We thank Sam Ling and David Carmel, as well as other Carrasco Lab

members, for their helpful comments. This research was funded by grants NIH

R01-EY016200 to M.C. and NIH R01-MH062104 to E.A.P.

31

INTRODUCTION

Faces are special visual objects that we encounter every day. Not only

are they complex and ever-changing, they are a portal into the thoughts and

intentions of others, providing information necessary for navigating our

dynamic social world. Perhaps for these reasons, we are particularly

responsive to faces; we rapidly evaluate them (Haxby, Hoffman & Gobbini,

2002) and use them to make predictions of social outcomes (Oosterhof &

Todorov, 2008). Furthermore, faces have the ability to automatically draw our

attention, more so when they depict a fearful rather than neutral expression

(Phelps, Ling & Carrasco, 2006). This ability is particularly important because

it is one of the first steps necessary to begin the process of evaluation and

prediction formation in our chaotic visual world.

Selective attention can be deployed covertly (without eye movements)

to a region in space and improve performance on visual discrimination tasks in

that location (Carrasco, 2006; Kinchla, 1992). This is true whether attention is

deployed voluntarily (endogenously), or driven involuntarily by a transient

change in the visual field (exogenously) (Nakayama & Mackeben, 1989).

Typically, psychophysicists use peripheral cues consisting of dots or bars to

direct exogenous attention. When cued with dots or bars, not only does

exogenous attention improve performance at cued locations (Carrasco,

32

Penpeci-Talgar & Eckstein, 2000; Ling & Carrasco, 2006a), it also impairs

performance at uncued locations. This is true even though the cues are

uninformative (i.e., they do not predict the target location) and observers are

explicitly told that this is the case (Montagna, Pestilli & Carrasco, 2009; Pestilli

& Carrasco, 2005). These trade-offs in performance have been interpreted as

resulting from the allocation of limited resources. Faces are effective as

exogenous cues and can reflexively draw attention to task-relevant locations,

perhaps because of their ecological validity and social value (Phelps et al.,

2006). However, it is unknown whether there is a corresponding cost, as with

dot cues, at irrelevant locations. How do face cues modulate the benefits and

costs of attention, at attended and unattended locations respectively?

Based on the finding that face cues result in greater attentional benefit

when they depict fearful than neutral expressions (Phelps et al., 2006), in a

pilot experiment we tested if we would find both differential benefits (at cued

locations) and costs (at uncued locations) for fearful and neutral faces.

Although we found no effect of emotion (see Discussion), we did discover an

intriguing pattern of results mediated by handedness: for left-handers the

cueing effect depended on the location of the target in the visual field.

Interestingly, lesion and imaging studies with right-handers have

revealed that face perception and visuospatial attention are hemispherically

lateralized. Face recognition is a specialized process of the right hemisphere

33

(Luh, Redl & Levy, 1994). Consistent with this finding, people are better at

recognizing faces in the left than right visual field (LVF; RVF) (Rhodes, 1985).

Greater face-related activity in the right than left fusiform face area, as

assessed by EEG and fMRI, is thought to underlie this LVF advantage (Yovel,

Levy, Grabowecky & Paller, 2003; Yovel, Tambini & Brandman 2008).

Visuospatial attention is also associated with greater activity in the right

hemisphere (Siman-Tov, Mendelsohn, Schonberg, Avidan, Podlipsky et al.,

2007), with attention benefiting detection (Fecteau, Enns & Kingstone, 2000)

and discrimination (Evert, McGlinchey-Berroth, Verfaellie & Milberg, 2003)

tasks more in the LVF than RVF. Correspondingly, more severe attention

deficits result from lesions to the right- than left- parietal lobe (Mesulam, 1999).

As a group, compared to right-handers, left-handers show more inter-subject

variability in these lateralized brain functions (e.g., Dronkers & Knight, 1989;

Luh et al., 1994).

It is unknown whether brain lateralization differences observed in right-

and left-handers could lead them to exhibit different behavior in experiments

that tap into the lateralized functions of face processing and covert attention.

Hence, in the present study, using an exogenous attention procedure (Pestilli

& Carrasco, 2005), we systematically investigated how the interaction of

handedness and attention cued with faces (Experiment 1) or with dots

(Experiment 2) affects visual performance.

34

METHODS

Experiment 1: Effects of faces as exogenous cues

Participants

Six right-handed (2 males, 20-34 years, M=26.8) and six left-handed (1

male, 24-31 years, M=27.5) observers participated. All had normal or

corrected-to-normal vision and completed the 10-item Edinburgh Handedness

Inventory (Oldfield, 1971). A score of +100 on the inventory indicates complete

right-hand dominance whereas a score of -100 indicates complete left-hand

dominance. Right-handers scored +78 (SD=21) and left-handers scored -83

(SD=14) on average.

Apparatus

Stimuli were presented on a 21” ViewSonic P220f monitor (1600x1200

pixels; 75 Hz) connected to a Power Macintosh G4 computer via an

attenuator. Background luminance was set to 16.5 cd/m2. During the

experiment participants’ heads were stabilized using a chin rest 57 cm from

the monitor.

35

Stimuli

Face stimuli consisted of 22 contrast- and luminance-equated grayscale

pictures of fearful and neutral faces from the Pictures of Facial Affect series

(Ekman & Friesen, 1976). Gabor patches (sinusoidal gratings in a Gaussian

envelope, SD=1˚; 4 cpd) were created using Matlab 5.2.1 and the

Psychophysics Toolbox (Brainard, 1997). The face cues subtended 4x5.3˚,

and were centered 5˚ horizontally and 2.65˚ above fixation. The Gabor

patches subtended 6x6˚ and were centered 5˚ horizontally and 4˚ below

fixation. Gabor patch contrast ranged from 3.4% to 56.7% in 7 log steps.

Gabor tilt ranged from 3 to 6˚, chosen for each observer individually based on

a ~62.5%-correct criterion in pretesting.

Procedure

Observers were seated in a darkened room. On each trial, they fixated

a central cross for 500 ms; then a face cue was presented to the left, right or

on both sides above fixation for 80 ms to manipulate exogenous attention;

following a 53 ms ISI, one tilted (the target) and one vertical Gabor patch were

presented, one on either side below fixation, for 40 ms. Participants indicated

the target location (left or right) and orientation (counterclockwise or

clockwise), with a single button press (Figure 1). Feedback was given after

each trial by a high tone for correct and a low tone for incorrect responses.

36

Cues appeared on the same side as targets (Valid), the opposite side (Invalid),

and on both sides (Distributed) with equal probability (1/3). Observers

completed 3,340 trials on average. See Appendix A for detailed task

instructions.

Analysis

For each condition, we calculated percent correct as a function of

contrast. Psychometric functions were fitted using psignifit 2.5.6 (Weibull;

http://bootstrap-software.org/psignifit/; Wichmann & Hill, 2001). Contrast

threshold was indexed by the stimulus intensity at which observers were

correct 67% of the time (about halfway between chance, 25%, and perfect

performance, 100%). The primary dependent variable was contrast sensitivity

(CS), which is inverse contrast threshold. Observers’ CS scores were

individually normalized by dividing each condition mean by the average of all

conditions to reduce the influence of baseline CS differences across

observers. Normalized CS scores were then averaged across observers in

each handedness group. Reaction times (RTs) were also measured.

Experiment 2: Effects of dots as exogenous cues (Control)

All experimental parameters for Experiment 2 were the same as for

Experiment 1 except for the following: (1) five out of six observers from each

37

handedness group in Experiment 1 participated in Experiment 2; (2) black dot

cues (0.3˚ diameter, 1˚ above and 5˚ horizontally from fixation) were used

instead of face cues; (3) Gabor targets were always tilted ± 4˚; and (4)

observers completed 4,000 trials on average.

RESULTS

Given that there were no differences in performance or RT between

facial expression conditions (fearful vs. neutral face cues, p>0.1), the data

were averaged across both expressions. Here we report detailed statistics for

CS, and note that the RT analyses showed no speed-accuracy trade-offs for

any comparison. For each experiment, there are two within-subject factors:

cue validity (Valid, Distributed, Invalid), and visual field (LVF, RVF); there is

also one between-subject factor: handedness (Left, Right).

To determine whether cue validity interacted with handedness and

visual field and whether this interaction depended on cue type, three-way

mixed factorial ANOVAs were performed for face and dot data separately with

cue validity, visual field, and handedness as factors. There was a significant

interaction of the three factors for faces (F(2,20)=14.349, p<0.001), but not for

dots (F(2,16)=1.652, p>0.10). To better understand how faces are different

from dots, we first present an analysis of the effect of cue type alone. Then,

38

because our pilot study indicated left-hander performance depended on the

target location in the visual field, we investigate the effect of handedness, as

well as the effect of visual field. Lastly, we examine the relation between

degree of handedness and cue validity effect.

Effects of Cue Type

To evaluate the effect of cue type, CS was averaged over both visual

fields and handedness conditions separately for face and dot cue data (Figure

2A). One-way repeated measures ANOVAs performed on cue validity (Valid,

Distributed, or Invalid) indicated that it changed CS marginally when faces

were used (F(2,22)=3.091, p=0.066, !2RM=0.22); however it changed CS

significantly when dots were used (F(2,18)=10.663, p<0.001, !2RM=0.54). The

results replicated previous findings for dots (Carrasco et al., 2000; Ling &

Carrasco, 2006; Pestilli & Carrasco, 2005), but not for faces (Phelps et al.,

2006). Faces decrease the magnitude of the cue validity effect but, at this

point in the analysis, it is unclear why this is the case.

Effects of Handedness

We then split the data to evaluate the effect of handedness. When right-

handers were cued with either faces or dots, cue validity significantly changed

39

CS (face: F(2,10)=5.992, p<0.02, !2RM=0.55; dot: F(2,8)=11.838, p<0.005,

!2RM=0.75; Figure 2B). Although when left-handers were cued with dots cue

validity significantly changed CS (F(2,8)=5.904, p<0.05, !2RM=0.6), this was

not the case when they were cued with faces, (F(2,10)<1). The decreased

magnitude of the initial face-cue validity effect across handedness appears to

be due to the lack of CS modulation in left-handers when cued with faces.

Effects of Handedness and Visual Field

Next, to evaluate the effect of visual field, the data were split based on

whether the target appeared on the left or right side of the screen. Within each

handedness group, two-way repeated measures ANOVAs were performed on

target visual field (LVF, RVF) and cue validity (Valid, Distributed, Invalid)

separately for face and dot cues. Left-hander face data revealed a significant

interaction between VF and cue validity (F(2,10)=6.519, p<0.02; Figure 3A).

When targets were in the LVF, valid-face cues resulted in the highest CS,

followed by distributed and invalid cues (Ms = 1.15, 1.10, and 1.01,

respectively). However, a different pattern was found when targets were in the

RVF: invalid cues led to the highest CS, followed by distributed and valid cues

(Ms = 0.97, 0.90, and 0.86, respectively). In contrast, left-hander dot data

revealed a main effect of cue validity (F(2,8)=5.905, p<0.05) with valid-dot

40

cues leading to the highest CS followed by distributed and invalid cues (Ms =

1.12, 1.02, and 0.86, respectively). VF and cue validity did not significantly

interact (F(2,8)=1.699, p>0.10).

Right-hander face data revealed a significant interaction of VF and cue

validity (F(2,10)=7.93, p<0.01; Figure 3B): CS was higher in the RVF than

LVF, and there were greater differences in CS due to cue validity in the RVF

than LVF (Valid–Invalid CS = 0.5 and 0.18, respectively). Right-hander dot

data revealed a main effect of VF (F(2,8)=18.954, p<0.02) with CS being

higher in the RVF than LVF (Ms = 1.18 and 0.82, respectively). There was

also a main effect of cue validity (F(2,8)=11.837, p<0.005) with valid cues

resulting in the highest CS, followed by distributed and invalid cues (Ms =

1.52, 0.98 and 0.5, respectively). VF and cue validity did not significantly

interact (F(2,8)=1.5, p>0.10).

Lastly, we examined the correlation between each individual’s cue

validity effect (Valid–Invalid CS) and their handedness score (Figure 4).

These two indices were positively and significantly correlated when targets

appeared in the RVF with both face (R2=0.56, p<0.01) and dot (R2=0.45,

p<0.05) cues; however, it appears that these correlations may each be due to

different underlying mechanisms (see Discussion). When targets appeared in

the LVF, this effect was marginal with dot cues (R2=0.38, p=0.058) but no

such correlation emerged with face cues (R2=0.03, p>0.1).

41

DISCUSSION

Does covert attention evoked by face and dot cues have comparable

benefits and costs on contrast sensitivity? Critically, the answer to this

question depends on the observer’s handedness. For right-handers, both

faces and dots are effective at eliciting attention, resulting in a benefit at cued

and cost at uncued locations (Figure 2B, third and fourth triplet from L to R).

Conversely, for left-handers, faces and dots elicit attention differentially:

whereas dot cues result in enhanced CS with attention, face cues have a

different pattern in each hemifield. When faces cued LVF targets, valid cues

increased CS, and invalid cues decreased CS, relative to distributed cues.

However, when faces cued RVF targets, valid cues decreased CS and invalid

cues increased CS relative to distributed cues (Figure 3A, first and third triplet

from L to R). Consequently, averaging over both hemifields resulted in no net

effect of cue validity in left-handers (Figure 2B, first triplet on left).

Previous studies have shown that when exogenous attention is

manipulated via dot or bar cues, it elicits attentional benefits and costs at cued

and uncued locations, respectively (Carrasco et al., 2000; Montagna et al.,

2009; Pestilli & Carrasco, 2005), and that face cues elicit attentional benefits

at cued locations (Phelps et al., 2006). The present study replicated previous

dot cue findings for all observers, and revealed that, for right-handers, the

42

benefits of face cues were accompanied by costs at the uncued locations.

These findings support selective attention’s role in helping to manage limited

resources that result in processing trade-offs (Carrasco, 2006; Kinchla, 1992;

Pestilli & Carrasco, 2005).

Although we had expected both benefits and costs of attention on CS to

be mediated by facial expression (Phelps et al., 2006), no such differences

emerged. A recent study suggests a possible explanation: the valence effect

of facial expression interacts with Gabor spatial frequency. There is no

advantage of fearful faces on the perception of oriented stimuli with spatial

frequency greater than 2 cpd (Bocanegra & Zeelenberg, 2009). These results

suggest that the beneficial effects of emotion are restricted to low spatial

frequencies. Whereas in our previous study we used 2 cpd stimuli, in the

present experiment we used 4 cpd stimuli, which may have resulted in the null

effect.

Could cue complexity rather than “faceness” account for the visual field

and handedness effects? A recent study suggests that the effect is face-

specific. Face cues produce greater differences in RT to detect cued and

uncued targets than equivalently complex phase-scrambled and inverted

faces, but only in the RVF (Elder, Balaban, Kamyab, Wilcox & Hou, 2008).

Consistent with that study, the present results show that performance with

face cues is also affected by visual field asymmetries. One aspect of the data

43

that can be explained by differential cue complexity is that in general, dot cues

result in overall greater cue validity effects and contrast sensitivity than face

cues. To make expressions discriminable, the size of the faces are much

larger than the dots, which may have resulted in a more diffuse attention boost

due to the trade-off between attention field size and spatial resolution (Eriksen

& St. James, 1986). However, the contrast and luminance of the faces were

equated across the whole set, resulting in much lower contrast for faces than

dots, which could also make them less effective exogenous cues (Fuller, Park

& Carrasco, 2009).

Furthermore, the present results show that the cue validity effects are

stronger in the RVF for right-handers, and LVF for left-handers. This is

consistent with the dominant-hand attentional bias seen in the Simon Effect

(Rubichi & Nicoletti, 2006), which reflects an interaction of target location and

the location of the hand used to make the response on RT. Responses are

faster when made with the hand adjacent to the target, compared to the

opposite hand. A larger Simon Effect is observed in the hemifield

corresponding to the dominant hand: for right-handers, the difference in RT

between hands to make an RVF response is larger than the corresponding

difference for an LVF response. Crucially, right-handers have faster RT for

RVF targets when they respond with their right-hand, and slower RT for LVF

targets when they respond with their left-hand (vice versa for left-handers).

44

This effect is thought to be due to spatial attention, which allows a more

efficient response selection for the dominant hand. In the present experiments,

this attentional bias may also explain the increased CS and cue validity effects

in the visual fields corresponding to each group’s dominant hand.

The degree of handedness and the magnitude of the attention effect

were significantly correlated in the RVF for both cue types, whereas the LVF

correlation was insignificant for faces and only marginal for dots (Figure 4).

However, the significant RVF correlations for faces vs. dots may have different

underlying phenomena. The significant face correlation is driven both by a

decrease in left-handers’ and increase in right-handers’ cue validity effect

(Figure 4, top-right panel); the use of face cues seems to affect attentional

deployment to the RVF in opposing ways in these two groups. This pattern of

results is consistent with the difference in degree of lateralization and inter-

subject variability for these two groups (Boles, 1989; Luh et al., 1994),

especially with regard to face processing (Bourne 2008). In contrast, the

significant dot correlation is primarily driven by closer clustering of right-

handers’ cue validity effect, with no real change in left-handers’ cue validity

(Figure 4, bottom-right panel); this finding indicates that attentional

deployment to the LVF results in an increase of CS for everyone but to

different degrees. This pattern of results is consistent with the existence of

attentional asymmetries across the visual field (e.g., Fecteau et al., 2000),

45

which depends in part on handedness (i.e., Simon Effect: Rubichi & Nicoletti,

2006).

Regardless of the differences between visual fields, why might the

effect of faces on covert exogenous attention depend on handedness? It is

possible that in left-hander brains, attention-related signals have to travel

farther to boost the processing of spatially specific locations cued by faces

than by dots. The right hemisphere of the right-handed brain is dominant for

both face and attention processing, allowing for efficient interactions of face

cues and attention signals. However, given their variability in degree of

lateralization, the functions of left-handed brains may be more distributed,

leading to greater distances between face- and attention-related regions. As a

result, left-handers as a group may not experience the same benefits and

costs of attention on CS when cued with faces as right-handers.

Even though left-handers comprise 10% of the population (Raymond,

Pontier, Dufour & M"ller, 1996), they are excluded from most cognitive

psychology and cognitive neuroscience studies because researchers are

concerned with laterality issues. Conversely, visual perception data from right-

and left-handers are usually averaged. However, we show here that

handedness is a critical variable affecting not only higher cognitive processes

but also perception. Our visual systems have evolved to become “face

46

recognition experts”, a specialization that interacts differently with attention in

right- and left-handers.

47

CHAPTER 2

Emotion and Attention Costs on Contrast Sensitivity:

Influences of Anxiety and Sex

(Unpublished, under review at Emotion)

Emma Ferneyhough1, Min K. Kim1, Elizabeth A. Phelps1,2 & Marisa Carrasco1,2

1New York University Psychology Department

2New York University Center for Neural Science

48

Abstract

Emotion and attention affect accuracy and response time in visual

detection and discrimination tasks. Anxiety modulates these effects. Here we

investigate how individual differences in trait anxiety and sex influence the

interaction of emotion and attention on contrast sensitivity, a basic visual

dimension. In two experiments, non-predictive precues directed exogenous

(involuntary) attention to contrast-varying stimuli (Gabor patches). Precues

were faces with either neutral or fearful expressions and were presented to

one or both sides of central fixation along the horizontal meridian (Experiment

1) or at one or four locations along the intercardinal meridians (Experiment 2).

On each trial, a tilted Gabor target was displayed randomly at one of the

possible task locations, concurrently with distracter(s). Attention was thus

randomly cued toward the target (valid cue), a distracter (invalid cue), or

distributed over all locations. Observers discriminated target orientation on

each trial, and completed self-report measures of anxiety. Consistent with

previous research, fear-distributed cues significantly improved contrast

sensitivity compared to neutral-distributed cues (Expt. 1). We also found that

emotion significantly interacted with attention resulting in perceptual benefits

and costs, but this depended on trait anxiety (Expts. 1 & 2) as well as sex

(Expt. 1) of the observer. Specifically, with two task locations high trait anxious

females showed increased contrast sensitivity with fear-valid cues and

49

decreased contrast sensitivity with fear-invalid cues while males showed no

effect (Expt. 1). With four task locations, all high trait anxious individuals

showed costs of emotion (Expt. 2), suggesting sex differences are reduced

with greater attentional demand. These findings are discussed in regards to

known sex differences in facial expression recognition and effects of anxiety

on response to threat-related stimuli.

Acknowledgements

We thank Damian Stanley, Tobias Brosch and David Carmel for helpful

discussions, as well as other Phelps and Carrasco Lab members for

comments on earlier versions of this manuscript. This research was funded by

grants NIH R01-EY016200 to M.C. and NIH R01-MH062104 to E.A.P.

50

INTRODUCTION

Emotion influences many cognitive processes such as learning,

memory, motivation, and decision making, and has been found to have at

least two distinct effects on visual attention and low-level visual perception.

Emotion can improve attention and perception under certain circumstances,

but it can also impair them in others. These two effects are consequences of

the finding that, compared to neutrally-valenced stimuli, emotional stimuli and

stimuli associated with emotions are preferentially processed in terms of

speed and depth (for a review see Compton 2003). This preferential

processing occurs especially for stimuli endowed with negatively arousing and

potentially threatening emotions such as fear and anger. When threat stimuli

attract attention to the location of an experimental task, for example,

performance typically improves; however when threat stimuli distract attention

away, performance is typically impaired. These benefits and costs due to the

interaction of emotion and attention have been shown in studies measuring

reaction time, whereas only the benefits have been demonstrated on

perception, specifically the basic visual feature of contrast sensitivity (Phelps,

Ling & Carrasco, 2006). Here we ask whether there is also a cost of the

interaction of emotion and attention on contrast sensitivity.

51

Fast, exhaustive processing of threat is crucial for survival (LeDoux

1996), but this threat-advantage is manifested differently in our attentional and

perceptual abilities. In the absence of emotion, it is known that covert

exogenous attention (i.e., attending reflexively while gaze is fixed) is a finite

cognitive resource that improves early visual processes at attended locations

but impairs them at unattended locations (for reviews see Carrasco, 2006,

2011). Attention researchers use peripheral cues consisting of dots or bars to

direct exogenous attention, which is driven involuntarily by a transient change

in the visual field. The effect is maximal at about 100-120 ms post cue onset,

and decays shortly thereafter (Nakayama & Mackeben, 1989; Cheal & Lyon,

1991; Fuller, Rodriguez & Carrasco, 2008). When exogenous attention is cued

to a spatial location, performance on visual tasks is improved there (Carrasco,

Penpeci-Talgar & Eckstein, 2000; Ling & Carrasco, 2006a), but this comes at

a cost to performance at uncued locations. These changes in performance

occur even though the cues are uninformative (i.e., they do not predict the

target location). Most relevant for the present study are the benefits and costs

of exogenous covert attention on contrast sensitivity (Pestilli & Carrasco, 2005;

Pestilli, Ling & Carrasco, 2009; Pestilli, Viera & Carrasco, 2007).

The conjoint effect of an emotional stimulus with attention results in

greater benefits and costs in performance compared to a neutral control.

Evidence for these tradeoffs have been mounting steadily, but have been

52

primarily in the form of reaction time (RT) differences in attention tasks, when

either emotional or neutral stimuli were used. As described in detail below,

tasks with emotional stimuli are typically completed faster at attended

locations, but are slower at unattended locations, compared to neutral stimuli.

Whether enhanced emotion processing produces benefits or costs to

attention and perception critically depends on how relevant the emotional

stimulus or cue is to the task at hand. One way to manipulate stimulus/cue

relevance is by changing its spatial location in relation to a target task

stimulus. Using Posner and colleagues’ (Posner & Petersen, 1990) three

components of spatial attention (“shift-engage-disengage”) as a simple model,

researchers have conducted experiments to investigate what effects emotion

can have on the shifting, engagement, and disengagement of attention (e.g.,

Derryberry & Reed, 2002; Yiend & Mathews, 2001). An emotional cue such as

a picture or word, for example, may improve target processing in its vicinity

due to the beneficial effects of attentional shifting to, and engagement with,

task-relevant locations (valid cue). This enhanced attentional engagement with

emotional stimuli, compared to a neutral control, results in greater accuracy

and faster RT on experimental tasks. If that same emotional stimulus is at a

task-irrelevant location, however, it can impair target processing due to costs

of attentional disengagement and shifting from the task-irrelevant back to the -

relevant location (invalid cue). This impaired attentional disengagement from

53

emotional stimuli, compared to a neutral control, results in decreased accuracy

and slower RT on experimental tasks (e.g., Fox, Russo, Bowles & Dutton,

2001).

Benefits and costs to RT have also been demonstrated in a spatial

cueing experiment using fear conditioning. Koster, Crombez, Van Damme,

Verschuere and De Houwer (2004) measured detection RTs for targets cued

with stimuli paired (conditioned stimulus: CS+; predicts threat) and not paired

(CS-; does not predict threat) with an unconditioned stimulus (an aversive

white noise burst). Targets that were validly cued by a CS+ were detected

faster compared to those cued by a CS-. On the other hand, RTs were slower

when these targets were invalidly cued by a CS+ compared to CS-. The

authors concluded that the slowing of RT was due to a delayed

disengagement from the stimulus that predicted threat.

In addition to effects on RT, research has recently focused on

psychophysical investigations of the interaction of emotion and attention on

fundamental dimensions of visual perception. Contrast is a visual feature that

underlies stimulus visibility. Perception of contrast occurs at the earliest levels

of the cortical visual hierarchy, area V1. Thus, contrast sensitivity, unlike RT,

carries important information about the strength of the initial perceptual signal

as it enters primary visual cortex (e.g., Boynton, Demb, Glover & Heeger,

1999; Graham, 2011). As a consequence of its elementary nature,

54

improvement (or impairment) of this signal by emotion and/or attention can

then influence a vast array of perceptual and cognitive processes downstream.

Phelps et al. (2006) used visual psychophysics methodology to

investigate how emotion and attention interact to affect contrast sensitivity. On

each trial, a fearful face precue was briefly presented, reflexively drawing

exogenous, covert (automatic and transient, without eye movements) attention

to its location. When this precue appeared just prior to the onset of a tilted

target Gabor patch (a luminance-defined, sinusoidal grating convolved with a

Gaussian), participants’ orientation discrimination improved, compared to the

presentation of a neutral face precue. This improvement in performance was

more pronounced when the fearful face was a valid cue (spatially informative)

compared to a distributed cue (appearing at all possible locations, therefore

not spatially informative). Given that increased performance on orientation

discrimination tasks depends on increased contrast sensitivity (e.g., Carrasco

et al., 2000; Ling & Carrasco, 2006a; Pestilli et al., 2009), these results

showed that emotion improved contrast sensitivity, and this effect was

facilitated by the beneficial effect of attention in valid trials. There were no

invalid cues, however, leaving open the question of how emotion affects

contrast sensitivity at unattended locations.

The evidence described thus far indicates that emotion and attention

interact to produce benefits and costs for RTs (Koster et al., 2004) and only

55

benefits for contrast sensitivity (Phelps et al., 2006). It remains unclear,

however, whether there is also a cost of emotion for contrast sensitivity in

invalidly cued trials. In other words, does emotion affect the cost of attention

disengagement for the perceptual signal when cueing task-irrelevant (invalid)

locations? Here, we test the hypothesis that emotion, like attention, can

reflexively draw resources resulting in both improved contrast sensitivity at

validly cued locations and impaired contrast sensitivity at invalidly cued

locations compared to a neutral control.

Of particular relevance to our experimental design is a recent study that

showed both benefits and costs of emotion in orientation detection

performance (Bocanegra & Zeelenberg, 2009). Whether there was a benefit or

cost crucially depended on target spatial frequency: the orientation of low

spatial frequency (LSF) targets was detected better, whereas the opposite was

true for high spatial frequency (HSF) targets, when fearful (compared to

neutral) precues were used. Spatial frequency, like contrast, is another basic

feature of vision; unique neural pathways preferentially process low vs. high

spatial frequencies in natural images. This study did not manipulate spatial

attention (two spatially uninformative cues always appeared at the same time

adjacent to both possible target locations), precluding any conclusions

regarding the interaction of emotion and attention. However, the results of this

study underscore the fact that emotion and perception are not independent.

56

Furthermore, these results are consistent with a prior demonstration of the

importance of coarse, LSF information for recognizing emotional facial

expressions, especially threat (Vuilleumier, Armony, Driver & Dolan, 2003).

Within a short time window after fearful face presentation, visual channels

sensitive to LSFs may confer a perceptual benefit to other LSF stimuli in the

vicinity. In light of these results, we hypothesized that the LSF information in

fearful faces would lead participants to be more sensitive, not only to

orientation, but to the contrast of LSF target stimuli as well.

Manipulating externally observable aspects of stimuli in an attention

paradigm, such as cue emotionality or target size and frequency content,

undoubtedly have measurable consequences on the outcome of experimental

results. At the same time, experiment participants’ mental state, personality

characteristics, or sex, should be taken into consideration. In particular, there

exists a wide range of emotional dispositions in the general population. What

is considered “normal” is highly variable. Furthermore, differences between

males and females in regards to brain function (Cahill, 2006) as well as

personality tendencies have been well documented; for example, females are

more likely to have symptoms of anxiety than males (Kessler, Berglund,

Demler, et al., 2005). While investigating costs of emotion on attention, it is

therefore important to look at individual variability in factors known to modulate

emotional effects on attention. Anxiety and sex are two such factors.

57

Non-clinical trait anxiety, for example, correlates positively with both the

benefits (Macleod & Mathews, 1988; Mogg, Holmes, Garner & Bradley, 2008;

Öhman, Flykt & Esteves, 2001) and costs (Fox et al., 2001; Fox, Russo &

Dutton, 2002; Koster, Crombez, Verschuere & De Houwer, 2006; Smith, Most,

Newsome & Zald, 2006; Yiend & Mathews, 2001) of visual attention in tasks

measuring RT and accuracy. Heightened attention, and prolonged

maintenance of attention, to potential threat is thought to underlie anxiety

sufferers’ propensity to dwell on negative thoughts and feelings in the absence

of a trigger, thereby perpetuating anxious symptoms. It remains unknown,

however, whether anxiety modulates the effects of attention on basic visual

dimensions such as contrast sensitivity.

Participant sex may also mediate interactions of emotion and attention

on perception. Females are more sensitive to faces, performing better than

males on tasks involving facial expression discrimination, recognition and

identification (McClure, 2000; Thayer & Johnsen, 2000). Given that the

present study uses fear face stimuli, heightened sensitivity in females to these

threatening facial expressions could result in differences between the sexes in

attentional selectivity for faces. In addition, sex may interact with anxiety-

related attention biases. Anxious females and males are known to cope with

threat differently; while females internalize feelings and are avoidant of threat,

males are more likely to externalize feelings and act out (for a review see

58

Craske, 2003). Different overt coping strategies could be associated with

different covert attention biases in the presence of threat.

The goal of the present study was to investigate whether using fearful

(compared to neutral) faces as attentional cues would lead to benefits and

costs for contrast sensitivity, specifically with low spatial frequency targets. We

hypothesized that fearful face cues will exaggerate both the benefits and the

costs of attention on perception. To investigate whether there is a connection

between the magnitude of emotion’s effects on attention and trait anxiety, we

recruited participants who spanned a wide range on self-report measures of

state- and trait-anxiety. In Experiment 1, equal numbers of male and female

participants were recruited, enabling an investigation into possible interactions

of anxiety and sex. In Experiment 2, we increased the number of possible task

locations to further tax the limits of spatial attention cued with faces.

EXPERIMENT 1: Two task locations

METHODS

Participants

Fifty-six (28 female; age M = 21, SD = 4, range = 18-33) right-handed

observers were recruited. All had normal or corrected-to-normal vision and

completed the 10-item Edinburgh Handedness Inventory (Oldfield, 1971)

59

before participating. Possible scores ranged from -100 to +100 (completely

left- to completely right-handed, respectively). Only observers with

handedness scores ! 40 participated (M = 79, SD = 17). All observers

completed the 20-item Positive and Negative Affect Scale (PANAS: Watson,

Clark & Tellegen, 1988) and the 40-item State-Trait Anxiety Inventory (STAI:

Spielberger, Gorsuch, Lushene, Vagg & Jacobs, 1983) at the experiment’s

conclusion. The PANAS was used to assess the degree to which different

positive and negative emotions were experienced in general over the previous

six months, and scores could range from 10 to 50 within either positive or

negative affect (positive affect: M = 34, SD = 7; negative affect: M = 21, SD =

7). The STAI was used to assess degree of anxiety at the present moment

(state) and in general (trait), and scores could range from 20 to 80 within either

state or trait anxiety (state anxiety: M = 38, SD = 10; trait anxiety: M = 40, SD

= 12). Negative affect and trait anxiety are typically highly correlated with each

other.

Apparatus and Stimuli

Stimuli were presented on a 21” CRT monitor (1600 x 1200 pixels; 75

Hz) connected to a Power Macintosh G4 computer via an attenuator driving

just the green gun (providing a larger possible set of distinct luminance levels).

60

Background luminance was set to 16.5 candelas/m2. During the experiment,

participants’ heads were stabilized using a chin rest 57 cm from the monitor.

Face stimuli consisted of 22 contrast- and luminance-equated grayscale

pictures of fearful and neutral faces from the Pictures of Facial Affect series

(Ekman & Friesen, 1976; same as used in Phelps et al., 2006 and

Ferneyhough, Stanley, Phelps & Carrasco, 2010). Gabor patches (SD = 1

degree (deg); 1.5 cycles per deg (cpd)) were created using Matlab 5.2.1 and

the Psychophysics Toolbox (Brainard, 1997). The face cues subtended 3.5 x

4.6 deg and were centered 8 deg to the left and right of fixation along the

horizontal meridian. The Gabor patches subtended 3 deg and were centered 4

deg to the left and right of fixation. The Gabor target was always tilted 6 deg

right or left from vertical, whereas the distracter was vertical (0 deg; e.g., Liu,

Pestilli & Carrasco, 2005). Seven Gabor patch contrasts were chosen

individually per observer to obtain performance levels that ranged from chance

to asymptotic performance with the goal of having at least three contrasts

within the dynamic range of fitted psychometric functions.

Procedure

Observers were seated in a darkened room and completed an

orientation discrimination task. Performance on orientation discrimination tasks

such as this, in which the target Gabor contrast varies systematically from trial

61

to trial, is commonly used to assess contrast sensitivity (e.g., Carrasco et al.,

2000; Ling & Carrasco, 2006a; Pestilli & Carrasco, 2005; Pestilli et al., 2009).

On each trial, observers fixated a central cross for 500 ms; then a face precue

(fearful or neutral) was presented to the left, right or on both sides of fixation

for 80 ms to manipulate exogenous attention; following a 53 ms ISI, one tilted

(the target) and one vertical Gabor patch were presented, one on either side of

fixation, for 40 ms. Participants were instructed to indicate the target location

(left or right) and orientation (counterclockwise or clockwise), with a single

button press within a 2000 ms response window (Figure 5). Feedback was

given after each trial by a high tone for correct and a low tone for incorrect

responses. Valid cues appeared on the same side as targets, invalid cues

appeared on the opposite side, and distributed cues appeared on both sides.

Each cue type appeared in 1/3 of the trials. Observers completed 672 trials

each (16 trials per condition per contrast level) in one 1-hour session. See

Appendix B for detailed task instructions.

Prior to the 1-hour experimental session with face cues, each observer

completed a half-hour practice session with black dot cues (0.3 deg diameter,

8 deg eccentricity) to avoid habituation to facial expression. From performance

on the practice session, we determined each individual’s contrast range to

allow an average performance of ~67% (about halfway between chance, 25%,

and perfect, 100%, performance) in the distributed dot cue condition.

62

Analysis

For each condition, we calculated percent correct as a function of

contrast. Psychometric Weibull functions were fitted using psignifit 2.5.6

(http://bootstrap-software.org/psignifit/; Wichmann & Hill, 2001). Contrast

threshold was defined as the estimated stimulus intensity at which observers

would be correct 67% of the time. The primary dependent variable was

contrast sensitivity, which is the inverse of contrast threshold. Observers’

contrast sensitivity scores were individually normalized by dividing each

condition mean by the average of all conditions; such normalized scores

reduced the noise introduced by different baseline contrast sensitivity for

different observers (e.g., Ferneyhough et al., 2010). Normalized contrast

sensitivity scores were then averaged across all observers. Reaction times

were also measured as a secondary dependent variable. Here we report

detailed statistics for contrast sensitivity and note that for RT there were no

speed-accuracy trade-offs for any comparison.

RESULTS

One of the six conditions for seven of the observers (~2% of the total

experiment data) could not be reliably fit with a Weibull function (deviance

63

scores, which assess goodness of fit, exceeded #2.05(7) = 14.1; .05 refers to p-

value; 7 refers to number of contrast levels), so their data were discarded. In

addition, the data of three observers whose contrast sensitivity were > 3 SDs

from the mean on one of the six conditions were also discarded. There was no

consistency in which conditions could not be fit or resulted in outliers across

these observers. There were a total of 46 remaining observers for the data

analysis (24 male, 22 female). Self-reported state or trait anxiety were not

different between males and females (ps>.1), however, males reported greater

negative affect (males M = 22, females M = 18, t(44)=2.27, p<.05). All

remaining observers were classified as either low or high trait anxiety via a

median-split. All reported t-tests are two-tailed, unless noted otherwise.

Overall Contrast Sensitivity

A 2x3x2x2 mixed-model ANOVA was conducted on normalized contrast

sensitivity scores of all observers. Facial expression (fearful, neutral) and cue

(valid, distributed, invalid) served as within-subjects factors, and trait anxiety

(low, high) and sex (male, female) served as between-subjects factors. This

ANOVA revealed a marginally significant four-way interaction of face type, cue

condition, sex and trait anxiety (F(2,80)=2.53, p<.1), and a significant three-

way interaction of face type, cue condition and sex (F(2,80)=3.701, p<.05).

Furthermore, there was a marginal two-way interaction between face type and

64

cue condition (F(2,80)=2.551, p<.1). Follow-up paired t-tests comparing fear-

distributed with neutral-distributed cue conditions showed a significant

difference in contrast sensitivity (t(45)=2.76, p<.01) with fear-distributed cues

leading to higher contrast sensitivity (Figure 6). Fear-distributed cues also led

to higher contrast sensitivity than fear-invalid cues (t(45)=3.22, p<.01).

Sex Differences

To investigate the nature of these significant interactions in the overall

ANOVA, two 2x3x2 mixed-model ANOVAs with facial expression and cue as

within-subjects factors, and anxiety as a between-subjects factor were

conducted separately for males and females. Males showed a marginally

significant two-way interaction of face type and cue condition (F(2,40)=2.493,

p<.1). Follow-up paired t-tests revealed that fear-distributed contrast sensitivity

was greater than neutral-distributed (t(23)=1.75, p<.05, one-tailed), and

neutral-valid was greater than both neutral-distributed (t(23)=2.01, p<.05, one-

tailed) and neutral-invalid (t(23)=1.98, p<.05, one-tailed; Figure 7, top right).

Females also showed a marginally significant two-way interaction of

face type and cue condition (F(2,36)=3.128, p=.056). Fear-distributed contrast

sensitivity was significantly greater than both neutral-distributed (t(21)=2.21,

p<.05) and fear-invalid (t(21)=3.12, p<.01; Figure 7, top left). Within females,

there was also a significant three-way interaction of face type, cue condition

65

and anxiety (F(2,36)=3.282, p<.05; note that two females with median anxiety

scores were excluded from this interaction because observers were

categorized as either high or low anxiety based on a median split).

Anxiety by Sex

Interestingly, when the observers were divided into four groups based

on sex and trait anxiety scores (male low, male high, female low, female high),

we observed differences in patterns of contrast sensitivity, especially

comparing high trait anxious females to the other three groups. Figure 7

shows the contrast sensitivity of the fear and neutral expressions in the

distributed cue condition (middle bars) plotted for each of the four groups, as

well as the overall groups. Again, as described above, there was a significant

increase in contrast sensitivity with fear-distributed relative to neutral-

distributed cues across all observers (t(45)=2.76, p<.01). This difference was

significant across all female observers (n=22, t(21)=2.21, p<.05), and was

driven by the high trait anxious females (n=10, t(9)=2.59, p<.05). High trait

anxious males also showed greater contrast sensitivity in the fear-distributed

relative to neutral-distributed condition (n=11, t(10)=1.89, p<.05, one-tailed).

Next we evaluated the female participants’ difference in contrast

sensitivity with fear-valid vs. neutral-valid cues and found that it was

significantly greater than the corresponding difference in contrast sensitivity

66

with fear-invalid vs. neutral-invalid cues (t(21)=1.91, p<.05, one-tailed; Figure

7, top left). This comparison was stronger in high trait anxious females

(t(9)=2.68, p<.05; Figure 7, bottom left). The increase in contrast sensitivity

with fear-valid vs. neutral-valid cues in high trait anxious females was

marginally greater than zero (t(9)=1.72, p=.06, one-tailed). The decrease in

contrast sensitivity with fear-invalid vs. neutral-invalid cues, on the other hand,

was marginally less than zero (t(9)=-1.72, p=.06, one-tailed). Thus, significant

differences found across all females were driven by females with high trait

anxiety. There were no such differences in males.

Correlations

Trait- and state-anxiety, and trait-anxiety and negative affect scores

were significantly correlated with each other across all observers (rs=.75 and

.54, respectively; n=46, ps<.001), validating the self-report measures. Trait-

anxiety was not correlated with the difference between fear-invalid and

neutral-invalid contrast sensitivity as hypothesized, neither across all

observers nor within each sex. However, a correlation between female

negative affect scores and the difference of fear-distributed and neutral-

distributed contrast sensitivity was marginally significant (r=.4, n=22, p<.07).

Higher negative affect scores were associated with greater increases of

67

contrast sensitivity with fear- over neutral-distributed conditions. No other

correlations of self-report measures and behavior were significant.

EXPERIMENT 1 DISCUSSION

We investigated both the benefits and costs of emotion and attention on

contrast sensitivity. Our findings revealed that emotion interacted with

attention in a manner that was dependent on trait anxiety and sex. In addition,

fear-distributed cues significantly improved contrast sensitivity compared to

neutral-distributed cues, replicating a previous study (Phelps et al., 2006).

Although fear-valid and -invalid cues did not consistently modulate contrast

sensitivity across observers compared to neutral cues, group differences in

anxiety and sex indicate that only high trait anxious females demonstrated

both benefits and costs of emotion and attention on perception.

Some results of the present study were not consistent with earlier

research (Phelps et al., 2006). First, we were expecting benefits of emotion

across our whole subject population. However, we found that males,

regardless of anxiety level, and low-anxious females showed no significant

emotion effect. Second, the present research did not replicate the findings that

emotion and attention (fear-valid cues) improve contrast sensitivity above and

68

beyond that of emotion alone (fear-distributed cues). Instead, we found that

the benefits of both were similar in magnitude in high-anxious females.

Possible reasons for these two inconsistencies are the differences in

stimulus and experiment parameters between the two studies. Cue and target

sizes differed, being much smaller in the present study (cues: 5 vs. 3.5 deg

width; targets: 8 vs. 3 deg diameter). Cue and target locations in the visual

field also differed, with both cue and target eccentricity being smaller here as

well (cues: 5 vs. 4 deg eccentricity; targets: 11 vs. 8 deg eccentricity). Could

effects of emotion and attention be more exaggerated further out in the

periphery? This possibility rests on the larger receptive field sizes in peripheral

vision (DeValois & DeValois, 1988), the decreasing ratio of the number of

neurons tuned to high vs. low spatial frequencies with eccentricity (Azzopardi,

Jones, & Cowey, 1999), and the preference of the amygdala for low spatial

frequency information (Vuilleumier et al., 2003). Whereas effects of attention

increase with eccentricity (Carrasco & Yeshurun, 1998; Yeshurun & Carrasco,

1999), another study investigating emotion processing in the amygdala

showed no evidence of eccentricity effects (at -1.68, 5.6, and 11.25 deg;

Morawetz, Baudewig, Treue & Dechent, 2010). Given that trait anxiety data

were not collected in the Phelps et al. (2006) study it is possible that, by

chance, a greater proportion of this small group of observers (n=6) had higher

trait anxiety, which could have contributed to the prior results.

69

In addition, we used two stimulus locations, whereas the 2006 study

used four locations. Attentional resources may not have been sufficiently

taxed with only two locations to see a benefit of emotion and attention over

emotion alone. Anecdotally, many observers said it was easier to do the task

when two (distributed) face cues appeared, as opposed to one (valid or

invalid) face cue, because it made it easier to see both Gabor patches. Being

able to clearly see both stimuli provides an advantage because they can then

be more easily compared to one another. Given that one is always vertical and

one is always tilted, having information regarding both makes the orientation

discrimination task easier.

Experiment 2 addresses the concerns outlined above by using the

same stimulus parameters as Phelps et al. (2006) with four possible task

locations. Differences between the two experiments include: 1) instead of only

the target Gabor being tilted, all four Gabor stimuli were randomly tilted to

prevent any comparisons between target and vertical distracters; and 2) target

identity was revealed with a response cue at Gabor offset (e.g., Ling &

Carrasco, 2006b; Pestilli & Carrasco, 2005).

70

EXPERIMENT 2: Four task locations

METHODS

Participants

Forty-seven new observers were recruited (32 female; age M = 22, SD

= 4, range = 18-34). All observers had normal or corrected-to-normal vision

and were right-handed (Edinburgh Handedness Inventory M = 81, SD = 22;

Oldfield, 1971). All observers completed the PANAS (positive affect: M = 36,

SD = 6; negative affect: M = 21, SD = 7; Watson et al., 1988) and the STAI

(state anxiety: M = 38, SD = 10; trait anxiety: M = 39, SD = 10; Spielberger et

al., 1983).

Apparatus and Stimuli

Stimuli were presented on a 21” CRT monitor (1600 x 1200 pixels; 75

Hz) connected to a Macintosh Intel IMac computer. Background luminance

was set to 57 candelas/m2 (25% of its maximum luminance). The stimuli used

were the same as in Experiment 1, except they were enlarged (faces: 5 x 6.7

deg; Gabor patches: 7.9 deg, 2 cpd, tilted ± 5 deg from vertical).

71

Procedure

On day 1 each observer completed a half-hour training session with

black dot cues (0.3 deg diameter, 5 deg eccentricity). Participants who

performed ! 70% accuracy (about halfway between chance, 50%, and perfect

performance) on average throughout training continued on to the first 4 blocks

of the main experiment in which fearful or neutral faces were used as cues. On

day 2, observers returned to complete the other 8 blocks of the experiment

and to fill out the self-report surveys. In total, observers completed 1,344 trials

(112 trials per block).

Observers performed an orientation discrimination task. The trial

sequence was similar to that of Experiment 1 with the following differences: 1)

the precue was presented to either one (valid or invalid) or four (distributed)

locations along the intercardinal merdians (5 deg eccentricity); 2) four

randomly tilted Gabor patches were presented at each of four intercardinal

locations (11 deg eccentricity); 3) a response cue appeared for 100 ms at

Gabor offset indicating the location of the target Gabor; 4) participants were

instructed to indicate only the target orientation (counterclockwise or

clockwise; Figure 8).

72

RESULTS

Data from seven observers could not be reliably fit with a Weibull

function (see fitting criteria Experiment 1: Results) and four observers had

contrast sensitivity > 3 SDs from the mean, so only data from the 36 remaining

observers (23 females, 13 males) were included in the ANOVAs and t-tests.

Overall Contrast Sensitivity

A 2x3x2 mixed-model ANOVA was conducted on normalized contrast

sensitivity scores of all observers. Facial expression (fearful, neutral) and cue

(valid, distributed, invalid) served as within-subjects factors, and trait anxiety

(low, high) served as a between-subjects factor. This ANOVA resulted in a

marginally significant main effect of cue condition (F(2,68)=2.694, p=.075), in

which contrast sensitivity was highest with valid cues, then distributed and

finally invalid cues (Figure 9, top). Planned paired t-tests across observers

revealed a significant increase in contrast sensitivity in the valid cue relative to

the invalid cue condition (t(35)=1.94, p<.05, one tailed). The main effect was

qualified by a significant two-way interaction of face type and trait anxiety

(F(1,34)=4.278, p<.05).

73

Anxiety Differences

To investigate the nature of the significant interaction of face type and

trait anxiety in the overall ANOVA, planned paired t-tests were conducted

comparing the effects of fear vs. neutral face expression in each cue condition

for both anxiety groups resulting from a median split on the STAI survey. In the

high trait anxious group there was a significant decrease in contrast sensitivity

in the fear-invalid relative to both the neutral-invalid condition (t(17)=-2.741,

p<.05), and the fear-distributed condition (t(17)=2.197, p<.05; Figure 9,

bottom). In the low trait anxious group, there was a significant decrease in

contrast sensitivity in the fear-distributed relative to the fear-valid condition

(t(17)=2.432, p<.05; Figure 9, middle). No other comparisons were significant

(ps >.1).

Correlations

As in Experiment 1, trait- and state-anxiety, and trait-anxiety and

negative affect were significantly correlated with each other across observers

(rs=.66 and .74 respectively; n=38, ps<.001). Trait anxiety and the difference

in contrast sensitivity in the invalid cue condition (fear-invalid minus neutral-

invalid) were significantly correlated (r=-.36, n=36, p<.05). As trait anxiety

increased, the difference in contrast sensitivity between the fear-invalid and

neutral-invalid cue conditions also increased in the hypothesized direction.

74

EXPERIMENT 2 DISCUSSION

In Experiment 2 we modified cue and target parameters and increased

the number of possible task locations, from two to four, to address several

concerns from Experiment 1. Overall there was a significant effect of attention

in which, regardless of emotion, contrast sensitivity was highest at attended

locations (valid), intermediate with diffused attention (distributed), and lowest

at unattended locations (invalid) consistent with previous research

(Ferneyhough et al, 2010; Pestilli & Carrasco, 2005; Pestilli et al., 2007). This

result provides support for our hypothesis that attentional resources were not

sufficiently taxed with only two task locations in Experiment 1 to see benefits

or costs of attention on contrast sensitivity.

In addition, although our results showed no significant differences due

to emotion across all subjects, we did find significant decreases in contrast

sensitivity in the fear-invalid relative to both the neutral-invalid and fear-

distributed cue conditions in the high trait anxiety group across both males and

females. This result is in agreement with the results from Experiment 1,

extending the findings to both high trait anxious males and females. The

significant negative correlation of trait anxiety with the size of the

disengagement cost with emotion (fear-invalid minus fear-valid contrast

75

sensitivity) provides further evidence in support of our hypothesis that anxiety

increases attention disengagement costs with emotion.

Lastly, in this experiment, we found that the low trait anxiety group

showed significantly greater contrast sensitivity in the fear-valid relative to

fear-distributed cue condition, indicating a significant effect of attention with

fear, but not neutral, face cues. This result is consistent with the previously

found facilitation effect of emotion on the benefit of attention to contrast

sensitivity (Phelps et al, 2006).

GENERAL DISCUSSION

Across both Experiments 1 and 2 we showed that trait anxiety

increases the cost of attention disengagement from fearful faces. When a fear-

invalid cue automatically directed attention to a location incongruent with the

target, contrast sensitivity was more impaired at the target location, relative to

a neutral-invalid cue. Moreover, in Experiment 1 we showed that fear-

distributed cues significantly improved contrast sensitivity compared to neutral-

distributed cues, replicating a previous study (Phelps et al., 2006). Further,

only high trait anxious females showed benefits and costs of emotion to

attention when only two task locations were used. Using four task locations in

76

Experiment 2, we additionally replicated previous findings that showed

benefits to contrast sensitivity at attended locations, and costs to contrast

sensitivity at unattended locations (Ferneyhough et al., 2010; Pestilli &

Carrasco, 2005; Pestilli et al., 2007). Importantly, we also replicated previous

findings that showed emotion enhances the beneficial effect of attention on

contrast sensitivity (Phelps et al., 2006) in low trait anxious observers.

Exactly how anxiety biases the allocation of spatial attention is debated.

Some research suggests that those who are more anxious will be more

strongly drawn to threatening stimuli such as faces with angry or fearful facial

expressions, experiencing benefits at these attended locations (Macleod &

Mathews, 1988; Mogg, Holmes, Garner & Bradley, 2008; Öhman et al., 2001).

Other research suggests they will instead be slower to disengage from

threatening stimuli, experiencing costs at unattended locations (Fox et al.,

2001; Fox et al., 2002; Koster et al., 2006; Smith et al., 2006; Yiend &

Mathews, 2001; for reviews: Bar-Haim, Lamy, Pergamin, Bakermans-

Kranenburg & van IJzendoorn, 2007; Weierich, Treat & Hollingworth, 2008).

Experiment 1 suggests females with increased anxiety are both more strongly

drawn to threat and also have greater difficulty disengaging from threat than

other participants. This results in enhanced contrast sensitivity following fear-

valid cues and impaired contrast sensitivity following fear-invalid cues, relative

to their neutral counterparts. Experiment 2 placed more stringent limitations on

77

attentional resources by using four task locations. Within this context, high trait

anxious observers had greater difficulty disengaging from threat, resulting in

impaired contrast sensitivity following fear-invalid cues relative to neutral. In

addition, low trait anxious observers were drawn more to threat resulting in

enhanced contrast sensitivity following fear-valid relative to fear-distributed

cues.

In our task, the rapid presentation of face cues directed exogenous,

bottom-up attention towards or away from target stimuli. We used fearful

faces, commonly used to recruit the amygdala (e.g., Bishop, Duncan &

Lawrence, 2004; Dickie & Armony, 2008; Morris, Friston, Buchel, Frith, Young,

Calder & Dolan, 1998; Morris, deGelder, Weiskrantz & Dolan, 2001; Whalen

1998; Vuilleumier et al., 2003; Vuilleumier, Richardson, Armony, Driver &

Dolan, 2004), which can then strengthen cue representation via feedback

connections throughout the ventral visual pathway (Freese & Amaral, 2005)

enhancing bottom-up attention allocation. Numerous studies have investigated

this possible link between amygdala activity and enhanced signal in visual

cortex (e.g., Amaral, Behniea & Kelly, 2003; Anderson & Phelps, 2001; Morris

et al., 1998, 2001; Vuilleumier et al., 2004). No studies that we know of,

however, have investigated how anxiety might modulate perception.

Neurocognitive theories of anxiety and attention have suggested

amygdala activity is heightened in anxiety in response to sources of potential

78

threat (e.g., Davis & Whalen, 2001). This hyperactivity could in turn bias

bottom-up attention allocation more strongly towards locations of threat,

resulting in both enhanced perception at cued locations and impaired

perception at uncued locations. Recent work investigating the role of frontal

brain regions in the top-down control of emotion have shown that anxious

individuals may have, not only increased amygdala activity (Bishop, Duncan &

Lawrence, 2004; Dickie & Armony, 2008), but decreased recruitment of frontal

control regions as well (Bishop, 2008). This imbalance between bottom-up

emotional response and top-down attention could underlie the difficulty

anxious individuals have in disengaging attention from threat. Consistent with

this imbalance, it has been shown with diffusion tensor imaging that

connections between amygdala and ventral medial prefrontal cortex are

weakened in anxiety (Kim & Whalen, 2009). Furthermore, voxel-based

morphometry research has shown that increased anxiety is associated with

decreased cortical volume in brain regions implicated in anxiety disorders,

such as the amygdala, ventromedial and dorsolateral prefrontal cortex

(Spampinato, Wood, De Simone & Grafman, 2009). These studies provide

support for the idea that the expression of anxiety in an individual is closely

linked to impaired amygdala-frontal cortex interactions, which could result in

increased bottom-up response to threat. In tasks such as ours, in which both

exogenous attention and emotion are manipulated, feedback to V1 from the

79

amygdala and brain regions involved in exogenous attention shifts may

interactively modulate V1 activity resulting in the benefits and costs to contrast

sensitivity described above. With increased anxiety, stronger feedback from

the amygdala may result in greater costs. It is an open question, however, why

the benefits with emotion are less consistently found than the costs, in both

the present study and previous studies (e.g., Fox et al., 2001).

The fact that anxiety can modulate contrast sensitivity, regardless of the

specific direction of these exogenous attention effects, indicates a prioritization

of resources that enhances processing of possibly threatening stimuli in the

environment. Greater sensitivity to differences between light and dark

enhances the perception of borders and outlines of objects, which provides an

advantage in efficiently parsing threat from non-threat. Higher anxiety, at least

to some extent, may impart an even greater advantage in this process, but as

we show here this threat-advantage can come at a cost of performing visual

tasks that do not pose a threat. Evolutionarily, this is often an acceptable cost

in comparison to those from real threats. Today, however, life-threatening

situations are rare, and attentional biases due to anxiety can impair our ability

to focus on task-relevant items.

Whereas high trait anxious females showed the hypothesized benefits

and costs of emotion in Experiment 1, male participants did not. These

differences may have depended on the availability of attentional resources.

80

When attentional resources were more limited, as in Experiment 2, costs of

emotion were evident across all high trait anxious individuals. Although there

were almost twice as many females as males (n=23 and 13, respectively) both

high trait anxious groups showed decreased contrast sensitivity for fear-invalid

than for neutral-invalid trials. What could explain why only females show the

benefits and costs of emotion on attention in Experiment 1? Both females and

males experience the same degree of self-reported anxiety, however, it has

been shown that there are vast differences in their response to anxiety-

provoking stimuli (for a review see Craske, 2003). Females tend to internalize

their feelings and withdraw, avoiding threat, whereas males tend to externalize

their feelings, often resulting in more outwardly aggressive behavior. Here we

showed that there are differences in how anxious males and anxious females

allocate exogenous covert spatial attention in the presence of fearful face

expressions when attention is not severely taxed. Exactly how the

aforementioned differences in overt behavior may be related to these covert

differences cannot presently be determined, however, meriting further

investigation.

As mentioned above, males and females show different patterns of

avoidance of threat. High anxious females tend to overtly avoid threatening

situations, which may have important links with the literature on attentional

avoidance of threat. The ‘vigilance-avoidance’ hypothesis (e.g., Mogg,

81

Bradley, Miles & Dixon, 2004), for example, states that after a fast, initial

orienting to locations of threat, a voluntary avoidance component directs

attention away from the threatening stimulus. This avoidance serves to protect

the individual from further exposure, yet by doing so it tends to maintain

anxious traits because the individual is rarely able to habituate to the threat.

Evidence for avoidance has been shown in many (e.g., Holmes, Nielsen &

Green, 2008; Koster, Verschuere, Crombez & Van Damme, 2005) but not all

(e.g., Bradley, Mogg, Falla & Hamilton, 1998) studies of anxiety. Our contrast

sensitivity results for high anxious females are consistent with reaction time

studies showing vigilance towards threat. However, given that the cue and

target were presented in less than 180 ms our task was not designed to

evaluate avoidance, which has been shown to require more than 1000 ms to

emerge (Koster et al, 2005; Mogg et al., 2004). Instead our paradigm was able

to provide evidence for impaired disengagement of attention from fear cues

relative to neutral cues on high anxious females’ contrast sensitivity, which

was observable in our time frame. This unique finding extends prior work that

has shown benefits and costs of exogenous attention on contrast sensitivity, in

which non-emotional cues direct attention to the location of a target or

distracter (Pestilli & Carrasco, 2005). Here we find that fear cues provide

greater benefits and greater costs than neutral cues in high anxious females.

82

In addition to having different reactions to threat than males, females

have been shown to be more sensitive in recognizing and classifying

emotional facial expressions (McClure, 2000; Thayer & Johnsen, 2000).

Perhaps when attention is not as severely limited, this greater sensitivity is

manifested by females with higher anxiety relative to males with equivalent

anxiety levels, leading to increased attentional allocation to the face cues.

Consistent with this notion, results from an ERP study showed anxious

females had greater processing of stimuli than anxious males at an early stage

(P100, ~100 ms), however this early responding was not modulated by

valence of the stimuli (Sass, Heller, Stewart, Silton, Edgar, Fisher & Miller,

2010). Moreover, fMRI studies have shown that females have greater activity

in primary and secondary visual cortex to unpleasant relative to pleasant

stimuli, with the opposite pattern in males (Lang, Bradley, Fitzsimmons,

Cuthbert, Scott, Moulder et al., 1998), and that high trait anxious females have

increased amygdala response to unattended fearful faces compared to high

trait anxious males (Dickie & Armony, 2008). These studies add to a body of

knowledge demonstrating large differences in brain systems between males

and females (for a review see Cahill, 2006), the most relevant difference in

processing being, in many cases, that of emotional stimuli in anxiety.

It is interesting to note that the stimulus parameters and relative

locations we used in Experiment 1 closely match Bocanegra & Zeelenberg’s

83

(2009) configuration. Though they studied effects of emotion on spatial

frequency resolution and we studied effects of emotion and attention on

contrast sensitivity, we generally observed the same increase in perceptual

sensitivity with emotion alone (i.e., fear-distributed cues). Perhaps this

particular configuration used in both studies is especially well-suited for

demonstrating effects of emotion, but is relatively weak for investigating the

interaction of emotion with attention because attentional resources are not

strongly taxed. Nevertheless, even using this task design we demonstrated

that the emotion effects on attention and perception were driven by high trait

anxious females.

With the addition of two more possible target locations in Experiment 2,

we taxed spatial attention even further. For all observers, regardless of

gender, there was a benefit to contrast sensitivity with fear-valid cues relative

to fear-invalid cues. The same pattern was present with neutral cues but the

valid-invalid difference was not as great, suggesting that fearful faces more

strongly capture attention. Overall, the hypothesized benefits and costs of

attention to contrast sensitivity were augmented with four possible target

locations, and fear-valid cues significantly increased contrast sensitivity

beyond that of fear-distributed cues in low trait anxious observers. However

fear-valid cues did not significantly increase contrast sensitivity above that of

neutral-valid cues. A possible explanation for this is that Phelps et al. (2006)

84

tested fewer, more experienced observers (n=6) and collected significantly

more data from each person, over the course of six days, than the present

study. Collecting more data from a small group of experienced observers

could have lead to a more refined measurement, however, in the present case

more observers were needed in order to investigate the effects of anxiety and

sex.

In our previous work (Ferneyhough et al., 2010) that investigated the

effect of handedness on contrast sensitivity we used a similar experimental

design as Experiment 1, with two task locations. We found a significant

attention effect but no effect of facial expression, even when accounting for

negative affect (we collected PANAS self-report only). A plausible explanation

for this lies in the choice of Gabor target spatial frequency used in each study.

Specifically, we used 4 cpd Gabor targets in the 2010 study, whereas we used

2 cpd targets in the 2006 study, in which there was an interaction of emotion

with attention. Following Bocanegra & Zeelenberg’s 2009 study, we

hypothesized that the amygdala is more sensitive to the low spatial

frequencies in the fearful relative to neutral face cues (e.g., Vuilleumier,

Armony, Driver & Dolan, 2003), which then sends modulatory feedback to

ventral visual areas along magnocellular pathways (Amaral, Behniea & Kelly,

2003). This feedback may then preferentially augment processing of 2 cpd

relative to 4 cpd targets. Future work is needed to better understand the roles

85

of stimulus parameters, configuration and observer factors on how the

interaction of emotion and attention mediates perception.

In conclusion, emotion improves contrast sensitivity at attended

locations and impairs it at unattended locations when attentional resources are

limited. Furthermore, under less stringent attentional limitations, only high trait

anxious females demonstrate both benefits and costs of emotion and attention

on perception. Given that contrast sensitivity is one of the most basic

characteristics of the primary visual cortex, we show that emotion and

attention can modulate the actual perceptual signal representing a stimulus.

86

CHAPTER 3

The role of intraparietal sulcus in the emotional cost to temporal

attention

(Unpublished and not submitted)

Emma Ferneyhough1 & Elizabeth A. Phelps1,2

1New York University Psychology Department

2New York University Center for Neural Science

87

Abstract

The attentional blink (AB) task assesses the temporal limitations of

attention. In a rapid visual serial presentation, identifying a target stimulus

impairs identification of a second target stimulus that follows soon after (early

lag), but not later (late lag). Emotion has been shown to influence temporal

attention in two ways: (1) an emotional second target facilitates attention, and

(2) an emotional first target impairs attention. Previous research exploring the

neural systems of emotion’s facilitation of temporal attention has implicated a

role for the amygdala in driving bottom-up emotional responses. The goal of

the present study was to explore the neural systems linked to the emotional

cost of temporal attention, which may involve competition between bottom-up

and top-down components. Whereas the orbitofrontal cortex (OFC) and the

intraparietal sulcus (IPS) have been implicated in spatial attention costs, the

dorsolateral prefrontal cortex (DLPFC) is important for attentional control

processes. In Experiment 1, we replicate the behavioral emotional cost to

attention. In Experiment 2, we conducted an AB task in the scanner

investigating the role of these regions in the interplay of bottom-up and top-

down attentional processes that may underlie emotional costs. Emotional or

neutral distracter words appeared 3, 4, 7 or 8 lags prior to a single neutral

target. Lags 3 and 4 were within the AB window (early: 270, 360 ms

88

respectively) whereas lags 7 and 8 appeared outside (late: 630, 720 ms

respectively). Participants provided self-report anxiety and attentional control

measures. We observed greater activation in (1) the IPS, OFC and amygdala

for emotional distracter trials, and (2) DLPFC for neutral distracter trials, at

early, but not late, lags. Moreover, amygdala activity differences were driven

by high state anxiety whereas DLPFC activity differences were driven by low

trait anxiety and high attentional control, consistent with prior research. Thus,

IPS and OFC may be part of a frontoparietal network underlying attentional

costs with emotion, not only in spatial, but also temporal domains.

Acknowledgements

We would like to thank research assistants Rita Ludwig and Caroline McClave

for their help in collecting and processing data and all Phelps and Carrasco

Lab members for helpful feedback. This research was funded by Grant NIH

R01-MH062104 to E.A.P.

89

INTRODUCTION

Emotionally arousing information can both facilitate attention as well as

impair it. Being completely engrossed in a Hollywood blockbuster, for

example, can impair your ability to hear the doorbell announcing the pizza you

ordered just minutes before. Thrillers have thus perfected the art of capturing

our attention, melding together fast-paced story arcs full of action, violence

and sex. These elements engage us by eliciting emotions associated with

physiological arousal, like fear, anger and surprise. Greater engagement by

emotional stimuli, however, is accompanied by the exclusion of the low-

arousal, mundane aspects of everyday life. Costs of emotionally arousing

stimuli to attention and perception of the mundane have been studied

experimentally in both the spatial and the temporal domains, but our

understanding of the neural mechanisms underlying this phenomenon is

incomplete (for a review, Stanley, Ferneyhough & Phelps, 2009).

In the current study we use an attentional blink task (AB: Raymond et

al, 1992) to investigate the neural correlates of costs of emotion to attention in

the temporal domain. This task, in which target stimuli are embedded in a

rapid serial visual presentation (RSVP) of masking stimuli, has been used

extensively to study the temporal limitations of attention (for a review, Dux &

Marois, 2009). The AB is an impairment in correctly identifying a second target

90

stimulus (T2) that has appeared within 500 ms of correctly identifying a

preceding first target stimulus (T1). As the amount of time between T1 and T2

increases, lessening attentional demands, performance on T2 identification

progressively improves.

When T2 is an emotionally arousing stimulus, however, the AB is

attenuated at short T1-T2 time intervals relative to when T2 is a neutral

stimulus (Anderson, 2005; Anderson & Phelps, 2001; De Martino, Strange &

Dolan, 2008; Fox, Russo & Georgiou, 2005; Mathewson, Arnell & Mansfield,

2008). In this case, emotion facilitates performance when the task-relevant

item is emotional. For this AB ‘facilitation’ effect, it has been proposed that the

amygdala plays a crucial role in enhancing bottom-up processing (Anderson,

2005; Anderson & Phelps, 2001; Lim, Padmala, & Pessoa, 2009; Schwabe,

Merz, Walter, Vaitl, Wolf & Stark, 2010). The amygdala is important for

emotion processing broadly (for a review, Phelps 2006), and has feedback

connections throughout ventral visual cortex (Amaral et al., 2003) including

both primary visual cortex and ventral occipitotemporal regions involved in

visual word form recognition (e.g., Cohen, Dehaene, Naccache, Lehéricy,

Dehaene-Lambertz, Hénaff & Michel, 2000). Re-entrant processing from the

amygdala could enhance early activity (<300 ms post-stimulus onset) in these

regions (e.g., Kissler, Herbert, Peyk & Junghofer, 2007; Luo, Peng, Jin, Xu,

Xiao & Ding, 2004) improving the perception, and identification accuracy, of

91

emotional T2 stimuli. Consistent with this idea, a recent fMRI study of

emotion’s interaction with spatial attention has shown increased visual cortical

activity following fear-valid relative to fear-invalid face cues (Pourtois et al,

2006). Another fMRI study using fear-conditioned cues found both enhanced

amygdala and visual cortical activity (Armony & Dolan, 2002). Moreover, left

amygdala patients do not experience attenuation of the AB with emotional T2

stimuli (Anderson & Phelps, 2001), providing further support for the idea that

the facilitation effect is due to enhanced bottom-up processing of emotion.

An alternative variant of the AB paradigm requires identifying a single

neutral target preceded by an emotionally arousing, task-irrelevant distracter.

Instead of facilitating performance, these to-be-ignored emotional distracters

impair target identification and produce an AB at short distracter-target

intervals. This ‘capture’ task shows that even ignored emotional stimuli can

automatically capture attention and divert it from neutral, task-relevant targets

(Arnell, Killman & Fijavs, 2007; Keil & Ihssen 2004; Most, Chun, Widders &

Zald, 2005; Most, Smith, Cooter, Levy & Zald, 2007; Smith, Most, Newsome &

Zald, 2006). Neutral distracters, on the other hand, are easily disregarded.

The analogous capture effect in studies of spatial attention occurs when

emotional attention cues direct attention away from target stimuli, resulting in a

delayed disengagement from emotional, relative to neutral, cues before

attention is reoriented to the target. The magnitude of the delayed

92

disengagement effect on task performance has been found to correlate

positively with anxiety (e.g., Fox et al., 2001) and negatively with attentional

control (Derryberry & Reed, 2002), a measure of one’s ability to ignore

distractions and regulate attention allocation (for reviews: Bar-Haim, Lamy,

Pergamin, Bakerman-Kranenburg & van IJzendoorn, 2007; Weierich, Treat &

Hollingworth, 2008). Similarly, the degree to which task-irrelevant emotional

distracters disrupt target processing in the AB have also been shown to

depend on self-reported anxiety (Most et al, 2005). Higher anxiety is thus

associated with a greater inability to suppress processing of emotional

information (Fox et al, 2005).

It is believed that this inability to suppress emotion processing, both in

the spatial and temporal domains, is due to failures in top-down attentional

control, functions of prefrontal and parietal cortices (for a review: Cisler &

Koster, 2010). When emotional distractions automatically capture attention

through facilitative bottom-up mechanisms centered on the amygdala,

reorienting attention back to a task-relevant target likely requires top-down

cognitive processes. These processes may involve both suppressing

emotional responses and engaging regions important for voluntary attention.

Whereas the emotional facilitation effect in the AB is thought to be a result of

bottom-up enhancement of target stimuli, the neural mechanisms underlying

the emotional capture of attention may involve competition among regions

93

important for both bottom-up and top-down information processing (e.g.,

Desimone & Duncan, 1995; Pessoa, Kastner & Ungerleider, 2002). One

imaging study has investigated costs of task-irrelevant emotional distracters in

the AB (Most et al., 2006), however, it leaves the nature of this competition

unclear.

The main question addressed in the study of Most and colleagues’

(2006) concerned how attentional modulation of amygdala activity is correlated

with self-reported harm avoidance. Attention was manipulated with task

instructions that resulted in participants forming either a ‘specific’ or ‘non-

specific’ attentional set as they searched for a target during an RSVP task.

The authors found that decreased amygdala and increased rostral anterior

cingulate cortex (rACC) activity were linked to trials with emotional distracters,

but only among high harm avoidant (similar to anxious) participants who were

maintaining a specific attentional set (Most et al., 2006). Previous research

has suggested that rACC can inhibit the amygdala response (Kim & Whalen,

2009). Consistent with this, Most et al. (2006) suggested that increased rACC

activity is indicative of the extra effort high harm avoidant participants must

exert in order to ignore emotionally distracting stimuli, presumably by down-

regulating the amygdala. In contrast, while maintaining a non-specific

attentional set the same participants showed increased amygdala and

decreased rACC. Moreover, the authors found that the degree to which

94

dorsolateral prefrontal cortex (DLPFC) activation was modulated by attentional

set depended on self-reported attentional control. Individuals with higher

attentional control engaged the DLPFC more during the ‘specific’ relative to

‘non-specific’ attentional set, consistent with DLPFC’s known role in top-down

attentional control functions (e.g., Macdonald, Cohen, Stenger & Carter,

2000).

These findings by Most et al. (2006) confirm the role of the amygdala in

response to emotional distracters, and suggest roles for the rACC in inhibiting

the amygdala’s impact on perception, and the DLPFC in maintaining attention

on task-relevant goals. However, it is important to note that all imaging

analyses were performed on trials without a target, and only early, but not late,

lags were included in the task. As such, these results do not have a direct

bearing on the behavioral emotional capture of attention effect, i.e., reduced

accuracy on emotional relative to neutral distracter trials, at early but not late

lags. In fact, no imaging study to date has specifically investigated the neural

mechanisms underlying costs of task-irrelevant, emotional distracters to

attention at early vs. late lags. Thus, the question of how bottom-up amygdala

signals may interact with brain regions underlying shifts of attention in the AB

has remained unanswered.

Although little is known about the temporal cost of emotion to attention,

studies investigating costs of emotion to spatial attention may provide useful

95

information regarding which brain areas are involved in shifting attention

towards emotional distracters. One study found that both benefits and costs of

emotion to spatial attention increased intraparietal sulcus (IPS) and lateral

orbitofrontal cortex (OFC) activity, and costs alone increased posterior OFC

activity (Armony & Dolan, 2002). The authors conclude that amygdala

responses to emotional stimuli are relayed to the posterior OFC during invalid

trials. The OFC then modulates activity in IPS, which re-directs attention

towards the location of target stimuli. Another study similarly showed

increased lateral OFC activation for costs of emotion, and decreased IPS

activity to invalidly cued targets (Pourtois, Schwartz, Seghier, Lazeyras &

Vuilleumier, 2006). Results from both spatial studies are consistent with OFC’s

role in the re-orienting of attention after unexpected events (Coull, Frith,

Buchel & Nobre, 2000; Nobre, Coull, Frith & Mesulam, 1999). Furthermore,

these results are consistent with the rACC results of Most et al. (2006). OFC

and rACC are highly interconnected brain regions. Both regions additionally

share extensive reciprocal connections with the amygdala (Bush, Luu &

Posner, 2000) and have shown modulatory effects on the amygdala across a

range of tasks (e.g., Kim & Whalen, 2009; for reviews, Hartley & Phelps, 2010;

Quirk & Mueller, 2008). Given these results, IPS and OFC/rACC may be

important brain areas underlying emotional costs to attention, not only in the

spatial domain, but the temporal domain as well, specifically within the AB.

96

In the present study, we use fMRI to explore the neural systems

underlying costs of emotion in the AB to answer the following questions. 1)

How do emotion-sensitive brain regions such as the amygdala modulate

bottom-up and top-down attention systems to result in emotional costs at early

but not late lags? 2) What are the roles of IPS, OFC and rACC in these costs?

3) And how does anxiety or attentional control affect emotional costs to

attention in the brain?

Given the findings from the emotional facilitation of the AB, and the

spatial attention invalidity effects, we are able to make the following

predictions. We hypothesize that goal-driven attention used to search for a

target in the emotional capture of attention task may be disrupted in a bottom-

up fashion by emotional stimuli, resulting in the AB at short, but not long,

distracter-target intervals. For example, while looking for a target word

embedded in an RSVP of distracter words, DLPFC, which is known to play a

role in maintaining task-relevant goals (e.g., Corbetta & Shulman, 2002), may

be engaged. However, when attention is automatically captured by an

emotional distracter word, the amygdala may become more active. While the

amygdala does not have direct projections to DLPFC (Barbas 2000;

McDonald, Mascagni & Guo, 1996) or IPS (Saygin, Osher, Augustinack, Fischl

& Gabrieli, 2011), it can signal the presence of the emotional conflict to

OFC/rACC which may serve as a hub and momentarily alter responses in

97

attentional control regions such as DLPFC or IPS (Yamasaki, LaBar, &

McCarthy, 2002). Depending on the relative strength of the bottom-up

emotional disruption to the top-down attentional control, as well as the

distracter-target lag, the visibility of the target word is determined. At short time

intervals, we expect that emotional distracters will have a greater influence on

this competition and decrease target word visibility during the AB, while at long

time intervals we expect less impairment due to top-down attentional control

mechanisms coming online, refocusing attention on the task at hand.

In addition, we expect that individuals who score high on anxiety

measures will have less attentional control and greater behavioral costs, than

lower scoring individuals. This will be accompanied by greater amygdala

activity in response to emotional stimuli and attenuated activity in regions

important for attentional control such as DLPFC, consistent with research

showing amygdala activity is correlated with anxiety (Bishop, Duncan &

Lawrence, 2004) and DLPFC activity is correlated with attentional control

(Bishop, Duncan, Brett & Lawrence, 2004; Most et al., 2006). This model

includes elements of previously proposed models (Bishop 2007; Taylor &

Fragopanagos, 2004, 2005), however, here we have the opportunity to test it

using the emotional capture of attention AB task in the scanner.

In Experiment 1 we test a variant of the emotional capture of attention

AB task in which participants make a 4-alternative forced choice (4AFC)

98

response with the goal of replicating previous behavioral findings (e.g.,

Mathewson et al., 2008). In Experiment 2 we test the proposed model by

measuring BOLD signal as participants complete the task in the scanner.

EXPERIMENT 1: Task validation

In pilot studies not described here, we replicated the attention capture

by emotion effect previously found by other groups (Arnell, Killman & Fijavs,

2007; Keil & Ihssen 2004; Mathewson et al, 2008; Most, Chun, Widders &

Zald, 2005; Most, Smith, Cooter, Levy & Zald, 2007; Smith, Most, Newsome &

Zald, 2006). To bring the experiment into an fMRI environment without the use

of a full keyboard to identify target words, we changed the word identification

task to be a 4AFC task. The purpose of Experiment 1 was to validate this task

for the scanner.

METHODS

Participants

Twenty undergraduates from the NYU Psychology subject pool

participated for course credit. Two participants were later excluded because

they learned English as a second language. Given that we used English words

99

as stimuli, one of our requirements for participation was that English be the

first language learned to reduce interference of the first acquired language.

Thus, we analyzed data from 18 total participants (13 female).

Apparatus

The experiment was conducted on a Windows PC running MATLAB

(The Mathworks, Natick, MA) and Psychophysics Toolbox (version 3;

Brainard, 1997). Observers made responses by choosing numbers 1-4 on the

top row of the keyboard with their index through pinky fingers.

Stimuli

The same word stimuli as was used in Anderson (2005) were used here

with supplemental words (see Appendix C). Highly arousing words that could

be either negative or positive in valence were used as emotional distracters,

and words with neutral valence and low arousal were used as neutral

distracters, targets, and maskers in our RSVP. The target was printed in one

of 16 shades of green by changing the RGB levels to manipulate ease of

visibility, which was adjusted for each individual observer to keep performance

between 65 and 75% correct on average. After each block, accuracy was

assessed and the visibility was adjusted to maintain a challenging level. The

distracter and filler words were printed in black. The background was mean

100

gray. All words were in Arial font, size 38. The distracter word was either

neutral or emotional with equal probability, and the target word was always

neutral. The black masking words were 10 letters in length on average. Both

the distracter and target words were 5 letters in length on average.

Procedure

Participants were seated in front of the computer with an approximate

viewing distance of 40 cm. After 2 short practice blocks where only neutral

words were presented, participants completed 4 blocks of 56 trials each.

Within each trial, a fixation point appeared for 500 ms, followed by an RSVP

stream of 15 items. The RSVP stream contained 13 filler words, 1 distracter

word inserted in position 2, 3, 4 or 5, and 1 target word inserted 1, 2, 3, 4, 5, 6,

or 7 positions after the distracter. Each word was on the screen for 90 ms and

immediately replaced by the following word with no intervening blank. After the

RSVP stream completed, there was a 500 ms blank screen, then participants

were shown a list of 4 words and they had to choose which word matched the

target. There was 1500 ms to make a response by pressing 1 of 4 buttons with

their right hand (Figure 10, top panel).

101

Analysis

The data were divided by condition (emotion vs. neutral distracter) and

lag (positions 1-7 after distracter). Accuracy was averaged over every trial

within a condition at every lag. Average accuracy in the emotion condition was

compared to the average accuracy in the neutral condition for all 7 lags. A 2 x

7 repeated measures ANOVA was conducted on the data with distracter type

and lag as the two factors, and individual t-tests comparing the emotional and

neutral conditions for each lag was performed.

RESULTS

The ANOVA revealed significant main effects of distracter emotion

(F(1,17)=17.55, p<0.001), and lag (F(6,17)=25.67, p<0.001) as well as a

significant interaction of the two factors (F(6,17)=3.592, p< 0.01; Figure 11).

The difference in performance for lags 3, 4, 5, and 6 were all individually

statistically significant (ps<.01 at the two-tailed level, except lag 5 p<.05 at the

one-tailed level) with neutral distracter trials leading to greater accuracy

(Figure 11, left panel). The strongest effect was found at lag 4 (t(17)=4.35,

p=0.0004), or 360ms after presentation of the distracter. In addition we

averaged target accuracy separately for neutral and emotional distracter trials

102

over lags 3-4. Performance for these two types of trials was significantly

different: neutral distracter trial performance was 81%, while for emotion

distracters performance was 68% (t(17)=4.58, p<0.001). Importantly,

performance for lag 7 was the same for emotion and neutral distracter trials

(77%; p>0.1; Figure 11, right panel).

EXPERIMENT 1 DISCUSSION

This task successfully produced a strong capture effect at early lags,

but not late lags. These results provided the information we needed regarding

the efficacy of using a 4AFC task, so we felt free to further modify the

experiment for use in the scanner in a second pilot experiment not described

here. To maximize the amount of data we could collect in the shortest amount

of time in the scanner, we tested a simplified version of Experiment 1, which

included only 2 distracter positions (3 and 5) and 4 distracter-target lags (3, 4,

7, and 8). We used lag 8 as a second “late” lag, given the results showing lag

6 still showed a cost of emotion. In addition, we introduced a jittered ITI in a

rapid event-related design, which allowed us to include trials closer together in

time without having to wait for the brain’s hemodynamic response to return to

baseline between every trial. The results of this second pilot replicated

103

Experiment 1, and showed enhanced attention capture by emotion with high

trait anxiety relative to low trait anxiety, indicating that participants with

elevated trait anxiety should be preferentially recruited for Experiment 2 in the

scanner.

EXPERIMENT 2: fMRI component

METHODS

Participants

31 naïve observers (19 female) from the NYU community participated in

this study. They were screened to ensure they are native English speakers

with corrected or normal vision, no history of psychiatric disorder, not on any

psychotropic medication, and met all criteria for safe scanning. In addition,

given pilot results indicating that individuals high in trait anxiety and negative

affect show the strongest emotion capture effect, we mainly recruited

volunteers from an Introduction to Psychology course whose trait anxiety

scores on the 40-item State-Trait Anxiety Inventory (STAI: Spielberger,

Gorsuch, Lushene, Vagg & Jacobs, 1983), and whose negative affect scores

on the 20-item Positive and Negative Affect Scale (PANAS: Watson, Clark &

104

Tellegen, 1988), were in the top half of the total range (collected several

months in advance of this study in a general battery of surveys).

Apparatus

A 3-Tesla Siemens head-only scanner housed in the NYU Center for

Brain Imaging was used for collecting functional (T2*-weighted EPI) and

anatomical (T1-weighted) data. For each participant, 830 volumes of functional

data were collected (166 volumes/run x 5 runs), with each volume consisting

of 34 interleaved 3mm slices oriented approximately parallel with the

anterior/posterior commissure (inplane resolution = 3mm2, interslice gap =

0mm, flip angle = 82°, TE = 15ms, TR = 2s) providing whole-brain coverage in

most participants. Anatomical data had a resolution of 1mm3. A Dell PC

computer running MATLAB 7.5 (The MathWorks, Natick, MA) and the

Psychophysics Toolbox (version 3; Brainard, 1997) controlled timing of

stimulus presentation. The display was back-projected into the bore of the

magnet via an Eiki LC-XG250 projector approximately 57 cm from the

observers’ eyes. Observers made their choices using 1 of 4 possible buttons

on a button-box in their right hand.

105

Stimuli

Same as in Experiment 1 (see Appendix C).

Procedure

On the day of the scan participants completed 2 short practice blocks

(20 trials each) using only neutral distracter words outside the scanner to

familiarize themselves with the task. Participants then completed 5 functional

runs of the task containing 32 trials each (4 trials per condition per run) in the

scanner. See Appendix C for detailed task instructions. The trial structure was

identical to that of Experiment 1, except for (1) an additional 8-second initial

fixation period and 12-second ending fixation period appended to the

beginning and end of each run, (2) the length of the inter-trial-intervals (ITIs)

were jittered and could range from 2 to 16 seconds long (Figure 10, bottom;

ITI duration and trial ordering was optimized using optseq2:

http://surfer.nmr.mgh.harvard.edu/optseq/), and (3) only lags 3, 4, 7 and 8

were tested. There was a ~10 minute T1-MPRAGE structural scan after

completion of the 5 runs, to which functional data were aligned in order to

localize active brain regions.

All observers filled out the following self-report measures at the

experiment’s conclusion: the PANAS (Watson et al., 1988), the STAI

106

(Spielberger et al., 1983), the 20-item Attentional Control Scale (ACS:

Derryberry & Reed, 2002), and the Edinburgh Handedness Inventory (EHI;

Oldfield, 1971). The PANAS was used to assess the degree to which different

positive and negative emotions were experienced in general over the previous

six months, and scores could range from 10 to 50 within either positive or

negative affect. The STAI was used to assess degree of anxiety at the present

moment (state) and in general (trait), and scores could range from 20 to 80

within either state or trait anxiety. The ACS was used to assess the degree of

attentional control participants have over distractions, and scores could range

from 20 to 80. EHI scores could range from -100 (completely left-handed) to

+100 (completely right-handed) and were collected in the case of differences

in functional lateralization due to handedness.

Behavioral Analysis

Each participant completed 160 experimental trials, equally divided

among 8 conditions, created by crossing 2 distracter types (emotional, neutral)

with 4 lags (3, 4, 7, 8). The emotional experiment conditions will be referred to

as E3, E4, E7, and E8. The neutral experiment conditions will be referred to as

N3, N4, N7 and N8. Since distracter position is not a variable of interest, data

were averaged over both distracter positions 3 and 5. Lags 3 and 4 comprised

the “early” component (the AB period) and Lags 7 and 8 comprised the “late”

107

component. Accuracy and RT was averaged over every trial within a condition

at both early and late lags.

Functional and Anatomical Data Preprocessing

The first 2 TRs (4 s) of each functional run were discarded before

preprocessing. Data quality of each functional run was visually inspected in a

time course movie to note any major movement. Functional runs then

underwent the following preprocessing steps: (1) slice scan time correction, (2)

temporal filtering (linear trend removal and high pass filter of 3 cycles), and (3)

motion correction to the last image closest to the T1-MPRAGE. No participants

were excluded due to excessive (>3 mm) movement. For group whole-brain

analyses, the functional data were spatially smoothed (Gaussian kernal spatial

smoothing of 6 mm full width half maximum).

The T1-MPRAGE for each participant was cleaned, AC-PC aligned,

and then morphed into Talairach space (Talairach & Tournoux, 1988). The

anatomical data were then co-registered with the corresponding functional

data.

BOLD Response Analysis

Based on our a priori hypotheses regarding the brain regions involved

in the emotional cost to attention, we defined regions of interest (ROIs) in the

108

DLPFC, OFC and rACC using a whole-brain contrast of all task-related activity

greater than baseline. IPS was defined individually for each subject using the

same contrast (with a separate anterior and posterior region on the left side)

due to the variability in the location of IPS activation across subjects. Given

that we used a contrast that revealed regions involved in the task generally,

ROI definition was orthogonal to our specific contrasts of interest. The

amygdala was defined two ways: (1) anatomically based on each subject’s T1

scan, and (2) based on a contrast of all emotional trial-related activity greater

than all neutral trial-related activity. The mean Talairach coordinates and

number of voxels for each region are listed in Table 1A (Table 2B for

amygdala ROIs based on E>N contrast). We performed convolution analyses

using these ROIs, in which we estimated betas for each trial type. The

resultant betas from conditions E3 and E4, as well as E7 and E8, were

averaged together to form the “early” and “late” components, respectively. The

same was done for the corresponding neutral conditions.

Whole-brain contrasts were conducted across all participants to

observe global brain activity correlated with main effects and interactions of

the 2 experimental factors (distracter type and lag) to serve as confirmation of

the ROI results. Multi-subject design matrices containing stick predictors

109

convolved with a hemodynamic response function (HRF)1 were constructed to

fit a random effects general linear model to the 3D functional data using

BrainVoyager 2.0 (Brain Innovation, Maastricht, the Netherlands). The stick

predictors’ onsets were placed at the beginning of the RSVP for each trial. The

first 8 predictors in the design matrix corresponded with the 8 conditions (as

defined under Behavioral Analysis), and the last predictor was a constant2.

The fixation/ITI periods served as baseline. The following whole-brain

contrasts were conducted:

Contrast Effect 1 E3+E4+E7+E8+N3+N4+N7+N8 task-related activity 2 E3+E4+E7+E8 > N3+N4+N7+N8 main effect of emotion 3 E3+E4+N3+N4 > E7+E8+N7+N8 main effect of attention 4 E3+E4 > N3+N4 simple effect of emotion (early lags) 5 E7+E8 > N7+N8 simple effect of emotion (late lags) 6 N3+N4 > N7+N8 simple effect of attention (neutral) 7 E3+E4 > E7+E8 simple effect of attention (emotional) 8 E3correct + E4correct >

E3incorrect + E4incorrect simple effect of accuracy (early emotion lags)

1 The HRF used in the convolution of the design matrices had a peak response at 10 seconds post-stimulus-onset due to a data processing error, however, the standard HRF has a peak response at 6 seconds. New analyses will be conducted using a 6 s HRF, and the results are expected to be similar, and may even be stronger. 2 An additional analysis was conducted including reaction time as a regressor of no interest to ensure that any activity revealed in whole-brain contrasts was due to our contrasts of interest rather than a main effect of RT. Contrasts using this GLM revealed overlapping regions of activation compared to when RT was not entered in the model, suggesting that differences in brain activity are not due only to systematic differences in RT from trial to trial.

110

RESULTS

Data from three participants (1 female) were discarded. The first

participant failed to make a response on over 25% of the trials; the second

scored more than 2 SDs away from the mean on both the PANAS and STAI,

and 2 out of 8 experiment conditions; and the third scored more than 2 SDs

away from the mean on 2 out of 8 conditions. 1 out of 28 participants had a

negative EHI score indicating left-handedness, however, their behavioral and

imaging data were no different from the other 27 participants so they were not

excluded. Behavioral and imaging data of the remaining 28 participants were

analyzed.

Self-Report

Scale means and standard errors are listed in Table 1B. ACS and

STAI-T were negatively correlated (r(28)=-.38, p<.05), confirming prior

research showing an inverse relationship between attentional control and trait

anxiety. Positive affect (PA) and negative affect (NA) were positively

correlated (r(28)=+.44, p<.05), and PA and STAI-S were negatively correlated

(r(28)=-.39, p<.05).

111

Target Identification Accuracy

A 2x2x2 repeated measures ANOVA on participants’ accuracy data

with distracter-target lag (early, late) and distracter type (emotional, neutral) as

within-subjects factors, and trait anxiety (low, high) as a between-subjects

factor was conducted. (Attentional control was not included as a factor

because it was significantly correlated with trait anxiety, but for the imaging

data both were analyzed separately.) A main effect of lag was revealed

(F(1,27)=5.98, p<.05) in which early lags resulted in decreased accuracy

compared to late lags. The main effect of distracter type was marginally

significant (F(1,27)=3.63, p<.1), with emotional distracter trials resulting in

decreased accuracy compared to neutral distracter trials. In addition there was

a main effect of anxiety (F(1,26)=5.13, p<.05) in which high trait anxious

individuals had greater accuracy overall than low trait anxious. Furthermore,

lag and distracter type significantly interacted (F(1,27)=5.66, p<.05). At the

early lag, neutral distracters resulted in greater target identification accuracy

than emotional distracters (t(27)=2.7, p<.05), however at the late lag, there

was no difference in accuracy (Figure 12, left panel). Additionally, early

emotion trials resulted in significantly decreased accuracy relative to late

emotion trials (t(27)=-3.14, p<.01). Across all four tested lags (3, 4, 7 and 8),

the difference between neutral and emotional distracter trials was greatest at

lag 4, whereas the magnitude of the difference at lags 7 and 8 were

112

equivalent.

Although neither STAI-T nor ACS was correlated with the size of the

emotional capture of attention effect, only high trait anxious individuals

showed the significant effects described above. In addition, PA was

significantly and negatively correlated with the behavioral effect (r=+.48, n=28,

p<.05). Thus, a greater behavioral cost was associated with lower self-

reported positive affect.

Reaction Time

A 2x2 repeated measures ANOVA on participants’ reaction time data

showed a main effect of lag (F(1,27)=6.63, p<.05) in which early lags resulted

in increased response latency compared to late lags. There was a marginal

main effect of distracter type (F(1,27)=3.79, p<.1) in which emotional

distracters resulted in slowed response times. Lag and distracter type

interacted as well (F(1,27)=6.12, p<.05) with early emotion trials resulting in

slower reaction time than early neutral trials (t(27)=2.53, p<.05) and late

emotion trials (t(27)=3.01, p<.01; Figure 12, right panel). Across all four tested

lags (3, 4, 7 and 8), the difference between neutral and emotional distracter

trials was greatest at lag 3, whereas the magnitude of the difference at lags 7

and 8 were equivalent.

113

Regions of Interest Analysis

In addition to examining the beta values derived from our ROI analyses

across the entire sample, we also divided the sample based on median splits

of the self-report measures. Presented below are the mean results across all

subjects, and analysis for both anxiety and attentional control.

Amygdala

Given that the amygdala is sensitive to emotionally salient information,

and may be the source of bottom-up emotional facilitation effects in the AB, we

hypothesized that amygdala activity would increase in response to emotional

distracter trials relative to neutral. Greater amygdala activity to emotional

distracter trials may then signal the presence of emotional stimuli to OFC and

rACC. Consistent with this hypothesis, a contrast of all emotional distracter

trials vs. neutral trials (Contrast 2; Table 2B) revealed activation in bilateral

amygdala. No other contrasts involving interactions with attention resulted in

amygdala activation, indicating that the amygdala response was specific to

emotional distracters regardless of lag.

Two ROI analyses were conducted on bilateral amygdala. The first

used anatomically defined ROIs, based on each subject’s T1 scan. The

second used functionally defined ROIs, based on the above contrast of E>N

across all subjects. Betas for each condition were extracted from each ROI for

each subject, and these betas were then averaged together. The results of

114

both ROI analyses yielded similar results. Both showed a main effect of

emotion, confirmed in repeated measures ANOVAs on the betas with

distracter type, E or N, and lag, 3, 4, 7 or 8, as within-subjects factors

[functional ROIs: R amygdala: F(1,27)=5.11, p<.05; L amygdala:

F(1,27)=11.96, p<.001] [anatomical ROIs: R amygdala: F(1,27)=4.88, p<.05; L

Amygdala: F(1,27)=4.9, p<.05].

Upon closer inspection with paired t-tests, it was found that right

amygdala activity was greater for early emotional relative to neutral trials

(functional ROI: t(27)=1.83, p<.05 one-tailed; anatomical ROI: t(27)=2.54,

p<.05). This difference was driven by lag 3 trials in which there was greater

activity in lag 3 emotion compared to neutral trials (functional ROI: t(27)=2.73,

p<.05; anatomical ROI: t(27)=3.15, p<.01).

Intraparietal Sulcus

Consistent with its role in top-down shifts of attention, we expected IPS

activity to be differentially modulated during early emotional vs. neutral

distracter trials. IPS may direct attention towards emotional distracters via top-

down signals from OFC and rACC.

Bilateral posterior IPS (left n=28, right n=27; right IPS could not be

defined in one subject even at very low thresholds) showed significantly

greater activity for emotional distracter trials relative to neutral at early lags

115

(left t(27)=2.36, p<.05; right t(26)=2.47, p<.05), but not at late lags (Figure 13).

In addition, early emotional distracter trials resulted in greater activity than late

emotional distracter trials (left t(27)=3.24, p<.01, right t(26)=2.79, p<.01). This

was not the case for the neutral distracter condition. Left anterior IPS (n=24)

showed the same pattern of results. Early emotion was greater than early

neutral (t(23)=2.06, p=.05) and early emotion was greater than late emotion

(t(23)=2.97, p<.01). In both right and left (posterior and anterior) IPS, this

pattern was driven by greater activity for emotional distracter trials at lag 3 (all

ts>2.45, ps<.05), but not lag 4 (all ps>.1).

When the data were examined based on median splits of the anxiety

and attentional control self-report measures, no obvious pattern emerged,

suggesting that the activity differences were not driven by any particular group

in any of the 3 IPS ROIs.

Dorsolateral Prefrontal Cortex

DLPFC has been shown to be broadly involved in attentional control

functions in the absence of emotional stimuli. Given that greater attentional

control is needed during early lag trials within the AB, we expected greater

DLPFC activity in early vs. late neutral distracter trials. For emotional distracter

trials, we expected that DLPFC activity would decrease due to inhibitory

connections with OFC/rACC. Based on previous research showing increased

116

DLPFC activity with increased self-reported attentional control (Most et al.,

2006), we also predicted that DLPFC activity would be modulated by degree of

self-reported attentional control.

Consistent with our hypotheses, right DLPFC (n=28) showed

decreased activity in emotional distracter trials at early lags relative to neutral

(t(27)=-2.38, p<.05; Figure 14). In addition, early neutral distracter trials

resulted in decreased activity relative to late neutral trials (t(27)=2.51, p<.05).

These differences were driven by increased activity for neutral distracter trials

relative to emotional at lag 4 (t(27)=2.92, p<.01), but not lag 3 (p>.1), though

the comparison is in the same direction at lag 3. The difference between lag 3

and lag 4 emotion trials was marginally significant (p<.1), with a greater

reduction in DLPFC activity at lag 4. A whole-brain contrast of early emotional

greater than early neutral revealed negative activation in right DLPFC,

corroborating these ROI results.

When the data were examined based on median splits of the anxiety

and attentional control self-report measures, these differences were only

significant for individuals low in trait (early lags, emotional vs. neutral: t(13)=

4.28, p<.001; early neutral vs. late neutral: t(13)=3, p<.05) and state (early

lags, emotional vs. neutral: t(13)= 3.66, p<.01) anxiety, and high in attentional

control (early lags, emotional vs. neutral: t(13)=2.53, p<.05; early neutral vs.

late neutral: t(13)=2.55, p<.05). High positive affect led to greater differences

117

between emotional and neutral at early lags, while low positive affect led to a

greater difference between early and late neutral trials. No significant

differences between emotional vs. neutral, or early vs. late, conditions were

found within the left DLPFC ROI.

Orbitofrontal Cortex

The OFC receives input from the amygdala and was expected to play a

role in relaying emotional signals to DLPFC and IPS during early emotional

distracter trials. No significant differences between emotional vs. neutral, or

early vs. late, conditions were found within the OFC ROI. However, the main

whole-brain contrast of interest comparing emotional distracter trials vs.

neutral trials at early lags only (Contrast 4; Table 3B) revealed activation in

the OFC, consistent with our hypothesis.

Rostral Anterior Cingulate Cortex

Given that the rACC has been implicated in the processing of emotional

stimuli in cases of attentional conflict, we hypothesized that rACC would be

preferentially active for early emotional distracter trials. Partially inconsistent

with this notion, our ROI analyses revealed rACC activity was marginally less

active in early vs. late trials (p<.1, one-tailed), but was not differentially active

for emotional vs. neutral conditions. Furthermore, no whole-brain contrast

comparing emotional vs. neutral distracter trials showed rACC activity.

118

Comparing all late vs. early lags (Contrast 3; Table 3A), on the other hand,

revealed a single activation in right rACC. Restricting this contrast to only

neutral (Contrast 6; Table 3D) or only emotional (Contrast 7, Table 4A) trials

revealed very similar regions of right rACC. Overall, right rACC was

preferentially active in late lag trials regardless of whether the distracter was

emotional or neutral.

Early Emotion Performance Analysis

We had hypothesized that during early emotional distracter trials,

emotion signals from the amygdala would activate OFC, which would then

inhibit DLPFC activity while exciting IPS activity. Given DLPFC’s role in

attentional control and IPS’s role in attention shifts, we predicted less DLPFC

activity would lead to more errors and greater IPS activity would lead to

greater orienting towards emotional distracters, also leading to more errors. To

test this hypothesis, a GLM was fit to the data including predictors for only

early emotional distracter trials differentiating between correct and incorrect

responses on each of these trials (E3correct, E3incorrect, E4correct,

E4incorrect).

Consistent with the hypothesized role of IPS activity increasing during

trials in which emotional distracter words captured attention leading to

incorrect responses, bilateral IPS is shown to be more active during early

119

emotional incorrect trials (Table 4B). In addition, a region that we did not

previously report, medial PFC, is shown to be more active for correct vs.

incorrect early emotional trials. This is consistent with prior research showing

that medial frontal cortex can regulate amygdala responses, suggesting that in

the early emotional distracter trials when this region is successfully recruited,

participants are able to correctly identify target words.3

EXPERIMENT 2 DISCUSSION

Our aim was to investigate the neural mechanisms underlying the

emotional cost to temporal attention in the attentional blink task. Based upon

previous research on the emotional facilitation effect in the AB, and on imaging

studies investigating emotional costs to spatial attention, we proposed a model

in which bottom-up emotional signals compete with top-down attentional

control for processing resources during early emotional distracter trials. We

hypothesized that the amygdala provides the bottom-up emotional signal. This

3 Previously reported DLPFC and OFC regions were not active in this contrast, suggesting that these regions play no role in whether a correct response is made during emotional trials but may be responding to the emotion itself. However, it could also be a result of insufficient power: only 18% of all trials resulted in an incorrect response.

120

signal may disrupt DLPFC control processes while engaging IPS to shift

attention towards distracters, through mediating OFC and rACC

interconnections. Given that the amygdala may be important for the bottom-up

emotional signal, whereas the frontoparietal regions mentioned above may be

important for the top-down response to task demands, we might expect

different patterns of brain activity underlying these respective functions. For

example, while the bottom-up response may not necessarily be modulated by

task demands, the top-down response may differ depending on distracter-

target lag, with early lags requiring observers to exert greater attentional

control to perform as well as in late lags. We also hypothesized that self-

reported anxiety and attentional control would modulate behavioral costs and

corresponding brain activity. Our results show that both the behavioral and

imaging results are largely consistent with these predictions.

We replicated the behavioral emotional capture of attention effect in

which emotional distracters produced a cost in target identification at early, but

not late, lags relative to neutral distracters (Mathewson et al, 2008; Keil &

Ihssen, 2004). In addition, we found that emotional costs to target identification

accuracy were greatest in high trait anxious individuals, suggesting that they

processed emotional stimuli to a greater extent than low trait anxious

individuals (e.g., Fox et al., 2005; Most et al., 2005). Furthermore, RT was

slowed in early emotional, relative to early neutral, trials. This indicates that

121

subjects were less confident of their responses as well as less accurate.

Consistent with our proposed model, we found that the amygdala was

maximally active for emotional distracter trials relative to neutral, regardless of

attentional demands. Although simple effects analysis of extracted beta values

from both anatomical and functional ROIs revealed stronger amygdala activity

to early vs. late emotional trials, these results did not survive whole-brain

contrasts or overall ANOVAs. In other words, significant amygdala activity was

not revealed in any contrast of early and late lags. Greater amygdala activity

thus facilitates the processing of emotional distracter stimuli at both early and

late lags. However, we propose that whether the emotional distracter impedes

target visibility is determined through the amygdala’s connections with regions

involved in shifts of attention under increased task demands.

Mirroring the amygdala results, we found that IPS was more active for

emotional distracter trials relative to neutral, however this was only true at

early lags. In addition, IPS activity was greater for early emotional, relative to

late emotional, trials. This suggests that the role of IPS in emotional costs to

spatial attention (Pourtois et al., 2006) generalizes to attentional selection in

time. Importantly this also provides evidence for the notion that IPS activity is

specifically modulated by emotion at early but not late lags, suggesting that

the availability of attentional resources determines the IPS response to

emotion. In other words, IPS activity in this task is not a general arousal

122

response. Studies of emotional costs to spatial attention have shown

decreased IPS activity in response to targets invalidly cued by fear faces

(Pourtois et al., 2006), and increased IPS activity in response to the emotional

cues themselves (Armony & Dolan, 2002; Pourtois et al., 2006). Together,

these results combined with the present findings demonstrate that IPS is

preferentially responsive to emotional relative to neutral stimuli under

increased attentional demands, whether they are spatial attention cues or

distracters in the AB task.

Emotional modulation of amygdala and IPS was accompanied by a

relative increase of right DLPFC activity for early neutral relative to both early

emotional and late neutral distracter trials. In contrast, there was no difference

in activity for early emotion vs. late emotion. Given the role of DLPFC in

attentional control (e.g., Corbetta & Shulman, 2002), it appears that this region

was successfully recruited during early neutral, but not emotional, distracter

trials, in order to maintain attention on the target identification task. In addition,

the DLPFC response during neutral trials was specific to early lags, indicating

that increased top-down control was required to meet the increased demands

of attention at early distracter-target lags. Although all early lag trials generate

a challenge for limited attentional resources, bottom-up emotion signals seem

to preferentially disrupt the ability of DLPFC to exert top-down control over

distracters.

123

How might bottom-up emotional signals from the amygdala interact with

top-down attentional signals from IPS and DLPFC? We show that IPS is more

responsive to early emotional relative to late emotional trials, which suggests

that its response is temporally specific. The amygdala is known to respond

quickly to emotionally arousing stimuli (e.g., LeDoux 1996) and this is

confirmed in our results. Hence, the initial IPS response may be driven by

bottom-up signals from the amygdala. Although the amygdala does not project

directly to IPS (Saygin et al., 2011) or DLPFC (Barbas 2000; McDonald et al.,

1996), it has reciprocal connections with OFC (Barbas 2000; Carmichael &

Price, 1995), a region that was more active for emotional vs. neutral trials

specifically at early lags. Modulation of activity in IPS and DLPFC regions may

thus be mediated by the OFC. Specifically, OFC may inhibit DLPFC

(Yamasaki et al., 2002) while engaging IPS activity (Morecraft, Geula &

Mesulam, 1993; Cavada & Goldman-Racik, 1989; Cavada, Compañy, Tejedor,

Cruz-Rizzolo & Reinoso-Suárez, 2000). During early emotional distracter

trials, increased IPS and reduced DLPFC-mediated attentional control could

lead to greater attentional orienting to emotional distracters and, ultimately, to

more errors in the word identification task. During late lag trials regardless of

distracter type, demands on attention are reduced, and less DLPFC-mediated

attentional control may be needed to perform accurately.

Two further analyses were conducted based on behavioral

124

performance. An analysis of accuracy was conducted to test whether our

regions of interest were differentially activated by correct vs. incorrect trials. A

contrast of early emotional incorrect vs. correct trials showed that bilateral IPS

was preferentially active, suggesting greater capture by emotional distracters

(Table 4B). In the opposite contrast, bilateral medial PFC activity was

revealed, suggesting that this region is recruited during correct trials, and may

influence activity in other regions to reduce the influence of emotional

distracters, such as IPS or amygdala. These results should be taken

cautiously however, as only 18% of the trials across all participants resulted in

an incorrect response. Another analysis, this time on RT, was conducted to

ensure that IPS activity was not solely driven by RT. A GLM with RT as a

regressor of no interest showed that the same regions (e.g., IPS) were active

in our contrasts of interest. This suggests that RT alone is not driving the

activity in these regions, however it may very well be the case that there is a

common underlying process, such as emotional capture of attention, that

drives both the RT and IPS activity differences. For example, when attention is

shifted away from a target word due to an emotional distracter, RT to identify

that target slows down due to a noisier perceptual signal.

Although the data are largely consistent with our proposed model, the

amygdala and IPS may also interact through an alternative visual cortical

route. Representations of emotional distracter words may be strengthened by

125

feedback from the amygdala (Freese & Amaral, 2005), consistent with the

greater visual cortex activity revealed in the early emotion vs. early neutral

whole-brain contrast (Table 3B). IPS may receive relatively greater

feedforward input from visual cortex in response to emotional vs. neutral

distracters. Greater attention may consequently be diverted to the emotional

distracter via either amgydala-OFC-IPS connections, or amygdala-occipital-

IPS connections, or both, resulting in poorer perceptual encoding of the

neutral target stimulus.

Somewhat inconsistent with our proposed model, we found that rACC

activity increased in all late compared to early lag trials, but was not

differentially modulated by emotional relative to neutral distracters (similar to

Yamasaki et al., 2002). Previous research has linked rACC to increased target

identification accuracy in the face of emotional and attentional conflict (De

Martino et al., 2009; Most et al., 2006; Schwabe et al., 2010). These studies,

however, did not include late lag trials, precluding a comparison of early vs.

late lags. It may be the case that rACC is preferentially active when attentional

control is exerted in order to ignore processing of distracter stimuli (regardless

of valence). This view is in accordance with previous research demonstrating

rACC’s general involvement in response conflict (e.g., Carter, Braver, Barch,

Botvinivk, Noll & Cohen, 1998; MacDonald, Cohen, Stenger & Carter, 2000).

Differences in self-reported attentional control or anxiety have been

126

associated with differences in frontoparietal cortex or amygdala function

respectively (e.g., Bishop 2007). How do anxiety and attentional control affect

brain activity in the present results? We found that activity differences in the

DLPFC were primarily driven by individuals low in trait anxiety and high in

attentional control. This is consistent with other prior research demonstrating

that the activity in DLPFC has been linked to attentional control (Corbetta &

Shulman, 2002; Corbetta, Patel & Shulman, 2008, Most et al., 2006). Although

we were expecting IPS activity to be modulated by trait anxiety and attentional

control, this was not consistently the case across IPS ROIs or conditions. It is

possible that because of its placement between bottom-up and top-down

attention processes, IPS activity is modulated by a mix of signals via the

amygdala and via DLPFC that, with the current experiment, cannot be

disentangled.

The cost of emotion to attention in the AB task is likely due to failures of

top-down attention to control strong bottom-up responses towards emotional

distracters. This competition for attentional resources (e.g., Desimone &

Duncan, 1995; Pessoa, Kastner & Ungerleider, 2002) indicates that there is

some degree of automaticity in emotional stimulus processing (e.g., Carretie,

Hinojosa, Martin-Loeches, Mercado & Tapia, 2004; Ortigue, Michel, Murray,

Mohr, Carbonnel & Landis, 2004), but we show that the degree to which this

disrupts target processing may also be dependent on the relative strength of

127

top-down attentional control over bottom-up distractions (Fox et al, 2005; Most

et al, 2005). Indeed, individual differences in the degree to which emotional

distracter words disrupt neutral target identification are associated with

differences in self-reported attentional control and anxiety.

Consistent with its involvement in emotional costs to spatial attention,

we show that IPS is involved with emotional costs to temporal attention,

specifically within the attentional blink task. The early amygdala response to

emotion may facilitate the early IPS response through common links with

OFC. However, IPS is maximally responsive to emotion at early, but not late,

distracter-target lags, indicating that the pattern of IPS activity described here

is not just an arousal response. Instead we provide evidence for the notion

that IPS is a site underlying the temporally-dependent capture of attention by

emotion.

128

CONCLUSION

The interactive effects of emotion with attention can both improve and

impair visual processing. Across three chapters I have shown how this

interaction modulates our perceptual experience, with an emphasis on how

emotion can incur attentional costs to visual processes. We have also

explored how external characteristics of visual stimuli and differences among

individual observers affect this interaction.

Whereas individual differences in how we experience emotion, and how

we respond to emotionally-evocative stimuli, are accepted and often expected

in psychological research, individual differences in how we perceive, and how

we pay attention to, visual stimuli is a relatively new idea. Not only do we show

emotion modulates attention and perception, we also show that individual

variability across a number of factors can affect the way we pay attention.

Handedness, a marker of cerebral lateralization (e.g., Bourne, 2008;

Hellige, Bloch, Cowin, Eng, Eviatar & Sergent, 1994), interacts with the type of

stimulus used to cue exogenous attention, resulting in different effects of

attention across the visual field. Anxiety exacerbates costs of emotion to

attention, and under some circumstances affects females more than males.

Finally, the degree to which we can exert control over distracters while

performing a task can determine whether we see a target stimulus. By

129

changing the way we pay attention, these factors critically influence which

information is processed at higher cognitive levels.

Handedness and Attention

Chapter 1 describes a study that was our initial attempt at investigating

costs of emotion to attention and contrast sensitivity. Instead, we found that

observer handedness modulated the degree to which attention cued with

pictures of faces affected contrast sensitivity, regardless of the emotional

expression of the cues. No significant differences between left- and right-

handers were found when simple dots were used to cue exogenous attention:

both groups demonstrated the same pattern of results of increased contrast

sensitivity following valid dot cues, and decreased contrast sensitivity following

invalid dot cues, relative to a distributed dot condition. When face cues were

used to cue attention, right-handers had the expected benefits of valid face

cues and the expected costs of invalid face cues relative to a distributed face

cue condition. Notably, using the same face cues to direct attention in left-

handers resulted in no change of contrast sensitivity across cueing conditions,

suggesting that the handedness effect is specific to faces.

While asymmetries in visual attention, face and emotion recognition

have been previously found to interact with handedness (Bourne, 2008;

Dronkers & Knight, 1989; Luh, Redl & Levy, 1994; Rubichi & Nicoletti, 2006),

130

our study was the first to demonstrate that handedness affects the way in

which attention alters sensitivity to contrast, a low-level visual feature which

modulates cortical (V1) activity at early levels of visual analysis.

We hypothesized that these behavioral differences are due to

differences in the functional organization of brain regions supporting spatial

attention and face processing between handedness groups. Previous laterality

research reports that left-hander brains as a whole are less functionally

lateralized than right-handers (e.g., Boles, 1989; Luh et al., 1994). Significantly

more right-handers, for example, are right-hemisphere dominant for face

processing (e.g., Badzakova-Trajkov, Häberling, Roberts & Corballis, 2010;

Bourne 2008) and for spatial attention (e.g., Flöel, Buyx, Breitenstein,

Lohmann & Knecht, 2005) than left-handers. It may be the case that in order

to elicit the beneficial effects of attention using face cues, signals must travel

between face and attention regions within the time limits of exogenous

attention (100 to 120 ms; Nakayama & Mackeben, 1989; Cheal & Lyon, 1991).

Given that left-handers have a lesser degree of functional laterality these

signals may, on average, travel farther, take a longer amount of time to

interact, and result in no effect. Regardless of the true explanation, Chapter 1

shows that individual differences in handedness can affect attention and

perception at very early levels of processing, indicating that researchers with

interests in these areas should adopt appropriate controls.

131

Emotion, Anxiety, Gender and Attention

We revisited the question of whether emotion produces a cost to

attention and contrast sensitivity in Chapter 2. Although we were initially

puzzled with regard to the lack of a significant emotion effect in Chapter 1, an

article was recently published that proved very influential for our later

experiments. Bocanegra & Zeelenberg (2009) found that emotion’s effects on

orientation perception depend on spatial frequency. Emotion improved

perception of low spatial frequency targets (<3 cpd) and impaired perception

of high spatial frequency targets (>3 cpd). The authors suggest this is due to

the amygdala’s preference for the low spatial frequency components in

pictures of fearful faces, which may boost processing of these spatial

frequencies in retinotopic cortex. Rather than using 4 cpd Gabor stimuli as we

did in Chapter 1, this information led us to use low spatial frequency Gabor

stimuli (1.5 and 2 cpd) in the experiments reported in Chapter 2.

Our main unique finding was that costs of emotion to attention and

contrast sensitivity were modulated by trait anxiety (Experiments 1 and 2) and

sex (Experiment 1). Relative to neutral face cues that invalidly cued attention

to a distracter location, invalid fearful face cues resulted in decreased contrast

sensitivity for target stimuli. In Experiment 1, in which we tested two task

locations, this cost was found most strongly in high trait anxious females,

whereas in Experiment 2, in which we tested four task locations, this cost was

132

found across all high trait anxious individuals regardless of sex. We also

replicated previous research (Phelps et al., 2006) showing that fear cues

increased contrast sensitivity beyond that of neutral cues when they (1)

directed attention in a distributed fashion across all task locations, and (2)

directed selective attention to the target’s location.

The detrimental effects of anxiety on attention have been studied for

decades, with reaction time in dot probe or Stroop tasks serving as the main

dependent measure of attention allocation (e.g., Macleod, Mathews & Tata,

1986; Mathews & Macleod, 1985; Richards & Millwood, 1989; Fox 1993). The

present research is the first to show perceptual consequences of anxiety’s

influence on attention. In addition, our results suggest that known differences

in behavioral and neural responses to emotional or anxiety-provoking stimuli

between males and females, such as better recognition of facial affect in

females and greater amygdala activity in response to fearful faces in high

anxious females (Cahill, 2006; Craske, 2003; Dickie & Armony, 2008; Kemp,

Silberstein, Armstrong & Nathan, 2004; Lang et al., 1998; McClure, 2000;

Thayer & Johnsen, 2000) are extended to differences at the perceptual level.

Neural Correlates of Emotion’s Cost to Attention

Chapter 3 explored the neural correlates of emotion’s cost to attention

within the attentional blink. We tested a model in which bottom-up amygdala

133

responses to emotion influence activity in brain regions involved in the top-

down control of attention such as the OFC, IPS and DLPFC. We found that the

amygdala was more active for all emotion vs. neutral distracter trials,

consistent with its broad role in emotion processing (e.g., Phelps 2006), and

its specific role in the emotional facilitation effect in the AB (Anderson &

Phelps, 2001). We also had a specific interest in the IPS based on previous

reports of this region’s involvement in spatial attention invalidity effects, in

tasks where attention has been directed to non-target locations by emotional

face cues (Armony & Dolan, 2002; Pourtois et al., 2006). Consistent with these

accounts, we found that IPS activity increased in response to emotional

distracters within the early, but not late, AB window. Rather than being

responsive based solely on stimulus-evoked arousal, this time-dependent

activity suggests IPS is sensitive to emotion in the context of increased task

demands.

A specific contrast comparing early emotional vs. early neutral

distracter trials further revealed a region in OFC, while the opposite contrast

revealed DLPFC. We propose that DLPFC activity is reduced during early

emotional distracter trials via inhibitory connections from OFC (Yamasaki et

al., 2002). These events involving both bottom-up emotion and top-down

attention processes result in greater attentional resources being funneled into

emotional distracter processing, at the cost of neutral target processing. Our

134

results are consistent with research on the frontoparietal attention network as

reviewed by Corbetta & Shulman (2002). Ventral frontoparietal regions, which

respond automatically to visually salient and/or behaviorally relevant objects in

the environment, can disrupt ongoing goal-directed attention under the control

of dorsal frontoparietal regions.

Furthermore our results extend prior imaging work on the AB (De

Martino et al., 2009; Most et al., 2006; Schwabe et al., 2010) by showing that

differences in brain activity correspond to differences in self-reported anxiety

and attentional control. Increased DLPFC activity, which we hypothesized is

related to control over early neutral distracters, was driven by individuals who

self-reported high attentional control and low trait anxiety. The impact these

traits had on brain activity is consistent with earlier research on the

neurocognitive mechanisms of anxiety and attentional control (Bishop et al.,

2004; Bishop 2009).

Individual Differences in Perception

The first two chapters of this dissertation have shown that there are

significant variations in our ability to perceive contrast. While individual

differences in perception have been studied throughout the history of modern

psychology, they have mostly pertained to our high-level conceptualization of

objects or scenes. For example, one study found that gender influenced

135

interpretations of ambiguous drawings (Coren, Porac & Ward, 1978) and

another found that mood (happy, critical or anxious) affected the way

individuals interpreted a scene (Leuba & Lucas, 1945).

More recently, there has been an increased interest in inter-subject

variability of low-level visual perception. The individual differences that are

reported in low-level perception research, however, are usually described and

explained in terms of how they are associated with other visual perceptual

abilities (e.g., Peterzell & Teller, 1996; Simpson & McFadden, 2005) rather

than innately non-perceptual qualities such as personality or gender, or non-

visual cognitive processes such as emotion. Psychological research in general

is becoming more open to research at the intersections of multiple “distinct”

sub-fields that for most of the last century were traditionally studied separately.

Our investigations of how attention affects perception, and how emotion

affects both perception and attention, make an important contribution to this

interdisciplinary movement. In particular, emotion’s effects on perception, for

which there are currently only two papers published that I’m aware of (spatial

frequency: Bocanegra & Zeelenberg, 2009; contrast sensitivity: Phelps et al.,

2006), is relatively unexplored territory. Given the importance of both attention

and emotion in filtering large amounts of information for what is most relevant,

there is sure to be many new advances in this area in the near future.

136

Other Relevant Issues

Observer expertise in visual psychophysics

Studies of attention and perception traditionally recruit a small number

of well-trained psychophysical observers. The main reason being that

experienced observers provide less noisy data, and data are analyzed per

individual. For a given stimulus intensity, experienced observers give more

consistent responses, whereas completely naïve observers may at first

answer randomly during perceptually difficult trials. In addition, naïve

observers are more likely to press the wrong button even if they know the

correct answer, adding even more variance to the data. Given that we cannot

know which trials were answered incorrectly due to finger-error, it is impossible

to remove these trials from analysis.

Observer expertise may have influenced the results of Experiment 2 in

Chapter 2. Fear-valid and fear-distributed cuing conditions were expected to

produce greater contrast sensitivity than their neutral counterparts across all

observers, replicating the results of Phelps et al. (2006). Instead the

differences we found were a result of the interaction of cuing condition and

anxiety. Specifically, low trait anxious observers demonstrated greater contrast

sensitivity following fear-valid cues, but this was relative to the fear-distributed

rather than the neutral-valid cuing condition. Furthermore, high trait anxious

observers demonstrated greater contrast sensitivity following fear-distributed

137

cues, but this was relative to the fear-invalid rather than the neutral-distributed

cueing condition. We hypothesize that our large and inexperienced observer

population, made necessary by the fact that we were also interested in

individual differences in anxiety and sex, underlies the lack of statistically

significant differences between fear and neutral conditions in these cases.

Costs to spatial vs. temporal attention

Given that we did not scan our spatial attention studies we cannot say

for sure that the spatial disengagement cost due to invalid cuing with fearful

faces has common underlying brain regions as the emotional capture of

attention in the AB task. That is, it is unclear whether the spatial

disengagement cost is a result of enhanced amygdala activity driving OFC and

IPS relative to attentional control brain regions such as DLPFC. Moreover, an

important point to consider is that the attentional systems engaged during

exogenous spatial cuing, and those engaged in the attentional blink task, may

be subserved by different underlying neural networks. However, given the IPS

and OFC results of Pourtois et al. (2006) and Armony and Dolan (2002), which

were both spatial attention studies, and our current imaging study of the

attentional blink, we hypothesize that similar neural circuitry may underlie

emotional costs in space as well as at different points in time.

138

If we were to scan Study 2 Experiment 2 (the four locations task), we

would need to test ~6-10 observers at their distributed cue condition contrast

threshold only, due to the fact that changes in contrast modulate V1 activity on

their own (Boynton, Demb, Glover & Heeger, 1999). This would necessitate

pre-testing outside of the scanner in order to obtain reliable estimates of each

person’s contrast threshold. We would also need to create retinotopic maps for

all observers, from data collected in a second scanning session, and localize

the four possible target locations in visual cortex.

To separate cue- vs. target-related activity, we would utilize two types

of trials: one in which the cue precedes the target, and another in which the

target precedes the cue (similar to Liu et al., 2005). While the overall visual

stimulation is equivalent, there will only be cue-related effects on the target

during the trials where the cue precedes the target. What we might expect is

that visual cortical activity would increase at validly cued target locations, and

decrease at invalidly cued target locations, relative to when there were

distributed cues. These changes may be accompanied by similar changes in

IPS as attention is transiently directed towards cued locations. We might also

expect graded activity changes across the four locations. For example, if a

target in the upper left quadrant was validly cued, we might see the greatest

increase in activity in the corresponding brain region, the greatest decrease in

activity in the diagonal region, and intermediate changes in the other two. As

139

for frontal cortex, it may be difficult to tease apart responses to, say, the upper

left vs. the lower left quadrant of the visual field. A general contrast of fear-

invalid vs. fear-valid, however, may reveal OFC activity with decreases in

DLPFC, whereas a contrast of neutral-invalid vs. neutral-valid may reveal

increases in DLPFC due to the attentional control required in shifting from

task-irrelevant back to task-relevant locations.

Current Projects and Future Directions

The results of Chapter 3 confirmed that the amygdala is involved in

emotional costs to attention, consistent with other imaging studies of the AB

(Most et al., 2006; Schwabe et al., 2010). We also show that the right

amygdala may play more of a role than the left amygdala given that only the

right side had differential activity in response to emotional distracters at early

compared to late lags. What Chapter 3 is not able to answer, however, is

whether the (right) amygdala is necessary for these costs in behavior to occur.

Following Anderson and Phelps’ (2001) patient study showing that the left

amygdala was necessary for emotion’s facilitative effect during the AB, we

now are interested in investigating whether the amygdala is necessary for

emotion’s detrimental effect during the modified AB.

To answer this question, we are currently collecting data from anterior

temporal lobectomy (ATL) volunteers as they participate in the same modified

140

AB task as described in Chapter 3. ATL patients elect to have surgery to

excise the anterior temporal lobe on one side as a treatment for intractable

epilepsy. It includes removal of the amygdala, hippocampus and anterior

temporal cortex. At the moment we have a participant pool of 13 left ATL and

9 right ATL patients, with the goal of having 15 in each group. In addition, we

have data from a bilateral amygdala patient, who had a right ATL and a more

focal left amygdala lesion. Our control group consists of 14 healthy age-

matched volunteers.

Thus far, our results indicate that the left ATL patients, and the bilateral

amygdala patient, show the emotional capture of attention effect (same as

controls). While the results are trending in the same direction for the right ATL

patients, they are not significant. These results are tentative, but results

suggest that the left amygdala is not necessary for the capture effect to occur,

while the right amygdala is necessary. However, the bilateral results appear to

be contradictory. Given that we are still collecting data, it is possible that there

is not enough power in the right ATL group to show the effect.

If after collecting more data, the right ATL results become significant,

this would indicate that the emotional capture effect is not dependent on the

amygdala. This would raise an interesting conundrum. While the emotional

capture of attention effect may be more dependent on failures of top-down

attentional control rather than bottom-up emotional responses, the effect still

141

requires a strong emotional response to distract attention in the first place. If

the amygdala is not the source of bottom-up emotional prioritization of

attentional resources, which brain region(s) could it be? A recent patient study

showed that two bilateral amygdala patients showed the emotional facilitation

effect (Bach, Talmi, Hurlemann, Patin & Dolan, 2011), in direct contrast to

Anderson & Phelps (2001) and consistent with another study showing the

amygdala is not necessary for non-conscious processing of fearful faces

(Tsuchiya, Moradi, Felsen, Yamasaki & Adolphs, 2009). These authors

suggest the pulvinar nucleus of the thalamus and visual cortex could be the

source of this automatic relevance detection, and the amygdala modulates

other cognitive processes after considerable cortical computation has already

been completed. Although this is a debate that cannot currently be settled, it

demonstrates that more research is needed to understand the role of the

amygdala in this process of attentional prioritization.

Concluding Remarks

Emotion and attention are highly interactive processes that, for the most

part, allow us to dedicate limited metabolic resources to the important events

in our lives, while simultaneously discarding irrelevant events. At times,

however, emotionally significant information can disrupt ongoing attentional

processes. We have all experienced the distraction caused by the aftermath of

142

a car accident, for example, while trying to concentrate on driving safely down

the freeway.

The consequences of emotion and attention’s interaction, both the

beneficial and the costly, are far-reaching and occur at multiple levels of visual

stimulus processing. We show that it impacts both contrast sensitivity at a

lower level, and word identification at a higher level, of object recognition.

Furthermore, we show that a number of factors not traditionally associated

with visual perception have significant effects on the perception of contrast via

their interactions with attention. Whereas handedness interacts with the type

of stimulus that directs spatial attention, anxiety and sex interact to produce

different outcomes depending on cue validity, cue valence, and the availability

of attentional resources.

One of the themes common to Chapters 2 and 3 is our demonstration

of a significant positive relationship between the magnitude of emotion’s cost

to attention and trait anxiety. The fact that we show this for both contrast

sensitivity and word identification accuracy in the AB suggests that common

underlying mechanisms may be responsible. Weakened top-down attentional

control processes that are associated with anxiety may result in enhanced

sensitivity to bottom-up stimulation, especially under conditions of task-

irrelevant emotional distraction. This emphasis on processing distractions may

divert attentional resources from processing task-relevant targets, resulting in

143

decreased contrast sensitivity. Contrast processing, impoverished or not, is an

important first step in the recognition of complex visual stimuli such as words.

The outcomes of the three chapters in this dissertation just begin to hint

at some of the other perceptual and cognitive processes that are likely

impacted by emotional and attentional selection. Our results also emphasize

the importance of taking into account the significant individual variability in a

given subject pool, because it may indicate not only behavioral differences but

also functional differences at the neural level. If we can link differences in

function to differences among individuals, then we are a step closer to

understanding emotion and attention’s conjoint effects on how we see.

144

FIGURES

Figure 1. Chapter 1: Trial sequence for Experiment 1 (face cues) and

Experiment 2 (dot cues). Cues preceded Gabor stimuli in this exogenous

cuing task. Participants indicated both the location and the orientation of the

tilted target Gabor using a single button press on each trial. Images not to

scale; contrast and target Gabor tilt emphasized for clarity.

145

Figure 2. Chapter 1: (A) Contrast sensitivity data for all observers (face n=12;

dot n=10), averaged over handedness group and target visual field. (B)

Contrast sensitivity data split by handedness group (each group: face n=6; dot

n=5). Error bars are ± 1 standard error of the mean. *p=.05. **p=.01.

***p=.001.

146

Figure 3. Chapter 1: Contrast sensitivity data from Figure 2B, split by target

visual field. (A) Left-hander face (n=6) and dot (n=5) data. (B) Right-hander

face (n=6) and dot (n=5) data. Error bars are ± 1 standard error of the mean.

*p=.05.

147

Figure 4. Chapter 1: Correlation of handedness score with cue validity effect.

For RVF targets cued with faces, this significant correlation is driven by both a

decrease in attention effect in left-handers and an increase in attention effect

in right-handers, whereas for dots, the significant correlation is driven mostly

by closer clustering of attention effect in right-handers.

148

Figure 5. Chapter 2: Experiment 1 trial sequence. Face cues preceded Gabor

stimuli in this exogenous cuing task. Participants indicated both the location

and the orientation of the tilted target Gabor using a single button press on

each trial. Images not to scale; contrast, target Gabor tilt and spatial frequency

emphasized for clarity.

149

Figure 6. Chapter 2: Experiment 1 cueing effects: all observers. The Y-axis is

normalized contrast sensitivity. The X-axis is spatial cueing condition. ($$)

indicates a significant two-tailed comparison. Error bars are ± 1 SE of mean.

150

151

Figure 7. Chapter 2: Experiment 1 cueing effects: by anxiety and sex. The Y-

axis is normalized contrast sensitivity. The X-axis is spatial cueing condition.

Top row: all females and all males; middle row: low trait anxious females and

males; bottom row: high trait anxious females and males. ($$) indicates a

significant two-tailed comparison, ($) indicates a significant one-tailed

comparison, and (!) indicates a marginal one-tailed comparison. Error bars

are ± 1 SE of mean. Note: two female and two male observers with median

trait anxiety scores were not included in either the low or high groups.

152

Figure 8. Chapter 2: Experiment 2 trial sequence. Face cues preceded Gabor

stimuli in this exogenous cuing task. Participants indicated the orientation of

the tilted target Gabor using a single button press on each trial. Images not to

scale; contrast, target Gabor tilt and spatial frequency emphasized for clarity.

153

154

Figure 9. Chapter 2: Experiment 2 cueing effects: all observers and by

anxiety. The Y-axis is normalized contrast sensitivity. The X-axis is spatial

cueing condition. Top row: all observers; middle row: low trait anxious

observers; bottom row: high trait anxious observers. ($$) indicates a significant

two-tailed comparison, ($) indicates a significant one-tailed comparison. Error

bars are ± 1 SE of mean.

155

Figure 10. Chapter 3: Top: Trial sequence. Bottom: fMRI trial events.

156

Figure 11. Chapter 3: Behavioral results of Experiment 1 (n=18). Left:

Accuracy data across all 7 lags. Right: Data split by early vs. late lags (note:

late lags consist only of lag 7). Error bars are ± 1 SEM. (*) indicates significant

comparison.

157

Figure 12. Chapter 3: Behavioral results of Experiment 2 (n=28). Left:

Accuracy data across early and late lags. Right: Reaction time data across

early and late lags (note: late lags consist of both lags 7 and 8). Error bars are

± 1 SEM. (*) indicates significant comparison.

158

Figure 13. Chapter 3: IPS results. Top: Bilateral IPS region active for all task-

related activity across the whole group. ROIs were defined individually per

subject using this contrast (Table 1A), and beta values were extracted per

subject; this figure is for illustrative purposes only. Bottom left: left posterior

IPS. Bottom right: right posterior IPS. Error bars are ± 1 SEM. (*) indicates a

significant comparison.

159

Figure 14. Chapter 3: Right DLPFC results. Top: Right DLPFC region active

for all task-related activity across the whole group. The group ROI was defined

160

across the whole group using this contrast (Table 1A), and beta values were

extracted individually per subject; this figure is for illustrative purposes only.

Error bars are ± 1 SEM. (*) indicates a significant comparison.

161

TABLES

Table 1A. Chapter 3: Mean Talairach coordinates of a priori defined ROIs. All

n=28 except: L Ant. IPS (n=24), R Post. IPS (n=27). Coordinates of center of

gravity in Talairach space. ‘Nr of Voxels’ refers to voxels interpolated into

1x1x1 space.

Region x y z Nr of Voxels

L Posterior IPS -27 -65 38 1655 L Anterior IPS -34 -52 39 1208 R Posterior IPS 28 -60 38 1181 L DLPFC -24 25 43 738 R DLPFC 24 21 48 810 rACC -2 30 7 594 OFC -11 36 -8 1266 L Amygdala (anat.) -18 -5 -13 1067 R Amygdala (anat.) 17 5 -12 1128 Table 1B. Self-report scores. n=28. Scale Mean SE Min Max Positive Affect 35.04 1.35 13 46 Negative Affect 25.07 1.20 14 42 State Anxiety 41.18 1.68 24 60 Trait Anxiety 44.79 1.62 27 64 Attentional Control 51.14 1.17 43 66 Handedness 78.46 5.54 -25 100

162

Table 2. Chapter 3: Whole-brain Contrasts. n = 28. Coordinates of center of gravity in Talairach space. ‘Nr of Voxels’ refers to voxels interpolated into 1x1x1 space. t-scores are of peak voxel within cluster. 300 voxel threshold. Table 2A. Contrast 1 All > Baseline Region x y z t(27)-score Nr of

Voxels p (corr)

R Post Occ Ctx 5 -92 -3 8.44 4752 <.05 L Post Occ Ctx -19 -95 1 10.61 1215 <.05 Calcarine Sulcus -4 -79 -2 8.98 1728 <.05 L Fusiform Gyrus -42 -48 -15 8.78 1890 <.05 L Post IPS -24 -65 36 8.21 351 <.05 L Ant IPS -35 -47 36 8.26 1404 <.05 R white matter (thal) 24 -24 2 7.72 378 <.05 L white matter (thal) -25 -26 2 9.92 1134 <.05 L white matter (caud) -20 -12 16 8.48 486 <.05 L Sup Frontal Gyr -45 -1 33 8.79 1026 <.05 Baseline > All PCC -2 -52 27 11.75 12825 <.05 rACC / PFC -2 34 11 12.21 19467 <.05 R DLPFC 24 21 48 8.02 810 <.05 L DLPFC -24 25 43 7.98 756 <.05 R Ant Temp Lobe 50 -6 -10 9.92 1539 <.05 L Ant Temp Lobe -55 -12 -13 8.59 621 <.05 L Sup Occ Ctx -43 -77 27 10.06 2052 <.05 L Frontal Pole -20 56 21 7.64 324 <.05 Sup Colliculus -2 -34 -9 7.94 432 <.05 Table 2B. Contrast 2 E > N p (unc) R Middle Temp Gyr 60 -37 -1 4.66 405 <.001 R Ant Temp/Post

OFC 28 14 -17 4.64 324 <.001

L Ant Temp Lobe -50 10 -20 6.15 1053 <.001 L VLPFC -48 38 -3 4.44 324 <.001 L Amygdala -16 -3 -16 3.31 847 <.01 R Amygdala 12 -3 -16 3.06 269 <.01

163

Table 3. Chapter 3: Whole-brain Contrasts. n = 28. Coordinates of center of gravity in Talairach space. ‘Nr of Voxels’ refers to voxels interpolated into 1x1x1 space. t-scores are of peak voxel within cluster.

Table 3A. Contrast 3 Late > Early Region x y z t(27)-score Nr of

Voxels p (unc)

R rACC 7 43 14 5.04 918 <.001 Table 3B. Contrast 4 Early E > N

R Post Lat Occ 36 -86 -2 4.39 405 <.005 R Cing Ctx 11 -33 18 3.72 189 <.005 Medial OFC 0 35 -20 4.46 162 <.005 L DPFC -13 58 31 4.08 162 <.005 L VLPFC -27 51 -12 3.81 108 <.005 L Post OFC/Ant Temp -45 18 -4 3.26 108 <.005 L Ant Temp Lobe -53 11 -18 4.27 297 <.005 Early N > E R DLPFC 24 18 46 3.40 216 <.005 L TPJ -44 -40 29 3.34 135 <.005 Table 3C. Contrast 5 Late E > N

R Inf Temp Gyr 46 -22 -10 5.45 405 <.001 R Ant Temp Lobe -48 0 -23 5.54 405 <.001 L Ant Temp Lobe -52 1 -24 5.67 189 <.001 Table 3D. Contrast 6 Early N > Late N

L Posterior Caudate -10 -27 18 4.79 216 <.001 Late N > Early N R rACC 3 36 15 4.33 135 <.001

164

Table 4. Chapter 3: Whole-brain Contrasts. n = 28. Coordinates of center of gravity in Talairach space. ‘Nr of Voxels’ refers to voxels interpolated into 1x1x1 space. t-scores are of peak voxel within cluster. Table 4A. Contrast 7 Early E > Late E

Region x y z t(27)-score Nr of Voxels

p (unc)

R Posterior Occ Ctx 22 -93 7 4.97 1836 <.01 L Posterior Occ Ctx -29 -96 2 4.08 1296 <.01 Late E > Early E R rACC 7 43 15 4.11 810 <.01 Table 4B. Contrast 8 Early Emotion Correct > Incorrect

R MPFC 6 58 11 3.99 432 <.005 L MPFC -7 48 1 3.9 945 <.005 Early Emotion Incorrect > Correct L IPS -28 -57 32 4.63 1593 <.005 R Ant IPS 24 -51 33 3.71 297 <.005 R Post IPS 21 -60 36 4.17 270 <.005

165

APPENDICES

Appendix A: Chapter 1

Study 1 Instructions

Thanks in advance for your participation. The programs for this

experiment are “do-it-yourself” so you may run at your convenience without

having to change settings or filenames.

Experiment Set-up: Located in the first room on the left (testing room “L1”) in

the Carrasco Lab, room 970 on the 9th floor of Meyer Hall.

When you come in to run yourself, please go over this checklist first to ensure

the experiment set-up is correct:

1) The monitor must be 57 cm from the chinrest and the edge of the

table (there is a piece of tape on the table marking 57 cm).

2) The monitor video attenuator must be attached (the little metal box

on the video cable that makes the screen green).

3) The monitor resolution must be 1600 x 1200 at 75Hz – please check

this every time you run by going to the Monitor Control Panel under

the apple menu (top left).

166

If these settings are not correct and you run in the experiment, I will have to

discard your data for that session.

You will have your own experiment file in the folder “EmoAttention” on the

Desktop that you will use each time you run.

To run the experiment:

1) Put up the “Experiment in Progress” sign on the door so no one will

interrupt you.

2) Turn out all the lights, and allow your eyes to adapt to the darkness

for a minute.

3) Double click on the folder “EmoAttention” on the Desktop.

4) Find your experiment file (it will have your initials at the end of the

filename) and double click it. This will open the file in Matlab. Do not

modify the file.

5) With the file open, type “Apple-E”. This saves the file and executes

the program.

6) Enter your subject ID in addition to the year, month and day,

followed by the run number for that day. For example, if I ran myself

on June 20th, 2008 I would use this ID: “ef080620_r1”.

167

7) Click OK and you’re ready to run. The program will ask you to press

the spacebar to begin.

Experiment Details: Each run of the program consists of 672 trials in 6 blocks

of 112 trials each. Please feel free to take a break in between blocks. After 3

blocks I would ask that you force yourself to take a break so your eyes don’t

get overtired. Each run takes about 45 minutes to 1 hour to complete. In all, I

ask that you complete 6 sessions over the next 2 weeks. You may do multiple

sessions in one day but please leave a few hours in between each session.

Task Instructions: This experiment requires that you make visual

discriminations about images that appear on the screen. On each trial a

fixation point appears, followed by a brief presentation of 1 or 2 faces located

on either side of the fixation point. (Your eyes must be looking at the fixation

point at all times, unless you are taking a break.) The faces are followed by a

very brief presentation of two tilted gratings in positions just underneath the

faces. One of the gratings will be tilted slightly to the left or the right. Your task

is to indicate which grating is tilted, and which direction it is tilted.

There are 4 response keys: ‘x’, ’c’, ’<’, and ’>’. If you position your pinky

fingers on the shift keys, your middle and index fingers will fall on the correct

168

keys. Use your left hand to indicate if the left grating is tilted and your right

hand if the right grating is tilted. Indicate the direction of tilt using the key that

corresponds to that direction.

Response Keys

X = grating on left, left tilted

C = grating on left, right tilted

< = grating on right, left tilted

> = grating on right, right tilted

Following the presentation of the gratings, you have 2 seconds to make

a response. Try your hardest to push the correct button. We are also

measuring reaction time so try to answer correctly as quickly as possible, but

not at the cost of making more errors. The contrast of the gratings will vary

from trial to trial so they may actually be quite difficult to see on some trials. If

you are unsure which grating was tilted in which direction, please guess.

On each trial you will receive feedback in the form of a tone. A high

tone means you were correct, while a low tone means you were incorrect. No

tone means that the response period ended before you responded. PLEASE

DO YOUR BEST TO GUESS BEFORE THE RESPONSE PERIOD ENDS.

Trials in which we don’t get a response in time will have to be discarded.

169

When the experiment is over, please close all program windows and

quit Matlab. Remove the “Experiment in Progress” sign and, if you’re the only

one in the lab, please close the lab door ensuring it is locked.

Troubleshooting:

- If the monitor will not display anything (it’s black) but the computer is

on, this is probably a problem with the video attenuator. Sometimes if

the computer is started up with the attenuator in place it allows only one

choice for video resolution, but not the one we want. The computer

should be started up again without the attenuator attached, and then

the video resolution should be changed to 1600 x 1200 in the Control

Panel. Only then should the attenuator be attached.

- If the experiment freezes or crashes, please exit the program by

pressing “Apple-period, Apple-0” and type “clear all” in the command

window. Then just re-start the program by typing “Apple-E”.

170

Appendix B: Chapter 2

Study 2, Experiment 2 Instructions

During this experiment, you will be presented with a picture of a face or

four faces, followed by four gratings that are all tilted randomly either left or

right. After the gratings disappear, a line indicating the location of the target

will appear. Your task is to indicate the tilt of the target grating. To indicate

your answer, press either the ‘<’ button for a left tilted grating, or the ‘>’ button

for a right tilted grating. If you get the answer right, you will hear a high tone,

and if you get the answer wrong, you will hear a low tone. You will not be

penalized for wrong answers so make your best guess if you are not sure of

your response.

This experiment will take about an hour on day 1 and an additional hour

on day 2 (two days in the same week).

Schedule for each day:

On day 1, you will arrive at the Carrasco Lab, room 970, at your

designated appointment time. The experiment for this day will take about an

hour. Before we begin the training session, we will go over the task

instructions, and then ask if you have any questions about procedures,

remuneration, time, etc. Then you will begin the training session, which

consists of 3 six-minute blocks of 112 trials each. Instead of pictures of faces

171

you will see black dots, but your task is still to indicate the tilt of the target

grating. After this ~20 minute training session we will check your data and

choose appropriate contrast levels for the main experiment. Next the

experimental session will begin. There will be four blocks and each block will

take about 5-6 minutes each. Thus, if you do not take any breaks in between

each block, you will finish the experiment in about 40 minutes.

On day 2, you will complete the second half of the experiment. There

will be eight blocks total. One block will take around 5-6 minutes. After you are

finished with the experiment, we will ask you to fill out a short questionnaire

about your emotions during and after the experiment. You will then be

debriefed as to the purpose of the experiment and will be paid twenty dollars.

Important:

1. You MUST choose either left or right for every trial. If you do not answer,

the trial will have to be discarded. It is also important to make your

response within two seconds after the gratings disappear from the

screen.

2. Keep your eyes focused on the black cross in the middle of the screen. It

is very important that your eyes don’t wander around to look for the

target grating.

3. Please don’t move the position of the chin rest. We will adjust the chin

172

rest before the experiment so that it is comfortable for you. The chin

rest ensures that you stay the same distance from the screen

throughout the experiment.

4. If, during the training, you cannot get more than 70-75% right, we cannot

have you continue to the main experiment. For that reason, we will not

be able to give you the full $20 remuneration. However, we will

remunerate you with $7 for coming to our lab and doing the training

session part of the experiment. If you do qualify, the $20 remuneration

will be paid on the second day.

5. If your eyes are getting tired, you may utilize a short, less than two-minute

break after each 5-6 minute block.

173

Appendix C: Chapter 3

Study 3 Instructions

Please read the instructions along with the experimenter.

During this experiment you will view a stream of rapidly changing words

in the center of the screen. Each word will immediately replace the one that

came before it. All the words will be printed in black, however there is one

word which will be printed in green. This is the target word. After the word

presentation is completed, you will be presented with a list of 4 possible

choices.

Your task is to view the words and to pick the one word out of the list of

four words that matches the target word. To do so, place your fingers along

the top number row on the keyboard, with your index finger lightly resting on

the 1, middle finger on the 2, ring finger on the 3 and pinky on the 4. We ask

you to rest your hand there in order for you to make your responses as

efficiently as possible.

You won’t get any feedback when you make a response. The choices

will be on the screen for a full 2 seconds (3 seconds during practice). The next

trial will begin automatically even if you don’t make a response.

There will be 5 blocks with breaks in between. Each block is about 5

minutes long. After each block ends, you must notify the experimenter who will

then start you on the next block. They will record how well you do and adjust

174

the difficulty of the task to keep it challenging but not impossible to do fairly

well.

Things to remember:

It is very important that you make a response on every trial. Even if

you are unsure of your response, which is highly likely, you must make a

guess. If you don’t answer too many trials (more than 8) we cannot use your

data.

You must make your response while the choices are on the screen

(approximately 2 seconds). If you don’t, your answer will not be recorded.

Please do not try to lean closer to the screen. Stay a constant

distance so you do not have an unfair advantage over other participants.

After all 5 blocks have been completed there are 3 short surveys to fill

out. If you are interested, the experimenter will then debrief you on the

purpose of the experiment and answer any questions you might have.

To begin, we’ll start with 2 short practice blocks.

175

Neutral Distracters (80)

ART FUSSY PEAR SNAP BAKED GEAR PERCOLATE SOLID

BIN HIKERS PINKY SOLVENT BOARDING HILL PISTON SON

BOOTH HITCH POETIC SPEND CAPE ICICLES PURSE SPIN

CARGOS INSECT RATE STORIED CENT JAZZ REPORT SUM

COLUMN LEASHES RIGGED TACTICS COMPUTER LONER RIVETED THAWS

CORN LOWBROW RUDDER THIS CUPID LUCK RUMOR THORN DATES LUNCH SANDBAR TOCK DOCK MASTERMIND SAY TOGGLE

ENABLES MELODY SEEMS TOURISM ETHICS MILL SHERPA TREADS EXALTS MOOD SHORE VARIOUS FENCE MUST SHUT WAGON FLATS NUN SLAB YOGA FLOOD PAVING SLIPPED ZITS

176

Emotional Distracters (56)

AGONY FAGGOT PENIS ANUS FART PISS

AROUSAL FECES PUBIC ASSHOLE FETISH PUSSY

BARF FONDLE RAPE BASTARD FUCK SCROTUM

BITCH HERPES SEMEN BLOWJOB HORNY SEX

BONER INCEST SHIT BOOBS KILL SHRIEK BREAST KINKY SLAVE

CLITORIS LESBIAN SLUT COCK LEWD TESTICLE

CONDOM LUST TITS CUM LYNCH TUMOR CUNT MASTURBATE VAGINA DICK NIPPLES VIBRATOR

DILDO ORGASM WHORE EJACULATE ORGY

177

Neutral Targets (160)

ABOVE EITHER JAW ORDER TAIL ACADEMY EMBASSY JOIN OWNER TAN

ACTOR EON JOKE PADDLE TASTE AROUND EVERY JUDGE PAGE THINK

AVID FADE JUMP PALE TICKET BALCONY FATIGUED JUNE PATROL TRIVIALITY

BASIC FERN KARMA PLAIN TROPHY BELIEVE FIELD KAYAK PLASTER TUITION

BEND FLAG KEY POINTY TURNED BINDER FLEW KIN QUEEN UNDER BRAVE FOLDER KIND QUICK UPON

BUFFER FOREIGN KNOWN QUIETLY USUAL BUILT FORK LATER RADIO UTTER CALM GARDEN LAZY RAINBOW VACATION

CANVAS GENTLE LEARN RAMBLING VALET CARRIER GIVES LED RELAXATION VALIANT CASUAL GRAIN LUXURIOUS REPEAT VEND CEDAR GRAY MAGNETIC RULE VERSE CHAPEL GROUP MANAGE RYE VIEW CHEEK HALF MEAL SADDLE VOICE CHIN HANGAR MEND SCARCE WAS COP HEAR MONK SECURE WHILE

CUBIC HEAVY MOVIE SIT WIND DEALT HOME NATIVE SOFT WOULD

DEEPEST HOW NEEDLE SOLAR YARD DISCUSS HUNT NERVE SOLD YELLOW

DISK INFER NIECE SPEAKER YESTERDAY DIVIDE INSIDE NOTE SPILL YOUNG

DREARY INVENT OATMEAL SPONSOR ZANY DRYING IRATE OBOE STABLE ZEBRA

DUG IRONED OCCUPANT STAR ZERO EAST JACKET OKAY STRETCHY ZONING

178

Maskers (120)

ABBREVIATION GOVERNMENT RHODODENDRON ALBUQUERQUE HALLUCINATION RIGAMAROLE AMPITHEATER HANDKERCHIEF RIGHTEOUSLY

ANTHROPOLOGY HIEROGLYPHIC RITUALIZATION BEATIFICATION HOUSEKEEPER SIMULTANEOUS

BIOGRAPHY HOUSESITTER SINUSOIDAL BOOTSTRAPPING HUMMINGBIRD SOUNDLEVEL

BUSINESSMAN HYPOTHETICAL TABLEWARES CHRYSANTHEMUM ILLUMINATION THOROUGHBRED CIRCUMFERENCE INCOMPREHENSIBLE THOROUGHLY CLASSIFICATION INTELLECTUALISM THREADBARE

COMEDIAN INTERNALIZATION TIDDLYWINKS CONDENSATION INVESTIGATION TOPOGRAPHY

CONFEDERATION IRREPROACHABLE TOURNAMENT CONGREGATION IRRESPECTIVE TRANSPORTATION CONTORTIONIST JOURNEYMAN ULTRASONIC

DELETERIOUS JUSTIFICATION UNQUESTIONABLY DELICATESSEN JUXTAPOSITION UNSYSTEMATIC

DEMOGRAPHICS KALEIDOSCOPE UPHOLSTERY DEMYSTIFY KINDERGARTENER UTILITARIAN

DESCRIPTION KNOWLEDGEABLE VEGETATION DICTIONARY LEGISLATORSHIP VERBALIZATION

DIFFERENTIATION LOGROLLING VIDEOCASSETTE DISAMBIGUATE LOQUACIOUSNESS VIRTUOSITY

DISAPPEARANCE LUNCHEONETTE VOCALIZATION DISINTEGRATE MERETRICIOUS VOLUMINOUS

EMULATION MOTORCYCLE WINDSHIELD ENCYCLOPEDIA NOTEWORTHINESS WINTERLAND ENTREPRENEUR NOTWITHSTANDING WOODPECKER

ERADICATE NULLIFICATION WOOLGATHERER ESTABLISHMENT NUTRITIONIST WORKSTATION

EXCEEDINGLY OBSERVATORY WRAPAROUND EXIGENCIES ORGANIZATION XYLOPHONIST

EXTRAPOLATION OSCILLATION YARDMASTERS FERTILIZER PERTURBATIONS YESTERYEAR

FORESHADOW PHOTOGRAPHED YOUTHFULLY FUNCTIONALITY PONTIFICATION YOUTHFULNESS FURTHERMORE PURIFICATION ZESTFULNESS

GENERALIZATION QUADRICEPS ZILLIONAIRE GLOBALIZATION RATIONALISTIC ZOOLOGICALLY

179

BIBLIOGRAPHY

Adolphs, R. (2002). Neural systems for recognizing emotion. Current Opinion

in Neurobiology, 12, 169-177.

Adolphs, R., Tranel, D., Damasio, H., & Damasio, A. (1994). Impaired

recognition of emotion in facial expressions following bilateral damage to the

human amygdala. Nature, 372, 669-672.

Amaral, D. G., Behniea, H., & Kelly, J. (2003). Topographic organization of

projections from the amygdala to the visual cortex in the macaque monkey.

Neuroscience, 118(4), 1099-1120.

Anderson, A. K. (2005). Affective influences on the attentional dynamics

supporting awareness. Journal of Experimental Psychology: General, 134(2),

258-281.

Anderson, A. K., & Phelps, E. A. (2001). Lesions of the human amygdala

impair enhanced perception of emotionally salient events. Nature, 411(6835),

305-309.

Armony J. L. & Dolan, R. J. (2002). Modulation of spatial attention by fear-

conditioned stimuli: an event-related fMRI study. Neuropsychologia, 40(7),

817-826.

180

Arnell, K. M., Killman, K. V., & Fijavs, D. (2007). Blinded by emotion: target

misses follow attention capture by arousing distractors in RSVP. Emotion,

7(3), 465-477.

Azzopardi, P., Jones, K. E., & Cowey, A. (1999). Uneven mapping of

magnocellular and parvocellular projections from the lateral geniculate nucleus

to the striate cortex in the macaque monkey. Vision Research, 39(13), 2179-

2189.

Bach, D. R., Talmi, D., Hurlemann, R., Patin, A, & Dolan, R. J. (2011).

Automatic relevance detection in the absence of a functional amygdala.

Neuropsychologia, 49(5), 1302-1305.

Badzakova-Trajkov, G., Häberling, I. S., Roberts, R. P., & Corballis, M. C.

(2010). Cerebral asymmetries: complementary and independent processes.

PLoS ONE, 5(3), e9682.

Barbas, H. (2000). Connections underlying the synthesis of cognition,

memory, and emotion in primate prefrontal cortices. Brain Research Bulletin,

52, 5, 319-330.

Bar-Haim, Y., Lamy, D., Pergamin, L., Bakermans-Kranenburg, M. J., & van

IJzendoorn, M. H. (2007). Threat-related attentional bias in anxious and

nonanxious individuals: a meta-analytic study. Psychological Bulletin, 133(1),

1-24.

181

Bechara, A., Damasio, H., Tranel, D. and Damasio, A. R. (1997). Deciding

advantageously before knowing the advantageous strategy. Science,

275(5304), 1293-1295.

Bishop, S. J. (2007). Neurocognitive mechanisms of anxiety: an integrative

account. Trends in Cognitive Sciences, 11(7), 307-316.

Bishop, S. J. (2009). Trait anxiety and impoverished prefrontal control of

attention. Nature Neuroscience, 12(1), 92-98.

Bishop, S. J., Duncan, J., & Lawrence, A. D. (2004). State anxiety modulation

of the amygdala response to unattended threat-related stimuli. Journal of

Neuroscience, 24(46), 10364-10368.

Bishop, S. J., Duncan, J., Brett, M., & Lawrence, A. D. (2004). Prefrontal

cortical function and anxiety: controlling attention to threat-related stimuli.

Nature Neuroscience, 7(2), 184-188.

Bocanegra, B. R., & Zeelenberg, R. (2009). Emotion improves and impairs

early vision. Psychological Science, 20, 707-713. doi:10.1111/j.1467-

9280.2009.02354.x

Boles, D. B. (1989). Do visual field asymmetries intercorrelate?

Neuropsychologia, 27, 697-704. doi:10.1016/0028-3932(89)90114-0

182

Bourne, V. J. (2008). Examining the relationship between degree of

handedness and degree of cerebral lateralization for processing facial

emotion. Neuropsychologia, 22, 350-356. doi:10.1037/0894-4105.22.3.350

Boynton, G. M., Demb, J. B., Glover, G. H. & Heeger, D. J. (1999). Neuronal

basis of contrast discrimination. Vision Research, 39, 257-269.

Bradley, B., Mogg, K., Falla, S., & Hamilton, L. (1998). Attentional bias for

threatening facial expressions in anxiety: Manipulation of stimulus duration.

Cognition and Emotion, 12(6), 737-753.

Brainard, D. H. (1997). The Psychophysics Toolbox. Spatial Vision, 10, 433-

436. doi: 10.1163/156856897X00357

Broadbent, D.E. (1954). The role of auditory localization in attention and

memory span. Journal of Experimental Psychology, 47(3), 1-6.

Brown, R. & Kulik, J. (1977). Flashbulb memories. Cognition, 5, 73-99.

Bush, G., Luu, P. & Posner, M. I. (2000). Cognitive and emotional influences in

anterior cingulate cortex. Trends in Cognitive Sciences, 4, 6, 215-222.

Cahill, L. (2006). Why sex matters for neuroscience. Nature Reviews

Neuroscience, 7(6), 477-484.

183

Carmichael, S. T., & Price, J. L. (1995). Sensory and premotor connections of

the orbital and medial prefrontal cortex of macaque monkeys. The Journal of

Comparative Neurology, 363, 642-664.

Carrasco, M. (2006). Covert attention increases contrast sensitivity:

Psychophysical, neurophysiological and neuroimaging studies. Progress in

Brain Research, 154, 33-70.

Carrasco, M. (2011). Visual attention: The past 25 years. Vision Research,

doi:10.1016/j.visres.2011.04.012.

Carrasco, M., Penpeci-Talgar, C., & Eckstein, M. P. (2000). Spatial covert

attention increases contrast sensitivity across the CSF: support for signal

enhancement. Vision Research, 40, 1203-1215. doi:10.1016/S0042-

6989(00)00024-9

Carrasco, M. & Yeshurun, Y. (1998). The contribution of covert attention to the

set-size and eccentricity effects in visual search. Journal of Experimental

Psychology: Human Perception & Performance, 24(2), 673-692.

Carretie, L., Hinojosa, J. A., Martin-Loeches, M., Mercado, F. & Tapia, M.

(2004). Automatic attention to emotional stimuli: neural correlates. Human

Brain Mapping, 22, 290-299.

Carter, C. S., Braver, T. S., Barch, D. M., Botvinivk, M. M., Noll, D., & Cohen,

J. D. (1998). Anterior cingulate cortex, error detection, and online monitoring of

performance. Science, 280, 747-749

184

Cavada, C. & Goldman-Racik, P. S. (1989). Posterior parietal cortex in rhesus

monkey. II. Evidence for segregated corticocortical networks linking sensory

and limbic areas with the frontal lobe. Journal of Comparative Neurology, 287,

422-445.

Cavada,, C., Compañy, T., Tejedor, J., Cruz-Rizzolo, R. J., & Reinoso-Suárez,

F. (2000). The anatomical connections of the macaque monkey orbitofrontal

cortex: a review. Cerebral Cortex, 10, 220-242.

Cheal, M. & Lyon, D. R. (1991). Central and peripheral precuing of forced-

choice discrimination. Quarterly Journal of Experimental Psychology A., 43(4),

859-880.

Cherry, E. C. (1953). Some experiments on the recognition of speech, with

one and with two ears. The Journal of the Acoustical Society of America,

25(5), 975-979.

Chun, M. M. and Potter, M. C. (1995). A two-stage model for multiple target

detection in rapid serial visual presentation. Journal of Experimental

Psychology: Human Perception & Performance, 21(1), 109-127.

Cisler, J. M. & Koster, E. H. W. (2010). Mechanisms of attentional biases

towards threat in anxiety disorders: An integrative review. Clinical Psychology

Review, 30, 203-216.

185

Cohen, L., Dehaene, S., Naccache, L., Lehéricy, S., Dehaene-Lambertz, G.,

Hénaff, M-A., & Michel, F. (2000). The visual word form area: spatial and

temporal characterization of an initial state of reading in normal subjects and

posterior split-brain patients. Brain, 123, 291-307.

Compton, R. (2003). The interface between emotion and attention: A review of

evidence from psychology and neuroscience. Behavioral and Cognitive

Neuroscience Reviews, 2(2), 115-129.

Corbetta, M. & Shulman, G. L. (2002). Control of goal-directed and stimulus-

driven attention in the brain. Nature Reviews Neuroscience, 3, 201-215.

Corbetta, M. Patel, G. & Shulman, G. L. (2008). The reorienting system of the

human brain: from environment to theory of mind. Neuron, 58, 306-324.

Coren, S., Porac, G. and Ward, L.M. (1978). Sensation and Perception. New

York: Academic Press.

Coull, J. T., Frith, C. D., Buchel, C., & Nobre, A. C. (2000). Orienting attention

in time: behavioral and neuroanatomical distinction between exogenous and

endogenous shifts. Neuropsychologia, 38, 808-819.

Craske, M. G. (2003). Origins of phobias and anxiety disorders: Why more

women than men? Oxford: Elsevier.

Davis, M. & Whalen, P. J. (2001). The amygdala: vigilance and emotion.

Molecular Psychiatry, 6(1), 13-34.

186

De Martino, B., Strange, B. A., & Dolan, R., J. (2008). Noradrenergic

neuromodulation of human attention for emotional and neutral stimuli.

Psychopharmacology, 197(1), 127-136.

De Martino, B., Kalisch, R., Rees, G., & Dolan, R. J. (2009). Enhanced

processing of threat stimuli under limited attentional resources. Cerebral

Cortex, 19(1), 127-133.

Derryberry, D., & Reed, M. A. (2002). Anxiety-related attentional biases and

their regulation by attentional control. Journal of Abnormal Psychology, 111(2),

225-236.

DeValois, R.L. & DeValois, K. K. (1988). Spatial Vision. New York: Oxford

University Press.

Desimone, R. & Duncan, J. (1995). Neural mechanisms of selective visual

attention. Annual Review of Neuroscience, 18(1), 193-222.

Dickie, E. W., & Armony, J. L. (2008). Amygdala responses to unattended

fearful faces: Interaction between sex and trait anxiety. Psychiatry Research,

162(1), 51-57.

Dronkers, N. F., & Knight, R. T. (1989). Right-sided neglect in a left-hander:

Evidence for reversed hemispheric specialization of attention capacity.

Neuropsychologia, 27, 729-735. doi:10.1016/0028-3932(89)90118-8

187

Dux, P. E., & Marois, R. (2009). The attentional blink: a review of data and

theory. Attention, Perception & Psychophysics, 71(8), 1683-1700.

Easterbrook, J. A. (1959). The effect of emotion on cue utilization and the

organization of behavior. Psychological Review, 66(3), 183-201.

Eimer, M. & Holmes, A (2002). An ERP study on the time course of emotional

face processing. NeuroReport, 13, 427-431.

Ekman, P., & Friesen, W. (1976). Pictures of Facial Affect. Palo Alto, CA:

Consulting Psychologists Press.

Elder, J. H., Balaban, D. Y., Kamyab, A., Wilcox, L., & Hou, Y. (2008, May).

Selectivity for faces as exogenous attentional cues [Abstract]. Journal of

Vision, 8, 685a.

Eriksen, C.W., & St. James, J.D. (1986). Visual attention within and around the

field of focal attention: A zoom lens model. Perception & Psychophysics, 40,

225-240.

Evert, D. L., McGlinchey-Berroth, R., Verfaellie, M., & Milberg, W. P. (2003).

Hemispheric asymmetries for selective attention apparent only with increased

task demands in healthy participants. Brain and Cognition, 53, 34-41. doi:

10.1016/S0278-2626(03)00207-0

Eysenck, M. W., Derakshan, N., Santos, R., & Calvo, M. G. (2007). Anxiety

and cognitive performance: attentional control theory. Emotion, 7(2), 336-353.

188

Fecteau, J. H., Enns, J. T., & Kingstone, A. (2000). Competition-induced visual

field differences in search. Psychological Science, 11, 386-393.

doi:10.1111/1467-9280.00275

Ferneyhough, E., Stanley, D. A., Phelps, E. A., & Carrasco, M. (2010). Cuing

effects of faces are dependent on handedness and visual field. Psychonomic

Bulletin & Review, 17(4), 529-535.

Flöel, A., Buyx, A., Breitenstein, C., Lohmann, H., & Knecht, S. (2005).

Hemispheric lateralization of spatial attention in right- and left-hemispheric

language dominance. Behavioral Brain Research, 158(2), 269-275.

Fox, E. (1993). Attentional bias in anxiety: selective or not? Behavioral

Research and Therapy, 31(5), 487-493.

Fox, E. (2002). Processing emotional facial expressions: the role of anxiety

and awareness. Cognitive & Affective Behavioral Neuroscience, 2, 1, 52-63.

Fox, E., Russo, R., Bowles, R. J., & Dutton, K. (2001). Do threatening stimuli

draw or hold visual attention in subclinical anxiety? Journal of Experimental

Psychology: General, 130(4), 681-700.

Fox, E., Russo, R., & Dutton, K. (2002). Attentional Bias for Threat: Evidence

for Delayed Disengagement from Emotional Faces. Cognition & Emotion,

16(3), 355-379.

189

Fox, E., Russo, R., & Georgiou, G. A. (2005). Anxiety modulates the degree of

attentive resources required to process emotional faces. Cognitive & Affective

Behavioral Neuroscience, 5(4), 396-404.

Freese, J. L., & Amaral, D. G. (2005). The organization of projections from the

amygdala to visual cortical areas TE and V1 in the macaque monkey. The

Journal of Comparative Neurology, 1-23.

Frijda, N. H. (1986). The Emotions. New York: Cambridge University Press.

Fuller, S., Park, Y. & Carrasco, M (2009). Cue contrast modulates the effects

of exogenous attention on appearance. Vision Research, 49, 1825-1837.

doi:10.1016/j.visres.2009.04.019

Fuller, S., Rodriguez, R. Z., & Carrasco, M. (2008). Apparent contrast differs

across the vertical meridian: visual and attentional factors. Journal of Vision,

8(1), 1-16, http://journalofvision.org/8/1/16/, doi:10.1167/8.1.16.

Graham, N. (2011). Beyond multiple pattern analyzers modeled as linear filters

(as classical V1 simple cells): useful additions of the last 25 years. Vision

Research, doi:10.1016/j.visres.2011.02.007.

Hartley, C. A. & Phelps, E. A. (2010). Changing fear: the neurocircuitry of

emotion regulation. Neuropsychopharmacology Reviews, 35, 136-146.

190

Haxby, J. V., Hoffman, E. A., & Gobbini, M. I. (2002). Human neural systems

for face recognition and social communication. Biological Psychiatry, 51, 59-

67.

Hellige, J. B., Bloch, M. I., Cowin, E. L., Eng, T. L., Eviatar, Z. & Sergent, V.

(1994). Individual variation in hemispheric asymmetry: multitask study of

effects related to handedness and sex. Journal of Experimental Psychology:

General, 123, 3, 2353-256.

Holmes, A., Nielsen, M. K., & Green, S. (2008). Effects of anxiety on the

processing of fearful and happy faces: an event-related potential study.

Biological Psychology, 77(2), 159-173.

Jefferies, L. N., Smilek, D., Eich, E., & Enns, J. T. (2008). Emotional valence

and arousal interact in attentional control. Psychological Science, 19(3), 290-

295.

Keil, A. & Ihssen, N. (2004). Identification facilitation for emotionally arousing

verbs during the attentional blink. Emotion, 4(1), 23-35.

Kelly, C., Uddin, L. Q., Shehzad, Z., Margulies, D. S., Castellanos, X., Milham,

M. P. & Petrides, M. (2010). Broca’s region: linking human brain functional

connectivity data and non-human primate tracing anatomy studies. European

Journal of Neuroscience, 32, 3, 383-398.

191

Kemp, A. H., Silberstein, R. B., Armstrong, S. M., & Nathan, P. J. (2004).

Gender differences in the cortical electrophysiological processing of visual

emotional stimuli. Neuroimage, 21(2), 632-646.

Kessler R. C., Berglund P., Demler O., et al. (2005). Lifetime prevalence and

age-of-onset distributions of DSM- IV disorders in the National Comorbidity

Survey Replication. Archives of General Psychiatry, 62, 593-602.

Kim, M. J., & Whalen, P. J. (2009). The Structural Integrity of an Amygdala-

Prefrontal Pathway Predicts Trait Anxiety. Journal of Neuroscience, 29(37),

11614-11618.

Kissler, J., Herbert, C., Peyk, P. & Junghofer, M. (2007). Buzzwords: early

cortical responses to emotional words during reading. Psychological Science,

18, 6, 475-480.

Kinchla, R. A. (1992). Attention. Annual Review of Psychology, 43, 711-743.

doi: 10.1146/annurev.ps.43.020192.003431

Koster, E. H. W., Crombez, G., Van Damme, S., Verschuere, B., & De

Houwer, J. (2004). Does imminent threat capture and hold attention? Emotion,

4(3), 312-317.

Koster, E. H. W., Verschuere, B., Crombez, G., & Van Damme, S. (2005).

Time-course of attention for threatening pictures in high and low trait

anxiety. Behaviour Research and Therapy, 43, 1087-1098.

192

Koster, E. H. W., Crombez, G., Verschuere, B., & De Houwer, J. (2006).

Attention to threat in anxiety-prone individuals: Mechanisms underlying

attentional bias. Cognitive Therapy and Research, 30(5), 635-643.

Lang, P. J., Bradley, M. M., Fitzsimmons, J. R., Cuthbert, B. N., Scott, J. D.,

Moulder, B., et al. (1998). Emotional arousal and activation of the visual

cortex: an fMRI analysis. Psychophysiology, 35(2), 199-210.

LeDoux, J. E. (1996). The Emotional Brain: The Mysterious Underpinnings of

Emotional Life. Simon & Schuster: New York.

LeDoux, J. (2000). Emotion circuits in the brain. Annual Review of

Neuroscience, 23, 155-184.

LeDoux, J. (2002). Synaptic self: How our brains become who we are. New

York: Viking Penguin.

Lennie, P. (2003). The cost of cortical computation. Current Biology, 13(6),

493-497.

Leuba, C. & Lucas, C. (1945). The effects of attitudes on descriptions of

pictures. Journal of Experimental Psychology, 35, 517-524.

Lim, S-L., Padmala, S., & Pessoa, L. (2008). Affective learning modulates

spatial competition during low-load attentional conditions. Neuropsychologia,

46(5), 1267-1278.

193

Ling, S., & Carrasco, M. (2006a). Sustained and transient covert attention

enhance the signal via different contrast response functions. Vision Research,

46, 1210-1220. doi:10.1038/nn1761

Ling, S. & Carrasco, M. (2006b). When sustained attention impairs perception.

Nature Neuroscience, 9,1243-1245.

Liu, T., Pestilli, F., & Carrasco, M. (2005). Transient attention enhances

perceptual performance and fMRI response in human visual cortex. Neuron,

45(3), 469-477.

Luh, K. E., Redl, J., & Levy, J. (1994). Left- and right-handers see people

differently: Free-vision perceptual asymmetries for chimeric stimuli. Brain &

Cognition, 25, 141-160. doi: 10.1006/brcg.1994.1028

Luo, Q., Peng, D., Jin, Z., Xu, D., Xiao, L. & Ding, G. (2004). Emotional

valence of words modulates the subliminal repetition priming effect in the left

fusiform gyrus: an event-related fMRI study. Neuroimage, 21, 414-421.

Macdonald, A. W., Cohen, J. D., Stenger, A. & Carter, C. S. (2000).

Dissociating the role of the dorsolateral prefrontal cortex in cognitive control.

Science, 288, 1835-1838.

Macleod, C., Mathews, A. & Tata, P. (1986). Attentional bias in emotional

disorders. Journal of Abnormal Psychology, 95, 1, 15-20.

194

Macleod, C., & Mathews, A. (1988). Anxiety and the allocation of attention to

threat. Quarterly Journal of Experimental Psychology A, 40(4), 653-670.

Marois, R. & Ivanoff, J. (2005). Capacity limits of information processing in the

brain. Trends in Cognitive Sciences, 9, 6, 296-305.

Mathews, A., & Macleod, C. (1985). Selective processing of threat cues in

anxiety states. Behavioral Research & Therapy, 23, 5, 563-569.

Mathewson, K. J., Arnell, K. M., & Mansfield, C. A. (2008). Capturing and

holding attention: the impact of emotional words in rapid serial visual

presentation. Memory & Cognition, 36, 1, 182-200.

McClure, E. B. (2000). A meta-analytic review of sex differences in facial

expression processing and their development in infants, children, and

adolescents. Psychological Bulletin, 126(3), 424-453.

McDonald, A. J., Mascagni, F. & Guo, L. (1996). Projections of the medial and

lateral prefrontal cortices to the amygdala: a Phaseolus vulgaris

leucoagglutinin study in the rat. Neuroscience, 71, 55-75.

Mesulam, M. M. (1999) Spatial attention and neglect: Parietal, frontal and

cingulate contributions to the mental representation and attentional targeting

of salient extrapersonal events. Philosophical Transactions of the Royal

Society London B Biological Sciences, 354, 1325–1346.

195

Mogg, K. & Bradley, B. P. (1999). Some methodological issues in assessing

attentional biases for threatening faces in anxiety: A replication study using a

modified version of the probe detection task. Behavioral Research & Therapy,

37, 6, 595-604.

Mogg, K., Holmes, A., Garner, M., Bradley, B. P. (2008). Effects of threat cues

on attentional shifting, disengagement and response slowing in anxious

individuals. Behavioral Research & Therapy, 46(5), 656-667.

Mogg, K., Bradley, B.P., Miles, F., and Dixon, R. (2004). Time course of

attentional bias for threat scenes: testing the vigilance—avoidance hypothesis,

Cognition & Emotion, 18, 689–700

Montagna, B., Pestilli, F., & Carrasco, M. (2009). Attention trades off spatial

acuity. Vision Research, 49, 735-745. doi:10.1016/j.visres.2009.02.001

Morawetz, C., Baudewig, J., Treue, S., & Dechent, P. (2010). Diverting

attention suppresses human amygdala responses to faces. Frontiers in

Human Neuroscience, 4, 226.

Morecraft, R. J., Geula, C. & Mesulam, M. M. (1993). Architecture of

connectivity within a cingulo-fronto-parietal neurocognitive network for directed

attention. Archives of Neurology, 50, 3, 279-284.

Morris, J. S., de Gelder, B., Weiskrantz, L., & Dolan, R. J. (2001). Differential

extrageniculostriate and amygdala responses to presentation of emotional

faces in a cortically blind field. Brain, 124(Pt 6), 1241-1252.

196

Morris, J. S., Friston, K. J., Büchel, C., Frith, C., Young, A. W., Calder, A. J., et

al. (1998). A neuromodulatory role for the human amygdala in processing

emotional facial expressions. Brain, 121(Pt 1), 47-57.

Morris, J. S., Frith, C. D., Perrett, D. I., Rowland, D., Young, A. W., Calder, A.

J. & Dolan, R., J. (1996). A differential neural response in the human

amygdala to fearful and happy facial expressions. Nature, 383, 812-815.

Most, S. B., Chun, M. M., Widders, D. M., & Zald, D. H. (2005). Attentional

rubbernecking: cognitive control and personality in emotion-induced blindness.

Psychonomic Bulletin & Review, 12(4), 654-661.

Most, S. B., Chun, M. M., Johnson, M. R., & Kiehl, K. A. (2006). Attentional

modulation of the amygdala varies with personality. Neuroimage, 31, 2, 934-

944.

Most, S. B., Smith, S. D., Cooter, A. B., Levy, B. N., & Zald, D. H. (2007). The

naked truth: positive, arousing distractors impair rapid target perception.

Cognition and Emotion, 21, 5, 964-981.

Müller, H. J. & Rabbitt, P. M. A. (1989). Reflexive and voluntary orienting of

visual attention: time course of activation and resistance to interruption.

Journal of Experimental Psychology: Human Perception & Performance, 15, 2,

315-330.

197

Nakayama, K., & Mackeben, M. (1989). Sustained and transient components

of focal visual attention. Vision Research, 29, 1631-1647. doi: 10.1016/0042-

6989(89)90144-2

Nobre, A. C., Coull, J. T., Frith, C. D., & Mesulam, M. M. (1999). Orbitofrontal

cortex is activated during breaches of expectation in tasks of visual attention.

Nature Neuroscience, 2, 1, 11-12.

Notebaert, L., Crombez, G., Van Damme, S., De Houwer, J. & Theeuwes, J.

(2011). Signals of threat do not capture, but prioritize, attention: A conditioning

approach. Emotion, 11, 1, 81-89.

Öhman, A. (2009). Of snakes and faces: an evolutionary perspective on the

psychology of fear. Scandinavian Journal of Psychology, 50, 6, 543-552.

Öhman, A., Flykt, A., & Esteves, F. (2001). Emotion Drives Attention:

Detecting the Snake in the Grass. Journal of Experimental Psychology:

General. 130(3), 466-478.

Oldfield, R. C. (1971). The assessment and analysis of handedness: The

Edinburgh Inventory. Neuropsychologia, 9, 97 - 113. doi: 10.1016/0028-

3932(71)90067-4.

Olsson, A., Nearing, K. I., & Phelps, E. A. (2007). Learning fears by observing

others: the neural systems of social fear transmission. Social Cognitive and

Affective Neuroscience, 2, 3-11.

198

Olsson, A. & Phelps, E. A. (2007). Social learning of fear. Nature

Neuroscience, 10, 9, 1095-1102.

Oosterhof, N. N., & Todorov, A. (2008). The functional basis of face

evaluation. Proceedings of the National Academy of Sciences, 105, 11087-

11092. doi: 10.1073/pnas.0805664105

Ortigue, S., Michel, C. M., Murray, M. M., Mohr, C., Carbonnel S. & Landis, T.

(2004). Electrical neuroimaging reveals early generator modulation to

emotional words. Neuroimage, 21, 4, 1242-1251.

Pessoa, L. (2010). Emotion and attention effects: is it all a matter of timing?

Not yet. Frontiers in Human Neuroscience, 4, 172, 1-5.

Pessoa, L. & Adolphs, R. (2010). Emotion processing and the amygdala: from

a ‘low road’ to ‘many roads’ of evaluating biological significance. Nature

Reviews Neuroscience, 11, 11, 773-783.

Pessoa, L., Kastner, S. & Ungerleider, L. G. (2002). Attentional control of the

processing of neutral and emotional stimuli. Cognitive Brain Research, 15, 1,

31-45.

Pestilli, F., & Carrasco, M. (2005). Attention enhances contrast sensitivity at

cued and impairs it at uncued locations. Vision Research, 45, 1867-1875.

doi:10.1016/j.visres.2005.01.019

199

Pestilli, F., Ling, S., & Carrasco, M. (2009). A population-coding model of

attention's influence on contrast response: Estimating neural effects from

psychophysical data. Vision Research, 49(10), 1144-1153.

Pestilli, F., Viera, G., & Carrasco, M. (2007). How do attention and adaptation

affect contrast sensitivity? Journal of Vision, 7(7), 1-12.

Peterzell, D. H. & Teller, D. Y. (1996). Individual differences in contrast

sensitivity functions: the lowest spatial frequency channels. Vision Research,

36, 19, 3077-3085.

Phelps, E. A. (2006). Emotion and cognition: insights from studies of the

human amygdala. Annual Review of Psychology, 57, 1, 27-53.

Phelps, E. A., Ling, S., & Carrasco, M. (2006). Emotion facilitates perception

and potentiates the perceptual benefits of attention. Psychological Science,

17, 292-299. doi:10.1111/j.1467-9280.2006.01701.x

Posner, M. I. (1980). Orienting of attention. The Quarterly Journal of

Experimental Psychology, 32, 1, 3-25.

Posner, M. I., & Petersen, S. (1990). The attention system of the human brain.

Annual Review of Neuroscience, 13(1), 25-42.

Pourtois, G., Grandjean, D., Sander D., Vuilleumier, P. (2004).

Electrophysiological correlates of rapid spatial orienting towards fearful faces.

Cerebral Cortex, 14, 6, 619-633.

200

Pourtois, G., Schwartz,S., Seghier, M. L., Lazeyras, F., & Vuilleumier, P.

(2006). Neural systems for orienting attention to the location of threat signals:

an event-related fMRI study. Neuroimage, 31, 2, 920-933.

Quirk, G. J., & Mueller, D. (2008). Neural mechanisms of extinction learning

and retrieval. Neuropsychopharmacology, 33 1, 56-72.

Raymond, J. E., Shapiro, K. L., & Arnell, K. M. (1992). Temporary suppression

of visual processing in an RSVP task: an attentional blink? Journal of

Experimental Psychology: Human Perception and Performance, 18, 3, 849-

860.

Raymond, M., Pontier, D., Dufour, A., & M"ller, A. P. (1996). Frequency-

dependent maintenance of left-handedness in humans. Proceedings of the

Royal Society of London, B, 263, 1627-1633.

Rhodes, G. (1985). Lateralized processes in face recognition. British Journal

of Psychology, 76, 249 – 271.

Richards, A. & Millwood, B. (1989). Colour-identification of differentially

valenced words in anxiety. Cognition & Emotion, 3, 2, 171-176.

Rubichi, S., & Nicoletti R. (2006). The Simon effect and handedness: Evidence

for a dominant-hand attentional bias in spatial coding. Perception &

Psychophysics, 68, 1059-1069.

201

Sass, S. M., Heller, W., Stewart, J. L., Silton, R. L., Edgar, J. C., Fisher, J. E.,

et al. (2010). Time course of attentional bias in anxiety: Emotion and gender

specificity. Psychophysiology, 47(2), 247-259.

Saygin, Z. M., Osher, D. E., Augustinack, J., Fischl, B. & Gabrieli, J. D. E.

(2011). Connectivity-based segmentation of amygdala nuclei using

probabilistic tractography. Neuroimage, 56, 1353-1361.

Schupp, H. T., Markus, J., Weike, A. I., Hamm, A. D. (2003). Emotional

facilitation of sensory processing in the visual cortex. Psychological Science,

14, 1, 7-13.

Schwabe, L., Merz, C. J., Walter, B., Vaitl, D., Wolf, O. T., & Stark, R. (2010).

Emotional modulation of the attentional blink: the neural structures involved in

capturing and holding attention. Neuropsychologia, 49, 3, 416-425.

Sharot, T., Delgado, M. R., & Phelps, E. A. (2004). How emotion enhances the

feeling of remembering. Nature Neuroscience, 7(12), 1376-1380.

Siman-Tov, T., Mendelsohn, A., Schonberg, T., Avidan, G., Podlipsky, I.,

Pessoa, L., Gadoth, N., Ungerleider, L. G., & Hendler, T. (2007).

Bihemispheric leftward bias in a visuospatial attention-related network. Journal

of Neuroscience, 27, 11271-11278. doi: 10.1523/JNEUROSCI.0599-07.2007

Simpson, W. A. & McFadden, S. M. (2005). Spatial frequency channels

derived from individual differences. Vision Research, 45, 2723-2727.

202

Smith, S. D., Most, S. B., Newsome, L. A., & Zald, D. H. (2006). An emotion-

induced attentional blink elicited by aversively conditioned stimuli. Emotion,

6(3), 523-527.

Spampinato, M. V., Wood, J. N., De Simone, V., & Grafman, J. (2009). Neural

correlates of anxiety in healthy volunteers: a voxel-based morphometry study.

Journal of Neuropsychiatry & Clinical Neuroscience, 21(2), 199-205.

Spielberger, C.D., Gorsuch, R.L., Lushene, R., Vagg, P.R. and Jacobs, G.A.

(1983). Manual for the State-Trait Anxiety Inventory. Consulting Psychologists

Press: Palo Alto.

Stanley, D.A., Ferneyhough, E., & Phelps, E.A. (2009). Neural Perspectives

on Emotion: Impact on Perception, Attention and Memory. In G.G. Berntson &

J.T. Cacciopo (Eds.), Handbook of Neuroscience for the Behavioral Sciences.

New York: John Wiley & Sons.

Talairach, J. & Tournoux, P. (1988). Co-planar stereotaxic atlas of the human

brain: 3-dimensional proportional system; an approach to cerebral imaging.

Stuttgart: George Thieme Verlag.

Taylor, J. G. & Fragopanagos, N. F. (2004). Modelling human attention and

emotions. Proceedings of the International Joint Conference on Neural

Networks.

Taylor, J. G. & Fragopanagos, N. F. (2005). The interaction of attention and

emotion. Neural Networks, 18, 353-369.

203

Thayer, J. F., & Johnsen, B. H. (2000). Sex differences in judgement of facial

affect: a multivariate analysis of recognition errors. Scandinavian Journal of

Psychology, 41(3), 243-246.

Tsuchiya, N., Moradi, F., Felsen, C., Yamazaki, M., & Adolphs, R. (2009).

Intact rapid detection of fearful faces in the absence of the amygdala. Nature

Neuroscience, 12, 10, 1224-1225.

Vaughan, K. B. & Lanzetta, J. T. (1980). Vicarious instigation and conditioning

of facial expressive and autonomic responses to a model’s expressive display

of pain. Journal of Personality and Social Psychology, 38, 6, 909-923.

Vuilleumier, P., Armony, J. L., Driver, J., & Dolan, R. J. (2003). Distinct spatial

frequency sensitivities for processing faces and emotional expressions. Nature

Neuroscience, 6(6), 624-631.

Vuilleumier, P., Richardson, M., Armony, J. L., Driver, J., & Dolan, R. J.

(2004). Distant influences of amygdala lesion on visual cortical activation

during emotional face processing. Nature Neuroscience, 7(11), 1271-1278.

Vuilleumier, P., Schwartz, S., Verdon, V., Maravita, A., Hutton, C., Husain, M.,

& Driver, J. (2008). Abnormal attentional modulation of retinotopic cortex in

parietal patients with spatial neglect. Current Biology, 18, 1525-1529.

204

Watson, D., Clark, L.A. and Tellegen, A. (1988). Development and validation

of brief measures of positive and negative affect: The PANAS scales. Journal

of Personality and Social Psychology, 54, 1063–1070.

Weierich, M., Treat, T., & Hollingworth, A. (2008). Theories and measurement

of visual attentional processing in anxiety. Cognition & Emotion, 22(6), 985-

1018.

Whalen, P. J. (1998). Fear, vigilance, and ambiguity: initial neuroimaging

studies of the human amygdala. Current Directions in Psychological Science,

7, 177-188.

Wichmann, F. A., & Hill, N. J. (2001): The psychometric function: I. Fitting,

sampling and goodness-of-fit. Perception & Psychophysics, 63, 1293-1313.

Yamasaki, H., LaBar, K. S. & McCarthy, G. (2002). Dissociable prefrontal

brain systems for attention and emotion. Proceedings of the National Academy

of Sciences, 99, 11447-11451.

Yeshurun, Y. & Carrasco, M. (1999). Spatial attention improves performance

in spatial resolution tasks. Vision Research, 39, 293-306.

Yiend, J., & Mathews, A. (2001). Anxiety and attention to threatening pictures.

The Quarterly Journal of Experimental Psychology Section A, 54(3), 665-681.

Yovel, G., Levy, J., Grabowecky, M., & Paller, K. A. (2003). Neural correlates

of the left-visual-field superiority in face perception appear at multiple stages of

205

face processing. Journal of Cognitive Neuroscience, 15, 462–474. doi:

10.1162/089892903321593162

Yovel, G., Tambini, A., & Brandman, T. (2008). The asymmetry of the fusiform

face area is a stable individual characteristic that underlies the left-visual-field

superiority for faces. Neuropsychologia, 46, 3061-3068. doi:

10.1016/j.neuropsychologia.2008.06.017