Transcript
Page 1: A unique look at face processing: the impact of masked faces on the processing of facial features

A unique look at face processing: the impact of

masked faces on the processing of facial features

Mark A. Williamsa,*, Simon A. Mossb, John L. Bradshawb

aDepartment of Psychology, School of Behavioural Science, University of Melbourne,

Parkville, Victoria, 3010, AustraliabDepartment of Psychology, School of Psychiatry, Psychology and Psychological Medicine,

Monash University, Clayton, Victoria, 3800, Australia

Received 20 September 2002; revised 14 May 2003; accepted 28 August 2003

Abstract

This experiment utilized a masked priming paradigm to explore the early processes involved in

face recognition. The first experiment investigated implicit processing of the eyes and mouth in an

upright face, using prime durations of 33 and 50 ms. The results demonstrate implicit processing of

both the eyes and mouth, and support the configural processing theory of face processing. The second

experiment used the same method with inverted faces and the third experiment was a combination of

Experiments 1 and 2. The fourth experiment utilized misaligned faces as the primes. Based on the

pattern of results from these experiments, we suggest that, when a face is inverted, the eyes and

mouth are initially processed individually and are not linked until a later stage of processing. An

upright face is proposed to be processed by analysis of its configuration, whereas an inverted face is

initially processed using first-order relational information, and then converted to an upright

representation and transferred to face specific regions for configural analysis.

q 2003 Elsevier B.V. All rights reserved.

Keywords: Face perception; Holistic processing; Configural processing; Masked priming

1. Introduction

Face perception and the subsequent recognition of social cues is a vital aspect of human

functioning. The face provides information not only about the age, gender and identity of

the individual, but also the intention and emotion. Faces are based on a similar

0022-2860/$ - see front matter q 2003 Elsevier B.V. All rights reserved.

doi:10.1016/j.cognition.2003.08.002

Cognition 91 (2004) 155–172

www.elsevier.com/locate/COGNIT

* Corresponding author. Fax: þ61-3-9347-6618.

E-mail address: [email protected] (M.A. Williams).

Page 2: A unique look at face processing: the impact of masked faces on the processing of facial features

configuration, and are continually changing. Despite this dynamic interplay of movement

and emotions, we are able to recognize hundreds of individuals under dramatically

different lighting conditions and orientations. This ability raises the question of how we

are able to differentiate individuals with such expertise. It has been claimed that the

specific process that provides us with this critical skill only occurs for faces, rather than

being a general process involved in the recognition of all objects (for a review, see

Kanwisher & Moscovitch, 2000).

All faces are comprised of the same fundamental configuration or arrangement of

features. Hence, some researchers claim that we process the relationship amongst these

features, and not merely the features themselves, to differentiate faces (for a review, see

Maurer, Le Grand, & Mondloch, 2002). In his classic paper, Yin (1969) demonstrated that

inversion resulted in a more pronounced deleterious effect on memory recognition for

faces than for other object categories including houses, airplanes, men in motion, or

faceless figures. This dramatic effect was ascribed to the disruption of configural

processing, which only affected faces. Since this time, the effect of face inversion on

processing has been studied extensively (e.g. Diamond & Carey, 1986; Farah, Wilson, &

Drain, 1998; Freire, Lee, & Symons, 2000; Haxby et al., 1999; Hillis, Hiscock, & Rexer,

1995; Kanwisher, Tong, & Nakayama, 1998; Leder & Bruce, 2000; Leder, Candrian,

Huber, & Bruce, 2001; Parr, Dove, & Hopkins, 1998; Rhodes, Brake, & Atkinson, 1993;

Tanaka & Farah, 1993). In a recent review, Maurer et al. (2002) discusses converging

results from many studies that demonstrate that face perception proceeds configurally and

that effects of inversion can be ascribed to the disruption of this process.

Many variants of configural processing have been proposed (Farah et al., 1998).

Specifically, three primary classes of processes have been posited. Conceivably, each class

may apply under different conditions. First-order relational processing involves the

determination of whether the structure matches a face-like configuration. In other words,

these processes determine the presence of facial features in a face-like configuration,

rather than an intricate analysis of the configuration of the face. Following these first-order

relational processes, which recognize the object as a face, additional processes that are

specific to facial analysis are invoked (Maurer et al., 2002).

First-order relational processing of faces has been demonstrated under a variety of

experimental conditions. For instance, experiments using schematic faces with only two

circles representing eyes and one line for the mouth have demonstrated patterns of results

that are specific to faces, such as fusiform face area (FFA) activation (Tong, Nakayama,

Moscovitch, Weinrib, & Kanwisher, 2000). Patients with spatial neglect seem to be less

likely to neglect a schematic face than a scrambled face (Vuilleumier, 2000). Priming

extinction patients with either two circles or two crosses within the context of a schematic

face reduces extinction of the two circles or crosses on subsequent presentations, despite

the absence of any schematic face surrounding them (Vuilleumier & Sagiv, 2001).

According to Moscovitch and Moscovitch (2000), when a face is inverted the object

processing system initially creates an upright representation of the face that is then

transferred to the FFA. To initiate transfer to the FFA, first-order configural processing is

suggested to be responsible for identifying the object as a face (Maurer et al., 2002).

Second-order relational processing is thought to be utilized when the identity of a face

needs to be ascertained. Second-order relational processing has been posited to compare

M.A. Williams et al. / Cognition 91 (2004) 155–172156

Page 3: A unique look at face processing: the impact of masked faces on the processing of facial features

the specific parameters of the target face with a prototype. These parameters reflect the

spacing between facial features (Diamond & Carey, 1986). Thus, to identify faces, these

processes compare the spacing of the features of a target face with the configuration of a

generic template (Leopold, O’Toole, Vetter, & Blanz, 2001). Each individual will exhibit

consistent deviations from this generic template, which enables familiar faces to be

recognized. Research investigating second-order relational processing has involved

changing the spacing between features. Several studies have demonstrated that even

minute changes to the spacing between the features can be readily perceived when the

faces are upright; however, this ability is dramatically affected when the faces are inverted.

This detrimental effect of inversion is not observed, however, when the features

themselves are changed (Freire et al., 2000; Leder & Bruce, 1998, 2000; Leder et al., 2001;

Macho & Leder, 1998).

The Thatcher Illusion, first demonstrated by Thompson (1980), provides striking

evidence that second-order relational processing arises only when the face is upright. This

illusion arises when the eyes and mouth are rotated 1808 within a face. When the stimulus

is upright, this change results in a bizarre face. When the altered face is inverted, however,

the stimulus appears less unusual. Presumably, these second-order relational processes

explore the configuration of features. When the face is inverted, these processes are

thwarted and thus modifications to facial configurations are overlooked.

Proponents of holistic processing dismiss the notion of a generic template face from

which discrepancies are compared. Instead, they propose that we store a separate gestalt

template for each and every face. In other words, faces cannot be reduced to a finite set of

features or spaces between features; instead, each face is stored as a unique form or gestalt

(Farah et al., 1998). Tanaka and Farah (1993) found that individual features such as eyes,

mouth, and nose are recognized more readily when displayed as part of a face than when

displayed in isolation. This effect, however, did not extend to scrambled faces, inverted

faces, or images of houses. Accordingly, these findings are compatible with the idea that

the face is processed holistically rather than piecemeal. This evidence for holistic

processing, however, could be ascribed to alternative mechanisms. The removal of facial

features also limits the information that is utilized by first- and second-order relational

processing.

The evidence for second-order relational processing has also been contested. The ease

of identification of subtle changes to the spacing between features of upright faces,

compared to inverted faces, could be attributed to a mismatch between the altered face and

the stored template or gestalt of the particular individual’s face rather than the change in

second-order relational information. Likewise, the Thatcher Illusion could be due a

mismatch with the stored gestalt, rather than the change in the spatial relationships.

The competing explanations of various findings highlight that theories of configural

processing are still debatable. There is no argument that inversion disrupts the

configuration of the face and causes a delay in recognition. The debate surrounds the

types of configural processing that underpin face perception.

We explored the types of configural processing involved in face perception, using a

masking paradigm. Masking is a technique commonly used to disrupt the processing of a

visual stimulus, which otherwise may continue to be processed after it is has been

physically terminated (Keyser & Perrett, 2002). Using a mask that precedes (forward

M.A. Williams et al. / Cognition 91 (2004) 155–172 157

Page 4: A unique look at face processing: the impact of masked faces on the processing of facial features

mask), and follows (backward mask), the presentation of a visual stimulus enables the

experimenter to control the duration of processing that is dedicated to the target.

Previous studies have demonstrated that the emotions of masked faces are implicitly

processed. Even when the face is masked to such an extent that participants are not aware

of its presentation, appropriate changes in physiological and brain activations have been

recorded (Morris, Ohman, & Dolan, 1998; Whalen et al., 1998). Masking therefore affords

us the opportunity to investigate the relationship between the features of the face prior to

awareness, as individual features of a masked face may be manipulated and the subsequent

effects on recognition can be observed. This paradigm has the potential to illuminate the

critical question of how they are processed.

2. Experiment 1

This experiment was concerned with the question of whether the specific features of the

eyes and mouth are initially processed by first-order relational, second-order relational, or

holistic perception. The aim of this experiment was first to investigate whether priming

could be achieved using individual facial features, and second to identify, through specific

condition contrasts, the early mechanisms involved in face perception.

Three types of prime–target pairs were used: (1) congruent (e.g. open eyes only in both

prime and target); (2) incongruent (e.g. open eyes only in prime and open mouth only in

target); and (3) dual (both mouth and eyes open in the prime only, followed by either type of

target, i.e. either eyes or mouth – but not both – open, see Fig. 1). Participants made a

speeded decision regarding the target face: whether eyes (response 1) or mouth (response 2)

Fig. 1. An example of the series of presentations for (A) congruent, (B) incongruent, and (C) dual trials

(not to scale).

M.A. Williams et al. / Cognition 91 (2004) 155–172158

Page 5: A unique look at face processing: the impact of masked faces on the processing of facial features

or neither (catch trials – withhold responding) were open. Note that the identity of the

faces changed between prime and target.

The dual prime condition, in which both eyes and mouth are open, yields different

predictions based on holistic, first-order relational, and second-order relational processing

theories. According to holistic processing theories, in the dual condition, the overall form

or gestalt of the prime and target will differ. Hence, this condition should yield the same

response times as incongruent trials. Conversely, according to first-order relational

processing theories, the features are processed independently, and thus the dual condition

should yield analogous response times to the congruent trials. Second-order relational

processing theories predict that the relationship between the features rather than the

features themselves are processed. Presumably, then, primes in the dual condition

comprise a more similar configuration to the target than incongruent primes, but less

similar configuration to the target than congruent primes. In other words, response times in

this dual condition should be intermediate between the congruent and incongruent trials.

2.1. Method

2.1.1. Participants

Twelve right handed University students (six male and six female, mean age 24.6 years,

SD ¼ 3:32) participated and were paid for their time.

2.1.2. Apparatus and stimuli

Coloured photographs were generated using a digital camera and edited using Adobew

Photoshop. The background was black and the mean luminance was approximately the

same for all pictures. These pictures were converted to 24-bit bitmaps for display. The

forward mask and target (visual angle 6.48) were 25% larger than the prime (visual angle

5.18). The test computer was an IBM compatible PC with a 750 Hz Intel Pentium III

processor, 128 MB RAM, a Trident CyberBlade video card with 16 MB video memory,

and a MAG Innovision DJ530 15-inch CRT monitor. The video card was set at a refresh

rate of 60 Hz and screen resolution of 800 £ 600 £ 16. The program was written in Visual

Basic (Version 6) using Direct-X 8 technology. Priority settings were optimized to ensure

accurate display durations.

The forward mask was the experimenter’s face with both eyes and mouth closed.

Primes were also the experimenter’s face with either eyes, mouth, or both open. Note that

primes were 25% smaller than the forward mask to avoid any apparent movement. The use

of the experimenter’s face ensured that all participants were familiar with the face and the

facial features remained constant. The target face was a male of approximately the same

age as the experimenter and was the same size as the forward mask, acting as a complete

backward mask of the prime (see Fig. 1). It has been demonstrated that face masks are

more appropriate than other stimuli when masking a face (Costen, Shepherd, Ellis, &

Craw, 1994). The target face had open mouth or open eyes, except on 50% of trials in

which both eyes and mouth were closed (catch trials). It should be noted that the same two

faces were used throughout the experiments and therefore caution should be exercised

when the generalizability of these results is contemplated.

M.A. Williams et al. / Cognition 91 (2004) 155–172 159

Page 6: A unique look at face processing: the impact of masked faces on the processing of facial features

2.1.3. Design

A three-factor within-subjects design was used, in which factors were Prime duration

(33 or 50 ms), Target type (open mouth, open eyes) and Congruency (congruent,

incongruent, dual). Prime duration was blocked whereas Target type and Congruency

were randomized within each block. All factors were fully crossed, yielding 12

experimental conditions.

2.1.4. Procedure

Fig. 1 shows the sequence of stimuli in a single trial. Participants were asked to

maintain fixation on the centre of the screen throughout the experiment. Each trial

commenced with the forward mask appearing in the centre of the screen for 1500 ms. This

mask was then replaced with the prime in the same location. The prime remained on the

screen for either 33 or 50 ms and was then replaced with the target. Participants pressed

one button to indicate whether the target face’s mouth was open, and another if the eyes

were open. They were instructed to respond as quickly as possible. Both eyes and mouth

were never simultaneously open in the target face. The position (left/right) of the buttons

was counterbalanced between subjects. In 50% of trials, neither the eyes nor mouth were

open on the target face (catch trials), and no response was required. Participants completed

four blocks of 240 trials, two blocks at each prime duration resulting in a total of 960 trials

(40 trials per condition). Following each experimental block, participants were asked to

describe what they perceived between the fixation and target displays. Participants were

then asked if they could identify the face that appeared. The criterion for exclusion from

this study was explicit recognition of the prime face as the experimenter.

2.2. Results

No participant was able to identify the prime. Outliers were defined as reaction times

(RT) greater than 3 SD from each individual’s mean or less than 150 ms, and were

excluded from analysis (less than 2%). Mean correct RT were calculated for each of the 12

conditions. A three-way within-subjects analysis of variance (ANOVA) conducted on the

RT data yielded a significant main effect of Target (Fð1; 11Þ ¼ 5:06, P , 0:05).

Participants were faster to respond to open eyes (M ¼ 421:8 ms, SE ¼ 2:9) than an

open mouth (M ¼ 428:4 ms, SE ¼ 3:1). A significant main effect of Congruency

(Fð1; 11Þ ¼ 51:62, P , 0:001) and a significant Duration £ Congruency interaction

(Fð2; 22Þ ¼ 9:22, P , 0:001) were also evident. No other effects or interactions reached

significance (P . 0:1).

Simple main effects analysis (Bonferroni adjusted) of the Duration £ Congruency

interaction uncovered a significant difference between each of the congruency conditions

at both durations. Fig. 2 illustrates that, at a prime duration of 33 ms, RT on congruent

trials were significantly shorter than on incongruent trials and dual trials, and RT on

incongruent trials were significantly longer than on dual trials. At a prime duration of

50 ms, an analogous pattern emerged. The only significant difference evident between

durations was slower mean RT in the incongruent condition, when prime duration was

50 ms rather than 33 ms (P , 0:05). It can be seen from the increase in the percentage

M.A. Williams et al. / Cognition 91 (2004) 155–172160

Page 7: A unique look at face processing: the impact of masked faces on the processing of facial features

of errors in concordance with the RT data that there was no speed/accuracy trade-off

(see Fig. 2).

2.3. Discussion

The results of Experiment 1 show that prime faces influence RT to target faces at prime

durations of 33 and 50 ms. Congruent prime–target combinations resulted in faster RT

than incongruent prime–target combinations. More importantly, the dual condition, in

which both the eyes and mouth were open in the prime, yielded RT that were faster than

incongruent trials yet slower than congruent trials. The only difference in results between

the prime durations was that responses to incongruent trials were slower at the longer

prime duration. Target type also had an effect, with participants reacting faster to open

eyes (M ¼ 422 ms) in the target face than to an open mouth (M ¼ 428 ms).

The congruency effects observed at both prime durations provide evidence for implicit

processing of the eyes and mouth. At both prime durations, participants were unable to identify

the prime. Nevertheless, these brief primes influenced subsequent responses to the target face.

The dual condition, in which the eyes and mouth were both open in the prime, resulted

in intermediate RT in comparison to the other congruency conditions for both target types.

This result suggests that the eyes and mouth are processed together rather than as

individual parts. That is, if the eyes and mouth were processed independently, open eyes in

the prime would not influence responses to open mouth in the target, and vice versa. If

holistic processing occurred, responses should be akin to incongruent trial responses. The

results, therefore, demonstrate that second-order relational information of the face is

processed, rather than the holistic information.

The only difference observed between the two prime durations was that participants

exhibited slower RT in the incongruent condition when the prime duration was 50 ms as

compared with 33 ms. It is likely that a stimulus that is presented for a longer duration

Fig. 2. Mean reaction times in milliseconds for each of the congruency conditions at prime durations of either 33

or 50 ms collapsed across Target type. The mean percentage of errors in each condition is displayed in

parentheses and the error bars reflect one standard error.

M.A. Williams et al. / Cognition 91 (2004) 155–172 161

Page 8: A unique look at face processing: the impact of masked faces on the processing of facial features

produces more extensive or protracted activation as the processing is more in depth. In an

incongruent condition, then, increased inhibition may be required to overcome the prime

activation, resulting in protracted RT. The absence of any corresponding decrease in RT to

congruent trials is likely to reflect a ceiling effect: participants were receiving maximal

assistance from the prime congruency even at the 33 ms duration, and therefore no further

improvement could be achieved.

There are fundamental differences between the eyes and the mouth that account for the

faster responses to open eye compared with open mouth targets. Specifically, there is a

luminance discrepancy between the white sclera of the eyes and the rest of the face that

may act as an exogenous cue that is not present in an open mouth.

In summary, the results suggest that the eyes and mouth are processed together prior to

awareness. This finding supports the second-order relational theory of face recognition,

that the relationship between the parts is processed, which is then compared to a generic

template. Inversion of the face, however, has been claimed to affect configural processing.

The next experiment explored this idea by inverting the stimuli.

3. Experiment 2

In this experiment, all stimuli from Experiment 1 were rotated 1808 to create inverted

faces. The adverse effect of inversion on face recognition and memory has been well

documented (e.g. Bruce & Langton, 1994; Farah et al., 1998; Tanaka & Farah, 1993). As

discussed previously, this finding has been used as evidence for configural encoding

theories of face perception. When a face is inverted, the spatial relationships change and

hence, this affects the processing of second-order relational information.

We examined the effect of inversion on the implicit processing of the eyes and mouth, as

demonstrated in Experiment 1. If an inverted face is processed by first-order relational

information, that is, the parts, then presenting both eyes and mouth open in the prime (the dual

condition) should result in response times analogous to those in the congruent trials,

regardless of the target type. For example, an open mouth in the prime should not

compromise processing of open eyes in the target if these parts are processed independently.

If, however, inverted faces are processed by second-order relational information, reflecting

inter-feature spacing, we should observe analogous results to those of Experiment 1, with RT

in the dual condition intermediate between the congruent and incongruent conditions.

3.1. Method

3.1.1. Participants

Twelve right handed University students (five male and seven female, mean age 23.6

years, SD ¼ 3:94) participated in the experiment and were paid for their time.

3.1.2. Apparatus and stimuli

All stimuli used in Experiment 1 were rotated by 1808 to produce inverted masks, primes

and targets. All other apparatus and materials were identical to those in Experiment 1.

M.A. Williams et al. / Cognition 91 (2004) 155–172162

Page 9: A unique look at face processing: the impact of masked faces on the processing of facial features

3.1.3. Design and procedure

The design and procedure were the same as Experiment 1.

3.2. Results

Participants were again unable to identify the prime. Outliers were defined as RT

greater than 3 SD from each individual mean or less than 150 ms, and were removed prior

to analysis (less than 2%). As in Experiment 1, mean RT were calculated for each of the 12

conditions. A three-way within-subjects ANOVA conducted on the RT data yielded a

significant main effect of Target type (Fð1; 11Þ ¼ 18:28, P , 0:001). Responses were

again faster to open eyes (M ¼ 441:9 ms, SE ¼ 2:6) than to open mouth (M ¼ 453:1 ms,

SE ¼ 2:8). A significant main effect of Congruency (Fð1; 11Þ ¼ 64:37, P , 0:001) and a

significant Duration £ Congruency interaction (Fð2; 22Þ ¼ 5:76, P , 0:01) were also

observed. No other effects or interactions reached significance (P . 0:1).

Simple main effects analysis (Bonferroni adjusted) of the Duration £ Congruency

interaction showed a significant difference between congruent and incongruent conditions

at both durations, as can be seen in Fig. 3. At a prime duration of 33 ms, RT on congruent

trials were significantly shorter than on incongruent trials, and the difference between the

dual condition and the incongruent condition was also significant. There was no difference

observed between the dual condition and the congruent condition. At a prime duration of

50 ms, a similar pattern emerged, with the exception that a significant difference between

the congruent and dual conditions was also observed (P , 0:05). No other simple main

effects reached significance (P . 0:1). It can be seen from the increase in the percentage of

errors in concordance with the RT data that there is no speed/accuracy trade-off (see Fig. 3).

Fig. 3. Mean reaction times in milliseconds for each of the congruency conditions at prime durations of either 33

or 50 ms. The percentage of errors in each condition is displayed in parentheses and the error bars reflect one

standard error.

M.A. Williams et al. / Cognition 91 (2004) 155–172 163

Page 10: A unique look at face processing: the impact of masked faces on the processing of facial features

3.3. Discussion

The results from Experiment 2 show that inverted face primes influence RT to inverted

target faces, at both prime durations. Congruent prime–target trials resulted in faster RT

than incongruent prime–target combinations. Unlike Experiment 1, however, there was

no significant difference in RT between the congruent and dual conditions at the shortest

prime duration of 33 ms, suggesting a difference in the perceptual mechanisms involved.

At the longer prime duration (50 ms), the dual condition mimicked the result of

Experiment 1, with the dual condition mean RT intermediate between incongruent and

congruent response times. Analogous to Experiment 1, responses to open eyes (M ¼ 442

ms) were faster than to open mouth (M ¼ 453 ms).

As in Experiment 1, the observed congruency effects provide evidence for implicit

processing of the eyes and mouth. At a prime duration of 33 ms, there was no significant

difference between the dual and the congruent conditions, indicating that the eyes and

mouth may be processed independently. At the prime duration of 50 ms, however, the

mean RT in the dual condition was intermediate between the means for congruent and

incongruent conditions. At the longer prime duration, therefore, the eyes and mouth may

be linked in a way that does not seem to occur at earlier stages of processing. This pattern

suggests that, initially, first-order relational information is processed, with second-order

relational information encoding occurring slightly later.

These results have interesting implications for theories that concern the effect of face

inversion on face recognition. Only at the short prime duration was there evidence for

inverted faces being processed by first-order relational information. At the longer prime

duration, the results show a different pattern indicating an interaction in processing of the

parts. These findings suggest that inversion may not completely disrupt second-order

relational information processing, but rather protract an initial parts-based phase. It could

be argued, however, that the absence of a difference between congruent and dual inverted

conditions may reflect inadequate power. In Experiment 3, therefore, we sought to

replicate our findings and strengthen our conclusions by directly examining the interaction

between the upright and inverted faces at the shortest prime duration.

4. Experiment 3

In this experiment, we examined whether the patterns of results observed at the shortest

prime duration in the first two experiments were robust and replicable or simply a

consequence of limited statistical power. The same protocol was utilized as in the previous

experiments, with the exception of limiting the prime duration to 33 ms, and using both

upright and inverted stimuli. If our interpretation of the results of Experiments 1 and 2 is

correct, then upright faces are processed by configural information, whereas inverted faces

are initially processed by parts. This outcome would be demonstrated by a difference in

results for inverted versus upright stimuli at this short prime duration. An interaction

between stimuli orientation types, therefore, would support and replicate our findings from

the previous two experiments.

M.A. Williams et al. / Cognition 91 (2004) 155–172164

Page 11: A unique look at face processing: the impact of masked faces on the processing of facial features

4.1. Method

4.1.1. Participants

Twelve right handed University students (six male and six female, mean age 26.5 years,

SD ¼ 2:0) participated in the experiment and were paid for their time.

4.1.2. Apparatus and stimuli

All stimuli used in Experiments 1 and 2 were utilized. All other apparatus and materials

were identical to those in Experiment 1.

4.1.3. Design

A two-factor within-subjects design was used, in which factors were Orientation

(upright or inverted) and Congruency (congruent, incongruent, dual). Orientation was

blocked whereas Congruency was randomized within each block. All factors were fully

crossed, yielding six experimental conditions.

4.1.4. Procedure

The procedure was the same as Experiment 1.

4.2. Results

Participants were again unable to identify the prime. Outliers were defined as RT

greater than 3 SD from each individual mean or less than 150 ms, and were removed prior

to analysis (less than 2%). Mean RT were calculated for each of the six conditions. A two-

way within-subjects ANOVA conducted on the RT data yielded a significant main effect

of Congruency (Fð1; 11Þ ¼ 42:18, P , 0:001) and a significant Orientation £ Congruency

interaction (Fð2; 22Þ ¼ 4:39, P , 0:05) was also observed. No other effects or interactions

reached significance (P . 0:1).

Simple main effects analysis (Bonferroni adjusted) of the Orientation £ Congruency

interaction showed that the impact of congruency differed for upright and inverted faces,

as can be seen in Fig. 4. The upright manipulation yielded congruent RT that were

significantly shorter than those on the dual trials which, in turn, were significantly shorter

than the incongruent RT (P , 0:05). In contrast, the inversion manipulation resulted in

congruent RT that were not significantly different from dual RT (P ¼ 0:13), whilst the

difference between the dual condition and the incongruent condition was significant.

The only significant main effect evident between orientations was the slower mean RT in

the dual condition, when stimuli were upright rather than inverted (P , 0:05). No other

simple main effects reached significance (P . 0:1). It can be seen from the increase in the

percentage of errors in concordance with the RT data that there is no speed/accuracy trade-

off (see Fig. 4).

4.3. Discussion

The results from Experiment 3 demonstrate that both upright and inverted face primes

influence RT to target faces, consistent with Experiments 1 and 2. Congruent prime–target

M.A. Williams et al. / Cognition 91 (2004) 155–172 165

Page 12: A unique look at face processing: the impact of masked faces on the processing of facial features

trials resulted in faster RT than incongruent prime–target combinations for both upright

and inverted faces. Critically, an interaction between orientation and congruency was

observed, due to a difference between congruent and dual conditions with upright stimuli

which was not present when the stimuli were inverted. This finding demonstrates that the

differential effects observed in Experiments 1 and 2 are reliable.

As in Experiments 1 and 2, the observed congruency effects provide evidence for

implicit processing of the eyes and mouth. Again, inversion of the stimuli removed the

difference between the dual and congruent conditions that was present for upright stimuli.

This finding indicates that inversion causes the eyes and mouth to be processed

independently.

There is, however, another possible explanation for this particular pattern of results. It

is conceivable that there are two separate processing systems for the eyes and mouth,

perhaps based on simple visual cues such as luminance or contrast changes: if so, the

results observed in Experiments 1 and 3 for upright faces could easily be explained. While

congruent and incongruent trials would cause facilitation and interference effects,

respectively, dual trials would result in interference via one system and facilitation via the

other. The net effect would be intermediate RT on dual trials, as observed.

On inversion the effects of such systems should still occur, assuming these processes

are equally active. It has been suggested, however, as discussed earlier, that inversion

disrupts face processing (Yin, 1969). In this case, it could be argued that inversion also

prevents these two systems from being triggered and as such the inversion effects

observed in the previous experiments could be explained as a disruption of these two

separate systems as opposed to second-order configural processing. Alternatively,

inversion may retard response times because of limited experience with inverted faces.

As such, decision-associated factors may underlie the pattern of results observed. To

address these issues, we constructed a novel priming paradigm using ‘misaligned’ faces

(Young, Hellawell, & Hay, 1987).

Fig. 4. Mean reaction times in milliseconds for each of the congruency conditions for upright and inverted faces at

a prime duration of 33 ms. The percentage of errors in each condition is displayed in parentheses and the error

bars reflect one standard error.

M.A. Williams et al. / Cognition 91 (2004) 155–172166

Page 13: A unique look at face processing: the impact of masked faces on the processing of facial features

5. Experiment 4

Young et al. (1987) reported a striking demonstration of second-order configural

processing that arose when the top half of one face was aligned with the bottom half of

another to create a composite face. They found that participants were slower to recognize

either the top or bottom half of these composite faces relative to faces in which the two

halves were misaligned or the entire stimulus was inverted. They argued that aligned

composite faces are fused automatically and perceived as a new whole face rather than two

different halves (Young et al., 1987). Importantly for our purposes, misaligned faces were

not processed via a second-order configural processing system, suggesting that misaligned

faces are appropriate controls to test the possibility that eyes and mouth processing reflect

two parallel systems.

In Experiment 4, we misaligned the prime faces to interrupt second-order configural

information (see Fig. 5). If separate processing systems for the eyes and mouth underlie the

intermediate RT observed in Experiments 1 and 3 for upright faces, then the same effect

should be evident when the upper and lower parts of the prime are misaligned. If, on the

other hand, the effect evident for upright faces is due to second-order configural

processing, misalignment should generate a pattern of findings that is analogous to the

results that were observed for inverted faces.

5.1. Method

5.1.1. Participants

Twelve right handed University students (seven male and five female, mean age 27

years, SD ¼ 2:61) participated in the experiment and were paid for their time.

5.1.2. Apparatus and stimuli

The top and bottom halves of the primes from Experiment 1 were misaligned by

moving them horizontally to ensure an overlap of approximately 66% of the face (Fig. 5).

Fig. 5. An example of a misaligned prime face.

M.A. Williams et al. / Cognition 91 (2004) 155–172 167

Page 14: A unique look at face processing: the impact of masked faces on the processing of facial features

Both combinations of left and right adjustments were utilized. Note that as the prime

face was 25% smaller than the target face, misaligned primes were still within the

boundaries of targets. All other apparatus and materials were identical to those in

Experiment 1.

5.1.3. Design

A within-subjects design was used, with the factor of Congruency (congruent,

incongruent, dual).

5.1.4. Procedure

The procedure was the same as Experiment 1.

5.2. Results

Participants were again unable to identify the prime. Outliers were defined as RT

greater than 3 SD from each individual mean or less than 150 ms, and were removed prior

to analysis (less than 2%). Mean RT were calculated. A one-way within-subjects ANOVA

conducted on the RT data yielded a significant main effect of Congruency

(Fð2; 22Þ ¼ 27:49, P , 0:001).

Simple main effects analysis (Bonferroni adjusted) of the Congruency main effect

showed that there was a significant difference between congruent and incongruent, and

between the dual and incongruent conditions (P , 0:05); however, congruent trials were

not significantly different than dual trials (P ¼ 0:17), as can be seen in Fig. 6. It can be

seen from the increase in the percentage of errors in concordance with the RT data that

there is no speed/accuracy trade-off (see Fig. 6).

Fig. 6. Mean reaction times in milliseconds for each of the congruency conditions at a prime duration of 33 ms.

The percentage of errors in each condition is displayed in parentheses and the error bars reflect one standard error.

M.A. Williams et al. / Cognition 91 (2004) 155–172168

Page 15: A unique look at face processing: the impact of masked faces on the processing of facial features

5.3. Discussion

The results of Experiment 4, as in the previous experiments, demonstrate that prime

faces influence RT to target faces. Congruent prime–target combinations resulted in faster

RT than incongruent prime–target combinations. Critically, in the dual condition, in

which both the eyes and mouth were open in the prime, RTs were not different from

congruent trials and were faster than incongruent trials.

Misalignment of a face interrupts the second-order configural processing (Young et al.,

1987). In the current experiment, we used this property of misalignment to ascertain that

the results obtained in the previous experiments may not be attributed to independent

processing of the eyes and mouth nor decision-associated factors. When second-order

configural processes are disrupted, the pattern of results observed for upright faces mimics

the pattern of results observed for inverted faces, as revealed in Experiments 2 and 3. The

results for upright faces in the previous experiments can, therefore, be ascribed to second-

order configural processing.

6. General discussion

The main focus of this study was to explore configural theories of face processing by

investigating implicit processing of two important parts of the face: the eyes and mouth.

Experiment 1 revealed a significant congruency effect at prime durations of 33 and 50 ms,

with participants fastest to react to targets preceded by a congruent prime and slowest in

incongruent conditions. Critically, the dual condition, in which the prime contained both

open eyes and open mouth, produced intermediate response times between the congruent

and incongruent conditions. In Experiment 2, the effect of inversion on this implicit

processing was examined. The dual condition at the shortest prime duration produced RT

analogous to the congruent condition responses. At the longer prime duration, the results

replicated the pattern observed in Experiment 1. Experiment 3 investigated whether these

differences between Experiments 1 and 2 could be replicated and statistically validated.

An interaction between upright and inverted faces was demonstrated and the overall

pattern of results showed that the original findings were robust. Experiment 4 utilized

‘misaligned faces’ as primes to falsify an alternative explanation for our results.

In each experiment, participants executed a speeded judgement about the eyes and

mouth. As participants were never required to identify the faces, it could be argued that

subjects were able to focus on these parts of the face, which precludes the activation of

holistic processes. However, the pattern of results in each experiment and the changes

observed when the configuration of the primes was disrupted, via inversion or

misalignment, challenge this argument. Furthermore, there is evidence that faces are

processed automatically (Boutet, Gentes-Hawn, & Chaudhuri, 2002; Vuilleumier, 2000;

Vuilleumier & Sagiv, 2001; Winston, Strange, O’Doherty, & Dolan, 2002) even when

detrimental to the task (Young et al., 1987) and, therefore, simply attending to the face

should activate face processes.

We have applied a strict definition of holistic processing in this study. Of course,

holistic processing could be defined as merely focusing on the similarities between the real

M.A. Williams et al. / Cognition 91 (2004) 155–172 169

Page 16: A unique look at face processing: the impact of masked faces on the processing of facial features

and stored representations of the object (Schwarzer, Kuefer, & Wilkening, 1999).

According to this definition, second-order configural and holistic processing cannot be

disentangled. However, a model must be operationalized to be tested and to that end we

have used a strict interpretation of both processing types to enable a systematic

investigation.

Overall, the results support the concept of implicit processing of the eyes and mouth

prior to awareness. Upright faces were found to be processed by second-order relational

information. Inverting or misaligning the faces, however, resulted in first-order relational

processing. Intriguingly, inverted faces were found to later undergo second-order

relational processing, supporting the notion that they may be transferred to the FFA after

initial processing and identification as faces. This proposition is consistent with

neuroimaging studies that have found the FFA, which is specific to face processing, to

be more active in response to inverted faces than other objects (Haxby et al., 1999;

Kanwisher et al., 1998; Sagiv & Bentin, 2001; Tong et al., 2000).

Moscovitch and Moscovitch (2000) have suggested that, when a face is inverted, the

object processing system initially creates an upright representation of the face that it then

transfers to the FFA. Our current results support this theory; with the face inverted, the

eyes and mouth were processed independently (first-order relational information) at 33 ms,

yet at 50 ms they appeared to be processed by second-order relational information

consistent with upright face processing. In other words, when a face is inverted, the

configuration is changed resulting in initial processing by first-order relational information

for identification as a face and transformation to an upright representation. Following this

phase, second-order relational processing can then proceed. Of course, whether or not

these two stages are completely distinct or part of a cascade of neural processes cannot yet

be established.

There has been a large number of studies demonstrating the detrimental effect of

inversion on face recognition (e.g. Diamond & Carey, 1986; Farah et al., 1998; Freire et al.,

2000; Haxby et al., 1999; Hillis et al., 1995; Kanwisher et al., 1998; Leder & Bruce, 2000;

Leder et al., 2001; Parr et al., 1998; Rhodes et al., 1993; Tanaka & Farah, 1993); however,

the current results suggest that 50 ms presentation of an inverted face is sufficient to allow

conversion to an upright representation and subsequent second-order relational

processing. This study did not examine whether this transformation to an upright

representation is precise and, as such, it cannot be assumed that a person would be accurate

at recognizing or matching the faces, instead of merely following the more limited

requirements of the current paradigm. Based on the previous literature on face inversion, it

appears that the transformation may in fact be crude. As such, although configural

processing occurs, the transformed representation may be degraded resulting in decreased

accuracy in recognition tasks.

As discussed, backward masking is a technique used to control the available time of

initial processing of a visual stimulus and is used to investigate how that stimulus is

processed during the first stages of perception. Interestingly, even at the shortest prime

duration, second-order relational processing was observed for upright faces, suggesting

that face perception occurs pre-attentively. These results support several previous studies

demonstrating automatic pre-attentive processing of faces and facial expressions (Boutet

et al., 2002; De Gelder, Pourtois, Van Raamsdonk, Vroomen, & Weiskrantz, 2001;

M.A. Williams et al. / Cognition 91 (2004) 155–172170

Page 17: A unique look at face processing: the impact of masked faces on the processing of facial features

De Gelder, Vroomen, Pourtois, & Weiskrantz, 1999; Morris, De Gelder, Weiskrantz, &

Dolan, 2001; Morris et al., 1998; Vuilleumier, 2000; Vuilleumier & Sagiv, 2001; Whalen

et al., 1998; Winston et al., 2002).

In summary, we have conclusively shown in a series of four experiments that within the

context of task requirements second-order configural processing, rather than holistic

processing, underlies face perception. In addition, we have demonstrated that inverted

faces are initially processed by first-order (parts-based) assessment before second-order

relational processing is initiated. These experiments show the value of systematic

investigation of implicit face perception using masked priming.

Acknowledgements

We thank Chris Chambers, Belinda Howard and Anina Rich for their suggestions on an

earlier draft of the manuscript.

References

Boutet, I., Gentes-Hawn, A., & Chaudhuri, A. (2002). The influence of attention on holistic face encoding.

Cognition, 84, 321–341.

Bruce, V., & Langton, S. R. H. (1994). The use of pigmentation and shading information in recognizing the sex

and identities of faces. Perception, 23, 803–822.

Costen, N. P., Shepherd, J. W., Ellis, H. D., & Craw, I. (1994). Masking of faces by facial and non-facial stimuli.

In G. W. Humphreys (Ed.), Object and face recognition (1) (pp. 227–251). Special issue of visual cognition,

Hillsdale, NJ: Lawrence Erlbaum Associates.

De Gelder, B., Pourtois, G., Van Raamsdonk, M., Vroomen, J., & Weiskrantz, L. (2001). Unseen stimuli

modulate conscious visual experience: evidence from inter-hemispheric summation. Cognitive Neuroscience

and Neuropsychology, 12(2), 385–391.

De Gelder, B., Vroomen, J., Pourtois, G., & Weiskrantz, L. (1999). Non-conscious recognition of affect in the

absence of striate cortex. NeuroReport, 10, 3759–3763.

Diamond, R., & Carey, S. (1986). Why faces are and are not special: an effect of expertise. Journal of

Experimental Psychology: General, 115(2), 107–117.

Farah, M. J., Wilson, K. D., & Drain, M. (1998). What is “special” about face perception? Psychological Review,

105(3), 482–498.

Freire, A., Lee, K., & Symons, L. A. (2000). The face-inversion effect as a deficit in the encoding of configural

information: direct evidence. Perception, 29, 159–170.

Haxby, J. V., Ungerleider, L. G., Clark, V. P., Schouten, J. L., Hoffman, E. A., & Martin, A. (1999). The

effect of face inversion on activity in human neural systems for face and object perception. Neuron, 22,

189–199.

Hillis, S. K., Hiscock, M., & Rexer, J. L. (1995). Dual-task interference patterns reveal differential processing of

upright and inverted faces. Brain and Cognition, 28, 155–172.

Kanwisher, N., & Moscovitch, M. (2000). The cognitive neuroscience of face processing: an introduction.

Cognitive Neuropsychology, 17(1/2/3), 1–11.

Kanwisher, N., Tong, F., & Nakayama, K. (1998). The effect of face inversion on the human fusiform face area.

Cognition, 68, B1–B11.

Keyser, C., & Perrett, D. (2002). Visual masking and RSVP reveal neural competition. Trends in Cognitive

Sciences, 6, 120–125.

Leder, H., & Bruce, V. (1998). Local and relational aspects of face distinctiveness. Quarterly Journal of

Experimental Psychology: A, Human Experimental Psychology, 51A(3), 449–473.

M.A. Williams et al. / Cognition 91 (2004) 155–172 171

Page 18: A unique look at face processing: the impact of masked faces on the processing of facial features

Leder, H., & Bruce, V. (2000). When inverted faces are recognised: the role of configural information in face

recognition. Quarterly Journal of Experimental Psychology: A, Human Experimental Psychology, 53,

513–536.

Leder, H., Candrian, G., Huber, O., & Bruce, V. (2001). Configural features in the context of upright and inverted

faces. Perception, 30, 73–83.

Leopold, D. A., O’Toole, A. J., Vetter, T., & Blanz, V. (2001). Prototype-referenced shape encoding revealed by

high-level aftereffects. Nature Neuroscience, 4, 89–94.

Macho, S., & Leder, H. (1998). Your eyes only? A test of interactive influence in the processing of facial features.

Journal of Experimental Psychology: Human Perception and Performance, 24(5), 1486–1500.

Maurer, D., Le Grand, R., & Mondloch, C. J. (2002). The many faces of configural processing. Trends in

Cognitive Sciences, 6(6), 255–260.

Morris, J. S., De Gelder, B., Weiskrantz, L., & Dolan, R. J. (2001). Differential extrageniculostriate and amygdala

responses to presentation of emotional faces in a cortically blind field. Brain, 124, 1241–1252.

Morris, J. S., Ohman, A., & Dolan, R. J. (1998). Conscious and unconscious emotional learning in the human

amygdala. Nature, 393, 467–470.

Moscovitch, M., & Moscovitch, D. A. (2000). Super face-inversion effects for isolated internal features and

fractured faces. Cognitive Neuropsychology, 17, 201–219.

Parr, L. A., Dove, T., & Hopkins, W. D. (1998). Why faces may be special: evidence of the inversion effect in

chimpanzees. Journal of Cognitive Neuroscience, 10(5), 615–622.

Rhodes, G., Brake, S., & Atkinson, A. P. (1993). What’s lost in inverted faces? Cognition, 47, 25–57.

Sagiv, N., & Bentin, S. (2001). Structural encoding of human and schematic faces: holistic and part-based

processes. Journal of Cognitive Neuroscience, 13(7), 937–951.

Schwarzer, G., Kuefer, I., & Wilkening, F. (1999). Learning categories by touch: on the development of holistic

and analytic processing. Memory and Cognition, 27(5), 868–877.

Tanaka, J. W., & Farah, M. J. (1993). Parts and wholes in face recognition. Quarterly Journal of Experimental

Psychology, 46A, 225–245.

Thompson, P. (1980). Margaret Thatcher – a new illusion. Perception, 9, 483–484.

Tong, F., Nakayama, K., Moscovitch, M., Weinrib, O., & Kanwisher, N. (2000). Response properties of the

human fusiform face area. Cognitive Neuropsychology, 17(1/2/3), 257–279.

Vuilleumier, P. (2000). Faces call for attention: evidence from patients with visual extinction. Neuropsychologia,

38, 693–700.

Vuilleumier, P., & Sagiv, N. (2001). Two eyes make a pair: facial organization and perceptual learning reduce

visual extinction. Neuropsychologia, 39, 1144–1149.

Whalen, P. J., Rauch, S. L., Etcoff, N. L., McInerney, S. C., Lee, M. B., & Jenike, M. A. (1998). Masked

presentations of emotional facial expressions modulate amygdala activity without explicit knowledge.

Journal of Neuroscience, 18(1), 411–418.

Winston, J. S., Strange, B. A., O’Doherty, J., & Dolan, R. J. (2002). Automatic and intentional brain responses

during evaluation of trustworthiness of faces. Nature Neuroscience, 5(3), 277–283.

Yin, R. K. (1969). Looking at upside-down faces. Journal of Experimental Psychology, 81, 141–145.

Young, A. W., Hellawell, D., & Hay, D. C. (1987). Configural information in face perception. Perception, 16,

747–759.

M.A. Williams et al. / Cognition 91 (2004) 155–172172


Top Related