Download - A unique look at face processing: the impact of masked faces on the processing of facial features
A unique look at face processing: the impact of
masked faces on the processing of facial features
Mark A. Williamsa,*, Simon A. Mossb, John L. Bradshawb
aDepartment of Psychology, School of Behavioural Science, University of Melbourne,
Parkville, Victoria, 3010, AustraliabDepartment of Psychology, School of Psychiatry, Psychology and Psychological Medicine,
Monash University, Clayton, Victoria, 3800, Australia
Received 20 September 2002; revised 14 May 2003; accepted 28 August 2003
Abstract
This experiment utilized a masked priming paradigm to explore the early processes involved in
face recognition. The first experiment investigated implicit processing of the eyes and mouth in an
upright face, using prime durations of 33 and 50 ms. The results demonstrate implicit processing of
both the eyes and mouth, and support the configural processing theory of face processing. The second
experiment used the same method with inverted faces and the third experiment was a combination of
Experiments 1 and 2. The fourth experiment utilized misaligned faces as the primes. Based on the
pattern of results from these experiments, we suggest that, when a face is inverted, the eyes and
mouth are initially processed individually and are not linked until a later stage of processing. An
upright face is proposed to be processed by analysis of its configuration, whereas an inverted face is
initially processed using first-order relational information, and then converted to an upright
representation and transferred to face specific regions for configural analysis.
q 2003 Elsevier B.V. All rights reserved.
Keywords: Face perception; Holistic processing; Configural processing; Masked priming
1. Introduction
Face perception and the subsequent recognition of social cues is a vital aspect of human
functioning. The face provides information not only about the age, gender and identity of
the individual, but also the intention and emotion. Faces are based on a similar
0022-2860/$ - see front matter q 2003 Elsevier B.V. All rights reserved.
doi:10.1016/j.cognition.2003.08.002
Cognition 91 (2004) 155–172
www.elsevier.com/locate/COGNIT
* Corresponding author. Fax: þ61-3-9347-6618.
E-mail address: [email protected] (M.A. Williams).
configuration, and are continually changing. Despite this dynamic interplay of movement
and emotions, we are able to recognize hundreds of individuals under dramatically
different lighting conditions and orientations. This ability raises the question of how we
are able to differentiate individuals with such expertise. It has been claimed that the
specific process that provides us with this critical skill only occurs for faces, rather than
being a general process involved in the recognition of all objects (for a review, see
Kanwisher & Moscovitch, 2000).
All faces are comprised of the same fundamental configuration or arrangement of
features. Hence, some researchers claim that we process the relationship amongst these
features, and not merely the features themselves, to differentiate faces (for a review, see
Maurer, Le Grand, & Mondloch, 2002). In his classic paper, Yin (1969) demonstrated that
inversion resulted in a more pronounced deleterious effect on memory recognition for
faces than for other object categories including houses, airplanes, men in motion, or
faceless figures. This dramatic effect was ascribed to the disruption of configural
processing, which only affected faces. Since this time, the effect of face inversion on
processing has been studied extensively (e.g. Diamond & Carey, 1986; Farah, Wilson, &
Drain, 1998; Freire, Lee, & Symons, 2000; Haxby et al., 1999; Hillis, Hiscock, & Rexer,
1995; Kanwisher, Tong, & Nakayama, 1998; Leder & Bruce, 2000; Leder, Candrian,
Huber, & Bruce, 2001; Parr, Dove, & Hopkins, 1998; Rhodes, Brake, & Atkinson, 1993;
Tanaka & Farah, 1993). In a recent review, Maurer et al. (2002) discusses converging
results from many studies that demonstrate that face perception proceeds configurally and
that effects of inversion can be ascribed to the disruption of this process.
Many variants of configural processing have been proposed (Farah et al., 1998).
Specifically, three primary classes of processes have been posited. Conceivably, each class
may apply under different conditions. First-order relational processing involves the
determination of whether the structure matches a face-like configuration. In other words,
these processes determine the presence of facial features in a face-like configuration,
rather than an intricate analysis of the configuration of the face. Following these first-order
relational processes, which recognize the object as a face, additional processes that are
specific to facial analysis are invoked (Maurer et al., 2002).
First-order relational processing of faces has been demonstrated under a variety of
experimental conditions. For instance, experiments using schematic faces with only two
circles representing eyes and one line for the mouth have demonstrated patterns of results
that are specific to faces, such as fusiform face area (FFA) activation (Tong, Nakayama,
Moscovitch, Weinrib, & Kanwisher, 2000). Patients with spatial neglect seem to be less
likely to neglect a schematic face than a scrambled face (Vuilleumier, 2000). Priming
extinction patients with either two circles or two crosses within the context of a schematic
face reduces extinction of the two circles or crosses on subsequent presentations, despite
the absence of any schematic face surrounding them (Vuilleumier & Sagiv, 2001).
According to Moscovitch and Moscovitch (2000), when a face is inverted the object
processing system initially creates an upright representation of the face that is then
transferred to the FFA. To initiate transfer to the FFA, first-order configural processing is
suggested to be responsible for identifying the object as a face (Maurer et al., 2002).
Second-order relational processing is thought to be utilized when the identity of a face
needs to be ascertained. Second-order relational processing has been posited to compare
M.A. Williams et al. / Cognition 91 (2004) 155–172156
the specific parameters of the target face with a prototype. These parameters reflect the
spacing between facial features (Diamond & Carey, 1986). Thus, to identify faces, these
processes compare the spacing of the features of a target face with the configuration of a
generic template (Leopold, O’Toole, Vetter, & Blanz, 2001). Each individual will exhibit
consistent deviations from this generic template, which enables familiar faces to be
recognized. Research investigating second-order relational processing has involved
changing the spacing between features. Several studies have demonstrated that even
minute changes to the spacing between the features can be readily perceived when the
faces are upright; however, this ability is dramatically affected when the faces are inverted.
This detrimental effect of inversion is not observed, however, when the features
themselves are changed (Freire et al., 2000; Leder & Bruce, 1998, 2000; Leder et al., 2001;
Macho & Leder, 1998).
The Thatcher Illusion, first demonstrated by Thompson (1980), provides striking
evidence that second-order relational processing arises only when the face is upright. This
illusion arises when the eyes and mouth are rotated 1808 within a face. When the stimulus
is upright, this change results in a bizarre face. When the altered face is inverted, however,
the stimulus appears less unusual. Presumably, these second-order relational processes
explore the configuration of features. When the face is inverted, these processes are
thwarted and thus modifications to facial configurations are overlooked.
Proponents of holistic processing dismiss the notion of a generic template face from
which discrepancies are compared. Instead, they propose that we store a separate gestalt
template for each and every face. In other words, faces cannot be reduced to a finite set of
features or spaces between features; instead, each face is stored as a unique form or gestalt
(Farah et al., 1998). Tanaka and Farah (1993) found that individual features such as eyes,
mouth, and nose are recognized more readily when displayed as part of a face than when
displayed in isolation. This effect, however, did not extend to scrambled faces, inverted
faces, or images of houses. Accordingly, these findings are compatible with the idea that
the face is processed holistically rather than piecemeal. This evidence for holistic
processing, however, could be ascribed to alternative mechanisms. The removal of facial
features also limits the information that is utilized by first- and second-order relational
processing.
The evidence for second-order relational processing has also been contested. The ease
of identification of subtle changes to the spacing between features of upright faces,
compared to inverted faces, could be attributed to a mismatch between the altered face and
the stored template or gestalt of the particular individual’s face rather than the change in
second-order relational information. Likewise, the Thatcher Illusion could be due a
mismatch with the stored gestalt, rather than the change in the spatial relationships.
The competing explanations of various findings highlight that theories of configural
processing are still debatable. There is no argument that inversion disrupts the
configuration of the face and causes a delay in recognition. The debate surrounds the
types of configural processing that underpin face perception.
We explored the types of configural processing involved in face perception, using a
masking paradigm. Masking is a technique commonly used to disrupt the processing of a
visual stimulus, which otherwise may continue to be processed after it is has been
physically terminated (Keyser & Perrett, 2002). Using a mask that precedes (forward
M.A. Williams et al. / Cognition 91 (2004) 155–172 157
mask), and follows (backward mask), the presentation of a visual stimulus enables the
experimenter to control the duration of processing that is dedicated to the target.
Previous studies have demonstrated that the emotions of masked faces are implicitly
processed. Even when the face is masked to such an extent that participants are not aware
of its presentation, appropriate changes in physiological and brain activations have been
recorded (Morris, Ohman, & Dolan, 1998; Whalen et al., 1998). Masking therefore affords
us the opportunity to investigate the relationship between the features of the face prior to
awareness, as individual features of a masked face may be manipulated and the subsequent
effects on recognition can be observed. This paradigm has the potential to illuminate the
critical question of how they are processed.
2. Experiment 1
This experiment was concerned with the question of whether the specific features of the
eyes and mouth are initially processed by first-order relational, second-order relational, or
holistic perception. The aim of this experiment was first to investigate whether priming
could be achieved using individual facial features, and second to identify, through specific
condition contrasts, the early mechanisms involved in face perception.
Three types of prime–target pairs were used: (1) congruent (e.g. open eyes only in both
prime and target); (2) incongruent (e.g. open eyes only in prime and open mouth only in
target); and (3) dual (both mouth and eyes open in the prime only, followed by either type of
target, i.e. either eyes or mouth – but not both – open, see Fig. 1). Participants made a
speeded decision regarding the target face: whether eyes (response 1) or mouth (response 2)
Fig. 1. An example of the series of presentations for (A) congruent, (B) incongruent, and (C) dual trials
(not to scale).
M.A. Williams et al. / Cognition 91 (2004) 155–172158
or neither (catch trials – withhold responding) were open. Note that the identity of the
faces changed between prime and target.
The dual prime condition, in which both eyes and mouth are open, yields different
predictions based on holistic, first-order relational, and second-order relational processing
theories. According to holistic processing theories, in the dual condition, the overall form
or gestalt of the prime and target will differ. Hence, this condition should yield the same
response times as incongruent trials. Conversely, according to first-order relational
processing theories, the features are processed independently, and thus the dual condition
should yield analogous response times to the congruent trials. Second-order relational
processing theories predict that the relationship between the features rather than the
features themselves are processed. Presumably, then, primes in the dual condition
comprise a more similar configuration to the target than incongruent primes, but less
similar configuration to the target than congruent primes. In other words, response times in
this dual condition should be intermediate between the congruent and incongruent trials.
2.1. Method
2.1.1. Participants
Twelve right handed University students (six male and six female, mean age 24.6 years,
SD ¼ 3:32) participated and were paid for their time.
2.1.2. Apparatus and stimuli
Coloured photographs were generated using a digital camera and edited using Adobew
Photoshop. The background was black and the mean luminance was approximately the
same for all pictures. These pictures were converted to 24-bit bitmaps for display. The
forward mask and target (visual angle 6.48) were 25% larger than the prime (visual angle
5.18). The test computer was an IBM compatible PC with a 750 Hz Intel Pentium III
processor, 128 MB RAM, a Trident CyberBlade video card with 16 MB video memory,
and a MAG Innovision DJ530 15-inch CRT monitor. The video card was set at a refresh
rate of 60 Hz and screen resolution of 800 £ 600 £ 16. The program was written in Visual
Basic (Version 6) using Direct-X 8 technology. Priority settings were optimized to ensure
accurate display durations.
The forward mask was the experimenter’s face with both eyes and mouth closed.
Primes were also the experimenter’s face with either eyes, mouth, or both open. Note that
primes were 25% smaller than the forward mask to avoid any apparent movement. The use
of the experimenter’s face ensured that all participants were familiar with the face and the
facial features remained constant. The target face was a male of approximately the same
age as the experimenter and was the same size as the forward mask, acting as a complete
backward mask of the prime (see Fig. 1). It has been demonstrated that face masks are
more appropriate than other stimuli when masking a face (Costen, Shepherd, Ellis, &
Craw, 1994). The target face had open mouth or open eyes, except on 50% of trials in
which both eyes and mouth were closed (catch trials). It should be noted that the same two
faces were used throughout the experiments and therefore caution should be exercised
when the generalizability of these results is contemplated.
M.A. Williams et al. / Cognition 91 (2004) 155–172 159
2.1.3. Design
A three-factor within-subjects design was used, in which factors were Prime duration
(33 or 50 ms), Target type (open mouth, open eyes) and Congruency (congruent,
incongruent, dual). Prime duration was blocked whereas Target type and Congruency
were randomized within each block. All factors were fully crossed, yielding 12
experimental conditions.
2.1.4. Procedure
Fig. 1 shows the sequence of stimuli in a single trial. Participants were asked to
maintain fixation on the centre of the screen throughout the experiment. Each trial
commenced with the forward mask appearing in the centre of the screen for 1500 ms. This
mask was then replaced with the prime in the same location. The prime remained on the
screen for either 33 or 50 ms and was then replaced with the target. Participants pressed
one button to indicate whether the target face’s mouth was open, and another if the eyes
were open. They were instructed to respond as quickly as possible. Both eyes and mouth
were never simultaneously open in the target face. The position (left/right) of the buttons
was counterbalanced between subjects. In 50% of trials, neither the eyes nor mouth were
open on the target face (catch trials), and no response was required. Participants completed
four blocks of 240 trials, two blocks at each prime duration resulting in a total of 960 trials
(40 trials per condition). Following each experimental block, participants were asked to
describe what they perceived between the fixation and target displays. Participants were
then asked if they could identify the face that appeared. The criterion for exclusion from
this study was explicit recognition of the prime face as the experimenter.
2.2. Results
No participant was able to identify the prime. Outliers were defined as reaction times
(RT) greater than 3 SD from each individual’s mean or less than 150 ms, and were
excluded from analysis (less than 2%). Mean correct RT were calculated for each of the 12
conditions. A three-way within-subjects analysis of variance (ANOVA) conducted on the
RT data yielded a significant main effect of Target (Fð1; 11Þ ¼ 5:06, P , 0:05).
Participants were faster to respond to open eyes (M ¼ 421:8 ms, SE ¼ 2:9) than an
open mouth (M ¼ 428:4 ms, SE ¼ 3:1). A significant main effect of Congruency
(Fð1; 11Þ ¼ 51:62, P , 0:001) and a significant Duration £ Congruency interaction
(Fð2; 22Þ ¼ 9:22, P , 0:001) were also evident. No other effects or interactions reached
significance (P . 0:1).
Simple main effects analysis (Bonferroni adjusted) of the Duration £ Congruency
interaction uncovered a significant difference between each of the congruency conditions
at both durations. Fig. 2 illustrates that, at a prime duration of 33 ms, RT on congruent
trials were significantly shorter than on incongruent trials and dual trials, and RT on
incongruent trials were significantly longer than on dual trials. At a prime duration of
50 ms, an analogous pattern emerged. The only significant difference evident between
durations was slower mean RT in the incongruent condition, when prime duration was
50 ms rather than 33 ms (P , 0:05). It can be seen from the increase in the percentage
M.A. Williams et al. / Cognition 91 (2004) 155–172160
of errors in concordance with the RT data that there was no speed/accuracy trade-off
(see Fig. 2).
2.3. Discussion
The results of Experiment 1 show that prime faces influence RT to target faces at prime
durations of 33 and 50 ms. Congruent prime–target combinations resulted in faster RT
than incongruent prime–target combinations. More importantly, the dual condition, in
which both the eyes and mouth were open in the prime, yielded RT that were faster than
incongruent trials yet slower than congruent trials. The only difference in results between
the prime durations was that responses to incongruent trials were slower at the longer
prime duration. Target type also had an effect, with participants reacting faster to open
eyes (M ¼ 422 ms) in the target face than to an open mouth (M ¼ 428 ms).
The congruency effects observed at both prime durations provide evidence for implicit
processing of the eyes and mouth. At both prime durations, participants were unable to identify
the prime. Nevertheless, these brief primes influenced subsequent responses to the target face.
The dual condition, in which the eyes and mouth were both open in the prime, resulted
in intermediate RT in comparison to the other congruency conditions for both target types.
This result suggests that the eyes and mouth are processed together rather than as
individual parts. That is, if the eyes and mouth were processed independently, open eyes in
the prime would not influence responses to open mouth in the target, and vice versa. If
holistic processing occurred, responses should be akin to incongruent trial responses. The
results, therefore, demonstrate that second-order relational information of the face is
processed, rather than the holistic information.
The only difference observed between the two prime durations was that participants
exhibited slower RT in the incongruent condition when the prime duration was 50 ms as
compared with 33 ms. It is likely that a stimulus that is presented for a longer duration
Fig. 2. Mean reaction times in milliseconds for each of the congruency conditions at prime durations of either 33
or 50 ms collapsed across Target type. The mean percentage of errors in each condition is displayed in
parentheses and the error bars reflect one standard error.
M.A. Williams et al. / Cognition 91 (2004) 155–172 161
produces more extensive or protracted activation as the processing is more in depth. In an
incongruent condition, then, increased inhibition may be required to overcome the prime
activation, resulting in protracted RT. The absence of any corresponding decrease in RT to
congruent trials is likely to reflect a ceiling effect: participants were receiving maximal
assistance from the prime congruency even at the 33 ms duration, and therefore no further
improvement could be achieved.
There are fundamental differences between the eyes and the mouth that account for the
faster responses to open eye compared with open mouth targets. Specifically, there is a
luminance discrepancy between the white sclera of the eyes and the rest of the face that
may act as an exogenous cue that is not present in an open mouth.
In summary, the results suggest that the eyes and mouth are processed together prior to
awareness. This finding supports the second-order relational theory of face recognition,
that the relationship between the parts is processed, which is then compared to a generic
template. Inversion of the face, however, has been claimed to affect configural processing.
The next experiment explored this idea by inverting the stimuli.
3. Experiment 2
In this experiment, all stimuli from Experiment 1 were rotated 1808 to create inverted
faces. The adverse effect of inversion on face recognition and memory has been well
documented (e.g. Bruce & Langton, 1994; Farah et al., 1998; Tanaka & Farah, 1993). As
discussed previously, this finding has been used as evidence for configural encoding
theories of face perception. When a face is inverted, the spatial relationships change and
hence, this affects the processing of second-order relational information.
We examined the effect of inversion on the implicit processing of the eyes and mouth, as
demonstrated in Experiment 1. If an inverted face is processed by first-order relational
information, that is, the parts, then presenting both eyes and mouth open in the prime (the dual
condition) should result in response times analogous to those in the congruent trials,
regardless of the target type. For example, an open mouth in the prime should not
compromise processing of open eyes in the target if these parts are processed independently.
If, however, inverted faces are processed by second-order relational information, reflecting
inter-feature spacing, we should observe analogous results to those of Experiment 1, with RT
in the dual condition intermediate between the congruent and incongruent conditions.
3.1. Method
3.1.1. Participants
Twelve right handed University students (five male and seven female, mean age 23.6
years, SD ¼ 3:94) participated in the experiment and were paid for their time.
3.1.2. Apparatus and stimuli
All stimuli used in Experiment 1 were rotated by 1808 to produce inverted masks, primes
and targets. All other apparatus and materials were identical to those in Experiment 1.
M.A. Williams et al. / Cognition 91 (2004) 155–172162
3.1.3. Design and procedure
The design and procedure were the same as Experiment 1.
3.2. Results
Participants were again unable to identify the prime. Outliers were defined as RT
greater than 3 SD from each individual mean or less than 150 ms, and were removed prior
to analysis (less than 2%). As in Experiment 1, mean RT were calculated for each of the 12
conditions. A three-way within-subjects ANOVA conducted on the RT data yielded a
significant main effect of Target type (Fð1; 11Þ ¼ 18:28, P , 0:001). Responses were
again faster to open eyes (M ¼ 441:9 ms, SE ¼ 2:6) than to open mouth (M ¼ 453:1 ms,
SE ¼ 2:8). A significant main effect of Congruency (Fð1; 11Þ ¼ 64:37, P , 0:001) and a
significant Duration £ Congruency interaction (Fð2; 22Þ ¼ 5:76, P , 0:01) were also
observed. No other effects or interactions reached significance (P . 0:1).
Simple main effects analysis (Bonferroni adjusted) of the Duration £ Congruency
interaction showed a significant difference between congruent and incongruent conditions
at both durations, as can be seen in Fig. 3. At a prime duration of 33 ms, RT on congruent
trials were significantly shorter than on incongruent trials, and the difference between the
dual condition and the incongruent condition was also significant. There was no difference
observed between the dual condition and the congruent condition. At a prime duration of
50 ms, a similar pattern emerged, with the exception that a significant difference between
the congruent and dual conditions was also observed (P , 0:05). No other simple main
effects reached significance (P . 0:1). It can be seen from the increase in the percentage of
errors in concordance with the RT data that there is no speed/accuracy trade-off (see Fig. 3).
Fig. 3. Mean reaction times in milliseconds for each of the congruency conditions at prime durations of either 33
or 50 ms. The percentage of errors in each condition is displayed in parentheses and the error bars reflect one
standard error.
M.A. Williams et al. / Cognition 91 (2004) 155–172 163
3.3. Discussion
The results from Experiment 2 show that inverted face primes influence RT to inverted
target faces, at both prime durations. Congruent prime–target trials resulted in faster RT
than incongruent prime–target combinations. Unlike Experiment 1, however, there was
no significant difference in RT between the congruent and dual conditions at the shortest
prime duration of 33 ms, suggesting a difference in the perceptual mechanisms involved.
At the longer prime duration (50 ms), the dual condition mimicked the result of
Experiment 1, with the dual condition mean RT intermediate between incongruent and
congruent response times. Analogous to Experiment 1, responses to open eyes (M ¼ 442
ms) were faster than to open mouth (M ¼ 453 ms).
As in Experiment 1, the observed congruency effects provide evidence for implicit
processing of the eyes and mouth. At a prime duration of 33 ms, there was no significant
difference between the dual and the congruent conditions, indicating that the eyes and
mouth may be processed independently. At the prime duration of 50 ms, however, the
mean RT in the dual condition was intermediate between the means for congruent and
incongruent conditions. At the longer prime duration, therefore, the eyes and mouth may
be linked in a way that does not seem to occur at earlier stages of processing. This pattern
suggests that, initially, first-order relational information is processed, with second-order
relational information encoding occurring slightly later.
These results have interesting implications for theories that concern the effect of face
inversion on face recognition. Only at the short prime duration was there evidence for
inverted faces being processed by first-order relational information. At the longer prime
duration, the results show a different pattern indicating an interaction in processing of the
parts. These findings suggest that inversion may not completely disrupt second-order
relational information processing, but rather protract an initial parts-based phase. It could
be argued, however, that the absence of a difference between congruent and dual inverted
conditions may reflect inadequate power. In Experiment 3, therefore, we sought to
replicate our findings and strengthen our conclusions by directly examining the interaction
between the upright and inverted faces at the shortest prime duration.
4. Experiment 3
In this experiment, we examined whether the patterns of results observed at the shortest
prime duration in the first two experiments were robust and replicable or simply a
consequence of limited statistical power. The same protocol was utilized as in the previous
experiments, with the exception of limiting the prime duration to 33 ms, and using both
upright and inverted stimuli. If our interpretation of the results of Experiments 1 and 2 is
correct, then upright faces are processed by configural information, whereas inverted faces
are initially processed by parts. This outcome would be demonstrated by a difference in
results for inverted versus upright stimuli at this short prime duration. An interaction
between stimuli orientation types, therefore, would support and replicate our findings from
the previous two experiments.
M.A. Williams et al. / Cognition 91 (2004) 155–172164
4.1. Method
4.1.1. Participants
Twelve right handed University students (six male and six female, mean age 26.5 years,
SD ¼ 2:0) participated in the experiment and were paid for their time.
4.1.2. Apparatus and stimuli
All stimuli used in Experiments 1 and 2 were utilized. All other apparatus and materials
were identical to those in Experiment 1.
4.1.3. Design
A two-factor within-subjects design was used, in which factors were Orientation
(upright or inverted) and Congruency (congruent, incongruent, dual). Orientation was
blocked whereas Congruency was randomized within each block. All factors were fully
crossed, yielding six experimental conditions.
4.1.4. Procedure
The procedure was the same as Experiment 1.
4.2. Results
Participants were again unable to identify the prime. Outliers were defined as RT
greater than 3 SD from each individual mean or less than 150 ms, and were removed prior
to analysis (less than 2%). Mean RT were calculated for each of the six conditions. A two-
way within-subjects ANOVA conducted on the RT data yielded a significant main effect
of Congruency (Fð1; 11Þ ¼ 42:18, P , 0:001) and a significant Orientation £ Congruency
interaction (Fð2; 22Þ ¼ 4:39, P , 0:05) was also observed. No other effects or interactions
reached significance (P . 0:1).
Simple main effects analysis (Bonferroni adjusted) of the Orientation £ Congruency
interaction showed that the impact of congruency differed for upright and inverted faces,
as can be seen in Fig. 4. The upright manipulation yielded congruent RT that were
significantly shorter than those on the dual trials which, in turn, were significantly shorter
than the incongruent RT (P , 0:05). In contrast, the inversion manipulation resulted in
congruent RT that were not significantly different from dual RT (P ¼ 0:13), whilst the
difference between the dual condition and the incongruent condition was significant.
The only significant main effect evident between orientations was the slower mean RT in
the dual condition, when stimuli were upright rather than inverted (P , 0:05). No other
simple main effects reached significance (P . 0:1). It can be seen from the increase in the
percentage of errors in concordance with the RT data that there is no speed/accuracy trade-
off (see Fig. 4).
4.3. Discussion
The results from Experiment 3 demonstrate that both upright and inverted face primes
influence RT to target faces, consistent with Experiments 1 and 2. Congruent prime–target
M.A. Williams et al. / Cognition 91 (2004) 155–172 165
trials resulted in faster RT than incongruent prime–target combinations for both upright
and inverted faces. Critically, an interaction between orientation and congruency was
observed, due to a difference between congruent and dual conditions with upright stimuli
which was not present when the stimuli were inverted. This finding demonstrates that the
differential effects observed in Experiments 1 and 2 are reliable.
As in Experiments 1 and 2, the observed congruency effects provide evidence for
implicit processing of the eyes and mouth. Again, inversion of the stimuli removed the
difference between the dual and congruent conditions that was present for upright stimuli.
This finding indicates that inversion causes the eyes and mouth to be processed
independently.
There is, however, another possible explanation for this particular pattern of results. It
is conceivable that there are two separate processing systems for the eyes and mouth,
perhaps based on simple visual cues such as luminance or contrast changes: if so, the
results observed in Experiments 1 and 3 for upright faces could easily be explained. While
congruent and incongruent trials would cause facilitation and interference effects,
respectively, dual trials would result in interference via one system and facilitation via the
other. The net effect would be intermediate RT on dual trials, as observed.
On inversion the effects of such systems should still occur, assuming these processes
are equally active. It has been suggested, however, as discussed earlier, that inversion
disrupts face processing (Yin, 1969). In this case, it could be argued that inversion also
prevents these two systems from being triggered and as such the inversion effects
observed in the previous experiments could be explained as a disruption of these two
separate systems as opposed to second-order configural processing. Alternatively,
inversion may retard response times because of limited experience with inverted faces.
As such, decision-associated factors may underlie the pattern of results observed. To
address these issues, we constructed a novel priming paradigm using ‘misaligned’ faces
(Young, Hellawell, & Hay, 1987).
Fig. 4. Mean reaction times in milliseconds for each of the congruency conditions for upright and inverted faces at
a prime duration of 33 ms. The percentage of errors in each condition is displayed in parentheses and the error
bars reflect one standard error.
M.A. Williams et al. / Cognition 91 (2004) 155–172166
5. Experiment 4
Young et al. (1987) reported a striking demonstration of second-order configural
processing that arose when the top half of one face was aligned with the bottom half of
another to create a composite face. They found that participants were slower to recognize
either the top or bottom half of these composite faces relative to faces in which the two
halves were misaligned or the entire stimulus was inverted. They argued that aligned
composite faces are fused automatically and perceived as a new whole face rather than two
different halves (Young et al., 1987). Importantly for our purposes, misaligned faces were
not processed via a second-order configural processing system, suggesting that misaligned
faces are appropriate controls to test the possibility that eyes and mouth processing reflect
two parallel systems.
In Experiment 4, we misaligned the prime faces to interrupt second-order configural
information (see Fig. 5). If separate processing systems for the eyes and mouth underlie the
intermediate RT observed in Experiments 1 and 3 for upright faces, then the same effect
should be evident when the upper and lower parts of the prime are misaligned. If, on the
other hand, the effect evident for upright faces is due to second-order configural
processing, misalignment should generate a pattern of findings that is analogous to the
results that were observed for inverted faces.
5.1. Method
5.1.1. Participants
Twelve right handed University students (seven male and five female, mean age 27
years, SD ¼ 2:61) participated in the experiment and were paid for their time.
5.1.2. Apparatus and stimuli
The top and bottom halves of the primes from Experiment 1 were misaligned by
moving them horizontally to ensure an overlap of approximately 66% of the face (Fig. 5).
Fig. 5. An example of a misaligned prime face.
M.A. Williams et al. / Cognition 91 (2004) 155–172 167
Both combinations of left and right adjustments were utilized. Note that as the prime
face was 25% smaller than the target face, misaligned primes were still within the
boundaries of targets. All other apparatus and materials were identical to those in
Experiment 1.
5.1.3. Design
A within-subjects design was used, with the factor of Congruency (congruent,
incongruent, dual).
5.1.4. Procedure
The procedure was the same as Experiment 1.
5.2. Results
Participants were again unable to identify the prime. Outliers were defined as RT
greater than 3 SD from each individual mean or less than 150 ms, and were removed prior
to analysis (less than 2%). Mean RT were calculated. A one-way within-subjects ANOVA
conducted on the RT data yielded a significant main effect of Congruency
(Fð2; 22Þ ¼ 27:49, P , 0:001).
Simple main effects analysis (Bonferroni adjusted) of the Congruency main effect
showed that there was a significant difference between congruent and incongruent, and
between the dual and incongruent conditions (P , 0:05); however, congruent trials were
not significantly different than dual trials (P ¼ 0:17), as can be seen in Fig. 6. It can be
seen from the increase in the percentage of errors in concordance with the RT data that
there is no speed/accuracy trade-off (see Fig. 6).
Fig. 6. Mean reaction times in milliseconds for each of the congruency conditions at a prime duration of 33 ms.
The percentage of errors in each condition is displayed in parentheses and the error bars reflect one standard error.
M.A. Williams et al. / Cognition 91 (2004) 155–172168
5.3. Discussion
The results of Experiment 4, as in the previous experiments, demonstrate that prime
faces influence RT to target faces. Congruent prime–target combinations resulted in faster
RT than incongruent prime–target combinations. Critically, in the dual condition, in
which both the eyes and mouth were open in the prime, RTs were not different from
congruent trials and were faster than incongruent trials.
Misalignment of a face interrupts the second-order configural processing (Young et al.,
1987). In the current experiment, we used this property of misalignment to ascertain that
the results obtained in the previous experiments may not be attributed to independent
processing of the eyes and mouth nor decision-associated factors. When second-order
configural processes are disrupted, the pattern of results observed for upright faces mimics
the pattern of results observed for inverted faces, as revealed in Experiments 2 and 3. The
results for upright faces in the previous experiments can, therefore, be ascribed to second-
order configural processing.
6. General discussion
The main focus of this study was to explore configural theories of face processing by
investigating implicit processing of two important parts of the face: the eyes and mouth.
Experiment 1 revealed a significant congruency effect at prime durations of 33 and 50 ms,
with participants fastest to react to targets preceded by a congruent prime and slowest in
incongruent conditions. Critically, the dual condition, in which the prime contained both
open eyes and open mouth, produced intermediate response times between the congruent
and incongruent conditions. In Experiment 2, the effect of inversion on this implicit
processing was examined. The dual condition at the shortest prime duration produced RT
analogous to the congruent condition responses. At the longer prime duration, the results
replicated the pattern observed in Experiment 1. Experiment 3 investigated whether these
differences between Experiments 1 and 2 could be replicated and statistically validated.
An interaction between upright and inverted faces was demonstrated and the overall
pattern of results showed that the original findings were robust. Experiment 4 utilized
‘misaligned faces’ as primes to falsify an alternative explanation for our results.
In each experiment, participants executed a speeded judgement about the eyes and
mouth. As participants were never required to identify the faces, it could be argued that
subjects were able to focus on these parts of the face, which precludes the activation of
holistic processes. However, the pattern of results in each experiment and the changes
observed when the configuration of the primes was disrupted, via inversion or
misalignment, challenge this argument. Furthermore, there is evidence that faces are
processed automatically (Boutet, Gentes-Hawn, & Chaudhuri, 2002; Vuilleumier, 2000;
Vuilleumier & Sagiv, 2001; Winston, Strange, O’Doherty, & Dolan, 2002) even when
detrimental to the task (Young et al., 1987) and, therefore, simply attending to the face
should activate face processes.
We have applied a strict definition of holistic processing in this study. Of course,
holistic processing could be defined as merely focusing on the similarities between the real
M.A. Williams et al. / Cognition 91 (2004) 155–172 169
and stored representations of the object (Schwarzer, Kuefer, & Wilkening, 1999).
According to this definition, second-order configural and holistic processing cannot be
disentangled. However, a model must be operationalized to be tested and to that end we
have used a strict interpretation of both processing types to enable a systematic
investigation.
Overall, the results support the concept of implicit processing of the eyes and mouth
prior to awareness. Upright faces were found to be processed by second-order relational
information. Inverting or misaligning the faces, however, resulted in first-order relational
processing. Intriguingly, inverted faces were found to later undergo second-order
relational processing, supporting the notion that they may be transferred to the FFA after
initial processing and identification as faces. This proposition is consistent with
neuroimaging studies that have found the FFA, which is specific to face processing, to
be more active in response to inverted faces than other objects (Haxby et al., 1999;
Kanwisher et al., 1998; Sagiv & Bentin, 2001; Tong et al., 2000).
Moscovitch and Moscovitch (2000) have suggested that, when a face is inverted, the
object processing system initially creates an upright representation of the face that it then
transfers to the FFA. Our current results support this theory; with the face inverted, the
eyes and mouth were processed independently (first-order relational information) at 33 ms,
yet at 50 ms they appeared to be processed by second-order relational information
consistent with upright face processing. In other words, when a face is inverted, the
configuration is changed resulting in initial processing by first-order relational information
for identification as a face and transformation to an upright representation. Following this
phase, second-order relational processing can then proceed. Of course, whether or not
these two stages are completely distinct or part of a cascade of neural processes cannot yet
be established.
There has been a large number of studies demonstrating the detrimental effect of
inversion on face recognition (e.g. Diamond & Carey, 1986; Farah et al., 1998; Freire et al.,
2000; Haxby et al., 1999; Hillis et al., 1995; Kanwisher et al., 1998; Leder & Bruce, 2000;
Leder et al., 2001; Parr et al., 1998; Rhodes et al., 1993; Tanaka & Farah, 1993); however,
the current results suggest that 50 ms presentation of an inverted face is sufficient to allow
conversion to an upright representation and subsequent second-order relational
processing. This study did not examine whether this transformation to an upright
representation is precise and, as such, it cannot be assumed that a person would be accurate
at recognizing or matching the faces, instead of merely following the more limited
requirements of the current paradigm. Based on the previous literature on face inversion, it
appears that the transformation may in fact be crude. As such, although configural
processing occurs, the transformed representation may be degraded resulting in decreased
accuracy in recognition tasks.
As discussed, backward masking is a technique used to control the available time of
initial processing of a visual stimulus and is used to investigate how that stimulus is
processed during the first stages of perception. Interestingly, even at the shortest prime
duration, second-order relational processing was observed for upright faces, suggesting
that face perception occurs pre-attentively. These results support several previous studies
demonstrating automatic pre-attentive processing of faces and facial expressions (Boutet
et al., 2002; De Gelder, Pourtois, Van Raamsdonk, Vroomen, & Weiskrantz, 2001;
M.A. Williams et al. / Cognition 91 (2004) 155–172170
De Gelder, Vroomen, Pourtois, & Weiskrantz, 1999; Morris, De Gelder, Weiskrantz, &
Dolan, 2001; Morris et al., 1998; Vuilleumier, 2000; Vuilleumier & Sagiv, 2001; Whalen
et al., 1998; Winston et al., 2002).
In summary, we have conclusively shown in a series of four experiments that within the
context of task requirements second-order configural processing, rather than holistic
processing, underlies face perception. In addition, we have demonstrated that inverted
faces are initially processed by first-order (parts-based) assessment before second-order
relational processing is initiated. These experiments show the value of systematic
investigation of implicit face perception using masked priming.
Acknowledgements
We thank Chris Chambers, Belinda Howard and Anina Rich for their suggestions on an
earlier draft of the manuscript.
References
Boutet, I., Gentes-Hawn, A., & Chaudhuri, A. (2002). The influence of attention on holistic face encoding.
Cognition, 84, 321–341.
Bruce, V., & Langton, S. R. H. (1994). The use of pigmentation and shading information in recognizing the sex
and identities of faces. Perception, 23, 803–822.
Costen, N. P., Shepherd, J. W., Ellis, H. D., & Craw, I. (1994). Masking of faces by facial and non-facial stimuli.
In G. W. Humphreys (Ed.), Object and face recognition (1) (pp. 227–251). Special issue of visual cognition,
Hillsdale, NJ: Lawrence Erlbaum Associates.
De Gelder, B., Pourtois, G., Van Raamsdonk, M., Vroomen, J., & Weiskrantz, L. (2001). Unseen stimuli
modulate conscious visual experience: evidence from inter-hemispheric summation. Cognitive Neuroscience
and Neuropsychology, 12(2), 385–391.
De Gelder, B., Vroomen, J., Pourtois, G., & Weiskrantz, L. (1999). Non-conscious recognition of affect in the
absence of striate cortex. NeuroReport, 10, 3759–3763.
Diamond, R., & Carey, S. (1986). Why faces are and are not special: an effect of expertise. Journal of
Experimental Psychology: General, 115(2), 107–117.
Farah, M. J., Wilson, K. D., & Drain, M. (1998). What is “special” about face perception? Psychological Review,
105(3), 482–498.
Freire, A., Lee, K., & Symons, L. A. (2000). The face-inversion effect as a deficit in the encoding of configural
information: direct evidence. Perception, 29, 159–170.
Haxby, J. V., Ungerleider, L. G., Clark, V. P., Schouten, J. L., Hoffman, E. A., & Martin, A. (1999). The
effect of face inversion on activity in human neural systems for face and object perception. Neuron, 22,
189–199.
Hillis, S. K., Hiscock, M., & Rexer, J. L. (1995). Dual-task interference patterns reveal differential processing of
upright and inverted faces. Brain and Cognition, 28, 155–172.
Kanwisher, N., & Moscovitch, M. (2000). The cognitive neuroscience of face processing: an introduction.
Cognitive Neuropsychology, 17(1/2/3), 1–11.
Kanwisher, N., Tong, F., & Nakayama, K. (1998). The effect of face inversion on the human fusiform face area.
Cognition, 68, B1–B11.
Keyser, C., & Perrett, D. (2002). Visual masking and RSVP reveal neural competition. Trends in Cognitive
Sciences, 6, 120–125.
Leder, H., & Bruce, V. (1998). Local and relational aspects of face distinctiveness. Quarterly Journal of
Experimental Psychology: A, Human Experimental Psychology, 51A(3), 449–473.
M.A. Williams et al. / Cognition 91 (2004) 155–172 171
Leder, H., & Bruce, V. (2000). When inverted faces are recognised: the role of configural information in face
recognition. Quarterly Journal of Experimental Psychology: A, Human Experimental Psychology, 53,
513–536.
Leder, H., Candrian, G., Huber, O., & Bruce, V. (2001). Configural features in the context of upright and inverted
faces. Perception, 30, 73–83.
Leopold, D. A., O’Toole, A. J., Vetter, T., & Blanz, V. (2001). Prototype-referenced shape encoding revealed by
high-level aftereffects. Nature Neuroscience, 4, 89–94.
Macho, S., & Leder, H. (1998). Your eyes only? A test of interactive influence in the processing of facial features.
Journal of Experimental Psychology: Human Perception and Performance, 24(5), 1486–1500.
Maurer, D., Le Grand, R., & Mondloch, C. J. (2002). The many faces of configural processing. Trends in
Cognitive Sciences, 6(6), 255–260.
Morris, J. S., De Gelder, B., Weiskrantz, L., & Dolan, R. J. (2001). Differential extrageniculostriate and amygdala
responses to presentation of emotional faces in a cortically blind field. Brain, 124, 1241–1252.
Morris, J. S., Ohman, A., & Dolan, R. J. (1998). Conscious and unconscious emotional learning in the human
amygdala. Nature, 393, 467–470.
Moscovitch, M., & Moscovitch, D. A. (2000). Super face-inversion effects for isolated internal features and
fractured faces. Cognitive Neuropsychology, 17, 201–219.
Parr, L. A., Dove, T., & Hopkins, W. D. (1998). Why faces may be special: evidence of the inversion effect in
chimpanzees. Journal of Cognitive Neuroscience, 10(5), 615–622.
Rhodes, G., Brake, S., & Atkinson, A. P. (1993). What’s lost in inverted faces? Cognition, 47, 25–57.
Sagiv, N., & Bentin, S. (2001). Structural encoding of human and schematic faces: holistic and part-based
processes. Journal of Cognitive Neuroscience, 13(7), 937–951.
Schwarzer, G., Kuefer, I., & Wilkening, F. (1999). Learning categories by touch: on the development of holistic
and analytic processing. Memory and Cognition, 27(5), 868–877.
Tanaka, J. W., & Farah, M. J. (1993). Parts and wholes in face recognition. Quarterly Journal of Experimental
Psychology, 46A, 225–245.
Thompson, P. (1980). Margaret Thatcher – a new illusion. Perception, 9, 483–484.
Tong, F., Nakayama, K., Moscovitch, M., Weinrib, O., & Kanwisher, N. (2000). Response properties of the
human fusiform face area. Cognitive Neuropsychology, 17(1/2/3), 257–279.
Vuilleumier, P. (2000). Faces call for attention: evidence from patients with visual extinction. Neuropsychologia,
38, 693–700.
Vuilleumier, P., & Sagiv, N. (2001). Two eyes make a pair: facial organization and perceptual learning reduce
visual extinction. Neuropsychologia, 39, 1144–1149.
Whalen, P. J., Rauch, S. L., Etcoff, N. L., McInerney, S. C., Lee, M. B., & Jenike, M. A. (1998). Masked
presentations of emotional facial expressions modulate amygdala activity without explicit knowledge.
Journal of Neuroscience, 18(1), 411–418.
Winston, J. S., Strange, B. A., O’Doherty, J., & Dolan, R. J. (2002). Automatic and intentional brain responses
during evaluation of trustworthiness of faces. Nature Neuroscience, 5(3), 277–283.
Yin, R. K. (1969). Looking at upside-down faces. Journal of Experimental Psychology, 81, 141–145.
Young, A. W., Hellawell, D., & Hay, D. C. (1987). Configural information in face perception. Perception, 16,
747–759.
M.A. Williams et al. / Cognition 91 (2004) 155–172172