The effects of valenced odors onfacial perception Elmeri Syrjänen

Elmeri Syrjänen    T

he effects of valen

ced odors on facial perception

Doctoral Thesis in Psychology at Stockholm University, Sweden 2020

Department of Psychology

ISBN 978-91-7911-040-6

Elmeri Syrjänen Gösta Ekmans Laboratory

In this thesis, I present my research on how valenced (unpleasant andpleasant) odors affect the perception of faces with emotionalexpressions. I have focused on happiness and disgust because they areassumed to correspond to the emotional responses to pleasant andunpleasant odors. Experimentally combining odor and face stimuliallows the investigation of the effects of emotional odor-facecongruency (e.g., perceiving an unpleasant odor and a disgusted face). Ihave also investigated the overall effects of odors on facial perception,irrespective of the facial expression. My overall research question is:Do pleasant and unpleasant odors affect how we process emotionallyexpressive faces. This thesis centers on emotion recognitionperformance, perceptual evaluation of faces, cortical brain responses,and attentional processes in valenced odor contexts.

The effects of valenced odors on facial perceptionElmeri Syrjänen

Academic dissertation for the Degree of Doctor of Philosophy in Psychology at StockholmUniversity to be publicly defended on Friday 24 April 2020 at 13.00 in David Magnussonsalen(U31), Frescati Hagväg 8.

AbstractWe use our senses to navigate in the world. An important property of olfaction, the sense of smell, is to enable us to approachbeneficial things, and to avoid what might be toxic or otherwise harmful in our environment. Other peoples’ behaviors arealso paramount for our survival. Thus, we use our vision to decode their internal states from their facial expressions. Formany modalities, multiple senses are integrated to enhance sensory percepts. In this thesis, I investigated how valencedodors affect the perception of facial expressions. Specifically, using a multi-method approach, I studied the integration ofunpleasant and pleasant odor contexts on odor-congruent and incongruent facial expressions, disgusted, and happy faces.

The effects I am interested in are those that valenced odors have on face perception, attention to faces, and the corticalprocessing of faces. To answer these questions, I used questionnaires, ratings, EEG, and behavioral measures such asreaction times. Across studies, ratings of face valence are affected in the direction of the odor valence (e.g., faces arerated more negatively in the context of an unpleasant odor). Also, overall, the results in my studies indicate that faces areperceived as more arousing in valenced odor contexts; however, these effects occur regardless of facial expression.

In study 1, I found that valenced odors and facial expressions are integrated at an earlier time-frame than previouslythought. Specifically, I found that the N170 event-related potential component (ERP) to disgusted facial expressions waslower in amplitude in the unpleasant odor condition than in the pleasant odor condition. This effect was not present forhappy faces in the N170 component. An unpleasant odor might thus facilitate the processing of threat-related information.

In study 2, I found evidence that odors, in general, did not affect the recognition speed of facial expressions that changedfrom neutral to disgusted or happy over 3 seconds. Also, I found robust evidence against congruency effects in facialexpression recognition reaction times (RTs). The results indicated that faces overall were recognized faster in the unpleasantodor condition. Further, these results were not qualified by individual differences in body odor disgust. Thus, unpleasantodors might facilitate the recognition of facial expressions regardless of trait body odor disgust.

In study 3, I studied whether valenced odors directed spatial attention toward odor-congruent facial expressions in a “dot-probe” task. I found decisive evidence that odors do not affect attention towards disgusted and happy facial expressions,casting doubt on the dot-probe experiment. However, I found that probes were detected faster as a function of time-on-task in the unpleasant odor condition. I hypothesized that this effect might be due to maintained vigilance in the presenceof an unpleasant odor and task fluency effects.

In summary, the results indicate that valenced odors affect facial perception. Generally, faces are perceived as morevalenced and arousing in odor contexts. Further, an unpleasant odor may decrease RTs; however, this effect seems to beirrespective of the target type. Also, odor face integration may happen earlier than thought; yet, evidence in the literatureis mixed, and more research is needed. The methods I have used may increase transparency and robustness of publishedresults, and help accelerate knowledge development in this field of research.

Keywords: Valenced odors, Facial expressions, Disgust, Happiness, Olfaction, Emotion, Reaction Times, Evaluation,Ratings, EEG, ERP, N170.

Stockholm 2020http://urn.kb.se/resolve?urn=urn:nbn:se:su:diva-179663

ISBN 978-91-7911-040-6ISBN 978-91-7911-041-3

Department of Psychology

Stockholm University, 106 91 Stockholm

THE EFFECTS OF VALENCED ODORS ON FACIAL PERCEPTION 

Elmeri Syrjänen

The effects of valenced odorson facial perception 

Elmeri Syrjänen

©Elmeri Syrjänen, Stockholm University 2020 ISBN print 978-91-7911-040-6ISBN PDF 978-91-7911-041-3 Cover illustration by ©Petter Kallioinen Printed in Sweden by Universitetsservice US-AB, Stockholm 2020

To the lights of my life:EmmaEnarVera

i

Svensk sammanfattning

Våra sinnen är verktyg genom vilka vi navigerar i vår omvärld. Synsinnet hjälper oss att förstå andra människors sinnesstämningar genom att vi kan tolka deras ansiktsuttryck. Eftersom människan är en social varelse, är andra människor livsviktiga för vår överlevnad. På samma sätt hjälper luktsinnet att närma oss det som är fördelaktigt och att undvika sådant som kan vara farligt för oss. Vi använder ofta flera sinnen tillsammans för att förstärka sinnesförnimmel-ser. Genom historien kan vi se att människor på olika sätt har förbättrat sitt utseende, men vi har också en lång historia av användande av olika parfymer och aromatiska oljor. Lukters och dofters påverkan på hur vi bedömer andra personers ansikten är ett forskningsområde som är eftersatt. Hur påverkar luk-ter förnimmelsen av ansiktsuttryck? Denna avhandling undersöker hur behag-liga och obehagliga bakgrundslukter påverkar ansikten som uttrycker kongru-enta och inkongruenta ansiktsuttryck, dvs. ansikten som uttrycker glädje eller äckel. För att besvara frågeställningarna har jag använt ett flertal metoder för att testa hur lukter påverkar ansiktsperception, uppmärksamhetsprocesser till ansikten och kortikal bearbetning av ansikten. Bland annat har jag använt frågeformu-lär som mäter personlighetsdrag, EEG, beteendemått såsom reaktionstid, skattningar av ansikten och lukter. I samtliga studier som genomförts har för-sökspersonerna fått skatta valens (positiv/negativ känsla) och arousal (känslo-mässig aktivering eller styrka) av ansiktena. Totalt genomfördes tre studier för att testa effekterna. I studie 1 användes EEG för att mäta deltagarnas hjärnaktivitet vid visandet av bilder på glada, neutrala och äcklade ansikten i kombination med en be-haglig eller obehaglig bakgrundslukt. Resultaten visade att lukterna påverkar hjärnans bearbetning av ansikten i ett tidigare skede (ca 170 ms) än vad före-gående forskning funnit. Resultaten kan tolkas som att när information från flera sinnen är sammanstämmiga behöver hjärnan en mindre grad av bearbet-ning av informationen. Dock återfanns denna effekt bara för äcklade ansikten, något som kan bero på att negativ information är viktigare att bearbeta jämfört med positiv information.

ii

I studie 2 undersöktes om lukter påverkade hur snabbt glädje eller äckel kan identifieras i dynamiska ansikten. Deltagarna fick se korta filmsnuttar där ne-utrala ansikten omvandlades till glada eller äcklade ansiktsuttryck i en behag-lig eller obehaglig luktkontext. Resultaten visade att lukterna inte hjälpte del-tagarna att känna igen specifika ansiktsuttryck, men när bakgrundslukten var obehaglig var deltagarna generellt snabbare på igenkänning av ansiktsuttryck. Resultaten kan tolkas som att en obehaglig lukt kan påverka hur alerta delta-garna är. Resultaten visade även att mellan-individfaktorer som äckelkänslig-het för kroppslukter inte kunde förklara dessa effekter. I studie 3 undersöktes om bakgrundslukter kunde påverka uppmärksamhets-processer till ansikten. I studien användes en experimentell procedur som kal-las dot-probe. I en dot-probe-uppgift visas två ansikten (glad eller äcklad på ena sidan och neutral på andra) på vardera sidan av en datorskärm samtidigt som deltagarna har blicken i mitten av skärmen. Efter en tid försvinner ansik-tena och bakom en av dessa finns en punkt. Deltagarnas uppgift var att besvara om punkten var på höger eller vänster sida av skärmen, en uppgift som anses mäta snabb förflyttning av uppmärksamhet. Resultaten visade på att deltagarna blev snabbare på att respondera till punkten över tid, men bara under exponering för den obehagliga bakgrundslukten. Re-sultaten kan bero på att en obehaglig luktkontext leder till att deltagarna blir mer vaksamma och därför bättre på uppgiften över tid. I övrigt fanns det inga effekter av varken lukter eller ansiktsuttryck på hur snabbt deltagarna respon-derade, vilket ligger i linje med ny forskning som har ifrågasatt om dot-pro-beexperiment verkligen mäter hur uppmärksamhet förflyttas. Sammanfattningsvis visar resultaten i avhandlingen att bakgrundslukterna på-verkar ansiktsperception. Generellt uppfattas ansikten som mer negativa eller positiva och mer känslomässigt starka beroende på luktkontext. Ansiktena skattades något mer positivt i en behaglig luktkontext och något mer negativt i en obehaglig luktkontext. De skattades även som något mer känslomässigt starka i en luktkontext i jämförelse med utan luktkontext. Dessa effekter var oberoende av ansiktsuttryck. Vidare visar resultaten på att särskilt obehagliga lukter kan snabba upp reaktionsförmågan, men att detta sker oberoende av ansiktsuttryck, och vare sig det är igenkänning av ansiktsuttryck eller punkter i dot-probe uppgiften. EEG-resultaten visade att affektiva lukter och äcklade ansikten kan integreras i hjärnan tidigare än man tidigare trott, men det behövs mer forskning om denna effekt. Sist men inte minst, trots luktsinnets betydelse för vår överlevnad är det fortfarande relativt outforskat vetenskapligt. Resul-taten i mina studier kan ge några viktiga pusselbitar i hur lukter kan påverka ansiktsperception.

iii

List of studies

This doctoral thesis is based on the following studies: 1. Syrjänen, E., Wiens, S., Fischer, H., Zakrzewska, M., Wartel, A., Larsson, M.,

& Olofsson, J. K. (2018). Background Odors Modulate N170 ERP Component and Perception of Emotional Facial Stimuli. Frontiers in Psychology, 9, 1000. doi: 10.3389/fpsyg.2018.01000

2. Syrjänen, E., Liuzza, M. T., Fischer, H., & Olofsson, J. K. (2017). Do Valenced Odors and Trait Body Odor Disgust Affect Evaluation of Emotion in Dynamic Faces? Perception, doi: 10.1177/0301006617720831

3. Syrjänen, E., Fischer, H., & Olofsson, J. K. (2019). Background odors affect behavior in a dot-probe task with emotionally expressive faces. Physiology and Behavior. doi: 10.1016/j.physbeh.2019.05.001

iv

v

Contents

Svensk sammanfattning .......................................................................... i

List of studies .......................................................................................... iii

Contents .................................................................................................... v

Abbreviations .......................................................................................... vii

Introduction ............................................................................................. 1

Our senses ............................................................................................... 5 Vision........................................................................................................................... 5 Olfaction ..................................................................................................................... 6

Role of attention in perception .............................................................. 9

What are emotions made of? ...............................................................11

Disgusted and happy facial expressions .............................................19

The Behavioral Immune System .........................................................21

Cross-modal perception: Integrating vision and olfaction ...............23 Review of behavioral effects of odors on face perception ............................... 24

Aims of the thesis ..................................................................................31

Methods ...................................................................................................33 General methods ..................................................................................................... 33 Ethics ......................................................................................................................... 33 Open science............................................................................................................ 33 Statistics ................................................................................................................... 34 Stimuli ....................................................................................................................... 35 Ratings ...................................................................................................................... 37 RTs ............................................................................................................................. 37 EEG ............................................................................................................................ 38

Summary of studies...............................................................................39 Study 1 ..................................................................................................................... 39 Study 2 ..................................................................................................................... 47 Study 3 ..................................................................................................................... 53

vi

Aggregated data: Odor effects on facial perception .........................61

General summary ..................................................................................65 Do valenced odors influence how faces are perceived? ............................. 65 Do odors affect how faces are processed in the brain? .............................. 66 Do congruent odors and facial expressions influence the amount of information needed to recognize happy and disgusted faces? .................. 69 Do congruent odors, and facial expressions influence attentional processes? .......................................................................................................... 70

Other results of interest ........................................................................................ 71 Strengths and limitations ...................................................................................... 72 Ethical considerations............................................................................................. 76 Future directions ..................................................................................................... 77 Concluding remarks ................................................................................................ 79

Acknowledgments ..................................................................................81

References ..............................................................................................83

vii

Abbreviations

ANOVA Analysis of variance BET Basic emotion theory BF Bayes factor BIS Behavioral immune system BO Body odor BODS Body odor disgust scale CET Constructed emotion theory CS Conditioned stimuli EEG Electroencephalography EPN Early posterior negativity ERP Event-related potential FFA Fusiform face area fMRI Functional magnetic resonance imaging LMM Linear mixed models LPP Late positive potential NHST Null hypothesis significance testing RT Reaction time SAM Self-assessment manikin SD Standard deviation UCS Unconditioned stimuli VAS Visual analogue scale VPP Vertex positive potential

viii

1

Introduction

Throughout recorded history humans have enhanced their visual appearance using make-up and rejuvenating oils (Classen, Howes, & Synnott, 2002). From an evolutionary perspective these practices may have evolved to in-crease reproductive fit by making the practitioner more attractive (Etcoff, 1999). A less obvious fact is that humans throughout history have also used perfumes and fragrances to hide or modify their natural body odors (Allen, Havlíček, & Roberts, 2019). The topic of my thesis is how odors affect the perception and psychological processing of faces. In everyday life, our senses provide us with relevant information from our environment. Imagine being in a train and smelling a foul smell. You scan your surroundings and see that some people are disgusted and some seem oblivious and carry on as though nothing is wrong. Or imagine the opposite, that you look up and see some people expressing disgust and only then notice that there is an unpleasant smell. What do you do? You can try and identify the source and the threat it might pose to your personal health. If you interpret the smell as being dangerous, you might move to a safer place. Or wait and see how other people react and follow suit. Odors in our environment can alert us to imminent danger from possible path-ogen sources. Olfaction has not been well appreciated throughout western world history. In ancient Greece, olfaction was by some regarded as a lower sense not worth discussing, and for Romans odors signaled social identity, but also the contagious properties of others’ bodies (Bradley, 2015). Among Christians, some pleasant odors were seen as sacred, as incenses were associ-ated with heaven, and unpleasant odors were associated with hell and demons. The name of the devil, Mephistoteles, means pestilential odor (Miller, 1997). In a similar vein, until the last couple decades, disgust has been less studied than other emotions. A hypothesis as to why disgust has been relatively ig-nored is that disgusting things are seen as something civilized people should not converse about, the contagious properties of disgust sticks to the bearer of message (Miller, 1997). No wonder then that the sense of smell and disgust has received less attention in the past. Research in vision has received a disproportionate amount of attention from the research community in the past century. This might not be surprising as

2

vision has been described as the most complex sense that involves the largest area of the brain. Vision is also the dominating sense in western cultures and vision is arguably the modality that is easiest to experimentally investigate with available methods and technology (Hutmacher, 2019). In a similar man-ner, emotion research has been dominated by the use of visual stimuli. Olfac-tion however is intimately associated with emotion in humans (Adolph & Pause, 2012). Knowing how olfaction affects vision might tell us a different tale of what happens when emotional information from different modalities is integrated in the brain. Research is not exempt from regular market forces, and as the big questions in one modality or emotion are answered, researchers scramble for finding new gaps to fill. In this vein, the interest in olfaction and disgust seem to go hand in hand. If we look at the research output, it seems that an increase in olfactory research is accompanied by an increase in research on disgust (cf. Figure 1). Both of these research areas have gained in popularity the last cou-ple of decades. This might be due to that they have been under-researched areas for a long time, but also because of the intrinsic connection between them.

Figure 1. Number of peer-reviewed articles published by year between 1945 and 2019 for the search term: (ALL=(disgust AND olfac*) OR ALL=(disgust AND odor)) in Web of Science.

In the following thesis, I will present my research on how valenced (unpleas-ant and pleasant) odors affect the perception of faces with emotional expres-sions. I have focused on expressions of happiness and disgust, because they are assumed to correspond to the emotional responses to pleasant and unpleas-ant odors (Bensafi, Rouby, et al., 2002). Experimentally combining odor and face stimuli allows me to investigate effects of emotional odor-face

3

congruency (e.g. perceiving an unpleasant odor and a disgusted face), as well as overall effects that are independent of such congruency, for example changes in perception of faces due to the presence of an odor, irrespective of the facial expression or odor valence. My overall research question can be described as: Does pleasant and unpleasant odors affect how we process emo-tionally expressive faces. I will focus on emotion recognition performance, perceptual evaluation of faces, cortical brain responses, and attentional pro-cesses.

4

5

Our senses

We use our senses to interact with our environment. The introduction above highlighted two of these senses, vision and olfaction, and how these might affect behavior when we navigate the world. Apart from these, we can also perceive sound, taste and touch; these five senses form what commonly called the classical or basic senses. However, apart from these, depending on how you define a sense, we have up to 20 different senses, including the trigeminal sense that perceives chemosensory irritation in the eyes, nose and mouth. Alt-hough all senses are important in everyday life, this thesis revolves around vision and olfaction and how these two senses together are involved in how we perceive and process social information via facial expressions and odors.

Vision When you look at this thesis, do you see it objectively, as it really is? If we consider the physical process of perceiving an object what we see is light from a narrow band in the electro-magnetic spectrum reflected from the object. The light goes through your pupil and is focused on your retina by the lens. On the retina, cone receptors are activated by the light (photons) and transmit the in-formation to the visual cortex located at the rear part of your brain. In the visual cortex each receptor on the retina map to specific neurons. From here neurons process the visual information in a hierarchical manner from ever the smallest details to the whole that we perceive as an object. At every step of this process, in the lens, on the retina, through the optic nerve, thalamus and regions of the visual cortex, this information is filtered and transformed in various ways. At the same time, higher order brain areas affect the type of filtering and transformation at every step through attentional processes, ex-pectations, memory etc. Now, consider that you looked at this thesis first in-doors and the second time outdoors. Objectively the visual input to your retina will be different, indoors light from white LEDs will dominated by the shorter wavelength bluish light in the electromagnetic spectrum, whereas outdoors, sun light consists of broad band of the whole visible spectrum. Still, even though the conditions are very different, we will recognize this thesis as being the same object (Goldstein & Brockmole, 2017).

6

A special case of visual perception is faces. Humans are social creatures who depend on each other for survival, but other humans can also be a threat. For example, we know from animal research that vigilance for predators is de-creased as a function of herd size (i.e., safety in numbers), and similar results have been found in human research (Dunbar, Cornah, Daly, & Bowyer, 2002; Kameda & Tamura, 2007). The fact that humans live in groups has put a se-lection pressure in the ability to decode other peoples’ intentions and emotions (Schmidt & Cohn, 2001). The importance of facial information results in that we automatically attend more to faces than other stimuli, and it might be that we can process faces faster than we process other objects (Crouzet, Kirchner, & Thorpe, 2010; Palermo & Rhodes, 2007). We also know that there are spe-cific regions in the brain that are particularly sensitive to faces. EEG research using ERP (event-related potentials) indicate that faces are processed early in the perceptual process, as signs of unique processing of faces starts as fast as 120 ms after stimulus onset. This face-selective ERP component is the N170, a wave with negative polarity peaking at about 170 ms after stimulus onset (Bentin, Allison, Puce, Perez, & McCarthy, 1996). The N170 component has been shown to be sensitive to facial emotion as indicated in a recent meta-analysis (Hinojosa, Mercado, & Carretie, 2015). The source of the N170 has been located to the FFA (fusiform face area), located in the STS (superior temporal sulcus), an area that has been shown to be activated in fMRI experi-ments to facial stimuli (Kanwisher, McDermott, & Chun, 1997), although re-cent evidence indicate that the FFA activations reflects object expertise, where faces are something we are all knowledgeable about (Burns, Arnold, & Bukach, 2019). Whether the FFA is sensitive to facial expressions of emotions is still debated (Calder & Young, 2005; Harry, Williams, Davis, & Kim, 2013). Thus, the importance of facial information is reflected in both behav-ioral effects and possibly dedicated areas in the brain that process socially rel-evant information.

Olfaction Now if we consider olfaction, take a sniff of this thesis. Although it does not have the complex and rich smell of old books, most people will perceive a pleasant smell with sweet undertones, perhaps from the paper or glue that binds the pages. In vision, photons hitting the retina set a cascade of activity that leads to us perceiving the object. What is the process of sensing smells? In olfaction, actual molecules released by physical objects are inhaled through the nose and attach to the olfactory mucosa in the roof of our nasal cavity. Embedded in the mucosa are olfactory receptors neurons (350 – 400 types that are sensitive to a narrow range of molecules, these olfactory receptor neurons (ORN) in different patterns can lead to a remarkable number of odors that can be discriminated. Once an ORN is activated the signal is transmitted to the

7

olfactory bulb. Animal work has shown that the ORNs map to specific loca-tions in the olfactory bulb, a similar mapping as for vision, where specific receptors map to specific areas in the primary visual cortex, however, the exact organization of the olfactory bulb is not known for humans yet (Goldstein & Brockmole, 2017). From here olfactory signals are transmitted in parallel to the primary olfactory areas (e.g., piriform cortex and amygdala) in bidirec-tional connections. Secondary olfactory areas are the OFC (orbitofrontal cor-tex), agranular insula and basolateral amygdala (Gottfried, 2010). With only this limited background we can see that vision and olfaction differ substan-tially. For example, olfactory receptor neurons are the only part of the brain that is in direct contact with the outside world. Whereas visual input is relayed through the thalamus, olfaction is in just a few steps in connection with both prefrontal (areas involved in cognition) and limbic (areas involved in emotion) brain areas. Here, a key area is the anterior insula, which has been implicated in the integration of both disgusted faces and unpleasant odors. Both stimuli types resulted in similar fMRI activations in the anterior insula (Wicker et al., 2003). The tight integration of olfaction with the limbic system is a sign of the close connection of olfaction with our survival. Olfaction is involved in both the detection of danger (e.g., fire, bad food), and nutrition. This link between ol-faction and survival explains the emotional nature of the olfactory system. There is a wide agreement that the main hedonic dimensions of olfaction are valence and arousal (Anderson et al., 2003; Bensafi, 2002). In addition, it has been shown that valenced odors may evoke discrete emotions such as happi-ness and disgust (Alaoui-Ismaïli, Robin, Rada, Dittmar, & Vernet-Maury, 1997; Croy, Olgun, & Joraschky, 2011; Glass, Lingg, & Heuberger, 2014). Although habituation to valenced odors have been reported in the literature (Ferdenzi, Poncelet, Rouby, & Bensafi, 2014), some authors argue that va-lenced odors are more potent at eliciting emotions than valenced pictures (Adolph & Pause, 2012). Thus, the effect of odors can have profound motiva-tional effects on human behavior by their strong ties with the emotional sys-tem.

8

9

Role of attention in perception

What is the role of attention in perception? If we consider the vast amount of information that is available from all of our senses at the same time. One way to think about this is what the requirements would be for a brain to fully pro-cess all available information. It would require a much larger brain that uses much more energy, however, the human brain already is the largest energy consumer in the body and consumes about 20% of the energy (Raichle & Gusnard, 2002). Thus, attention can be described as a mechanism that en-hances the processing of attended stimuli while suppressing unattended stim-uli (Handel, Haarmeier, & Jensen, 2011). Attention is the mechanism that brings our perceptions to awareness. There is no need to be aware of all the available information in our environment at all times, for example, what your toes are up to while you sit and read this thesis (Goldstein & Brockmole, 2017). Thus, attention is a mechanism that helps in selecting and processing of information in a cost-effective way. But what happens if there is important information that requires access to awareness? Attentional mechanisms are very flexible. In general, attention can be influenced by both top-down (endogenous) and bottom-up processes (ex-ogenous). Top-down processes can be described as processes that are under cognitive control, such as reading this thesis, you attend to the book and read the words and process the information, while not attending to other sensory input (Moore & Zirnsak, 2017). Bottom-up attention can be described as processes outside of cognitive con-trol, such as change detection; did anything change in our environment? If you are out in the forest and suddenly it becomes completely quiet, it might be advantageous to quickly find out why. We also automatically attend to sur-vival-relevant stimuli in our environment, both when it presents a danger or an opportunity for us (Bradley, Keil, & Lang, 2012). Research also indicates that the attentional effects of exogenous (external) and endogenous (internal) cues increase additively (Brosch, Pourtois, Sander, & Vuilleumier, 2011; Pourtois, Schettino, & Vuilleumier, 2013). Thus, both saliency (e.g., survival relevance) and deliberate attention (e.g., task effects) affect visual perception.

10

Odor effects on visual attention Human olfaction differs substantially from the visual sense in that we are gen-erally bad at identifying odors (Olofsson & Gottfried, 2015). In addition, typ-ically, we are not usually aware of our olfactory surroundings. It has been suggested that the role of olfaction is change detection, alerting us to possible dangers or when we encounter odors in unexpected circumstances (Koster, Moller, & Mojet, 2014). Further, humans are bad at explicitly detecting the location of odors (Moessnang, Finkelmeyer, Vossen, Schneider, & Habel, 2011). These properties of olfaction highlight the need for using other senses in tandem to identify or find the source of an odor. Although the link between odors (a bottom-up cue) and visual attention (top-down attention) has not been fully explored yet, there is some evidence that odors may affect attentional processes. For example, in two experiments, Pool, Brosch, Delplanque, and Sander (2014) could show faster spatial orienting to visual shapes associated with chocolate odor, an effect that was diminished when the value of the re-warding odor was devalued. Studies show also that participants attend more to objects that are congruent with a contextual odor (Seo, Arshamian, et al., 2010; Seo, Roidl, Muller, & Negoias, 2010). In line with these results, un-pleasant odors associated with objects shift visual attention away and pleasant odors toward these associated objects (Rinaldi, Maggioni, Olivero, Maravita, & Girelli, 2018). These results indicate that odors may provide powerful ex-ogenous cues, and may influence attentional effects.

11

What are emotions made of?

From everyday experience, what emotions are is straightforward. But how do we define emotions in a scientific context? It proves more difficult than you might imagine. The word emotion itself is a fairly recent word for emotions in English language. Earlier, what we now refer to as emotions were usually called “appetites,” “passions,” “affections,” or “sentiments”. Etymological records show that the word emotion was imported from the French word for physical disturbance “émotion” in the 17th century, and although several writ-ers such as Descartes used the new term, it did not catch on until mid-18th century (Dixon, 2012). Modern day emotional theories started to emerge in the early 18th century. The modern definition of emotion was formed in the early 18th century by two Scottish philosophers, Thomas Brown and Charles Bell from Edinburgh. Ac-cording to Thomas Brown, the exact definition of emotion was difficult to define in words but was based on feelings toward objects that are perceived, remembered or imagined. Charles Bell, a philosopher and physician, argued that emotions were a result of bodily changes, being it changes in heart rate or facial expression, these bodily changes strengthened and directed emotions. Especially Darwin’s work on “The expression of the emotions in man and animals” was influenced by Charles Bell’s ideas (Darwin, 1872; Dixon, 2012). The word emotion finally gained traction when William James wrote an article called “What is an emotion?” in 1884, integrating the ideas of Brown, Bell and Darwin. In this article, James defined emotions as clear mental feelings of bodily sensations caused by perception. Although James notion was heavily criticized, and the scientific community has not been in agreement on what emotions are since then, it was influential in establishing emotion as a separate category, opposed to passions and affections, that was free from the religious baggage that were attached to those words during that time (Dixon, 2012). During the writing of this thesis, I have noticed the major impact that philos-ophers have had on emotion theories. While emotion researchers usually try to answer specific questions empirically and theorize about specific processes; philosophers on the other hand have had a birds eye view and tried to integrate both philosophical and literary sources, as well as empirical data to form a coherent theory of emotions. In many cases they have been able to critique psychological emotion theories on logical grounds by finding inconsistencies

12

not noticed by scientists. From a philosophical point of view, emotion theories can be sorted in three broad categories; feelings, motivations and evaluations (Scarantino, 2018). The “feeling” tradition broadly encompasses subjective properties of emotion, how emotions “feel” in layman terms, and have dominated thoughts about emotions from ancient Greece until today. Here William James “The Physical Basis of Emotion” (1894) was an influential modern theory on emotions; James critiqued previous emotion research for being merely descriptive. James insight was that emotion research should investigate the physiological causal mechanisms of emotion. According to Mandler (1990), William James was the first to introduce the idea that what we perceive changes in bodily states, thus, emotions are constructed from changes in what we feel. In modern day, influential constructionist emotion theories that are inspired by William James include Lisa Feldman-Barrett (2006), but more on this later. The motivational tradition is rooted in the ideas of Plato, who suggested that “pathos” energized the body and prepared it for pleasure or pain. Modern day motivational emotion theory starts with John Dewey in late 1800s; he wanted to combine Darwin’s ideas about the functional aspects of emotions with Wil-liam James emotion theory (Scarantino, 2018). The main points in Dewey’s theory was first that certain expressions promoted survival, for example baring teeth was originally preparing to bite, according to Darwin, this function then adapted and started signaling threat. Second, in contrast to James where emo-tions were based on feelings, emotion was associated with behavior, where behaviors affected the feeling. Further, Dewey’s contribution was that feel-ings, in contrast to for example moods, were always directed toward some-thing, an object or person, thus we feel because we have to act on something. While not explicitly in the motivational tradition, James Russell (1980) was very influential in defining the affective space into valence and arousal in his two-dimensional “circumplex model” of affect. This model is still widely used today, mostly in the form of a pictorial version called the self-assessment man-ikin where participants rated their subjective experience of emotional stimuli (SAM; Bradley & Lang, 1994). The SAM-scale is widely used in a branch of emotion research that is firmly rooted in the motivational tradition, the affec-tive neuroscience that uses the ERP-method (Olofsson, Nordin, Sequeira, & Polich, 2008). This line of research is limited, in that typically ERPs cannot readily differentiate between positive and negative valence, but a common finding is larger ERP amplitudes as a function of higher arousal (Wiens & Syrjänen, 2013). In much of this research using emotional pictures, rated va-lence and arousal go hand in hand, i.e. higher valence (positive or negative) is associated with higher arousal (Bradley, Codispoti, Cuthbert, & Lang, 2001; Olofsson et al., 2008). Thus, even if some methods are unable to differentiate

13

between specific emotional states, they can inform us of the significance of survival-relevant stimuli. How do we know what emotion we are experiencing? In the evaluative tradi-tion, this is a cognitive process that depends on the situation. Similar to the other emotion theories, the evaluative tradition has a long history, starting in ancient Greece. Modern day evaluative emotion theories started in the 1960s with the cognitive revolution in psychology as a reaction against behaviorism that had dominated psychological thinking in the preceding decades. One of the pioneers in this field, Magda Arnold, wanted to explain how perceiving something could cause an emotion, something that was largely ignored until then. As discussed by Scarantino (2018), Arnold (1960) stated that “to arouse an emotion, the object must be appraised as affecting me in some way, affect-ing me personally as an individual with my particular experience and my par-ticular aims”. Later appraisal theories, such as the component process model proposed by Scherer (2009a), define appraisals as recurrent emotional pro-cesses affected at multiple levels that will prepare an organism for survival-relevant behavior. Appraisal processes answer questions such as, what is the relevance to self or others, consequences of the event, the ability to cope in the situation, and how important is the event for a self-concept in respect to social norms and values. These appraisal processes are processed at different levels in the brain. The processes can be divided by automatic processes out-side of cognitive control at a neural level where simple template matching is performed on biologically innate representations of objects such as snakes (Ohman & Mineka, 2001), and a schematic level based on previous learning (Scherer, 2009a). Lastly, purely cognitive appraisal at a conceptual level in-volves frontal areas that requires explicit knowledge and may be influenced by cultural factors (Scherer, 2009b). In this model, appraisal processes at dif-ferent levels interact with each other and ultimately produce an emotion. Most modern emotional theories have incorporated ideas from appraisal theories in their theoretical frameworks, that is, how someone evaluate a situation will determine how they will experience the emotion (Scarantino, 2018). Today, most emotion theories contain some combination of subjective feel-ings, motivation and evaluations/appraisal. But no emotion theory has been able to integrate all aspects of emotions into a single coherent emotion theory. From here, I will discuss two theories that are prominent in today’s research, basic emotion theory (BET) of Paul Ekman, which has been very influential for the last 50 years, and a more recent theory by Lisa Feldman-Barret, the constructed emotion theory (CET). Although these researchers do not exist in a vacuum, Ekman collaborated with Friesen in forming the BET, and both of them were students of Silvan Tomkins and their work represents a continua-tion of Tomkin’s work on discrete emotion theory. For the CET, James Russel has been instrumental in much of the work. However, I have chosen to

14

concentrate on two specific persons that are more or less synonymous with the theories that they propose, Ekman and Feldman-Barret. In basic emotion theory, facial expressions are an integral part of emotion. BET has been one of the most influential theories of facial emotion for dec-ades (Hutto, Robertson, & Kirchhoff, 2018). The story of basic emotions be-gins in the 1970s, when Paul Ekman did not agree with the current ideas about facial expressions being socially learned and variable between cultures. He was inspired by the evolutionary approach to emotions as suggested by Dar-win and the work on discrete emotions by Tomkins (Ekman, 1992). Darwin’s revolutionary idea was that emotions were a solution to everyday problems that our ancestors encountered. In Darwin’s theory certain emotions were in-herited, meaning they were innate and universal within and between different species. Emotional events were associated with bodily changes such as auto-nomic activity that prepared the organism to act. These emotional experiences were also associated with physical changes such as posture and facial expres-sions. Darwin’s idea was that the original function of the facial musculature was for chewing, shading of the eyes etc. This musculature adapted to other behaviors such as baring of the teeth in preparation of an attack. As these facial expressions had added value, such as signaling aggression, they were pre-served even in species that no longer had this behavior (Darwin, 1872). To test Darwin’s ideas on universal facial expressions, Ekman & Friesen travelled to Papua New Guinea in 1969 bringing with them photographs of faces dis-playing prototypical expressions of anger, fear, happiness, sadness, disgust, and surprise. In Papua New Guinea Ekman and Friesen contacted the Fore tribe that previously had limited contact with westerners and no access to me-dia. They could show that people in this remote culture could recognize the displayed emotions just as well as westerners. As the Fore tribe could not have learned western facial expressions of emotions, the researchers concluded that this was evidence for the universality of emotional facial expressions. In the last 50 years basic emotion theory has evolved as a result of criticism and as more empirical data has been collected. The latest version of BET as defined by Ekman and Cordaro (2011) present the current version of the BET. The foundation of BET is that emotions are discrete and that that the emotions have evolved trough adaptation to our environment. To distinguish between basic emotions and other affective states such as mood, a basic emotion has to meet these thirteen criteria: distinctive universal signals; distinctive physio-logical effects; automatic appraisal; universal cause or event that elicit the emotion; presence in other primates; quick onset; may be brief in duration; outside of voluntary control; distinctive thoughts, memories and images; dis-tinctive subjective experience; affect perception toward emotionally congru-ent information; may be directed toward any person and lastly emotions may affect behavior in both constructive and destructive ways. According to

15

Ekman and Cordaro (2011), there is sufficient evidence for universality in these seven specific emotions: anger, fear, surprise, sadness, contempt, as well as the two facial expressions of interest for my present work, disgust and hap-piness. In their view, additional emotions may be added when there is enough empirical support that these emotions meet the above listed criteria. Basic emotion theory has been heavily criticized, and some of the most damn-ing evidence against BET regards the notion of universality. For example, there are clear differences between cultures in how emotions are expressed and recognized (Gendron, Crivelli, & Barrett, 2018). Further, the notion that discrete emotions should have distinct physiological fingerprints has been challenged. Several meta-analyses have not found reliable physiological fin-gerprints in autonomic changes (Siegel et al., 2018), nor fMRI activation pat-terns (Lindquist, Satpute, Wager, Weber, & Barrett, 2016), tied to specific emotions. Further, the methods that have been used to investigate universality of basic emotion facial expressions have been questioned (Barrett, Adolphs, Marsella, Martinez, & Pollak, 2019; Russell, 1993). For example, some of the problems mentioned are that in the photos, the facial expressions used were poses and do not reflect natural emotional expressions, and that contextual effects are not considered (Barrett et al., 2019). Specific studies investigating universality of emotion recognition in remote cultures have also given varied results (Gendron et al., 2018). The methods that use predefined emotional la-bels skew the results in typical emotional expression tasks (Russell, 1993). Although, some authors argue that even when accounting for the methodolog-ical criticism, there is still evidence for universality in both recognition and expression of emotions (Cordaro et al., 2019; Cordaro et al., 2018; Majid, Burenhult, Stensmyr, de Valk, & Hansson, 2018). Constructed emotion theory has a somewhat different approach to what emo-tions are. CET is based on theories of how the brain process perceptual infor-mation. The key idea behind these theories, predictive coding (Clark, 2013; Rao & Ballard, 1999), or the free energy principle (Friston, 2010), is that it is costly, requires a lot of energy, to process incoming sensory information. As we already know a lot about the world, it is more efficient for the brain to predict what we are perceiving and only process the information that does not match our internal model of the world (Hutchinson & Barrett, 2019). Previ-ously it has been thought that the function of the brain is to maintain homeo-stasis in the body, however, a newer model suggests that this is very ineffi-cient, for example, to seek out food only when hungry. Instead, the brain tries to predict from previous experiences and current status a future state and main-tain allostasis, i.e. anticipate hunger and prepare before it happens (Sterling, 2012). These ideas form the basis of constructed emotion theory (Barrett, 2017b).

16

In the constructivist view, emotions are constructed on the fly by using active inference to monitor bodily changes and by categorization. All our previous emotional experiences create a model of emotional experience that affect how we react and express specific emotions. Although biological factors are still important, the appropriate way to experience and express emotions is a learned process that is influenced by the cultural background. In the constructed emo-tion view, context is an important factor. In this view, we use all available information to predict and correct our model of the world using active infer-ence (Barrett, 2017a, 2017b). How do basic findings in emotion research fit with the constructed emotion theory? What hypothesis can be formulated on basis of the CET? And how do we experimentally test these hypotheses? My view is that much effort has been spent on criticizing previous research and less on developing CET (Barrett, 2017b). In most published research, experimental stimuli are for methodolog-ical reasons often presented in random order. Thus, as stimuli are presented in an unpredictable manner, prediction error should be high. However, compared to neutral stimuli, we see larger activations for emotional stimuli. However, few authors appear to discuss how to interpret the results in previous emotion research. These types of effects are briefly mentioned by Satpute, Hanington, and Barrett (2016). The best evidence for CET comes from studies with re-peated presentation of affective and neutral stimuli (Satpute et al., 2016), showing that as stimuli are repeated and become familiar and less arousing, prediction error is reduced. Trapp and Kotz (2016), suggest that because of the importance of affective information, bottom-up processes may over-shadow prediction error processes. I think that the future for empirical evi-dence for CET lies in experimentally varying the predictability of emotional stimuli. Previous emotion research might reflect combinations of top-down and bottom-up processes. A top-down process would be that the brain tries to predict upcoming survival relevant events. However, when survival relevant stimuli is unpredictable, they might receive enhanced activations that are mostly dependent on bottom-up processes. Throughout the years, BET has been modified as a result of criticism and new empirical evidence against some of its foundations. Still, BET shows its age, being a theory from a time when biological determinism was prevalent, the brain was seen as modular and not much emphasis was put on cultural factors shaping emotions. Perhaps the research community is starting to shift towards a more constructivist approach. Still, in my view, one of the greatest ad-vantages of BET is that it is well defined. For me, a well-defined theory is a theory that is useful for both generating testable hypotheses and that can be falsified. CET, on the other hand, is still developing. Much focus has been put on criticizing other approaches, but it is unclear what the key findings are that can only be explained by the CET.

17

In the research for my thesis, I have used stimuli that are based on BET, as my face stimuli represented prototypical expressions of happiness and disgust, and I selected odors that I assumed to be congruent with these facial expres-sions. However, some aspects of CET are also present in my studies, for ex-ample the focus on individual differences and context-dependent effects. These concepts are more central in CET than in BET. In the next chapter I will turn to facial expressions of happiness and disgust, the two emotional expres-sions that have been instrumental in my studies on how odors and facial ex-pressions are integrated.

18

19

Disgusted and happy facial expressions

Darwin was the first author to present a coherent theory of facial emotion. His groundbreaking idea was that facial expressions did not originally evolve for social communication (Darwin, 1872). Facial muscles were for eating, aiding in breathing and protecting the eyes. As facial musculature evolved, the mus-culature gained additional functions that were beneficial for survival. The ha-bitual use of these functions was then adapted for other functions, such as signaling social cues. One of the assumptions in this theory is that facial emo-tional expressions are universal. There is a long history of criticism against the notion of universality in facial expressions of emotion (Russell, 1993). Mainly, this criticism has been based on concerns on the forced choice methods Ekman, Sorenson, and Friesen (1969) used in their original study. Recently, these results have been chal-lenged by empirical data showing that recognition of facial expressions is much more diverse than previously thought (Gendron et al., 2018). However, there is also recent evidence for the opposite interpretation. When presenting pleasant and unpleasant odors to Dutch and Jahai participants (a group of hunter-gatherers in Malaysia), both groups expressed pleasure and disgust in a similar manner, indicating that, at least for odors, there seems to be a clear connection between valenced odors and facial expressions across two very culturally diverse groups (Majid et al., 2018). Further, even the harshest critics of basic emotion theory agree that within cultures, emotion recognition from facial expressions is high (Barrett et al., 2019). Thus, even though there are some problems with the basic emotion theory, it is still a useful theory to use within a given cultural context. In my research I have used two distinct facial expressions, disgust and happi-ness. From a perceptual perspective these both emotional expressions are fairly similar. Both of these expressions are “mouth emotions” as the main emotional content is expressed by the mouth. This is reflected in very similar gaze patterns in emotion recognition. Participants initial gaze is directed to the mouth and they view this area longest for both disgusted and happy expres-sions compared to other emotional expressions (Schurgin et al., 2014). How-ever, disgust and happiness differ in one important aspect. Disgust is associ-ated with negativity and happiness with safety and security. These expressions also differ in other aspects, for example happy faces are the most frequent

20

facial expression in naturalistic settings (Calvo, Gutiérrez-García, Fernández-Martín, & Nummenmaa, 2014), which could explain why it is the most accu-rately recognized emotion (Barrett et al., 2019), a finding that generalizes across cultures (Gendron et al., 2018). Thus, both disgusted and happy facial expressions may act as important social signals in our environment. In the next section I will discuss disgust, an emotion that is well-characterized both in terms of its evolutionary origin and the adaptations that it has resulted in.

21

The Behavioral Immune System

Disgust is important for human behavior; this has been described in the Be-havioral Immune System (BIS) theory. A starting point for the theory is that throughout human and pre-human history, pathogens have been one of the deadliest threats and, as such, a major selective force in the evolution of all species. This threat has resulted in many adaptations to counteract the dangers of pathogens, such as an immune system that can generate antibodies against infection. However, fighting an infection is costly; for an organism, it would be much more efficient to avoid getting infected in the first place (Murray & Schaller, 2016). The Behavioral Immune System (BIS), a theory suggested by Schaller and Park (2011), tries to explain how an external immune system works. Cues that signal the presence of pathogens are preferentially processed. The detection of pathogen cues causes arousal and aversive feelings; this ac-tivation of the BIS leads to behaviors that result in avoidance of the pathogen source (Schaller & Park, 2011). There are several strategies to avoid patho-gens, such as proactive and reactive behaviors (Schaller, 2014). Proactive be-haviors include hygienic customs such as washing hands and food cooking practices. Reactive behaviors are typical reactions that involve the emotion disgust, such as sight and smell of feces, or facial and bodily disfigurement. This raises the question of what is the relation between BIS and disgust (Lieberman & Patrick, 2014), clearly proactive BIS-related behavior does not entail the emotion disgust proper. Schaller (2014) argues that activating the emotion disgust is costly. However, disgust reactions are potent in forming memories, and these memories can then form a basis for proactive behaviors and reduce the number of situations where BIS is activated. As pathogens are invisible to the naked eye, we have to use cues that signal the presence of pathogens to detect them (Tybur & Lieberman, 2016). For example, humans can use olfactory cues of sickness in body odor (Olsson et al., 2014) and they like photos of faces of sick individuals less than those of healthy individuals (Regenbogen et al., 2017). Some research also indicates that the immune system becomes active when seeing sick faces (Schaller, Miller, Gervais, Yager, & Chen, 2010). The detection of pathogen cues is as-sociated with behavioral changes, for example, facial expressions of disgust for unpleasant odors (Majid et al., 2018), social distance to diseased (Crandall & Moriarty, 1995), and avoidance of persons who display pathogen cues (van Leeuwen & Petersen, 2018). Behaviorally, expressing disgust leads to a

22

narrower visual field, slower eye movements, and a decrease in nasal volume and air velocity during inspiration, all behaviors that limit the exposure to harmful substances (Susskind et al., 2008). Activation of the BIS system is associated with physiological changes, such as faster breathing, lower body temperature, and decreased respiration volume, all signs of parasympathetic activity that has an inhibitory role in organisms (Kreibig, 2010; Stark, Walter, Schienle, & Vaitl, 2005; Susskind et al., 2008). All of these results indicate that pathogen cues may be detected from olfactory and visual cues. This de-tection also leads to changes in behavior and autonomic activity. In the next chapter, I will review research on how the olfactory and visual systems inter-act.

23

Cross-modal perception: Integrating vision and olfaction

Although our experience is typically dominated by visual cues, we can use a wide variety of our senses when we navigate the world. Previous research has shown that when we use multiple modalities, our ability to recognize objects becomes more accurate and faster. For example in a noisy environment, seeing someone’s lips while he/she talks will improve a person’s ability to hear the content of that someone’s speech (Ross, Saint-Amour, Leavitt, Javitt, & Foxe, 2007). Odors can, in a similar manner as hearing, influence visual perception. For example, smelling an orange can increase visual attention towards oranges (Seo, Roidl, et al., 2010). In general, the arousing properties of odors may facilitate the processing of stimuli. In this line, Millot, Brand, and Morand (2002) found faster RTs to simple visual and auditory cues in both unpleasant and pleasant odor context in comparison to a no-odor context. We use multiple sensory modalities in relation to objects. But do we use our noses in social communication? We know from ancient texts and archeologi-cal findings that humans have used perfumes for a long time (Classen et al., 2002). Groyecka et al. (2017) review multimodal effects on attractiveness, among other modalities also odor effects. Research shows that loss of smell, anosmia, may lead to loneliness; people who cannot smell themselves may be hesitant to engage with other people (Croy, Nordin, & Hummel, 2014). In a seminal study, Wicker et al. (2003) showed that disgusting odors and dis-gusted faces activate the same brain area (anterior insula), indicating that there is a cortical overlap between being disgusted by a malodor and seeing some-one expressing disgust. This provides an interesting avenue for disgusting odors and facial expressions to integrate. It might be that seeing someone ex-press disgust activates the expressed emotion in the perceiver, as proposed by Tomkins’s affect theory. These facts in themselves are not evidence for the communicative functions of odors, and could be explained by cultural factors for example. There is now a growing body of research showing effects of odors on facial perception, which I review in the section below.

24

Review of behavioral effects of odors on face perception Because my empirical research concerns the role of background odors in the perception of facial emotional expressions, I have performed a review for this thesis on previously published experimental research where odors were used to modulate the perception of facial expressions. Note that some of the re-viewed research has been published during and after my thesis studies were carried out.

Methods I performed searches in Google scholar, PsycINFO, PubMed and Web of Sci-ence for keywords: (face + odor | body odor | cross modal), as well as in my own database of articles. Suitable articles were also manually searched in the reference lists of all included articles. Below is a list of the inclusion criteria that was used. Inclusion criteria

1. Peer-reviewed articles 2. Empirical studies 3. Adult participants 4. Healthy participants with no reported mental illness 5. Reported behavioral effects of odors on faces

Results In total, I found 26 articles matching the inclusion criteria. These studies are presented with summaries in Table 1 in supplementary materials S1. All pub-lished studies using natural or chemical odorants should be included. Alt-hough my review of studies using body odors may not be exhaustive, it should provide a good overview of the published work. The methods used in these studies can be roughly divided in two experimental paradigms, first, recogni-tion tests of facial emotion where reaction times and accuracy are measured and second, evaluations (typically ratings of facial valence and arousal) during exposure to various odors. In many of the reviewed studies the main experi-mental outcome was electrophysiological measures or brain activation using fMRI. Common odors used in these studies are coarsely divided in three groups, pleasant (vanillin and floral-based such as jasmine), body odors that are rated as neutral or negative (sport or anxiety induced BO), and unpleasant (valeric acid and chemically similar odors such as 4-methylpentanoic acid, hydro

25

sulfide, methyl mercaptan and fish). A majority of all studies also used clean air as a control condition. A general method in studies that use BO (body odors) is that sweat samples are collected from an independent sample of participants that do not partici-pate in the experimental condition. These BO donors are typically on a re-stricted diet and are asked to refrain from eating strong foods and use of per-fumes or deodorants. Sweat samples are collected while the donors engage in either a fear or anxiety inducing activity such as giving a public speech (Pause, Ohrt, Prehn, & Ferstl, 2004; Zernecke et al., 2011). As a control condition, clean air and/or sweat samples collected during exercise are used. Some stud-ies also have used artificially produced compounds found in human sweat such as androstenone (Hornung, Kogler, Erb, Freiherr, & Derntl, 2018; Hornung, Kogler, Wolpert, Freiherr, & Derntl, 2017). Odors in these studies are typically perceived as neutral and low in intensity or near subthreshold, some studies also have used a masking odor (Hornung et al., 2018; Hornung et al., 2017). The facial expressions in these studies typically are ecologically relevant to the emotions that the odors are meant to represent displaying anger or fear (Rocha, Parma, Lundstrom, & Soares, 2018; Zhou & Chen, 2009). Some studies have also investigated the effects of BO on neutral and happy expressions (Platek, Thomson, & Gallup, 2004; Regenbogen et al., 2017; Zernecke et al., 2011). The main measures in these studies are reaction times and accuracy in an emotion recognition task or effects of BO on facial valence and arousal ratings. Common facial expressions used in all reviewed studies were: happy faces, neutral (most prevalent in associative learning studies), disgusted, fear/anxiety (mostly in BO studies) and anger. See supplementary materials S1 for a table containing a complete list of studies. This table comprises of each study in the review, an overall description of the design, odors and facial expressions that were used, number of participants and a summary of the results.

Discussion The methods and stimuli used across the reviewed studies were quite hetero-geneous; however, the findings indicate that valenced odors affect the pro-cessing of faces. A few studies show facial and odor valence congruency ef-fects; the overall pattern suggests that the most consistent finding is that va-lenced odors affect behavior overall (e.g. faces are perceived as more unpleas-ant in an unpleasant odor condition). These results is discussed below. In the reviewed studies, the primary outcomes were often measures of brain activity using EEG or fMRI; whereas, the behavioral effects were often sec-ondary measures or control conditions. In these studies, the type of experi-mental paradigms can be divided in four coarse categories; first, effects of

26

valenced odors, second, body odor effects, third, classical conditioning with valenced odors and fourth, effects of valenced odors on ambiguous facial ex-pressions.

Valenced odor effects Several studies used reaction times as a measure to investigate the effect of valenced odors on emotion recognition. To my knowledge, Leppanen and Hietanen (2003) were the first to show that pleasant odors facilitated the recognition of happy faces, whereas the speed of recognizing disgusted faces was not affected by either pleasant or unpleasant odors. In the same study a second experiment confirmed the effect of pleasant odors on happy faces. Alt-hough several studies have used similar methods as Leppanen and Hietanen (2003), the odor-face congruency effects in this study have not yet been rep-licated. In two separate studies by Seubert and colleagues, the results indicated only overall effects of odors (Seubert, Kellermann, et al., 2010; Seubert, Loughead, et al., 2010). In one of these studies the results showed that partic-ipants were more accurate and faster in recognizing disgusted expressions in both the pleasant and unpleasant odor conditions and slower at recognizing happy faces compared with the neutral condition (Seubert, Kellermann, et al., 2010); in the second study, the results were similar but this time only present for disgusted faces (Seubert, Loughead, et al., 2010). At onset, these incon-sistent results are hard to consolidate, one reason might be that in a simple task such as emotion recognition, performance is already at ceiling level and if odor effects generally are fairly small there is not much room for improve-ment. The results in Leleu, Demily, et al. (2015) hints at this possibility. In this study emotional expressions were morphed with neutral faces to show various levels of emotional intensity. The results showed that facial expres-sions were recognized at lower intensities in congruent odor conditions (e.g. happy faces in the pleasant odor condition), for happy, disgusted and angry facial expressions (Leleu, Demily, et al., 2015). Another clue to these incon-sistent results might be found in a study by Damjanovic, Wilkinson, and Lloyd (2018) in which the authors found that happy faces were recognized faster initially, however, this effect was diminished at the end of the task. Here dif-ferences in experiment length might explain the mixed findings, perhaps time-on-task effects might conceal odor effects in facial emotion recognition. I will return to this issue in Study 3 of this thesis. Many of these reviewed studies have also measured how valenced odors affect subjective properties of faces. Two studies have shown that neutral faces are rated as more unpleasant in the context of an unpleasant odor (Cook et al., 2015; Cook et al., 2018), and more pleasant during a pleasant odor context (Cook et al., 2015), however, one study did not find any effect of pleasant

27

odors on classifying neutral faces as either pleasant or unpleasant (Bensafi, Pierson, et al., 2002). Odor effects have also been found for valenced expres-sions; faces displaying happiness were rated as more pleasant during exposure to a pleasant odor and, vice versa, disgusted faces as more unpleasant during exposure to unpleasant odor (Cook et al., 2017). Odors may also affect facial attractiveness; in one study faces were rated as younger and more attractive in a pleasant odor context (Seubert, Gregory, Chamberland, Dessirier, & Lundstrom, 2014) and another study found that faces were rated as less attrac-tive in the presence of an unpleasant odor (Dematte, Osterbauer, & Spence, 2007). Finally, one study indicated that subthreshold odors may affect facial evaluation; Li, Moallem, Paller, and Gottfried (2007) found that participants who were unaware of the odor rated neutral faces as less likeable after expo-sure to an unpleasant odor.

Body odor effects There is a great heterogeneity in experimental design in the BO studies. For emotion recognition, one study showed that recognition in dynamic faces was more accurate during exposure to anxiety induced BO (Rocha et al., 2018), one study showed that own faces were identified faster when exposed to own body odor (Platek et al., 2004) and in one study the findings indicated that there was less interference in men when they categorized angry faces during androstenone exposure in an emotional Stroop-task (Hornung et al., 2017), however, this effect was not present in a similar experiment with a larger sam-ple (Hornung et al., 2018). For ambiguous faces expressing different levels of happiness and fear in anxiety-induced BO, more faces were categorized as fearful (Zhou & Chen, 2009), and less happy (Zernecke et al., 2011), notably these effects were the largest for the most ambiguous expressions of fear and happiness. Body odor effects on facial valence showed that neutral faces were rated as more pleasant when preceded by a happy face during sport-induced BO, for anxiety-induced BO this effect was not present in females (Pause et al., 2004), faces were also rated as more negative during anxiety BO (Adolph, Meister, & Pause, 2013) and less liked in sickness-induced BO (Regenbogen et al., 2017). One study showed that faces were rated as more arousing during both sport- and anxiety-induced BO (Adolph et al., 2013).

Olfactory conditioning In conditioning studies, an unpleasant odor (UCS; unconditioned stimuli) is usually associated with a face (CS+) by presenting these concurrently; faces that are not associated with the UCS (CS-) are used as a control. Some studies have used odors as a contextual cue and use startle sounds (loud sudden noises) as UCS (Hermann, Ziegler, Birbaumer, & Flor, 2000). The idea is that

28

faces that are associated with an unpleasant odors CS+, differ in brain activity and behavioral measures compared with the CS- faces. Two studies have shown temporal effects in a gender recognition task; CS+ faces associated with an unpleasant UCS were initially faster in the beginning of the task (Gottfried & Dolan, 2004), however, in another study (Gottfried, O'Doherty, & Dolan, 2002), this effect was reversed, with slower responses to the CS+ faces conditioned with pleasant and unpleasant UCS odors. Several studies have shown that faces conditioned with an unpleasant odor are evaluated more negatively (Hermann et al., 2000; Steinberg et al., 2013; Steinberg et al., 2012). In addition, Steinberg et al. (2013) found that faces conditioned with pleasant odors were preferred over faces paired with clean air and an unpleas-ant odor. One study has shown that faces paired with pleasant and unpleasant odors are rated as more arousing (Hermann et al., 2000). Considering the results in these olfactory conditioning studies. The results for RTs do not show a clear pattern. For ratings of valence the results are stronger, although for arousal only one study collected ratings, the results indicate that both pleasant and unpleasant odors might affect facial evaluation. Notable in the reviewed studies is that odors seem to affect associative learning for faces when used in classical conditioning paradigms. This capacity is highlighted by Steinberg et al. (2012). In this study, the authors showed that in a stimulus set of 208 faces, only two pairings of the CS+ (neutral faces) with the unpleas-ant UCS odor was enough to affect facial evaluations negatively. We know from previous work that animals have the capacity to associate aversive odors with objects in a fast and flexible manner (Handler et al., 2019). These find-ings show a similar remarkable capacity in humans to associate faces with aversive odors. These behavioral effects are complemented by results from psychophysiolog-ical measures. fMRI results indicate that the amygdala and orbitofrontal cor-tex, areas that are closely associated with olfaction are involved in olfactory associative learning (Gottfried & Dolan, 2004; Gottfried et al., 2002). MEG results indicate that activity in the prefrontal cortex to CS+ faces can be dif-ferentiated already 50-80 ms after stimulus onset (Steinberg et al., 2013; Steinberg et al., 2012). Lastly, CS+ faces to an unpleasant odor can affect fa-cial muscle activity (Hermann et al., 2000). In summary, consistent olfactory conditioning effects on liking and cortical responses have been found for face stimuli associated with odors.

Ambiguity in emotional expression A few of the reviewed studies investigated the effects of odors on neutral or ambiguous facial expressions. The results of studies using BO have shown

29

that overall, facial emotion is recognized more accurately (Rocha et al., 2018) and faces are perceived as less happy in anxiety-induced odor conditions (Zernecke et al., 2011). In Zernecke et al. (2011) this effect was largest in the most ambiguous facial expressions. Zhou and Chen (2009) found also that ambiguous fearful faces were categorized as more fearful in the fear-related BO condition. Similar results have been found for neutral faces. Neutral faces were rated as more pleasant after pleasant odor priming (Cook et al., 2015) and less pleasant after unpleasant odor priming (Cook et al., 2015; Cook et al., 2018). Thus, odor effects on facial perception might be more pronounced for neutral and ambiguous facial expressions.

Conclusions The reviewed studies span almost 2 decades of olfactory research on odor ef-fects on facial perception. Generally the results indicate that facial expressions are recognized faster in the context of odors. Apart from a few studies that show face-odor valence congruency effects, the most consistent finding is that valenced odors affect perceived face valence overall (e.g. faces are perceived as more unpleasant in an unpleasant odor condition). Odor effects on neutral and ambiguous facial expressions appear consistent. A plausible explanation is that recognition of facial expressions is such an easy task that there is not much room for improvement even when there is congruent olfactory infor-mation available. Notable in these studies is the lack of open research practices, such as trans-parent study protocols and open data. Considering that at least half of the stud-ies utilized small sample sizes and that odor effects typically are fairly small, we can expect that some of the findings in this area are false positives. Almost all studies found some odor effects on facial perception, if we assume that many studies in the field are underpowered, there might be a file drawer prob-lem of studies that were conducted but not published because of non-signifi-cant findings. I also think that research in this area could benefit from practices such as standardized measures, testing procedures and stimuli. Practices that have been highly successful for emotion research is the use of standardized stimuli, international affective picture set (IAPS; Lang, Bradley, & Cuthbert, 2008) and measurement of affective properties of stimuli using the self-as-sessment manikin (SAM; Bradley & Lang, 1994). Further, because of the small number of published studies in this line of research, open data would be beneficial for the whole community of researchers. Open data practices enable reinterpretation and creation of novel hypothesis using existing data, and lessen the amount of duplicated work. In the following empirical section of this thesis, I will try to resolve some of the inconsistent results in the literature and fill the gaps where there is no research.

30

31

Aims of the thesis

The aim of this thesis is to investigate how odors affect perception and pro-cessing of facial expressions of emotion. To achieve this, I have used multiple methods in three studies. Throughout these three studies, I have used the same odors (valeric acid and lilac), the first of these is an unpleasant odor and the second is a pleasant odor. In addition, in all studies I have used a no-odor control condition. To assess face and odor integration, I have used facial ex-pressions that are congruent with these odors (i.e. disgusted faces with the unpleasant odor and happy faces with the pleasant odor), conditions that can be compared with incongruent conditions or no-odor control. In the studies, in statistical terms, I have investigated both main effects of odors on facial perception and interaction effects. Conceptually, main effects can be described as that the manipulation affects a measure similarly across conditions. For example, arousal effects would be present if faces regardless of facial expression are rated as more arousing in odor conditions relative to the no-odor condition. In a similar vein, odor valence effects would manifest in that faces in general were rated as more valenced in the direction of the odor (e.g., more negative in the unpleasant odor condition). Interactions on the other hand can be described as an odor having different effects depending on the facial expression. Thus, I have defined signs of face-odor integration, in statistical terms, as the interaction between facial expressions and odors. Most relevant of these are congruency effects between valenced odors and facial expressions. For example, a disgusted facial expression might be rated as more negative in an unpleasant odor context versus a pleasant odor, and vice versa for happy facial expressions, a face might be rated as more positive in a pleas-ant odor condition versus the unpleasant odor condition. The following ques-tions will be addressed in the thesis:

1. Do affective odors influence how faces are perceived? 2. Do odors affect how faces are processed in the brain? 3. Do congruent odors and facial expressions influence the amount of

information needed to recognize happy and disgusted faces? 4. Do congruent odors and facial expressions influence attentional pro-

cesses?

32

33

Methods

General methods The participants in my studies have consisted of young and middle-aged adults, predominantly university students. The participants have been re-cruited trough bulletin board ads, e-mails to student lists and an online recruit-ment panel at www.studentkaninen.se. I have compensated participants by course credit or movie vouchers. I have used a blocked design in the studies included in this thesis. In general, the participants have performed tasks during a period of exposure to valenced odors. In total, the duration of each block was 5-10 minutes. In each block, the participants rated the odors, performed the main experimental task involving faces while exposed to the odor, and at the end, rated the faces that were shown during the task.

Ethics All studies in this thesis received approval from the regional ethics board 2014/2129-31/2 (www.epn.se/stockholm). We followed the ethical principles as stated in the Declaration of Helsinki. All participants were fully informed about potential risks and procedures and gave written informed consent prior to participating. Some ethical aspects of this research are discussed in the gen-eral discussion.

Open science In recent years, there has been a move in psychology toward more open prac-tices. This move has been motivated because of the low reproducibility of psychological findings (Open Science Collaboration, 2015). The main objec-tive of this open science movement has been to reduce practices that are harm-ful to science. In general, the move has been toward more transparent prac-tices, such as open study protocols, sampling plans and pre-registered hypoth-eses to reduce hypothesizing after results are known (HARKing; Kerr, 1998). In research there are many incentives for using bad practices, such as

34

difficulties to publish null-results and that many metrics used to evaluate re-searchers, premiere number of published articles. Some might interpret these incentives as something that induces dishonesty. However, the scientific pro-cess contains several steps that might insert bias without the need for dishon-esty. Typically, there is a considerable time that passes from an idea until the data is collected and analyzed. Thus, when the results are obtained the exact hypotheses could be fuzzy. In addition, data is often messy and need consid-erable pre-processing. How one preprocesses the data may influence the re-sults considerably. Further, in the publication process, reviewers may ask for dropping of some analyses or additional analyses that might influence type-1 error rates. Pre-registration shifts the process and puts more emphasis in which order you do things and makes the researchers to think more about the various hypotheses and decisions before data collection. This should lead to better de-signed studies and removes the doubt what the original plan was at the time of the idea. Pre-registered studies also provide evidence for the severity of a test and increases trust in the obtained results (Allen & Mehler, 2019; Lakens, 2019, November 18). During my PhD project, I have become more aware of the problems that are present in research and tried to adjust my research to-ward more transparent practices. Thus, in this thesis the first study was not pre-registered, but we have openly shared the data. Study two and three, how-ever, were both pre-registered, in addition to all data and analysis scripts being freely available.

Statistics In study 1 I used classical NHST (null hypothesis significance testing) statis-tical testing. In studies 2 and 3, I have used Bayesian statistics. In experimental research, ANOVAs have long been the standard method to perform statistical hypothesis tests and have greatly influenced the design of experiments. A large portion of my undergraduate and graduate statistics and methods classes focused on ANOVAs. So much that it influenced the research design in my studies; how do we answer this research question with an ANOVA? However, ANOVAs are difficult to interpret; they only provide sig-nificance of the whole model, but not the direction of the effects. In most cases this leads to many post-hoc tests using t-tests, which in turn leads to a loss of control of the alpha level and may result in false positives. Another problem with ANOVAs is that analysis is performed on aggregated data, leading to a loss of information that resides in how much variation there is within an indi-vidual and the stimuli (Judd, Westfall, & Kenny, 2012). In summary, in re-peated measures ANOVA, only participant is considered as a random factor. However, all the stimuli that are used in experiments are a sample of all the possible stimuli; thus, these should also be considered as a random factor. One

35

solution to this problem is Linear Mixed Models (LMMs). In LMMs both par-ticipants and stimuli are treated as random factors. This leads to a better esti-mate of the sources of variance in the data. Statistical power can be increased by considering the source of variance. Large variance due to stimuli can easily be remedied by increasing the number of stimuli, however, if large variance is due to participant, then adding more participants is instead the solution that will increase power (Judd et al., 2012). My move from NHST to Bayesian statistics was motivated in part by interest in Linear Mixed Models. While learning to use LMMs, I played around with it and noticed that I would often get a lot of significant results. In my studies, I have used fairly large sample sizes and had many trials for each condition. With lots of data for estimation, as is the case in my studies, even small effects can be statistically significant. This raises the question, which effects are im-portant? In my search for a solution, I found that Bayesian statistics were more conservative and acted as a filter to weed out spurious effects.

Stimuli In all 3 studies in this thesis I have used the same 3 odor conditions, valeric acid (a sweat- or cheese-like odor), no-odor and lilac essence. In study 1 I used psychophysiological methods to determine a moderately intensive odor con-centration for each participant. In studies 2 and 3, I used the mean concentra-tions determined in study 1 for all participants to achieve a moderate odor intensity. In study 1 I established that the perceptual properties of the two odors that have been used throughout this thesis. In this study I calculated odor concen-trations that matched a moderate odor intensity, and ensured that the odors were “mirrored” in the valence dimension, such that the unpleasant odor was rated as negatively as the pleasant odor was rated positively. The balanced valence and arousal properties were confirmed in the following studies. The idea behind a moderate intensity level was to lessen participant discom-fort, less habituation (strong odors initially habituate fast) and equate the odors so that they are matched in intensity and valence. A common finding in emo-tion research using valenced picture stimuli is that valence and arousal are asymmetrically correlated. Between negatively and positively valenced stim-uli, as valence increases, arousal or intensity typically also increases. Neu-trally valenced stimuli are typically low in arousal. Valence and arousal in pictures are highly dependent of the semantic content (Anderson et al., 2003). However, odors are not affected by this type of confound because odor va-lence may be manipulated by varying odor concentration. Previous research

36

has shown that for many odors, an increased odor concentration is associated with increasing intensity and valence (Doty, 1975). In study 1, the participants rated the odors on intensity and valence in three different concentrations (1%, 40% and 80%). Figure 2 displays the relationship between rated intensity and valence for the two odors. The results indicate that for the pleasant odor, pos-itive valence increases with higher intensity, and for the unpleasant odor, neg-ative valence increases with increased intensity.

Figure 2. Rated odor intensity and valence for three different concentrations of odors used in my studies, valeric acid (unpleasant) and lilac essence (pleasant). Black lines show predicted linear relation between intensity and valence. Uncer-tainty of the linear effect (colored lines) is visualized with a sample of draws from the posterior distribution.

We also asked participants to match the odors to different facial expressions, using an independent sample; these results were published as a supplementary material in study 3. Briefly, we found that valeric acid was most strongly as-sociated with a disgusted expression, and lilac was most strongly associated with a happy expression. These results are also in concordance to verbal re-ports of joy and disgust respectively for pleasant and unpleasant odors (Bensafi, Rouby, et al., 2002). This observation supports our notion of odor-face congruency.

37

For facial stimuli, I used two freely available databases containing actors ex-pressing various emotions (Ebner, Riediger, & Lindenberger, 2010; Lundqvist, Flykt, & Öhman, 1998). In study 1, I used the FACES database (Ebner et al., 2010). This database contains normatively rated picture stimuli from young, middle-aged and old actors. From this database, I selected 24 actors (6 young women, 6 young men, 6 elderly women and 6 elderly men) expressing three facial expressions (disgust, neutral and happy). In total, the stimulus set consisted of 72 unique pictures. The pictures were selected such that the emotions expressed by each actor were recognized by more than 80% of the participants in a normative sample (Ebner et al., 2010). These pictures were then realigned so that the eyes of the actors conformed to a common position. In study 2 and 3, I used the KDEF database (Lundqvist et al., 1998). A good property of the KDEF database is that the images require less manual pre-processing to be useful.

Ratings In all of my studies I was interested in whether the odors are perceived as expected in terms of valence and arousal (e.g. the unpleasant odor is rated as unpleasant and arousing when compared to the no-odor condition). I have also been interested in how odors affect perception of faces; do the odors affect how the faces are perceived and processed? We used a CR100 scale for odor intensity ratings in study 1 (Borg & Borg, 2002). This category ratio scale has been validated as an appropriate scale for magnitude estimation. We used linear interpolation to estimate an equal odor concentration. We used SAM-scales in study 1 and 3, this has been extensively validated as an appropriate scale for measuring affective properties of visual stimuli (Bradley & Lang, 1994). For practical reasons, we used a VAS-scale in study 2. Previous research has shown that valence and arousal are appropri-ate dimensions for measuring odor properties (Bensafi, 2002).

RTs The idea behind reaction times (RTs), also called mental chronometry, is that we can infer how the brain operates by carefully designing experiments and measuring how long it takes to respond to a stimulus (Meyer, Osman, Irwin, & Yantis, 1988). If we are faster at processing a specific stimulus, then it is because we employ more processing or do it more efficiently. If we can find this type of time benefit for a specific stimulus (or condition), when everything else is equal,

38

then it indicates that there is something inherently special with that type of stimuli or condition, that it is important enough that there are dedicated path-ways for processing it or that more energy is expended in its processing.

EEG Although we have learned a great deal of how the brain works using carefully planned experiments with reaction times, there is a direct way to measure how the brain processes stimuli. Electroencephalography (EEG) is a method of measuring the electrical activity of the brain. By attaching electrodes on the scalp, the neuronal activity can be recorded. When recording EEG in experi-mental situations a trigger that is synchronized with stimuli is recorded along-side the EEG. From the EEG we can calculate event-related potentials (ERPs), by averaging the EEG activity time-locked to stimuli. This method summates the neuronal activity of a large number of cortical pyramidal cells that are oriented in the same direction, while averaging out random activity and activ-ity not associated with the specific stimuli. Previous research from the last eight decades have found ERP components for various types of stimuli and processes and defined their temporal and spatial properties (Luck, 2012). Thus, ERPs are suitable for measuring the time-locked brain activity to spe-cific stimuli, and this can be measured with great accuracy regarding the tim-ing of events. However, a drawback of the ERP methods is a poor spatial res-olution. We can know when cortical activity happens, but have little infor-mation on where in the brain it happens. In my research, I have followed the recommendations in publication guide-lines and recommendations for studies using EEG (Keil et al., 2014). Recently there has been criticism against the common use of identifying ERP compo-nents from the topographies and ERP-waves. These results may inflate the researchers’ degrees of freedom. Luck and Gaspelin (2017) recommends sev-eral different methods to combat these problems. A method that they recom-mend and that I used in study 1 is the grand averaged ERP-waves and topog-raphies to define ERP-components. Another way is to use more data-driven approaches such as non-parametric statistical methods to mitigate false posi-tives. I agree with these recommendations, however, I am also worried that data-driven practices can make it harder to compare results between studies. I have noticed that research that uses these types of data-driven methods find effects in time-points and scalp locations that do not match previously found effects. This makes the effects harder to understand because we often know a lot of the properties of standard ERP-components that have been studied in hundreds or even thousands of different studies. Findings that are data-driven might reflect specific experimental situations or a specific sample of partici-pants.

39

Summary of studies

Study 1 Syrjänen, E., Wiens, S., Fischer, H., Zakrzewska, M., Wartel, A., Larsson, M., & Olofsson, J. K. (2018). Background Odors Modulate N170 ERP Compo-nent and Perception of Emotional Facial Stimuli. Frontiers in Psychology, 9, 1000. doi: 10.3389/fpsyg.2018.01000

Aim The aim in this study was to investigate how affective background odors in-fluence perception of faces expressing disgusting, happy or neutral emotions. We measured how affective odors and faces are integrated, using ratings of valence and arousal in faces, and cortical activity using EEG.

Background Previous research on odor effects on facial processing has shown effects that might be attributed to arousal such that disgusted facial expressions are rec-ognized faster regardless of odor (Seubert, Kellermann, et al., 2010). Some studies have also shown that odors might enhance recognition RTs when odor and facial expressions are congruent (Leppanen & Hietanen, 2003). Only a few previous studies have researched the effects of odors on facial processing using ERPs. Event-related potentials is a very useful method for investigating the time course of different brain processes (Olofsson et al., 2008). Early and mid-la-tency ERPs such as the P1 and EPN are thought to capture automatic atten-tional processes for affective stimuli (Olofsson et al., 2008; Schupp, Flaisch, Stockburger, & Junghöfer, 2006). Late latency ERPs such as the LPP are thought to reflect allocation of attentional processes to affective stimuli (Hartigan & Richards, 2016). In the early ERP-components, a good candidate for odor effects is the face-sensitive N170 ERP-component (Rossion, 2014). One previous study has shown main effects of odors on the inverse of the N170 component (VPP; Rossion & Jacques, 2012). In this study, Leleu, Godard, et al. (2015) found that pleasant odors increased the amplitudes in the VPP regardless of facial expression and enhanced amplitudes at right temporal location to unpleasant odors. In the mid-latency components (200-300 ms),

40

the N2/EPN has both been associated with main effects and interactions be-tween odor and facial expression. For main effects, Leleu, Godard, et al. (2015) found reduced P2 (interpreted as an EPN) amplitudes to happy and disgusted faces in an unpleasant odor context. As for interactions, Cook et al. (2017) found stronger activation in the N2 to disgusted faces relative to happy in the unpleasant odor condition; this effect was reversed in the neutral odor condition. In a similar manner compared to the behavior effects and early-mid latency ERP-components, studies have found main and interaction effects for odor and facial expressions in later ERP-components. Cook et al. (2017) found larger amplitudes for disgusted faces in the pleasant odor condition and vice versa in the pleasant odor condition, larger amplitudes for happy faces in the unpleasant odor condition, suggesting interaction between congruent and in-congruent odor and facial expressions. However, Leleu, Godard, et al. (2015), found only enhanced LPPs to disgusted faces. Based on previous research, for the P1, we hypothesized only arousing effects of the odors (i.e., higher ERP amplitude in the odor vs. no-odor condition). For the face-selective components such as the N170 and VPP, we hypothe-sized that odors would lead to increased amplitudes regardless of face valence. Although no previous studies found odor congruency effects on the N170, we hypothesized effects in the N170 time range because of findings using odor conditioning and in studies using visual primes, and considering the close con-nection between olfactory and visual areas. We hypothesized that odor and emotional expression would interact at the P2 component. For the LPP we hypothesized only main effects of odor.

Methods We tested 60 participants (of which 2 were excluded due to technical prob-lems) in the lab. Participants were screened for olfactory and uncorrected vis-ual deficits, psychiatric and neurological disorders. We collected demographic and various personality measures. Means and SDs for the study variables are reported in Table 1. Participants signed an informed consent form and the study design was approved by the regional ethics board (2014/2129-31/2).

41

In short, the study consisted of two main parts. First, participants performed a psychophysical rating session of a pleasant (lilac essence) and unpleasant (va-leric acid) odor to ensure moderate odor intensity. Second, in a blocked design with these contextual odors and a neutral baseline condition, participants per-formed a simple 1-back task while watching faces displaying happiness, neu-trality and disgust. During this task we recorded the brain activity of the par-ticipants with EGG. At the end of each block participants also rated the va-lence and arousal of each picture that was displayed in the preceding block, for a general overview of the trial structure see Figure 3.

Table 1.

Demographics, personality measures and odor concentrations (N = 58). Three domain disgust scale (TDDS). Chemical sensitivity scale (CSS).

Mean SD

Age 25.57 5.76

Gender 57% female (n = 33)

Handedness 88.46% right-handed

Years in school 15.36 2.08

Positive affect 29.87 5.92

Negative affect 13.89 4.47

TDDS 26.21 9.02

CSS 32.29 7.37

Valeric acid concentration 33.66 26.43

Lilac concentration 29.72 22.46

Valeric valence -2.59 2.57

Lilac valence 3.95 2.43

42

Figure 3. Top row depicts the main block with the first two trials illustrating the 1-back task. The bottom row depicts the picture rating procedure with valence rating at the top and arousal at the bottom.

The ERP part of the experiment was divided in six blocks where each of the three odor conditions (pleasant, no-odor and unpleasant) was repeated. Odor order was randomized with the constraint that the same odor condition was not repeated immediately after the preceding block. To maintain participants’ attention to the faces the task was to report if a picture was repeated (1 in 12 pictures) from the previous trial (1-back task). At the end of each block, par-ticipants rated the valence and arousal (SAM; Lang et al., 2008) of each pic-ture in the preceding block. At the end of each block odors were removed and swapped for the following block. Average duration of each block was around 5 minutes; total EEG testing time was around 30 minutes. In the 1-back task, the mean hit rate was 80% (SD = 16.34), and the mean false alarm rate was 2% (SD = 7.34), indicating that participants were attend-ing to the picture stimuli. For the ratings and ERPs we first performed 3x3 repeated ANOVAs to investigate the overall pattern of results. To reduce model complexity we then calculated differences scores (Figure 4 shows the difference score calculation procedure) between the neutral baseline condi-tions (neutral facial expressions and the no-odor condition) and the affective conditions (happy and disgusted facial expressions, and pleasant and unpleas-ant odor conditions).

43

Figure 4. Illustration of our difference-score calculation procedure. Based on the original 3*3 design (left panel), we first subtracted emotional conditions from neu-tral conditions (second from left) separately for each emotional facial expression (il-lustrated within the red box for happy - neutral). Then, odor conditions were simi-larly subtracted from the no-odor condition (third from left; green box illustrates the subtraction of pleasant from no-odor condition). This resulted in four conditions, two that were congruent (happy face + pleasant odor; disgusted face + unpleasant odor) and two that were incongruent (happy face + unpleasant odor; disgusted face + pleasant odor; rightmost panel).

We used MATLAB and custom FieldTrip scripts to preprocess the EEG data offline. We identified each relevant ERP-component by grand-averaging all participants’ data across conditions and visually inspected the time-course and topography of each ERP-component (using descriptions of these components from previous research). The P1 component was identified as positive deflec-tion in the ERP-waves (120-160 ms after stimulus onset) in occipital locations around the O1 and O2 electrodes. The N170 was evident as a negative deflec-tion (160-200 ms) in right temporal electrodes around the P8 electrode. We also tested the VPP (inverse of the N170) in the same time range as the N170 by re-referencing to mastoid electrodes and extracted means at central elec-trodes around Cz. The P2 was identified as an occipital positivity around elec-trodes O1 and O2 at 220-300 ms after stimulus onset. Lastly the LPP was identified as a sustained positivity in parietal-occipital electrodes around Pz at 300-500 ms after stimulus onset. For all components, we computed the mean amplitudes across the relevant electrodes and time range for each condition and participant. To investigate the effects of odor and expression on ERP com-ponents, we followed the same analysis strategy as for odor ratings, we per-formed 3 by 3 repeated measures ANOVAs and then computed difference scores between affective and neutral stimuli on which we performed 2 by 2 repeated measures ANOVAs. To reduce clutter, the figures for the P1, VPP, P2, and LPP is in the Supplementary Materials. To further the cumulative pro-gress of science and help in establishing reproducible results and aid in future meta-analyses we provide full experimental data at the following url: https://figshare.com/s/252745f56a0f7b7115fc (Syrjänen, Wiens, et al., 2017).

Results We first performed manipulation checks and confirmed that the unpleasant odor was rated as more negative than the pleasant odor. We followed up with

44

confirming that happy faces were rated as more positive than neutral faces and that disgusted faces were rated as more negative. Face ratings were also rated as more negative in the unpleasant odor condition, a valence effect that was not present for the pleasant odor condition (left panel in Figure 5). Overall, happy and disgusted faces were rated as more arousing compared to the neu-tral faces. Interestingly, the arousal ratings showed that happy faces were rated as less arousing in the unpleasant odor condition, whereas, arousal ratings for happy faces in the pleasant, and happy and disgusted faces in the unpleasant odor condition were similar (right panel in Figure 5).

Figure 5. The panels show the rated valence (left) and arousal (right) difference scores for pleasant and unpleasant odors when subtracted from the no-odor condi-tion, separately for happy, and disgusted facial expressions (means and 95% confi-dence intervals) for difference scores. The 95% CIs around the means were cor-rected as suggested by Morey (2008) to account for within subject designs.

For the ERP results the main finding was an interaction in the N170 compo-nent indicating that happy faces were unaffected by odor context, whereas, for disgusted faces there was a relative decrease of amplitude in the pleasant odor context and increase of amplitude in the unpleasant odor context (Figure 6). The only other statistically significant result was a main effect of facial ex-pression in the P2 component (happy > neutral > disgusted).

45

Figure 6. The top row shows the raw topography of grand average (160-200 ms from stimulus onset) N170 for each odor-face condition. The left panel shows grand average ERP waves (of right temporal sensors, highlighted in the top row topogra-phies) in each condition. The insert zooms in at the difference waves between emo-tional and neutral conditions in the N170 peak. The right panel shows mean ampli-tudes and 95% confidence intervals (corrected as per Morey (2008) to account for within subject designs) around the mean amplitude difference between emotional and neutral conditions.

Conclusions The main finding in study 1 is that we find evidence of odor and face integra-tion at an earlier EEG time point compared to previous studies. Specifically, the N170 was larger for disgusted faces in the incongruent pleasant odor con-dition, whereas there was a decrease in amplitude in the congruent unpleasant odor condition. These results are in line with the notion that for disgusted faces, a congruent odor may facilitate the processing of the faces while the processing of faces in incongruent odor contexts needs more attentional re-sources. Although Leleu, Godard, et al. (2015) only found effects of facial emotion in this time-range, one previous fMRI study (Seubert, Kellermann, et al., 2010), found hypoactivation to disgusted facial expressions in an unpleas-ant odor context in areas that are associated with the integration and pro-cessing of facial information. In relation to other previous ERP studies, we could not replicate some prior odor-visual interactions. Mainly, Leleu, Go-dard, et al. (2015), the study that closest resembles the present study, found that unpleasant odors increased the amplitude relative to baseline for disgusted facial expressions. Although the findings are still mixed, a problem here is that there might be too few studies to indicate a clear pattern of results.

Another main contribution of this study to the research field was the odor val-idation procedure, where we showed that valeric acid and lilac were rated as isointense at the same concentrations, and although there were individual

46

variations in the ratings, all participants were exposed to subjectively “mod-erate” odor intensities. We could also found odor effects on how people per-ceived the facial expressions. Mainly, participants rated faces as more nega-tively valenced in the unpleasant odor condition. Faces were also rated as overall more arousing in an odor context. This indicates that persons, when making a judgement about a face in front of them, typically factor in the back-ground odor in their judgement, even when this is not asked for. This effect might be due to visual-olfactory interactions in the fusiform face area of the brain, the brain region where the N170 is generated.

47

Study 2 Syrjänen, E., Liuzza, M. T., Fischer, H., & Olofsson, J. K. (2017). Do Va-lenced Odors and Trait Body Odor Disgust Affect Evaluation of Emotion in Dynamic Faces? Perception, doi: 10.1177/0301006617720831

Aim The results in study 1 suggested that cortical processing of disgusted facial expressions were modulated by the unpleasant background odor condition. We sought to see if we could understand this effect with new behavioral measures. We measured RTs for emotion recognition in dynamic faces, short clips, going from neutral expression to a happy or disgusted expression, while participants were exposed to a pleasant odor, unpleasant odor, or no (neutral) odor context (same as in study 1). We were also interested in whether individ-ual differences in body odor sensitivity would explain differences in facial emotion recognition in the unpleasant odor condition (Liuzza, Lindholm, et al., 2017).

Background Leppanen and Hietanen (2003) is an influential study in the topic of odor ef-fects on visual processing of facial emotion. They found that happy faces were recognized faster in a pleasant odor context, whereas disgusted faces were recognized faster in an unpleasant odor context. In a similar vein, a recent study by Leleu, Demily, et al. (2015) had the amount of emotional information in faces manipulated by morphing emotional expressions with neutral expres-sions to different degrees. In this study, the authors found that contextual odors reduced the amount of emotional information needed to recognize the emotion if the odor was congruent with the facial expression. However, some studies have instead reported more general effects suggesting that the arousing prop-erties of odors might enhance recognition of facial expressions (Cook et al., 2015; Seubert, Kellermann, et al., 2010). The motivation for this study was to extend Leleu, Demily, et al. (2015), by using dynamic facial expressions that changes from neutral to emotional and that better resemble facial expressions we encounter in everyday life. If congruent odors affect the amount of emo-tional information required to recognize a facial expression, then participants should need less time to watch the dynamic faces and be faster in recognizing the displayed emotion.

As the results in previous research show both general and congruency effects of odors on emotion recognition, we investigated whether individual differ-ences in body odor disgust could explain some of these mixed results. To this

48

end, we used a newly developed body odor disgust scale (BODS; Liuzza, Lindholm, et al., 2017).

As the study design was novel, we pre-registered seven different hypotheses we wanted to test. Mainly, we were interested in main effects of odors and interactions between odors and facial expressions. We were also interested in interactions between the unpleasant odor and BODS score.

Methods In this pre-registered study we used a Sequential Bayes Factor design (Schonbrodt, Wagenmakers, Zehetleitner, & Perugini, 2015). We performed power calculations for a medium effect size (r = .3), and defined an N of at least 20 participants and a maximum of 70, after obtaining the required amount of participants we sought conclusive evidence for either the null or alternative hypothesis. For the null we defined BF > 3 (indicating a moderate level of evidence) and for the alternative we defined BF > 6 (indicating strong evidence for an effect) as a limit. The final sample consisted of 21 adult par-ticipants (eight male, mean age 31.86 years, SD = 10.93 years). We created a stimulus set of 16, 3-second long videos where a neutral face morphed to either a happy (8 videos) or a disgusted face (8 videos). We used a similar blocked design as in study 1; participants performed an emotion recognition task in three different odor conditions. We used the same delivery method and odors as in study 1, valeric acid, no-odor control and lilac essence diluted to concentrations that matched a moderate intensity. Each block started with valence ratings of the odor. This was followed by the emotion recognition task illustrated in Figure 7. At the end of each block par-ticipants rated the odor valence again and then rated how arousing and va-lenced the start (neutral) and end point (happy or disgusted) of each facial expression was.

Figure 7. Pictures of neutral faces morphed to faces expressing 50% of either happy or a disgust. Duration of each movie clip was 3 seconds. The task of the participants was to report the facial expression on a keyboard.

49

We used Bayesian linear models in the BayesFactor package to analyze the data, using actor and participant as random factors. Mainly we were interested in main effects of odor and odor visual interactions. We also performed anal-yses using BODS score as an interaction term.

Results Overall, the results indicated that the odors differed in valence as expected. Performing the pre-registered analysis plan, we found positive to strong evi-dence for no odor effects in emotion recognition speed, indicating odors did not affect speed of recognition. Performing exploratory analyses we found weak evidence for a general decrease in RTs for recognizing emotional ex-pressions (cf. Figure 8). We also found evidence that, regardless of odor con-dition, happy faces were both recognized more accurately and faster than dis-gusted facial expressions.

Figure 8. Credible intervals (CIs) around the median, separately for main effects of expression (left side), and odor (right side) on reaction times. Note that the medians and CIs for RTs reflect the difference from grand mean (indicated by the gray dashed line) for each condition.

50

We found strong evidence for facial expressions affecting valence ratings. There were no interactions between facial expressions and odors, and the re-sults also indicated positive to strong evidence for no odor effects on rated face valence illustrated in Figure 9.

Figure 9. Median face valence ratings with 95% credible intervals. Note that the er-ror bars are obscured by the dots. The left panel shows the ratings for each expres-sion. The right panel shows the effect of odors across facial expressions.

We found strong evidence for facial expressions affecting rated arousal. There were no interactions between facial expressions and odors, and the results also indicated positive to strong evidence for no odor effects on rated face arousal shown in Figure 10.

Figure 10. Median face valence ratings with 95% credible intervals. Note that the error bars are obscured by the dots. The left panel shows the ratings for each ex-pression. The right panel shows the effect of odors across facial expressions.

51

We also found positive to strong evidence for no interactions between BODS and unpleasant odors on odor or face rating s and RTs. Ratings of odor valence at beginning and end of each block indicated no habituation effects for the odors. Further, these effects did not interact with individual odor valence rat-ings.

Conclusions The main result in this study was that we found positive to strong evidence for no odor effects in the emotion recognition task involving dynamic face stimuli that change from a neutral to an emotional expression. This is the first study investigating odor effects on dynamic facial stimuli. The research on static facial stimuli show mixed results regarding odor effects on emotion recogni-tion, some studies show general odor effects (Leleu, Demily, et al., 2015; Seubert, Kellermann, et al., 2010; Seubert, Loughead, et al., 2010), and a few studies show odor-facial expression congruency effects (Adolph et al., 2013; Leppanen & Hietanen, 2003). Our study therefore suggests that perhaps re-sults from experiments with static face stimuli might not easily generalize to situations where expressions are dynamically changing. More research on this topic is needed. The discrepancy in results between Study 2 and previous work on static faces might be explained by task difficulty. Previously it has been argued that dy-namic facial expressions are easier to recognize (Krumhuber, Kappas, & Manstead, 2013). However, although accuracy was generally high, there was plenty of individual variation in accuracy. Also, happy faces were recognized more accurately and faster than disgusted faces. Thus theoretically, there was room for odors to affect accuracy and recognition speed. We could also see that happy faces were recognized faster than disgusted facial expressions, rep-licating previous findings in static faces (Palermo & Coltheart, 2004), further supporting that emotion recognition in dynamic and static faces might not dif-fer substantially, at least for some emotional expressions. The lack of odor effects in the present study on emotion recognition could be explained by individual variation in body odor disgust sensitivity. However, we found no interactions in any of our measures between BODS score and odors. Although we did not use actual body odors in the present sample, at least valeric acid has been indicated as an endogenous body odor compound (Doty, 1981; Shirasu & Touhara, 2011) and has been used extensively in re-search as a body odor related smell (Anderson et al., 2003). Although we did not measure disgust sensitivity in general, the BODS scale has been validated against existing disgust scales and show good construct validity (Liuzza, Lindholm, et al., 2017; Liuzza, Olofsson, Sabiniewicz, & Sorokowska, 2017).

52

We also found that participants did not rate the facial expressions as more valenced or arousing due to the odors, as has been seen in some previous stud-ies (Adolph et al., 2013; Cook et al., 2015; Syrjänen et al., 2018). Although the sample size was adequate to indicate decisive statistical conclusions, it was still on the small side. Thus, lack of odor effects might be due to an unrepre-sentative sample. Future research should follow up on our exploratory finding that there was weak supporting evidence that the participants were generally faster at recog-nizing faces in the unpleasant odor condition. We interpret these results in the light of the BIS (Behavioral Immune System) framework. In this view an un-pleasant odor might be considered a pathogen cue (Tybur, Lieberman, Kurzban, & DeScioli, 2013), which leads to a vigilance where attention is di-rected to visual cues in the environment (Adolph et al., 2013; Steinberg et al., 2012).

53

Study 3 Syrjänen, E., Fischer, H., & Olofsson, J. K. (2019). Background odors affect behavior in a dot-probe task with emotionally expressive faces. Physiology and Behavior. doi: 10.1016/j.physbeh.2019.05.001

Aim In this study we were interested in the effect of valenced odors on attentional processes. Using the emotional dot-probe task we collected RTs to probes re-placing happy or disgusted faces on either side of the screen, while odors were presented similarly as in previous studies. If participants were attending to odor-face congruent sides more than the opposite side that would indicate that integration of odor and visual information and can affect attentional processes.

Background Results in previous research have been mixed whether odors might affect the recognition of facial expressions. Findings have generally suggested two dif-ferent effects. First, it has been suggested that the arousing properties might enhance response speed in behavioral tasks (Millot et al., 2002; Seubert, Kellermann, et al., 2010). Second, some studies have shown face and odor congruency effects (Leppanen & Hietanen, 2003), indicating that odors might facilitate the processing of faces based on whether they correspond to a similar emotion. Here, a hypothesis is that odors might affect attentional processes and direct attentional resources toward congruent facial expressions. These types of effects have been seen for odors and objects (Aguillon-Hernandez et al., 2015). Besides averaging trials within each experimental block to obtain mean RT data, we tested whether effects of odors and odor-face interactions occurred as a function of time-on-task within each block. We designed a dot-probe task where we could test the hypotheses of odor-related arousal effects and valence congruency effects. In the dot-probe para-digm, visual stimuli are presented on either side of the screen. After a while these pictures are removed and in place of one of these pictures a dot appears. Participants report the side of this “dot-probe”. The idea is that if participants are faster at reporting sides that contained emotional stimuli, that indicates that they were directing spatial attention to this side. In the present study, if odors are more arousing, then participants would be faster overall in responding to the probes when an odor is present compared to when the odor is absent. On the other hand, if odors direct attention towards valence-congruent faces, then participants should be faster at responding to the probes when emotional faces are congruent with the odor context. In this study, we wanted to test four hy-potheses, defined in a pre-registered analysis plan. First, if the arousing prop-erties of odors compared to the no-odor condition would decrease RTs to the dot-probes. Second, whether unpleasant odors specifically would lead to faster

54

RTs. Third, if odors would affect spatial attention when odor and face stimuli are emotionally congruent. Fourth, based on results in an independent pilot study we were interested in whether RTs would be modulated by time-on-task in a general, or odor-specific, manner.

Methods In this pre-registered study, we tested 40 participants in the lab (22 female, mean age 26.2 SD = 6.82). We used the same type of blocked experimental design and odor delivery method as in study 1 and 2. We also used the same odor stimuli in a moderate intensity, pleasant (lilac), neutral (no-odor), and unpleasant (valeric acid), as in study 2. Similarly, facial stimuli consisted of happy, neutral and disgusted facial expressions. The experiment was conducted in a well-ventilated room. The experiment was divided in three blocks, with contextual odors. The whole experiment con-sisted of four tasks, all repeated in each block. First, participants rated the odors on valence and arousal. Second, participants performed the dot-probe task. Third, participants rated the odors again on valence and arousal. Fourth, participants rated the valence and arousal of the faces during exposure to re-spective odor.

55

Specifically, the dot-probe task consists of showing a neutral face on one side of the fixation cross and an emotional (disgusted or happy) face on the other side. After the faces are removed, a dot is present on one side. Participants used the keyboard to indicate which side the dot appeared (overview of the dot-probe task and the rating tasks are displayed in Figure 11).

Figure 11. The upper panel describes the dot-probe task and the lower panel the rating tasks. Note that in the odor rating task, the words “rate the odor” were used instead of faces in the visual display.

We first ensured task compliance by checking task accuracy in the dot-probe task. The results indicated near ceiling performance 97.8% (SD = 2.52), and as we did not have any prior hypothesis on accuracy, we did not analyze the accuracy results further. We analyzed the data using linear models with the lmbf function in the BayesFactor package in R and performed model comparison using Bayes fac-tors. In each statistical test we considered actor and participant as random fac-tors. In all models, we performed main effects models and interaction models and compared these with a model with participant as the only factor.

56

Results We failed to find effects of facial emotion on the dot-probe task, as there was no benefit of emotional expressions in responding to the dot. There were also no overall effects of odor arousal or valence interactions in the dot-probe task (cf Figure 12).

Figure 12. Reaction time difference scores between emotional versus neutral probes (with 95% credible intervals). Note that positive values here indicate slower re-sponses to the emotional side vs neutral.

57

The main positive result in this study was instead a time-on-task interaction effect driven by the unpleasant odor (cf. Figure 13). Specifically, participants responded faster on the side of the probe as a function of time, whereas, the response speeds were relatively stable in the neutral and pleasant odor condi-tion within their respective block.

Figure 13. The interaction effect of odor and time-on-task on reaction times. The shaded areas around the fit-lines show the 95% credible intervals (CIs) computed from the posterior distribution.

58

Odor valence rating results were as predicted, unpleasant odor was more neg-ative than neutral, with the pleasant odor as most positive. Odors were also rated as similarly arousing, compared to the neutral odor (Figure 14). Neither odor valence nor arousal ratings decreased between the start and end of the blocks, suggesting habituation was likely not an issue in this experiment.

Figure 14. The left panel shows the median valence ratings and 95% credible inter-vals (CIs) computed from the posterior distribution. The panel on right hand side shows the median arousal ratings with 95% CIs.

Although the observed effects were small, we found that faces were rated as being more valenced in the direction of the odors (Figure 15).

Figure 15. The left panel shows the median valence ratings and 95% credible inter-vals (CIs) computed from the posterior distribution. The panel on right hand side shows the median arousal ratings with 95% CIs.

59

Arousal ratings also indicated that faces were more arousing in the odor con-ditions compared to the neutral control condition (Figure 16).

Figure 16. The panel on left hand side shows the mean arousal ratings for each ex-pression and 95% credible intervals (CIs) computed from the posterior distribution. The right panel shows the effect of odor on face arousal ratings across facial ex-pressions with 95% CIs (note that as the effects are small, we present the means as deviation scores from the grand mean).

Conclusions The main finding in this study was that response speed decreased as a function of time in the unpleasant odor condition. As both the unpleasant and pleasant were matched in valence and arousal, we believe there is something special with the unpleasant odor. Differing rates of habituation cannot either explain this effect. Although there was a small decrease in odor valence and arousal ratings during the dot-probe task, this effect was “non-significant”, and im-portantly, this decrease was similar for both odors. Similar time-on-task ef-fects of an unpleasant odor has been seen previously in a visual search task (Damjanovic et al., 2018). Thus, although seldom assessed, time-on-task ef-fects could explain discrepant findings in previous research. This puts empha-sis of careful planning of number of trials and task length in future research. As task performance was near ceiling levels in the present study, we speculate that a combination of learning and sustained vigilance in the unpleasant odor condition could explain these results, whereas, in the neutral and pleasant odor conditions increases in response times could be explained by fatigue or related effects.

60

The null findings in the dot-probe task requires further discussion. Recent de-velopments in the field have shown that this task in fact has low test-retest reliability (Kappenman, Farrens, Luck, & Proudfit, 2014). There is a concern that attention has already shifted somewhere else when the probe appears, and there is no difference in response speed between conditions. For healthy indi-viduals this type of fast attentional shift might only be possible to measure with EEG or similar methods (Brosch et al., 2011; Eimer & Kiss, 2007; Holmes, Bradley, Kragh Nielsen, & Mogg, 2009). In fact, the type of spatial attention required in this task might only be relevant in specific clinical pop-ulations. The unreliability of this specific experimental paradigm might thus make it unsuitable for measuring attentional biases, at least in unselected pop-ulations. Lastly, similar to study 1, we found small but reliable effects of odor valence and arousal on facial emotion ratings. Overall, faces were rated as more va-lenced in the odor contexts, faces were rated more negatively in the unpleasant odor condition and more positively in the pleasant odor condition. Faces were also rated as more arousing in odor conditions compared to the neutral condi-tion. This indicates that background odors have an impact on the perceived arousal and valence of faces, even if no clear-cut attentional effects emerged.

61

Aggregated data: Odor effects on facial perception

Throughout my studies, I have asked the participants to rate the facial stimuli used in the experiments on valence and arousal in the context of valenced odors. By using the same expressions (happy, neutral and disgusted) and odors (pleasant, no-odor and unpleasant), it was possible to pool the data and per-form new analyses on the aggregated data. By aggregating the data, we can obtain a more precise estimate of the effect sizes, and estimate how robust the results were in the individual studies. Although these analyses are exploratory, the results may guide future research. I pre-processed the facial rating data in studies 1, 2 and 3 into a common for-mat and combined them. In addition, I included data from an unpublished manuscript in which I used the same facial stimuli and odors as in the previous studies (Syrjänen, Zakrzewska, & Olofsson, 2020). In total, the number of participants was 160 in this aggregated dataset. Because of the use of VAS-scales in study 2 that ranged from 1 to 100, I standardized the rating data to a mean of 0 and SD of 1 in all individual datasets. I used the BayesFactor pack-age in R to test five different models. First, an interaction model with the fac-tors odor and expression. Second, a main effects model with both odor and expression. Third and fourth, individual main effects model with either odor or expression as factors. Lastly, a participant-only model that served as the null model.

62

The aggregated analysis for valence effects showed that the two main effects model (odor + expression) received the most support (BF10 = 2.19^4736) fol-lowed by the expression only model (BF10 = 1.11^4726), the odor + expres-sion model was preferred over the expression-only model (BF10 = 19825126963). Figure 17 shows the main effects of facial expression and odors on valence ratings. The odor-only model received positive support (BF10

= 4.22). For the interaction model, the null received greatest support (BF01 = 19735).

Figure 17. The left panel shows median (with 95% credible intervals) valence rat-ings for happy, neutral and disgusted expressions. Note that the CIs in left panel are hidden behind the dots. Right panel shows median odor effects across facial expres-sions for the pleasant, no-odor and unpleasant odor contexts.

63

The analysis for arousal showed similar results as the results for valence. The two main effects model (odor + expression) received the most support (BF10 = 5.45^ 838) followed by the expression only model (BF10 = 4.81^ 833), the odor + expression model was preferred over the expression only model (BF10 = 113187). Figure 18 shows the main effects of facial expression and odors on arousal ratings. The odor-only model received strong support (BF10 = 2692). In contrast with the valence ratings, the interaction model was strongly preferred in comparison to the null model (BF10 = 1448).

Figure 18. The left panel shows median (with 95% credible intervals) arousal rat-ings for happy, neutral and disgusted expressions. Note that the CIs in left panel are obscured by the dots. Right panel shows median odor effects across facial expres-sions for the pleasant, no-odor and unpleasant odor contexts.

Taken together, the rating results paint the same picture as in the individual studies. The results indicated that facial expression has a large effect on the valence and arousal ratings in the predicted direction. Further, faces were rated as more valenced in the direction of the odor valence. In addition, faces were rated as more arousing in the presence of an odor compared to no odor. Alt-hough the odor effects are quite small, the large sample size (N = 160), and many ratings (~18000), results in a very precise estimate of the effect. Thus, odor effects reliably accounted for a small but significant proportion of the variance in the ratings. Further, the aggregated analysis also indicated that there was strong support for no interaction between odors and facial expres-sions for valence ratings. The results also indicated strong support for an in-teraction between odors and facial expressions for arousal ratings, however, as this model was less preferred than the main effects models, the results should be interpreted cautiously. This might indicate that there is variability between participants for arousal in the interaction between odors and facial expressions. In summary, the results provided strong evidence against

64

congruency effects of odors on facial expressions. The results suggest that odors affect ratings of facial valence and arousal regardless of facial expres-sion.

65

General summary

In this thesis, I have used a multiple methods approach to investigate how valenced odors affect facial perception. Specifically, I have used pleasant and unpleasant odors to investigate the influence of valenced odors on the pro-cessing and perception of disgusted, neutral, and happy facial expressions. In these studies (Syrjänen, Fischer, & Olofsson, 2019; Syrjänen, Liuzza, Fischer, & Olofsson, 2017; Syrjänen et al., 2018), I have tried to answer the following questions. How do odors affect the perception of faces? What are the neural correlates of facial processing during odor exposure? If valenced odors affect the amount of information needed to recognize congruent facial expressions. And lastly, if valenced odors affect attentional processes in face perception. These four main questions of the thesis are discussed below.

Do valenced odors influence how faces are perceived? In my studies, at the end of each block, I have asked the participants to rate the facial stimuli used in the preceding block, during the exposure to a va-lenced odor. The facial stimuli were rated on valence and arousal.

Valence effects on facial perception The results in studies 1 and 3 suggest that faces overall were rated in the di-rection of the odor valence; for an unpleasant odor, faces were rated more negatively and for a pleasant odor context they were rated as more positive. Aggregating the data across studies confirmed the results of the individual studies. In general, face valence ratings are affected in the direction of odor valence. Several previous studies have found similar effects that neutral facial expressions were rated as more negative in an unpleasant odor context (Cook et al., 2015; Cook et al., 2018), and in one study, faces were rated as more positive in a pleasant odor context (Cook et al., 2015). In a study by Cook et al. (2017), valence ratings for happy and disgusted faces were similarly af-fected by odor valence as in study 1 and 3. However, post-hoc tests in Cook et al. (2017) indicated that this effect was largest for congruent facial and odor combinations (e.g., disgusted faces were rated more unpleasant in the unpleas-ant odor condition). Other supporting evidence of odor effects on facial per-ception comes from studies that show that faces are rated as younger and more attractive in a pleasant odor context (Seubert et al., 2014) and less attractive

66

in an unpleasant odor context (Dematte et al., 2007). Thus, odors may affect how valenced faces are perceived.

Arousal effects on facial perception For facial arousal ratings, the results in studies 1 and 3 indicated that overall, faces were rated as more arousing in valenced odor contexts than in the no-odor contexts. In addition, in study 1, a significant interaction indicated that happy faces were rated as less arousing in the unpleasant odor context. The main effect results in studies 1 and 3 remained when aggregating the data. The results showed that faces overall were rated as more arousing in an odor con-text when compared to the no-odor condition. Only a few studies on odor face integration has measured arousal. One study using classical Pavlovian condi-tioning found that neutral faces paired with unpleasant and pleasant odors were rated as more arousing (Hermann et al., 2000). Further, in one study us-ing anxiety- and sport-induced body odors, the results indicated that partici-pants rated fearful faces as more arousing during such exposures than during a clean air exposure (Adolph et al., 2013). These results suggest that the arous-ing properties of odors on facial perception are undifferentiated of odor va-lence. However, more research is needed on these effects for conclusive evi-dence. In study 2, the results provided evidence for the null, for odor effects on both valence and arousal ratings of the faces. In study 2, the participants evaluated the faces using a VAS-scale, whereas in studies 1 and 3, the participants rated the faces on a SAM-scale where they are asked to report how they feel when they see the face. Although I have no empirical evidence, I wonder if this difference could have resulted in the null finding in Study 2. If the participants are asked to concentrate on their internal state when rating, odors might have a larger effect than when they are explicitly asked to rate visual features of faces. Previous research has described odors as inherently emotional (Yeshurun & Sobel, 2010). In one study that used the SAM-scale for affective ratings, researchers found that the affective properties of odors are harder to suppress than visual information (Adolph & Pause, 2012). However, to my knowledge, no one has previously compared how rating scale could affect the results in ratings. This might be an interesting avenue for future research.

Do odors affect how faces are processed in the brain? The results in study 1 suggest that odors might affect visual perception at early stages of facial processing. Specifically, we found that the N170 to disgusted facial expressions was smaller in the unpleasant odor condition than in the pleasant odor condition. We found no odor effects for happy faces in the N170 component. A smaller N170 in the presence of congruent odor and facial ex-pression might indicate facilitated processing, whereas more processing is

67

needed when odor and facial expression are incongruent, that is, when there are mixed threat and safety cues. We hypothesized that as disgusted expres-sions signal danger, these would receive preferential processing compared to happy facial expressions. In a typical survival scenario avoiding dangers that, in the worst case, may prove to be fatal is more important than missing a safety cue that might lead to an opportunity. Corroborating evidence for these find-ings comes from fMRI research where Seubert, Kellermann, et al. (2010) found hypoactivation in the fusiform face area (FFA) when participants saw disgusted facial expressions in the context of unpleasant odors. Previous re-search has shown that the source of the N170 components is the FFA (Joyce & Rossion, 2005), an area that recently has been indicated to be sensitive to cross-modal integration (Gerdes, Wieser, & Alpers, 2014; Park et al., 2010). These results might also be interpreted within the constructed emotion frame-work (Barrett, 2017b). In this view, differences in the N170 amplitude could be interpreted as reflecting predicting error. If unpleasant contextual odors provide valuable predictions for facial processing, then a smaller N170 could be interpreted as indicating that the disgusted facial expression was predicted correctly. However, in the pleasant odor condition, unpredicted disgusted fa-cial expressions would need enhanced processing. This might be an interesting avenue of research using specific methods that are more suitable to test this specific hypothesis. We also investigated other candidate components the P1, P2 and LPP that previous research has found to be sensitive to odor effects (Adolph et al., 2013; Leleu, Godard, et al., 2015; Rubin, Botanov, Hajcak, & Mujica-Parodi, 2012), or emotional facial processing (Trautmann-Lengsfeld, Dominguez-Borras, Escera, Herrmann, & Fehr, 2013). However, we could not find any statistically significant effects of odors on the P1, P2 or LPP. At the design phase of study 1, there was no previous research using ERPs. However, prior to the publication of study 1, a few similar studies were pub-lished, and since then, some additional studies have been published. One of these previous studies showed larger ERP amplitudes to faces in the P1 com-ponent in the context of body odors (Adolph et al., 2013). However, neither we in study 1 nor five other studies (Cook et al., 2015; Cook et al., 2017; Cook et al., 2018; Leleu, Godard, et al., 2015; Rubin et al., 2012) found support for this type of odor effect in the P1, in the context of unpleasant and pleasant odors or body odors. When it comes to the N170, in study 1, we found that odors affected the processing of disgusted facial expressions in an interactive way. Other research has found larger N170 amplitudes in the BO context (Adolph et al., 2013), and in frontal electrodes in pleasant and unpleasant odor context (Cook et al., 2017). On the other hand, three studies found no odor effects in the N170 component (Cook et al., 2015; Cook et al., 2018; Leleu, Godard, et al., 2015). For the closely related VPP, one study found larger am-plitudes in unpleasant and pleasant odor conditions to faces compared to neu-tral odor condition (Leleu, Godard, et al., 2015), a result that was not

68

replicated in study 1. Leleu, Godard, et al. (2015) also found weaker ampli-tudes in the P2 component for all expressions compared with neutral expres-sions in the unpleasant odor condition. However, no odor effects in the P2 component were observed in study 1, or in two other studies (Cook et al., 2017; Rubin et al., 2012). The N200, in the same time range as the P2, has also been implicated in odor effects on processing of facial expressions. Cook et al. (2017) found a more negative N200 for happy faces in the neutral odor condition and a more negative N200 for disgusted faces in the unpleasant odor condition. Cook et al. (2017) also found a more positive N400 for disgusted faces in neutral odor and a more positive N400 for happy faces in the unpleas-ant odor condition, but these effects were not replicated in two other similar studies by the same authors (Cook et al., 2015; Cook et al., 2018). Lastly, several studies have found odor effects to faces in the LPP component. Using body odors (Rubin et al., 2012) found that early LPP was enhanced in stress BO to neutral, ambiguous and angry facial expressions and that the late LPP was enhanced similarly for stress BO for all expressions, and for exercise BO only for angry faces, however, Adolph et al. (2013) found smaller LPP for faces in BO context. For pleasant and unpleasant odors, Cook et al. (2015) found smaller amplitude in the early phase of the LPP for faces during expo-sure to pleasant odor, whereas, the late portion of the LPP larger amplitudes for faces in the left hemisphere in pleasant odor condition and right in unpleas-ant odor condition. Lastly, Cook et al. (2018) found larger LPP (similar early onset as in Cook et al. (2015) in the unpleasant odor condition to faces. Notable in these reviewed studies is the heterogeneity of results. The only consistent result between all of these studies seems to be the main effect of stress-related body odor in Rubin et al. (2012) and unpleasant odor in Cook et al. (2018) on the early part of the LPP component. Apart from these results, the heterogeneity of results might be rather unsurprising given that the rele-vant studies here differ in several key aspects, such as the odors and facial expressions that were used, and the odor delivery method. One might even question whether there are any true odor effects on facial processing. How-ever, a more parsimonious explanation could be that most research in this area is underpowered. All effects seem to be within a range of .5 uV difference between conditions, a pretty small effect, and comparable to the results in study 1. Considering that the studies reviewed here had sample sizes between 18 and 26 participants, there might be power issues. In comparison with my study 1, the previous research might have been underpowered to detect inter-action effects, whereas study 1 had 2-3 times larger sample size in comparison. Large samples are clearly needed in future studies, given the small effect sizes. If we compare the ratings of facial arousal with the results in the N170 com-ponent we find a dissociation between congruency effects in behavioral and brain responses. The results showed that happy faces were rated as less

69

arousing in the unpleasant odor condition than in the pleasant odor condition, however, this difference in rated arousal was not reflected in amplitude differ-ences in the N170 component for happy faces. Disgusted faces on the other hand were rated as equivalently arousing in both odor conditions, however, here we find that they differed in amplitude in the N170 component. The faces were rated as more negatively valenced across facial expressions in the un-pleasant odor condition and as more positive in the pleasant odor condition. If we carefully consider the rating results and effects of odors on the N170 com-ponent, we might conclude that the difference in amplitude cannot be ex-plained by the arousing properties of the faces. As we used psychophysiolog-ical methods to equate odor intensities, we might tentatively conclude that the N170 effect was driven by odor valence rather than intensity.

Do congruent odors and facial expressions influence the amount of information needed to recognize happy and disgusted faces? To investigate if congruent odors influenced the amount of information needed to recognize facial expressions, I used dynamic facial expressions that morphed from neutral to a happy or disgusted facial expression. The idea to use dynamic facial stimuli is motivated by that in real life, facial expressions are dynamic and not static (Calvo & Nummenmaa, 2016). However, there are other benefits. Throughout my research, I have been wondering if emotion perception in faces is such an easy task for humans that additional information from other senses are unnecessary, thus, there is not much room for improve-ment in emotion recognition performance. However, in dynamic faces the amount of emotional information unfolds from neutral to full facial expres-sion, there is plenty of ambiguous information where a contextual odor can facilitate the recognition of the expression. This idea is not farfetched, as Leleu, Demily, et al. (2015) found that static facial expressions morphed with neutral were recognized at lower intensities in congruent odor contexts. Re-search using anxiety-induced body odors has found that emotion recognition accuracy in dynamic faces is facilitated overall for both happy and angry facial expressions (Rocha et al., 2018). The idea that the dynamic unfolding of facial emotion could be facilitated by valenced odors did not work out as hypothesized in study 2. We found positive evidence that odors in general did not affect the recognition speed of the facial expressions and very strong evidence against congruency effects in facial ex-pression recognition RTs. Similar as Rocha et al. (2018), we found that dy-namic happy faces were recognized faster and more accurately than disgusted and angry faces respectively, and these results were not qualified by a decrease in accuracy. We also performed some exploratory analyses and found weak

70

evidence that both disgusted and happy faces were recognized faster in the unpleasant odor condition. Why did we not see odor effects in RTs as we hypothesized? We replicated previous research that has shown that happy faces are recognized faster and more accurately than other emotions (Palermo & Coltheart, 2004). Odors were rated as expected and in line with studies 1 and 3. These results suggest that problems with stimuli cannot explain the lack of results. Another point to con-sider is the sample size in study 2; with only 21 participants we might have been unlucky and found a sample that was not sufficiently responsive to the experimental manipulation. Similarly, as small sample studies might be false positives (Forstmeier, Wagenmakers, & Parker, 2017), we may have obtained a false negative result. Further, if we consider that the odor effects in dynamic or degraded facial expressions has been in facial expression recognition accu-racy (Leleu, Demily, et al., 2015; Rocha et al., 2018), we might conclude that response times might be a poor measure for these types of effects. However, we could not replicate previous results of odor effects in accuracy either.

Do congruent odors, and facial expressions influence attentional processes? In study 3 we found decisive evidence that odors do not affect covert attention towards disgusted and happy facial expressions using the dot-probe task. In fact, the results showed that, on average, probe reaction times were within a few milliseconds regardless of probe location. From onset, the idea that emo-tional faces inherently attract attention has wide support from multiple sources (Vuilleumier, 2002). We also have evidence from eye-tracking studies that odors may direct attention to congruent objects (Aguillon-Hernandez et al., 2015; Seigneuric, Durand, Jiang, Baudouin, & Schaal, 2010; Seo, Roidl, et al., 2010), facilitate visual search (Chen, Zhou, Chen, He, & Zhou, 2013), and shift visuo-spatial attention (Rinaldi et al., 2018). Some evidence indicate that pleasant odors may facilitate the recognition of happy faces in a visual search task, but this advantage has a brief experimental effect (Damjanovic et al., 2018). Recently a few studies using the dot-probe task has investigated the effect of androstadienone (a component in human sweat) on attentional bias toward emotional faces. In one study, there was no attentional bias or effects of androstadienone (Hornung et al., 2017). In a second study with two exper-iments, the results in both experiments indicated attentional bias toward fear-ful faces, in the first experiment this attentional bias toward fearful faces was enhanced by androstadienone, but this effect was not replicated in the second experiment (Hornung, Noack, Kogler, & Derntl, 2019). Thus, odors may

71

affect visual attention, but not much research has been done for odor effects on attention to emotional facial expressions. In addition, recently the dot-probe task has received criticism for not being a reliable measure of attentional bias (Parsons, Kruijt, & Fox, 2019; Waechter, Nelson, Wright, Hyatt, & Oakman, 2013). Although the dot-probe task was originally intended for measuring attentional bias with RTs, there is some ev-idence from ERP research using the dot-probe task for attentional bias toward angry faces (e.g., Holmes et al., 2009) and threatening pictures (Kappenman, MacNamara, & Proudfit, 2015). In the study by Kappenman et al. (2015), the authors found no behavioral evidence for attentional bias, however, there was a rapid shift of attention toward threatening pictures indicated in the N2pc component. This initial shift of attention did not result in sustained attention toward the stimuli, which could explain the lack of behavioral results. Typi-cally in the dot-probe research, probes appear for 500 ms after the onset of the faces. Thus, attention might have shifted to another location when the probe appears, explaining the poor reliability of the dot-probe task. In addition, a recent review of the ERP research attentional bias toward faces found incon-sistent results and recommends that the dot-probe paradigm should be changed to better reflect the ERP method (Torrence & Troup, 2018).

Other results of interest My studies have generated several results that were outside of the specific research questions, but might be interesting to consider. Throughout my stud-ies I have used the same odors and facial expressions, although the main re-search themes have involved different measures and questions. The odors have been consistently rated as more or less equally intensive and/or arousing and valenced compared to the no-odor control in all studies. In studies 2 and 3 the participants rated the odors before and after the main experimental task. The results showed a slight decrease in arousal and valence, however this ef-fect was not statistically robust. I also asked the participants to rate the face stimuli during exposure to the odors. In studies 1 and 3 the participants rated the faces as more arousing and more valenced in the direction of the odor valence (negative and positive respectively), in study 1 we also found that happy faces were rated as less arousing in the unpleasant odor context. These general results were not replicated in study 2. In general, the results indicate that the odor effects on facial evaluations are unspecific regarding the facial emotion. In study 2 and 3 we found reduced reaction times in the unpleasant odor con-ditions. In study 2, this reduction generalized to dynamic expressions of both disgust and happiness. To my knowledge, no other studies have investigated

72

valenced odor effects on dynamic faces. Studies using static facial expressions has found that happy faces are recognized faster in a pleasant odor condition (Leppanen & Hietanen, 2003), that disgusted faces are recognized faster in pleasant and unpleasant odor conditions and slower at recognizing happy faces compared with the neutral condition (Seubert, Kellermann, et al., 2010); in the second study the results were similar but this time only for disgusted faces (Seubert, Loughead, et al., 2010). In study 3, the effect of an unpleasant odor was a function of experimental time and toward a non-facial target, the dot-probe. Previous research has shown that both unpleasant and pleasant odors may decrease RTs to simple auditory and visual cues (Millot et al., 2002) and faster Go responses in a Go/No-Go task (Albayay, Castiello, & Parma, 2019). We also have some evidence from visual search tasks for facial emotion, that the effect of valenced odors might depend on exposure time (Damjanovic et al., 2018). Some discrepant findings could perhaps be explained by time-on-task effects, something that most research to date has not investigated. The results in my studies indicate that an unpleasant odor may affect visual RTs, however, this effect seems to be irrespective of target type.

Strengths and limitations My thesis work has several notable strengths, as well as limitations, which are discussed below. Among the strengths are the use of open data (studies 1, 2, and 3), preregistered hypotheses and analysis plans (studies 2 and 3), and mod-ern statistical procedures such as Bayesian Mixed Linear Models. In general, I have used larger sample sizes than those commonly used in the field. I have also made an effort to validate and investigate the psychophysical properties of the odors used in my research. One of the big questions in this thesis is how olfactory and visual information is integrated. A recent study showed that olfactory and visual objects are pro-cessed independently (Hochenberger, Busch, & Ohla, 2015), specifically, in this study the researchers compared the response speeds and accuracies on visual and olfactory objects together and independently. The results showed that there was no additional advantage when congruent odors and visual stim-uli were presented together, indicating that information from both modalities were processed individually and there was no evidence for the integration of olfactory and visual information taking place, at least not in early stages of stimulus processing. When comparing these results with the findings in study 1, we have to keep in mind that the processing of object and social information such as facial expressions could be processed in different pathways, and thus these results do not necessarily indicate evidence against the notion of inte-gration of facial expressions and odors. Further, in the study by Hochenberger et al. (2015), odors were presented with an olfactometer, in study 1 odors were

73

present the whole time which might have a considerable difference in odor and visual integration. The odor cascade model suggests that odors are first detected, then the odor is identified before odor valence is determined (Olofsson, 2014). If true, this might be problematic for research on visual-odor integration that uses an olfactometer and does not consider that valence information in visual and odor objects are processed asynchronously. My studies used continuous (blocked) presentation of odors, which might corre-spond more closely to everyday situations. We used dynamic faces in study 2, to investigate whether valenced odors could affect the amount of information needed to recognize happy and dis-gusted faces. Dynamic faces are considered to be more naturalistic stimuli that is more easily recognized especially when the stimuli is degraded (Calvo & Nummenmaa, 2016). In the present study and many other studies in the field, stimuli morphs from neutral to emotional facial expression from 3 up to 15 seconds (Rocha et al., 2018; Trautmann-Lengsfeld et al., 2013). However, some research indicates that shorter emotion onset times are more natural. For happy and disgusted facial expressions 850-1100 ms has been suggested to be the most natural speed of morphing (Recio, Schacht, & Sommer, 2013). Thus, in future studies, emotion onset times are an important aspect to consider. Throughout my studies I have used only one unpleasant odor (valeric acid) and one pleasant odor (lilac), and a no-odor control. These odors were selected because I believed that they convey social information and that they are natu-rally connected to disgusted and happy facial expressions. Valeric acid con-notates to uncleanliness and can be described as an odor that could be emitted by the body. Lilac on the other hand conveys cleanliness, as it is a common perfume in soaps and detergents. By using the same odors in all studies we also can be fairly certain of the odor properties, and pool data across studies. But this comes at a price, as using only a few odors we do not know if the results in my studies will generalize to other odors. Many odorants involve a trigeminal component. Trigeminal sensations may be described as painful or irritating depending on the intensity of the trigemi-nal properties (Doty et al., 1978). Importantly, trigeminal properties are an important when it comes to odor valence (Licon, Manesse, Dantec, Fournel, & Bensafi, 2018). This could influence how odors are perceived and might be a problem when comparing two odors that are not matched on trigeminal prop-erties. When evaluating the odors in my studies I have not considered trigem-inal issues explicitly. However, odor intensity is a strong indicator for trigem-inal properties and may contribute additively with the odorant with larger ef-fects in high concentrations (Hummel & Livermore, 2002). In my studies, the used odors (valeric acid and lilac essence) were matched on intensity and were moderately intensive. Further, a recent study where participants rated valeric

74

acid using 146 odor descriptors on a 9 point scale found that the first descriptor with purely trigeminal properties, ammonia, was only the 9th highest odor quality in valeric acid (Bae, Yi, & Moon, 2019). These results indicate that although valeric acid might have some trigeminal odor properties, it is not one of the most prominent sensory property of valeric acid. A curious finding in my studies is that there is a much smaller habituation/ad-aptation to the odors than I would expect considering what is known about olfaction (Dalton, 2000; Ferdenzi et al., 2014). The results indicated that the ratings of odor valence (study 2 and 3) and arousal (study 3) did not signifi-cantly decrease pre-post odor exposure. A working hypothesis I have had is that strong odors adapt faster (Dalton, 2000; Winston, Gottfried, Kilner, & Dolan, 2005) whereas moderately intensive odors, such as we used, has a much smaller adaptation curve. A complicating matter is that odors might ha-bituate at different rates (Sinding et al., 2017). Thus, the specific odors (lilac and valeric acid) in moderate intensities used in this study might be resistant to habituation effects. Another idea I have is that there is adaptation as ex-pected but when participants are asked to rate the odor, attentional mecha-nisms boost the perception of the odor, in a similar manner as when we are not aware of the background humming of fluorescent lights in a room until we for some reason attend to the sound, then we can hear it clearly. A question here is where habituation occurs, it might be at receptor level in olfactory re-ceptors, or central areas such as the olfactory bulb, piriform cortex and orbito-frontal cortex (Sinding et al., 2017). Recent EEG work in humans on the ol-factory bulb indicates that the olfactory bulb responses do not habituate for prolonged odor exposures (Iravani, Arshamian, Ohla, Wilson, & Lundstrom, 2020). Thus, odor habituation might appear at later olfactory processing stages where they might be also affected by attentional effects. Odor delivery method might also affect how we interpret the results. Odors are thought to be powerful at affecting mood states (Knasko, 1995). This raises the question if the valenced odor effects on facial ratings in my studies could be explained by mood states. Studies using an event-related design, that is, odors are delivered with an olfactometer that is synced to visual stimuli (Cook et al., 2017) have shown odor-face congruency effects. Whereas, studies that have prolonged odor exposure such as study 1 and 3, indicated odor effects on face valence regardless of facial expression in the direction of the odor va-lence. Thus, participants might be in a negative mood state during exposure to unpleasant odors and thus rate the faces more negatively overall. Similarly, participants might be in a positive mood state during exposure to pleasant odors and rated the faces more positively. In the current studies, we did not consider mood effects, however, this could be an interesting avenue in future research.

75

I think that here we have to keep in mind that whatever odor delivery method that is used, there is always a trade-off between desirable and undesirable properties. By using contextual odors we can present visual stimuli in a fast pace, thus, we can have a much better estimate of the effect with many trials in a very short time. This method might also be more natural as odors typically linger in the air, especially in confined spaces. However, this type of design might not be optimal for detecting congruency effects. Using an event-related design, the interval between stimuli has to be much longer than for visual stimuli alone. Thus, for the same amount of trials experimental time could be as much as five times longer. Here we must consider factors such as boredom (experiments typically are not that fun to participate in, especially if they also are very slow and time consuming) and fatigue effects. I think that in the fu-ture, we need more empirical work and careful consideration what the appro-priate method is to investigate odor effects on facial perception. In general, we used sample sizes that are equivalent (study 2) or larger (study 1 and 3) than what is common in the field. Further, we thought that the studies were well powered to detect small effect sizes, and in studies 2 and 3, we used Linear Mixed Models that are more powerful than ANOVA types of statistics where important information is discarded by averaging responses. In hind-sight, I wonder if we had enough power to detect for example the 3-way in-teractions in studies 2 and 3. Power calculations for complex interactions have been traditionally difficult. Recently it has been suggested that sample sizes need to be much larger than previously thought, especially for interaction ef-fects. For example, the sample size for a moderate (d = .4) effect size in a typical repeated measures 2 x 2 ANOVA requires a sample size of 110 partic-ipants for 80% power. However, one way to increase power is to add more trials for a more precise estimate (Brysbaert, 2019). Something that I have always tried to have is many trials experiments. In addition, new methods have been developed to calculate power for these types of complex interactions and methods that were used in the present study (simR package; Green, MacLeod, & Nakagawa, 2016). This highlights that more precise power calculations can be performed nowadays, even for complex designs. Specific limitations in the studies were, in study 1; in relation to the other studies in this thesis, we did not pre-register our analysis plan or hypotheses, which might in comparison to the other studies be considered weaker. Further, the odor effect for disgusted faces in the N170 component was in absolute amplitude small and barely significant, although NHST significance results cannot be interpreted that way (they are either significant or not). In study 2, we had a small sample size due to sequential Bayes factor analysis (Rouder, 2014), this practice has been criticized recently (de Heide & Grunwald, 2018). In study 3, although there were odor effects, the main result was not related to faces. This was probably due to an experimental paradigm with questionable

76

reliability (Kappenman et al., 2014). All of these “mistakes” provide valuable lessons for my future research. One of the advantages of using Bayes factors over classical p-values is that it has been argued that it can be used for evidence for null effects (Dienes, 2011). Although Bayesian statistics are probably better than traditional NHST, some of the practices have recently received criticism (Tendeiro & Kiers, 2019). One of the points that are made is that Bayes factors might be sensitive to the priors that are used; especially default priors might be most problematic. Bayes factors also seem to be biased to favor the null when compared to clas-sical NHST. In addition, a Bayes factor only tells us how much to favor one model against another, but we cannot know whether the models in themselves are appropriate in describing the effect that is tested (Smithson, 2019; Tendeiro & Kiers, 2019). Further, it has been argued that optional stopping rules might inflate problems associated with Bayes factors (de Heide & Grunwald, 2018). Here, I think that the debate in the scientific community will continue and with time we will learn what the limitations and strengths of Bayesian statistics are. In the near future, I see a move from the use of Bayes factors toward Bayesian parameter estimations, for example.

Ethical considerations The current research, as all experimental research, needs to be considered in terms of the benefits provided by the research, and any potential harm to par-ticipants. There is always a risk that participating in a psychological experi-ment may lead to discomfort. I hope and think I have been successful in man-aging these risks by my soon decade-long experience of conducting these types of studies. I have learned that informing participants prior on all aspects of the experiment, and informing that participation is completely voluntary and that they can abort at any time without any consequence will have a calm-ing effect. Specifically, prior to any new step I have informed the participants on the exact details of the procedure and task and an estimate of the duration that this will take. Many or most of my participants have also been psychology students and generally aware of common procedures in research. Further, psy-chiatric disorders were exclusion criteria in my studies. As I have used unpleasant odors, they might cause discomfort to some partic-ipants. By using odors diluted to a moderate intensity level, I hope I have avoided causing excessive discomfort. To my knowledge, displaying pictures of facial emotion has not been shown to evoke any negative effects in partici-pants. Thus, I think that considering how little research there is on odor effects on facial perception, the scientific knowledge gained from my studies out-weigh the potential harm that my participants may have encountered.

77

In my research I have tried to follow open science practices. Apart from in-creasing trust in my studies and results, open science practices can also be viewed from an ethical aspect. We have to consider that those who participate in our research give us their time and effort in aiding science. By sharing data, we maximize this effort to the fullest. Sharing data will aid future research by availability to other researchers to re-analyze the data, to correct potential mis-takes, aid in power calculations for future studies and meta-analyses. Thus, when data is available to other researchers they can avoid spending resources and potential participants’ time in vain.

Future directions Given the present results, I am interested in pursuing research on the following question; how might individual differences in odor sensitivity contribute in our understanding of odor effects on face perception? Further, an interesting avenue is to investigate the effects of contextual vs. transient odor exposure. In an ongoing study, we used the emotional Stroop task to investigate the ef-fect of contextually valenced odors on Stroop interference (Syrjänen et al., 2020). In this study with 41 participants we used the same odor and facial stimuli as in our previous research. Faces were presented with the words “dis-gusted” or “happy” superimposed. The task was to classify the emotional ex-pression of the face, while ignoring the words (which were sometimes con-gruent and sometimes incongruent with the expressions). We found a success-ful Stroop effect, that is, participants were slower at classifying the facial ex-pressions when the words written on the faces did not match the facial expressions. However, this effect was not affected by the contextual odors. Interestingly, we found instead that the recognition of disgusted faces was faster in the unpleasant odor condition. In addition, this effect was qualified by trait body odor disgust sensitivity (BODS; Liuzza, Lindholm, et al., 2017), that is, the more disgust-sensitive participants were to body odors, the faster they were at classifying disgusted faces in the unpleasant odor condition. The BODS scale has previously been found to discriminate between participants’ in how they rate actual body odors (Liuzza, Olofsson, et al., 2017). These re-sults highlight that individual differences in trait disgust might be an important factor to consider in research when using unpleasant odors and expressions of disgust. In a broader sense, considering individual differences are in line with CET theory and recommendations put forward by Barrett et al. (2019). Unpublished results from a colleague in the research group (Marta Zakrzewska) have found interesting odor-face integration results. My col-league used the same unpleasant and pleasant odors and disgusted and happy

78

facial expressions as I have used in my studies. The results indicated odor-face congruency effects in facial expression recognition (i.e., happy faces were recognized faster in pleasant odor context than unpleasant and vice versa for disgusted faces). An important difference in this study compared to my studies was that my colleagues used an event-related design with an olfactom-eter to deliver a mixed sequence of odors to the participants. Similar odor-face congruency effects have been found for valence ratings of faces in event-re-lated designs (Cook et al., 2017). With these results in mind, it would be in-teresting to compare if odor delivery method matters for both how fast emo-tional expressions are detected and if delivery method influence how facial expressions are perceived. Here, I envision an experiment using an olfactom-eter where blocks of prolonged odor exposure and a fast pace of facial stimuli is interspersed by a slower pace of individual odor exposures to each displayed expression. Thus, with this kind of experimental setup, we can investigate odor-induced mood effects on facial perception and more transient odor-face integration. As noted in the introduction, we have a long history of using perfumes to en-hance our appearance (Classen et al., 2002). Today, the global fragrance in-dustry market size is estimated to over 70 billion USD (Grand View Research; 2019). In this market research report, the authors see a trend that the fragrance industry will continue to grow. This growth is driven by urbanization and ris-ing disposable incomes in developing countries. In a report for the Interna-tional Fragrance Association (IFRA), PwC, a large consulting company, esti-mates that the fragrance industry employs over 26 000 workers and spends 8% of their sales on research and development (PwC, 2019). Thus, people spend a lot of money to enhance their olfactory appearance, and companies make an effort to provide this enhancement. The practice of using odorants as appearance enhancers seem to translate to actual measurable behavior in facial perception according to research reviewed and presented in this thesis. In line with this observation, the results in my studies indicate that a simple, pleasant odor such as lilac is affecting participants evaluation positively and an un-pleasant odor negatively. There is also corroborating evidence that individuals are rated as both younger (Seubert et al., 2014), more trustworthy (van Nieuwenburg, de Groot, & Smeets, 2019), and more attractive in the presence of a pleasant odor (Dematte et al., 2007; Seubert et al., 2014). Similarly, we see the opposite effect of unpleasant odors on facial attractiveness (Dematte et al., 2007), and faces are rated as even less likable (Li et al., 2007). Compar-ing the size of the fragrance industry spending on research and the academic research output, we see a vast disparity. The fragrance industry spends a lot of money on research and development; however, this knowledge is not publicly available. This disparity in spending implies that more funding is needed for this type of research that may have a significant impact on social behavior.

79

Concluding remarks In this thesis, I have investigated how valenced odors affect the perception of congruent facial expressions using a multi-method approach. What I found suggests that, in general, ratings of facial expressions are affected in the direc-tion of the odor valence (e.g., faces, in general, are rated more negatively in the context of an unpleasant odor). Overall, the results in my studies indicate that faces are perceived as more arousing in the context of odors; however, this is regardless of valence. I found early cortical integration of valenced odors and facial expressions. Specifically, I found that the N170 to disgusted facial expressions was smaller in the unpleasant odor condition than in the pleasant odor condition. A smaller N170 in the presence of congruent odor and facial expression might indicate facilitated processing. However, considering that the results in this literature are quite heterogeneous, more research is needed on these effects for conclu-sive evidence. I found evidence that odors, in general, did not affect the recognition speed of dynamic facial expressions, and I found robust evidence against congruency effects in facial expression recognition RTs for these dynamic faces. This puts into question whether reported odor-face integration effects will generalize to everyday situations where facial expressions are changing. I hypothesized that valenced odors might attract attention toward congruent faces; however I found evidence against this hypothesis. Instead, exposure to an unpleasant odor leads to a gradually decreased response speed over time in an attention-demanding task, an effect that was independent of the face stim-ulus. Overall, the results in my studies indicate that an unpleasant odor may affect RTs; however, this effect seems to be irrespective of the target type. In summary, the results indicate that valenced odors have effects on facial perception and cortical processing. There is evidence for both general effects (odor valence and arousal properties have effects on face processing irrespec-tive of the facial expression), and specific effects (only present when odor and face correspond to the same emotion). While the evidence is not conclusive for odor-face integration and more research is needed, the methods I have in-troduced may increase transparency and robustness of published results, and help accelerate knowledge development in this field of research.

80

81

Acknowledgments

First of all, I would like to thank my main supervisor, Jonas Olofsson. He has given me plenty of freedom in my work, while at the same time trying to keep me on track while I have diverted to "greener grass" or "shiny, interesting things" besides the path. Although we have very different writing styles and we have not always agreed, I think his keen eye on details, and elegant style has made me a much better writer. As a supervisor, he has completed me per-fectly. Me being a detail-oriented person that starts digging into some in hind-sight insignificant details, he always dragged me up and forced me to consider the big picture. I think that the differences between us have taught me some-thing about myself, now I know where I fit in the big picture. I would also like to thank my second supervisor Håkan Fischer for his satellite view in my progress. I am also immensely grateful for Stefan Wiens. Stefan supervised my Bachelor's thesis and hired me as a research assistant. From the day we met, I knew what I wanted to do with my life. Stefan taught and chal-lenged me to learn how to program and think about experimental research and statistics. Other notable people have been Mats Nilsson, who taught me sta-tistics and psychophysics and Elisabet Borg, who gave me the opportunity to teach statistics. Lastly, Maria Larsson, who is a force of nature when dealing with the university administration. I would also like to thank my students for asking critical and sometimes strange questions. I never understood statistics intuitively until I started teach-ing it. I would also like to thank all the undergraduate and graduate students I supervised throughout the years, Nicki Basiri, Elena Ubaldini, Rebecca Lundberg, Ariana Dahlberg, Sverker Fällström, Elisabet Amann, David Sundberg, Julia Nielsen, Elin Wessman and Elin Eriksson. Many of you helped in data collections, and you have taught me just as much as I hope you learned from me. I would also like to thank my participants who contributed with their valuable time and data. I want to thank Pehr Granqvist (Stockholm University) and Daniel Västfjäll (Linköping University), who reviewed this thesis. Both of you agreed to re-view a long time ago and then patiently waited until I had a first draft. I really appreciate that both of you gave up some of your valuable time reviewing my

82

thesis. I know that this thesis is much better as a result of your thoughtful comments. I want to thank Petter Kallioinen for the cover art and always exciting discus-sions about all sorts of things (not only EEG). I have had many colleagues and friends throughout the years, I cannot mention them all. Some that I will never forget, just because we shared the room for so many years Anders Sand and Henrik Nordström. Some that always been there just for a chat Stina Cornell Kärnekull. Lastly, Ingrid Ekström, you left me alone with the rest of the freaks, and I miss our conversations ever since then. Lastly, my family and friends. The woman I call my "wife" Emma Persson. You have given me two beautiful children Enar and Vera. With these two bundles of joy, I have less time to work. But instead, I have gained in quality of life. Emma, you are the part of the reason I ended up on this academic journey. When I think back, what a trip, going from an incurable bachelor to some sort of grown-up. I think I am a better person because of you, or just maybe a little bit more well-rounded.

83

References

Adolph, D., Meister, L., & Pause, B. M. (2013). Context counts! social anxiety

modulates the processing of fearful faces in the context of chemosensory anxiety signals. Frontiers in Human Neuroscience, 7, 283. doi:10.3389/fnhum.2013.00283

Adolph, D., & Pause, B. M. (2012). Different time course of emotion regulation towards odors and pictures: are odors more potent than pictures? Biological Psychology, 91(1), 65-73. doi:10.1016/j.biopsycho.2012.05.003

Aguillon-Hernandez, N., Naudin, M., Roche, L., Bonnet-Brilhault, F., Belzung, C., Martineau, J., & Atanasova, B. (2015). An odor identification approach based on event-related pupil dilation and gaze focus. International Journal of Psychophysiology, 96(3), 201-209. doi:10.1016/j.ijpsycho.2015.03.009

Alaoui-Ismaïli, O., Robin, O., Rada, H., Dittmar, A., & Vernet-Maury, E. (1997). Basic Emotions Evoked by Odorants. Physiology and Behavior, 62(4), 713-720. doi:10.1016/s0031-9384(97)90016-0

Albayay, J., Castiello, U., & Parma, V. (2019). Task-irrelevant odours affect both response inhibition and response readiness in fast-paced Go/No-Go task: the case of valence. Scientific Reports, 9(1), 19329. doi:10.1038/s41598-019-55977-z

Allen, C., Havlíček, J., & Roberts, S. C. (2019). The Effects of Artificial Fragrances on Human Olfactory Communication, Cham.

Allen, C., & Mehler, D. M. A. (2019). Open science challenges, benefits and tips in early career and beyond. PLoS Biology, 17(5), e3000246. doi:10.1371/journal.pbio.3000246

Anderson, A. K., Christoff, K., Stappen, I., Panitz, D., Ghahremani, D. G., Glover, G., . . . Sobel, N. (2003). Dissociated neural representations of intensity and valence in human olfaction. Nature Neuroscience, 6(2), 196-202. doi:10.1038/nn1001

Bae, J., Yi, J. Y., & Moon, C. (2019). Odor quality profile is partially influenced by verbal cues. PloS One, 14(12), e0226385. doi:10.1371/journal.pone.0226385

Barrett, L. F. (2006). Are Emotions Natural Kinds? Perspectives on Psychological Science, 1(1), 28-58. doi:10.1111/j.1745-6916.2006.00003.x

84

Barrett, L. F. (2017a). How Emotions Are Made: The Secret Life of the Brain. New York: Houghton Mifflin Harcourt.

Barrett, L. F. (2017b). The theory of constructed emotion: an active inference account of interoception and categorization. Social Cognitive and Affective Neuroscience, 12(1), 1-23. doi:10.1093/scan/nsw154

Barrett, L. F., Adolphs, R., Marsella, S., Martinez, A. M., & Pollak, S. D. (2019). Emotional Expressions Reconsidered: Challenges to Inferring Emotion From Human Facial Movements. Psychological Science in the Public Interest, 20(1), 1-68. doi:10.1177/1529100619832930

Bensafi, M. (2002). Autonomic Nervous System Responses to Odours: the Role of Pleasantness and Arousal. Chemical Senses, 27(8), 703-709. doi:10.1093/chemse/27.8.703

Bensafi, M., Pierson, A., Rouby, C., Farget, V., Bertrand, B., Vigouroux, M., . . . Holley, A. (2002). Modulation of visual event-related potentials by emotional olfactory stimuli. Neurophysiologie Clinique/Clinical Neurophysiology, 32(6), 335-342. doi:10.1016/s0987-7053(02)00337-4

Bensafi, M., Rouby, C., Farget, V., Bertrand, B., Vigouroux, M., & Holley, A. (2002). Psychophysiological correlates of affects in human olfaction. Neurophysiologie Clinique/Clinical Neurophysiology, 32(5), 326-332. doi:10.1016/s0987-7053(02)00339-8

Bentin, S., Allison, T., Puce, A., Perez, E., & McCarthy, G. (1996). Electrophysiological Studies of Face Perception in Humans. Journal of Cognitive Neuroscience, 8(6), 551-565. doi:10.1162/jocn.1996.8.6.551

Borg, E., & Borg, G. (2002). A comparison of AME and CR100 for scaling perceived exertion. Acta Psychologica, 109(2), 157-175. doi:10.1016/s0001-6918(01)00055-5

Bradley, M. (2015). Smell and the Ancient Senses. New York: Routledge. Bradley, M. M., Codispoti, M., Cuthbert, B. N., & Lang, P. J. (2001). Emotion

and motivation I: Defensive and appetitive reactions in picture processing. Emotion, 1(3), 276-298. doi:10.1037//1528-3542.1.3.276

Bradley, M. M., Keil, A., & Lang, P. J. (2012). Orienting and emotional perception: facilitation, attenuation, and interference. Frontiers in Psychology, 3, 493. doi:10.3389/fpsyg.2012.00493

Bradley, M. M., & Lang, P. J. (1994). Measuring emotion: The self-assessment manikin and the semantic differential. Journal of Behavior Therapy and Experimental Psychiatry, 25(1), 49-59. doi:10.1016/0005-7916(94)90063-9

Brosch, T., Pourtois, G., Sander, D., & Vuilleumier, P. (2011). Additive effects of emotional, endogenous, and exogenous attention: behavioral and electrophysiological evidence. Neuropsychologia, 49(7), 1779-1787. doi:10.1016/j.neuropsychologia.2011.02.056

Brysbaert, M. (2019). How Many Participants Do We Have to Include in Properly Powered Experiments? A Tutorial of Power Analysis with Reference Tables. J Cogn, 2(1), 16. doi:10.5334/joc.72

85

Burns, E. J., Arnold, T., & Bukach, C. M. (2019). P-curving the fusiform face area: Meta-analyses support the expertise hypothesis. Neuroscience and Biobehavioral Reviews, 104, 209-221. doi:10.1016/j.neubiorev.2019.07.003

Calder, A. J., & Young, A. W. (2005). Understanding the recognition of facial identity and facial expression. Nature Reviews: Neuroscience, 6(8), 641-651. doi:10.1038/nrn1724

Calvo, M. G., Gutiérrez-García, A., Fernández-Martín, A., & Nummenmaa, L. (2014). Recognition of Facial Expressions of Emotion is Related to their Frequency in Everyday Life. Journal of Nonverbal Behavior, 38(4), 549-567. doi:10.1007/s10919-014-0191-3

Calvo, M. G., & Nummenmaa, L. (2016). Perceptual and affective mechanisms in facial expression recognition: An integrative review. Cognition & Emotion, 30(6), 1081-1106. doi:10.1080/02699931.2015.1049124

Chen, K., Zhou, B., Chen, S., He, S., & Zhou, W. (2013). Olfaction spontaneously highlights visual saliency map. Proceedings: Biological Sciences, 280(1768), 20131729. doi:10.1098/rspb.2013.1729

Clark, A. (2013). Whatever next? Predictive brains, situated agents, and the future of cognitive science. Behavioral and Brain Sciences, 36(3), 181-204. doi:10.1017/s0140525x12000477

Classen, C., Howes, D., & Synnott, A. (2002). Aroma: The Cultural History of Smell. London: Routledge.

Cook, S., Fallon, N., Wright, H., Thomas, A., Giesbrecht, T., Field, M., & Stancak, A. (2015). Pleasant and Unpleasant Odors Influence Hedonic Evaluations of Human Faces: An Event-Related Potential Study. Frontiers in Human Neuroscience, 9, 661. doi:10.3389/fnhum.2015.00661

Cook, S., Kokmotou, K., Soto, V., Fallon, N., Tyson-Carr, J., Thomas, A., . . . Stancak, A. (2017). Pleasant and unpleasant odour-face combinations influence face and odour perception: An event-related potential study. Behavioural Brain Research, 333, 304-313. doi:10.1016/j.bbr.2017.07.010

Cook, S., Kokmotou, K., Soto, V., Wright, H., Fallon, N., Thomas, A., . . . Stancak, A. (2018). Simultaneous odour-face presentation strengthens hedonic evaluations and event-related potential responses influenced by unpleasant odour. Neuroscience Letters, 672, 22-27. doi:10.1016/j.neulet.2018.02.032

Cordaro, D. T., Sun, R., Kamble, S., Hodder, N., Monroy, M., Cowen, A., . . . Keltner, D. (2019). The recognition of 18 facial-bodily expressions across nine cultures. Emotion. doi:10.1037/emo0000576

Cordaro, D. T., Sun, R., Keltner, D., Kamble, S., Huddar, N., & McNeil, G. (2018). Universals and cultural variations in 22 emotional expressions across five cultures. Emotion, 18(1), 75-93. doi:10.1037/emo0000302

86

Crandall, C. S., & Moriarty, D. (1995). Physical illness stigma and social rejection. British Journal of Social Psychology, 34 ( Pt 1), 67-83. doi:10.1111/j.2044-8309.1995.tb01049.x

Crouzet, S. M., Kirchner, H., & Thorpe, S. J. (2010). Fast saccades toward faces: face detection in just 100 ms. J Vis, 10(4), 16 11-17. doi:10.1167/10.4.16

Croy, I., Nordin, S., & Hummel, T. (2014). Olfactory disorders and quality of life--an updated review. Chemical Senses, 39(3), 185-194. doi:10.1093/chemse/bjt072

Croy, I., Olgun, S., & Joraschky, P. (2011). Basic emotions elicited by odors and pictures. Emotion, 11(6), 1331-1335. doi:10.1037/a0024437

Dalton, P. (2000). Psychophysical and Behavioral Characteristics of Olfactory Adaptation. Chemical Senses, 25(4), 487-492. doi:10.1093/chemse/25.4.487

Damjanovic, L., Wilkinson, H., & Lloyd, J. (2018). Sweet Emotion: The Role of Odor-induced Context in the Search Advantage for Happy Facial Expressions. Chemical Senses, 43(3), 139-150. doi:10.1093/chemse/bjx081

Darwin, C. (1872). The expression of the emotions in man and animals. New York: D. Appleton And Company

de Heide, R., & Grunwald, P. D. (2018). Why optional stopping is a problem for Bayesians. arXiv. doi:https://arxiv.org/abs/1708.08278v3

Dematte, M. L., Osterbauer, R., & Spence, C. (2007). Olfactory cues modulate facial attractiveness. Chemical Senses, 32(6), 603-610. doi:10.1093/chemse/bjm030

Dienes, Z. (2011). Bayesian Versus Orthodox Statistics: Which Side Are You On? Perspectives on Psychological Science, 6(3), 274-290. doi:10.1177/1745691611406920

Dixon, T. (2012). "Emotion": The History of a Keyword in Crisis. Emotion Review, 4(4), 338-344. doi:10.1177/1754073912445814

Doty, R. L. (1975). An examination of relationships between the pleasantness, intensity, and concentration of 10 odorous stimuli. Perception and Psychophysics, 17(5), 492-496. doi:10.3758/bf03203300

Doty, R. L. (1981). Olfactory communication in humans. Chemical Senses, 6(4), 351-376. doi:10.1093/chemse/6.4.351

Doty, R. L., Brugger, W. E., Jurs, P. C., Orndorff, M. A., Snyder, P. J., & Lowry, L. D. (1978). Intranasal trigeminal stimulation from odorous volatiles: Psychometric responses from anosmic and normal humans. Physiology and Behavior, 20(2), 175-185. doi:10.1016/0031-9384(78)90070-7

Dunbar, R. I. M., Cornah, L., Daly, F. J., & Bowyer, K. M. (2002). Vigilance in human groups: a test of alternative hypotheses. Behaviour, 139(5), 695-711. doi:10.1163/15685390260136771

Ebner, N. C., Riediger, M., & Lindenberger, U. (2010). FACES--a database of facial expressions in young, middle-aged, and older women and men: development and validation. Behavior Research Methods, 42(1), 351-362. doi:10.3758/BRM.42.1.351

87

Eimer, M., & Kiss, M. (2007). Attentional capture by task-irrelevant fearful faces is revealed by the N2pc component. Biological Psychology, 74(1), 108-112. doi:10.1016/j.biopsycho.2006.06.008

Ekman, P. (1992). An argument for basic emotions. Cognition and Emotion, 6(3-4), 169-200. doi:10.1080/02699939208411068

Ekman, P., & Cordaro, D. (2011). What is Meant by Calling Emotions Basic. Emotion Review, 3(4), 364-370. doi:10.1177/1754073911410740

Ekman, P., Sorenson, E. R., & Friesen, W. V. (1969). Pan-cultural elements in facial displays of emotion. Science, 164(3875), 86-88. doi:10.1126/science.164.3875.86

Etcoff, N. (1999). Survival of the Prettiest: The Science of Beauty. New York: Anchor Books.

Ferdenzi, C., Poncelet, J., Rouby, C., & Bensafi, M. (2014). Repeated exposure to odors induces affective habituation of perception and sniffing. Frontiers in Behavioral Neuroscience, 8, 119. doi:10.3389/fnbeh.2014.00119

Forstmeier, W., Wagenmakers, E. J., & Parker, T. H. (2017). Detecting and avoiding likely false-positive findings - a practical guide. Biological Reviews of the Cambridge Philosophical Society, 92(4), 1941-1968. doi:10.1111/brv.12315

Friston, K. (2010). The free-energy principle: a unified brain theory? Nature Reviews: Neuroscience, 11(2), 127-138. doi:10.1038/nrn2787

Gendron, M., Crivelli, C., & Barrett, L. F. (2018). Universality Reconsidered: Diversity in Making Meaning of Facial Expressions. Current Directions in Psychological Science, 27(4), 211-219. doi:10.1177/0963721417746794

Gerdes, A. B., Wieser, M. J., & Alpers, G. W. (2014). Emotional pictures and sounds: a review of multimodal interactions of emotion cues in multiple domains. Frontiers in Psychology, 5, 1351. doi:10.3389/fpsyg.2014.01351

Glass, S. T., Lingg, E., & Heuberger, E. (2014). Do ambient urban odors evoke basic emotions? Frontiers in Psychology, 5, 340. doi:10.3389/fpsyg.2014.00340

Goldstein, B. E., & Brockmole, J. R. (2017). Sensation and Perception (10th ed.). Boston: Cengage Learning.

Gottfried, J. A. (2010). Central mechanisms of odour object perception. Nature Reviews: Neuroscience, 11(9), 628-641. doi:10.1038/nrn2883

Gottfried, J. A., & Dolan, R. J. (2004). Human orbitofrontal cortex mediates extinction learning while accessing conditioned representations of value. Nature Neuroscience, 7(10), 1144-1152. doi:10.1038/nn1314

Gottfried, J. A., O'Doherty, J., & Dolan, R. J. (2002). Appetitive and aversive olfactory learning in humans studied using event-related functional magnetic resonance imaging. The Journal of Neuroscience, 22(24), 10829-10837.

88

Green, P., MacLeod, C. J., & Nakagawa, S. (2016). SIMR : an R package for power analysis of generalized linear mixed models by

simulation. Methods in Ecology and Evolution, 7(4), 493-498. doi:10.1111/2041-210x.12504

Groyecka, A., Pisanski, K., Sorokowska, A., Havlicek, J., Karwowski, M., Puts, D., . . . Sorokowski, P. (2017). Attractiveness Is Multimodal: Beauty Is Also in the Nose and Ear of the Beholder. Frontiers in Psychology, 8, 778. doi:10.3389/fpsyg.2017.00778

Handel, B. F., Haarmeier, T., & Jensen, O. (2011). Alpha oscillations correlate with the successful inhibition of unattended stimuli. Journal of Cognitive Neuroscience, 23(9), 2494-2502. doi:10.1162/jocn.2010.21557

Handler, A., Graham, T. G. W., Cohn, R., Morantte, I., Siliciano, A. F., Zeng, J., . . . Ruta, V. (2019). Distinct Dopamine Receptor Pathways Underlie the Temporal Sensitivity of Associative Learning. Cell, 178(1), 60-75 e19. doi:10.1016/j.cell.2019.05.040

Harry, B., Williams, M. A., Davis, C., & Kim, J. (2013). Emotional expressions evoke a differential response in the fusiform face area. Frontiers in Human Neuroscience, 7, 692. doi:10.3389/fnhum.2013.00692

Hartigan, A., & Richards, A. (2016). Disgust exposure and explicit emotional appraisal enhance the LPP in response to disgusted facial expressions. Social Neuroscience, 1-10. doi:10.1080/17470919.2016.1182067

Hermann, C., Ziegler, S., Birbaumer, N., & Flor, H. (2000). Pavlovian aversive and appetitive odor conditioning in humans: subjective, peripheral, and electrocortical changes. Experimental Brain Research, 132(2), 203-215. doi:10.1007/s002210000343

Hinojosa, J. A., Mercado, F., & Carretie, L. (2015). N170 sensitivity to facial expression: A meta-analysis. Neuroscience and Biobehavioral Reviews, 55, 498-509. doi:10.1016/j.neubiorev.2015.06.002

Hochenberger, R., Busch, N. A., & Ohla, K. (2015). Nonlinear response speedup in bimodal visual-olfactory object identification. Frontiers in Psychology, 6, 1477. doi:10.3389/fpsyg.2015.01477

Holmes, A., Bradley, B. P., Kragh Nielsen, M., & Mogg, K. (2009). Attentional selectivity for emotional faces: evidence from human electrophysiology. Psychophysiology, 46(1), 62-68. doi:10.1111/j.1469-8986.2008.00750.x

Hornung, J., Kogler, L., Erb, M., Freiherr, J., & Derntl, B. (2018). The human body odor compound androstadienone increases neural conflict coupled to higher behavioral costs during an emotional Stroop task. Neuroimage, 171, 364-375. doi:10.1016/j.neuroimage.2018.01.027

Hornung, J., Kogler, L., Wolpert, S., Freiherr, J., & Derntl, B. (2017). The human body odor compound androstadienone leads to anger-dependent effects in an emotional Stroop but not dot-probe task using human faces. PloS One, 12(4), e0175055. doi:10.1371/journal.pone.0175055

89

Hornung, J., Noack, H., Kogler, L., & Derntl, B. (2019). Exploring the fMRI based neural correlates of the dot probe task and its modulation by sex and body odor. Psychoneuroendocrinology, 99, 87-96. doi:10.1016/j.psyneuen.2018.08.036

Hummel, T., & Livermore, A. (2002). Intranasal chemosensory function of the trigeminal nerve and aspects of its relation to olfaction. International Archives of Occupational and Environmental Health, 75(5), 305-313. doi:10.1007/s00420-002-0315-7

Hutchinson, J. B., & Barrett, L. F. (2019). The Power of Predictions: An Emerging Paradigm for Psychological Research. Current Directions in Psychological Science, 28(3), 280-291. doi:10.1177/0963721419831992

Hutmacher, F. (2019). Why Is There So Much More Research on Vision Than on Any Other Sensory Modality? Frontiers in Psychology, 10, 2246. doi:10.3389/fpsyg.2019.02246

Hutto, D. D., Robertson, I., & Kirchhoff, M. D. (2018). A New, Better BET: Rescuing and Revising Basic Emotion Theory. Frontiers in Psychology, 9, 1217. doi:10.3389/fpsyg.2018.01217

Iravani, B., Arshamian, A., Ohla, K., Wilson, D. A., & Lundstrom, J. N. (2020). Non-invasive recording from the human olfactory bulb. Nature Communications, 11(1), 648. doi:10.1038/s41467-020-14520-9

Joyce, C., & Rossion, B. (2005). The face-sensitive N170 and VPP components manifest the same brain processes: the effect of reference electrode site. Clinical Neurophysiology, 116(11), 2613-2631. doi:10.1016/j.clinph.2005.07.005

Judd, C. M., Westfall, J., & Kenny, D. A. (2012). Treating stimuli as a random factor in social psychology: a new and comprehensive solution to a pervasive but largely ignored problem. Journal of Personality and Social Psychology, 103(1), 54-69. doi:10.1037/a0028347

Kameda, T., & Tamura, R. (2007). “To eat or not to be eaten?” Collective risk-monitoring in groups. Journal of Experimental Social Psychology, 43(2), 168-179. doi:10.1016/j.jesp.2006.02.003

Kanwisher, N., McDermott, J., & Chun, M. M. (1997). The Fusiform Face Area: A Module in Human Extrastriate Cortex Specialized for Face Perception. The Journal of Neuroscience, 17(11), 4302-4311. doi:10.1523/jneurosci.17-11-04302.1997

Kappenman, E. S., Farrens, J. L., Luck, S. J., & Proudfit, G. H. (2014). Behavioral and ERP measures of attentional bias to threat in the dot-probe task: poor reliability and lack of correlation with anxiety. Frontiers in Psychology, 5, 1368. doi:10.3389/fpsyg.2014.01368

Kappenman, E. S., MacNamara, A., & Proudfit, G. H. (2015). Electrocortical evidence for rapid allocation of attention to threat in the dot-probe task. Social Cognitive and Affective Neuroscience, 10(4), 577-583. doi:10.1093/scan/nsu098

90

Keil, A., Debener, S., Gratton, G., Junghofer, M., Kappenman, E. S., Luck, S. J., . . . Yee, C. M. (2014). Committee report: publication guidelines and recommendations for studies using electroencephalography and magnetoencephalography. Psychophysiology, 51(1), 1-21. doi:10.1111/psyp.12147

Kerr, N. L. (1998). HARKing: hypothesizing after the results are known. Personality and Social Psychology Review, 2(3), 196-217. doi:10.1207/s15327957pspr0203_4

Knasko, S. C. (1995). Pleasant odors and congruency: effects on approach behavior. Chemical Senses, 20(5), 479-487. doi:10.1093/chemse/20.5.479

Koster, E. P., Moller, P., & Mojet, J. (2014). A "Misfit" Theory of Spontaneous Conscious Odor Perception (MITSCOP): reflections on the role and function of odor memory in everyday life. Frontiers in Psychology, 5, 64. doi:10.3389/fpsyg.2014.00064

Kreibig, S. D. (2010). Autonomic nervous system activity in emotion: a review. Biological Psychology, 84(3), 394-421. doi:10.1016/j.biopsycho.2010.03.010

Krumhuber, E. G., Kappas, A., & Manstead, A. S. R. (2013). Effects of Dynamic Aspects of Facial Expressions: A Review. Emotion Review, 5(1), 41-46. doi:10.1177/1754073912451349

Lakens, D. (2019, November 18). The Value of Preregistration for Psychological Science: A Conceptual Analysis. PsyArXiv. doi:10.31234/osf.io/jbh4w

Lang, P. J., Bradley, M. M., & Cuthbert, B. N. (2008). International affective picture system (IAPS): Affective ratings of pictures and instruction manual. Technical report A-8.

Leleu, A., Demily, C., Franck, N., Durand, K., Schaal, B., & Baudouin, J. Y. (2015). The Odor Context Facilitates the Perception of Low-Intensity Facial Expressions of Emotion. PloS One, 10(9), e0138656. doi:10.1371/journal.pone.0138656

Leleu, A., Godard, O., Dollion, N., Durand, K., Schaal, B., & Baudouin, J. Y. (2015). Contextual odors modulate the visual processing of emotional facial expressions: An ERP study. Neuropsychologia, 77, 366-379. doi:10.1016/j.neuropsychologia.2015.09.014

Leppanen, J. M., & Hietanen, J. K. (2003). Affect and face perception: odors modulate the recognition advantage of happy faces. Emotion, 3(4), 315-326. doi:10.1037/1528-3542.3.4.315

Li, W., Moallem, I., Paller, K. A., & Gottfried, J. A. (2007). Subliminal smells can guide social preferences. Psychological Science, 18(12), 1044-1049. doi:10.1111/j.1467-9280.2007.02023.x

Licon, C. C., Manesse, C., Dantec, M., Fournel, A., & Bensafi, M. (2018). Pleasantness and trigeminal sensations as salient dimensions in organizing the semantic and physiological spaces of odors. Scientific Reports, 8(1), 8444. doi:10.1038/s41598-018-26510-5

91

Lieberman, D., & Patrick, C. (2014). Are the behavioral immune system and pathogen disgust identical? Evolutionary Behavioral Sciences, 8(4), 244-250. doi:10.1037/ebs0000018

Lindquist, K. A., Satpute, A. B., Wager, T. D., Weber, J., & Barrett, L. F. (2016). The Brain Basis of Positive and Negative Affect: Evidence from a Meta-Analysis of the Human Neuroimaging Literature. Cerebral Cortex, 26(5), 1910-1922. doi:10.1093/cercor/bhv001

Liuzza, M. T., Lindholm, T., Hawley, C., Senden, M. G., Ekstrom, I., Olsson, M. J., . . . Olofsson, J. K. (2017). The Body Odor Disgust Scale (BODS): Development and Validation of a Novel Olfactory Disgust Assessment. Chemical Senses, 42(6), 499-508. doi:10.1093/chemse/bjw107

Liuzza, M. T., Olofsson, J. K., Sabiniewicz, A., & Sorokowska, A. (2017). Body odor trait disgust sensitivity predicts perception of sweat bio-samples. Chemical Senses. doi:10.1093/chemse/bjx026

Luck, S. J. (2012). The Oxford Handbook of Event-Related Potential Components. In S. J. Luck & E. S. Kappenman (Eds.), The Oxford Handbook of Event-Related Potential Components: Oxford University Press.

Luck, S. J., & Gaspelin, N. (2017). How to get statistically significant effects in any ERP experiment (and why you shouldn't). Psychophysiology, 54(1), 146-157. doi:10.1111/psyp.12639

Lundqvist, D., Flykt, A., & , & Öhman, A. (1998). The Karolinska directed emotional faces (KDEF): CD ROM from Department of Clinical Neuroscience, Psychology section, Karolinska Institutet.

Majid, A., Burenhult, N., Stensmyr, M., de Valk, J., & Hansson, B. S. (2018). Olfactory language and abstraction across cultures. Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences, 373(1752). doi:10.1098/rstb.2017.0139

Mandler, G. (1990). William James and the Construction of Emotion. Psychological Science, 1(3), 179-180. doi:10.1111/j.1467-9280.1990.tb00193.x

Meyer, D. E., Osman, A. M., Irwin, D. E., & Yantis, S. (1988). Modern mental chronometry. Biological Psychology, 26(1-3), 3-67. doi:10.1016/0301-0511(88)90013-0

Miller, W. I. (1997). The Anatomy of disgust. Cambridge, Massachusetts: Harvard University Press.

Millot, J.-L., Brand, G., & Morand, N. (2002). Effects of ambient odors on reaction time in humans. Neuroscience Letters, 322(2), 79-82. doi:10.1016/s0304-3940(02)00092-7

Moessnang, C., Finkelmeyer, A., Vossen, A., Schneider, F., & Habel, U. (2011). Assessing implicit odor localization in humans using a cross-modal spatial cueing paradigm. PloS One, 6(12), e29614. doi:10.1371/journal.pone.0029614

Moore, T., & Zirnsak, M. (2017). Neural Mechanisms of Selective Visual Attention. Annual Review of Psychology, 68, 47-72. doi:10.1146/annurev-psych-122414-033400

92

Morey, R. D. (2008). Confidence Intervals from Normalized Data: A correction to Cousineau (2005). Tutorials in Quantitative Methods for Psychology, 4(2), 61-64. doi:10.20982/tqmp.04.2.p061

Murray, D. R., & Schaller, M. (2016). The Behavioral Immune System (pp. 75-129).

Ohman, A., & Mineka, S. (2001). Fears, phobias, and preparedness: toward an evolved module of fear and fear learning. Psychological Review, 108(3), 483-522. doi:10.1037/0033-295x.108.3.483

Olofsson, J. K. (2014). Time to smell: a cascade model of human olfactory perception based on response-time (RT) measurement. Frontiers in Psychology, 5, 33. doi:10.3389/fpsyg.2014.00033

Olofsson, J. K., & Gottfried, J. A. (2015). The muted sense: neurocognitive limitations of olfactory language. Trends in Cognitive Sciences, 19(6), 314-321. doi:10.1016/j.tics.2015.04.007

Olofsson, J. K., Nordin, S., Sequeira, H., & Polich, J. (2008). Affective picture processing: an integrative review of ERP findings. Biological Psychology, 77(3), 247-265. doi:10.1016/j.biopsycho.2007.11.006

Olsson, M. J., Lundstrom, J. N., Kimball, B. A., Gordon, A. R., Karshikoff, B., Hosseini, N., . . . Lekander, M. (2014). The scent of disease: human body odor contains an early chemosensory cue of sickness. Psychological Science, 25(3), 817-823. doi:10.1177/0956797613515681

Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716. doi:10.1126/science.aac4716

Palermo, R., & Coltheart, M. (2004). Photographs of facial expression: Accuracy, response times, and ratings of intensity. Behavior Research Methods, Instruments, & Computers, 36(4), 634-638. doi:10.3758/bf03206544

Palermo, R., & Rhodes, G. (2007). Are you always on my mind? A review of how face perception and attention interact. Neuropsychologia, 45(1), 75-92. doi:10.1016/j.neuropsychologia.2006.04.025

Park, J. Y., Gu, B. M., Kang, D. H., Shin, Y. W., Choi, C. H., Lee, J. M., & Kwon, J. S. (2010). Integration of cross-modal emotional information in the human brain: an fMRI study. Cortex, 46(2), 161-169. doi:10.1016/j.cortex.2008.06.008

Parsons, S., Kruijt, A.-W., & Fox, E. (2019). Psychological Science Needs a Standard Practice of Reporting the Reliability of Cognitive-Behavioral Measurements. Advances in Methods and Practices in Psychological Science, 2(4), 378-395. doi:10.1177/2515245919879695

Pause, B. M., Ohrt, A., Prehn, A., & Ferstl, R. (2004). Positive emotional priming of facial affect perception in females is diminished by chemosensory anxiety signals. Chemical Senses, 29(9), 797-805. doi:10.1093/chemse/bjh245

93

Platek, S. M., Thomson, J. W., & Gallup, G. G., Jr. (2004). Cross-modal self-recognition: the role of visual, auditory, and olfactory primes. Consciousness and Cognition, 13(1), 197-210. doi:10.1016/j.concog.2003.10.001

Pool, E., Brosch, T., Delplanque, S., & Sander, D. (2014). Where is the chocolate? Rapid spatial orienting toward stimuli associated with primary rewards. Cognition, 130(3), 348-359. doi:10.1016/j.cognition.2013.12.002

Pourtois, G., Schettino, A., & Vuilleumier, P. (2013). Brain mechanisms for emotional influences on perception and attention: what is magic and what is not. Biological Psychology, 92(3), 492-512. doi:10.1016/j.biopsycho.2012.02.007

PwC. (2019). The value of fragrance. Retrieved from https://ifrafragrance.org/docs/default-source/policy-documents/pwc-value-of-fragrance-report-2019.pdf?sfvrsn=b3d049c8_0

Raichle, M. E., & Gusnard, D. A. (2002). Appraising the brain's energy budget. Proceedings of the National Academy of Sciences of the United States of America, 99(16), 10237-10239. doi:10.1073/pnas.172399499

Rao, R. P., & Ballard, D. H. (1999). Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nature Neuroscience, 2(1), 79-87. doi:10.1038/4580

Recio, G., Schacht, A., & Sommer, W. (2013). Classification of dynamic facial expressions of emotion presented briefly. Cognition & Emotion, 27(8), 1486-1494. doi:10.1080/02699931.2013.794128

Regenbogen, C., Axelsson, J., Lasselin, J., Porada, D. K., Sundelin, T., Peter, M. G., . . . Olsson, M. J. (2017). Behavioral and neural correlates to multisensory detection of sick humans. Proceedings of the National Academy of Sciences of the United States of America, 114(24), 6400-6405. doi:10.1073/pnas.1617357114

Research, G. V. (2019). Fragrance Market Size, Share & Trends Analysis Report By Product (Deodorants, Perfumes), By Application (Personal Care, Household Care), By Distribution Channel (Offline, Online), And Segment Forecasts, 2019 - 2025. Retrieved from https://www.grandviewresearch.com/industry-analysis/fragrances-market

Rinaldi, L., Maggioni, E., Olivero, N., Maravita, A., & Girelli, L. (2018). Smelling the space around us: Odor pleasantness shifts visuospatial attention in humans. Emotion, 18(7), 971-979. doi:10.1037/emo0000335

Rocha, M., Parma, V., Lundstrom, J. N., & Soares, S. C. (2018). Anxiety Body Odors as Context for Dynamic Faces: Categorization and Psychophysiological Biases. Perception, 47(10-11), 1054-1069. doi:10.1177/0301006618797227

94

Ross, L. A., Saint-Amour, D., Leavitt, V. M., Javitt, D. C., & Foxe, J. J. (2007). Do you see what I am saying? Exploring visual enhancement of speech comprehension in noisy environments. Cerebral Cortex, 17(5), 1147-1153. doi:10.1093/cercor/bhl024

Rossion, B. (2014). Understanding face perception by means of human electrophysiology. Trends in Cognitive Sciences, 18(6), 310-318. doi:10.1016/j.tics.2014.02.013

Rossion, B., & Jacques, C. (2012). The Oxford Handbook of Event-Related Potential Components. In S. J. Luck & E. S. Kappenman (Eds.), The Oxford Handbook of Event-Related Potential Components: Oxford University Press.

Rouder, J. N. (2014). Optional stopping: no problem for Bayesians. Psychon Bull Rev, 21(2), 301-308. doi:10.3758/s13423-014-0595-4

Rubin, D., Botanov, Y., Hajcak, G., & Mujica-Parodi, L. R. (2012). Second-hand stress: inhalation of stress sweat enhances neural response to neutral faces. Social Cognitive and Affective Neuroscience, 7(2), 208-212. doi:10.1093/scan/nsq097

Russell, J. A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39(6), 1161-1178. doi:10.1037/h0077714

Russell, J. A. (1993). Forced-choice response format in the study of facial expression. Motivation and Emotion, 17(1), 41-51. doi:10.1007/bf00995206

Satpute, A. B., Hanington, L., & Barrett, L. F. (2016). Novel response patterns during repeated presentation of affective and neutral stimuli. Social Cognitive and Affective Neuroscience, 11(12), 1919-1932. doi:10.1093/scan/nsw104

Scarantino, A. (2018). The Philosophy of Emotions and Its Impact on Affective Science. In L. Feldman Barrett, M. Lewis, & J. M. Haviland-Jones (Eds.), The Handbook of Emotions (4th ed.). New York: Guildford University Press.

Schaller, M. (2014). When and how disgust is and is not implicated in the behavioral immune system. Evolutionary Behavioral Sciences, 8(4), 251-256. doi:10.1037/ebs0000019

Schaller, M., Miller, G. E., Gervais, W. M., Yager, S., & Chen, E. (2010). Mere visual perception of other people's disease symptoms facilitates a more aggressive immune response. Psychological Science, 21(5), 649-652. doi:10.1177/0956797610368064

Schaller, M., & Park, J. H. (2011). The Behavioral Immune System (and Why It Matters). Current Directions in Psychological Science, 20(2), 99-103. doi:10.1177/0963721411402596

Scherer, K. R. (2009a). The dynamic architecture of emotion: Evidence for the component process model. Cognition & Emotion, 23(7), 1307-1351. doi:10.1080/02699930902928969

Scherer, K. R. (2009b). Emotions are emergent processes: they require a dynamic computational architecture. Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences, 364(1535), 3459-3474. doi:10.1098/rstb.2009.0141

95

Schmidt, K. L., & Cohn, J. F. (2001). Human facial expressions as adaptations: Evolutionary questions in facial expression research. American Journal of Physical Anthropology, Suppl 33, 3-24.

Schonbrodt, F. D., Wagenmakers, E. J., Zehetleitner, M., & Perugini, M. (2015). Sequential Hypothesis Testing With Bayes Factors: Efficiently Testing Mean Differences. Psychological Methods. doi:10.1037/met0000061

Schupp, H. T., Flaisch, T., Stockburger, J., & Junghöfer, M. (2006). Emotion and attention: event-related brain potential studies. 156, 31-51. doi:10.1016/s0079-6123(06)56002-9

Schurgin, M. W., Nelson, J., Iida, S., Ohira, H., Chiao, J. Y., & Franconeri, S. L. (2014). Eye movements during emotion recognition in faces. J Vis, 14(13), 14. doi:10.1167/14.13.14

Seigneuric, A., Durand, K., Jiang, T., Baudouin, J.-Y., & Schaal, B. (2010). The Nose Tells it to the Eyes: Crossmodal Associations between Olfaction and Vision. Perception, 39(11), 1541-1554. doi:10.1068/p6740

Seo, H. S., Arshamian, A., Schemmer, K., Scheer, I., Sander, T., Ritter, G., & Hummel, T. (2010). Cross-modal integration between odors and abstract symbols. Neuroscience Letters, 478(3), 175-178. doi:10.1016/j.neulet.2010.05.011

Seo, H. S., Roidl, E., Muller, F., & Negoias, S. (2010). Odors enhance visual attention to congruent objects. Appetite, 54(3), 544-549. doi:10.1016/j.appet.2010.02.011

Seubert, J., Gregory, K. M., Chamberland, J., Dessirier, J. M., & Lundstrom, J. N. (2014). Odor valence linearly modulates attractiveness, but not age assessment, of invariant facial features in a memory-based rating task. PloS One, 9(5), e98347. doi:10.1371/journal.pone.0098347

Seubert, J., Kellermann, T., Loughead, J., Boers, F., Brensinger, C., Schneider, F., & Habel, U. (2010). Processing of disgusted faces is facilitated by odor primes: a functional MRI study. Neuroimage, 53(2), 746-756. doi:10.1016/j.neuroimage.2010.07.012

Seubert, J., Loughead, J., Kellermann, T., Boers, F., Brensinger, C. M., & Habel, U. (2010). Multisensory integration of emotionally valenced olfactory-visual information in patients with schizophrenia and healthy controls. Journal of Psychiatry and Neuroscience, 35(3), 185-194. doi:10.1503/jpn.090094

Shirasu, M., & Touhara, K. (2011). The scent of disease: volatile organic compounds of the human body related to disease and disorder. Journal of Biochemistry, 150(3), 257-266. doi:10.1093/jb/mvr090

Siegel, E. H., Sands, M. K., Van den Noortgate, W., Condon, P., Chang, Y., Dy, J., . . . Barrett, L. F. (2018). Emotion fingerprints or emotion populations? A meta-analytic investigation of autonomic features of emotion categories. Psychological Bulletin, 144(4), 343-393. doi:10.1037/bul0000128

96

Sinding, C., Valadier, F., Al-Hassani, V., Feron, G., Tromelin, A., Kontaris, I., & Hummel, T. (2017). New determinants of olfactory habituation. Scientific Reports, 7, 41047. doi:10.1038/srep41047

Smithson, M. (2019). Can Bayes Factors "Prove" the Null Hypothesis? Preprint. (1907.05583). arXiv.

Stark, R., Walter, B., Schienle, A., & Vaitl, D. (2005). Psychophysiological Correlates of Disgust and Disgust Sensitivity. Journal of Psychophysiology, 19(1), 50-60. doi:10.1027/0269-8803.19.1.50

Steinberg, C., Brockelmann, A. K., Dobel, C., Elling, L., Zwanzger, P., Pantev, C., & Junghofer, M. (2013). Preferential responses to extinguished face stimuli are preserved in frontal and occipito-temporal cortex at initial but not later stages of processing. Psychophysiology, 50(3), 230-239. doi:10.1111/psyp.12005

Steinberg, C., Dobel, C., Schupp, H. T., Kissler, J., Elling, L., Pantev, C., & Junghofer, M. (2012). Rapid and highly resolving: affective evaluation of olfactorily conditioned faces. Journal of Cognitive Neuroscience, 24(1), 17-27. doi:10.1162/jocn_a_00067

Sterling, P. (2012). Allostasis: a model of predictive regulation. Physiology and Behavior, 106(1), 5-15. doi:10.1016/j.physbeh.2011.06.004

Susskind, J. M., Lee, D. H., Cusi, A., Feiman, R., Grabski, W., & Anderson, A. K. (2008). Expressing fear enhances sensory acquisition. Nature Neuroscience, 11(7), 843-850. doi:10.1038/nn.2138

Syrjänen, E., Fischer, H., & Olofsson, J. K. (2019). Background odors affect behavior in a dot-probe task with emotionally expressive faces. Physiology and Behavior. doi:10.1016/j.physbeh.2019.05.001

Syrjänen, E., Liuzza, M. T., Fischer, H., & Olofsson, J. K. (2017). Do Valenced Odors and Trait Body Odor Disgust Affect Evaluation of Emotion in Dynamic Faces? Perception, 301006617720831. doi:10.1177/0301006617720831

Syrjänen, E., Wiens, S., Fischer, H., Zakrzewska, M., Wartel, A., Larsson, M., & Olofsson, J. K. (2017). Dataset for: Cortical integration of odor-visual emotional signals in the N170 ERP component. Retrieved from: https://figshare.com/s/252745f56a0f7b7115fc

Syrjänen, E., Wiens, S., Fischer, H., Zakrzewska, M., Wartel, A., Larsson, M., & Olofsson, J. K. (2018). Background Odors Modulate N170 ERP Component and Perception of Emotional Facial Stimuli. Frontiers in Psychology, 9, 1000. doi:10.3389/fpsyg.2018.01000

Syrjänen, E., Zakrzewska, M., & Olofsson, J. (2020). Trait body odor disgust sensitivity (BODS) is associated with enhanced odor-visual integration of disgust cues [Unpublished manuscript]. Department of Psychology, Stockholm University.

Tendeiro, J. N., & Kiers, H. A. L. (2019). A review of issues about null hypothesis Bayesian testing. Psychological Methods, 24(6), 774-795. doi:10.1037/met0000221

Torrence, R. D., & Troup, L. J. (2018). Event-related potentials of attentional bias toward faces in the dot-probe task: A systematic review. Psychophysiology, 55(6), e13051. doi:10.1111/psyp.13051

97

Trapp, S., & Kotz, S. A. (2016). Predicting Affective Information - An Evaluation of Repetition Suppression Effects. Frontiers in Psychology, 7, 1365. doi:10.3389/fpsyg.2016.01365

Trautmann-Lengsfeld, S. A., Dominguez-Borras, J., Escera, C., Herrmann, M., & Fehr, T. (2013). The perception of dynamic and static facial expressions of happiness and disgust investigated by ERPs and fMRI constrained source analysis. PloS One, 8(6), e66997. doi:10.1371/journal.pone.0066997

Tybur, J. M., & Lieberman, D. (2016). Human pathogen avoidance adaptations. Current Opinion in Psychology, 7, 6-11. doi:10.1016/j.copsyc.2015.06.005

Tybur, J. M., Lieberman, D., Kurzban, R., & DeScioli, P. (2013). Disgust: evolved function and structure. Psychological Review, 120(1), 65-84. doi:10.1037/a0030778

van Leeuwen, F., & Petersen, M. B. (2018). The behavioral immune system is designed to avoid infected individuals, not outgroups. Evolution and Human Behavior, 39(2), 226-234. doi:10.1016/j.evolhumbehav.2017.12.003

van Nieuwenburg, D., de Groot, J. H. B., & Smeets, M. A. M. (2019). The Subtle Signaling Strength of Smells: A Masked Odor Enhances Interpersonal Trust. Frontiers in Psychology, 10, 1890. doi:10.3389/fpsyg.2019.01890

Vuilleumier, P. (2002). Facial expression and selective attention. Current Opinion in Psychiatry, 15(3), 291-300. doi:10.1097/00001504-200205000-00011

Waechter, S., Nelson, A. L., Wright, C., Hyatt, A., & Oakman, J. (2013). Measuring Attentional Bias to Threat: Reliability of Dot Probe and Eye Movement Indices. Cognitive Therapy and Research, 38(3), 313-333. doi:10.1007/s10608-013-9588-2

Wicker, B., Keysers, C., Plailly, J., Royet, J.-P., Gallese, V., & Rizzolatti, G. (2003). Both of Us Disgusted in My InsulaThe Common Neural Basis of Seeing and Feeling Disgust. Neuron, 40(3), 655-664. doi:10.1016/s0896-6273(03)00679-2

Wiens, S., & Syrjänen, E. (2013). Directed attention reduces processing of emotional distracters irrespective of valence and arousal level. Biological Psychology, 94(1), 44-54. doi:10.1016/j.biopsycho.2013.05.001

Winston, J. S., Gottfried, J. A., Kilner, J. M., & Dolan, R. J. (2005). Integrated neural representations of odor intensity and affective valence in human amygdala. Journal of Neuroscience, 25(39), 8903-8907. doi:10.1523/JNEUROSCI.1569-05.2005

Yeshurun, Y., & Sobel, N. (2010). An odor is not worth a thousand words: from multidimensional odors to unidimensional odor objects. Annual Review of Psychology, 61, 219-241, C211-215. doi:10.1146/annurev.psych.60.110707.163639

98

Zernecke, R., Haegler, K., Kleemann, A. M., Albrecht, J., Frank, T., Linn, J., . . . Wiesmann, M. (2011). Effects of Male Anxiety Chemosignals on the Evaluation of Happy Facial Expressions. Journal of Psychophysiology, 25(3), 116-123. doi:10.1027/0269-8803/a000047

Zhou, W., & Chen, D. (2009). Fear-related chemosignals modulate recognition of fear in ambiguous facial expressions. Psychological Science, 20(2), 177-183. doi:10.1111/j.1467-9280.2009.02263.x

S1. Supplementary material for chapter “Review of behavioral effects of odors on face perceptions”

Table 1 Description and main results of studies with odor effects on faces

Study (year)

Design

Stimuli Odor/Body Odor/Face (O/BO/F)

N Female/male (F/M)

Conclusion

Hermann, Ziegler, Birbaumer, and Flor (2000)

Two groups of participants rated the valence and arousal (SAM-scale) of neutral faces conditioned by startle sounds during exposure to either a pleasant (N=15) or unpleasant (N=15) odor.

O: Pleasant & Unpleasant F:Neutral

30 M

Faces that were paired with a startle sound (CS+) during an unpleasant odor (UCS) were rated as both more arousing and negative than the CS- and faces during pleasant UCS in acquisition phase. Faces paired with a startle sound (CS+) during a pleasant odor (UCS) were also rated as more arousing than the CS- but not more valenced.

Gottfried, O'Doherty, and Dolan (2002)

3 CS+ neutral faces paired with UCS pleasant, neutral or unpleasant, 1 face CS- in the neutral odor condition. RTs for gender identification

O: Pleasant, neutral & unpleasant F: Neutral

17 (10 F)

No decrease in odor valence or intensity. Slower RTs to CS+ faces for the pleasant and unpleasant USC in first block, "no-effect" of USC in 2nd and 3rd block.

Bensafi, Pierson, et al. (2002)

Classification of neutral faces as either pleasant or unpleasant during exposure to a pleasant odor or clean air

O: Pleasant & neutral F: Neutral

15 F Pleasant odor did not affect classifying faces as pleasant or unpleasant

Leppanen and Hietanen (2003)

Exp 1. Emotion recognition task (on happy or disgusted faces) in a blocked design with a pleasant or an unpleasant odor. Exp 2. Similar as Exp 1 but with a between groups design and the addition of neutral faces and no-odor control condition

Exp 1. O: Pleasant & unpleasant F: Happy & Disgusted Exp 2. O: Pleasant, no-odor & unpleasant F: Happy, neutral & disgusted

Exp 1: 20 (15 F) Exp 2: 45 (40 F)

Exp 1. Happy faces were recognized faster in the pleasant odor context, this advantage was not present in the unpleasant odor condition, where disgusted faces were recognized slightly faster. Exp 2. Similar results as in Exp 1.Happy faces were recognized fastest in the pleasant odor context, intermediate in the no-odor control and slowest in the unpleasant odor condition.

Gottfried and Dolan (2004)

2 CS+ neutral faces paired with UCS odors, 2 faces CS- not paired with an unpleasant odor. RTs for gender identification, faces were rated (VAS) on intensity and valence

O: Unpleasant F: Neutral 18 (10 F) Faster RTs to CS+ first half, CS+ faces

rated as more negatively valenced

Platek et al. (2004)

RTs were measured in an own face recognition task while exposed to subthreshold levels of own BO, friend BO, male BO, female BO, androstenone, phenyl ethanol or no-odor

BO: Neutral F: Neutral 11 (7 F) Own faces were recognized faster

during exposure to own BO

Pause et al. (2004)

Judgement of neutral face valence after briefly shown pleasant or unpleasant facial expressions during exposure to either sport or anxiety induced BO

BO: Valence not reported F: Primes: Happy, fear & sad Target: Neutral

16 (8 F)

Neutral target faces in sport BO context were judged as more pleasant when they were preceded by a happy prime. For anxiety BO this priming effect was present in females only

Li, Moallem, Paller, and Gottfried (2007)

Likeability of neutral faces was rated (VAS) after exposure to subthreshold pleasant, neutral and unpleasant odors

O: Pleasant, neutral & unpleasant F: Neutral

31 (18 F) Non-odor-aware subgroup (N=15) rated faces as less likeable after unpleasant odor than pleasant

Dematte, Osterbauer, and Spence (2007)

Female participants rated (VAS) attractiveness of male faces during exposure to pleasant and unpleasant odors

O: Pleasant & unpleasant F: Neutral, varying in attractiveness

16 F Faces were rated as less attractive in the presence of an unpleasant odor.

Zhou and Chen (2009)

Exp 1, emotion recognition task of somewhat happy, ambiguous and somewhat fearful faces with fear, control and happiness induced BO. Exp 2, emotion recognition task on ambiguous fearful and happy faces during exposure to fear induced BO

Exp 1. BO: Neutral F: Happy & fear Exp 2. BO: Neutral & unpleasant F: Happy & fear

Exp 1: 48 F Exp 2: 16 F

In both experiments ambiguous facial expressions were categorized as fearful in the fear related BO condition

Seubert, Loughead, et al. (2010)

Accuracy and RTs were measured while in an emotion recognition task (on happy, neutral and disgusted faces) in an event related design with a pleasant, neutral and unpleasant odors

O: Pleasant, neutral and unpleasant F: Happy, neutral & disgusted

24 (10 F)

Participants were more accurate and faster in the pleasant and unpleasant odor conditions on recognizing disgusted expression compared to the neutral odor condition. Response times to happy faces were increased in the pleasant and unpleasant odor conditions on recognizing disgusted expression compared to the neutral odor condition.

Seubert, Kellermann, et al. (2010)

Emotion recognition task (on happy, neutral and disgusted faces) in an event related design with a pleasant, neutral and unpleasant odors

O: Pleasant, neutral and unpleasant F: Happy, neutral & disgusted

44 (23 F)

Participants were faster in the pleasant and unpleasant odor conditions on recognizing disgusted expression compared to the neutral odor condition. Similar trends in accuracy data as in Seubert (2010), however, did not survive multiple comparison correction of p-values.

Zernecke et al. (2011)

Evaluation of morphed neutral to happy faces on a VAS-scale on face valence during exposure to fear or exercise induced BO

BO: Slightly unpleasant F: Happy & neutral

15 M

Faces were rated as less happy during anxiety sweat condition, most pronounced effect for ambiguous expressions of happiness

Steinberg et al. (2012)

Face valence and arousal ratings (SAM) to CS+ (unpleasant odor) and CS- faces

O: Unpleasant F: Neutral 24 (12 F) CS+ faces rated as more negative post

conditioning, no arousal effects

Steinberg et al. (2013)

Preference for CS (pleasant, neutral and unpleasant odors) to neutral faces

O: Pleasant, neutral & unpleasant F: Neutral

23 (12 F) Preference for CS was pleasant > neutral > unpleasant

Adolph, Meister, and Pause (2013)

Exp 1. Ratings (SAM) of valence and arousal for faces displaying anxiety during exposure to either an anxiety induced BO or a sport induced BO. Exp 2 included a clean air condition

BO: Mildly negative F: Anxious

Exp 1: 40 F Exp 2. 36 F

Exp 1. Participants rated their emotions as more negative during the anxiety induced BO condition. Exp 2. Participants rated being more aroused during both the anxiety induced and sport BO condition

Seubert, Gregory, Chamberland, Dessirier, and Lundstrom (2014)

Classification of target faces as either older or younger than a cue face. Ratings (VAS) of target faces on either age or attractiveness

O: Pleasant & unpleasant linearly increasing valence F: Neutral, varying in attractiveness and age

18 (12 F) Faces were rated as more attractive and younger in the pleasant odor conditions.

Leleu, Demily, et al. (2015)

Matching of faces displaying various levels of emotions with standard faces expressing 100% emotion during the exposure to pleasant, neutral and unpleasant odor.

O: Pleasant, neutral, & unpleasant F: Happy, angry, fear & sad morphed with neutral in steps of 10% for different levels of emotion

48 (31 F)

Facial expressions were recognized at lower intensities in congruent odor conditions (e.g. happy faces in the pleasant odor condition), for happy, disgusted and angry expressions. There were fewer false alarms for disgusted faces in the unpleasant odor condition when emotion labels for faces were provided. Without labels, in the pleasant odor context more false alarms were provided for negative emotions.

Cook et al. (2015)

Ratings (VAS) of the valence of neutral faces primed by a pleasant, unpleasant or a neutral odor

O: Pleasant, neutral & unpleasant F: Neutral

23 (12 F)

The faces were rated as more pleasant after pleasant odor priming, less pleasant after unpleasant odor priming and in the between the valenced odors during clean air exposure

Regenbogen et al. (2017)

Ratings (VAS) of faces of sick and healthy individuals during exposure to sick and healthy BO and no BO control condition

BO: Sickness induced (unpleasant) & healthy (neutral) F: Sick & healthy, neutral

30 (15 F) Faces overall less liked in the sick BO condition compared to control

Hornung et al. (2017)

RTs and accuracy were measured in an emotional Stroop task, during exposure to either androstadienone masked by musk oil or only the musk oil

O: Neutral F: Happy, anger & fear

56 (29 F) Men exhibited less interference to angry faces during AND exposure.

Cook et al. (2017)

Ratings (VAS) of the valence of neutral faces during exposure to a pleasant, unpleasant or a neutral odor

O: Pleasant, neutral & unpleasant F: Happy & disgusted

23 (13 F)

Happy faces were rated as more pleasant during exposure to a pleasant odor, disgusted faces were rated as less pleasant during the exposure to an unpleasant odor

Rocha et al. (2018)

Two groups performed an emotion recognition task on dynamic faces during exposure to either anxiety induced or neutral BO.

BO: Similar in intensity, pleasantness F: Dynamic faces morphing from neutral to happy or angry

46 F

Overall, facial emotion recognition was more accurate in the anxiety induced odor condition. Overall, faster recognition of happy faces

Hornung et al. (2018)

RTs and task accuracy was measured in an emotional Stroop task, during exposure to either androstadienone masked by musk oil or only the musk oil

BO: Neutral F: Happy, angry & fearful

80 (51 F) No odor effects

Damjanovic, Wilkinson, and Lloyd (2018)

RTs and accuracy was measured in a detection of same different facial emotion in a visual search task during odor exposure in three groups of participants (pleasant, neutral or unpleasant)

O: Pleasant, neutral & unpleasant F: Happy, angry & neutral

54 (43 F)

Faster detection of happy faces in the neutral and unpleasant odor conditions. Faster detection of happy faces in the pleasant odor condition early in the task, but this effect decreased toward the end. In contrast this effect was reversed for happy faces in the unpleasant task, participants were slower at detecting happy faces toward the end of the task

Cook et al. (2018)

Ratings of valence of neutral faces either during exposure to a pleasant, unpleasant or a neutral odor or one second after exposure

O: Pleasant, neutral & unpleasant F: Neutral

28 (18 F)

Faces were rated as less pleasant when primed with an unpleasant odor, effect was largest when odor and faces were presented concurrently