embodied cognition in music

Korsakova-Kreyn, Embodied Cognition in Music, June 2015

1

Embodied Cognition in Music

MARINA KORSAKOVA-KREYN Touro College, New York, NY, USA

ABSTRACT: In this paper, I propose that embodied cognition in music has two distinct levels: 1) the apparent corporeal articulation of music by performers and listeners, which reflects either a desire to make visible their emotional responses to the music or rhythmic entrainment, and 2) the principal (though concealed) level of transient muscular reactions to the main coding aspects in music: the tonal relationships arranged in time. This paper argues that the apparent corporeal articulation with regard to the entrainment effect and dance (Leman & Maes, 2014) is more related to the multimodal integration that is characteristic of attending to such a multidisciplinary performing art as opera and ballet than to purely musical content. I also present empirical data on the perception of tonal distances (Korsakova-Kreyn & Dowling, 2014) and suggest an explanation of why listeners’ intuitive navigation in tonal and temporal space lies at the heart of emotional responses to music, including corporeal articulation. In addition, the paper touches on the research into temporality in music, such as memory constraints in the perception of tonal structures (Tillmann & Bigand, 2004). The main emphasis of this paper is on the principal two dimensions of music: tonal relationships and time. Understanding the primacy of these dimensions is important for defining music cognition and music in general. The paper also identifies the need for collaboration among various subdisciplines in musicology and the cognitive sciences so as to further the development of the nascent field of embodied cognition in music. KEYWORDS: music perception, embodied cognition, emotional processing in music

Expressive distinctions are easily encoded by the listeners through the verbal labels, but they are practically untranslatable by bodily mediation, when body expression is induced by the musical stimulus. Frances and Bruchon-Schweitzer (1983) The concept of embodied cognition is based on the understanding that our emotions, memory, speech, and imagination are inseparable from the experiences of our bodies. To say it differently, a mind is shaped by the motor and somatosensory experience of the body that houses that mind. Music, a very special form of communication between humans, illustrates two levels of embodied cognition. The first, “surface” level is the influence of visible bodily movement on music perception and cognition. The second, “deep” level deals with melodic morphology. Our minds read melodic information by comparing differences in the perceived tonal stability of melodic elements. The sense of stability is directly related to a physical sensation of perceived tension, which means that melodic morphology is based on a highly primitive principle of perception that involves changes in somatic tension (Radchenko et al., 2015)—changes that most likely include transient actions of the musculature in response to tonal and temporal patterns. This is why tonal music presents what is probably the most obvious and holistic example of embodied cognition.

SPEECH AND EMBODIED COGNITION

The theory of embodied cognition postulates that sensory information and motor activity are essential for understanding the surrounding world and for developing the abilities that are important for abstract reasoning (Foglia & Wilson, 2013). Because both memory and speech include sensorimotor representations, our imagination relies on previously experienced gestures and movements (Wellsby &


2

Pexman, 2014). This is why people’s reactions to speech are faster when they see the physical movements described by the speech (Glenberg & Kaschak, 2002). Imaging studies have shown that attending to a list of adjectives and verbs generates activation in the motor cortex for verbs only, which demonstrates an association between speech and specific muscular experience (James & Maouene, 2009). The perception of abstract words such as courage and dignity produces changes in activation in the areas of the brain associated with emotional processing (Kousta et al., 2011). Since emotion is an affective state of mind that is accompanied by physiological changes, the perception of such abstract words may illustrate embodied cognition as well.

PERCEIVED TENSION AND MELODIC MORPHOLOGY

When thinking about music, we must first of all consider melodic line and rhythm. In other words, the two main dimensions of music are the space of tones and the arrow of time. Currently, the theoretical background for studies in embodied cognition in music often resembles the theoretical underpinnings of embodied cognition in linguistic language, which means emphasizing the effects of the observable bodily movements of music performers and listeners on our understanding of music’s content (Leman & Maes, 2014). An image of a performing musician could indeed influence music perception, but a statement that the musician’s corporeal articulation convey the content of music is not necessarily precise, considering that such great musicians as Svyatoslav Richter and Glenn Gould believed that watching a pianist performing music distracts from listening to the music and understanding its content. There are, however, certain “covert” properties of melodies and harmonies that allow us to describe music as a particularly significant case of embodied cognition. These properties manifest the nature of music’s melodic morphology that is defined by perceived tension. For the theory of embodied cognition in music, the most noteworthy aspect of melodic morphology is the phenomenon of intuitive muscular actions of music listeners in response to the basic melodic elements that constitute musical matter. Among the varieties of human communication, music occupies a special place because it is able to transmit complex information in the absence of symbolic representation. People use universally adopted symbols to communicate mathematical thoughts, and people use semantic units (“constants of consciousness,” Solntsev, 1974) to inform about things and actions and their qualia via linguistic language. In music, however, there are neither words nor familiar visual images and, unlike the language of mathematics and a foreign tongue, music cognition does not require special training (Fritz et al., 2009). Nevertheless, musical compositions are able to elicit aesthetic emotions and convey general ideas, which means that music incorporates a highly abstract way of transferring complex information. The aim of musical information is to influence human psychological states. We know that listening to music generates changes in vital signs (Bernardi et al., 2006) and neurochemistry (Chanda & Levitin, 2013). These changes speak for the involvement of the autonomic nervous system during attending to music, and they highlight the fundamentally intuitive nature of music perception. Merely averaging physiological measures during music listening, however, does not explain the dynamic nature of musical emotions (Krumhansl, 1997). The method used to code and transfer information in music is very simple. This simplicity begins with the parsimony of the basic elements used in the construction of musical matter (here we are speaking of the European musical tradition that dominates the world today). Musical matter consists of seven diatonic and five chromatic tones; these twelve tones are repeated in different registers, creating a musical diapason. While the twelve tones differ in frequency (pitch), the most important quality of their relationships is the difference in degree of attraction to a stable tonal center, the tonic. The difference in tonal attraction is perceived as a difference in tension. Music teachers casually use the concept of tonal tension when explaining the character of tonal relationships with regard to musical phrasing and expressiveness. When people listen to music, the degrees of perceived tension define musical expectations and generate a sense of melodic and harmonic motion in music. The human mind reads musical information by using an appropriate system of reference that is formed spontaneously by the mind as soon as we hear different musical sounds. This system of reference, or perceptual schema, is known colloquially as a musical scale (see Figure 1). The nonlinear distances between the tonic and the other tones of a scale represent melodic intervals that are characterized by the intensity of tonal attraction, that is, by the degree of potential tonal energy. A diatonic scale has eight basic intervals, ranging from unison to octave.


3

Fig. 1. The tones of a scale differ in pitch and in the degree of their attraction to a tonic. In the C major scale, the leading tone B (diatonic step VII) is perceived as unstable (high potential energy), whereas the tonic C represents a potential energy value of zero. Music is constructed of melodic intervals and their combinations, chords. Considering that the distribution of tones and their groupings in the tonal space can be explained in terms of differences in the intensity of tonal attraction to a center of stability (the tonic), we can say that music, as a pattern of melodic information, communicates a configuration of levels of tonal attraction that are arranged in time. The character of the relationships among different tones allows us to discuss music in terms of “phenomenal gravity” (Scruton, 1997) or the “tonal force field.” Any conventional musical composition exists in a certain key (tonality), which means a basic set of tones that are organized into a system of reference around a given tonal center. The distance between two tonalities—between their tonal centers—is usually illustrated using the “circle of fifths.” The greater the number of shared tones between the two scales (the greater the amount of common information between them), the closer the two tonalities are on the circle of fifths (see Figure 2).

Fig. 2. The greater the number of shared tones between two scales, the closer the two tonalities are on the circle of fifths. Tonal distance increases gradually as the tonic of the second scale shifts away from the first by intervals of a fifth, so that fewer and fewer tones are shared. The expression tonal attraction belongs to a standard vocabulary for teaching music theory to students of musical schools of all levels. Beginning in the 1980s, perceived tension in music became a focus of empirical studies that used musical stimuli of various complexities, from the separate tones of a scale to fragments from musical compositions (Krumhansl & Kessler, 1982; Bigand et al., 1996; Krumhansl, 1996; Lehne et al., 2013). One of the important findings of the studies was a link between perceived tonal tension and physical tension (Madsen & Fredrickson, 1993). This discovery elucidates a distinction between the imaginary sense of gesture and movement in music and the concrete experience of perceived tonal tension that underlies the musical imagery. The combinations of tones that are perceived as tense are called dissonances, whereas those combinations that are pleasing to the ear are called consonances. The brain’s activation for consonant and dissonant sounds differs for one-day-old babies (Virtala et al., 2013), while two-month-old babies already demonstrate a preference for consonances (Trainor, 2004). In other words, human beings are born equipped to perceive the differences between the basic psychophysical characteristics of consonant and dissonant sounds that are the foundation of tonal patterns and therefore of music perception. Moreover, the sensation of musical dissonance and consonance is available to one-day-old chicks (Chiandetti & Vallortigara, 2011), which suggests that the melodic matter of music uses mechanisms of perception that do not require cognitive efforts of a higher order.


4

The consonant and dissonant qualia of combinations of musical sounds could be explained by differences in the amount of effort required by the auditory system to process melodic information. The hypothesis of a gradient of neuronal cost of auditory processing (Korsakova-Kreyn, 2005, 2014) proposes that when musical sounds share important spectral information, namely the strongest harmonics in their overtone series, the sharing makes it easier for the auditory system to process a combination of these sounds, which generates an agreeable auditory experience. For instance, the “happy” triad in the major mode and the sonically pleasant Pythagorean intervals of an octave, a fifth, and a fourth all present a combination of tones that belong to the beginning of a given overtone series (counting octave equivalence). At the level of neuronal nets, this redundancy of important spectral information may create the conditions for more efficient processing. In other words, the psychophysics of melodic compounds—the agreeable quality of musical consonants and the perceived tension of dissonant sounds—may be related to the degree of effort demanded from the auditory system to process the particular musical sound combinations. The hypothesis of a gradient of neuronal cost of auditory processing corresponds to the principle of less effort (Ferrero, 1894); here, the novelty lies in assigning the degree of pleasantness to the degree of effort (or economy of dedicated resources) on the level of neuronal processing. Research into neuronal activity in the brain stem in response to combinations of musical sounds has demonstrated that the magnitude of brain stem frequency-following responses (FFR) was more robust and more coherent for a consonant combination as compared with a dissonant combination (Bidelman & Krishnan, 2009). For example, the magnitude of FFR for triads was distributed as follows: major > minor > diminished > augmented. The study also found that the strongest neural pitch salience was produced for the unison, that is, for two tones that have identical fundamental tones and overtone series. Major-mode triads are known for their agreeable sonic quality and “happy” character. A major triad is made up of a fundamental tone and its four strongest harmonics, which are generated at the very beginning of the overtone series for that tone. This spectral characteristic emphasizes the importance of the “hidden” dimension of overtones in music. When the harmonic series of different tones have their beginning (strongest) overtones in common, sounding them together produces the consonant Pythagorean intervals and a major triad, whereas when the tones are the “distant relatives” with regard to their overtone series, this produces dissonances. It is precisely the interaction among different overtone series that generates the tonal force field, in which melodic structures appear as “folds” of tonal space in the same sense that material objects are “folds” and “wrinkles” in 3-D space (Florensky, 1925/1993). Conceptualizing melodies and other tonal structures as objects shaped by phenomenal tonal gravity suggests the possibility of explaining music with the help of analytical geometry, that is, in terms of melodic topology. The best candidate for this task seems to be a quaternionic and octonionic algebra that is also believed to be important for explaining human intelligence (Goertzel et al., 2008). Taking into consideration the hypothesis of supramodality in music perception (Korsakova-Kreyn, 2005), which proposes that the perception of music on the level of gestalt involves abstraction from the modality of incoming information during assessment of the configuration of homogeneous elements (tones) in a dynamic field, the mathematical model of melodic objects could advance our understanding of abstract and spatial reasoning.

UNIVERSAL LANGUAGE A conventional musical composition includes stable and unstable tonal elements. The listener’s intuition for tonal instability and stability determines the tonal expectations that guide the perception of music and creates a feeling of melodic and harmonic motion. Because tonal music affects listeners by means of auditory patterns made up of different levels of perceived tonal stability or different levels of perceived tension, it is safe to assume that music perception involves a corresponding pattern of somatic changes, including a pattern of muscular tension and release. A sensation of auditory comfort when listening to consonant sounds as compared with auditory uneasiness in response to disagreeable dissonant sounds illustrates the main morphological principle of music: perceived tension (Lerdahl & Krumhansl, 2007). The simplicity, if not primitivism of this principle, explains music’s ability to reach people in various cognitive states, including children with autism and elderly people with Alzheimer’s disease. Music has the ability to influence psychological states by utilizing the simplest type of reaction of the living organism to its environment, which is variation in the degree of experienced physical tension..


5

EMOTION IN MUSIC AND TONAL DISTANCES

Our exploratory study in affective responses to reorientation of the tonal system of reference, or tonal modulation, revealed that perception of the fine aspects of tonal organization in music does not require special musical training (Korsakova-Kreyn & Dowling, 2014). Testing the nonmusicians generated a map of tonal relationships that corresponds essentially with the accepted theory of functional harmony (see Figure 3). The study included two experiments. Experiment 1 used a set of 48 brief harmonic progressions modulating to all available degrees and in all available modal conditions for the beginning and the ending of the stimuli. Experiment 2 used a set of 24 harmonic progressions and a balanced set of real music excerpts from the compositions of the First Viennese School and the Romantics. Experiment 2 explored affective responses to modulation to three selected targets: the subdominant, the dominant, and a step 8, and in the major-major condition only (the modulating stimuli began and ended in the major mode). The perception of tonal distances was measured using bipolar adjective scales related to valence, synesthesia, potency, and tension. The results demonstrated that increased tonal distance was felt as colder, darker, and tenser than near tonal proximity (see Figure 4).

Warm%

Cold%

Bright%

Dark%

Happy%

Sad%

Pleasant%

Unpleasant%

Firm%

Wavering%

Strong%

Weak%

<2.0%

<1.0%

0.0%

1.0%

2.0%

<3.0% <2.0% <1.0% 0.0% 1.0% 2.0% 3.0%

Second

'Com

pone

nt'

First'Component'

48%Harmonic%Progressions%(12%Steps%x%4%Modal%CondiHons)%τ1%=%85.2%,%τ2%=%10.9%%

Major<Major%minor<Major%Major<minor%minor<minor%Warm<Cold%Bright<Dark%Happy<Sad%Pleasant<Unpleasant%Firm<Wavering%Strong<Weak%

1%

2%

3%4%

5%

6%

7%8%

9%

10%

11%

0%

1%

2%

3%

4%

5%

6%

7%

8%9%

10%

11%

0%

1%

2%

3%

5%6%

7%

8% 9%10%11%

0%0%

1%

2%

3%

4%

5%

7%

6%

8%

9%

10% 11%

4%

0%<%11%Steps%

Fig. 3. A PCA map of tonal distances for Experiment 1. The strongest variable commonality is related to the major and minor modes. Target tonalities in the major mode (diamonds and squares) are all on the side with the positive connotations (happy, pleasant, bright, and warm), whereas target tonalities in the minor mode (triangles and dots) gravitate to the negative side (sad, unpleasant, dark, and cold). The participants sensed different affective content in the different degrees of modulation. They recognized the importance of the subdominant (5) and dominant (7), which are both close in terms of key proximity, and the “pleasantness” of modulations in a major mode to the distant lowered second (up 1 semitone), the leading tone (up 11 semitones), and the minor sixth (up 8 semitones)—steps 1, 8, and 11 are in fact popular targets of deceptive cadence. The participants indicated negative feelings about modulation to the tritone (often termed the diabolo in musica, up 6 semitones) and the flattened leading tone (up 10 semitones). The tonal patterns of musical compositions can be imagined as artfully arranged sequences of auditory triggers that generate transient reactions of tension and relaxation. The accepted understanding of


6

embodied cognition presumes reiterating a previous modality-specific experience. Music is able to re-create various psychological states in their dynamics by using artfully sequenced gradations of perceived tension. This feeling of tension and relaxation is related to what is commonly called “life experience.” The episodes of people’s life experiences are generally infused with emotional components of various valence and intensity; these experiences are accompanied by somatic changes, including patterns of physical tension and relaxation. The re-creation of psychological states in music listeners happens through integration of the fleeting sensations of perceived tension encoded in the tonal relationships and the musical rhythms. An artful sequencing of musical sounds and their combinations has the ability to transmit the process of feeling. Moreover, in music we are dealing with aesthetic emotion, which includes the sense of beauty, nonpragmatic interests, cultural constructs, personal experiences, and the idiosyncrasies of personality.

Fig. 4. Experiment 2: A PCA map of affective responses to tonal distances for Real Music excerpts (RMs). Modulation to the distant step 8 was sensed as colder, darker, and tenser than modulation to the near subdominant (5) and dominant (7) steps. Target-steps are numbered for the C major as the beginning tonality.

APPARENT BODILY MOVEMENT AND MUSIC COGNITION: THE CRITIQUE OF DOMINATING IDEOLOGY

The existing studies in embodied cognition in music focus on visible bodily movements during music production and perception. This approach has a certain problem, however. The crux of the problem is that the prevalent theory explains embodied cognition in music from the perspective of kinesthetic information but without any explication of the causes of the sensations that give rise to bodily movements. For example, when a discussion of the effects of musical phrasing on corporeal articulation does not mention the actual musical source of melodic phrasing, which is the tonal relationships, this leads to equating musical phrasing with phrasing in speech. This equation does not have a strong foundation, however. First, music does not have cognitive constants similar to words. Second, melodic forms are defined by tonal gravity, which makes music perception more akin to navigation in phenomenal tonal space and even quasispatial reasoning than to speech. Complex musical compositions such as fugues and those written in sonata form provide ample evidence for melodic object–oriented cognition. Studies in the perception of melodic contour (Dowling, 1972; Korsakova-Kreyn & Dowling, under review) reaffirm the idea of a melody as a melodic object that can still be available to perception when transformed in the ways that resemble conventional visuospatial transformation (Shepard & Cooper, 1982). An astute critique of the idea of corporeal articulation informing the meaning of music came from the research of the pioneers in the field of embodied cognition, Frances and Bruchon-Schweitzer (1983), who wrote, “[E]xpressive distinctions are easily encoded by the listeners through the verbal labels, but they are practically untranslatable by bodily mediation, when body expression is induced by the musical stimulus.”


7

There is indeed a shared primeval root for music and speech, which is vocal production. The evolutionary bifurcation of vocal production into speech and music did not leave much space for relating the formation of content in speech to formation of content in music, however. Speech provides high-level precision of transfer of information: Speech operates with semantic units representing actions, things, phenomena, and qualia. In contrast, music dwells on the emotional component of vocal production: Music operates with musical sounds organized hierarchically according to perceived tension. Apart from the shared auditory modality, another point of confusion for studies in music cognition has been the concept of syntax in music. Some of the studies that suggest an affinity between music and linguistic language mention linguistic syntax and musical syntax in the same breath. But these two kinds of syntax have nothing in common. Whereas in linguistic language words are arranged according to the rules of parts of speech (the rules of linguistic syntax), music has neither cognitive constants similar to words, nor parts of speech. Instead, music uses a tonal “gravitational pull.” The conventional syntax of music (dictated by the gravitational pull) is expressed in a succinct formula of tonal harmony made up of a sequence of four chords: a triad on a tonic (the expression of stability), a triad on the subdominant tone (the expression of mild plagal instability), a triad on the dominant tone (the expression of instability, thanks to the leading tone), and the tonic triad (a movement back to stability). This formula of motion in tonal space came down to us from the 18th century, and it explains the “Newtonian” tonal universe of the First Viennese School. The majority of the music we hear today on headphones and in the concert hall relies on this formula; empirical studies have demonstrated that people have an innate understanding of tonal harmony and that they intuitively apply the rules of tonal harmony even when listening to a solo melody (Holleran et al., 1995). The tonal relationships, as explained by the basic formula of tonal harmony, define melodic phrasing. When we “feel” the completion of a musical phrase as a return to tonal stability, we experience an emotional component of the primeval vocal production that was harnessed by the tonal hierarchy. These two properties—the emotional component of voice production and the melodic line of speech (the latter emphasizes the content of speech)—exhaust the concrete shared properties between music and language. Intoning the words in speech brings out the emotional component of human communication, but the intoning can sharpen the meaning of words if and only if the words belong to a familiar vocabulary for the listener. There is no real ground for linking linguistic syntax with musical syntax since the conventional musical system, as Bigand et al. (2014) explain, “is rooted in psychoacoustic properties of sound, and this is not the case for linguistic syntax.” The nonsymbolic, supremely abstract way of communication used in music stands in stark contrast to the precision of linguistic language. The practice of translating the words into different languages illustrates the reliability of the semantic certainty of words. Music offers precision of a very different kind, related to the emotional aspect of human life. In the words of Susanne Langer (1942), “Because the forms of human feeling are much more congruent with music form than with the forms of language, music can reveal the nature of feeling and a detail and truth that language cannot approach.” Musical performance does, of course, demand physical effort and bodily movement. Skillful playing on a musical instrument can give a tremendous pleasure to the performing musician, a pleasure that is inseparable from the precision of conveyance of the melodic image and form. When the well-practiced gestures inadvertently deliver wrong notes, it can be devastating for a musician and for the content of the musical piece (even if the visible bodily movement looks the same for the wrong notes and the right notes). Furthermore, it is the melodic thinking that dictates the visual appearance of the physical efforts of a performing musician. In symmetry with the psychomotor program of the performer, the perception of music consists of the integration of transitory moments of perceived tension; these moments are the result of somatic responsiveness to details of the tonal relationships and rhythm. Pedagogical practice does employ gestures to illustrate the expressive value of the sound volume and melodic direction and to indicate the rounding of a musical phrase. Yet overall, neither music pedagogy nor performing practice sees bodily movement as playing a significant role in expressing the content of music. Unlike the great attention given to corporeal articulation in theater, where the meaning of words can be modulated by gesture and by physical appearance in general, music pedagogy and performing practices have been at best indifferent and at worst nonapproving towards expressive gesticulation during instrumental performance. If music pedagogy had ever found an expressive value of bodily movements (apart from solving specific technical problems related to playing a musical instrument), these movements would have been studied and incorporated into pedagogical practice—but this is not the case.


8

From the nonexpert perspective, seeing a virtuoso musician playing a musical instrument is exciting because of the ability of the musician to make music. For an expert musician, observing another performing musician is usually of interest because the corporeal articulation can provide a hint as to how a certain technical problem can be solved. For some listeners, the sight of an enthusiastically gesticulating music performer can be entertaining, while for others it can be quite annoying and distracting from the content of music. Differences in the interpretation of the same musical composition show that a deeply expressive performance can be visually modest. Furthermore, playing the same melody on different musical instruments produces different gestures that do not necessarily affect the “meaning” of the melody. The problem of connecting observable gestures with musical content emerges as soon as we try to compare the ordinary visuospatial perception that takes place in the 3-D world with the melodic and harmonic reasoning used in the cyclical tonal space, which is devoid of symbolic representation. For example, our research into tonal modulation revealed that the affective influence of melodic direction (an aspect of music that is easy to illustrate with a gesture) depends on the overall tonal context. Specifically, a melodic contour—visualized as a line in two-dimensional space—exerts more affective influence in the case of a musical phrase that modulates to a near key than it does in the case of a musical phrase that moves to a distant key (currently, the 12-tone tonal space has its explanation in a toroidal model [Purwins, 2005]). Unlike the readily available possibilities of connecting a certain gesture with a certain action in speech, in music tonal tension and release are the minute constituents of a flow of sounds, where a specific arrangement of the sounds in time generates a melodic and harmonic image that can be sensed as a gesture. Other aspects of music—such as tempo and expressive timing (agogica and rubato), timbre, accentuation, and variation in sound volume—belong to secondary qualia in relation to the tonal and temporal relationships (Scruton, 1997). If a music performer’s gestures appear meaningful for the listeners, this is so because these gestures skillfully follow the program of musical intentions encoded in the pattern of tonal tension and release. This is why the empathic involvement and sense of gesture in music are related not so much to visible corporeal articulation but, first of all, to tonal relationships arranged in time. The unfolding of tonal and rhythmical patterns generates an emotion-forming process in music. The corporeal articulation visible in music performance is simply the incidental by-product of playing a musical instrument, one that generally tells us very little—and in a rather sporadic manner—about the content of a particular piece of music. Some of the gestures of a performing musician may occasionally exert an affective influence on listeners, yet a rationale for the importance of seeing the performer in order to understand the logic of emotion in music, particularly in a relatively complex musical composition, is not clear. When a violinist makes a spectacular gesture after completing a virtuoso passage, this gesture is locally specific and tells us little about the overall content of the musical composition. When, during the rounding of a musical phrase, a pianist slows down the music’s pace and, on touching the final chord, makes a visible transition from being physically tense to becoming physically relaxed, the causes of the observable transformation reside in the tonal relationships that dictate the psychomotor program. A discussion of musical content needs to be clear about the hierarchy of the musical aspects that, when integrated, produce the complex phenomenon of aesthetic emotion. At the apex of this hierarchy are the tonal and temporal relationships that are primary with regard to the secondary and tertiary properties in music, such as timbre, sound volume, and the physical location of the sound source. For example, the first few notes of a favorite melody can be initiated by a human voice, and then the melody can be picked up and completed by a violin (or a trumpet, or a xylophone)—yet the division of the melody among different sound sources does not destroy the wholeness of the melodic object. This kind of distribution of a single motif among different musical instruments can be found in Western classical chamber music and symphonies, among other types of music. The source of musical sounds and their acoustical characteristics such as spectral envelope (timbre), amplitude (sound volume), and register (diapason) constitute secondary qualia with regard to the primary qualia of tonal relationships and the temporal organization of tones. Such technology-based improvements as headphones and recontextualizations of tracks to obtain better sound quality therefore do not belong to the musicological core. The distinction between the acoustical characteristics of musical sounds and the tonal relationships in the acousmatic space of melodic elements and musical forms is crucially important for describing music and music cognition. The two main dimensions in music—the tonal space and the arrow of time (rhythm)—are interwoven; together they constitute the tonal chronotope, the musical space populated by melodic/harmonic structures. An intuition for tonal hierarchy and musical rhythm lets listeners recognize music in an acoustic array; this intuition permits us to sense the difference between music and speech and between music and the environmental noise emitted, for instance, by a working engine. When the acoustic


9

array “offers information about the identity of and location of vibratory events” (Kersten, 2014), this kind of information is not imperative for music perception: The artifactual nature of music relies first and foremost on the tonal and temporal organization of musical sounds. Philosophic discourse on the temporality of musical experience and on the effect of musical context (Phillips, 2014; Kon, 2014) could benefit from the studies addressing the “sliding temporal window” in music perception (Tillmann & Bigand, 2004) and the time constraints for melodic perception (for example, the concept of “melodic fission,” Dowling, 1973). The involvement of short-term memory allows for the temporal integration of tonal events arranged in time in a specific order; the cognition of melodic structures relies to a large extent on the sliding temporal window of music perception, which is affected by the speed and complexity of tonal changes. For instance, the perception of reorientation of a tonal scale on a different tonal center (tonal modulation) and the perception of melodic transformation, when a transformed melody is compared to a template, both require a brief memory buffer. The influence of tonal proximity on the estimate of duration of a modulating musical fragment (Firmino et al., 2009) showed a fascinating connection between tonal distance and time perception in music, when modulation to the furthest tonal distance elicited the shortest time estimation. An ongoing EEG study by Radchenko et al. (in print) reveals the dependence of somatic changes on tonal proximity and on the interaction of tonal proximity and the major and minor modes. These studies provide new, important evidence for embodied cognition in music and indicate a new direction for understanding the process of musical meaning formation.

CORPOREAL ARTICULATION AND MULTISENSORY INTEGRATION Leman’s concept of the “human body as mediator in a musical meaning formation process” (2010) is irrefutably fruitful for the field of music cognition. As a novel development in cognitive science, the field of embodied cognition in music has been undergoing a process of clarification of criteria. Perhaps the research into the effect of visible bodily movement on making sense of music actually addresses the multisensory integration characteristic of attending to multidisciplinary performing art forms such as opera, pop-music stage production, ballet, and the like. The strongest argument for “surface”-level embodied cognition in music comes from the studies in the synchronization of people’s movements in response to tempo, meter, and rhythm (Naveda & Leman, 2009; Burger et al., 2013). Music has been used for millennia for accompanying important ceremonies and for dancing and marching. Apart from the pleasantness of melody, the main value of music for collective movement is musical meter and rhythm. The repetitiveness of musical meter makes it easier for a group of people to move together (Repp & Su, 2013). The power of repetitiveness is so great that a melody can be redundant with regard to eliciting synchronized movement among the listeners. Actually, synchronized movement can be achieved without any melody. (Large & Gray [2015] showed that rhythmic entrainment can be elicited from nonhuman primates.) Even for an art form as complex as ballet, until recently music was recognized primarily as a handmaiden, a provider of musical meter (Martens, 1918). Among outstanding attempts that have been made to illustrate music’s content with bodily movement are the innovative works of the great contemporary choreographers Jiří Kylián and Nacho Duato Bárcia.

VISUAL GESTURES AND MEANING IN MUSIC There is a special case in music performance when corporeal articulation is thoughtfully studied and practiced with an eye to achieving technical virtuosity of execution. The art of conducting is defined by the expressiveness and precision of gestures whose purpose is to assist a group of musicians in realizing the shared goal of conveying the images and structure of a musical composition. A conductor’s gestures take into consideration the specific nature of different musical instruments (such as brass versus stringed instruments) and of the voice. These gestures are suggestive of the character of the music in relation to the phrasing, variations in sound volume, melodic direction, and the details of tempo and rhythm, including accentuation and grouping; all these aspects of conducting are dictated, first of all, by the tonal relationships unfolding in time that generate musical images.

CONCLUSION


10

The proposed integrative theory of embodied cognition in music identifies two levels of bodily involvement in music perception: the “surface” level of corporeal articulation of music and the “deep” level of momentary somatic changes (including minute muscular actions) in response to a pattern of tonal and temporal relationships. It is the joint task of the core areas of musicology (music theory, functional analysis, and the analysis of musical form in historical context) and the science of music (particularly the research into the psychophysics of melodic elements and the neurophysiology of psychomotor program in music performance) to reinforce the theoretical foundation for the nascent understanding of music perception as embodied cognition. Bearing in mind Panksepp’s proposal that “our subneocortical animalian brain, with its many basic attentional, emotional, and motivational systems, may actually lie at the center of our mental universe” (2004), research into music, the universally acknowledged language of emotion, could prove to be crucially important for our understanding of the human mind.

ACKNOWLEDGMENTS I would like to express my gratitude to Peter Borten for his valuable assistance with regard to the text of this article.

NOTES

[1] Correspondence can be addressed to: Dr. Marina Korsakova-Kreyn, Touro College, E-mail: [email protected] or [email protected]

REFERENCES Bernardi, L., Porta, C., & Sleight, L. (2006). Cardiovascular, cerebrovascular, and respiratory changes induced by different types of music in musicians and non-musicians: The importance of silence. Heart, 92, 445-452 Bidelman, G. M. & A. Krishnan. (2009). Neural correlates of consonance, dissonance, and the hierarchy of musical pitch in the human brainstem. The Journal of Neuroscience, 29 (42), 13165–13171 Bigand, E., Delbé, C., Poulin-Charronnat, B., Leman, M., & Tillmann, B. (2014). Empirical evidence for musical syntax processing? Computer simulations reveal the contribution of auditory short-term memory. Frontiers in Systems Neuroscience, 8: 94, doi: 10.3389/fnsys.2014.00094 Bigand, E., Parncutt, R., & Lerdahl, F. (1996). Perception of musical tension in short chord sequences: The influence of harmonic function, sensory dissonance, horizontal motion, and musical training. Perception and Psychophysics, 58, 125-141 Burger B., Thompson M. R., Luck G., Saarikallio S. and Toiviainen P. (2013) Influences of rhythm- and timbre-related musical features on characteristics of music-induced movement. Frontiers of Psychology, 4(183), 1-10 Chanda, M. L. & D. J. Levitin (2013). The neurochemistry of music. Trends in Cognitive Sciences, 17 (4), 179-193 Chiandetti, C. & Vallortigara, G. (2011). Chicks like consonant music. Psychological science, 22(10), 1270-1273 Dowling W. J. (1972). Recognition of melodic transformations: Inversion, retrograde, and retrograde inversion. Perception and Psychophysics, 12 (50), 417-421 Dowling, W. J. (1973). The perception of interleaved melodies. Cognitive Psychology, 5, 322-337


11

Ferrero, G. (1894). L’inertie mentale et la loi du moindre effort. Revue Philosophique de la France et de l’Étranger, 37, 169–182 Firmino, E. A., Bueno, J. L. O., & Bigand, E. (2009). Travelling through pitch space speeds up musical time. Music Perception, 26(3), 205–209 Florensky, P. A. (1925/1993). Анализ пространственности и времени в художественно-изобразительных произведениях [Analysis of spatiality and time in the representational art]. Москва: Прогресс [Moscow: Progress] Foglia, L. & Wilson, R. A. (2013) Embodied cognition. WIREs Cognitive Science, doi: 10.1002/wcs.1226 Fritz, T., Jentschke, S., Gosselin, N., Sammler, D., Peretz, I., Turner, R., Friederici, A., & Koelsch, S. (2009). Universal recognition of three basic emotions in music. Current Biology, 19 (7), 573-576 Frances, R. & Bruchon-Schweitzer, M. (1983). Musical expressions and body expression. Perceptual and Motor Skills, 57, 587-595, pg. 594 Glenberg, A. M. & Kaschak, M. P. (2002). Grounding language in action. Psychonomic Bulletin & Review, 9, 558–565 Goertzel, B., Aam, O., F. Smith, A. & Palmer, K. (2008). Mirror neurons, mirrorhouses, and the algebraic structure of the self. Cybernetics and Human Knowing 15(1), 9-28 Holleran, S., Jones, M. & Butler, D. (1995). Perceived implied harmony. Journal of Experimental Psychology, 21 (3), 737-753 James, K. H. & Maouene, J. (2009). Auditory verb perception recruits motor systems in the developing brain: an fMRI investigation. Developmental Science, 12, F26–F34 Kersten, L. (2014). Music and Cognitive Extension. Empirical Musicology Review, 9 (3-4), 193-202, pg. 194 Kon, M. (2014). The temporality of musical experience: philosophical models and embodiment. Empirical Musicology Review, 9 (3-4), 213-223 Korsakova-Kreyn, M. (2005). Tonal processing and musical imagery: the tonal chronotope of music. Dissertation Proposal, on file at The University of Texas at Dallas, web source: http://universe-of-music.com/ Korsakova-Kreyn, M. (2014). Bach, Escher, and mental rotation: An empirical study in the perception of visual and melodic congruency. In A. Kozbelt (Ed.), Proceedings of the Twenty-Third Biennial Congress of the International Association of Empirical Aesthetics, 495-504 Korsakova-Kreyn, M. & Dowling, W. J. (under review). Mental Rotation in Visual and Musical Space: Comparing Pattern Recognition in Different Modalities Korsakova-Kreyn, M. & W. Jay Dowling. (2014). Emotional processing in music: Study in affective responses to tonal modulation in controlled harmonic progressions and real music. Psychomusicology, 24(1), 4-20 Kousta, S.-T., Vigliocco, G., Vinson, D.P., Andrews, M. & Del Campo, E. (2011). The representation of abstract words: why emotion matters. Journal of Experimental Psychology, 140(1), 14-34 Krumhansl, C. L. (1996). A perceptual analysis of Mozart’s piano sonata K. 282: Segmentation, tension, and musical ideas. Music Perception, 13, 401–432


12

Krumhansl, C. L. (1997). An exploratory study of musical emotions and psychophysiology. Canadian Journal of Experimental Psychology, 51, 336-335 Krumhansl, C.L. & Kessler, E. (1982). Tracing the dynamic changes in perceived tonal organization in a spatial representation of musical keys. Psychological Review, 89, 334–368 Langer, S. (1942/2009). Philosophy in a New Key: A Study in the Symbolism of Reason, Rite and Art. Harvard University Press. pg. 235 Large, E.W. & Gray, P.M. (2015). Spontaneous Tempo and Rhythmic Entrainment in a Bonobo. Journal of Comparative Psychology, Advance online publication: http://dx.doi.org/10.1037/com0000011 Lehne, M., Rohrmeier,M., Gollmann, D.,& Koelsch, S. (2013). The influence of different structural features on felt musical tension in two piano pieces by Mozart and Mendelssohn. Music Perception: An Interdisciplinary Journal, 31(2), 171-185 Leman, M. (2010). An embodied approach to music semantics. Musicae Scientiae, 14(1), 43-67; doi: 10.1177/10298649100140S104 Leman, M. & Maes, P.-J. (2014). The Role of Embodiment in the Perception of Music. Empirical Musicology Review, 9 (3-4), 236-246 Lerdahl, F. & Krumhansl, C. L. (2007). Modelling tonal tension. Music Perception, 24(4), 329-366 Madsen, C. K. & Fredrickson, W. E. (1993). The experience of musical tension: A replication of Nielsen's research using the continuous response digital interface. Journal of Music Therapy, 30, 46-63 Martens, F. H. (1918). The attitude of the dancer toward music. The Musical Quarterly, 4(3), 440-449 Naveda, L. & Leman, M. (2009). A cross-modal heuristic for periodic pattern analysis of samba music and dance. Journal of New Music Research, 38(3), 255–283 Panksepp, J. (2005) Affective consciousness: Core emotional feelings in animals and humans. Consciousness and Cognition, 14(1), 30-80 Phillips, M. (2014). A discussion of the effect of musical context on perception of succession of musical events, and its relevance for philosophical models of temporal experience. Empirical Musicology Review, 9 (3-4), 230-235 Purwins H. (2005). Profiles of pitch classes circularity of relative pitch and key – experiments, models, computational music analysis, and perspectives, Ph.D. Dissertation. Elektrotechnik und Informatik der Technischen Universität Berlin Radchenko, G. S., Bakhchina, A. V., Parin, S. A., Polevaya, S. A., Korsakova-Kreyn, M. N., & Fedotchev, A. I. (in press). Psychophysical correlates of the perception of motion in tonal space. Repp, B.H. & Su, Y.H. (2013). Sensorimotor synchronization: a review of recent research (2006-2012). Psychonomic Bulletin & Review, 20(3), 403-452 Scruton, R. (1997). The Aesthetics of Music. Oxford University Press Shepard, R. & Cooper, L. (1982). Mental images and their transformations. Cambridge, MA: MIT Press


13

Solntsev, V. M. (1974). К вопросу о семантике, или языковом значении (вместо предисловия) [On a question of semantics or linguistic meaning]. Проблемы семантики [Problems of semantics], Москва: Наука [Moscow: Science] Tillmann, B. & E. Bigand (2004). The relative importance of local and global structures in music perception. Journal of Aesthetics and Art Criticism 62, (2), 211–222 Trainor, L.J. (2004). Are there critical periods for music development? Developmental Psychobiology, 46, 262-278 Virtala, P., Huotilainen, M., Partanen, E., Fellman, V. & Tervaniemi, M. (2013). Newborn infants' auditory system is sensitive to Western music chord categories. Frontiers in Psychology, 4, 492. doi: 10.3389/fpsyg.2013.00492 Wellsby, M. & Pexman, P.M. (2014). Developing embodied cognition: insights from children’s concepts and language processing. Frontiers in Psychology, 5, 1-10

embodied cognition in music

Documents