Gesture Workshop: Gesture Analysis

Applications to Second Language Development

Aug. 10, 2012Steve McCafferty, University of Nevada, Las Vegas, Dept. of

Educational Psychology and Higher Education

Gesture/Gesticulation

• The manual modality just as easily could have been the primary modality for language (Goldin-Meadow & McNeill, 1999).

• Spontaneous gestures are at once both analogic and mimetic (ibid, p. 163).

Gestures/Gesticulations

• Gestures/Gesticulations appear when there is something “newsworthy” as with the introduction of a new character in a narrative, a contrast of some sort, and so on (linked to the psychological predicate by McNeill)

• Gesture/Gestiulations: Movement of the hands and arms

• Gesticulations: Always occur with speech, and are spontaneous and not conventionalized

Gesture

• Pantomime: imagistic portrayal of things and events without speech (includes facial expression and other forms of embodiment)

• Sign language: fully conventionalized lg

• Warning: These categories are not absolute and do overlap at times

Gesticulation

• Global and synthetic: This property contrasts with speech which is linear and segmented•Noncombinatoric: Gesticulations don’t form hierarchical units of meaning without speech (additive). Most follow one another at the clausal level•Context sensitive: Gesticulations highlight what is relative to contexts – the speaker’s image for self, image for interlocutor, image in relation to discourse and setting

Gesticulation

• Timing: Gesticulations are integrated into speech output and synchronize with speech phonologically at the stroke level (the gesture itself). This comes at the peak syllable. Context is also embodied in timing

• Co-expressiveness: gestures and speech come together in a dialectical snythesis of linear and imagistic form to form a whole presentation of meaning in relation to context

• Catchments: gesticulations that become a focal point of discourse

Gesture

• Language-slotted gestures look like gesticulations (the least language-like pole) but differ in how they combine with speech. They occupy a grammatical slot, become part of the syntax of the sentence, and acquire what Saussure (1959) called linguistic value. They fill the slot of speech

• Gestures can also become formal signs as in a work place where there is a lot of noise. There are also classroom management gestures (see Kendon, 2004)

Classification of gesticulations

• Imagistic gestures: Those that depict the shape of an object or activity

– Iconics: Resemble actual object or activity depicted in speech (illustrators)

–Metaphorics: Also concrete but represents an abstract concepts (illustrators)

– Butterworths: Gesticulations when a speaker is trying to recall a word or other verbal expression (not always imagistic)

– Deictics: pointing (can be iconic or abstract)

Classification of gesticulations

- Emblems: cultural gestures (sometimes imagistic, sometimes not)

• Non-imagistic gestures

– Beats: up and down movements that highlight some element of discourse or rhythmic structure of speech

– Gestures of identity: although this category includes emblems, it also includes ways of gesturing that belong to a specific culture or subculture as associated with identity (to inhabit)

Classification of gesticulation

• Warning: Gesture types can be combined so that more than any one gesture type can occur for a single gesture. As such McNeill (2005; 2012) suggests gesture dimensions as opposed to hard categories. Morover, gestures may have elements of emblematicity without being fully conventionalized emblems (McNeill 2012), or catchments can become emblematic within the discourse contexts

Classification of gesticulation

• Additionally, gestures can appear in abbreviated form and carry different levels of materialization, again, depending on contexts (Vygotsky, 1987, called gestures “material carriers”)

• Gestures can also take the point of view of the speaker or be the re-enactment of another person or character in the discourse

Gesture/Gesticulation production

• Preparation: (optional) the limbs or hand(s) move to the position preceding the stroke

• Pre-stroke hold: (optional) the limbs and hands hold before producing the stroke

• Stroke: The meaning unit of gesture – when the gesture occurs in relation to speech

Gesture/Gesticulation production

• Post-stroke hold: (optional)holding the final position of the stroke

• Retraction: (optional) limbs and hands return to rest position

• Gesture Unit: All of the above together (handout)

Gesture/Gesticulation production

• Gestures are 3 dimensional in that they occupy time and space. They can be quick or slow, and depending on culture they typically occupy a small “box” or a large box in space. Japanese is usually kept within a small box, American gestures occupy a somewhat larger box, Spanish more so, and Italians Much more so

Growth Point Hypothesis

• McNeill’s (1992; 2005; 2012) Growth Point Hypothesis: Speaking is more than uttering speech sounds with meaning; more deeply it is also “inhabiting” language (a term from Merleau-Ponty 1962). The GP can be regarded as a hypothesis of how this multilayered inhabitance takes place, using vocal, manual and body movements, all orchestrated by imagery and incorporating context, purpose and thought

Growth Point Hypothesis

• How speech and gesture organize is a dynamic process depending on contexts and is dependent on Vygotsky’s ideas concerning the psychological predicate

• Language is a linear, hierarchical, sequential, abstract contribution to thought and communication

• Gesture is a holistic, imagistic, concrete contribution to thought and communication

Growth Point Hypothesis

• There is a dialectical synthesis of these two opposite modes of semiotic packaging in relation to contexts in order to make meaning spontaneously

Ecosocial seimotics

• How people shape their environment semiotically and how this in turn shapes them as part of the complex, dynamic, and nonlinear process “in which the world and its meanings are actively produced by us” (Thibault 2004, p. 184)

Ecosocial semiotics

• In turn, this conceptualization suggests, as Thibault (2004, p. 234) further argues, that “[m]eaning is not … something which is isolable in just one part of the system. Rather it is a result of the relation between the agent-observer and the overall system of relations on all levels.”

Ecosocial semiotics

• As such, it is further argued by Thibault (2004, p. 176), that “… meaning is stored, not at the level of the individual per se, but at the level of … contextual configurations … which integrate individuals to their ecosocial environment and therefore, to the systems of interpretance that are embedded in these.”

Ecosocial semiotics

• Newcomers are subjected to mastering the “use of objects and symbols, pragmatically, semiotically, and semantically, something that can only be done by participating in socio-cultural practice” (Rosa, 2007, p. 305)

• Newcomers construct a sense of self “as an object among others, as an agent and as an actor” (ibid, p. 308)

Actuations

• How organisms come to act on affordances in the environment is the process of actuation

• Mediational Actuations of Communication: Three stages: (Rosa, 2007, p. 303).

• 1) exposure to an aspect of communication that is the object to be understood

• 2) engaging in the act itself• 3) conventional use of object in social settings

L2 gesture actuation

• A comparative study on the gestures of immigrant and assimilated Southern Italians (Sicilians and Neapolitans) and Eastern European Jews (Polish and Lithuanian) in New York City found that the gestures produced by the assimilated groups differed from those produced by the immigrant groups. The gestures of the assimilated groups had become both similar to one another and generally more American (Effron, 1941/1972)

L2 gesture actuation

• L1 Anglophone children deploy L2 Francophone emblematic gestures when speaking to age-matched L1 French speakers (Von Raffler-Engler, 1976)

• Mohen &Helmer (1988): L2 children acquire L2 emblematic gestures at very young ages

• McCafferty (1998; 2008), McCafferty & Ahmed (2000): acquisition of L2 gestures and gesturing

Bilingual gesture actuation

• Nicoladis (2007) argues that with bilinguals the weaker language includes the use of more deictic gestures than with the stronger language, although she offers no real explanation for this, other gesture types appear to occur at the same level of frequency

L2 classroom learning

• Jungheim (1991) found that comprehension of selected emblems was increased if students were explicitly taught through watching the teacher perform the gestures in the L2 classroom (adult Japanese L2 learners of English in Japan)

•Allen (1995), although researching lexical retention, found that when coupled with emblems (teacher gesturing), students remembered the items tested significantly better than without such coupling

L2 classroom learning

• Tellier (2008) found that when students enacted gestures as portrayed by the teacher along with new vocabulary they retained the vocabulary better than when they did not

L2 classroom learning

• Haught and McCafferty (2008) focused on L2 university students in the U.S., finding that taking a direct dramaturgical approach – verbally rehearsing short scenes of a highly cultural nature – led students to imitate gestures first inadvertently produced by the instructor, a former actor, and then to use them on their own (the scenes were practiced recursively). Only his gestures were copied

L2 classroom learning

• Nardotto Peltier & McCafferty (2010) found that university instructors of Italian in the U.S. who had spent time in Italy engaged in an almost constant use of Italian emblematic gestures as well as Italian ways of gesturing (growth point acquisition as suggested by McNeill, personal communication, October, 2011)

L2 classroom learning

• In follow-up interviews, all of the instructors noted that communicating in Italian goes beyond just speaking, emphasizing the embodied nature of this practice. Students in the study were found to mirror their instructors’ Italian gestures and to gesture in an Italian fashion. Also, instructors had not been video recorded previously and all were surprised at the extent of their Italian gestures.

Motion events

• A number of researchers have investigated the possible acquisition of motion events from one language typology to another in which gesture plays a significant role for verb frame as opposed to satellite frame languages, for example Spanish and English. Results to date are inconclusive

Foreigner talk and gesture

• It appears that some native speakers, when they feel that the L2 interlocutor is of beginning or intermediate proficiency in the L2 will resort to concretizing their gestures to provide explicit reference (sense) for the language learner (McCafferty & Rosborough, 2008; Stam & Tellier, forthcoming)

Foreigner talk and gesture

• Characteristics of FT often include the following:

–Simplified grammatical structures

–Slowed speech

–Limited vocabulary

–Exaggerated intonation

–Rise in voice pitch

Foreigner talk and gesture

• “For the party performing a gesture, a gesture does not represent at all; the gesture is created by the speaker as a materialization of meaning” (p. 58).

• “By performing the gesture, a core idea is brought into concrete existence and becomes part of the speaker’s own existence at the moment” (p. 99).

Foreigner talk and gesture

• Mimesis is associated with the terms “representation” and “imitation” (Maran, 2003).

• “[Mimesis] manifests itself in pantomine, imitation, gesturing, sharing attention, ritualized behaviors, and many games” (Donald 2001, p. 40).

Foreigner talk and gesture

• Linguistically, FT is mimetic: materialization of language - basic structure is brought out - centered in the here and now, grammar is reduced, FT is concrete and iconic when possible.

• FT is in fact primarily mimetic and gesture is the fundamental mode of mimetic representation (Goldin-Meadow & McNeill 1999).