multimodal analysis1

13
Multimodal Interactional Analysis

Upload: bobbadave

Post on 28-May-2015

339 views

Category:

Technology


7 download

DESCRIPTION

Multimodal session

TRANSCRIPT

Page 1: Multimodal analysis1

Multimodal Interactional Analysis

Page 2: Multimodal analysis1

Capturing Different Media

1. Sound2. Video3. Photo4. Observation (Field Notes)

Page 3: Multimodal analysis1

MediaMedia ModesModesSoundSound Speech, music, laughter, background noiseSpeech, music, laughter, background noise

VideoVideo LightingLighting Ambient lighting in Ambient lighting in homes, daylight homes, daylight

Physical presencePhysical presence Posture, facial and Posture, facial and bodily gestures, eye bodily gestures, eye contact, pointingcontact, pointing

Production elementsProduction elements On screen effects, On screen effects, framing of shots, framing of shots, cutting between cutting between scenes, etc.scenes, etc.

Physical objectsPhysical objects Placing objects in field Placing objects in field of view, gesturing of view, gesturing with objects, with objects,

PhotographyPhotography Similar to video but without the Similar to video but without the movement/time aspectsmovement/time aspects

Observation Observation (field notes)(field notes)

Spoken language, physical behaviours, social Spoken language, physical behaviours, social dynamics, printed text/signs, etc. dynamics, printed text/signs, etc.

Page 4: Multimodal analysis1

Multimodal Activity

Observe and record examples of how technologies influence the way that people relate to one another

Use one of the following media: 1.Sound Recording2.Observation (Field Notes)3.Still Photos4.Video Recording

Page 5: Multimodal analysis1

Things to look out for?Reciprocity in communication

coordination of turn taking in conversation‘gift’ exchange

Ritualised behaviour openings and closings

Defining the ‘meaning’ of physical spaceWhat behaviours are

allowed/expected/normsReinforcement of these norms

Divided Consciousness/Absent PresenceDealing with immediate and distant social

contact simultaneously

Page 6: Multimodal analysis1

Feedback SessionWhat modes have been captured?What different stories do they tell?Were there any issues with collecting/using

particular media?

Page 7: Multimodal analysis1

1. Watch the videos/listen to recordings/view images

2. Identify themes◦ Observe and identify significant moments that highlight

multimodal interaction as an aspect of communication – note the time

◦ Refer to existing literature on multimodal aspects of communication to check what you see and hear – for example Goffman (1959)

3. Watch the videos4. Describe the multimodal aspects of the videos5. Roughly transcribe the video data to clarify significant

moments in communication and identify commonalities across videos

6. Watch the videos7. Final transcription in a suitable form to explain themes

and subthemes◦ Include standard annotation conventions

Developing Multimodal Themes

Page 8: Multimodal analysis1

Multimodal transcription of video data (using my own coding scheme)

Page 9: Multimodal analysis1

Multimodal Transcription

www.sigridnorris.com

Page 10: Multimodal analysis1

Multimodal transcription – from conversational analysis:

(n) pause noted in seconds = joining of words : an extension of the preceding syllable; ; rising or falling intonation for subsequent

utterance underlined text indicates spoken with

particular vocal energy - a sudden cut off to an utterance; ? a rising tone . a falling tone , a shifting continuous tone bridging

utterances.

Page 11: Multimodal analysis1

Multimodal Description

Page 12: Multimodal analysis1
Page 13: Multimodal analysis1

ReferencesClark, H. H. (1992). Arenas of language use. Chicago: The

University of Chicago Press.Clark, H. H., & Schaefer, E. F. (1989). Contributing to

discourse. Cognitive Science, 13, 259–294Kendon, A., Harris, R. M., & Ritchie Key, M. (1975).

Organization of behavior in faceto-face interaction. Paris: Mouton Publishers.

Kendon, A. (1990). Conducting interaction: Patterns of behavior in focused encounters.Cambridge: Cambridge University Press.

Goffman, E. (1963). Behavior in public places: Notes on the social organization of gatherings. New York: The Free Press.

Ten Have, P. (1999). Doing conversation analysis. A practical guide. London: Sage Publications.