Perceptual Simulation of Verbs and Pictures

Michelle VergesIndiana University, South Bend

Sean DuffyRutgers University

Research Question:Do object images and motion words direct one’s spatial attention in mental imagery?

• Interplay between language and perception

• Representational processes between symbols and referents

Perceptual Simulation Spatial Cuing Experiment 1 Experiment 2 Conclusions

Perceptual Symbol Systems Barsalou (1999, 2008)

• Sensorimotor representations that simulate perceptual, motor, and introspective processes

• Integrate constituent features and orientations to form a single, multimodal representation

• Contrasted with amodal symbol systems

Perceptual Simulation Spatial Cuing Experiment 1 Experiment 2 Conclusions

Perceptual-Interference Effects:Mental imagery can interfere with the direct perception of another stimulus

• If mental image and physical stimulus overlap spatially (Craver-Lemley & Arteberry, 2001)

• If mental image and physical stimulus activate different perceptual representations (Estes, Verges, & Barsalou, 2008)

Perceptual Simulation Spatial Cuing Experiment 1 Experiment 2 Conclusions

Spatial Cuing Paradigm

Richardson, Spivey, Barsalou, & McRae (2003) horizontal/vertical sentences

Bergen, Lindsay, Matlock, & Narayanan (2007) up/down sentences

Estes, Verges, & Barsalou (2008) up/down nouns

Perceptual Simulation Spatial Cuing Experiment 1 Experiment 2 Conclusions

+

cue

(ISI)

o

Prior Research Cue

Experiment 1 (N = 28)Object Images and Words

• 32 object images and corresponding labels denoted up or down spatial prime

Up: cloud, flag, hat

Down: foot, whale, snake

Non-spatial: cake, lemon, comb

Perceptual Simulation Spatial Cuing Experiment 1 Experiment 2 Conclusions

Experiment 1 (N = 28)Object Images and Words

• 32 object images and corresponding labels denoted up or down spatial prime

• Primes presented centered of computer screen

• Target letter (X, O) shown at top of bottom of display

• Picture/word conditions counterbalanced Perceptual Simulation Spatial Cuing Experiment 1 Experiment 2 Conclusions

Procedure

Perceptual Simulation Spatial Cuing Experiment 1 Experiment 2 Conclusions

Fixation (250 ms)

hat Prime (100 ms)

ISI (50 ms)

x

Target (respond)

Results:Object Words

Perceptual Simulation Spatial Cuing Experiment 1 Experiment 2 Conclusions

Spatial Prime

Results:Object Images

Perceptual Simulation Spatial Cuing Experiment 1 Experiment 2 Conclusions

Spatial Prime

Experiment 2 (N = 48)Do perceptual-interference effects occur

for motion words?

• Verbs require holistic representations found in literal sentences (Bergen et al., 2007)

• But maybe not (Richardson et al., 2003)

• Verbs serve as the backbone of sentences (Pulvermüller, 2005)

Perceptual Simulation Spatial Cuing Experiment 1 Experiment 2 Conclusions

Experiment 2 (N = 48)

• 32 verbs denoted up or down spatial prime

Up: climb, lift, rise

Down: dig, dive, fall

Non-spatial: choose, draw, tickle

• Procedure identical to Experiment 1Perceptual Simulation Spatial Cuing Experiment 1 Experiment 2 Conclusions

Perceptual Simulation Spatial Cuing Experiment 1 Experiment 2 Conclusions

Results:Motion Words

Spatial Prime

Conclusions

• Object and motion words automatically orient attention to their typical location

• Interference effects due to sensorimotor representations (Barsalou, 2008; Pulvermüller, 2005)

• Object images do not automatically elicit perceptual simulations in mental imagery

Perceptual Simulation Spatial Cuing Experiment 1 Experiment 2 Conclusions

Implications

• Spatial representations elicit dual-coding effects in mental imagery (Paivio 1971, 1986, 2007)

• Developmental processes associated with children’s reading ability

• Long-term effects of perceptual simulation and mental imagery

Perceptual Simulation Spatial Cuing Experiment 1 Experiment 2 Conclusions

Thanks for your attention!

Perceptual Simulation Spatial Cuing Experiment 1 Experiment 2 Conclusions