Eye tracking for linguistics: Day 2

Office hours:

today: 3:30 - 4:30 tomorrow: 3:30 - 4:30 (if these times work I’ll keep using them all month)

Location:

MDS 160 (phonetics lab) 725 Rose Street (across the intersection from JSB)

Weaknesses of eye tracking

• Delicate experimental design: Creating a task to investigate a particular question in a new way can be challenging

• Complicated data analysis: Eye trackers generate a tremendous amount of data and statistical analysis can be difficult (but see: Growth Curve Analysis)

• Expensive: Fast eye trackers are not very affordable and affordable eye trackers are not very fast

• Slow data collection: It is faster and easier to collect and analyze reaction time data. This is why eye tracking is so often part of a follow-up experiment rather than an initial exploration.

Strengths of eye tracking

• Behavioral: eye movements can be explicitly linked to task goals (c.f. brain imaging techniques)

• Implicit: usually not subject to awareness or conscious control

• Fast: 2000 Hz trackers are as fast as button boxes

• Continuous: lets us look at the time course of task behavior

• Natural: Allows comparison across populations • Across development (infants, toddlers, adults, the oldest old) • Across patient populations • Across cultural divides…

Visual world + video chat! Eye tracking is fast

• Colleagues and I ran into a problem with a reaction time experiment at Stanford

• We measured semantic priming of (fast) casual speech

• Listeners’ responses to the baseline stimuli were already so fast that it appeared they simply could not respond faster when provided with a prime

• In other words, this could be a floor effect (sometimes ‘basement’ effect) OR it could be that casual speech isn’t as useful/informative to listeners

area of interest (AOI)

0 2.332-0.99

0.7281

0.7281

-0.990 2.3321.28

The cow was milked in the barn

T h e c o w w a s m i l k e d i n t h e b a r n

Auditory Stimulus

Participants: 48 Stanford University undergraduates Between Subjects Design: 8 Careful or Casual stimulus lists Stimuli: Sentences from experiments 1 & 2, images from Snodgrass & Vanderwart (1980)

Interest Period

Eye tracking removes floor effect

Mean looks to targetMean looks to distractors

Foveal vision Visual processing

The fovea is roughly 1% of the surface of the retina but takes up more than 50% of the visual cortex

http://xkcd.com/1080/

Visual processing

• The human visual field is roughly 220º

• Foveal vision : information is reaching the most sensitive portion of the retina (approximately 2º (1º around the center fixation point)

• Parafoveal vision: 10º circle around the fovea. In written languages (left to right), the next word to the right is called the “parafoveal word” and influences processing of the foveal word.

• Peripheral vision: (aka perifoveal vision) is mainly good at detecting movement and contrasts

Visual Processing: Smooth pursuit

• Smooth pursuit : Slow eye movements that stabilize the image of a slowly moving target on or near the fovea

• Typical speed is < 30°/s, the eye movements are initiated within 90-150 ms

• Above 30°/s smooth pursuit requires catch up saccades

• We perform better when we follow objects moving horizontally, than vertically

• Other objects beside the target are poorly processed

Attention & Eye Tracking

• Attention and eye movements are not the same thing (e.g. Rayner, 1998; 374)

• Although eye movements and attention are normally coupled, they can be decoupled

• particularly in simple discrimination tasks (Posner, 1980) reading and complex tasks are less susceptible to this

• attentional movement often precedes a saccade

• we can fixate without attending

Linking Hypothesis

• When we design an experiment there is always a linking hypothesis that maps the thing we can actually measure onto the theoretical construct we hope to better understand

• Grammaticality judgments

• Self-paced reading

• Categorical perception AXB

• Ultrasound tongue images

• others??

Linking Hypothesis for Eye Tracking

• What, according to Tanenhaus, Magnusun, Dahan, and Chambers (2000) is the linking hypothesis in eye tracking?

• Well… it depends, right? On how well we understand the task, how constrained the thing we are trying to model, how well-developed theory is.

• Sentence Processing?

• Reading?

• Lexical Access?

Linking Hypotheses for Eye Tracking

• One thing that might not be clear from Tanenhaus et al. (but is definitely implicit) is that hypothesis testing experiments can be used to develop and refine linking hypotheses between eye movements and linguistic processing

Linking Hypotheses for Eye Tracking

• If our theory predicts a particular type of eye movement and we observe a completely different type of movement, there are many possibilities:

• Our theory is wrong

• Our stimuli were badly developed/chosen

• We happened onto an uncharacteristic group of participants

• The participants used one or more task-specific strategies that interfere with our predictions

• The visual system doesn’t work the way we expected it to and we need to develop a different task to better understand this initial result

Linking Hypotheses for Eye Tracking

• That sais, If our theory predicts a particular type of eye movement and we observe precisely that type of movement, there are still many possibilities:

• Our theory is right

• Our stimuli were badly developed/chosen

• We happened onto an uncharacteristic group of participants

• The participants used one or more task-specific strategies that interfere with our predictions

• The visual system doesn’t work the way we expected it to and we need to develop a different task to better understand this initial result

Task-specific strategies

Task-Specific Strategies

• What were the task-specific strategies Tanenhaus et al. were concerned about?

• Life rarely presents us with a set of 4 to 8 pictures and restricts what can be said about them

• ( Why is this a problem? )

• ( What do we do about it? )

(aside: how did I get those images?)

Finding, creating, or buying

• Image libraries exist that have been used in many studies (e.g. Snodgrass image database, many others)

• I have hired cartoonists, caricature artists, and graphic designers to create stimuli of my design (e.g. Beddor et al. 2012)

• Google image search (and similar search engines) can be used to find appropriately-licensed images for use in research

Norming

• Once collected, images need to be normed for visual attractiveness, size, colors, and any other variables that matter to your study

• Norming can be done easily in the laboratory (perhaps as an add-on or distractor task for some other study?)

• I typically use Amazon’s Mechanical Turk because I can get hundreds of judgments in an afternoon

• word association

• naming

• ratings on various scales

• etc.

Norming

Our interests

• speech perception (x 5)

• phonetics (x 2)

• phonology (x 2)

• sentence processing (x 5)

• reading

• L2 reading

• L2 sentence processing

• orthography

• acquisition

• competing form-meaning mappings

• discourse level processing

• multilingualism

• pronoun resolution

• psycholinguistics

• sign language

• sociophonetic perception

• testing Sapir-Whorf hypothesis

Task-specific strategies

Choosing a research question Syllabus change?