two /b/ or not “too bee”:
DESCRIPTION
Two /b/ or not “too bee”: Gradient sensitivity to subphonemic variation, categorical perception and the effect of task. Bob McMurray. With thanks to. Outline. Invariance, Covariance and Gradient Sensitivity in speech perception. Categorical Perception and other previous research. - PowerPoint PPT PresentationTRANSCRIPT
Two /b/ or not “too bee”:
Gradient sensitivity to subphonemic variation,
categorical perception and the effect of task.Bob McMurray
Michael K. Tanenhaus Michael J. Spivey
Richard N. Aslin Dana Subik
With thanks to
• Invariance, Covariance and Gradient Sensitivity in speech perception.
• Categorical Perception and other previous research.
• Experiment 1: Gradient sensitivity in Word Recognition
• Experiment 2-5: The effect of experimental task
• Targets & competitors, gradient sensitivity and temporal dynamics,
• Conclusions
OutlineOutline
Problem of InvarianceProblem of InvarianceProblem of InvarianceProblem of Invariance
Phonetic features are correlated with many acoustic realizations.
Acoustic realization of a phonetic feature depends on context.
How do we extract invariant linguistic representations from a variable acoustic signal?
What properties of the signal provide an invariant mapping to linguistic representations?
How do we extract discrete units from a graded signal?
Problem of InvarianceProblem of InvarianceProblem of InvarianceProblem of Invariance
Two Solutions
• Motor Theory: acoustic invariance does not exist, but specialized mechanisms allow us to unpack speech into invariant motor representations (Liberman & Mattingly, 1985; Fowler, 1986).
• Acoustic Invariance: better computational methods and neurologically inspired models may find invariant acoustic properties of the signal (Blumstein,1998; Sussman et al, 1998)
The Fundamental Approach How do we pay attention to the right
parts of the signal and ignore the variation?
However, recent work suggests that this the variation is actually highly informative covariation.
Rethinking InvarianceRethinking InvarianceRethinking InvarianceRethinking Invariance
In measurements of productions, effects of
• speaking rate on VOT (e.g. Kessinger & Blumstein)• prosodic domain and VOT and articulatory
strength (Fougeron and Keating)• Place of articulation and vowel quality 5 syllables away (Local)• Between-consonant coarticulation (Mann & Repp)
suggest that a system sensitive to fine grained detail could take advantage of all of this information.
Rethinking InvarianceRethinking InvarianceRethinking InvarianceRethinking Invariance
Speech perception shows probabilistic effects of many information sources:
Lexical Context Spectral vs. Temporal CuesVisual Information Transition StatisticsSpeech Rate Stimulus NaturalnessSentential Context Compensatory
CoarticulationEmbeddings Syllabic StressLexical Stress Phrasal Stress
A system that was sensitive to fine-grained acoustic detail might be much more efficient than one that was not.
Tracking covariance may help solve the problem of invariance.
Rethinking InvarianceRethinking InvarianceRethinking InvarianceRethinking Invariance
What sort of sensitivity is What sort of sensitivity is needed? needed? What sort of sensitivity is What sort of sensitivity is needed? needed?
Gradient Sensitivity:
As fundamentally graded acoustic information changes (even changes that still result in the same “category”), activation of lexical or sublexical representation changes monotonically.
Activation of linguistic units reflects the probability that a that unit is instantiated by the acoustic signal.
Categorical PerceptionCategorical PerceptionCP suggests listeners do not show gradient
sensitivity to subphonemic information.
• Sharp identification of speech sounds on a continuum
ID (%/pa/)
VOT
0
100
PB
% /
p/B
P
Discrimination
• Discrimination poor within a phonetic category
Evidence for Categorical PerceptionEvidence for Categorical Perception
Supported by:
• Work on VOT and place of articulation.
• Ubiquity of steep identification functions.
• Recent electrophysiological data (e.g. Philips, Pellathy, Marantz, Yellin, Wexler, Poeppel, McGinnis & Roberts, 2000; Sharma & Dorman, 1999)
Revisiting Categorical Perception?Revisiting Categorical Perception?Evidence against CP comes from
Discrimination Tasks Pisoni and Tash (1974) Pisoni & Lazarus
(1974)Carney, Widin & Viemeister (1977)
Training Samuel (1977)Pisoni, Aslin, Perey & Hennessy (1982)
Goodness Ratings Miller (1997) Massaro & Cohen,
1983
Only goodness ratings show any hint of gradiency.No gradient effects from identification tasks.
But, 2AFC metalinguistic tasks may underestimate sensitivity to subphonemic acoustic information
Lexical sensitivityLexical sensitivity
Andruski, Blumstein & Burton (1994)
Created stimuli that were either voiceless, 1/3 or 2/3 voiced.
2/3 voiced stimuli primed semantic associates more weakly than fully voiceless or 1/3 voiced tokens
First demonstration of lexical sensitivity to natural variation in consonants.
However:• 2/3 voiced stimuli were close to category boundary.• No evidence for gradiency—difference between 2
items.• Hard to interpret temporal dynamics in priming
tasks.
Remaining QuestionsRemaining QuestionsRemaining QuestionsRemaining Questions
•Is sensitivity to subphonemic differences gradient?
•Is it symmetrical (I.e. gradiency on both sides of category boundary)?
•Are differences preserved long enough to be usefully combined with subsequent input?
Perhaps a more sensitive measure….
250 Hz realtime stream of eye positions.
Parsed into Saccades, Fixations, Blinks, etc…
Head movement compensation.
Output in ~screen coordinates.
Head-Tracker Cam Monitor
IR Headtracker Emitters
EyetrackerComputer
SubjectComputer
Computers connected via Ethernet
2 Eye cameras
Head
Eye-TrackingEye-TrackingEye-TrackingEye-Tracking
Eye-TrackingEye-TrackingEye-TrackingEye-Tracking
Fixations to object in response to spoken instructions:
•are time locked to incoming information (Tanenhaus, Spivey-Knowlton, Ebehart and Sedivy, 1995)
•can be easily mapped onto lexical activation from models like TRACE (Allopenna, Magnuson and Tanenhaus, 1998)
•show effects of non-displayed competitors (Dahan, Magnuson, Tanenhaus & Hogen)
•provide a glimpse at how activation for competitors unfolds in parallel over time.
Experiment 1Lexical Identification
“too bee”
Experiment 1Lexical Identification
“too bee”
Can we use eye-tracking methodologies to find evidence for graded perception of VOT?
Experiment 1: Lexical IdentificationExperiment 1: Lexical Identification
Six 9-step /ba/ - /pa/ VOT continuum (0-40ms)Bear/Pear Beach/PeachButter/Putter Bale/PaleBump/Pump Bomb/Palm
12 L- and Sh- Filler itemsLeaf Lamp Ladder LockLip Leg Shark ShipShirt Shoe Shell Sheep
Identification indicated by mouse click on pictureEye movements monitored at 250 hz17 Subjects
A moment to view the items
Experiment 1: Lexical IdentificationExperiment 1: Lexical Identification
500 ms later
Experiment 1: Lexical IdentificationExperiment 1: Lexical Identification
Experiment 1: Lexical IdentificationExperiment 1: Lexical Identification
Bear
Experiment 1: Identification ResultsExperiment 1: Identification Results
By subject: 17.25 +/- 1.33ms By item: 17.24 +/- 1.24ms
0 5 10 15 20 25 30 35 400
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
VOT (ms)
pro
port
ion
/p
/
B P
High agreement across subjects and items for category boundary
Analysis of fixationsAnalysis of fixations
0 5 10 15 20 25 30 35 400
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
VOT (ms)
pro
port
ion
/p
/
B P
Actual Exp2 Data
Trials with low-frequency response excluded.
ID Function after filtering
Yields a “perfect” categorization function.
+
Target = bug
Competitor = bus
Unrelated = cat, fish
Time
200 ms
1
2
3
4
5
Trials
Analysis of fixationsAnalysis of fixations
Experiment 1: Eye Movement ResultsExperiment 1: Eye Movement Results
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
0 400 800 1200 1600 0 400 800 1200 1600 2000
Time (ms)
More looks to competitor than unrelated items
VOT=0 Response= VOT=40 Response=
Fix
ati
on
p
rop
ort
ion
e.g. Given that • the subject heard bomb• clicked on “bomb”…
time
Fix
ati
on
pro
port
ion
target
competitor
How often was the Subject looking at the “palm”?
time
Fix
ati
on
pro
port
ion
target
competitor
Categorical Results Gradient Effect
Analysis of fixationsAnalysis of fixations
Gradient “competitor” effects
0 400 800 1200 16000
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0 ms5 ms10 ms15 ms
VOT
0 400 800 1200 1600 2000
20 ms25 ms30 ms35 ms40 ms
VOT
Fix
ati
on
pro
port
ion
Time since word onset (ms)
Experiment 1: Eye Movement ResultsExperiment 1: Eye Movement Results
Smaller effect on the amplitude of activation—more effect on the duration:
Competitors stay active longer as VOT approaches the category boundary.
Response= Response=
Gradient competitor effects of VOT?
Experiment 1: Gradiency?Experiment 1: Gradiency?
0 5 10 15 20 25 30 35 400.02
0.03
0.04
0.05
0.06
0.07
0.08
VOT (ms)
Fix
ati
on
p
rop
ort
ion
Looks to Looks to
Gradient Sensitivity
“Categorical” Perception
Andruski et al (schematic)
0 5 10 15 20 25 30 35 400.02
0.03
0.04
0.05
0.06
0.07
0.08
Experiment 1: Eye Movement ResultsExperiment 1: Eye Movement Results
VOT (ms)
Fix
ati
on
p
rop
ort
ion
CategoryBoundary
B: p=.017* P: p<.0001***Clear effects of VOT
Response= Response=
Looks to
Looks to
Linear Trend B: p=.023* P: p=.002**
0 5 10 15 20 25 30 35 400.02
0.03
0.04
0.05
0.06
0.07
0.08
Experiment 1: Eye Movement ResultsExperiment 1: Eye Movement Results
VOT (ms)
Fix
ati
on
p
rop
ort
ion
CategoryBoundary
Response= Response=
Looks to
Looks to
B: p=.017* P: p<.0001***Clear effects of VOTUnambiguous Stimuli Only
Linear Trend B: p=.023* P: p=.002**
Experiment 1: Results and ConclusionsExperiment 1: Results and Conclusions
Subphonemic acoustic differences in VOT affect lexical activation.
• Gradient effect of VOT on looks to the competitor.
• Effect seems to be long-lasting (we’ll get back to that).
• Effect holds even for unambiguous stimuli.
Conservative Test• Filter out “incorrect” responses.• Use unambiguous stimuli only.
Why was it so hard to find evidence for gradiency in CP tasks?
However…However…
* Steep identification function consistently replicated.
What aspects of the task affect our ability to see gradient sensitivity?
• Phoneme ID vs. Lexical ID?• Number of Alternatives?• Type of Stimuli? • Sensitivity of response measure
Experiment 2Categorical Perception
2 /b/, not “too bee”
Experiment 2Categorical Perception
2 /b/, not “too bee”
What can the eye-tracking paradigm reveal about ordinary phoneme identification experiments?
Replicates “classic” task:
9-step /ba/ - /pa/ VOT continuum (0-40ms)
2AFC Identification indicated by mouse click.
Eye movements monitored at 250 hz.
17 Subjects
Experiment 2: Categorical PerceptionExperiment 2: Categorical Perception
1
2
B P Ba3
Experiment 2: Categorical PerceptionExperiment 2: Categorical Perception
Experiment 2: Identification ResultsExperiment 2: Identification Results
0 5 10 15 20 25 30 35 400
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
VOT (ms)
pro
port
ion
/p
/
Exp 2: BPExp 1: Words
Phoneme ID function steeper
BP: 17.5 +/- .83ms Wordssubject:17.25 +/-1.33ms Wordsitem: 17.24 +/- 1.24ms
Boundaries
Category boundaries are the same.
B P
0 5 10 15 20 25 30 35 400
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
VOT (ms)
Pro
port
ion
of
/p/
resp
on
se
Experiment 2: Data Analysis Experiment 2: Data Analysis
Trials with low-frequency response excluded.Effectively yields a “perfect” categorization function.
Actual ID FunctionEffective ID Function
Time (ms)0 400 800 1200 1600
0
0.05
0.1
0.15
0.2
0.25
0.3
0 400 800 1200 1600 2000
0 ms5 ms10 ms15 ms
VOT20 ms25 ms
VOT
30 ms35 ms40 ms
Some hints of gradiency for /p/. Even less for /b/.• Difference between stimuli near boundary and endpoints.• Perhaps more for /p/.
Experiment 2: Eye movement dataExperiment 2: Eye movement data
Fix
ati
on
p
rop
ort
ion
Response = B Response = P
Looks to P Looks to B
Experiment 2: Eye movement dataExperiment 2: Eye movement data
/b/: p =.044* ptrend=.055
/p/: p<.001*** ptrend=.005***
Could be driven by differences near category boundary.
0 5 10 15 20 25 30 35 400
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
VOT (ms)
CategoryBoundary
Response=BLooks to P
Response=BLooks to P
Fix
ati
on
p
rop
ort
ion
Experiment 2: Eye movement dataExperiment 2: Eye movement data
Unambiguous Stimuli Only/b/: p =.884 ptrend=.678/p/: p =.013* ptrend=.003***
0 5 10 15 20 25 30 35 400
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
VOT (ms)
CategoryBoundary
Response=BLooks to P
Response=BLooks to P
Fix
ati
on
p
rop
ort
ion
Experiment 2: Results and ConclusionsExperiment 2: Results and Conclusions
•Very steep slope for mouse response curves. consistent with traditional results
•Identical category boundary to experiment 1
validates stimuli
•Small difference between stimuli near category boundary and others.
similar to Pisoni & Tash, Andruski, et al.
•Gradient effect weak for /ba/, moderate for
/pa/
Experiment 3Number of Response Alternatives
Not 2 but /b/?
Experiment 3Number of Response Alternatives
Not 2 but /b/?
compare to experiment 2 (BaPa)
Experiment 3: BaPaLaShaExperiment 3: BaPaLaSha
Given the strong evidence for gradiency in Experiment 1 and the weaker evidence in Experiment 2, what is the effect of number of response alternatives?
• Same 9-step /ba/ - /pa/ VOT continuum (0-40ms) as experiment 2.
• La and Sha filler items added.
• 4AFC Identification indicated by mouse click. Button locations randomized between
subjects.
• Eye movements monitored at 250 hz.
•17 Subjects
Experiment 3: BaPaLaShaExperiment 3: BaPaLaSha
P
B Sh
L
La
0 5 10 15 20 25 30 35 400
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
VOT
pro
po
rtio
n /
p/
resp
on
se
Experiment 3: Identification ResultsExperiment 3: Identification Results
Exp. 2 (BaPa)
Exp. 1 (words)
Exp. 3 (BaPaLaSha)
Number of response alternatives accounts for some of the difference in slope.
Effective ID Function
0 5 10 15 20 25 30 35 400
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
VOT (ms)
Pro
port
ion
of
/p/
resp
on
se
Experiment 3: Data Analysis Experiment 3: Data Analysis
Trials with low-frequency response excluded.Effectively yields a “perfect” categorization function.
Actual ID Function
More looks to competitor than unrelated stimuli (p<.001).• Eye movements in “phoneme ID” tasks are sensitive
to acoustic similarity.
Experiment 3: Eye movement dataExperiment 3: Eye movement data
0 400 800 1200 1600
0 400 800 1200 1600 20000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Time (ms)
VOT=0 Response=b VOT=40 Response=P
Fix
ati
on
p
rop
ort
ion
B
P
UR
Time (ms)
0 ms5 ms10 ms15 ms
VOT
Difference between stimuli near boundary and endpoints
Experiment 3: Eye movement dataExperiment 3: Eye movement data
Fix
ati
on
p
rop
ort
ion
0 400 800 1200 16000
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0 400 800 1200 1600 2000
20 ms25 ms
VOT
30 ms35 ms40 ms
Response = B Response = P
Looks to P Looks to B
Experiment 3: Eye movement dataExperiment 3: Eye movement data
Close but no star: Nothing reaches significance/b/: p=.055 ptrend=.068
/p/: p=.510 ptrend=.199
0 5 10 15 20 25 30 35 40
VOT (ms)
CategoryBoundary
Response=BLooks to P
Response=BLooks to P
Fix
ati
on
p
rop
ort
ion
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
Experiment 3: Eye movement dataExperiment 3: Eye movement data
Unambiguous Stimuli Only: even worse
/b/: p=.374 ptrend=.419
/p/: p=.356 ptrend=.151
0 5 10 15 20 25 30 35 40
VOT (ms)
CategoryBoundary
Response=BLooks to P
Response=BLooks to P
Fix
ati
on
p
rop
ort
ion
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
Eye movements in phoneme ID tasks are sensitive to acoustic similarity between target and competitor.
Number of alternatives explains some of differences in ID function.
VERY weak subphonemic effects on lexical activation.
Experiment 3: ResultsExperiment 3: Results
Experiment 4Response Type
“too” /b/
Experiment 4Response Type
“too” /b/
Is there a difference between phoneme and lexical identification tasks?
compare to experiment 1 (words)
Experiment 4: Response TypeExperiment 4: Response Type
• Same 6 VOT continua (0-40ms) as experiment 1
beach/peach bear/pear bomb/palm
bale/pail bump/pump butter/putter
• Same 12 L- and Sh- filler items.
• 4AFC phoneme identification indicated by mouse click. Button locations randomized between subjects.
• Eye movements monitored at 250 hz.
• 17 Subjects
Experiment 4: Response TypeExperiment 4: Response Type
P
B Sh
L
Ship
VOT
pro
po
rtio
n /
p/
resp
on
se
Experiment 4: Identification ResultsExperiment 4: Identification Results
0 5 10 15 20 25 30 35 400
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Exp. 2 (BaPa)
Exp. 1 (words)
Exp. 4 (Response Type)
Similar category boundary and slope to Exp 1 Exp 1: 17.25 +/- 1.33ms Exp 4: 16.34 +/- 1.52ms
0 400 800 1200 16000
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0 400 800 1200 1600 2000
0 ms5 ms10 ms15 ms
VOT20 ms25 ms
VOT
30 ms35 ms40 ms
Time (ms)
Fix
ati
on
p
rop
ort
ion
Experiment 4: Eye movement dataExperiment 4: Eye movement data
Small differences in the right direction
Response = B Response = P
Looks to PLooks to B
Experiment 4: Eye movement dataExperiment 4: Eye movement data
Gradient effects using the whole range of stimuli/b/: p<.001 ptrend=.002
/p/: p=.001 ptrend=.031
0 5 10 15 20 25 30 35 40
VOT (ms)
CategoryBoundary
Fix
ati
on
p
rop
ort
ion
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.1
Response=BLooks to B Response=P
Looks to B
Experiment 4: Eye movement dataExperiment 4: Eye movement data
Marginal effects using “unambiguous” stimuli only.
/b/: p=.074 ptrend=.074
/p/: p=.137 ptrend=.108
0 5 10 15 20 25 30 35 40
VOT (ms)
CategoryBoundary
Fix
ati
on
p
rop
ort
ion
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.1
Response=BLooks to B Response=P
Looks to B
Weaker subphonemic effect suggests that offline “metalinguistic” tasks are less sensitive to fine-grained phonetic detail than online tasks.
Some detail is preserved in these tasks (at least with word stimuli)…
Experiment 4: ResultsExperiment 4: Results
Experiment 52AFC Words
2 “bee”
Experiment 52AFC Words
2 “bee”
Bringing it all together
Experiment 5: 2-WordsExperiment 5: 2-Words
Is the difference in ID curve slopes purely the result of number of response alternatives or does task play a role?
• Same 6 VOT continua (0-40ms) as experiment 1
beach/peach bear/pear bomb/palm
bale/pail bump/pump butter/putter
• 0 filler items.
• 2AFC phoneme identification indicated by mouse click.
• Eye movements monitored at 250 hz.
Pear
Experiment 5: TaskExperiment 5: Task
VOT
pro
po
rtio
n /
p/
resp
on
se
Experiment 5: Identification ResultsExperiment 5: Identification Results
0 5 10 15 20 25 30 35 400
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Exp. 2 (BaPa)
Exp. 1 (words)
Exp. 5 (2-words)
Similar category boundary and slope to Exp 1 Exp 1: 17.25 +/- 1.33ms Exp 5: 16.18 +/- 1.74ms
B P
0 ms5 ms10 ms15 ms
VOT20 ms25 ms
VOT
30 ms35 ms40 ms
Time (ms)
Fix
ati
on
p
rop
ort
ion
Experiment 5: Eye movement dataExperiment 5: Eye movement data
0 400 800 1200 16000
0.05
0.1
0.15
0.2
0.25
0.3
0 400 800 1200 1600 2000
Response= Response=
Clean, but small, gradient effects for /p/Effects for /b/ near the category boundary.
Experiment 5: Eye movement dataExperiment 5: Eye movement data
Gradient effects using the whole range of stimuli/b/: p<.001 ptrend=.005
/p/: p=.017 ptrend=.026
0 5 10 15 20 25 30 35 40
VOT (ms)
CategoryBoundary
Fix
ati
on
p
rop
ort
ion
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.1
Response=
Looks to
Response=
Looks to
Experiment 5: Eye movement dataExperiment 5: Eye movement data
Weaker effects using the “prototypical” range/b/: p<.443 ptrend=.802
/p/: p=.044* ptrend=.052
0 5 10 15 20 25 30 35 40
VOT (ms)
CategoryBoundary
Fix
ati
on
p
rop
ort
ion
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.1
Response=
Looks to
Response=
Looks to
Shallow ID curve slope suggests 2AFC alone is not enough to create steep slope: 2AFC- phoneme ID is needed.
Weaker gradient effects: fixed response locations and no filler items make this task more “explicit”?
Experiment 5: ResultsExperiment 5: Results
Trying to make sense out of it all…
Being and Nothingness?
Trying to make sense out of it all…
Being and Nothingness?
Slope of ID FunctionSlope of ID Function
Exp 1 (words)
Exp 2 (BaPa)
Exp 3 (BaPaLaSha)
Exp 4Exp 5
(2 Words)
-1
0
1
2
Slo
pe
BP > BaPaLaSha > all others (p<.05)Words ~= Exp 4 ~= 2 Words (p>.1)
2AFC results in less sensitivity (in ID function) than 4AFC for non-word stimuli.
Gradient Effect across experimentsGradient Effect across experiments
1 Words
2BP
3 BPLS
4 Phoneme
ID
5 2
words
B ?
P X
B X X ? X
P X X
Allstimuli
Without stimuli
near c.b.
Pooled eye movement dataPooled eye movement data
0 5 10 15 20 25 30 35 40
VOT (ms)
Fix
ati
on
p
rop
ort
ion
Response=BLooks to P
Response=PLooks to B
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
CategoryBoundary
Overall B: pvot<.001 ptrend<.001 pvot x exp>.15P: pvot<.001 ptrend<.001 pvot x exp>.2
0 5 10 15 20 25 30 35 40
VOT (ms)
Fix
ati
on
p
rop
ort
ion
Response=BLooks to P
Response=PLooks to B
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
CategoryBoundary
Without stimuli near category boundaryB: pvot=.005 ptrend<.019 pvot x exp>.1P: pvot<.001 ptrend<.001 pvot x exp>.2
Pooled eye movement dataPooled eye movement data
Conclusions on Task ManipulationsConclusions on Task Manipulations
Phoneme ID tasks with non-words yield the sharpest categorization functions—may mask subphonemic sensitivity.
Even within these tasks, the number of response alternatives makes a big difference.
Identification Functions
“Natural”, 4AFC lexical identification provides cleanest evidence for gradiency (measured by fixations to the competitor) for both /p/ and /b/ halves of continuum.
Conclusions on Task ManipulationsConclusions on Task ManipulationsCompetitor Effects (eye-movements)
All experiments offer evidence of subphonemic sensitivity when we include stimuli near the category boundary.
Eye-movements provide much more sensitive measure for assessing the role of fine-grained phonetic detail.
Most experiments showed weak evidence for gradient effect, but larger effects for /p/ than /b/.
Conclusions on Task ManipulationsConclusions on Task ManipulationsCompetitor Effects (eye-movements)
• Differences in the variance of the distribution of /b/ and /p/ in the learning environment? (Lisker & Abramson, Gerken & Maye)
• Auditory locus? Double peaked firing in
auditory cortex differs shows more VOT sensitivity to voiceless than voiced stops. (Steinshneider et al; Sharma et al)
No one factor seems to account for presence or absence of gradient effect.
Targets and Competitors, Gradient Effects and Temporal Dynamics
Targets and Competitors, Gradient Effects and Temporal Dynamics
and a return to experiment 1
Targets and CompetitorsTargets and Competitors
Why look at exclusively at the competitor?
Do subphonemic differences affect activation of the target?
Andruski et al suggests it does.
Experiment 1: Target ActivationExperiment 1: Target Activation
0 400 800 1200 16000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0 400 800 1200 1600 2000
0 ms5 ms10 ms15 ms
VOT20 ms25 ms
VOT
30 ms35 ms40 ms
Response= Response=
Time (ms)
Fix
ati
on
p
rop
ort
ion
Target effects much weaker, even in Experiment 1May be limited to range near category boundary.
Experiment 1: Target ActivationExperiment 1: Target Activation
0 5 10 15 20 25 30 35 40
VOT (ms)
CategoryBoundary
Fix
ati
on
p
rop
ort
ion
Response=
Looks to
Response=
Looks to
0.4
0.45
0.5
0.55
0.6
0.65
0.7
Overall B: pvot=.035 ptrend=.103P: pvot<.001 ptrend<.010
Experiment 1: Target ActivationExperiment 1: Target Activation
0 5 10 15 20 25 30 35 40
VOT (ms)
CategoryBoundary
Fix
ati
on
p
rop
ort
ion
Response=
Looks to
Response=
Looks to
0.4
0.45
0.5
0.55
0.6
0.65
0.7
Unambiguous Stimuli OnlyB: pvot=.44 ptrend=.98P: pvot=.33 ptrend=.22
Target Activation: ConclusionsTarget Activation: Conclusions
Target sensitivity to subphonemic differences is carried by differences between ambiguous and prototypical stimuli.
Consistent with previous research•Andruski et al: 2/3 voicing is close to ambiguous region (~27 ms).
•Pisoni & Tash: increased RT near boundary.
Target Activation: ConclusionsTarget Activation: Conclusions
Gradient sensitivity to subphonemic differences is stronger in competitor activation that target activation.
Consistent with Misuirski, Blumstein, Rissman and Berman (in press)
This makes sense:•Degrading target activation isn’t likely to
be helpful in word recognition.
•Augmenting competitor activation could be very helpful.
Gradiency and TimeGradiency and Time
Phonetic context in speech perception isn’t simultaneous.
• Rate information (vowel length) arrives after consonant.
• Coarticulation occurs across multiple segments.
• Lexical information has a large scope than phonetic information.
Simply tracking graded acoustic features is not enough.
Graded activation of lexical or sublexical units must persist over time to be integrated.
Temporal ambiguity resolutionTemporal ambiguity resolution
The lexical/phonetic identity of a segment can be determined by acoustic features that arrive after the segment in question.
pbrown
The ambiguous first consonant of
is clearly a /b/ after hearing ”rown”
Thus, like in higher level language comprehension, temporal ambiguity resolution is an important issue.
Temporal ambiguity resolutionTemporal ambiguity resolution
Lexical/Phonetic Temporal Ambiguity can be caused by
• Vowel length (cue to speaking rate and stress)• Lexical/Statistical effects • Embedded words
Subphonemic sensitivity can minimize or eliminate the effects of temporary phonetic ambiguity by
• Storing how ambiguous a segment is• Keeping competitors active until resolution occurs.
Experiment 1: Effect of Time?Experiment 1: Effect of Time?
How long does the gradient sensitivity to VOT remain?
Need to examine:• the effect of time on competitor fixations• interaction with VOT
0 400 800 1200 16000
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0 ms5 ms10 ms15 ms
VOT
0 400 800 1200 1600 2000
20 ms25 ms30 ms35 ms40 ms
VOT
Fix
ati
on
pro
port
ion
Time since word onset (ms)
Experiment 1: Effect of time?Experiment 1: Effect of time?
Time course data suggests that gradiency is sticking around at least 1600 milliseconds after syllable onset.
Response= Response=
Experiment 1: Effect of Time?Experiment 1: Effect of Time?
Analysis:
early late
Trial 1Trial 2 Tria
l 3
Trial 4
Trial 5Trial 6
Trial 7Trial 8
• Randomly sorted trials into two groups (early and
late).
Early
Late
• For each group, fixations from
only 1 time-bin were usedEarly: 300-1100ms
Late: 1100-1900ms
• Ensures independence of data in each time-bin (since each trial only contributes to one)
Experiment 1: VOT x TimeExperiment 1: VOT x Time
0 5 10 15 20 25 30 35 400.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.1
0.11
VOT (ms)
CategoryBoundary
Late (1100-1900ms)
Early (300-1100ms)
Main effect of time /b/: p=.001*** /p/: p=.0001****
Response= Response=
Looks to
Fix
ati
on
p
rop
ort
ion
Main effect of VOT /b/: p=.015* /p/: p=.001***Linear Trend for VOT /b/: p=.022* /p/: p=.009**No Interaction p>.1
Looks to
Experiment 1: VOT x TimeExperiment 1: VOT x Time
0 5 10 15 20 25 30 35 400.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.1
0.11
VOT (ms)
Late (1100-1900ms)
Early (300-1100ms)
Response= Response=
Looks to
Looks to
Fix
ati
on
p
rop
ort
ion
CategoryBoundary
Main effect of time/b/: p=.001*** /p/: p=.0001****Main effect of VOT /b/: p=.006** /p/: p=.013*Linear Trend for VOT /b/: p=.0012** /p/: p=.02**
No Interaction p>.1
Finally some conclusionsFinally some conclusions
Lexical activation exhibits gradient effects of subphonemic (VOT) variation.
Effect is robust and long-lasting—could potentially be very helpful for resolving temporal ambiguity and integrating information over time.
Effect of subphonemic variation is stronger for competitors than targets.
Finally some conclusionsFinally some conclusions
Experimental task is crucial to see sensitivity: more responses + less metalinguistic = more gradiency.
ID Functions influenced by type of stimuli (e.g. words/nonwords) as well as number of response alternatives. Realistic tasks = more gradient ID functions.
Finally some conclusionsFinally some conclusions
Subphonemic variation in VOT is
not discarded
It is not but signal.