research questions results - speech learning...
TRANSCRIPT
This work was supported by SSHRC grant 435-2016-0747 to Meghan Clayards.
Participants
• 54 monolingual speakers of Canadian English
Dual task
• 2AFC + visual search
• 2AFC (head or had): 5 spectral (TANDEM-STRAIGHT [7]) x 5 duration steps (PSOLA in Praat)
• Visual search: A black diamond is present?
Gradiency in phoneme categorization
• Visual Analogue Scaling (VAS: heed—hid): 7 spectral x 7 duration steps
Cognitive abilities
• Working memory (Backward Digit Span, Reading Span), Inhibitory control (Stroop, Go/No-go) [8]
Do listeners show adaptive strategies for phonetic categories in the face of cognitive load? If so, what makes some listeners better adaptors?
Speech perception under cognitive load
• Speech perception is an inherently attention demanding process and limited attentional resources have been shown to disruptive effects on speech perception [1, 2].
Cognitive abilities in speech perception
• Cognitive abilities (e.g. inhibitory control, working memory) play a role in speech perception in adverse conditions [3, 4].
Gradiency in phoneme categorization
• Listeners who have more gradient categorization patterns are more sensitive to acoustic-phonetic details [5, 6].
Research Questions
References: [1] Mattys, S. L., & Wiget, L. (2011). Effects of cognitive load on speech recognition. Journal of Memory and Language, 65(2), 145–160. [2] Mitterer, H., & Mattys, S. L. (2017). How does cognitive load influence speech perception? An encoding hypothesis. Attention, Perception, & Psychophysics, 79(1), 344–351. [3] Janse, E., & Adank, P. M. (2012). Predicting foreign-accent adaptation in older adults. Quarterly Journal of Experimental Psychology, 65(8), 1563–1585. [4] Tamati, T. N., Gilbert, J. L., & Pisoni, D. B. (2013). Some factors underlying individual differences in speech recognition on PRESTO: A first report. Journal of the American Academy of Audiology, 24(7), 616–634. [5] Kapnoula, E. C., Winn, M. B., Kong, E. J., Edwards, J. R., & McMurray, B. (2017). Evaluating the sources and functions of gradiency in phoneme categorization: An individual differences approach. Journal of Experimental Psychology: Human Perception and Performance, 43(9), 1594–1611. [6] Kong, E. J., & Edwards, J. R. (2016). Individual differences in categorical perception of speech: Cue weighting and executive function. Journal of Phonetics, 59, 40–57. [7] Kawahara, H., Takahashi, T., Morise, M., & Banno, H. (2009). Development of exploratory research tools based on TANDEM-STRAIGHT. Proceedings of Asia-Pacific Signal and Information Processing Association, 2009 Annual Summit and Conference. pp. 111–120. [8] Mueller, S. T., & Piper, B. J. (2014). The psychology experiment building language (PEBL) and PEBL test battery. Journal of Neuroscience Methods, 222, 250–259. [9] Heald, S. L. M., & Nusbaum, H. C. (2014). Speech perception as an active cognitive process. Frontiers in Systems Neuroscience, 8.
RQ1: Listeners flexibly adapted to unfamiliar vowels by up-weighting a secondary cue when a primary cue is not informative.
A: Primary use of spectral differences at Baseline, B1: Up-weighting of the duration cue when the spectral cue is not informative at Exposure, B2: Differences in proportion /æ/ responses (330ms – 130ms)
RQ2: Individuals varied widely in categorization gradiency and this variability was related to their use of a secondary cue, but the link between gradiency and adaptation was not found.
RQ3: Individual differences in inhibitory control correlated with the amount of adaptation.
Results
0.00
0.25
0.50
0.75
1.00
1 2 3 4 5 6 7Spectral step (/ɛ/ to /æ/)
Prop
ortio
n /æ
/ res
pons
es
Duration130 ms330 ms
Vowel categorization at BaselineA
0.00
0.25
0.50
0.75
1.00
baseline1 exposure baseline2Block
Prop
ortio
n /æ
/ res
pons
es
Duration130 ms330 ms
Categorization of Test stimuliB1
0.00
0.25
0.50
0.75
1.00
baseline1 exposure baseline2Block
Diff
in p
ropo
rtion
/æ/ r
espo
nses
Diff. /æ/ responses (long-short)B2
178178178178178178178178178178178178178178
179179179179179179179179179179179179179179
180180180180180180180180180180180180180180
181181181181181181181181181181181181181181
182182182182182182182182182182182182182182
183183183183183183183183183183183183183183
184184184184184184184184184184184184184184
185185185185185185185185185185185185185185
186186186186186186186186186186186186186186
167167167167167167167167167167167167167167
187187187187187187187187187187187187187187
188188188188188188188188188188188188188188
189189189189189189189189189189189189189189
190190190190190190190190190190190190190190
191191191191191191191191191191191191191191
192192192192192192192192192192192192192192
193193193193193193193193193193193193193193
195195195195195195195195195195195195195195
8585858585858585858585858585175175175175175175175175175175175175175175
151151151151151151151151151151151151151151
196196196196196196196196196196196196196196
197197197197197197197197197197197197197197
200200200200200200200200200200200200200200
201201201201201201201201201201201201201201
6363636363636363636363636363
148148148148148148148148148148148148148148202202202202202202202202202202202202202202
203203203203203203203203203203203203203203
7878787878787878787878787878 204204204204204204204204204204204204204204
208208208208208208208208208208208208208208
209209209209209209209209209209209209209209
211211211211211211211211211211211211211211
205205205205205205205205205205205205205205
206206206206206206206206206206206206206206
178178178178178178178178178178178178178178178178178178178178178178178178
179179179179179179179179179179179179179179179179179179179179179179179179
180180180180180180180180180180180180180180180180180180180180180180180180
181181181181181181181181181181181181181181181181181181181181181181181181
182182182182182182182182182182182182182182182182182182182182182182182182
183183183183183183183183183183183183183183183183183183183183183183183183
184184184184184184184184184184184184184184184184184184184184184184184184
185185185185185185185185185185185185185185185185185185185185185185185185
186186186186186186186186186186186186186186186186186186186186186186186186
167167167167167167167167167167167167167167167167167167167167167167167167
187187187187187187187187187187187187187187187187187187187187187187187187
188188188188188188188188188188188188188188188188188188188188188188188188
189189189189189189189189189189189189189189189189189189189189189189189189
190190190190190190190190190190190190190190190190190190190190190190190190
191191191191191191191191191191191191191191191191191191191191191191191191
192192192192192192192192192192192192192192192192192192192192192192192192
193193193193193193193193193193193193193193193193193193193193193193193193
195195195195195195195195195195195195195195195195195195195195195195195195
858585858585858585858585858585858585858585858585175175175175175175175175175175175175175175175175175175175175175175175175
151151151151151151151151151151151151151151151151151151151151151151151151
196196196196196196196196196196196196196196196196196196196196196196196196
197197197197197197197197197197197197197197197197197197197197197197197197
200200200200200200200200200200200200200200200200200200200200200200200200
201201201201201201201201201201201201201201201201201201201201201201201201
636363636363636363636363636363636363636363636363
148148148148148148148148148148148148148148148148148148148148148148148148202202202202202202202202202202202202202202202202202202202202202202202202
203203203203203203203203203203203203203203203203203203203203203203203203
787878787878787878787878787878787878787878787878 204204204204204204204204204204204204204204204204204204204204204204204204
208208208208208208208208208208208208208208208208208208208208208208208208
209209209209209209209209209209209209209209209209209209209209209209209209
211211211211211211211211211211211211211211211211211211211211211211211211
205205205205205205205205205205205205205205205205205205205205205205205205
206206206206206206206206206206206206206206206206206206206206206206206206
178178178178178178178178178178178178178178
179179179179179179179179179179179179179179
180180180180180180180180180180180180180180
181181181181181181181181181181181181181181
182182182182182182182182182182182182182182
183183183183183183183183183183183183183183
184184184184184184184184184184184184184184
185185185185185185185185185185185185185185
186186186186186186186186186186186186186186
167167167167167167167167167167167167167167
187187187187187187187187187187187187187187
188188188188188188188188188188188188188188
189189189189189189189189189189189189189189
190190190190190190190190190190190190190190
191191191191191191191191191191191191191191
192192192192192192192192192192192192192192
193193193193193193193193193193193193193193
195195195195195195195195195195195195195195
8585858585858585858585858585175175175175175175175175175175175175175175
151151151151151151151151151151151151151151
196196196196196196196196196196196196196196
197197197197197197197197197197197197197197
200200200200200200200200200200200200200200
201201201201201201201201201201201201201201
6363636363636363636363636363
148148148148148148148148148148148148148148202202202202202202202202202202202202202202
203203203203203203203203203203203203203203
7878787878787878787878787878 204204204204204204204204204204204204204204
208208208208208208208208208208208208208208
209209209209209209209209209209209209209209
211211211211211211211211211211211211211211
205205205205205205205205205205205205205205
206206206206206206206206206206206206206206
1
2
-100 -50Categorization gradiency (VAS slope)
Seco
ndar
y cu
e us
e (c
ross
over
diff
)
Gradiency and secondary cue useB A: Considerable individual differences in listeners’ VAS responses, B: The more gradient, the more secondary cue use
178178178178178178178178178178178178178
178 179179
179
179
179
179179179179179179
179179179
180180
180
180180
180
180
180180180
180
180
180
180181181181181181
181
181181
181
181181181181181
182182182
182182
182
182182
182182182
182182182183183
183
183
183183183
183
183
183
183183
183183184184184184184184184184
184
184184
184
184184
185185
185185185
185185185185185
185
185185
185186186186
186186
186
186186
186186186186
186186
167
167167
167
167167167167167167
167167167167187187187187
187
187187187
187187187187
187187188
188188
188188
188
188
188188
188188188188188
189189
189189189189189189189
189
189189
189189190
190190
190190190190
190
190190190
190
190
190 191191
191191
191191191
191
191
191
191
191
191
191
192192192192192192192192192192192192192
192
193193193193193193193193193193193193193193195195195
195195
195195
195
195195195
195195195 85
85858585
85
85
85
858585858585175175175
175175175175
175
175
175175
175
175175 151151151151151151151151151151151151151151196196196196196
196
196
196196196
196196196196197197197
197
197197197197197197197197197
197
200
200200200200
200200
200
200
200200200200200
201201201201201201201
201201201201201201201
63
63
63
636363
63
63
63
63
636363
63
148
148
148
148
148148
148148
148148148148148148202
202202202202
202202
202202202
202202202202 203203
203
203203203203203203203203
203
203
203
7878787878
787878787878
787878 204204204
204
204204
204204
204204204204204204 208
208
208208208208208
208
208208
208208208208
209
209
209209209
209
209
209209209
209209209209211211211211211211
211
211211211211211211211
205
205
205205205205205205205205
205
205205205
206206206206206206206206206206206206
206206 178
178
178
178
178178178178
178178178178178178178178
178
178178178178178178
178 179179179179179179179179179179179179179179179179179179
179179179
179179179
180180
180180
180180180
180180180180180180180180
180
180
180
180
180180180
180180181181181
181
181
181
181
181181
181
181
181181
181181
181
181181181181
181181
181181
182182
182
182182182182
182
182182
182
182
182
182
182182182
182
182182182
182
182
182183
183
183183
183183
183183
183183
183183
183
183183
183
183183
183183183183
183
183
184184
184
184184184184184184184184184
184184184184184184
184184
184
184184184 185185185185185
185185
185185185185185185185185
185185185
185
185185185
185
185
186
186
186
186
186186186186
186
186
186
186186
186186186
186
186186186
186186
186186167
167
167
167167167167
167
167167
167
167167167167
167167
167
167167167
167
167
167
187187
187187
187187
187187
187
187187187
187
187187
187
187187
187187187187
187
187188
188
188188
188
188188
188188
188188188
188188188188188188188188
188188
188188189189
189189189189
189
189
189
189189
189
189189189189189189189189
189189189
189190190
190190190190
190190
190190190190190190190
190
190190190190190
190
190190 191191191191191191191191191191191191191191191191191191191191191191
191
191192
192
192192192192
192
192
192
192192192192192192
192192192192192192192192192193193193193193193193193193193193193193193193193193193193193193193193193
195195195
195
195195195
195195
195
195
195195
195195195
195
195195195
195195
195195 85
85
85
85858585
85858585
85858585
8585
85
858585
85
85
85175175175175175175175175
175175
175175
175
175
175
175175175175
175175175175175 151151151151151151151151151
151
151
151
151151151151
151
151151151151151151151196196196196
196196196
196196196196
196196196196
196196
196
196196196
196
196196
197
197
197
197
197197197197
197
197
197
197197
197197197
197
197197197
197197
197197 200200
200200200
200200
200
200
200200200
200
200
200
200200
200
200
200
200
200200200
201201201
201
201201201201
201
201201201201
201201201
201201
201201201
201201201
63
63
63
636363
6363
63
63
63
636363
63
63
63
6363636363
63
63
148
148148148
148148148148148148148148148
148148
148148
148
148
148148148148148202202202202
202202
202202
202
202202
202202
202
202202
202
202
202202
202202202202 203
203
203
203
203203203203
203
203
203
203203
203203203
203
203203203
203203
20320378
78
78
78787878
78787878
78787878
78
7878
787878
78
78
78 204
204
204
204204
204204204
204
204
204204
204
204
204
204
204
204204204204204204204
208
208
208
208
208208208208
208
208208
208
208
208208208
208
208208208208
208
208208
209209209209
209
209209209
209209209
209209209
209209209
209
209
209209209209209211
211
211
211211211211
211
211211211
211211
211211211211211211211
211211
211211
205205205205205205205205205
205205
205205
205205205
205205
205205
205205
205205
206
206
206
206206
206206206
206
206
206
206206206206
206
206
206206
206206206206206
178
178178
178178178178178178178
178
178178178 179179
179
179
179
179179
179179
179179179179179180180
180180180180180180
180
180180
180180
180181
181181181181181181
181181181181
181181181 182182182182182
182
182182
182
182182182182182
183183183
183
183
183
183
183
183183183
183
183
183
184
184184
184184
184184184184
184
184184184184 185185185
185185185185
185185185185185185
185186186186
186
186
186
186186186
186186
186
186186167167
167
167
167167
167167
167
167167167
167
167187187187187
187
187
187
187187
187
187187
187
187
188
188188188188
188
188188188
188188188
188
188189
189
189
189189189189189189189189189189189
190190
190190
190
190190190
190
190190
190
190190
191191191
191
191
191191
191191191191191
191191
192
192192192192192192
192192
192
192
192192192193193193193193193193193193193193193193193195
195195
195195195195
195195
195
195195
195195 8585858585858585858585858585
175
175175175175175175175175
175
175175175175
151151151
151151151151151151151151151151151196196196196
196196196196
196196196196196196197197
197
197197197197
197
197197
197197
197197
200200200
200
200200
200
200200200200
200200200201
201201
201201201201201201201201201201201
63
636363
6363
63
63
63636363
6363
148148148148148148148148148148148
148
148
148
202
202
202202202202202202202
202202
202
202
202 203
203203
203
203
203203203
203
203
203
20320320378
78
787878
78787878
7878787878
204
204
204
204204
204
204204
204204204
204204204
208208208
208
208
208208
208208208208
208208208209209209
209
209209209
209
209209209
209
209209211
211211
211211211
211211211
211
211
211211211
205205205
205205
205
205
205205205
205
205205
205
206206206
206206206
206206206206206
206206206
baseline1 exposure baseline2
-0.1 0.0 0.1 0.2 -0.1 0.0 0.1 0.2 -0.1 0.0 0.1 0.2
0.00
0.25
0.50
0.75
1.00
Stroop effect (log)
Prop
ortio
n /æ
/ res
pons
es
Duration130 ms330 ms
Inhibitory control and perceptual adaptation
Individuals with poor inhibitory control showed more adaptation at Exposure.
208 209 211 63 78 85
201 202 203 204 205 206
192 193 195 196 197 200
186 187 188 189 190 191
180 181 182 183 184 185
148 151 167 175 178 179
0 25 50 75100 0 25 50 75100 0 25 50 75100 0 25 50 75100 0 25 50 75100 0 25 50 75100
04080
120
04080
120
04080
120
04080
120
04080
120
04080
120
VAS responses
coun
t
Individual listeners' VAS responsesA
Individual differences in plasticity in speech perception under cognitive load
CHSCOM2019Linköping, Sweden
June 9-12, 2019
Donghyun Kim, Meghan ClayardsContact: [email protected]
1. How and to what extent are speech perception abilities modulated by cognitive load?
2. Do individuals differ in their use of acoustic cues in phonetic categorization under cognitive load?
3. Are individual cue weighting strategies under cognitive load related to individuals’ cognitive abilities and gradiency in phoneme categorization?
Research QuestionsRQ1: Listeners overall showed an increased reliance on the primary (spectral quality) and the secondary cue (duration) under cognitive load.
RQ2: There were considerable differences across individuals in the effect of cognitive load on perceptual cue weighting.
RQ3: Individual differences in adaptive cue weighting strategies under cognitive load were linked to cognitive abilities (but were not linked to gradiency in phoneme categorization).
Individuals with better inhibitory control showed more adaptive spectral change.
Methods
Background
Results
Increased cue weights under cognitive load may be interpreted as an active cognitive process [9]
Some listeners showed an increased (decreased) reliance on spectral quality whereas others showed an increased (decreased) reliance on vowel duration under cognitive load.
No Load Cognitive Load
120 180 240 300 360 120 180 240 300 360
/ɛ/
50%
/æ/
Duration step (ms)
Spec
tral s
tep
(/ɛ/ t
o /æ
/)
0.250.500.75
/æ/ response
Average /æ/ responses by conditionA
No Load Cognitive Load
1 2 3 4 5 1 2 3 4 50.00
0.25
0.50
0.75
1.00
Spectral step (/ɛ/ to /æ/)Pr
opor
tion
/æ/ r
espo
nses
Duration120 ms180 ms240 ms300 ms360 ms
Cue weighting by conditionB
24
28
56
78
132
142
151
167
181
187
222
224
234
237
239
244246
248
249
250
251252
253
254
255256 257
259
260
261
262
263
264
265
266
267
268
270
271
272
273
274
275 276
277
278
279
280
281
282
283
284
285
286
0
1
2
-2.5 0.0 2.5Working memory composite score
Dur
atio
n cu
e w
eigh
t diff
eren
ce(C
ogni
tive
load
- N
o lo
ad)
Duration cue change and working memory
24
28
56
78
132
142
151
167
181
187
222
224
234
237
239
244246
248
249
250
251
252
253
255
256
257
259260
261
262
263 264
265
266
267
268
270
271
272
273274
275
276277
278
279
280
281
282
283
284
285
286
-2
0
2
0 2 4Go/No-go score
Spec
tral c
ue w
eigh
t diff
eren
ce(C
ogni
tive
load
- N
o lo
ad)
Spectral cue change and Go/No-go
No Load Cognitive Load
120 180 240 300 360 120 180 240 300 360
/ɛ/
50%
/æ/
Duration step (ms)Sp
ectra
l ste
p (/ɛ
/ to
/æ/)
0.000.250.500.751.00
/æ/ response
Duration weight increase (Listener 234)A
No Load Cognitive Load
120 180 240 300 360 120 180 240 300 360
/ɛ/
50%
/æ/
Duration step (ms)
Spec
tral s
tep
(/ɛ/ t
o /æ
/)
0.000.250.500.751.00
/æ/ response
Duration weight decrease (Listener 278)B
No Load Cognitive Load
120 180 240 300 360 120 180 240 300 360
/ɛ/
50%
/æ/
Duration step (ms)
Spec
tral s
tep
(/ɛ/ t
o /æ
/)
0.000.250.500.751.00
/æ/ response
Spectral weight increase (Listener 261)A
No Load Cognitive Load
120 180 240 300 360 120 180 240 300 360
/ɛ/
50%
/æ/
Duration step (ms)
Spec
tral s
tep
(/ɛ/ t
o /æ
/)
0.000.250.500.751.00
/æ/ response
Spectral weight decrease (Listener 267)B
/ɪ/
50%
/i/
80 125 170 215 260 305 350Duration steps (ms)
Spec
tral s
teps
(/ɪ/
to /i
/)
Visual Analog Scaling (VAS)
/ɛ/
50%
/æ/
120 180 240 300 360Duration steps (ms)
Spec
tral s
teps
(/ɛ/
to /æ
/)
Dual-task
heed hid
Individuals with better working memory showed more adaptive duration change.
Individual differences in adaptive cue weighting strategies under cognitive load, which may be interpreted as an active cognitive process, were linked to listeners’ cognitive abilities.