sound localization strategies in simple and complex environments norbert kopčo department of...
TRANSCRIPT
Sound Localization Strategies Sound Localization Strategies in Simple and Complex in Simple and Complex
Environments Environments
Norbert Kopčo
Department of Cybernetics and AI, TU Košice, Slovakia
Hearing Research Center and Center for Computational NeuroscienceBoston University
2
Overview
Intro – Horizontal sound localization
Two studies that examine auditory localization in more complex scenes:
Experiment 1: Shifts in localization due to contextual factors during sequential stimulus presentation
Experiment 2: A priori information and localization of a talker in a multitalker environment
Horizontal Sound localization
Is based on binaural cues (ITD, ILD)
depends on:• stimulus type: spectrum, temporal aspects• environment: anechoic, reverberant• source movement: static, dynamic
But also on:• presence of other stimuli• a priori knowledge / expectations about the
scene
Effect of additional stimuli
The extra sound can act as a:
Masker – localization worse (Gilkey & Good, 1986)
Adaptor – localization biased (Attraction/Repulsion) (Carlile et al., 1986)
Real sound (of which the target is a reflection) – localization worse/suppressed (Litovsky et al., 1999)
Perceptual stream of which the target is or is not a part – localization better or worse (Clifton & Freyman, 1997)
Cue – localization better (Rhodes, 1987; Sach et al., 2000)
Anchor – change localization strategy (Durlach & Braida , 1969)
Current Experiments
Understand auditory localization in a more complex scenes:
Exp 1:- when target is preceded by another identical sound (distractor) from a known location that the listener should ignore- when the task of single target localization is interleaved with the task of localization of a target preceded by a distractor
Exp 2:- when localizing a talker in a multi-talker environment- when we know the location of the distractor talkers
Goal: Show that listeners change how they interpret the binaural cues depending on the context, strategy, and a priori information
Experiment 1: Perceptual and central effects in a
click-sound localization with a preceding distractor
Preceding distractor: Intro
Several preceding studies indicated that preceding stimulus influences localization at SOAs of several hundreds milliseconds (Kopco et al., 2001, Perrott and Pacheco, 1989)
Goal:
- Characterize this influence (bias and std.dev. in responses)
- Determine its cause. Candidates:- short-term adaptation in brainstem representations- reverberation suppression and acoustics- strategy- perceptual organization- attention: focused away from distractor location
Hypotheses
Peripheral factors will have short-term effects
Central factors will influence results at longer separations
Effect of reverberation can be separated by comparing performance in anechoic and echoic rooms
Effect of perceptual organization can be addressed by modifying the stimuli
9
Methods
We measured azimuthal localization performance for a click target stimulus when preceded by another identical click (coming from a known location):
- presented with a short onset asynchrony
- from a different azimuthal location
- in either an anechoic chamber or an ordinary classroom
Performance was compared to that in a control in which there was no preceding distractor click.
Perceived target location of frontal sources is “attracted” by lateral distractors
Interactions for SOAs up to 200 ms
Possible mechanisms:
-Precedence effect
-Short-term adaptation
-Grouping of identical target and distractor
Results: Bias due to Distractor
Replacing the single-click distractor by a 5-Hz click-train (such that the target does not fall into the temporal pattern of the distractor) eliminates the “attractive” bias.
Effects not due to acoustics because correct representation is available
Exp 1 – Perceptual organization
Contextual plasticity
There is bias also in the no-distractor responses
The bias is always away from the non-present distractor
Because the runs were interleaved, this bias had to build up anew during each run
Contextual plasticityDifference in no-distractor responses in the frontal and lateral distractor context
-Is independent of azimuth
-Grows over time
-Slightly stronger for the 8-click train context
Contextual plasticity on time scale of minutes
Similar to effects of long-term exposure
Either due to bottom-up factors (distribution of stimuli) or top-down factors (focusing away from distractor)
Follow-up experiments and analysis to explore the contextual effect
Con
text
ual b
ias
Context Exp 1a – Reverberation?
Experimental Design:Compare results from room and anechoic chamber.
Result:Contextual plasticity equally strong in anechoic and reverberant spaces
Context Exp 1b: Effect of timing
Experimental Design:Fix Distractor aheadAdd no-distractor runSystematically vary:- SOA (25 – 400 ms)- distr/no-distr trial occurance ratio
Result:Contextual plasticity- modulated by SOA- modulated by occurance ratio- persists even at longest SOA & lowest dist-trial occuranceOccurance frequency does not matter
Context Exp 1c: Target-Distr. Order
Experimental Design:Use only longest SOAFix ratio at 75% distractorsReverse the Distractor-Target order
Results:Contextual plasticity- persists even for T-D ordering
SOA and D-T T-D order does not matter
Context Exp 1c: Target-Distr. Order
Experimental Design:Use only longest SOAFix ratio at 75% distractorsReverse the Distractor-Target order
Results:Contextual plasticity- persists even for T-D ordering- affects equally dist and no-dist resps- has sharp on- / off- setsBaseline changes over timeIs D used as Anchor?
Context Exp 1d: Sp. Arrangement
Experimental Design:In separate runs, vary which speakers present targets on distractor-trialsNo-distractor targets come from all locations
Result:Contextual plasticity - strongest where distractor-targets are present (+ D location)
- weaker if distr targets on both sides
D-T reference required for contextual plasticity
Context Exp 1d: Abs and Rel Info
Questions:Does distractor provide additional info?Do we combine D-T relative and T-only absolute info?
Analysis:Correlation coefficient between actual location and responses on no-distr trials
Result:Performance consistency improves when distractor- trials are present
Similar results for standard deviations (not shown)
Exp 1 - SummaryPreceding distractor:
-Affects perceived target location at SOAs of up to 200 ms
-Is likely due to combination of - short-term plasticity, and - perceptual-organization factors
Contextual plasticity:
-Is modulated by temporal and spatial characteristics of the inducing task
-Persists even when SOA is 400 ms or when target precedes distractor
-Is likely due to an automatic process that combines the distractor and target localization information, e.g., to provide the optimum target location estimate based on absolute and relative target location
-Results in improvement in localization in terms of correlation coefficient, likely because distractor used as anchor for relative localization (automatic change of strategy?)
24
Introduction
Spatial separation of sources enhances speech perception
In complex environments (e.g., with multiple talkers), spatial perception also important for “sorting” acoustic scene into objects and focusing attention on sources of interest (Brungart et al 2001; Freyman et al 1999; Kidd et al 2005; Best et al 2007; Shinn-Cunningham 2008)
Relatively few studies actually measured localization of speech in a multitalker environment (Yost et al., 1996; Hawley et al.1999; Drullman and Bronkhorst 2000; Brungart et al. 2006)
25
Experiment and GoalsStudy horizontal localization of speech in a multitalker environment
Question 1: How does presence of maskers influence localization performance?
Evaluate the effect of maskers on RMS errors in localization responses. Separate effect of detection on localization errors.
Question 2: Is performance affected by a priori knowledge / uncertainty about distribution of masker locations?
Compare performance when masker distribution fixed vs. varied from trial to trial.
Hypotheses: 1. Masker location uncertainty will hurt performance.2. A priori information will eliminate some of the loss, in particular if a simple strategy can be employed to use it.
26
Setup and masker patterns
Pattern 1 Pattern 3 Pattern 5Pattern 4Pattern 2
Room:approx. 3 m x 5 m
Speakers:
only presenting targets
presenting targets andmaskers (see panel b)presenting targets andmaskers (see panel b)
27
MethodsStimuli:
Target: word “two” spoken by a female talkerMaskers: 4 different monosyllabic words,
spoken by 4 male talkers (all longer than target)Target-to-Masker energy ratios: 0 dB or -5 dB
Task: Subjects pointed head to perceived target locationSubjects asked to indicate location only if target heard (5 catch trials with no target per block to monitor
obedience)
Conditions (separate blocks):- Control: No masker- Fixed: Masker pattern fixed across block of trials- Mixed: Masker pattern randomly chosen for each trial
28
Detection
Detection worse at lower TMR, similar in both uncertainty conditions
0 dB -5 dB0
5
10
15
20
25
False Alarm Rate
Pe
rce
nt (
ou
t of 5
ca
tch
tria
ls p
er
blo
ck)
N = 7
TMR0 dB -5 dB
0
5
10
15
20
25
Miss Rate
TMR
Pe
rce
nt
FixedMixed
29
Localization: Control
Good performance with no maskers
All effects of maskers plotted re. control performance on following slides
30
Average across patterns
Detrimental effect of maskers is strong, both for fixed and mixed conditions.
Averaged across patterns and target locations, a priori knowledge helps slightly, by approximately 20%.
31
Average across patterns
When looking only at off-masker locations, a priori knowledge helps dramatically (by 36%)
32
Average across patterns
When looking only at on-masker locations, a priori knowledge has no effect (or hurts performance)
33
Interim SummaryPresence of maskers hurts performance (H1 confirmed), even after accounting for lower detectability.
A priori knowledge of masker locations influences target talker localizability:
- Improving performance at locations from which (the subject knows) no masker can come
- Not affecting (or worsening) performance at locations from which (the subject knows) maskers will come (H2 partially confirmed)
Possible mechanism: - Redistribution of processing resources - “incorrect” strategy: focusing only on off-masker locations
Next, analyze patterns separately to gain more insight into behavior re. H2.
34
Raw Data
Complex effect of target location, masking pattern, uncertainty and TMR
-50 0 500
10
20
Sta
nd
ard
De
via
tion
[°]
-50 0 50
-50 0 50
TMR 0 dB
-50 0 50
-50 0 50
FixedMixed
-50 0 500
10
20
-50 0 50
-50 0 50
TMR -5 dB
Target Azimuth [°]-50 0 50
-50 0 50
35
Analysis of Patterns
A priori information helps for off-masker targets - in almost all patterns- at both TMRs (more at -5 dB)
A priori information can hurt for on-masker targets, mainly for patterns 1 and 2
Overall, effects large for Patts 1 & 2, small for Patt 5.
Complexity of pattern limits use of a priori information.
36
Bias due to Maskers
-50 0 50
-20
0
20
Me
an
Re
spo
nse
Bia
s [°
]
-50 0 50
-50 0 50
TMR 0 dB
-50 0 50
-50 0 50
FixedMixed
-50 0 50
-20
0
20
-50 0 50
-50 0 50
TMR -5 dB
Target Azimuth [°]-50 0 50
-50 0 50
Compression strongest for targets near peripheral maskers
Leftw
ard
R
ight
war
d
37
Masker Uncertainty and Bias
When masker pattern fixed throughout a block, responses biased away from maskers
Bia
s du
e to
Mas
ker
Unc
erta
inty
Left
Rig
ht
-50 0 50
-20
0
20
Me
an
Re
spo
nse
[°]
-50 0 50
-50 0 50
Average across TMR
-50 0 50
-50 0 50
FixedMixed
-50 0 50
-10
-5
0
5
10
m
ea
n [°
]
-50 0 50
-50 0 50
Target Azimuth [°]
Difference between meanFixed
and meanMixed
-50 0 50
-50 0 50
38
Exp 2: Summary1. Mixture has complex effects on localization performance
- compression of mean localization responses near peripheral maskers- it increases localization errors, even after detection errors are eliminated- effect depends on masker pattern, location of target re. maskers, and TMR
2. A priori information about the distribution of speech maskers modulates the effect of masking:
- reducing it (as expected)- but sometimes increasing it (unexpected)
3. These modulatory effects are - likely to be due to change in strategy / assignment of resources:
focusing on off-masker locations in fixed condition- most useful when a priori information can be simply applied (simple patterns)- least useful when a priori information cannot be simply applied (complex
patterns)
4. Can models like Feller and Merima (2004) or Dietz, Ewert and Hohmann (2010) predict these results?
Kopčo N, Best V, Carlile S (2010). Journal of the Acoustical Society of America, 127, 1450-1457
39
Exp 2a: Hearing Impairment
Q: Is the effect of complex masker mixture similar for Hearing-Impaired listeners?
Methods:As in Exp. 2, but only the Mixed condition.
Result:HI listeners performance only affected in mixture (re. normal hearing listeners)
Best, Carlile, Kopčo, van Schaik (2011) J of the Acoust Soc of Am, 129, EL210-EL215
40
Overall Summary
Previous studies showed that the effect of attention and a priori information on sound localization is small in simple scenarios.
Here we show that localization performance does change - even in simple conditions when the listeners can change
their response strategy
- In complex scenarios (e.g., in mixture of 5 talker):- People use a priori information to optimize their
localization performance.- The result is not always an improvement.
Beáta Tomoriová, Ľuboš Hládek, Rudolf AndogaPerception and Cognition Lab, TU Košice, Slovakia
Gin BestHearing Research Center, Boston U & Australian National Acoustics Lab
Barbara Shinn-CunninghamHearing Research Center, Boston University
Simon CarlileUniversity of Sydney
Financial Support: Human Frontiers Science Program, US National Institutes of Health, US National Science Foundation,US National Academy of Sciences, Slovak Science Grant Agency
Collaborators and Support
Current Projects
TU Košice & Boston University:Contextual Plasticity in Sound Localization (US NIH)
Boston University, Harvard Medical School/MGH, UC Riverside, University of Edinburgh:Perceptual and cross-modal learning in auditory distance perception (Marie Curie Project, 7FP EU)
More info: http://pcl.tuke.sk, http://cns.bu.edu/~kopco