sound field analysis and control for virtual acoustics in cars

Sound Field Analysis and Control for Virtual Acoustics in Cars

PHD STUDENT: MICHAEL STRAUSS

SUPERVISORS: PROF. ALOIS SONTACCHI, PROF. ROBERT HÖLDRICH, PROF. MIKAEL STERNAD (UPPSALA UNIVERSITY)

PHD-FORUM SOMMERSEMESTER 2021, KU GRAZ

2021-06-19

CONTENT

Introduction Big picture: Virtual Product Development PhD topic

Part #I: Summary and key outcome

Current research Focus on spatial audio perception

Part #II: State of the art, auditory models

Experimental work Working with auditory models

Ongoing and next steps

Content

19.06.2021 [email protected] 3

INTRODUCTIONPHASE I

IntroductionStarting point: An incredible car audio system – WFS


Wave Field Synthesis and Virtual Room Acoustics rendering system capabilities (Audi Sound Concept) [Strauss, Gleim, AES2012]

62 Loudspeakers (52 midrange, 5 tweeters, 4 woofers, 1 sub-woofer) Wave Field Synthesis rendering Virtual Room Acoustics processing Authoring and Sound Management GUI

IntroductionStarting point: An incredible car audio system …

… and a lot of question marks!


Let‘s frame this research into the bigger picture of industrialproduct development … virtual product development …

IntroductionSound Field Analysis & Control for Virtual Acoustics in Cars


Acquisition

Manipulation

Assessment

Simulation

Measurement

Subjective(Auralization, VR, …)

Objective(Quantification, …)

Virtual Tuning

Optimization

Data base Knowledge baseVirtual Model

(Car audio system)+ +

Listening evaluation

+ sound tuning

Numericaldescription

+ control

IntroductionVirtual Product Development applied to Acoustics (Industry 4.0)


Acquisition

Manipulation

Assessment

Simulation

Measurement



Virtual Tuning

Optimization

Data base Knowledge baseVirtualModel

+ +

Sound Field Analysis & Control for Virtual Acoustics in Cars

IntroductionVirtual Product Development applied to Acoustics (Industry 4.0)


Acquisition

Manipulation

Assessment

Simulation

Measurement



Virtual Tuning

Optimization

Data base Knowledge baseVirtualModel

+ +

Optimization

Sound Field Analysis & Control for Virtual Acoustics in Cars

IntroductionPart I: Overview

Tracing back to the origin of wave field artefacts How to measure/analyse the performance of a sound field reproduction method (e.g. Wave Field Synthesis)?

How to measure the quality or characteristics of virtual acoustics?

What is the car interior acoustics impact on the perceived (virtual) sound attributes?

Research topics Multi-channel data acquisition

Measurement error compensation

Multi-dimensional signal processing

Objective analysis of car interior acoustics


IntroductionPart I: Building the data base

• In-situ analysis of a sound field reproduction method requires a dense grid ofdata points across the reproduction area (spatial sampling theorem!)

• Measurement grid 57x60 positions, 2cm sampling distance

3420 RIRs per loudspeaker (62)

3 BRIR sets per loudspeaker


Processing steps to get a valid data set

array position

IntroductionPart I: Wave Field Analysis Toolbox

Multi-dimensional Fourier Transform Spatial decomposition of sound events

Analysis in spatial-frequency domain

Attenuation of specific directions of arrival

Extrapolation from 2D plane into 3D space


Fourier Domain – Spatial Filtering

• 2D mapping onto kx-ky cone• 3D mapping onto kxkykz-sphere

IntroductionPart I: Wave Field Analysis Toolbox

Application example Sound field of right side loudspeaker 𝑝(𝑥, 𝑦, 𝑡)

Wave front is invoking a significant artefact

Where does this origin from?

Processing 3D-FT into spatial-frequecy domain 𝑃(𝑘𝑥, 𝑘𝑦, 𝑤)

Apply spatial filter to suppress horizontal waves

Back-transformation into original domain

Now, just components arriving from above are included

Artefact origin identified as upper door handle (roof)


CURRENT RESEARCHPHASE II

Current researchOverview Part#2


Perceptual attributes & closing the optimization loop Which cues are supportive to create the illusion of an acoustic space,

larger than the car interior cabin?

Which sound field parameters are linked to such perceptual cues?

What are the objective and subjective analysis methods, suitable to assess the criteria mentioned above?

How to influence those properties on a target system (optimization-loop)?

Workflow What sounds good? target definition, parameter selection

Link perceptual and physical domain stability, repeatability

Test specific method(s) to reproduce/control transferability

focus = virtual room acoustics

Current researchRoom acoustics spatial perception

Searching for:

• (virtual) room acoustics

• room size/extent

• envelopment

• immersion

• perception of room boundaries,

• dissolving (reproduction) room boundaries,

• room-in-room,

• auditory model,

…


Concert hall acousticsmusical performance

Artistical aspects

Virtual AcousticsVAE

VR/AR, Games, Entertainment, Education

Maths/Physicssignal processing, numerical

simulation, optimization, machine learning, …

Psycho-acoustics, Auditory modeling, Hearing,

Audiology, Medical aspects

Car audio / interioracoustics, Industrial aspects,

measurement based, subjective/sensory

evaluation


Most prominent measures for spaciousness are based on: Interaural time difference (ITD)

Interaural level difference (ILD)

Interaural cross correlation (IACC)

Lateral engery fraction (LF)

Corresponding perceptual attributes: Apparent source width (ASW)

Listener envelopment (LEV)

Additional factors of influence: room-in-room perception, proximity/echolocation, visual context


room-in-room acoustics situation [Strauss]

Map of perceptual attributes [Kaplanis2017]


Apparent Source Width (ASW)

» Source perceived wider than visual/physicalsize of source [Keet1968]

» Spatially and teporally fused auditory image of the original sound and early reflections [Kaplanis2017]

» Related physical parameters:

(early) Lateral energy fraction (LF) and

Early Interaural CC (IACC_e) [ISO3382]

» Relevant f-bands and reliability of IACC under discussion [Witew2010, Beranek2008]

» In concert halls, high ASW seen as beneficial [Beranek2008]

» Listener Envelopment (LEV)

» Feeling of beeing surrounded by sound

» Mainly related to late lateral energy[Bradley and Soulodre, 1995]

» Related physical parameters

Late lateral sound level (GLL) and

Late Interaural CC (IACC_l) [ISO3382]

» Principal cue are ITD fluctuations (asymetriclateral and median room modes, fluctuation freq. ~ 3 - 20Hz) [Griesinger1997]

» Strength of late arriving energy plays a role [Furuya2005, Soulodre2003a]

» Fully de-correlated reverberation causes maximum envelopment [Blauert1996, Griesinger1997]


Current researchAuditory models

Auditory Modeling for Assessing Room Acoustics

- Described in [van Dorp Schuitman2010]- based on [Dau1996], extended by [Breebaart2001]

Model features:- foreground stream: direct sound- background stream: reverberant sound- monoaural output:

room attributes reverberance and clarity (S_REV, S_CLA)- binauraul output:

spaciousness related (S_ASW, S_LEV)

Stage #1: Peripheral processor:- Ear canal = bandpass, Cochlea = Gammatone filters- Hair cells = half-wave rectification- Neural adaptors (5 feedback loops)

Stage #2: Binaural and Central processor (+ Parameter calculation)- Split into direct and reverberant stream (peak detection, per channel)- Binaural: ITD calculated for each f-band, late/early values calculated using direct/reverberant streams- Central: combines peripheral and binaural processor (running ITD) outputs + calculates the 4 objective parameters


AMARA, processing blocks of auditory model [van Dorp Schuitman2010]

Current researchAuditory models

Room Acoustical Perception

- Described in [Klockgether2014]- Psychoacoustic experiments showed that listeners may have difficulties

to segregate between source/room stream- Model takes into account human insensitivity to changes in IACC [Klockgether2013]

Stage #1: Extract binaural cues from stereo input signalPeripheral processing- separately for left/right ear channel, frequency processing of inner ear- Gammatone filter bank (42 filters, 100Hz - 10kHz), half-wave-rectification,

low-pass filter (4th order Butterworth)Binaural processing- partitioning (40ms-blocks), ICC for each block and filter (weighted around zero with

Gaussian distribution, sd of 250us)- ILD calculation for each block and filter, then accumulation using random weighting factors- ICC are accumulated, point of gravity is ITD estimate [Stern&Shear1996]

Stage #2: Predict perceptual attributes (ASW, LEV)- Energy ratio determines direct and diffuse part- Modified ICC (power of 4) to model human insensitivity at low ICC values!- ICC weighted by Energy ratio


RAP model [Klockgether2014]

EXPERIMENTS& RESULTS

ExperimentsVirtual room acoustics – Test conditions

- Car audio system model = Binaural database from Part #1

- Virtual room acoustics rendered by Wave Field Synthesis audio system

- Virtual Stereo configuration, playback of Stereo audio material


Condition Number ofoptions

Description

Seating position 2 positions Driver (front left)Backseat (back right)

Room acoustics 4 rooms Big hall (T60 > 1.2s)Large room (T60 > 0.6s)Small room ( T60 > 0.3s)No-room (T60 ~ 0.05s)

Loudspeakers 2 configurations Midrange + tweeters onlyFull system (incl. sub-woofers)

Stimuli 4 items Acoustic guitar solo (dry recording)Pop song1 (strong rhythm section)Pop song2 (synthesizers, bass)Song (female voice and piano)

ExperimentsAMARA

Virtual room acoustics in car compartment

• S_ASW: relatively constant across conditions

• S_CLA: significant impact of sub-woofers

• S_LEV: closely following room size, also detecting the sub-woofer condition

• S_REV: decreasing with room size, but „no-room“ detected to be more reverberantthan „small room“

Need to study in-depth!


AMARA parameters @Driver seat: All conditions, 1 stimulus (pop-synth)

ExperimentsAMARA


• Comparison between driver and back seatfor 1 single stimulus (pop song2)

• Relative difference calculated as

𝑣𝑎𝑙𝑢𝑒@𝑏𝑎𝑐𝑘 − 𝑣𝑎𝑙𝑢𝑒@𝑑𝑟𝑖𝑣𝑒𝑟

𝑣𝑎𝑙𝑢𝑒@𝑑𝑟𝑖𝑣𝑒𝑟

• S_CLA: significantly higher on back seat

• All other parameters lower at back seat

Need to study in-depth!


AMARA parameters: relative difference due to seating positions, 1 stimulus

ExperimentsAMARA



Frequency [Hz]

Frequency [Hz]Frequency [Hz]

Subwoofer: 1st cabin mode @46Hz

SPL across car cabin:

front

back

front row

back row

right

left

right

left

Impact on decay time (T10) across cabin:

right

left

T10 [s]

0.2 s

0.4 s

ASW (coloured) and S_ASW (grey) on driver seat (front-left) LEV (coloured) and S_LEV (grey) on driver seat

ExperimentsRAP vs. AMARA

Comparison:ASW and LEV spaciousness attributes @driver seat


Audio examples for download!! Link shared via Zoom chat

ExperimentsStimulus

Any way to standardize the input signal? • Comparison of music signal (pop song) vs. pink burst signal (500ms, windowed)

• Virtual room acoustics, different virtual source positions (left/right Stereo source position)

• Binaural recording at different seating positions

Can „pink burst“ stimuli deliver more stable results (smaller variance)?


music pink burst

vQ: left right left right

------- driver seat ------- -------- back seat ---------

music pink burst

left right left rightvQ:

------- driver seat ------- -------- back seat ---------

Used in automotive audio qualityassessment [e.g. Azzali, Farina]

ExperimentsOpen: Design of the control loop

How to integrate perceptual attributes into the control loop?

• Estimation of actual perceptual attributes (e.g. ASW, LEV)

• a) Binaural optimization Target sound Filters

• b) Use microphone signals

• c) Binaural to microphone processing

• Manipulation of IACC [Grosse2014]

• Control frameworks [Brännmark2013, 2015]


Control loop concept is currently at the stage of „thoughts on paper“

ExperimentsNext steps

• Define test conditions for a structured analysis of auditory model performance

• Identify model deficiencies and accompany by in-depth analysis of acousticparameters• Influence of stimuli, adjustment of model parameters

• Go in-depth with further aspects of very small reproduction rooms:• Room-in-room perception

• Acoustic proximity/echolocation

• Integration of auditory assessment into control-loop• binaural-to-microphone, usage of non-binaural microphone measurements

• If required: tuning of auditory models or find other perceptual attributes

• Prepare for a subjective test (validate perceptual relevance of findings)


[email protected]

ReferencesPhase II


[Keet 1968] Keet W.d.V., “The influence of early lateral reflections on spatial impression". In Proceedings of the 6th International Congress on Acoustics (ICA 68), Tokyo, Japan, pages E53-E56.

[Kaplanis2014] Kaplanis N., Bech S., Jensen S. H. & van Waterschoot T., “Perception of reverberation in small rooms: a literature study”. Audio Engineering Society, 55th International Conference, 2014.

[ISO 3382] ISO (2009). \ISO 3382-1:2009: Acoustics - Measurement of room acoustic parameters - Part 1: Performance spaces". International Organization for Standardization.

[Witew2010] Witew I.B., Lindau A., Van Dorp Schuitman J., Vorländer M., Weinzierl S., De Vries D., “Uncertainties of IACC related to dummy head orientation". In Proceedings of DAGA 2010, Berlin, Germany

[Beranek2008] Beranek L. L., “Concert hall acoustics - 2008". Journal of the Audio Engineering Society, 56(7/8), pp. 532-544.

[Bradley and Soulodre1995] Bradley J. S., Soulodre G. A., “Objective measures of listener envelopment". Journal of the Acoustical Society of America, 98(5), pp. 2590-2597.

[Griesinger1997] Griesinger D., “The psychoacoustics of apparent source width, spaciousness and envelopment in performance spaces". Acta Acustica united with Acustica, 83(4), pp. 721-731.

[Furuya2005] Furuya H., Fujimoto K., Wakuda A. and Nakano Y., “The influence of total and directional energy of late sound on listener envelopment". Acoustic Science and Technology, 26(2), pp. 208-211.

[Soulodre2003] Soulodre G. A., Lavoie M. C. and Norcross S. G. “Objective measures of listener envelopment in multichannel surround systems". Journal of the Audio Engineering Society, 51(9), pp. 826-840.

[Blauert1996] Blauert, J. (1996). Spatial hearing. The MIT Press, Cambridge, Massachusetts, revised edition.

[Griesinger1997a]Griesinger D., “Spatial impression and envelopment in small rooms”. Presented at AES 103rd Convention, New York, USA. Preprint 4638, 1997.

[Dau1996] Dau, T., P•uschel, D. and Kohlrausch, A. (1996a). “A quantitative model of the “effective" signal processing in the auditory system. I. Model structure". JASA, 99(6), pp. 3615-3622.

[Breebaart2001] Breebaart D. J., Modeling binaural signal detection. PhD thesis, Eindhoven University of Technology, Eindhoven. 2001.

[Klockgether2014] Klockgether, S. & van de Par, S. A Model for the Prediction of Room Acoustical Perception Based on the Just Noticeable Differences of Spatial Perception Acta Acustica united with Acustica, 2014, 100, 964-971

[Brannmark2015] Brännmark L.-J., Sternad M., Controlling the impulse responses and the spatial variability in digital loudspeaker-room correction, International Symposium on Electro Acoustic Technologies – ISEAT, Shenzen, China, 2015

[Brännmark2013]Brännmark L.-J., Bahne A., Sternad M., Compensation of loudspeaker-room responses in a robust MIMO control framework. IEEE Trans. Audio, Speech, and Lang. Process., 21(6):1201-1216, 2013.

ReferencesPhase I


[2007 ICA 19] Strauß M., Munderloh M., "Influence of Loudspeaker Displacement on the Reproduction Quality of Wave Field Synthesis Systems", 19th Int. Congress on Acoustics, Madrid, 2007.

[2008 DAGA 33] Strauß M., Zhykhar A., Heeg S., "Einsatz von Mikrofonarrays zur Schallfeldanalyse in Fahrzeugen", 34. Jahrestagung für Akustik DAGA, Dresden, Germany, 2008.

[2008 AES 125Conv] Strauß M., de Vries D., "Application of Multichannel Impulse Response Measurement and Analysis to Automotive Audio", AES 125th Convention, San Francisco, USA, 2008.

[2009 DAGA 35] Strauß M., Nowak H., de Vries D., "Analysis of Enclosed Sound Fields using Multichannel Impulse Response Measurement", invited paper 35. Jahrestagung für Akustik DAGA, Rotterdam, The Netherlands, 2009.

[2009 AES 36Conf] Strauß M., Nowak H., de Vries D., "Approach to Sound Field Analysis and Simulation inside a Car Cabin”, AES 36th Int. Conference, Dearborn, USA, 2009

[2011 AES 131Conv] Strauß M., Gleim P., "Application of Wave Field Synthesis and Analysis on Automotive Audio", AES 131th Convention, New York, 2011.

[2012 AES 48Conf] Strauß M., Nowak H., " Sound Field Reproduction Analysis in a Car Cabin Based on Microphone Array Measurements”, AES 48th Int. Conference, Munich, Germany, 2012.

[2016 IEEE Spectrum] Strauß, M. et al, Carver L., "Virtually Tuning an Automotive Audio System”, Multiphysics Simulation, Insert to IEEE Spectrum, September 2016.

[2019 IEEE Spectrum] Strauß, M. et al, Forrister T., "Interactive Product Development – Simulation Applications Help Shape the Design of Car Audio Systems”, Multiphysics Simulation, Insert to IEEE Spectrum, December 2019.

[2019 AES AutoConf] Strauß M., Malbos F., Bogdanski M., "Virtual Reality Experience for the Optimization of a Car Audio System”, AES Int. Conference on Automotive Audio, Ingolstadt, Germany, 2019.

sound field analysis and control for virtual acoustics in cars

Documents