principles and applications of 3d reproduction using...

Principles and applications of 3D reproduction using Ambisonics

Alois Sontacchi

Principles and applications of 3D reproduction using Ambisonics Alois Sontacchi 2

@ the focus of Art and Technology

Institute of Electronic Music and Acoustics

University of Music and Performing Arts Graz

AUSTRIA - 8010 Graz - Inffeldgasse 10/3

3

artistic/scientific technical/scientific

Professors 1 1

Academic non-professorial teaching staff (scientists) 6 12

Academic non-professorial teaching staff (lectors) 2 7

Other staff members5

Principles and applications of 3D reproduction using Ambisonics Alois Sontacchi

4

Education

• Musicology (Bachelor, Master) in cooperation with KFU Graz

• Composition - Computer Music (Master + NEW: Bachelor)

• Electrical and Audio Engineering (Bachelor, Master) in cooperation with TU Graz

• Doctoral school


5

Artistic Research

sound & spaceembodimentalgorithmic composition

Signal Processing and Acoustics

spatial audioaudio signal processingpsychoacoustics

Computer Music

sonificationinteraction design


Research Areas Sound & Music Computing

6

Motivation

What is the necessity for 3D Audio production ?

Games & Entertainment

Music Performance / Composition

Listening Test: In-Situ Simulation- speech intelligibility in noise

- evaluation of hearing-aids

- improvements of ANC-applications

Immersive Telepresence

LoRA A Loudspeaker-Based Room Auralization SystemS. Favrot, J. M. Buchholz, Acta Acoustica Vol. 96, p.364-375, 2010.


7

Outline

Introduction

Ambisonics Theory

Optimization criteria for reproduction quality

Applications


8

Psychologists/psychophysicists discover „phantom source“ around 1900

- „phantom“ sound localised, where there‘s nothing physicalmedical term such as „phantom pain“


Introduction

9

X/Y uses first-order transduer characteristics to produce psychoacoustic effect

Introduction


10



Introduction

11


-15dB


Introduction

12

M/S is useful for modifying coincidence stereophony image

- + +

+- ++Encoding


Introduction

13


- + +

+- ++Encoding


Introduction

14


- + +

+- ++

- + +

+-++

But don‘t increase S: out-of-phase

Encoding

Decoding

Introduction

Advancements of Ambisonics

15

Soundfield microphone

A equilateral triangle is a regular 2D simplex; for 3D: tetrahedron

- +-

+

+

+ + - -

FLD FRU

BLU BRD

Front/Back,Left/Right,Up/Down

+

y

FRU

BRD

FLD

BLUxy

z

+-

x z w

+ - - + + + +-+ - +

Encoding


Introduction

16

Decoding w by sampling

Simplistically

-+

- ++

+

+

+

+


Introduction

17

Decoding x by sampling

Simplistically

-+

- ++

- +


Introduction

18

Simplistically

-+

- ++

-

+

Decoding y by sampling


Introduction

19

Matrix(Decoder)

Lspk. setup

The entire encoding and decoding signal chain

Matrix(Encoder)

Mic. arrayconfig. Andproperties

Ambisonics format

instead of array recording, monosignals can be encoded by vector.


Introduction

20

Matrix(Decoder)

Lspk. setup

The entire encoding and decoding signal chain

Matrix(Encoder)

Mic. arrayconfig. Andproperties

instead of array recording, monosignals can be encoded by vector.

Experimental evidence: what is heard ?

How to arrange loudspeakers optimally?

Ambisonics format

How to best shape the decoder?

Is it sound field synthesis? What‘s its theory?

Higher-order recording unclearbefore 2000… It won‘t workanymore using 2D/3D directionvectors


Introduction

The theory behind sound field synthesis lies in theKirchhoff-Helmholtz integral (KHI) equation

21

Theory of sound field synthesis, see Schenk (CHIEF point method) or Burton-Miller solution method


22

d/dn p|Γ is the surface normal derivative of sound pressure(proportional to normal component of particle velocity)

Theory of sound field synthesis, see Schenk (CHIEF point method) or Burton-Miller solution method


The theory behind sound field synthesis lies in the KHI equation and equivalent scattering problem

23

For simplicity (no encoding of source distance):Considering amplitude panning, the driving function onlyselects active sources on the boundary.

g is a virtual panning function: beause it is continuous(loudspeakers will be discrete sources)


The theory behind sound field synthesis lies in the KHI equation and equivalent scattering problem

24

Virtual panning function (with height)

red>0

blue<0

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Panning and decoding [Franz Zotter, Matthias Frank, „All-Round Ambisonic Panning and Decoding“, JAES, October 2012]

enc


25


red>0

blue<0


enc


26


red>0

blue<0

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .



27


red>0

blue<0

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .


How to optimally arrange withloudspeakers?

Virtual panning function must bedecoded to discrete loudspeakers

How can they represent it well?


Trying to do sound field synthesis seriously, with spherical functions:

Panning and decoding [Zotter, Pomberger, Frank, „An Alternative Ambisonic Formulation“, 126th AES Conv, May, 2009.]

29

“psychoacoustic” measures were considered for decoding.

Based on theoretical consideration

f<1.5kHz

loudness direction of localisation

f>1.5kHz


Makita theory.

What is relevant for broad-band sounds? What can we demand?

Trying it more psychoacoustically:Gerzons measures for the decoding problem



Jérôme Daniel‘s shapes for virtual panning functions: which is best?

Insertion of order weights an

basic

Max-rE

in-phase (a bit more complicated…)

30

Panning and decoding [Jérôme Daniel, PhD thesis, 2000]


31



Gerzons measures applied on virtual panning function


linear measures

quadratic measures


32

Virtual panning functions have nice properties:

panning-independent properties of Ambisonic panning functions:

• symmetric around panning direction ϑs

• amplitude integral is constant (V)

• directional amplitude concentration:- constant magnitude of concentration ||rV||

- always points towards panning direction ϑs=rV/||rV||

• energy integral is constant (E)

• directional energy concentration:- constant magnitude of concentration ||rE||

- always points towards panning direction ϑs=rE/||rE||



33

Nice properties might get lost after decoding:

• symmetric around panning direction ϑs

• amplitude integral is constant (V)

• directional amplitude concentration:- constant magnitude of concentration ||rV||

- always points towards panning direction ϑs=rV/||rV||

• energy integral is constant (E)

• directional energy concentration:- constant magnitude of concentration ||rE||

- always points towards panning direction ϑs=rE/||rE||


Which are relevant? Psychoacoustically?


Vector rE is most useful measuring localisation direction

34

broad-band sounds

VBAPVBIP

horizontal-only level panning

Psychoacoustically relevant factors [Matthias Frank… not yet published.]


Which sampling preserves all energetic measures?

35



36

Virtual t-design Ambisonics using VBAP

Fig. 6: Triangulation, energy measure, and spread measure as a function of the virtual source direction. [Zotter, Frank, Sontacchi, EAA 2010] [Zotter, Frank, JAES 2012],

PhD-Thesis Matthias Frank, 2013

VBAP

Virtualt-dsgn.Ambi.N = 5

00.510

10

20

30

40

50

60

70



37

VBAP and Ambisonics compared

Triplet-wise panning (VBAP)+ constant loudness+ arbitrary layout-- varying spread

Ambisonic Panning~+ constant loudness+ arbitrary layout~+ invariant spread



Psychoacousticians are sceptical aboutAmbisonics with height.

• Two vertically stacked loudspeakers do not yield a stablephantom source; should not be more than 30° separated

• A loudspeaker triplet does not yield stable phantom sources in all orientations, and results are individual

• However: What if we do not understand Ambisonics as a simple pair ortriplet?How diffuse must it be to obtain stable playback?

• [Results Braun/Frank/Pomberger 2011]

Advancements of Ambisonics 38

39

[Sabastian Braun, Matthias Frank, „Localization of 3D Ambisonic Recordings …“ ICSA, 2011]


40

[Sabastian Braun, Matthias Frank, „Localization of 3D Ambisonic Recordings …“ ICSA, 2011]


Application for Loudspeaker Reproduction

41Advancements of Ambisonics

1 2

Ambisonics Decoder

Encoderposition Encoder position

Ambi-sonicsBus

3 4

Encoderposition Encoder position

AmbisonicsRecording

rotation

Live electronics in modern operas with Ambisonics:

Alois Sontacchi 2002Thomas Musil

Markus Noisternig 2004Thomas Musil

Peter Plessas 2010

Pomberger, Zotter: An Ambisonics Format for flexible playback layouts, 1st Int. Ambisoncis Symposium, 2009.

Application for Headphone Reproduction

42Advancements of Ambisonics

Sources Encoding Room Head Rotation Filter Reproduction

43

I would particularly like to thank

Franz Zotter

Matthias Frank

Hannes Pomberger


44

Available software tools

• Tools:

CUBEmixer (IEM)

spat (ircam)

AmbDec, Amb plugins (Fons Adriaensen)

ICST Max/MSP plugins

plogue bidule

• Soon more to come.

• If you are a mixing desk manufacturer…


45

Format

• Format

• International Conference on Ambisonics and Spherical Acousticshttp://www.aes-uk.org/uk-conference/25th-call-for-papers/


46

How to record higher order? – upscaling methods from first order

- DirAC: Fraunhofer / V. Pulkki

builds intensity vector from p, vx, vy, vz at each frequencyto assign narrow direction

determines diffuseness

- HARPEX: S. Berge

can isolate even two signal directions at each frequency



47

Meyer, Elko 2000-

Petersen, Hald, Moller-Juhl, B&K, 2003-

Jin, v.Schaik, 2006-

Li, Duraiswami, O‘Donavan, Grassi, visisonics, 2004-

Rafaely&Park, 2004-

Higher order microphone arrays

Takashima, Nakagawa,Williams, 01db, 2008


Pausch, ZotterPomberger, 2013

40 years of coincidence recording yield Ambisonics

• 1930s- Blumlein‘s coincidentdirectional microphone pair (X-Y, M/S)

- Olson‘s ribbon microphones (figure-of-eight)

• 1940s- microphones with acoustically switchable characteristics

- Bauer‘s supercardioid has maximal front/back ratio

• 1950s- Neumann‘s U57 and AKG‘s C12 (double membrane) studio microphones allow electronic switching

- AKG‘s logo

- Vinyl and tape records available in stereo

- (many non-coincident A/B)

• 1960s- BBC broadcasts in stereo

- Psychoacoustics on stereo

• 1970s- Quadraphony

- Matrix stereophony: encoding/decoding in azimuthalharmonics (Cooper, Shiga)

- Ambisonics(Gerzon/Fellgett/Craven): recording: CALREC-triplet, Soundfield microphone, decoding/encoding circuits

- Panning in higher orders

Time-line

40 years later: a hand-full of higher orders, enriched by sound field synthesis theory, psychoacoustic experiments, and a community

• 1970s to 1990s: - theoretical psychoacoustic decoding, electronics (Gerzon) V,E,rV,rE

• 1990s: - binaural Ambisonics (Jot)- Sursound list- Pulkki‘s papers and experiments about 3D stereophony/VBAP

• 2000s: - Daniel (300 pages: theory), - Ward/Abhayapala truncation error- Malham (energetic distribution), - Sontacchi/Höldrich/Noisternig/Zmölnig(IEM-Bin-Ambi, IEM-CUBE

- Poletti („Mode-matching“),

• More in 2000s: - Meyer/Elko(„Eigenmike“, 4th-order recording)

• End of 2000s:- experimental psychoacosutics: Bertet, Solvang, Kratschmer, Frank, Bates, Marentakis

- Advancement in theory(concepts, sampling, truncation): Ahrens/Spors, Fazi, Li, Rafaely

- First Ambisonics Symposium: HRTFs, musical instrumentradiation, embedding of Ambiinto scene description, …

- Zotter/Pomberger/Frank …

Time-line

principles and applications of 3d reproduction using...

Documents