[134] immersive sound vr
TRANSCRIPT
![Page 1: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/1.jpg)
Immersive SOUND for VR
오현오 (Henney Oh) GAUDI AUDIO LAB
![Page 2: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/2.jpg)
SOUND is half the experience
![Page 3: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/3.jpg)
Listen
![Page 4: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/4.jpg)
SOUND is half the experience
- George Lucas
![Page 5: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/5.jpg)
Immersion into is a perception of being physically present in a non-physical world.
technology refers to technology that blurs the line between the physical world and digital or simulated world,
thereby creating a sense of immersion.
VR
Immersive
by wikipedia
Virtual Reality
![Page 6: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/6.jpg)
PS 게임_ “하쿠나 마타타”
![Page 7: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/7.jpg)
for
Immersive & Interactive
in Mobile
Head-Mounted Display
VR like HMD
![Page 8: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/8.jpg)
HMD is hot
Facebook, Oculus VR $2bil에인수(Mar. 25, 2014)
Samsung, Oculus와협력발표(May 31, 2014)
Sony, Project Morpheus 발표(Mar. 18, 2014)
Google, Cardboard VR(June 25, 2014)
Samsung Galaxy VR (Oculus + Note 4) 발표(Sep. 3, 2014)
Crescent Bay (Headphones)(Sep. 20, 2014)
Microsoft HoloLens(Jan. 21, 2015)
HTC – Valve, Revive (SteamVR)(Mar. 1, 2015)
LG – plastic cardboard(Feb. 10, 2015)
Apple acqr. Metaio(May 28, 2015)
Google invst. Magic Leap(Oct. 21, 2014)
![Page 9: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/9.jpg)
but, no one seriously thought SOUND
![Page 10: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/10.jpg)
Immersive & Interactive 3D SOUND
![Page 11: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/11.jpg)
Sound and Room
![Page 12: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/12.jpg)
Sound is a Wave
Sound = Longitudinal waveSound is a mechanical wave resulted from the longitudinal motion of the particles of the medium through which the sound wave is moving
![Page 13: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/13.jpg)
Anechoic and Reverberant Room
![Page 14: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/14.jpg)
Room Impulse Response
Direct Sound
sound object
microphone
0 Time
Freq
A
A
![Page 15: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/15.jpg)
Room Impulse Response
DS + 1 reflection (echo)
0 Time
Freq
![Page 16: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/16.jpg)
Room Impulse Response
DS + Early Reflections (Multiple Echoes)
0 Time
B-2案 _ IN SEARCH OF THE ORIGIN 음향 시스템의 기원인 축음기의 트랙으로부터 영감을 얻어 끊이지 않고 플레이되는 GAUDI 의 음향을 표현
장식적이고 복고스러운 개성이 강하여 아이덴티티로서 베스트안은 아님
![Page 17: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/17.jpg)
Room Impulse Response
DS + ER + Late Reverberations
0 Time
Late Reverberation
Direct Sound Early Reflections
B-2案 _ IN SEARCH OF THE ORIGIN 음향 시스템의 기원인 축음기의 트랙으로부터 영감을 얻어 끊이지 않고 플레이되는 GAUDI 의 음향을 표현
장식적이고 복고스러운 개성이 강하여 아이덴티티로서 베스트안은 아님
![Page 18: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/18.jpg)
Sound and Binaural
![Page 19: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/19.jpg)
Binaural
Widely used to mean “headphone virtualization”
Having or relating to two ears
![Page 20: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/20.jpg)
Binaural Hearing
High Frequency (above 4kHz)
• Inter-aural Level (intensity) Difference
• ILD
Low Frequency (Below 1kHz)
• Inter-aural Time (phase) Difference
• ITD L1 L2
![Page 21: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/21.jpg)
Cone of Confusion
How resolve front-back-above-below ?
?
![Page 22: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/22.jpg)
Head and Outer Ear (pinna)
We can since we are trained !!
![Page 23: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/23.jpg)
Spatial Cues
Azimuth cueCaused by head shape and pinna
ILD, ITD
Elevation cueCaused by shape of pinna
Freq. peak and notch
Range cueLoudness, motion parallax, excess ILD, ratio of direct to reverb
![Page 24: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/24.jpg)
HRTF
HRTF: Head Related Transfer FunctionHRIR: Head Related Impulse Response
![Page 25: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/25.jpg)
HRTF Measurement
![Page 26: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/26.jpg)
Binaural Room Impulse Response
BRIR = RIR + HRIR
B-2案 _ IN SEARCH OF THE ORIGIN 음향 시스템의 기원인 축음기의 트랙으로부터 영감을 얻어 끊이지 않고 플레이되는 GAUDI 의 음향을 표현
장식적이고 복고스러운 개성이 강하여 아이덴티티로서 베스트안은 아님
![Page 27: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/27.jpg)
Binaural Recording v. Binaural Rendering
![Page 28: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/28.jpg)
Binaural
Binaural Recording
Sound Source(Wave) RIR HRTF Human
Perception
Binaural Rendering
![Page 29: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/29.jpg)
Binaural Recording
Manikin measures binaural sound for “general” human
![Page 30: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/30.jpg)
Binaural Recording
Capture immersive sound there
![Page 31: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/31.jpg)
Binaural Rendering
Synthesize immersive sound here
*hL(t)
hR(t) *HRTF Database
mono
![Page 32: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/32.jpg)
Recording v. Rendering
Binaural Recording Binaural Rendering
OOOOO Complexity O
OOOO Immersive OOO
OO Interactive OOOOO
O Personalization OOOOO
Vs.
![Page 33: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/33.jpg)
Challenges in Implementation
![Page 34: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/34.jpg)
Technical Challenges
1. Interactive in Binaural Recording
2. Expensive to filter in real time
3. Hard to keep both timbre and localization
4. Hard to cover full 3D space with measured HRTF
5. Hard to match to visual content
6. Hard to get ‘one-size-fits-all’ HRTF
B-1案 _ A HOMAGE TO GAUDI 가우디 건축물의 도형으로부터 모티브를 얻어 기하학적인 형태로 문자를 표현
![Page 35: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/35.jpg)
Interactive in Binaural Recording
Issues on 360 binaural recording
B-1案 _ A HOMAGE TO GAUDI 가우디 건축물의 도형으로부터 모티브를 얻어 기하학적인 형태로 문자를 표현
![Page 36: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/36.jpg)
Match to visual content
B-1案 _ A HOMAGE TO GAUDI 가우디 건축물의 도형으로부터 모티브를 얻어 기하학적인 형태로 문자를 표현
![Page 37: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/37.jpg)
Implementation Example
A-1案
![Page 38: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/38.jpg)
Summary
Immersive sound is NOT conventional 3D soundFor HMD, Immersive and Interactive are essentialBinaural means Headphone VirtualizationBinaural Rendering vs. Binaural RecordingStill open issues
Demo is OPEN in booth #5
![Page 39: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/39.jpg)
Q&A
![Page 40: [134] immersive sound vr](https://reader033.vdocuments.mx/reader033/viewer/2022042907/5870660c1a28ab48378b5029/html5/thumbnails/40.jpg)
Thank You