signal processing for spatial sound control - ei.rub.de · • dolby axon → audio demonstration...

Signal Processing for Spatial Sound Control

Dr.-Ing. Gerald Enzner

Institut für Kommunikationsakustik (IKA)

Winter term, 2010/2011

Spatial Audio Systems Digital Signal Processing Rapid Prototyping Course Organization Student Project

Outline

◮ Spatial Audio Systems

◮ Digital Signal Processing

◮ Rapid Prototyping with Matlab

◮ Course Organization

◮ Previous Student Project: HRIR Demonstrator

Dr.-Ing. Gerald Enzner • Ruhr-Universität Bochum 2/31


Motivation of Spatial Sound Control

◮ The Dimension of Spatial Sound

• spaciousness or outdoorness (e.g., wind, backgroundnoise, roads, towns, birds, distant objects)

• indoor sound characteristics (e.g., room size,reverberation, timbre, reflections, closer objects)

• sound source properties (e.g., direction, distance, size,focus, diffuseness, directivity)



Motivation of Spatial Sound Control

◮ The Dimension of Spatial Sound

• spaciousness or outdoorness (e.g., wind, backgroundnoise, roads, towns, birds, distant objects)

• indoor sound characteristics (e.g., room size,reverberation, timbre, reflections, closer objects)

• sound source properties (e.g., direction, distance, size,focus, diffuseness, directivity)

◮ Human Auditory System

• resolves spatial sound cues (sound characteristics)

• interacts with the acoustical environments

• performs auditory scene analysis (grouping andsegregation of sound sources)



Spatial Sound Using Loudspeakers



History and Driving Forces

◮ Basic Evolution of Loudspeaker Reproduction Systems

• stereophonic sound: Ader’s experiments 1881, Blumlein’spatent 1931, consumer stereo after 1950’s (Vinyl and FM)

• multi-channel stereo (3,4,5,6-channel): first, using analogmatrix encoding, then, digital low-bit-rate encoding

• standardization: ITU-R 5.1 surround loudspeaker format



History and Driving Forces

◮ Basic Evolution of Loudspeaker Reproduction Systems

• stereophonic sound: Ader’s experiments 1881, Blumlein’spatent 1931, consumer stereo after 1950’s (Vinyl and FM)

• multi-channel stereo (3,4,5,6-channel): first, using analogmatrix encoding, then, digital low-bit-rate encoding

• standardization: ITU-R 5.1 surround loudspeaker format

◮ Accelerators

• moving pictures industry (music industry less successful):Dolby, Digital Theater Systems, Sony, MPEG

• storage media and audio codecs: CD, DVD, Blu-ray

• consumer electronics and related technical sound quality(i.e., SNR, dynamic range, frequency range)

• low-cost integrated circuits: DSP, ASIC, FPGA



Ader’s Stereo Experiment

Figure 3-1: The first stereophonic transmission by Clement Ader in the year 1881 (from

Daniels, 2002): Listeners enjoy a performance of the Paris opera house transmitted by two

telephone lines. Ader patented this stereo telephone.



Dolby Sorround Sound Formats

Dolby Sorround(for home media)

Mono OpticalSoundtrack

Dolby Digital 5.1 (AC−3 Transform Audio Coding)Dolby Digital Sorround EX (plus rear−center channel)

1976

Dolby SR1987

19991992

(improved dynamic range)

Dolby Stereo(left, center, right, surround)



Dolby Stereo/Surround Playback

Source: “Surround Sound: Past, Present, and Future”, Dolby



Dolby Digital Playback

Source: “Surround Sound: Past, Present, and Future”, Dolby



Virtual Auditory Space using Headphones

Examples of Auditory Virtual Environments (AVEs):

• Dolby Axon → Audio Demonstration (www.dolby.com)

• IKA-SIM (Institut für Kommunikationsakustik)



Head-Related Impulse Responses (HRIRs)

replacements

θ

x(k)

yr(k)

yℓ(k)

head-related impulse responses hr/ℓ(k, θk) describe thefree-field acoustical transfer from sound sources to thehuman ear canal – including reflection, diffraction, andrefraction of sound waves at head, pinnae, and torso:

yr/ℓ(k) =∑N

κ=0 hr/ℓ(κ, θk)x(k − κ)



Numerous Applications of AVEs

• auditory displays, e.g., in aircrafts and flight simulators

• design and evaluation of performance rooms / music halls

• audio effects, e.g., “spatializers”

• music, games, television

• usability research and product-sound design, e.g., in cars

• augmented reality, e.g., in navigation

• virtual museum, virtual tourism, audiological self-screening

• tele-conferencing, tele-presence, tele-operation

• individual interactive movie sound

• www interfaces, e.g., second-life

• . . .



Research Topics in Spatial Audio

◮ Rendering of 3D-Auditory Virtual Environments

◮ 3D-Audio Reproduction Systems (Beyond 5.1, 7.1, 10.2)

◮ Increased Compatibility of Sound Systems

◮ Standardization of Advanced Audio Formats and Codecs

◮ Spatial Sound Reproduction for Telecommunications:Telepresence, Hands-free Communication

◮ Binaural Sound for Digital Hearing Aids

◮ Acoustic Fiction or Physically Precise Rendering?

◮ ...



Outline








What is a Signal ?

• A signal is a function of one or more variables, e.g., time orlocation:

x : t → x(t)

x : t → ~x(t)

x : (u, v) → x(u, v)

• Examples are:

• audio signals (captured by a microphone)

• image and video signals (captured by a camera)

• radio signals (captured by an antenna)

• biological signals (e.g., captured by electrodes)

• seismic signals (captured by accelerometers)



Example: Speech Signal

0 1 2 3 4 5−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

t/s

x(t)



Example: CT Image



Example: Seismic Signals

Earthquake on Feb 22, 2003 with centre in the Vogese and with magnitude

5.5 (http://sdac.hannover.bgr.de).



What is Signal Processing ?

• Signal processing is the art of modifying signals for thepurpose of analysis, transmission, synthesis, etc.

• Typical signal processing tasks are

• filtering (e.g. highpass, lowpass)

• signal analysis and pattern recognition (e.g. ASR)

• signal compression (e.g. GSM/UMTS voice codecs)

• signal enhancement (e.g. echo and noise reduction)

• signal synthesis (e.g. auditory virtual environments)

• Various applications: audio signal processing, image andvideo signal processing, signal processing for radiocommunications, medical signal processing, etc.



What is Digital Signal Processing ?

• In general, there are continuous (analog), discrete-time,and digital signals.

• Discrete-time signals are sampled continuous signals.

• Digital signals are sampled and quantized continuous signals.

• Digital signal processing is the modification of digitalsignals of by means of Personal Computers (PC’s), DigitalSignal Processors (DSP’s) or Application SpecificIntegrated Circuits (ASIC’s).

• Digital signal processing is ubiquitous (e.g. MP-3 player,mobile phones, car-control)

• In MATLAB, digital signals (floating-point most of the times)are represented by vectors and matrices.



Outline








What is Rapid Prototyping ?

• First and quick (but not dirty!) implementation ofalgorithms, ideas, user interfaces, etc.

• Using tools which do not require extensive programmingeffort (e.g. provide for simple memory allocation)

• Using tools which provide for rich application-orientedlibraries and simple user-data interfaces

• No optimization w.r.t. memory, runtime efficiency, etc.

• Floating point arithmetics in most of the cases

• Platform independence (Operating System, Processor)

• Verification of algorithm functionality counts



What is MATLAB ?

“MATLAB is a high-performance language for technicalcomputing. It integrates computation, visualization, andprogramming in an easy-to-use environment where problemsand solutions are expressed in familiar mathematical notation.

Typical uses include

• Math and computation algorithm development

• Data acquisition, modeling, simulation, and prototyping

• Data analysis, exploration, and visualization

• Scientific and engineering graphics

• Application development, including graphical user interfacebuilding.”

(The Mathworks Inc.)



Why MATLAB ?

• Easy-to-use interpreter language.

• MATLAB is the de facto industry standard for thedevelopment of signal processing and control.

• Many application libraries (’toolboxes’) available.

• Excellent and easy-to-use graphics, I/O, and userinterfaces.

• C/C++ code generation and real-time capabilities.

• Frequently used in textbooks for illustration and demopurposes.



Why MATLAB ?

• Easy-to-use interpreter language.

• MATLAB is the de facto industry standard for thedevelopment of signal processing and control.

• Many application libraries (’toolboxes’) available.

• Excellent and easy-to-use graphics, I/O, and userinterfaces.

• C/C++ code generation and real-time capabilities.

• Frequently used in textbooks for illustration and demopurposes.

Disadvantages: price, licensing (check for student license!)Open source clone: OCTAVE (see www.octave.org)



Outline








Course Objectives

Basics of

Processing

Concepts of

SystemsDigital Signal Spatial Sound

Rapid Prototyping of

in Student Research Project"Signal Processing for Spatial Sound Control"



Course Outline

◮ Digital Signal Processing• Discrete Signals and Systems• Spectral Analysis• MATLAB Basics / Tips and Tricks / FIR Filtering• z-Transform Analysis



Course Outline


◮ Loudspeaker-Based Sound Systems

• “Audio Technology and Spatial Hearing” Lab• Room-Related Presentation of Auditory Scenes (Blauert)• Adaptive Filtering for Active Noise Control & Sound Projection



Course Outline


◮ Loudspeaker-Based Sound Systems

• “Audio Technology and Spatial Hearing” Lab• Room-Related Presentation of Auditory Scenes (Blauert)• Adaptive Filtering for Active Noise Control & Sound Projection

◮ Binaural Sound Systems (Headphone-Oriented)

• Auditory Virtual Environments using HRIR/HRTF• Transaural System with Crosstalk Cancellation• Student Research Project (Rapid Prototyping Project)



Course Info

• [email protected], (0234) 32 25392, ID/2/227

• Lecture with Integrated Exercises (1+1):Monday 16:00-17:30, ID/03/419 and ID/03/121 (CIP-Pool)

• Related Books:

• K.D. Kammeyer, K. Kroschel: “Digitale Signalverarbeitung”,Teubner, 2002

• Matlab Tutorial, The Mathworks, www.mathwork.com

• P. Vary, R. Martin, “Digital Speech Transmission”, Wiley, 2006

• F. Rumsey, “Spatial Audio”, Focal Press, 2001

• J. Blauert, “Spatial Hearing”, MIT Press, 1996

• Oral Exam: 25% DSP, 50% Audio, 25% Project, 5% Bonus



Outline








Task Overview (Shortened)

Create a Matlab tool for the calculation of HRIRs (head-relatedimpulse responses) and headphone-based presentation of theirspatial sound attributes:

• The HRIRs as required in auditory virtual environments shallbe calculated according to algorithms described in literature.

• Using anechoic input signals, the HRIRs shall then beemployed to synthesize the impression of fixed and virtuallymoving (rotating) sound sources in space.

• Using Matlab’s “callback” mechanism, a user-friendly GUIshall be designed in order to control and evaluate the virtualauditory space.



Sample Solution / Audio Demo


signal processing for spatial sound control - ei.rub.de · • dolby axon → audio demonstration...

Documents