trends dio and oustic signal...
TRANSCRIPT
Trends
in
Audio and Acoustic Signal Processing
ICASSP 2011
Malcolm Slaney, Yahoo! Research Silicon Valley, USA
Patrick A. Naylor, Imperial College London, UK
What do we mean by ‘Trends’?
ACTIVITY# papers submitted
ACTIVITY# papers submitted
ACHIEVEMENTS
Things we can do now
that we couldn’t beforeACHIEVEMENTS
Things we can do now
that we couldn’t before
CHALLENGES
Things we want to do
but can’t do yetCHALLENGES
Things we want to do
but can’t do yet
OPPORTUNITIES
Things we didn’t
know we
wanted to do
OPPORTUNITIES
Things we didn’t
know we
wanted to do
CURIOSITIES
Things that might be
interesting but we don’t
know what to do with them
CURIOSITIES
Things that might be
interesting but we don’t
know what to do with them
ResearchResearch
Historical Trends – ICASSP Submissions
Year
Su
bm
issi
on
s
Historical Trends – ICASSP Accepted Papers
Year
Acc
ep
ted
Pa
pe
rs
Music at ICASSP
• Three sessions
– SS-L5: Music Signal Processing Exploiting Musical Knowledge
– AE-L3: Music Signal Processing
– AE-P7: Music Signal Processing
• Reasons
– New EDICS
– More content
– Commercially relevant
Big Datasets
• Easy to rip CDs
– Copyright issues
• Million-Song Dataset
– Distribute features
– Columbia and EchoNest
MIREX Competition
Musical Separation
• Sound separation
– Uses
• Understanding (key, melody)
• Transcriptions
• Multipitch estimation
– With better models
• HMM
• Scores
– Techniques
• NMF
• Matching Pursuit
• PLCA
From: POLYPHONIC AUDIO-TO-SCORE ALIGNMENT BASED ON BAYESIAN LATENT HARMONIC ALLOCATION HIDDEN MARKOV MODEL. Akira Maezawa,
Hiroshi G. Okuno, Tetsuya Ogata, Kyoto University, Japan; Masataka Goto, National Institute of Advanced Industrial Science and Technology, Japan
Music Research
• Tagging
– Genre
– Emotion
• Miscellaneous
– Morphing
– Similarity
AASP-P2.1: SOUND MORPHING BY FEATURE INTERPOLATION, Marcelo Caetano, Xavier Rodet, Institut de Recherche
et Coordination Acoustique/Musique, France
Applications vs. Algorithms?
If I’m going to be Queen, I suppose I
will not have much time left for
Audio and Acoustic Signal Processing
If I’m going to be Queen, I suppose I
will not have much time left for
Audio and Acoustic Signal Processing
My Expectation
Maximization
algorithm has converged !
My Expectation
Maximization
algorithm has converged !
If there is a trend towards things that look nice (applications), let’s not
lose sight of the fundamental power behind them (algorithms).
Microphone Array Signal Processing
• Hearing aids
• TV / Entertainment
• Linear, planar/cylindrical,
spherical, distributed
• Spacing and orientation
• Localization of sources
• Tracking
• Extraction/Separation
• Inference of room geometry
APPLICATIONS
GEOMETRY and DISTRIBUTION
TASKS
the Eigenmike®
– mh Acoustics
32 elements
8.4 cm rigid sphere
2 - 64 elements, 0.5 m, linear ‘wing’ array
Planar Array Geometry Directivity Index
measured vs theoretical
Source Separation
• ‘Cocktail party problem’
– Colin Cherry in 1950s
– Audio signals from multi-talker
distant talking scenarios
– Behavior of a listener presented
with two speech signals
simultaneously
Colin Cherry
‘TRENDSETTER’
Source Separation
• Determined and underdetermined scenarios
– Clustering based blind source separation
– Permutation problem (EM)
– Reverberation times of, say, 100 - 500 ms
Speech Enhancement
• Dereverberation technology
real-world applications
– Single and multichannel
– Acoustic channel inversion
– Speech and Music
Speech Enhancement
• Dereverberation1 technology
real-world applications
– Single and multichannel
– Acoustic channel inversion
– Speech and Music
[1] P. A. Naylor and N. D. Gaubitch, Eds., Speech Dereverberation. Springer, 2010.
Synergies
•• Joint dereverberation and blind source separationJoint dereverberation and blind source separation
•• Speech recognition of reverberant speechSpeech recognition of reverberant speech
CleanSpeechHMM
CleanSpeechHMM
Reverb-erationModel
Reverb-erationModel
Recurring Theme
S P A R S I T Y!
History of Sparsity at ICASSP
Matching PursuitMatching Pursuit
Compressed SensingCompressed Sensing
Deep Belief NetworksDeep Belief Networks
L1 RegularizationL1 Regularization
I thought:
Dumb!
I thought:
Dumb!
Spatial-Temporal Receptive Fields
• Original sparse representation (spikes!)
SS-L7.1: SPEECH PROCESSING WITH A CORTICAL REPRESENTATION OF AUDIO, Nima Mesgarani, Shihab Shamma,
University of Maryland, United States
Deep Belief Network
From: SS-L7.4: LEARNING A BETTER REPRESENTATION OF SPEECH SOUND WAVES USING RESTRICTED BOLTZMANN
MACHINES, Navdeep Jaitly, Geoffrey Hinton, University of Toronto, Canada
These are NOT your average wavelet/Gabor response!!
Nonlinear Modeling via Sparsity
Basis 1
Basis 2
All-Possible Combinations Over-complete Basis
Industrial Perspectives
“Remaining challenges [in source separation] could include BSS for
unknown/dynamic number of sources.”Tomohiro Nakatani NTT Communication Science Laboratories
Industrial Perspectives
Mixed-signal ICs for mobile phones
“Moore’s Law is driving DSP speed and memory capacity … enabling
implementation of sophisticated DSP functions that have resulted from years of
research in acoustic signal processing. The end-user experience is one of natural
wideband voice communication, devoid of acoustic background noise and
unwanted artefacts.”
Anthony Magrath Director of DSP Technology, Wolfson Microelectronics
Industrial Perspectives
“The applications of sound capture, speech enhancement, and audio processing
technologies shift gradually from communications mostly, towards speech
recognition and building natural human-machine interfaces for mobile devices, in
cars, and in our living rooms.”Ivan Tashev Microsoft Research
iTunes Screen Shot
After-thought
• Trend
– Origin: Old English trendan ‘revolve, rotate’, of Germanic origin
• “What goes around, comes around” (?)
Texture
Sparsity
• Better representations
– Sparse
• Matching pursuit
• DBNs
• features for recognition
– New angles (cortical and textures)
– Subspaces (latent and otherwise)