Spatial Covariance Models For Under-Determined Reverberant

Audio Source Separation

N. Duong, E. Vincent and R. Gribonval

METISS project team, IRISA/INRIA, FranceOct. 2009

1

Content

2

Under-determined source separation

Spatial covariance models

Model parameter estimation

Experimental evaluation

Conclusion

Under-determined source separation

3

Use recorded mixture signals to separate sources , where

Convolutive mixing model: Denote the vector of mixing filters from source to microphone array, the contribution of to all microphones and the vector of mixture signals

are computed as:

img

1

( ) ( )J

jj

t t

x s

img ( ) ( ) ( )j j jt s t

s h

1( ) ( ),..., ( )T

It x t x tx( )js t

IJ I J

1 ,...,T

j j Ijh h hJ

( )js timg ( )j ts

BSS approaches

Sparsity assumption: only FEW sources are active at each time-frequency point

Binary masking: only ONE source is active at each time-frequency point

L1-norm minimization:

Short-term Fourier

transform

img ( )

( ) ( )

j t

j jj

t s t s

x h

( , )

( , ) ( ) ( , )

imgj n f

j jj

n f f s n f s

x h

,n f

( , ) 1

arg min ( , ) , s.t. j

J

js n f j

s n f

4

Beamforming model

is denoted as and approximated by the distance between each source to microphones [T. Gustafsson et.al.], i.e. in stereo mixture:

Covariance matrix of source images2

( )( , )

( , ) ( , ) ( ) ( )imgj

jj

Hj j j

fv n f

n f s n f f fs

R

R a a

Spatial covariance matrix (rank 1) modeling the mixing

process

( )j fa( )j fh

1

2

2

1

2

2

1 4( )

1 4

j

j

i f r c

j

j i f r c

j

r ef

r e

a

5

Sourcevariance

Spatial covariance models

6

Purpose of the paper: explore the extension of Gaussian framework, i.e. and , that better account for reverberation

We evaluate potential separation performance by estimating the spatial model parameter from training data

Source separation by Wiener filtering

Models for spatial covariance matrix: Rank-1 convolutive model Rank-1 anechoic model Full-rank direct+diffuse model Full-rank unconstrained model.

( , ) ( , ) ( )imgj

j jn f v n f fsR R ( , ) 0, ( , )j js n f N v n f

1imgˆ ( , ) ( , ) ( ) ( , ) ( ) ( , )j j j j jjn f v n f f v n f f n f

s R R x

Rank-1 models

7

Rank-1 anechoic model

Where is steering vector specified in the beamforming approach

Rank-1 convolutive model

Where is the Fourier transform of the mixing filters

( ) ( ) ( )Hj j jf f fR h h

( )j fh ( )j h

( ) ( ) ( )Hj j jf f fR a a

( )j fa

Full-rank direct+diffuse model

8

Assuming that the direct part and the reverberant part are uncorrelated and the reverberant part is diffuse

where and can be specified from statistical room acoustic, i.e. depends on the microphone distance , wall area , and wall reflection coefficient

- In the rectangular room:

2

covariance of direct partcovariance of reverberant part

1 ( , )( ) ( ) ( )

( , ) 1H

j j j rev

d ff f f

d f

R a a

2rev ( , )d f

sin(2 / )( , )

(2 / )

fd cd f

fd c

2

2

2

4

1rev

A

d A

Full-rank unconstrained model

9

- A more general model than the previous models where the coefficients of are not related a priori

- Allows more flexible modeling of the mixing process since the reverberation part is rarely diffuse and is correlated with the direct part in practice

- Expected to improve separation performance of real-worldconvolutive mixtures.

( )j fR

Model parameter estimation

We investigate the potential separation performance achievable via each model in:

Semi-blind context: Spatial covariance matrices are estimated from true source images but source variances are blindly estimated from the mixture in the ML sense

Where is the Kullback-Leibler (KL) divergence between the empirical covariance matrices and the model-based matrices.

Oracle context: Both and are estimated from the true source images.

( )j fR

( , )jv n f

( )j fR ( , )jv n f

( , ), ( )

ˆ ( , ) | ( , ) ( )arg minj j

KL j jjv n f f

D n f v n f fxR

R R

|KLD

Experiment Purpose:- Compare the source separation performance of the model-based algorithms

- Criteria: SDR, SIR, SAR

r

s1s2

s3

m1 m2

1.8m

1.5m

Room dimensions: 4.45 x 3.35 x 2.5 mSource and microphone height: 1.4 mMicrophone distance: d = 20 cm or 5 cmSource-to-microphone distance: 120 cm or 50 cm

Experimental setup:

- Speech length: 5 seconds

- Sampling rate: 16 kHz

- Sine window for STFT with length of 1024 taps

11

Experimental result

Context Covariance modelNumber of

spatial parameters

SDR SIR SAR

Semi-blind

Rank-1 anechoic 6 0.9 1.7 4.9Rank-1 convolutive 3078 4.0 6.4 6.5

Full-rank direct+diffuse

8 3.1 6.1 5.6

Full-rank unconstrained

6156 5.8 10.3 7.9

Binary masking 3078 3.3 10.3 2.9L1-norm minimization 3078 2.4 8.1 3.8

Oracle

Rank-1 anechoic 6 0.4 4.4 7.0Rank-1 convolutive 3078 4.2 10.2 5.3

Full-rank direct+diffuse

8 10.217.311.4

Full-rank unconstrained

6156 10.917.912.1

12

Conclusion

13

- Proposed to model the convolutive mixing process by full-rank spatial covariance matrices

- Experimental results confirm that full-rank spatial covariance matrices better account for reverberation and potentially improve separation performance compared to rank-1 matrices.

Work in progress- Validated the power of the proposed algorithms over real-world recordings with small source movement (demo session)- Blind context: learning the model parameters from the recorded mixture (submitted to ICASSP 2010 ).

Future work:- Consider separation of diffuse and semi-diffuse sources

Thanks for your attention!See you again in the demo session

tonight & Your comments…?

14