sound processing with short time fourier transform.pdf

8/17/2019 Sound Processing with Short Time Fourier Transform.pdf

1/15

2/5/2016 Sound Processing with Short Time Fourier Transform

http://www.numerical-tours.com/matlab/audio_1_processing/

Sound Processing with Short TimeFourier Transform

This numerical tour explores local Fourier analysis of sounds, and its application tosource denoising.

Contents

Installing toolboxes and setting up the path.Local Fourier analysis of sound.Short time Fourier transform.Audio DenoisingAudio Block Thresholding

Installing toolboxes and setting up the path.

You need to download the following files: signal toolbox and general toolbox.

You need to unzip these toolboxes in your working directory, so that you havetoolbox_signal and toolbox_general in your directory.

For Scilab user: you must replace the Matlab comment '%' by its Scilab counterpart'//'.

Recommandation: You should create a text file named for instance numericaltour.sce(in Scilab) or numericaltour.m (in Matlab) to write all the Scilab/Matlab command youwant to execute. Then, simply run exec('numericaltour.sce'); (in Scilab) or numericaltour;(in Matlab) to run the commands.

Execute this line only if you are using Matlab.

getd = @(p)path(p,path); % scilab users must *not* execute this

Then you can add the toolboxes to the path.

getd('toolbox_signal/');getd('toolbox_general/');

Warning: Name is nonexistent or not a directory: toolbox_signalWarning: Name is nonexistent or not a directory: toolbox_general

Local Fourier analysis of sound.

http://www.numerical-tours.com/matlab/toolbox_general.ziphttp://www.numerical-tours.com/matlab/toolbox_signal.zip


2/15


http://www.numerical-tours.com/matlab/audio_1_processing/ 2

A sound is a 1D signal that is locally highly oscillating and stationary. A local Fourieranalysis is thus usefull to study the property of the sound such as its local amplitudeand frequency.

First we load a sound, with a slight sub-sampling

n = 1024*16;options.n = n;[x,fs] = load_sound('bird', n);

Warning: WAVREAD will be removed in a future release. Use AUDIOREAD instead.

You can actually play a sound. In case this does not work, you need to run thecommand wavwrite(x(:)', 'tmp.wav') and click on the saved file 'tmp.wav' to read it.

sound(x(:)',fs);

We can display the sound.

clf;plot(1:n,x);axis('tight');set_graphic_sizes([], 20);title('Signal');

Local zooms on the sound show that it is highly oscilating.


3/15



p = 512;t = 1:n;clf;sel = n/4 + (0:p‐1);subplot(2,1,1);plot(t(sel),x(sel)); axis tight;sel = n/2 + (0:p‐1);subplot(2,1,2);plot(t(sel),x(sel)); axis tight;

Exercice 1: (check the solution) Compute the local Fourier transform around a point t0of x, which is the FFT (use the function fft) of the windowed signal x.*h where h issmooth windowing function located around t0. For instance you can use for h aGaussian bump centered at t0. To center the FFT for display, use fftshift.

exo1;

http://www.numerical-tours.com/matlab/missing-exo/


4/15



A good windowing function should balance both time localization and frequencylocalization.

t = linspace(‐10,10,2048);eta = 1e‐5;vmin = ‐2;

The block window has a sharp transition and thus a poor frequency localization.

h = double( abs(t)


5/15



A Hamming window is smoother.

h = cos(t*pi()/2) .* double(abs(t)


6/15



A Haning window has continuous derivatives.

h = (cos(t*pi())+1)/2 .* double(abs(t)


7/15



A normalized Haning window has a sharper transition. It has the advantage of generating a tight frame STFT, and is used in the following.

h = sqrt(2)/2 * (1+cos(t*pi())) ./ sqrt( 1+cos(t*pi()).^2 ) .* double(abs(t)


8/15



Short time Fourier transform.

Gathering a local Fourier transform at equispaced point create a local Fouriertransform, also called spectrogram. By carefully chosing the window, this transformcorresponds to the decomposition of the signal in a redundant tight frame. Theredundancy corresponds to the overlap of the windows, and the tight framecorresponds to the fact that the pseudo-inverse is simply the transposed of thetransform (it means that the same window can be used for synthesis with a simplesummation of the reconstructed signal over each window).

The only parameters of the transform are the size of the window and the overlap.

% size of the window w = 64*2;% overlap of the window q = w/2;

Gabor atoms are computed using a Haning window. The atoms are obtained bytranslating in time and in frequency (modulation) the window.

t = 0:3*w‐1;t1 = t‐2*w;f = w/8;% Position 0, frequency 0. g1 = sin( pi*t/w ).^2 .* double(t


9/15



% Position 2*w, frequency w/8

g4 = g2 .* sin( t * 2*pi/w * f);% display clf;subplot(2,2,1);plot(g1); axis('tight');title('Position 0, frequency 0');subplot(2,2,2);plot(g2); axis('tight');title('Position 2*w, frequency 0');

subplot(2,2,3);plot(g3); axis('tight');title('Position 0, frequency w/8');subplot(2,2,4);plot(g4); axis('tight');title('Position 2*w, frequency w/8');

We can compute a spectrogram of the sound to see its local Fourier content. Thenumber of windowed used is (n‐noverlap)/(w‐noverlap)

S = perform_stft(x,w,q, options);

To see more clearly the evolution of the harmonics, we can display the spectrogram inlog coordinates. The top of the spectrogram corresponds to low frequencies.

% display the spectrogram clf; imageplot(abs(S)); axis('on');% display log spectrogram plot_spectrogram(S,x);


10/15



The STFT transform is decomposing the signal in a redundant tight frame. This can bechecked by measuring the energy conservation.

% energy of the signal e = norm(x,'fro').^2;% energy of the coefficients eS = norm(abs(S),'fro').^2;

disp(strcat(['Energy conservation (should be 1)=' num2str(e/eS)]));

Energy conservation (should be 1)=1

One can also check that the inverse transform (which is just the transposed operator -it implements exactly the pseudo inverse) is working fine.

% one must give the signal size for the reconstruction x1 = perform_stft(S,w,q, options);disp(strcat(['Reconstruction error (should be 0)=' num2str( norm(x‐x1, 'fro')./norm(x,'fro') ) ])

Reconstruction error (should be 0)=2.2401e‐16

Audio Denoising

One can perform denosing by a non-linear thresholding over the transfomede Fourierdomain.

First we create a noisy signal


11/15



sigma = .2;xn = x + randn(size(x))*sigma;

Play the noisy sound.

sound(xn,fs);

Display the Sounds.

clf;subplot(2,1,1);plot(x); axis([1 n ‐1.2 1.2]);set_graphic_sizes([], 20);title('Original signal');subplot(2,1,2);plot(xn); axis([1 n ‐1.2 1.2]);set_graphic_sizes([], 20);title('Noisy signal');

One can threshold the spectrogram.

% perform thresholding Sn = perform_stft(xn,w,q, options);SnT = perform_thresholding(Sn, 2*sigma, 'hard');% display the results subplot(2,1,1);plot_spectrogram(Sn);subplot(2,1,2);plot_spectrogram(SnT);


12/15



Exercice 2: (check the solution) A denoising is performed by hard or soft thresholdingthe STFT of the noisy signal. Compute the denosing SNR with both soft and hardthresholding, and compute the threshold that minimize the SNR. Remember that asoft thresholding should be approximately twice smaller than a hard thresholding.Check the result by listening. What can you conclude about the quality of thedenoised signal ?

exo2;



13/15



Exercice 3: (check the solution) Display and hear the results. What do you notice ?

exo3;



14/15



Audio Block Thresholding

It is possible to remove musical noise by thresholding blocks of STFT coefficients.

Denoising is performed by block soft thresholding.

% perform thresholding Sn = perform_stft(xn,w,q, options);

SnT = perform_thresholding(Sn, sigma, 'block');% display the results subplot(2,1,1);plot_spectrogram(Sn);subplot(2,1,2);plot_spectrogram(SnT);

Exercice 4: (check the solution) Trie for various block sizes and report the bestresults.

exo4;



15/15


Copyright (c) 2010 Gabriel Peyre

sound processing with short time fourier transform.pdf

Documents