ee4c03 statistical digital signal processing (2016)

1 Introduction September 7, 2017

EE4C03 STATISTICAL DIGITAL SIGNAL PROCESSING (2017)

Alle-Jan van der Veen and Geert Leus

Summary

This is a second course in discrete-time signal processing:

Modeling discrete-time signals (and random processes),

Designing optimal filters and related adaptive filters (LMS, RLS, Kalman)

Estimation of the power spectrum of a random process

These are important topics that are frequently encountered in professional engi-

neering, and major applications such as digital communication, array processing,

and multimedia (speech and audio processing, image processing).

The course complements ET 4147 Signal Processing for Communications and

ET 4386 Estimation and detection.

1


EE4C03: STATISTICAL DIGITAL SIGNAL PROCESSING

Weight: 26 class hours + lab/ 5 EC credit points / 140 study hours

Classes: Wednesday 10:45, Friday 10:45

Week 1,2: Introduction; recapitulation of prior knowledge ( � -transform, discrete-

time Fourier transform, linear algebra); random processes

Week 3: Signal modeling, linear prediction

Week 3: Lunch meetings regarding lab assignments

Week 4: Examples and take-home Matlab exercise (1)

Week 4,5: Signal/system identification (Levinson, Schur algorithm)

Week 5: Spectrum estimation; frequency estimation (Pisarenko, MUSIC algo-

rithm)

Week 6: Optimal filtering (Wiener and Kalman filters)

Week 7,8: Adaptive filters (LMS, RLS algorithm)

Week 8: Examples and exercises (2)

2



Book:

Monson H. Hayes, "Statistical digital signal processing and modeling", John Wiley

and Sons, New York, 1996. ISBN: 0-471 59431-8

The website indicates which sections are part of the course.

Slides

The slides and video recordings of the course can be found on

� � � � � � ��

Exam:

Written (open book); Friday 10 November 2017, 13:30-16:30. The resit is in Jan-

uary 2018. Register at least 2 weeks in advance.

The website has examples of past exams (identical to ET4235). Questions will be

similar to those in the book.

3



Lab assignment:

The course also contains a lab assignment with size 1 EC (=28 study hours).

In week 3, we have lunch meetings introducing the (track-dependent) assignments.

In week 4, select one of the available options. Work in groups of 2. Most assign-

ments have the following structure:

Problem analysis, find related literature

“Solve” problem (Matlab)

Write a compact report

The assignment is pass/fail (need to pass), furthermore its grade counts for 20% of

your final grade.

Deadline for handing in the report: 15 November 2017.

4


1. Introduction

The (relative) importance of signal processing

IEEE has about 400,000 members. There are 42 Societies, and Signal Processing

is the 4rd largest (19,000 members), after Computers, Communications and Power

Electronics.

Downloads of articles from IEEE journals (2004):

1. Solid state circuits 1,500,000

2. Microwave technology 1,030,000

3. Signal processing 930,000

4. Magnetics 790,000

5. � � �

5


1. Introduction

Signal Processing:

Techniques and Methods

– DFT, filters, filter banks (signal analysis and reconstruction)

– Statistical signal processing (parameter estimation, detection)

– Adaptive filters, neural networks

– Analytical techniques (e.g. linear algebra, optimization)

– DSP hardware, fast algorithms/architectures (implementation)

Applications

– Communication, radar, sonar, sensor arrays (multichannel signal processing),

information theory

– Speech and audio processing

– Image, video and multimedia processing

– Biomedical/bioinformatics

6


1. Introduction

Speech and Audio Processing

The vowel /a/: � � � analog time, �� discrete time: 4 kHz, �� quantized (4 bits)

Examples: Compression (MP3), interference suppression (microphone arrays), speech

synthesis and understanding, wavefield synthesis

7


Introduction

Example: spectral analysis

Time

Frequency

8


1. Introduction

Sensor Array Processing

Examples: radio astronomy, seismic arrays, sonar arrays, synthetic aperture radar

9


1. Introduction

Signal Processing for Communication

INFORMATIONTHEORY

SIGNALPROCESSING

ELEKTRO-MAGNETISM

channel inversion(equalization)

parameter estimationdetection

SIGNALPROCESSING

ELEKTRONICS

TELECOM

ELEKTRONICS

coder

source/channel

��

D/AA/DBPFA/D

� ��

DSPD/A

� ��

� � �

� ��

Examples: multi-user multi-antenna (MIMO) communication, channel estimation,

baseband receiver design, VDSL, cognitive radio (compressive sensing), � � �

10


1. Introduction

This course

We consider mostly random signals (as opposed to deterministic signals):

Signals described by statistical properties: mean, variance, correlation, power

spectrum (i.e., just 2nd order statistics)

Signals often modeled by filters (output of filtered white noise)

• Prediction filters: minimize prediction error

• Optimal filters (Wiener/Kalman): minimize mean square error

• Adaptive filters (LMS/RLS)

Estimation of signal properties via filter parameter estimation

• Spectrum analysis

• Estimation of frequencies

11


1. Introduction

0 20 40 60 80 100−4

−3

−2

−1

0

1

2

3noisy time domain signal

time0 0.1 0.2 0.3 0.4 0.5

0

5

10

15

20

25

30

35

40

45

50corresponding frequency domain signal

normalized frequency

Two sinusoids buried in noise (e.g., a noisy music signal)

Modeling: what is a good (stochastic) model to describe the signal? How do we

estimate the model parameters?

Filtering: can we filter the noise away? Reconstruct the clean signal?

Spectrum: What is a good estimate of the spectrum? Resolution vs. noise aver-

aging: Can we detect the number of sinusoids and identify the two frequencies?

12

ee4c03 statistical digital signal processing (2016)

Documents