MATHEMATICS FOR

AUTOMATIC CONTROL

AND SIGNAL PROCESSING

Professor Jean Claude CARMONA

Some references

Analisis de seales, IRARRAZAVAL M. PABLO, Mac Graw Hill, 1999. Class.TK 5102.9 .I73 1999

Digital Signal Processsing: principles, algorithms and applications,PROAKIS J.G., and al., PrinticeHall 1996, TK 5102.9 .P757 1996

Introduction to random signals and applied Kalman filtering, GROVERBROWN R. and , PATRICK Y.C. HWANG P., Class. TK 5102.5 .B768I

Mathematical methods in Signal Processing and Digital Image,DAHLHAUS, R. ED, Springer 2008, Class. TK 5102.9 M3837 2008

Digital Signal Processsing laboratory: LabView based FPGAimplementation, KEHTARNAVAZ N. and MAHOTRA S.,Bown Walker Press,TK 5102.9 .K443 2010

Instrumentacion virtual: adquisicion, procesado y analisis de senales,ANTONI M., BIEL D., 2002, Class: TK 5102.9 .M36 2002

SUMMARY

Fourier transforms

Classical filtering

Adaptive filtering

Estimation: some basical concepts

Linear optimal festimation: Kallman filter

Optimisation: classical algorithms:

- application to system identification

- application to optimal control laws

Evaluation

Written evaluations: theoretical evaluations

mid term and final evaluations

Mini project:

work on experimental data coming from a real process:

- data acquisition

- treatment: filtering, estimation, .

- application to control ? .

I.1.1. Definition

For s(t) with period T : (1)

, and conversely : (2)

= =

+

+2

=1

0

()2

example ip(t) : periodic comb (T) , duration t and magnitude a

Chapter 1: Fourier analysis of an analogical signal

I.1. Fourier series of a periodical signal

1

=

(

)

=1

/2

+/2

2 =

()

therefore :

(3)

then : (4) =

=

+

(

) +2

I.1.2. Main properties

a. Power conservation ( Parceval theorem)

=

+

2 =

1

0

()2 (5)

b. Magnitude spectrum features i.e. function

Periodic signal (T) discrete spectrum (k/T)

ray spectrum : the harmonics 2

I.2. Fourier transform of a function

For any summable signal s(t) : (6)

with : (7)

S(f) is called the Fourier transform (FT) of hte signal s(t)

(6) is called the reverse Fourier transform formula.

=

+

()+2

=

+

()2

example i(t) : single pulse signal , duration t and magnitude a

I.2.1. Definition

3

=

+

()2 = /2

+/2

2

= ()

= ()

Let us compute its Fourier transform I(f) :

Its cancellation points are : , and: IMax =I(0) at

Infinite and multi lobed magitude spectrum (lobe width 1/t)

= 0 =

,

I.2.2. Main properties

a. Power conservation ( Parceval theorem)

+

() 2 = 0

()2

(8)

(9)

b. Differential theorem

+

2 = 2() (10)

4

c. Convolution theorem

= =

+

()

=

+

()2 .

+

()2

= . (11)

I.2.3. Case of the Dirac distribution

a. Definitions

+

= 1 (12)

b. Main property

, ,

+

= () (13)

Remarks :

1.

2.

0

= , (14) 5

I.2.4. Fourier transform of a distibution

()1From definition of d(t) :

d(t) is the unit (simplest one) in the "fequency" domain (TF)

Important consequence :

u(t) : the time unity comb (distribution)

t

period : T

level : 1

= lim0

= =

+

( )

u(t) periodic Fourier series development:

its Fourier transform:

mind the difference (1/T) between FS and FT !

U(f) is also a unit comb but in the frequency doamin (period 1/T) !!

, =1

= =

+

(

)

(15)

(16)

(17)

6

I.3. Application to continuous time signal sampling

1.3.1. modeling sampled continuous time signal

a. additive model :

= =

+

( ) (18)

b. multiplicative model :

= . (19)

remark : u(t) is already a sampled signal , since:

Dont forget that u(t) and U(f) are very particulars and play particular roles:

u(t) is the unique signal that have teh same shape that its TF U(f)

= lim0

=

+

( )

7

1.3.2. Spectrum of a sampled continuous time signal

Idea: use the multiplicative model and the convolution theorem

= . = =

+

= =

+

(

) =

=

+

( )replace with :

and use :

But, in the frequency domain !

Finally, it holds:

, ,

+

= ()

= =

+

( )

Important features:

The spectrum X*(f) is peridodic (Fe)

We dont have loss of information, on the contrary !

(20)

8

1.3.3. Sampling fundamental theorem, Shannon theorem

Principle: signal " well sampled" : we can retrieve it (the information it contains)

in fact, the problem is:

Let us draw the shape of the magnitude spectrum

2 cases :

1. Fe sufficiently large

It is easy to retrieve

a low pass filter at Fe /2

2. Fe too small

Spectrum aliasing problem ! (distorsion )

good sampling iff Fe > 2 fMax (the spectrum width of the signal) Shannon

9

I.4. Training exercices

1.4.1. FT of a rectangular impulse (see fig. 1.2.)

Let the signal s(t) be a rectangular impulse signal with length a and height

b.

Show that its TF is given as follows:

Draw the shape of the magnitude spectrum of s(t).

= ()

1.4.2. FT of a exponential impulse

Let the signal s(t) be an exponential impulse signal given as follows:

Show that its TF is given as follows:

Draw the shape of the magnitude spectrum of s(t).

Compare this result to the case of a rectangular impulse (the shape changes)

() =

1 + ()2

= 2 /

10

1.4.3. FT of a Lorentz impulse

Let the signal s(t) be a Lorentz impulse signal given as follows:

Show that its TF is given as follows:

Draw the shape of the magnitude spectrum of s(t).

hint : use the reciprocity theorem and the scale change theorem

=1

1 + (/)2

= () = ()

= () () =1

(

)

1.4.4. FT of a periodic signal

Let the signal s(t) be a positive square periodic signal, with period T,

Height 1 a and duty cycle 0,5.

Start with computing its Fourier series (FS).

Using:

( 0) = 20

and the reciprocity theorem (see above), compute its Fourier transform S(f).

11

I.5. Practical application:

Let us consider here the problem of (numerical) data acquisition on a process.

We calssically use a so called "acquistion chain" represented by the following scheme:

process sensor conditionning

Sample and hold

ADC

coding

anti-aliasing filter

bus

m-processor

coding/dcodage

Coding/decoding

Intranet/Internet

IN

Data acquisition chain

x(t)

xn

x*(t)

12

We are interested in the signals:

x(t) : the measure / analogic signal

i.e. exploitable signal given by the sensor + conditionning (pre treatment)

x*(t) : sampled (Te) / analogic signal

xn : converted (Te) / numerical signal ( i.e. the data )

and the way to obtain them in good conditions in order to conserve the information!

1. How sample x(t) without loosing information (choice of Te ? ) 2. How convertind (ADC) without loosing information ?

Idea: The frequency approach, i.e. using Fourier transforms

and analysing the magnitude spectrum of x(t) , x*(t) and xn .

The first question here is :How choosing the good Fe for a given signal, in particular in the general case ?

Methodology: 1. compute the TF X(f) of the signal x(t)

2. draw its magnitude spectrum , i.e. X(f)

3. determine the upper bound of its spectrum, i.e. fMax4. apply Shannon theorem , i.e. Fe > 2 fMax

Problem :

13

Difficulties :

1. compute the TF X(f) of the signal x(t)

easy for simple signals but for a general case (real) signal !!

Idea : the more a signal is rapid, the more it is difficult to sample it: fMax

Analysis the typical cases of:

1. a very rapide variation in the signal (i.e. a step dicontinuity type)

2. a progressive variation in the signal (i.e. a linear signal, triangular)

practically, we will analyse:1. the case of a rectangular impulse signal

2. the case of a triangular signal

and determine the frequency cost , i.e. fMax , to well sample them.

For a general signal, it will depend on the fact it contains these cases , for choosing the convenient estimation of fMax , and then deduce Fe.

14

1. Rectangular impulse signal

duration t , height aa. Show that the frequency harmonics decrease in 1/f .b. The spectrum is theoritically infinite, so we must decide to

make an approximation in order to fix fMax.Classically, we decide of a quality criterion, i.e. fMax for lessthan 1% of magnitude.Compute the cooresponding fMax , then propose Fe.

c. Numerical applications: t =100ms , t = 1 ms and t = 100 ms;Conclude on the sampling cost of these signals.

2. Triangular signal

To facilitate the calculus, choose a traingle with a for the base and the height, namely t.The same questions and conclusions.To numerically compare the two cases, take the same values of t, as in 1.

15

Chapter 2: Fourier analysis of an numerical signal

Interest:

- manipulate data records relevant informationex: RADAR signal extraction position/speed of an air plane

- modifications: add "good properties

ex: noise reduction/caneelation

sentences synthesis for telephone recording

information transmisson on a specific canal,

Interest of numerical signals

II.1. Time representation of a numerical signal

II.1.1. Definition

A numerical signal xn or x(n) can be modelised by a function from the Z set to the real set R.

II.1.2. Basical numerical signals

1. unit impulse defined as follows:

= 1 = 00

2. unit step defined as follows:

= 1 00

3. Sign function defined as follows:

Signn = 2 Un 1

4. Gate function defined as follows:

RectN n = Un+N Un-N

5. Sine function defined sas follows: (frequency f0)

xn= = X0 Sin (2pf0n + j)

6. Complex exponential function : = 0

20

II.2. Discrete time Fourier transform: dtFT

II.2.1. Definition

= =

+

2 (21)

where: g = f / Fe is the reduced (normalized) frequency, related to the discret time : n = t / Te (22)

2 = 2

= = = = (

) (23)

II.2.2. Main properties

X(g) is continue in g , and periodic in g, with period 1

vocabular: the basic period [0; 1[ or [-1/2 ; +1/2 [ is called the fundamental interval

the inverse dtFT :

linearity : axn + byn aX() + bY()

time translation :

(24)

= 1/2

+1/2

()+2 (25)

0 20

frequency translation : (modulation)

time inversion: x-n X(- )

convolution product: (26 )

Parseval formula : (27 )

20 0

= =

+

.

=

+

2 =

1/2

+1/2

() 2

II.2.3. Main example the dtFT of the unit gate signal (duration t 2N+1 samples)

= =

+

2 = +2 (1 + 2 .+ 2(2+1))

= ( 2+1 )

()(28)

the magnitude spectrum is not evident to draw in the fundametnal interval !

from Matlab:

application: multilobated spectrum but with a very fast decreasing

it is shown that almost 90% of the power spectrum is located in the first lobe !

We shall consider only this first (main) lobe. We measured appromitaly:

D 0,15 fMax 0,03 Fe : upper bound of the numerical signal.

Let us resume the analogical signal with width t =1 ms , sampled at Fe 2 . 33/t , 100kHz for example.

For the numerical signal obtain after numerization, we have: fMax 0,03 .105 = 3 kHz

conclusion: the numerical signal "occupies" 3 kHZ since the original continuous time signal "occupies" 33 kHz . NO COMMENT !!

() 2

II.3. Discrete Fourier transform: DFT

Motivation : Computing dtTF is too bulky :infinite sum in data (xn) + infinite number of frequencies (g)

Approximation : the DTF is an approxilation of the dtTF defined on:a large but finite number of data (xn) + contribution of a finite number of frequencies (g)

, where: k=1,2 ..N , g = k/N and: W e-i2p/N (29 ) = =0

1

In fact, we compute the DFT X(k) for N frequencies , on the base of N data xn of the signal.W is the Nth root of the unity, in the complex set C, i.e. the solution of zN = 1 in C

the DFT inverse is:

=1

=01()

In practical application, we want to optimize (reduce) the computaional time of DFT.

The FFT (Fast Fourier Transform)

remark : it is not an approximation of the DFT. Simply, it is a better way to compute it !

II.4. the Fast Fourier Transform (FFT ):

It is an optimization algorithm due to Cooley Tuckey

1. first idea: N = 2p . We say : a FFT on 2P points (4, 8, 16, 64, 128, 256, points)

2. second idea: Rewrite cleverly X(k) , that is:

for example: N = 8 points : data x0 , x1 , x2, x3 , x4 , x5 , x6 and x7 frequency contributions: X(1) , X(2) , X(3) , X(4) , X(5) , x(6) and X(7)

= 00+ 11+

22+ 33+

44+ 55+

66+ 77

= 00+ 22+

44+ 66+ (

11+ 33+

55+ 77)

Saying that: W0k = 1

= 0+ 22+

44+ 66+

1(1+ 23+

45+ 67)

= [(0+ 22)+

4(4+ 26)]+

1[(1+ 23)+

4(5+ 27)]

Moreover:

We notice that we can operate in the same time, x0 and x2, and , x4 and x6, and, x1 and x3, and , x5 and x7,

with the same operator:

1 time the first number + W2k the second number. Oreover

In fact, we can compute in parallel the 4 parenthesis ( xi + W2k xj) ,

i.e. the same operation with different entries time improvement (optimization)

The following (second) step: compute the brackets [ () + W4k () ] once more , using tehe same operator (a second one) with two couple of different entries , i.e. parallel computing.

Final step (third step); compute X(k) = [ ] + W8k []

Remark: the 3 calculus corresponding to the three steps , i.e. compute the 4 parenthesis (), than the 2 brackets [],

and finally X(k) itself, are of the same type:

a and b a + Wr.b with a certain power of W, Wr : W2k , then W4k and finally W8k

Let us introduce the following symbolic operator:

In fact, using the properties of the 8th roots of unity, we have some

useful symetric propereties of Wr, namely:

W4k = - W0k = -1 , W5k = -Wk , W6k = - W2k and W7k = - W3k

we can use the second (bottom) output of this operator !

Finally, sumarizing the three computing steps and using this operator, we have:

Chapter 3: Linear Invariant Discrete Systems (LIDS)

Idea: deal about nuemrical filters almost, linear invariant

Linearity : x1n y1n and x2n y2n ax1n + bx2n ay1n + by2n

Time invariance : xn yn xn-1 yn-1

Stability (1st form) : If xn is bounded, i.e. > 0 , , < > 0 , ,

III.1. Recurent equations:

example: yn = acxn-1 + bcxn-2 + cyn-1 (recursivity order 1 since yn-1 )

model closest to the implementation algorithm !

III.2. Impulse response:

Definition : input xn dn yn hn output

Fundamental property : (basis of linear dynamic systems) : convolution theorem

, , = (30)

Stability (2d form) : (31) =

+

< +

Causality: (32) < 0 , = 0

Discrete convolution: (33)

example: hn = an Un and : xn = bn Un yn = ?

= = =

+

= =

+

example : Classically, we introduce the elementary following operators

Addition operator:

Multiplication operator:

Delay operator

1. give the recursive equation of this "filter

2. deduce its impulse response hn

3. draw its shape

remark : examples of nonlinear and/or time varying disrete systems

Some basical relations

= ,

,

= =

+

= =

+

III.3. Frequency response:

Principle : Let us calculate the output of an DLIS with sine input with frequency f :

Let us take the DFT:

where H(jf) is the DFT of the impulse response hn.

Then , the output signal yn is a sine signal with frequency f , where:

(34)

= =1

2 + = 1 + 2

= = . , = 1,2

= cos + ,: = () , = ()

Some examples:

Give the recusrisve equations, the impulse responses hn and the frequency transfer H(if).

III.4. The z Transform of a numerical signal: (zT)

II.4.1. Definition

Let us remember the Laplace transfom (LT) of a continuous time signal x(t) :

(35)

For a numerical signal xn , for example the one given by the discretization of x(t) :

(36)

remark: we can say its a development in integer series of z .

examples :

=

+

()

() = =

+

= = ?

= = ?

gate signal: = 1 < , = = ?

exponential signal: = = ?

does X(z) ever exists ? pb of the integer series convergence !!

II.4.2. Convergence of the zT

Some results:

1. xn left bounded, i.e. 0 < 0 , = 0 X(z) converges for all z outside a given circle C

2. xn right bounded, i.e. 0 > 0 , = 0 X(z) converges for all z inside a given circle C

3. xn left-right bounded, i.e. of finite length ( n1 n n2 ) X(z) converges for all z except , maybe, for 0 and .

Practical consequence: We almost always use finite record "of signal" case 3 always convergence !

Examples: let us study the convergence of the folowing time series

1. xn = Un and yn = - Un 2. xn = a

n Cos (n).Un , with: a = 0.9 and = p/4

3. xn = an Sin (n).Un

Inverse transform:

from the residual theorem: (37)

, where C is a closed curve in the complex plane containing all the poles of X(z) and the origin O.

practically unusefull and unused !!

Therefore, how can we do to retrieve xn from X(z) ?

=1

2

() 1

1st method: the elementary decomposition of X(z)

Let us consider X(z) as he following rational fraction:

Idea: let us decompose X(z) in simple elements in order to use their well-known ZT; i.e.

xn = ..

2d method: the direct division of N(z) by D(z)

example:

=()

()=

0 + 11 + +

1 + 11 + +

=1

1 1

1 1

=1+21+2+3

1182+123= 1 + 31 + 122 + 253 +

(36)0= 1, 1 = 3 , 2 = 12, 3 = 25,

II.4.3. Usual Ztransforms table

II.4.4. Main properties of the zT

1. linearity: ax1n + bx2n aX1(z) + bX2(z)

2. time delay: xn-i z-iX(z)

3. product with an exponential time series: anxn X(z/a)

4. final and initial theorems:

0 = lim

= lim1

1 ()

5. time convolution: x1n * x2n X1(z).X2(z)

6. frequantial convolution: (practically unexploitable !!)

7. Parseval theorem

1. 2 1

2

1()2(

)1

=

+

2 =

1

2

(1)1

8. relation zT/dtFT :

with the formal link :

= =+

2 = =+

2

= 2

II.4.5. Discrete transfer function H(z)

a. definition : (38)

main interests:

1. provide the frequantial behaviour of the system:

(39)

2. avoid the convolution difficulty:

(40)

=

()

=

= . ()

H(z) computation

1. From hn , compute H(z) with (38) : for simple cases only !

2. Applied the delay theorem xn-i z-iX(z) to the recusive equation

example : give H(z) of the following system

II.4.6. Poles/Zeros diagram (description)

In a general way: (41)

, the recursive equation of a LIDS and therefore its distcrete tranfer function H(z) :

(42)

= =0

=1

=0 + 1

1 ++

1 + 11 + +

= 0 =1

( )

=1 ( )

The knowledge of the nA poles pj and the nB zeros zi + one constant defines the system !

1. Algebraic stability criterion

a LIDS is stable ifff all its poles pi are with modulus less than 1, i.e.

(43) , < 1

2. Minimum phase system (stable inverse system)

a LIDS is a minimum phase system, or is inverse stable, ifff all its zeros zji are "stable",

i.e.with modulus less than 1.

(44), < 1

3. Causality or feasability of a LIDS

In practical applications, systems are, of course, real and feasable, so nB nA

Important remark

There are strong lengths between the place of the poles and teh zeros of a model ,

and tits dynamical behaviour. We have seen the satbility (inside the unit circle), but also

The rapidity, the damping, as it can be easily shown on a 1st and 2d order models.

Main properties

2d order behaviour:

1st order behaviour:

III.4. The q backshift operator: application to SIDL modelling

II.4.1. Definition :

Let us introduce the time operator corresponding to a delay of one sampling period Te :

n, un-1 = q-1 un , for all numerical signal (time series) {un}n (45)

remark : taking zT of (45) on using the delay theorem of zT, we have :

zT{un} = z-1 U(z)

obviously the frequancy delay operator z-1 correspond to the time delay operator q-1

interest: use the practical notation q-1 while staying in time domain !

II.4.2. Important consequences :

a. Recursive equation (algorithm): see (45)

b. (47) = =0

=1

1

= =0

=1

We can then derive a discrete time transfer function:

(48)

Interest: we can know simply write in time domain:

(49)

1 =0 + 1

1 ++

1 + 11 + +

= (1)

b. Regression form:

We are using parameter models: recursive equation , transfer fucntions

Let us consider (47) introducing

notations: 1. let us define a parameter vector:

(50)

we have nA + nB +1 model parameters , i.e. dim q .

2. let us introduce the observation vector

(51)

= 1 2 01

= 1 2 11

We can use the regressive form of the model, rewriting equ (47)

(very helpfull for the Estimation Theory (and model Identification !)

(48)

remarks: It is an almost general framawork since:

case 1 . in the basic case , j do not depend on the parameter q. (48) is a pure linear regression.

Example in System Identification: we often employ the concept of true

system (see Ljung 99) to define the model of the system we are looking for:

* the true model with its true parameter vector q0 :

(49)

we use a family of parameter model within which we are looking for the best

estimate , with output signal the estimation of our systme output, i.e.

(50)

where the vector parameter is extimated on the base of N data.

=

= 0

=

See estimation theory and/or system identification theory,

we solve the following optimization problem :

(51)

Hoping

1. that problem has solution ( a set of solutions)

2. q0 belongs to its set (in general, we suppose it !)

3.

case 2 . , j depends on the parameter q , i.e. j j (q) (48) is a pseudolinear regression solving (51) is more complicated !

See further fondamental regressives models in control theory

= min ( )

lim

= 0

III.5. Black-Box models used in Automatic Control

Based on a parametric model structure as follows:

(52)

(53)

where:

un , yn are the input/output signals of the system

en the noise signal (centered white sequence with finite variance)with pdf fe(q)

q is the parameter vector of a given model of the structured the interger part of the time delay of the system

G the transfer function of the deterministic part of the model

H the transfer of the disturbance model (stochastic part)

{gk}k the impulse response of the process model

{hk}k the impulse response of the noise model

remark: noise output disturbance 1,

= 1, +

1,

, = 1

, (, ) = 1 +

1

remark: classically we define the one-step-ahead predictor related to, as:

() = 1 , , + [1

1(, )]

II.5.1. AR model :

= 11 22 . +

Let us introduce the predicted output (estimation)

linear regression

where:

=

A better approach, i.e. a polynomial approach facilitates estiamtion of Balck-Box mdels.

Teh general structure used is :

(53) =1

(1)

(1)

(1) +

(1)

(1)

= 1 2 .

= 1 2 .

(54)

II.5.2. ARX model :

II.5.3. ARMAX model :

= 11 22 . + 11 .+ + (55)

Let us introduce the predicted output (estimation)

linear regression

where:

=

= 1 . 1

= 1 2 . 1

= 11 . + 11 .+ + + 11 + (56)

Let us introduce the predicted output (estimation)

pseudo linear regression

where:

= (

)

= 1 2 . 1 1

II.5.5. Exercices:

II.5.4. Output Error model (OE):

and

= 1 . 1 1 ()

where is the prediction error at time n-i = ()

1. Demonstrate the preceeding expressions for ARMAX and OE models

2. Establish these expressions for the general case

of the following BoxJ enkins model

= 11 . + 11 .+ + + 11 + (57)

= (

)

We have the pseudo linear regression

with

= 1 . 1

= 1 1

III.6. Discretization of a continuous time system

Pb: What is exactly the system seen by the computer we are looking for its G(z-1) ?

Let us define the different functions to model: (under ideal asumptions)

- Sampling procedure "transparent "

- "Hold" treatment HBOZ(s)- Conversion A D "transparent "

- Continuous time process G(s) G(s) - Conversion D A "transparent "

Conclusion (58) 1 = () = 1 1

()

examples

1st order:

=

1 + = 1 /

/

2d order (with an integer):

with:

=

(1 + ) = 1( + 1)

(1 1)(1 1)

= (1 /)

= (1 /) /)

= /