systems id tifi tiidentificationprofsite.um.ac.ir/~karimpor/iden/sysiden_lec2_2012.pdf ·...

SYSTEMSSYSTEMSId tifi tiId tifi tiIdentificationIdentification

Ali KarimpourpAssistant Professor

Ferdowsi University of MashhadFerdowsi University of Mashhad

Reference: “System Identification Theory For The User” Lennart Ljung(1999)“Nonlinear System Identification: from Classical Approaches to

Neural Networks and Fuzzy Models” Oliver Nelles (2001)

lecture 2

Lecture 2

IntroductionIntroductionTopics to be covered include:� Impulse responses and transfer functions.� p p

� Frequency domain expression.

� Stochastic Process.

� Signal spectra� Signal spectra

� Disturbances

Ali Karimpour Sep 2012

22� Ergodicity

lecture 2

Impulse responses

It is well known that a linear, time-invariant, causal system can be described as:

τττ∫∞

−= )()()( dtugtyτττ∫ −=t

dutgty )()()(

Sampling

ττττ∫ =0

)()()( dtugty

τττ∫∞

)()()( dkTugkTy

τ∫ −∞=gy )()()(

ττττ∫ =

−=0

)()()( dkTugkTy

Most often, the input signal u(t) is kept constant between the sampling instants:

TktkTutu k )1()( +≤≤=So

∞

lT

l

lT

TldkTugdkTugkTy

∞∞

∞

=−=

∞

=

∫

∑∫∫⎤⎡

−=−=1

)1(0)()()()()( ττττττ

ττ


33lk

lTlk

l

lT

Tlulgudg −

=−

=−= ∑∑ ∫ =⎥⎦

⎤⎢⎣⎡=

11)1(

)()( τττ

lecture 2

Impulse responses

∞

∑lT

∫lkl

T ulgkTy −=∑=

1)()( )()(Where

)1(lgdg T

lT

Tl=∫ −=

τττ

For ease of notation assume that T is one time unit and use t to enumerate the sampling instantsp g

.....3,2,1,0)()()(1

=−=∑∞

=

tktukgtyk


4

lecture 2

Transfer functions

Define forward and backward shift operator q and q-1 as

)1()( += tutqu )1()(1 −=− tutuqNow we can write output as:

( )∑∑∞

−∞

=−= )()()()()( k tuqkgktukgty )()()()( tuqGtuqkg k =⎥⎦

⎤⎢⎣

⎡= ∑

∞−( )∑∑

==

==11

)()()()()(kk

tuqkgktukgty )()()()(1

qqgk

⎥⎦

⎢⎣∑=

G(q) is the transfer operator or transfer function

∑∞

)()( kkG ∑=

−=1

)()(k

kqkgqGSimilarly for disturbance we have

)()()( tuqHtv = )()()( q

So the basic description for a linear system with additive disturbance is:


5)()()()()( teqHtuqGty +=

lecture 2

Transfer functions

Some terminology

G(q) is the transfer operator or transfer functionG(q) is the transfer operator or transfer function

∑∑∞

=

−∞

=

− ==11

)()(or)()(k

k

k

k zkgzGqkgqG

We shall say that the transfer function G(q) is stable if

∞<∑∞

)(kg ∞<∑=1

)(k

kg

This means that G(z) is analytic on and outside the unit circle.

We shall say the filter H(q) is monic if h(0)=1:

∞


6∑∞

=

−=0

)()(k

kqkhqH

lecture 2

Frequency-domain expressions

Letttu ωcos)( =

It will convenient to writetjetu ωRe)( =

Now we can write output as:

∑∑∞

−∞

− == )()( )(ReRe)()( ktikti ekgekgty ωω ∑∑== 11 kk

( ) { })(Re)(.Re ωωωω itikiti eGeekge =⎭⎬⎫

⎩⎨⎧

= ∑∞

−

1k ⎭⎩ =

So we have

( )Ali Karimpour Sep 2012

7( ))(argcos)()( ωω ω ii eGteGty +=

lecture 2

Frequency-domain expressions

ttu ωcos)( =tjetu ωRe)( =

( ))(argcos)()( ωω ω ii eGteGty +=etu Re)(

⎧ 0 ⎧ 0R tjω

Now suppose

⎩⎨⎧

<≥

=000cos

)(ttt

tuω

⎩⎨⎧

<≥

=000Re

)(tte

tutjω

⎫⎧ ( )⎭⎬⎫

⎩⎨⎧

= ∑=

−t

k

kiti ekge1

)(.Re ωω∑=

−=t

k

ktiekgty1

)(Re)()( ω

⎫⎧⎫⎧⎫⎧ t

( ) ( ) ( )⎭⎬⎫

⎩⎨⎧

−⎭⎬⎫

⎩⎨⎧

+⎭⎬⎫

⎩⎨⎧

= ∑∑∑∞

=

−∞

=

−

=

−

tk

kiti

tk

kitit

k

kiti ekgeekgeekge ωωωωωω )(.Re)(.Re)(.Re1

For stable system


8( ) ( )

⎭⎬⎫

⎩⎨⎧

−+= ∑∞

=

−

tk

kitiii ekgeeGteGty ωωωω ω )(.Re)(argcos)()(For stable systemAnd large t

lecture 2

Periodograms of signals over finite intervals

tdetfg ti∫+∞ −= ωω )(

21)(

Fourier transform (FT)

)(tf )(ωg∫ ∞−π2

∫+∞

= ωω ω degtf ti)()( ∫ ∞−ωω degtf )()(

Discrete Fourier transform (DFT)

∑N

i1

)()4()3()2()1(

Nuuuuu

)2()8(

)6()4()2(

NUUN

UN

UN

U NNN

ππ

πππ∑=

−=t

tiN etu

NU

1

)(1)( ωω

)(......)4( Nuu )(......)(N

UN

U NN

Nu1 ∑=

=N

k

NtkiN eNkU

Ntu

1

/2)/2(1)( ππ )(ωNU


9

=kN 1

Exercise1: Show that u(t) can be derived by putting UN(ω) in u(t).

lecture 2


)()2( ωπω NN UU =+Some property of UN(ω)__________

)()( ωω NN UU =−

The function UN(ω) is therefore uniquely defined by its values over the intervalThe function UN(ω) is therefore uniquely defined by its values over the interval [ 0, 2π ]. It is, however, customary to consider UN(ω) for the interval [ - π, π ]. So u(t)can be defined as

2/1 N

∑+−=

=2/

12/

/2)/2(1)(N

Nk

NtkiN eNkU

Ntu ππ

The number UN(ω) tell us the weight that the frequency ω carries in theThe number UN(ω) tell us the weight that the frequency ω carries in the decomposition. So

2)(ωNU

Is known as the periodogram of the signal u(t), t= 1 , 2 , 3 , …..

Parseval’s relationship:NN


10∑∑==

=N

t

N

kN tuNkU

1

2

1

2 )()/2( π

lecture 2


Example: Periodogram of a sinusoid

tAtu 0cos)( ω= tAtu 0cos)( ωso periodic is )( Suppose tu

)(ofmultipleaiswhere321Let sNNNNNt

1integer somefor /2 000 >= NNπω

∑ −=N

tietAU cos1)( ωωω ( )∑ −− +=N

tititi eeeA00 ωωω

)(of multipleais where,...,3,2,1Let 00 sNNNNNt ==

∑=

=t

N etAN

U1

0cos)( ωω ( )∑=

+=t

eeeN 12

( )∑ +−− +=N

titi eeN

A )()( 00

2ωωωω( )∑

=tN 12

⎪⎪⎨

⎧±=±=±=

=s

NNifAN

U

224)( 0

0

2

2

ππωωω


11⎪⎪⎩

⎨≠=

=

skNkif

U N

,20)( 0

πωω

lecture 2



)3()2()1( uuu )6()4()2(N

UN

UN

U NNNπππ

∑ −N

tietuU )(1)( ωω)(......)4( Nuu )2(......)8(

NNU

NU

NNN

NNππ

Nu1

∑=

=t

N etuN

U1

)()(ω

1

The periodogram defines, in a sense, the frequency contents of a signal over afi it ti i t lfinite time interval.


1212

lecture 2

Stochastic Processes

A random variable (RV) is a rule (or function) that assigns a real number to every outcome of a random experiment.

The closing price of Iranian power market observed from Apr. 15 to Sep. 22, 2009.For scalar (RV) ( )∫=≤≤

b

a e dxxfbeaP )( For vector (RV) ( )∫ ∈=∈Bx e dxxfBeP )(

If e may assume a certain value with nonzero probability then feee contains δ function.d i bl d i d d if h

Probability density function (PDF)

Two random variables e1 and e2 are independent, if we have:

)().()( 22112211 xePxePxexeP ====∧=

Definition: The expectation E[e] of a random variable e is:Definition: The expectation E[e] of a random variable e is:

( )∫= NR e dxxxfEe

Definition: The variance Cov[e] of a random variable e is:


13

Definition: The variance, Cov[e], of a random variable, e, is:

( )( ) EemmemeEe T =−−= :Cov

lecture 2


A stochastic process is a rule (or function) that assigns a time function to every outcome of a random experiment.

• Consider the random experiment of tossing a dice at t = 0 and observing the number on the top face.

• The sample space of this experiment consists of the outcomes {1, 2, 3, · · · , 6}.

• For each outcome of the experiment, let us arbitrarily assign a function of time t in the following manner.


14

• The set of functions {x1(t), x2(t), · · , x6(t)} represents a stochastic process.

lecture 2


Mean of a random process X(t) is

I l ( ) i f i f i

Correlation RX(t1, t2) of a random process X(t) is

In general, mX(t) is a function of time.

Note RX(t1, t2) is a function of t1 and t2.X( 1 2) 1 2

Autocovariance CX(t1, t2) of a random process X(t) is defined as the covarianceof X(t1) and X(t2):

In particular when t = t = t we have


15

In particular, when t1 = t2 = t, we have

lecture 2


Example Sinusoid with random amplitude


16

lecture 2


Example Sinusoid with random phase


17

lecture 2


x(t) is stationary if

Example Sinusoid with random phase

Clearly x(t) is a stationary (WSS)Clearly x(t) is a stationary (WSS).

Example Sinusoid with random amplitude

Clearly x(t) is not a stationary


18

Clearly x(t) is not a stationary.

This may be a limiting definition. ?????

lecture 2

Signal Spectra

A Common Framework for Deterministic and Stochastic Signals

y(t) is not a)()()()()( tHtGt )()()( tGtE y(t) is not a stationary process)()()()()( teqHtuqGty += )()()( tuqGtEy =

This may be a limiting definition ?????

To deal with this problem, we introduce the following definition:

This may be a limiting definition. ?????


19Quasi-stationary signals

lecture 2


Quasi-stationary signals: A signal {s(t)} is said to be quasi-stationary if it is subject to

x(t) is stationary if

Q y g g { ( )} q y j

dand

( )If {s(t)} is a deterministic

sequence

Quasi-stationarymeans

{s(t)} is a bounded sequence and

Exist )()(1lim)(1∑=

∞→−=

N

tNs tstsN

R ττ

If { (t)} i t ti It i i t ti i


20

If {s(t)} is a stationary stochastic process

.on dependnot does )()()( tRtstEs s ττ =−

It is quasi stationary since

lecture 2

Signal Spectra

Notation: The notation means that the limit exists.

Quasi-stationary signals: A signal {s(t)} is said to be quasi-stationary if it is subject to

andand

( )

i i h b f i ll i i f i f


21

Sometimes with some abuse of notation, we call it Covariance function of s.

Exercise2: Show that sometime it is exactly covariance function of s.

lecture 2

Signal Spectra

Two signals {s(t)} and {w(t)} are jointly quasi-stationary if:

1- They both are quasi-stationary,

2- the cross-covariance function

exist.

Uncorrelated


22

lecture 2

Signal Spectra


)(......)4()3()2()1(

Nuuuuu

)2(......)8(

)6()4()2(

NNU

NU

NU

NU

NU

NN

NNN

ππ

πππ

∑=

−=N

t

tiN etu

NU

1

)(1)( ωω

Nu1

The periodogram defines, in a sense, the frequency contents of a signal over afinite time interval

23

finite time interval.

But we seek for a definition of a similar concept for signals over the interval [1, ∞).

∞→∑ − NetuUN

ti)(1)( ωω

But this limits fail to exist for many signals of practical interest.

∞→= ∑=

NetuN

Ut

N1

)()(ω


2323So we shall develop a frame work for describing signals and their spectra that isapplicable to deterministic as well as stochastic signals.

lecture 2

Signal Spectra

We define the (power) spectrum of {s(t)} as

Use Fourier transform of covariance function (Spectrum or Spectral density)

We define the (power) spectrum of {s(t)} as

∑∞

−=Φ τωτω iss eR )()(

When following limits exists:

∑−∞=τ

ss )()(

and cross spectrum between {s(t)} and {w(t)} asand cross spectrum between {s(t)} and {w(t)} as

∑∞

−=Φ τωτω ieR )()( ∑−∞=

=Φτ

τω swsw eR )()(

When following limits exists:


24Exercise3: Show that spectrum always is a real function but cross spectrum is ingeneral a complex-valued function.

lecture 2

Signal Spectra

Exercise4 : Spectra of a Periodic Signal: Consider a deterministic, periodicsignal with period M, i.e., s(t)=s(t+M)

Show that

),()()( MFpss ωωω Φ=Φ ),()()( ss ωωω

Where

∑−

−Φ1

)()(M

ip R ωτ ∑∞

−ilMMF ω)(d∑=

=Φ0

)()( is

ps eR

τ

ωττω ∑−∞=

=l

ilMeMF ωω ),( and

A d fi ll h th tAnd finally show that

πωπωδππω 20,)/2()/2(2)(1

<≤−Φ=Φ ∑−M

pss MkMk

M


25

0∑=kM

lecture 2

Signal Spectra

Exercise5: Spectra of a Sinusoid: Consider

),1[interval thetocos)( 0 ∞= tAtu ω

Show that

),[)( 0

Show that

( ) πωωδωωδω 2.)()(4

)( 00

2

++−=ΦA

u ( )4 00u


26

lecture 2

Signal Spectra

Example Stationary Stochastic Processes: Consider v(t) as a stationarystochastic processes

We will assume that e(t) has zero mean and variance λ . It is clear that:∞

)()()()()()()()(),0max(

IkhkhtvtEvtvtvERk

v ∑=

−=−=−=τ

τλτττ

The spectrum is:

∑∞

−∞=

−=Φτ

τωτω ivv eR )()( ∑ ∑

∞

−∞=

−∞

=

−=τ

τω

τ

τλ i

kekhkh

),0max()()( .............=

2)( ωλ ieH=

Where

∑∞

=

−=1

)()(s

isi esheH ωω


27

1s

Exercise6: Show (I)

lecture 2

Signal Spectra

Spectrum of Stationary Stochastic Processes

The stochastic process described by v(t)= H(q)e(t), where {e(t)}p y ( ) (q) ( ), { ( )}is a sequence of independent random variables with zero meanvalues and covariances λ , has the spectrum

2)()( ωλω i

v eH=Φ


28

lecture 2

Signal Spectra

Spectrum of a Mixed Deterministic and Stochastic Signal

deterministicStochastic:

stationary and zero mean

)()()( ωωω vus Φ+Φ=Φ

Exercise7: Proof it.

vus RRR +=


29

e c se7: oo .

lecture 2

Transformation of Spectra by Linear Systems

Theorem: Let{w(t)} be a quasi-stationary with spectrum , and let G(q) bea stable transfer function. Let

)(ωwΦ

Then {s(t)} is also quasi-stationary and

)()()(2

ωω ωieG ΦΦ )()()( ωω ωws eG Φ=Φ

)()()( ωω ωw

isw eG Φ=Φ


30

lecture 2

Disturbances

There are always signals beyond our control that also affect the system. We assume that such effects can be lumped into an additive term v(t) at the output

u(t) y(t)

v(t)

So

)()()()( tvktukgty +−=∑∞

+

)()()()(1

tvktukgtyk

+=∑=

There are many sources and causes for such a disturbance term.

• Measurement noise.

U ll bl i ( i d 100 W/ )


31

• Uncontrollable inputs. ( a person in a room produce 100 W/person)

lecture 2

Disturbances

Characterization of disturbances

• Its value is not known beforehand.

• Making qualified guesses about future values is possible.

• It is natural to employ a probabilistic framework to describe futureIt is natural to employ a probabilistic framework to describe future disturbances.We put ourselves at time t and would like to know disturbance at t+k, k ≥ 1 so we use the following approach.

)()()( ktekhtv −=∑∞

0k∑=

Where e(t) is a white noise.

This description does not allow completely general characteristic of


32

p p y gall possible probabilistic disturbances, but it is versatile enough.

lecture 2

Disturbances

Consider for example, the following PDF for e(t):

Small values of µ are suitable to describe classical disturbance patterns, steps, pulses, sinuses and ramps.


33

A realization of v(t) for propose e(t)

Exercise8: Derive above figure for µ=0.1 and µ=0.9 and a suitable h(k).

lecture 2

Disturbances

On the other hand, the PDF:


Often we only specify the second-order properties of the sequence {e(t)} that is themean and variances.



34

Exercise9: What is a white noise?

Exercise10: Derive above figure for δ=0.1 and δ=0.9 and a suitable h(k).

lecture 2

Disturbances

We will assume that e(t) has zero mean and variance λ . Now we want to knowthe characteristic of v(t) :

Mean:

C iCovariance:

∑ ∑∞

=

∞

=−−=

0 0)()()(

k sskshkh λτδ

∑∞−= )()( khkh τλ )(τR=


35

∑ =0)()(

kkhkh τλ )(τvR

causality) of (Since0for0)( that know We <= rrh

lecture 2

Disturbances

We will assume that e(t) has zero mean and variance λ . Now we want to knowthe characteristic of v(t) :

Mean: Covariance:Covariance:)()()( ττ −= tvtEvRv

Since the mean and covariance are do not depend on t, thei id t b t tiprocess is said to be stationary.


36

lecture 2

Ergodicity

Suppose you are concerned with determining what the most visited parks in acity are.

• One idea is to take a momentary snapshot: to see how many people are this momentin park A, how many are in park B and so on.

• Another idea is to look at one individual (or few of them) and to follow him for a certain period of time, e.g. a year.

The first one may not be representative for a longer period of time, while the secondone may not be representative for all the people.

The idea is that an ensemble is ergodic if the two types of statistics give the same result.Many ensembles, like the human populations, are not ergodic.


37

lecture 2

Ergodicity

Let x(t) is a stochastic process

Most of our computations will depend on a given realization of a quasi


38

stationary process.

Ergodicity will allow us to make statements about repeated experiments.

systems id tifi tiidentificationprofsite.um.ac.ir/~karimpor/iden/sysiden_lec2_2012.pdf ·...

Documents