eth chair of structural mechanics structural identiﬁcation ... · fundamentals of probability...

ETH Chair of Structural Mechanics

Structural Identification & Health Monitoring

Probability, random variables & stochastic processes

Dr. V.K. Dertimanis & Prof. Dr. E.N. Chatzi

Outline

Fundamentals of probabilityRandom variablesStochastic processesStationary time–seriesFurther Reading

ETH Chair of Structural Mechanics 25.03.2020 2

Fundamentals of probabilityDefinitions

ζ1 ζ2 ζ3ζ4

Ωζ5ζ6

ζ7ζ8

ζ9 ζ10

Re

x2

x4x3

x1

xx

xx

ζ1 ζ2 ζ3ζ4

Ω

A=ζ5,ζ6,ζ7

ζ5ζ6

ζ7ζ8

ζ9 ζ10

• Repeated experiment that consists of a number of trials

• The result (e.g. outcome) can not be determined a priori



ζ1 ζ2 ζ3ζ4

Ωζ5ζ6

ζ7ζ8

ζ9 ζ10

Re

x2

x4x3

x1

xx

xx

ζ1 ζ2 ζ3ζ4

Ω

A=ζ5,ζ6,ζ7

ζ5ζ6

ζ7ζ8

ζ9 ζ10

• Each outcome ζi is defined as the elementary event

• A set A of outcomes is defined as an event



ζ1 ζ2 ζ3ζ4

Ωζ5ζ6

ζ7ζ8

ζ9 ζ10

Re

x2

x4x3

x1

xx

xx

ζ1 ζ2 ζ3ζ4

Ω

A=ζ5,ζ6,ζ7

ζ5ζ6

ζ7ζ8

ζ9 ζ10

• The set of all elementary events defines the sample space Ω• The sample space of an experiment may be finite, or infinite



Axiomatic definition: the probability P of an event is defined as thenumber that fulfills the axioms

1. P(A) ≥ 0,

2. P(Ω) = 1, and

3. P(A ∪ B) = P(A) + P(B), if A ∩ B = ∅,



Classical definition: the probability P of an event is defined as the ratio ofthe outcomes favorable to an event (NA) to the total number of outcomes(N)

P(A) =NA

N



Conditional probability

P(A|B) =P(A ∩ B)

P(B)

Mutually independent events: P(A ∩ B) = P(A)P(B)


Fundamentals of probabilityExample 1: fair die

• Finite and discrete (e.g. countable) sample space

• Six elementary events (the number of dots on the faces)

• Possible event: A = 2, 4, 6


Fundamentals of probabilityExample 1: fair die

• Probability of each elementary event is P(ζi ) = 1/6

• Probability of the event A: is P(A) = 1/2

• P(ζ1|A) = P(ζ1 ∩ A)/P(A) = P(∅)/P(A) = 0


Fundamentals of probabilityExample 2: daily temperature

||Università degli Studi di Bergamo 28.02.2018 10

uncertainties

The System Identification ProblemChallenges

• Infinite and continuous (e.g. non–countable) sample space

• Elementary events: temperature values along the real axis

• Events are defined as intervals T1 ≤ T ≤ T2


Random variablesDefinitions

Random variable: a rule for assigning to every outcome ζ of anexperiment a number

: x(ζ) : S −→ R

ζ1

Rζ2

ζ3

ζ4S

xx

x

x

x1

x2

x3

x4



Random variable: a rule for assigning to every outcome ζ of anexperiment a number: x(ζ) : S −→ R

ζ1

Rζ2

ζ3

ζ4S

xx

x

x

x1

x2

x3

x4


Random variablesExample 3: daily temperature

||Università degli Studi di Bergamo 28.02.2018 10

uncertainties

The System Identification ProblemChallenges

• Define x(T ) = T

• Then x(T ) : R −→ R is a random variable

• A measurement is both an elementary event and a random variable



ζ1

Rζ2

ζ3

ζ4S

xx

x

x

x1

x2

x3

x4

The set x(ζ) = x1 represents a ran-dom event that contains all elementaryevents ζ for which x(ζ) = x1

The set x1 ≤ x(ζ) ≤ x2 is the randomevent that contains all elementary eventsζ for which x1 ≤ x(ζ) ≤ x2



ζ1

Rζ2

ζ3

ζ4S

xx

x

x

x1

x2

x3

x4Complete characterization of a ran-dom variable requires the calculationof probabilities associated to randomevents



ζ1

Rζ2

ζ3

ζ4S

xx

x

x

x1

x2

x3

x4

Probability distribution function:

F (x) = P(x(ζ) ≤ x)

Properties

F (x1) ≤ F (x2), if x1 ≤ x2

F (−∞) = 0

F (+∞) = 1



ζ1

Rζ2

ζ3

ζ4S

xx

x

x

x1

x2

x3

x4

Probability density function:

f (x) =dF (x)

dx

Properties

P(x1 ≤ x(ζ) ≤ x2) = F (x2)− F (x1)

=∫ x2

x1

f (x)dx


Random variablesExpected values

ζ1

Rζ2

ζ3

ζ4S

xx

x

x

x1

x2

x3

x4

mean value:

µx = Ex(ζ) ≡∫ ∞−∞

xf (x)dx

variance:

σ2x = E[x(ζ)−µx ]2 ≡

∫ ∞−∞

[x(ζ)−µx ]2f (x)dx


Random variablesExpected values

ζ1

Rζ2

ζ3

ζ4S

xx

x

x

x1

x2

x3

x4

mean square value:

ψ2x = Ex2(ζ) ≡

∫ ∞−∞

x2f (x)dx

basic equation:

ψ2x = σ2

x + µ2x


Random variablesExample 4: Gaussian distributionProbability and Stochastic Processes 209

!z1

(a) The shaded area equals P (z < −z1)

z1

(b) The shaded area equals P (z < z1)

FIGURE D.2Calculation of probability for the random event z < −z1 for z ∼ N(0, 1).

Assume that the random variable x(T ) = T defined in Example D.2 follows aGaussian distribution N(µ, σ2). Substituting f(x) in Eqs. D.8a–D.8b with the ex-pression given in Tab. D.1, it follows that µx = µ and that σ2

x = σ2. Noticethat, since x(T ) ∼ N(µx, σ2

x), its behaviour is completely determined by these twoparameters.

In calculating probabilities associated with random events of a Gaussian ran-dom variable, it is customary to normalize it as z(T ) = [x(T ) − µx]/σx, sothat z(T ) ∼ N(0, 1), for which standard tables with calculated probability dis-tribution function values exist. Indicatively, if µx = 20o C and σx = 10o C,the probability of measuring a temperature below 10o C is P (x(T ) < 10) =P (z(T ) < −0.83) = 1 − P (z(T ) < 0.83) = 0.2033, or 20.33%, where the prop-erty P (z(T ) < −z1) = 1− P (z(T ) < z1) has been applied (see Figure D.2).

Consider now the case of two random variables, x(ζ) : Ωx → R and y(ζ) :Ωy → R, as shown in Figs. D.3(a)–D.3(b), with individual probability distribu-tion functions notated as Fx(x) and Fy(y), respectively. Of interest is the studyof the joint behaviour of the two random variables, which implies the calcula-tion of probabilities for joint random events (Figs. D.3(c)–D.3(d)) of the formx1 ≤ x(ζ) ≤ x2 ∩ y1 ≤ y(ζ) ≤ y2 ≡ x1 ≤ x(ζ) ≤ x2, y1 ≤ y(ζ) ≤ y2.Following the single random variable case, the joint probability distributionfunction and the joint PDF are defined as

F (x, y) = P(x(ζ) ≤ x, y(ζ) ≤ y

)(D.9)

with F (−∞, y) = F (x,−∞) = 0 and F (∞,∞) = 1, and

f(x, y) =∂2F (x, y)

∂x∂y(D.10)

respectively, with f(x, y) > 0 and∫∞−∞

∫∞−∞ f(x, y)dxdy = 1. Similarly, the

calculation of probability for the random event x(ζ) ≤ x, y(ζ) ≤ y is givenby

P(x(ζ) ≤ x, y(ζ) ≤ y

)= F (x, y) =

∫ y

−∞

∫ x

−∞f(ξ1, ξ2)dξ1dxi2 (D.11)

f (x) =1

σx√

2πe− (x−µx )2

σ2x


Random variablesTwo random variables

210 Structural System Identification and Health Monitoring: A Primer

ζ1 ζ2 ζ3ζ4

Ωxζ5ζ6

ζ7ζ8

ζ9 ζ10

Re

x2

x4x3

x1

xx

xx

ζ1ζ2

ζ3ζ4

Ωy

ζ5

Rey3

y2

y1

y

yy

x

y

x1 x2 x3 x4

y1

y2

y3

x

y

x1x2 x3 x4

y1

y2

y3

(a) x(ζ) : Ωx → R

ζ1 ζ2 ζ3ζ4

Ωxζ5ζ6

ζ7ζ8

ζ9 ζ10

Re

x2

x4x3

x1

xx

xx

ζ1ζ2

ζ3ζ4

Ωy

ζ5

Rey3

y2

y1

y

yy

x

y

x1 x2 x3 x4

y1

y2

y3

x

y

x1x2 x3 x4

y1

y2

y3

(b) y(ζ) : Ωy → R

ζ1 ζ2 ζ3ζ4

Ωxζ5ζ6

ζ7ζ8

ζ9 ζ10

Re

x2

x4x3

x1

xx

xx

ζ1ζ2

ζ3ζ4

Ωy

ζ5

Rey3

y2

y1

y

yy

x

y

x1 x2 x3 x4

y1

y2

y3

x

y

x1x2 x3 x4

y1

y2

y3

(c) The x− y plane of random vari-ables. Each axis corresponds to therange of the random variable

ζ1 ζ2 ζ3ζ4

Ωxζ5ζ6

ζ7ζ8

ζ9 ζ10

Re

x2

x4x3

x1

xx

xx

ζ1ζ2

ζ3ζ4

Ωy

ζ5

Rey3

y2

y1

y

yy

x

y

x1 x2 x3 x4

y1

y2

y3

x

y

x1x2 x3 x4

y1

y2

y3

(d) The shaded quadrat (extendingto infinity) corresponds to the ran-dom event x < x1, y < y1

FIGURE D.3Two random variables and associated joint random events.

The individual distributions (e.g. Fx(x) and Fy(y)) and PDFs (e.g. fx(x)and fy(y)) of each random variable are referred to as marginal distributionsand PDFs, respectively. The former are recovered by the joint probabilitydistribution function as Fx(x) = F (x,∞) and Fy(y) = F (∞, y), while thelatter by the joint PDF as

fx(x) =

∫ ∞

−∞f(x, y)dy, fy(y) =

∫ ∞

−∞f(x, y)dx (D.12)

An important application of the marginal statistics pertains to the case wherethe random variables are (mutually) independent, that is, when

P(x(ζ) ≤ x, y(ζ) ≤ y

)= P

(x(ζ) ≤ x

)P(y(ζ) ≤ y

)(D.13)

implying that

F (x, y) = Fx(x)Fy(y) (D.14)

f(x, y) = fx(x)fy(y) (D.15)



210 Structural System Identification and Health Monitoring: A Primer

ζ1 ζ2 ζ3ζ4

Ωxζ5ζ6

ζ7ζ8

ζ9 ζ10

Re

x2

x4x3

x1

xx

xx

ζ1ζ2

ζ3ζ4

Ωy

ζ5

Rey3

y2

y1

y

yy

x

y

x1 x2 x3 x4

y1

y2

y3

x

y

x1x2 x3 x4

y1

y2

y3

(a) x(ζ) : Ωx → R

ζ1 ζ2 ζ3ζ4

Ωxζ5ζ6

ζ7ζ8

ζ9 ζ10

Re

x2

x4x3

x1

xx

xx

ζ1ζ2

ζ3ζ4

Ωy

ζ5

Rey3

y2

y1

y

yy

x

y

x1 x2 x3 x4

y1

y2

y3

x

y

x1x2 x3 x4

y1

y2

y3

(b) y(ζ) : Ωy → R

ζ1 ζ2 ζ3ζ4

Ωxζ5ζ6

ζ7ζ8

ζ9 ζ10

Re

x2

x4x3

x1

xx

xx

ζ1ζ2

ζ3ζ4

Ωy

ζ5

Rey3

y2

y1

y

yy

x

y

x1 x2 x3 x4

y1

y2

y3

x

y

x1x2 x3 x4

y1

y2

y3

(c) The x− y plane of random vari-ables. Each axis corresponds to therange of the random variable

ζ1 ζ2 ζ3ζ4

Ωxζ5ζ6

ζ7ζ8

ζ9 ζ10

Re

x2

x4x3

x1

xx

xx

ζ1ζ2

ζ3ζ4

Ωy

ζ5

Rey3

y2

y1

y

yy

x

y

x1 x2 x3 x4

y1

y2

y3

x

y

x1x2 x3 x4

y1

y2

y3

(d) The shaded quadrat (extendingto infinity) corresponds to the ran-dom event x < x1, y < y1

FIGURE D.3Two random variables and associated joint random events.

The individual distributions (e.g. Fx(x) and Fy(y)) and PDFs (e.g. fx(x)and fy(y)) of each random variable are referred to as marginal distributionsand PDFs, respectively. The former are recovered by the joint probabilitydistribution function as Fx(x) = F (x,∞) and Fy(y) = F (∞, y), while thelatter by the joint PDF as

fx(x) =

∫ ∞

−∞f(x, y)dy, fy(y) =

∫ ∞

−∞f(x, y)dx (D.12)

An important application of the marginal statistics pertains to the case wherethe random variables are (mutually) independent, that is, when

P(x(ζ) ≤ x, y(ζ) ≤ y

)= P

(x(ζ) ≤ x

)P(y(ζ) ≤ y

)(D.13)

implying that

F (x, y) = Fx(x)Fy(y) (D.14)

f(x, y) = fx(x)fy(y) (D.15)



Joint probability distribution function

F (x , y ) = P(x(ζ) ≤ x , y (ζ) ≤ y

)with F (−∞, y ) = F (x ,−∞) = 0 and F (∞,∞) = 1

Joint probability density function

f (x , y ) =∂2F (x , y )∂x∂y



Independence

P(x(ζ) ≤ x , y (ζ) ≤ y

)= P

(x(ζ) ≤ x

)P(y (ζ) ≤ y

)

F (x , y ) = Fx (x)Fy (y )

f (x , y ) = fx (x)fy (y )



Mean values. Define r(ζ) = [x(ζ) y (ζ)]T . Then:

µr ≡ Er(ζ) =

[Ex(ζ)Ey (ζ)

]=

[µx

µy

]

µx =∫ ∞−∞

∫ ∞−∞

xf (x , y )dxdy

µy =∫ ∞−∞

∫ ∞−∞

yf (x , y )dxdy



Covariance matrix

Crr ≡ E(r(ζ)− µr )(r(ζ)− µr )T )

=

[E(x − µx )2 E(x − µx )(y − µy )

E(y − µy )(x − µx ) E(y − µy )2

]

=

[σ2

x cxy

cxy σ2y

]



Covariance: a measure of the joint variability of x(ζ) and y (ζ)

cxy = cyx =∫ ∞−∞

∫ ∞−∞

[x(ζ)− µx ][y (ζ)− µy ]f (x , y )dxdy


Stochastic ProcessesDefinitions

Stochastic Process: a rule for assigning to every outcome ζ of anexperiment a function

: xk (t) : S −→ F, F .= the set of functions x(t)

ζ1

ζ2

ζ3

ζ4S

x1[t]

x2[t]

x3[t]

x4[t]


Stochastic ProcessesDefinitions

Stochastic Process: a rule for assigning to every outcome ζ of anexperiment a function: xk (t) : S −→ F, F .= the set of functions x(t)

ζ1

ζ2

ζ3

ζ4S

x1[t]

x2[t]

x3[t]

x4[t]


Stochastic ProcessesPreliminaries

A stochastic process xk (t) is thus a family of functions, where eachfunction corresponds to one experimental result ζk .

Equivalently, a stochastic process is an ensemble of functions,characterized by its probability structure.


Stochastic ProcessesPreliminaries

When both t and k are fixed, the quantity xk (t) is a number.

When t is fixed, the quantity xk (t) is a random variable.

When k is fixed, the quantity xk (t) is a sample function or a time–series.


Stochastic ProcessesExample 5: spring–mass–damper

m

k c

u(t)

x(t)

Steady–state response:

xk (t) = Xk cos(2πft + θk )

Assume that Xk and θk are random vari-ables. Then xk (t) is a sample function(e.g. a time–series)


Stochastic ProcessesComplete description

Requires knowledge of the joint PDFs of all (possibly infinite) randomvariables.

Times t1, t2 : f(

xk (t1), xk (t2))

= . . .

Times t1, t2, t3 : f(

xk (t1), xk (t2), xk (t3))

= . . ....

...


Stochastic ProcessesPartial description

Requires knowledge of the mean and covariance functions

µx (t) = Exk (t) cxx (t1, t2) = E

(xk (t1)− µx (t1)

)(xk (t2)− µx (t2)

)

µy (t) = Exy (t) cyy (t1, t2) = E

(yk (t1)− µy (t1)

)(xy (t2)− µy (t2)

)

cxy (t1, t2) = E

(xk (t1)− µx (t1)

)(yk (t2)− µy (t2)

)


Stochastic ProcessesStrong stationarity

Joint PDFs of all (possibly infinite) random variables are time invariant.

Times t1, t2 : f(

xk (t1))

= f(

xk (t2))

Times t1, t2, : f(

xk (t1), xk (t2))

= f(

xk (t1 + δt), xk (t2 + δt))

......


Stochastic ProcessesWeak stationarity

Mean and covariance functions are time invariant

µx (t) = µx cxx (tk , tm) = cxx (tk + δt , tm + δt) = cxx (tk − tm) = cxx (h)µy (t) = µy cyy (tk , tm) = cyy (tk + δt , tm + δt) = cyy (tk − tm) = cyy (h)

cxy (tk , tm) = cxy (tk + δt , tm + δt) = cxy (tk − tm) = cxy (h)


Stochastic ProcessesErgodicity

Strong ergodicity: all ensemble averaged statistical properties can becalculated from corresponding time averages of a single sample function(e.g. time series).

Weak ergodicity: ensemble mean and covariance functions can becalculated from corresponding time averages of a single sample function(e.g. time series).

µx (t) = µx cxx (h, k ) = cxx (h)µy (t) = µy cyy (h, k ) = cyy (h)

cxy (h, k ) = cxy (h)


Stochastic ProcessesGaussian processes

A stochastic process is Gaussian if the associated random variables forall times follow a multidimensional normal distribution.

If a stochastic process is Gaussian and weakly stationary, then is alsostrongly stationary.

A Gaussian stochastic process is ergodic if it is weakly stationary and

1T

∫ T

−T|cxx (h)|dh −→ 0, as T −→∞



m

k c

u(t)

x(t)

Steady–state response:

xk (t) = Xk cos(2πft + θk ), t = 0, 1, . . .

X : EX = 0 and EX 2 = 1θ : uniform distribution in [−π, π]X ,θ : independent



m

k c

u(t)

x(t)

Mean value:

Exk (t) = E

Xk cos(2πft + θk )

= EXkE

cos(2πft + θk )

= 0



m

k c

u(t)

x(t)

Covariance:

γxx [h] = Exk (t + h)xk (t)

= E

X 2k cos

(ω(t + h) + θk

)cos

(ωt + θk

)=

12

cos(ωh)


Further Reading

1. Papoulis, A. (1991), Probability, Random Variables & Stochastic Processes, 3thEd., McGraw–Hill, New York, USA.

2. Bendat, J.S. and Piersol, A.G. (2010), Random Data: Analysis and MeasurementProcedures, 4th Ed., John Wiley & Sons Ltd., Chichester, UK.


eth chair of structural mechanics structural identiﬁcation ... · fundamentals of probability...

Documents