KE-90.5100

Process Monitoring (4cr)

Course information

• The course consists of– Lectures:

• Tue 14 – 16, Ke3• Thu 12 – 14, Ke3

– Exercises: • Fri 10:00 – 12:00, computer class room

– Exam:• Oct 31 2008• Jan 8 2009

Course information

• The course consists of– 5 obligatory homeworks (presented during exercises):

• Assistant: M.Sc. Cheng Hui • submit report by email within 2 weeks (di.zhang@tkk.fi)• All exercises have to be OK before exam

– Assignments: Group work to be presented at the end of the course

• The grade consists of – Assignment (30%)– Exam (70%)

Course material

• Course web pages

• Slides

• Handouts

• Exercises/Homework

• Material from assignment

Course Staff

• Hui Cheng, hui.cheng@hut.fi, reception Thu 10 –11, F302

• Alexey Zakharov

• Fernando Dorado

• Di Zhang

Scope of the course

Tools of the process control engineer

ClassicalControl-PID-Bode, -Nyqvist-…

ModernControl- Multivariable control- MPC- IMC-…

Intelligentmethods- Neural Networks- Fuzzy logic- GA

Multivariate data analysis-PCA-PLS-SOM

KE-90.5100 Process Monitoring

KE-90.2100 Basics of processautomation

KE-90.4510 Control applications in process industries

Modeling andsimulation- First principles modeling- Identification- Simulation of dynamic systems- …

KE-90.3100 Process Modelling and Simulation

Scope of the course

• After the course you will know– How and when to use some statistical process

monitoring methods– The basics of neural networks and fuzzy

systems and how to utilize them in monitoring and control

– The basics of genetic algorithms

Introduction

• The idea of process monitoring• Goals of process monitoring• The process monitoring loop• Process monitoring methods• Data selection • Data pretreatment• Univariate vs. multivariate statistics

The idea

To monitor or monitoring generally means to be aware of the state of a system

The idea

• Multivariate data in industrial processes: impossible for human operator to monitor hundreds of measurements for possible faults

• Costs and safety issues with equipment malfunctions / process disturbances – Shutdowns expensive– Amount of maintenance breaks– New equipment: delivery time– Safe working environment for plant staff

The idea

• Process equipment malfunctions and process disturbances– E.g. contamination of sensors, faults of

analyzers, clogging of filters, degradation of catalyst, changing properties of feed stock, leaks, actuator faults etc…

– How to detect early enough?– How to distinguish between?

The goals

Get indication of– process disturbances and– malfunctions in process equipment

As early as possible to– Increase uniform quality of the products– Improve safety– Minimize maintenance costs

The process monitoring loop

FaultIdentification

FaultDiagnosis

Process Recovery

FaultDetection

Fault Detection = determining whether a fault has occurredFault Identification = identifying the variables most relevant for diagnosing the faultFault Diagnosis = determining which fault occurredProcess Recovery = Removing the effect of the fault

yes

no

Process monitoring methods

Process monitoring

Qualitative Quantitative

Data-BasedModel-Based

Neural Networks

Statistical

+ Includes process knowledge − Needs process models

+ Easier to implement− Don’t include process knowledge

Process monitoring methodsProcess monitoring

Qualitative Quantitative

Data-BasedModel-Based

Residual based observersParity-space basedCausal modelsSigned digraphs

Statistical

PCAPLSRPCA

Neural Networks

SOMMLPRBFN..

PCAPLSRPCA..

Trend analysisRule-based

Scope of the course

introductory

+ GA and some Control

Data selection

• Training data includes:– If goal is to verify that process is in normal state

(monitoring) the process data used for training should represent normal conditions

– If the goal is to identify if the process is in a normal or some specified faulty state the process data used for training should include all the possible faulty states as well as the normal state

• Testing data– Completely independent from training data!

Data pretreatment

• Main procedures in pretreatment are:– Removing variables

• The data set may contain variables that have no relevant information for monitoring

– Removing outliers• Outliers are isolated measurement values that are erroneous

and will cause biased parameter estimates for the method used. Methods for removing outliers include visual inspection and statistical tests.

– Autoscaling• Process data often needs to be scaled to avoid particular

variables dominating the process monitoring method (autoscaling = subtract mean and divide by standard deviation)

Univariate vs. multivariate statistics

• The most simple type of monitoring is based on univariate statistics

• Individual thresholds are determined for each variable (Shewart charts)

upper control limit

lower control limit

target

out-of-control

in-control

Univariate vs. multivariate statistics

• Tight thresholds result in a high false alarm rate but low missed alarm rate

• Limits too spread apart result in low false alarm rates but high missed alarm rates

• A trade off between false alarms and missed alarms

-5 0 5 100

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

threshold

normal faulty

false alarmsmissed alarms

Univariate vs. multivariate statistics

• Univariate methods determine threshold for each variable individually without considering other variables

• The fact that there are correlations between the variables is ignored

• The multivariate T2 statistic takes into account these correlations

• The T2 statistic is based on an eigenvector decompostion of the covariance matrix

Univariate vs. multivariate statistics

• Comparison of univariate statistics and T2

statistic

x1

x2

univariate statistical confidence region

T2 statistical confidenceregion

More on T2 later in the PCA part

Abnormal datacan be classifiedas ok!

Principal Component Analysis

Process monitoring

Qualitative Quantitative

Data-BasedModel-Based

Neural Networks

Statistical

PCA

Principal component analysis

• Linear method

• Greatly reduces the number of variables to be monitored – data compression

• Based on eigenvalue and eigenvector decomposition of the covariance matrix

• Simple indexes for monitoring

Principal component analysis• The idea of PCA is to form a minimum number of

new variables to describe the variation of the data by using linear combinations of the original variables

x2

PCA model

PC1: Direction of largest variationPC2: Direction of 2. largest variation

x1

PC1

PC2

PCA- principal components

• The new axes = principal components are selected according to the direction of highest variation in the original data set

• The new axes are orthogonal

x2

x1

PC2

PC1

PCA

PCA- principal components

x2

x1

pc2

pc1

• Principal components will rotate the data set so that the different groups might be separable

No separation offaulty data (red)with original axis

Separation of faulty dataon pc2-axis

PCA- scores

• The PCA scores are the values of the original data points on the principal component axes

PC1

PC2

Sample 1

PCA model calculation

• The direction of the principal components = eigenvectors

• The variation of the data along a eigenvector is given by the corresponding eigenvalue

x2

PCA model

x1

PC1=e1=w11x1+w12x

2

Principal component analysis

• Principal Component Analysis is based on a eigenvalue/eigenvector decomposition of the covariance matrix of the data set (X) with n observations (rows) and m variables (columns).

XX1-N

1C T

Covariancematrix C

The covariance matrix can be decomposed:

X is centered to have zero mean and scaled so that each variable has the same variance (especially if the variables have different units

M

1

M1

λ00

00

00λ

Λ

eeV

ΛC

TVV

Principal component analysis

• The decomposition can be done by solving

• In Matlab the eigenvalues and eigenvectors are obtained by:[eig_vec,eig_val] = eig(C)

• Each eigenvector is a column in eig_vec and the eigenvalues are on the diagonal of eig_val

• A score-matrix T can be calculated: T = XVk

• Vk is a transformation matrix containing the eigenvectors corresponding to the k largest eigenvalues. The user chooses k (more later)

• T is a ”compressed” version of X

0λICdet iii eλCe and

Principal component analysis

• The compressed data can be decompressed:

• There is a residual matrix E between the original data and the decompressed data

• Combining the equations and rewriting the TVkT term:

• pi are the principal components

• The scores ti are the distances along the principal component pi

XXE ˆ

TkTVX ˆ

EptETVXk

1i

Tii

Tk

Principal component analysis

• Matlab demo: Compressing and decompressing data via eigenvalue decomposition

Principal Component Analysis

• Example: Calculate (by hand) the eigenvalues and eigenvectors for:

Specify that

also verify

42

27C

TVVΛC

M

1

M1

λ00

00

00λ

Λ

eeV

Recall that 0λICdet

iii eλCe

1ei

Monitoring indexes

• SPE – variation outside the model, distance off the model plane

• Hotelling T2 – variation inside the model, distance from the origin along the model plane

PCA modelplane

Points with SPE violation

PC2

PC1

SPE

Points withT2 violation

Monitoring indexes

• T2: Measures systematic variations of the process. For individual observation:

• SPE: Measures the random variations of the process

tΛtxVΛVx)(xT -1K

Tnew

Tk

-1Kk

Tnewnew

2 ~~

newTkk

T x)VV(Ir whererrSPE ~

SPE

Different view

Monitoring indexes

• The process can be also monitored by tracking the score values for each principal component

PCA - calculate the model

1. Zero-mean the original data set.2. Compute the covariance matrix C3. Compute the eigenvalues and eigenvectors. Modify the

matrices so, that the eigenvalues are in decreasing order (remember to do the same operations to the eigenvectors)

4. Choose how many principal components to use. (Plot eigenvalues, captured variance)

5. Form the transformation matrix Vk (principal components) and eigenvalue matrix Λk.

6. Compute confidence limits for the scores of every PC7. Compute the Hotelling T2 & SPE limits

PCA - calculate the model1. Scale the original data set X (zero-mean, unit variance if necessary).

2. Calculate the covariance matrix

XX1-N

1C T

PCA - calculate the model

3. Compute the eigenvalues and eigenvectors. The eigenvalues are computed according to

The eigenvectors can be solved from the equation:

Remember to keep the eigenvectors in the same order as the eigenvalues

iii eλCe

0IλCdet i

m21 eeeV

PCA - calculate the model

4. Choose how many principal components to use. (Plot eigenvalues, captured variance)

100%λ

λ)Cvariance(P Captured m

1jk

ii

λ

var capt. %

tot. var capt. %

1 3.18 28.91 28.91

2 2.34 21.31 50.22

3 1.74 15.81 66.03

4 1.08 9.78 75.80

5 0.91 8.24 84.04

6 0.79 7.15 91.20

7 0.49 4.48 95.67

8 0.22 2.04 97.71

9 0.14 1.30 99.01

10 0.11 0.97 99.98

11 0.00 0.02 100.00

PCA - calculate the model

With 7 PCs 96% of the variance is captured

0 2 4 6 8 10 120

0.5

1

1.5

2

2.5

3

number of PC:s

96% of variance explained

PCA - calculate the model

5. Form the transformation matrix Vk (eigenvectors) and eigenvalue matrix ΛK .

K

2

1

K

λ00

0λ0

00λ

Λ

k21k eeeV

PCA - calculate the model

6. Compute confidence limits for the scores of every PC

α/2)1,t(Nλ)Conf(PC ii

sqrt(lamda)*tinv(alfa+(1-alfa)/2,N-1)In Matlab:

α

α = confidence level

PCA - calculate the model

7. Compute Hotelling T2 limit & SPE limit

αK,NK,FKN

1)K(NT2

lim

where the F(K,N-K,) corresponds to the probability point on the F-distribution with (K,N-K)

degrees of freedom and confidence level . N = # of data samples, K = # of PCs

0h1

21

002

1

20α1α Θ

1hhΘ1

Θ

2ΘhcΘSPE

22

310

m

1kj

iji 3Θ

Θ2Θ1hλΘ

where m= number of original variables, k=number of principal components in the model, cα= upper limit from normal distribution with conf. level α

“Unused” eigenvalues

PCA – the new data

1. Scale the new data set with training data scaling values .

2. Compute PCA transformation (i.e. scores for all the chosen principal components) using Vk.

3. Compare the scores to confidence limits. If inside the limits, then OK.

4. Compute the Hotelling T2 & SPE values for the new data set

5. Compare the Hotelling T2 & SPE values to the limits. If under the limit, then OK

PCA – the new data1. Scale the new data set with training data

scaling values

2. Compute PCA transformation (i.e. scores for all the chosen principal components) using Vk.

newTk xVt ~

train

trainnewnew σ

xxx

~

PCA – the new data

3. Compare the scores to confidence limits. If inside the limits, then OK4. Compute the Hotelling T2 & SPE values for the new data set

5. Compare the Hotelling T2 & SPE values to the limits. If under the limit, then OK

tΛtxVΛVx)(xT -1K

Tnew

Tk

-1Kk

Tnewnew

2 ~~

newTkk

T x)VV(Ir whererrSPE ~

Fault identification

• Once a fault has been detected, the next step is determine the cause of the out-of-control status

• One way to handle this is by using so called contribution plots

Contribution plots

• In response to T2 violations we can obtain contribution plots according to:

1. For observation xi, find the r cases when the normalized scores ti2/λi >

T2/a

2. Calculate the contribution of each variable xj to the out-of-control scores ti

3. When conti,j is negative set it equal to zero

4. Calculate the total contribution of the jth process variable

5. Plot CONTj for all process variables in a single plot

V matrix the of element j)(i, the is v xvλ

tcont th

ji,jji,i

iji,

~

r

1iji,j contCONT

Contribution plots

1 2 3 4 50

1

2

3

4

5

6

7

8Contribution plot

process variable

Contribution plots

• Can also be made from the SPE index

1 2 3 4 5 6 7 8 9 10 110

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

2iriSP

PCA derivation

• We want a linear combination of the elements of x that has maximal variance

• Next we look for a linear function eT2x that

is uncorrelated with eT1x but has maximum

variance and so on• We can write• We want to have unit length for the scaling

vector e (constraint)

nn1,11,1T xexexe1

1TT Ceexevar11

PCA derivation

• The maximization can be done using Lagrange multipliers

• We have

• Differentiation gives

• This is an eigenvalue problem. It is the largest eigenvalue that corresponds to the solution since

1eeλCeemax 1T11

T

1

Unit lenght

0λeCe 11

λeλeλeeCee 1T

1T

1T

111

Maximizing the variance = maximizing λ

PCA derivation

• For the second component we can write

• Differentiation gives

• Multiplication with eT1 gives

1T22

T22

T2 eφe1eeλCeemax

0φeλeCe 122

0φ

0eφeeλeCee 1T12

T12

T1

0λeCe 22

(A)

introduce in (A)

Again an eigenvalue problem

uncorrelated

PCA derivation

• Since e2 must be different from e1 and hence λ ≠ λ1

• We still want to maximize variation λ is the second largest eigenvalue

• A similar analysis can be done for the third, fourth etc. PCs.

0ee 1T2