Int. J. Appl. Math. Comput. Sci., 2006, Vol. 16, No. 1, 85–99

ADVANCES IN MODEL–BASED FAULT DIAGNOSIS WITH EVOLUTIONARYALGORITHMS AND NEURAL NETWORKS

MARCIN WITCZAK

Institute of Control and Computation EngineeringUniversity of Zielona Góra

ul. Podgórna 50, 65–246 Zielona Góra, Polande-mail: M.Witczak@issi.uz.zgora.pl

Challenging design problems arise regularly in modern fault diagnosis systems. Unfortunately, the classical analyticaltechniques often cannot provide acceptable solutions to such difficult tasks. This explains why soft computing techniquessuch as evolutionary algorithms and neural networks become more and more popular in industrial applications of faultdiagnosis. The main objective of this paper is to present recent developments regarding the application of evolutionaryalgorithms and neural networks to fault diagnosis. In particular, a brief introduction to these computational intelligenceparadigms is presented, and then a review of their fault detection and isolation applications is performed. Close attention ispaid to techniques that integrate the classical and soft computing methods. A selected group of them is carefully describedin the paper. The performance of the presented approaches is illustrated with the use of the DAMADICS fault detectionbenchmark that deals with a valve actuator.

Keywords: fault diagnosis, evolutionary algorithms, neural networks, robustness

1. Introduction

A continuous increase in the complexity, efficiency, andreliability of modern industrial systems necessities a con-tinuous development of the control and fault diagno-sis theory and practice (Blanke et al., 2003; Korbicz etal., 2004). These requirements extend beyond normallyaccepted safety-critical systems of nuclear reactors andchemical plants or aircrafts to new systems such as au-tonomous vehicles or fast rail systems. Early detectionand maintenance of faults can help avoid the system shut-down, breakdown and even catastrophes involving humanfatalities and material damage. A modern control systemthat is able to tackle such a challenging problem is pre-sented in Fig. 1.

Fault DiagnosisController re-design

Information about faults

Controller

yref

yu

Unknown Inputs

Faults

Process

Model

Analytic redundancy Sensors

Actuators

Act Senso

Fig. 1. Modern control system.

As can be observed, the controlled system is the main part

of the scheme, and it is composed of actuators, process dy-namics and sensors. Each of these parts is affected by theso-called unknown inputs that can be perceived as processand measurement noise as well as external disturbancesacting on the system. When model-based control and di-agnosis are utilized (Blanke et al., 2003; Korbicz et al.,2004), then the unknown input can also be extended bymodel uncertainty, i.e., the mismatch between a model andthe system being considered. The system may also be af-fected by faults, which can be divided into three groups,i.e., actuator faults, component (or process) faults, andsensor faults. The role of the fault diagnosis part is tomonitor the behaviour of the system and to provide allpossible information regarding the abnormal functioningof its components. As a result, the overall task of faultdiagnosis consists of three subtasks: fault detection, faultisolation, and fault identification (Chen and Patton, 1999).However, from a practical viewpoint, to pursue a completefault diagnosis, the following three steps have to be real-ized (Frank and Köppen-Seliger, 1997):

Residual generation: generation of the signals that re-flect the fault. Typically, the residual is defined asa difference between the system and model outputs.

Residual evaluation: logical decision making at the timeof the occurrence and location of faults.

Fault identification: determination of the type of a fault,its size and cause.

M. Witczak86

Knowledge resulting from these steps is then pro-vided to the controller re-design part, which is responsiblefor changing the control law in such a way as to maintain arequired system performance. Thus, the scheme presentedin Fig. 1 can be perceived as a fault-tolerant one.

Fault-Tolerant Control (FTC) (Blanke et al., 2003)is one of the most important research directions underly-ing contemporary control engineering. FTC can also beperceived as an optimized integration of advanced faultdiagnosis (Korbicz et al., 2004; Witczak, 2003) and con-trol (Blanke et al., 2003) techniques. There is no doubtthat the theory (and practice, as a consequence) of fault di-agnosis and control is well-developed and mature for lin-ear systems only (Chen and Patton, 1999; Korbicz et al.,2004). There are also a number of different approachesthat can be employed to settle the robustness problems re-garding model uncertainty (Chen and Patton, 1999; Ko-rbicz et al., 2004). Such robustness is especially impor-tant in practical implementations where various sourcesof uncertainty may be present, e.g., differences betweendifferent copies of a given component, time-varying prop-erties, noise, external disturbances, etc. Another kind ofsolutions that may increase the performance of the FTCscheme is based on appropriate scheduling of control testsignals in such a way as to gain as much information aspossible about the system being supervised (Delebecqueet al., 2003). Unfortunately, this technique can be usedfor linear systems only. In the light of the above discus-sion, it is clear that the development of new approachesfor non-linear systems is justified. From this discussion,it is also clear that fault diagnosis has become an impor-tant issue in the theory and practice of modern automaticcontrol. This justifies the objectives of this paper, whichaims at presenting the latest developments in fault diagno-sis with evolutionary algorithms and neural networks.

In spite of the fact that a large spectrum of analyticaltechniques for Fault Detection and Isolation (FDI) ofnon-linear systems can be found in the literature (Blankeet al., 2003; Chen and Patton, 1999; Korbicz et al., 2004),they usually all suffer from the lack of an appropriatemathematical description of the system being considered.If there are no sufficiently accurate analytical models,then the one feasible way is to use the so-called soft com-puting techniques (Korbicz et al., 2004). A large amountof knowledge on using these techniques for model-based fault diagnosis has been accumulated through theliterature since the beginning of the 1990s (see, e.g.,(Chen and Patton, 1999; Frank and Köppen-Seliger,1997; Korbicz et al., 2004; Ruano, 2005; Witczak, 2003)and the references therein). One objective of this paper isto show the benefits that can be gained while using thesetechniques in practice. Another objective is to show thatthe integration of soft computing and classical techniques

results in hybrid fault diagnosis techniques that are supe-rior to the classical ones.

The paper is organised as follows: Section 2 is de-voted to the application of evolutionary algorithms to faultdiagnosis, while in Section 3 the problem of fault di-agnosis is settled with neural networks. Both sectionspresent examples of integrating soft computing and an-alytical techniques. The presented approaches are illus-trated with results concerning the DAMADICS bench-mark (DAMADICS, 2004). Finally, the closing sectionconcludes the paper.

2. Evolutionary Algorithms in FDI

Evolutionary Algorithms (EAs) are a broad class of sto-chastic optimization algorithms inspired by some biolog-ical processes, which allow populations of organisms toadapt to their surrounding environment. Such algorithmshave been influenced by Darwin’s theory of natural se-lection, or the survival of the fittest (published in 1859).The idea behind it is that only certain organisms can sur-vive, i.e., only those which can adapt to the environmentand win the competition for food and shelter. Almost atthe same time that Darwin’s theory was presented (1865),Mendel published a short monograph about experimentswith plant hybridisation. He observed how traits of differ-ent parents are combined into offspring by sexual repro-duction. Darwinian evolutionary theory and Mendel’s in-vestigations of heredity in plants became the foundationsof evolutionary search methods and led to the creation ofthe neo-Darwinian paradigm (Fogel, 1995).

Evolutionary algorithms have been the subject of anuncountable number of papers (Bäck et al., 1997). Gen-erally, there is a large number of different kinds of EAsand the most popular of them are: genetic algorithms(GAs) (Holland, 1975), genetic programming (GP) (Grayet al., 1998; Koza, 1992), evolutionary programming (Fo-gel et al., 1999), evolutionary strategies (Michalewicz,1996), and evolutionary search with soft selection (Galar,1989).

Although the origins of evolutionary algorithms canbe traced back to the late 1950s (see Bäck et al., 1997 for acomprehensive introduction and survey on EAs), the firstworks on evolutionary algorithms in control engineeringwere published at the beginning of the 1990s. In 2002,Fleming and Purshouse (Fleming and Purshouse, 2002)tackled a challenging task of preparing a comprehensivesurvey on the application of evolutionary algorithms tocontrol engineering. As is indicated in (Fleming and Pur-shouse, 2002), there are relatively scarce publications onapplications of evolutionary algorithms to the design ofFDI systems.

Advances in model-based fault diagnosis. . . 87

This paper, rather than providing an exhaustive sur-vey on evolutionary algorithms in fault diagnosis, is aimedat providing a comprehensive account of the publishedwork that exploits the special nature of EAs. This meansthat the works dealing with EAs applied as alternative op-timisers, e.g., for training neural and/or fuzzy systems arenot included here. In other words, the main objective isto extend the material of (Fleming and Purshouse, 2002)by introducing the latest advances in fault diagnosis withevolutionary algorithms.

Irrespective of the identification method selectedfor designing the model, there always exists the prob-lem of model uncertainty, i.e., the model-reality mis-match. To overcome this problem, many approaches havebeen proposed (Chen and Patton, 1999; Korbicz et al.,2004). Undoubtedly, the most common approach is touse robust observers, such as the Unknown Input Ob-server (UIO) (Chen and Patton, 1999; Korbicz et al.,2004; Witczak, 2003), which can tolerate a degree ofmodel uncertainty and hence increase the reliability offault diagnosis. In such an approach, the model-realitymismatch can be represented by the so-called unknowninput. Hence the state estimate and, consequently, the out-put estimate are obtained taking into account model uncer-tainty. Unfortunately, when the direction of faults is sim-ilar to that of an unknown input, then the unknown inputdecoupling procedure may considerably impair fault sen-sitivity. In order to settle this problem, Chen et al. (1996)(see also Chen and Patton, 1999) formulated observer-based FDI as a multiobjective optimisation problem, inwhich the task was to maximise the effect of faults on theresidual, whilst minimising the effect of an unknown in-put. The approach was applied to the detection of sensorfaults in a flight control system. A similar approach wasproposed by Kowalczuk et al. (1999), where the observerdesign is founded on a Pareto-based approach, in whichthe ranking of an individual solution is based on the num-ber of solutions by which it is dominated. These two so-lutions can be applied to linear systems only.

In spite of the fact that a large amount of knowledgeon designing observers for non-linear systems has beenaccumulated through the literature since the beginning ofthe 1970s, a customary approach is to linearize the non-linear model around the current state estimate, and thento apply techniques for linear systems, as is the case forthe extended Kalman filter (Korbicz et al., 2004). Unfor-tunately, this strategy works well only when linearizationdoes not cause a large mismatch between the linear modeland non-linear behaviuor. To improve the effectiveness ofstate estimation, it is necessary to restrict the class of non-linear systems while designing observers. Unfortunately,the analytical design procedures resulting from such anapproach are usually very complex, even for simple labo-ratory systems (Zolghardi et al., 1996). To overcome this

problem, Porter and Passino proposed the so-called ge-netic adaptive observer (Porter and Passino, 1995). Theyshowed how to construct such an observer, where a ge-netic algorithm evolves the gain matrix of the observerin real time so that the output error is minimized. Apartfrom the relatively simple design procedure, the authorsdid not provide convergence conditions of the observer.They did not consider the robustness issues with respect tomodel uncertainty either. A solution that does not possessuch drawbacks was proposed by Witczak et al. (2002).In particular, the authors showed the convergence con-dition of the observer, and proposed a technique for in-creasing its convergence rate with genetic programming.This approach will be detailed in Section 2.2. It shouldbe strongly underlined that the application of observersis limited by the need for non-linear state-space modelsof the system being considered, which is usually a seri-ous problem in complex industrial systems. This explainswhy most of the examples considered in the literature aredevoted to simulated or laboratory systems, e.g., the cele-brated two- (three- or even four-) tank system, an invertedpendulum, a travelling crane, etc. To tackle this prob-lem, a genetic programming-based approach for designingstate-space models from input-output data was developedin (Witczak et al., 2002; Witczak, 2003). This approachwill be detailed in Section 2.1. A further developmentof this technique related to input-output models was pro-posed in (Metenidis et al., 2004).

Evolutionary algorithms have also been applied toFDI methods that are not based on the concept of resid-uals. Marcu (1997) formulated FDI design as a fea-ture selection and classifier design problem. EA hasalso been applied to the generalised task of determiningthe fault from a collection of symptoms (Miller, 1993).The method relied upon the availability of a-priori prob-abilities that a particular fault caused a particular symp-tom. In (Chen et al., 2003), the authors employedgenetic algorithms-based evolutionary strategy for faultdiagnosis-related classification problems, which includestwo aspects: evolutionary selection of training samplesand input features, and evolutionary construction of theneural network classifier. Finally, Sun et al. (2004) usedthe bootstrap technique to preprocess operational data ac-quired from a running diesel engine, and the genetic pro-gramming approach to find the best compound feature thatcan discriminate between the four kinds of commonly op-erating modes of the engine.

2.1. System Identification for FDI

Let us consider the following class of non-linear discrete-time systems:

xk+1 = g (xk, uk, p) + wk, (1)

M. Witczak88

yk+1 = Ck+1xk+1 + vk, (2)

where p ∈ Rnp is the parameter vector, xk ∈ R

n is thestate vector, uk ∈ R

r is the input vector, yk ∈ Rm is the

output vector, g (·) is a non-linear function, wk ∈ Rn

and vk ∈ Rm are the process and measurement noise, re-

spectively. With a slight abuse of notation, the parametervector will be neglected in model equations. Assume thatthe function g(·) has the form

g(xk, uk) = A(xk)xk + h(uk), (3)

where h(·) is a non-linear function, and A(·) is a matrixof functions.

The state-space model of the system (1)–(2) can beexpressed as

xk+1 = A(xk)xk + h(uk), (4)

yk+1 = Ck+1xk+1, (5)

where xk+1 ∈ Rn and yk+1 ∈ R

m stand for the stateand output estimates, respectively. The problem is to de-termine A(·), Ck+1 and h(·), given a set of input-outputmeasurements {(uk, yk)}nt−1

k=0 . Moreover, it is assumedthat the true state vector xk is, in particular, unknown.Without loss of generality, it is possible to assume that

A(xk) = diag[a1,1(xk), . . . , an,n(xk)]. (6)

Thus, the problem reduces to identifying a i,i(xk),hi(uk), i = 1, . . . , n, and Ck+1, i.e., to obtaining A(·)and h(·). Assuming that |ai,i(xk)| < 1, i = 1, . . . , n,it can be shown (Witczak et al., 2002) that the model(4)–(5) is globally asymptotically stable (Witczak et al.,2002). This implies that ai,i(xk) should have the follow-ing structure:

ai,i(xk) = tanh(si,i(xk)), i = 1, . . . , n, (7)

where tanh(·) is the hyperbolic tangent function, andsi,i(xk) is a function to be determined.

Undoubtedly, many tools can be employed to ob-tain (4)–(5), e.g., neural networks or Genetic Program-ming (GP) (Koza, 1992). GP is an extension of geneticalgorithms (Michalewicz, 1996), which are a broad classof stochastic optimization algorithms inspired by somebiological processes, which allow populations of organ-isms to adapt to their surrounding environment. The maindifference between these two approaches is that in GPthe evolving individuals are parse trees rather than fixed-length binary strings. The main advantage of GP overneural networks is that the models resulting from this ap-proach are less sophisticated (from the point of view ofthe number of parameters).

Since si,i(xk), hi(uk), i = 1, . . . , n, are assumedto be (in general) non-linear functions, they can easily be

x1,kx1,kx1,kx1,k x2,kx2,kx2,kx2,k

++

+

+

∗∗

/

p11

p1

p2 p3

p4 p5 p6 p7

p8 p9 p10 p12 p13 p14 p15

Fig. 2. Exemplary tree representing si,i(xk).

represented as trees, cf. Fig. 2. The language of trees inGP is formed by user-defined sets of functions F and ter-minals T, which form the nodes of the trees (cf. Fig. 2).The functions should be chosen so as to be a-priori use-ful in solving the problem, i.e., any knowledge concern-ing the system under consideration should be included inthe function set. This function set is very important andshould be universal enough to be capable of representinga wide range of non-linear systems. In the case of a pa-rameterized tree, as shown in Fig. 2, the terminal set iscomposed of variables only. Such a parameterization hasproven to be especially useful for model designing pur-poses (Witczak et al., 2002; Witczak, 2003; Witczak andKorbicz, 2004). On the other hand, it leads to the problemof non-linear parameter estimation, which has to be solvedby some non-linear programming tools, e.g., the AdaptiveRandom Search (ARS) algorithm (Walter and Pronzato,1997).

As a result of applying the above approach to theidentification of (1)–(2), each entry of A( xk) and h(uk)can be obtained with a population of trees evolved by theGP algorithm. It should be pointed out that for that par-ticular purpose two sets of terminals can be distinguished,i.e., one for A(xk) (T = {xk}) and the other for h(uk)(T = {uk}).

As can be observed, parameter estimation involvesthe computation of Ck, which is necessary to obtain theoutput error εk and, consequently, the value of the fitnessfunction. To tackle this problem, for each trial point pit is necessary to first set an initial state estimate x0, andthen to obtain the state estimate xk, k = 1, . . . , nt −1. Knowing the state estimate and using the least-squaresmethod, it is possible to obtain Ck (assuming Ck = C)by solving the equation

C

nt−1∑k=0

xkxTk =

nt−1∑k=0

ykxTk . (8)

It should also be pointed out that the order n of themodel is in general unknown and hence should be deter-mined through experiments.

Advances in model-based fault diagnosis. . . 89

2.2. Observer Design with Genetic Programming

Let us consider the class of non-linear systems describedby the following equations:

xk+1 = g (xk) + h(uk + L1,kfk) + Ekdk, (9)

yk+1 = Ck+1xk+1 + L2,k+1fk+1, (10)

where g (xk) is assumed to be continuously differen-tiable with respect to xk, fk ∈ R

s stands for the faultsignal, dk ∈ R

q is the unknown input, and L1,k, L2,k,Ek are their distribution matrices. Similarly to the Ex-tended Kalman Filter (Korbicz et al., 2004), the UIO pre-sented in (Chen and Patton, 1999, pp.98-108) can be ex-tended to the class of non-linear systems (9)–(10). Thisleads to the following structure of the Extended UIO(EUIO):

xk+1/k = g (xk) + h(uk), (11)

xk+1 = xk+1/k + Hk+1εk+1/k + K1,k+1εk, (12)

and

εk+1/k = yk −Ck+1xk+1/k, εk = yk −Ckxk, (13)

where the way of calculating the gain K1,k+1 and un-known input decoupling Hk+1 matrices is given in(Witczak et al., 2002; Witczak, 2003; Witczak and Kor-bicz, 2004).

It should also be pointed out that the matrix Ak usedin the designing procedure is now defined by

Ak =∂g (xk)

∂xk

∣∣∣∣xk=xk

. (14)

The main objective of this section is to show that the con-vergence of the EUIO strongly depends on an appropriatechoice of the instrumental matrices Rk and Qk (mea-surement and process noise covariance matrices, respec-tively, in the stochastic setting). Moreover, the fault-freemode is assumed, i.e., fk = 0.

For notational convenience, let us define the a-prioristate estimation error

ek+1/k = xk+1 − xk+1/k. (15)

As usual, to perform further derivations, it is neces-sary to linearize the model around the current state esti-mate xk. This leads directly to the classical approxima-tion

ek+1/k ≈ Akek + Ekdk. (16)

In order to avoid the above approximation, the di-agonal matrix αk = diag(α1,k, . . . , αn,k) can be intro-duced (Witczak et al., 2002), which makes it possible toestablish the following exact equality:

ek+1/k = αkAkek + Ekdk. (17)

The problem is to obtain an appropriate form of the in-strumental matrices Qk−1 and Rk in such a way as toensure the convergence of the observer or to adequatelymaximize the bounds of the diagonal elements of the ma-trix αk.

For that purpose, Witczak et al. (2002) performeda comprehensive convergence analysis with the Lyapunovmethod. As a result, they obtained the following condi-tions:

σ (αk) ≤ γ1 =σ (Ak)σ (Ak)

⎛⎝ (1 − ζ)σ (Pk)

σ(A1,kP ′

kAT1,k

)⎞⎠

12

,

and

σ (αk − I) ≤ γ2 =σ (Ak)σ (Ak)

×(

σ(CT

k

)σ (Ck)

σ(CT

k

)σ (Ck)

σ (Rk)σ(CkPkCT

k + Rk

)) 12

.

Bearing in mind the fact that αk is a diagonal ma-trix, the above inequalities can be expressed as

maxi=1,...,n

|αi,k| ≤ γ1, maxi=1,...,n

|αi,k − 1| ≤ γ2. (18)

Since (cf. Chen and Patton, 1999, pp. 98–108):

Pk = A1,kP ′kAT

1,k + TkQk−1TTk + HkRkHT

k , (19)

it is clear that an appropriate selection of the instrumentalmatrices Qk−1 and Rk may enlarge the bounds γ1 andγ2 and, consequently, the domain of attraction. Indeed,if the conditions (18) are satisfied, then xk converges toxk.

Unfortunately, analytical derivation of the matricesQk−1 and Rk seems to be an extremely difficult prob-lem. However, it is possible to set the above matricesas follows: Qk−1 = β1I, Rk = β1I, with β1 andβ1 large enough. On the other hand, it is well knownthat the convergence rate of such an EKF-like approachcan be increased by an appropriate selection of the covari-ance matrices Qk−1 and Rk, i.e., the more accurate (near“true” values) the covariance matrices, the better the con-vergence rate. This means that in the deterministic case(k = 0 and vk = 0), both matrices should be zero.Unfortunately, such an approach usually leads to the di-vergence of the observer as well as other computationalproblems. To tackle this issue, a compromise betweenthe convergence and the convergence rate should be estab-lished. This can easily be done by setting the instrumentalmatrices as

Qk−1 = β1εTk−1εk−1I + δ1I, (20)

Rk = β2εTk εkI + δ2I, (21)

M. Witczak90

Valve

S

Z1

Z2

Z3

T1 P1 P2

F

CV

X

ACQ

PP

PC

E/P CPU

PS

Fig. 3. Actuator and its scheme.

with β1, β2 large enough, and δ1, δ2 small enough. Al-though this approach is very simple, it is possible to fur-ther increase the convergence rate. Indeed, the instrumen-tal matrices can be set as

Qk−1 = q2(εk−1)I + δ1I,

Rk = r2(εk)I + δ2I, (22)

where q(εk−1) and r(εk) are non-linear functions of theoutput error εk (the squares are used to ensure the pos-itive definiteness of Qk−1 and Rk). Thus, the problemreduces to identifying the above functions. In particular,in (Witczak et al., 2002) it is shown how to reduce the taskof the designing of Qk−1 and Rk to the multi-objectiveand global structure optimization problem of q(εk−1)and r(εk). Genetic programming is utilized (Witczak etal., 2002; Witczak, 2003; Witczak and Korbicz, 2004) totackle this problem.

2.3. An Illustrative Example – the DAMADICSBenchmark

DAMADICS (Development and Application of Methodsfor Actuator Diagnosis in Industrial Control Systems)was a research project focused on drawing together wide-ranging techniques and fault diagnosis within the frame-work of a real application to on-line diagnosis of a 5-stageevaporisation plant of a sugar factory in Lublin, Poland.The project was focused on the diagnosis of valve (cf.Fig. 3) plant actuators and looked towards real implemen-tation methods for new actuator systems. The sugar fac-tory was a subcontractor (under the Warsaw University ofTechnology) providing real process data and the evalua-tion of trials of fault diagnosis methods.

The control valve constitutes the means used to pre-vent, permit and/or limit the flow of sugar juice throughthe control system (a detailed description of this actuatorcan be found in (DAMADICS, 2004)). As can be seen inFig. 3, the following process variables can be measured:CV is the control signal, P1 is the pressure at the inletof the valve, P2 is the pressure at the outlet of the valve,T 1 is the juice temperature at the inlet of the valve, Xis the servomotor rod displacement, F is the juice flow atthe outlet of the valve. In Fig. 3, three additional bypassvalves (denoted by z1, z2, and z3) can be seen. The stateof these valves can be controlled manually by the oper-ator. They are introduced for manual process operation,actuator maintenance and safety purposes. The data gath-ered from the real plant are available on the DAMADICSwebsite (DAMADICS, 2004). Although a large amountof real data are available, they do not cover all faulty situ-ations.

The objective of this section is to design the state-space model of the actuator being considered (cf. Fig. 3)according to the approach described in Section 2.1. Theparameters used during the identification process werenm = 200, nd = 10, ns = 10, F = {+, ∗, /}. Forthe sake of comparison, the linear state-space model wasobtained with the use of the MATLAB System Identifi-cation Toolbox. In both the linear and non-linear cases,the order of the model was tested between n = 2, . . . , 8.Unfortunately, the relation between the input uk =(CV, P1, P2, T 1) and the juice flow y1,k (yk = (F, X))cannot be modelled by a linear state-space model. Indeed,the modelling error was approximately 35%, thus mak-ing the linear model unacceptable. On the other hand, therelation between the input uk and the rod displacementy2,k can be modelled, with very good results, by the linear

Advances in model-based fault diagnosis. . . 91

state-space model. Bearing this in mind, the identificationprocess was decomposed into two phases, i.e.,

1. Derivation of a relation between the rod displace-ment and the input with a linear state-space model.

2. Derivation of a relation between the juice flow andthe input with a non-linear state-space model de-signed by GP.

Experimental results showed that the best-suited linearmodel is of the order n = 2. After 50 runs of the GPalgorithm performed for each model order, it was foundthat the order of the model which provides the best ap-proximation quality is n = 2. The mean-squared outputerror for the obtained model was 0.0079 (the model struc-ture can be found in (Witczak and Korbicz, 2004)). Theresponse of the model obtained for the validation data setis given in Fig. 4.

0 50 100 150 200 250 300 350 400 450 5000

0.2

0.4

0.6

0.8

1

1.2

1.4

y 1,k

Discrete time

(a)

0 50 100 150 200 250 300 350 400 450 5000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

y 2,k

Discrete time

(b)

Fig. 4. System (dotted line) and model (solid line)outputs (juice flow (a), rod displacement(b)) for the validation data set.

The main differences between the behaviour of themodel and the system were observed for the non-linearmodel (juice flow) during system saturation. This inaccu-racy constitutes the main part of modelling uncertainty.

Since the state-space model is given, it is possibleto design the EUIO presented in Section 2.2. To tacklethe determination of the unknown input distribution ma-trix Ek, the approach proposed in (Witczak and Korbicz,2004) was employed, and then the approach of Section 2.2along with the threshold selector described in (Witczakand Korbicz, 2004) were used for fault detection. Thisthreshold selector is based on a polynomial describing therelation between the control signal and the residual. Inparticular, adaptive residual bounds are provided based onthe parameter confidence region of the above polynomial.

Since the method of designing an appropriate thresh-old is known, it is possible to check fault detection ca-pabilities of the presented observer-based fault detectionscheme. Figure 5 presents the results of fault detection for

0 100 200 300 400 500 600 700 800−0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8r 1

Discrete time

(a)

0 100 200 300 400 500 600 700 800−0.12

−0.1

−0.08

−0.06

−0.04

−0.02

0

0.02

0.04

0.06

r 2

Discrete time

(b)

Fig. 5. Residuals for the flow sensor fault and its bounds(juice flow – (a), rod displacement – (b)).

M. Witczak92

the flow sensor fault (with a small magnitude). As can beobserved, the fault can be detected with the use of the juiceflow residual. A complete report regarding fault diagnosisof the DAMADICS benchmark can be found in (Witczakand Korbicz, 2004).

3. Neural Networks in FDI

Generally, neural networks (Gupta et al., 2003) can be per-ceived as a conveniently parameterized set of non-linearmaps. In the last fifteen years, neural networks have beensuccessfully used in solving complex problems in mod-elling and pattern recognition (see Gupta et al., 2003 andthe references therein). In the case of pattern recognition,a finite set of input-output pairs is given, where the inputsrepresents the objects to be recognised while the outputsstand for the pattern classes to which they belong. Thus,the role of a neural network is to approximate the mapbetween these two spaces. In the case of modelling, it isassumed that the input-output relation is formed by a non-linear system, and the role of a neural network is to ap-proximate the behaviour of this system. In both cases, theapplication of neural networks is justified by the assump-tion that there exists a non-linear input-output map. Akey theoretical result behind both applications is the factthat neural networks are universal approximators (Guptaet al., 2003). There are, of course, many different prop-erties (see, e.g. (Chandra and Sing, 2004)), which makeneural networks attractive for practical applications.

At the beginning of the 1990s, neural networks wereproposed for identification and control (see, e.g., (Naren-dra, 1990)). A rapid development concerning applicationsof neural networks to control engineering resulted in alarge number of publications related to this subject. Huntet al. (1992) confirmed the fast development of this re-search area by publishing a survey on neural networks incontrol engineering. In 1995, a similar work was pub-lished by Sjoberg et al. (1995) in the context of systemidentification with neural networks. Nowadays, the num-ber of applications has increased significantly. Fault diag-nosis constitutes one of the thrusts of the research efforton neural networks for control (Korbicz et al., 2004).

The main objective of the subsequent part of this pa-per is to present the development of this particular re-search area. Rather than providing an exhaustive surveyon neural networks in fault diagnosis, the aim is to pro-vide a comprehensive account of the published works thatexploit the special nature of neural networks. Indeed, it isimpossible to count all publications on fault diagnosis inwhich neural networks are used as models of the systemsbeing diagnosed. The strategy underlying such an ap-proach boils down to generating the residual with the sys-tem and neural network outputs, respectively. Examples

of using such an approach with the classical multi-layerperceptron are: leakage detection in an electro-hydrauliccylinder drive in a fluid power system (Watton and Pham,1997), the diagnosis of non-catastrophic faults in a nuclearplant (Weerasinghe et al., 1998), and process valve actu-ator fault diagnosis (Karpenko et al., 2003). Similar ex-amples relating to dynamic neural networks are: the diag-nosis of a chemical plant (Fuente and Saludes, 2000), thediagnosis of a valve actuator (Korbicz et al., 2004; Patanand Parisini, 2005), and the diagnosis of a steam evapora-tor (Janczak, 2005).

There are a number of works concerning observer de-sign with neural networks (Alessandri et al., 1997; Guoand Zhu, 2002). Thus, if non-linear state-space models areavailable, then these approaches can be utilized for resid-ual generation and fault diagnosis. Moreover, robustnesswith respect to model uncertainty can also be achieved byusing the concept of an unknown input. Unfortunately,when the direction of faults is similar to that of an un-known input, then the unknown input decoupling proce-dure may considerably impair the fault sensitivity. If theabove-mentioned approach fails, then describing modeluncertainty in a different way seems to be a good rem-edy. One of possible approaches is to use statistical tech-niques (Atkinson and Donev, 1992; Walter and Pronzato,1997) (for an example regarding different approaches, thereader is referred to (Delebecque et al., 2003)) to obtainparameter uncertainty of the model and, consequently,model output uncertainty. Such parameter uncertainty isdefined as the parameter confidence region (Atkinson andDonev, 1992; Walter and Pronzato, 1997) containing a setof admissible parameters that are consistent with the mea-sured data. Thus it is evident that parameter uncertaintydepends on measurement uncertainty, i.e., noise, distur-bances, etc.

Knowledge about parameter uncertainty makes itpossible to design the so-called adaptive threshold (Franket al., 1999). The adaptive threshold, contrary to the fixedone (cf. Fig. 6), bounds the residual at a level that is de-pendent on model uncertainty, and hence it provides morereliable fault detection. Contrary to the typical indus-trial applications of neural networks that are presentedin the literature (Chen and Patton, 1999; Karpenko etal., 2003; Korbicz et al., 2004), Witczak et al. (2006)defined the task of designing a neural network in sucha way as to obtain a model with possibly small uncer-tainty. Indeed, the approaches presented in the litera-ture try to obtain a model that is best suited to a par-ticular data set. This may result in a model with rela-tively large uncertainty. Degraded performance of faultdiagnosis constitutes a direct consequence of using suchmodels. To tackle this challenging problem, the GMDH(Group Method of Data Handling) approach was adaptedand modified (Ivakhnenko and Mueller, 1995; Korbicz et

Advances in model-based fault diagnosis. . . 93

residual

time

adaptive threshold

false alarm

residual signal

fault

fixed threshold

Tfa Tf

Fig. 6. Principle of an adaptive threshold.

al., 2004). They proposed a complete design procedureconcerning the application of GMDH neural networks torobust fault detection. Starting from a set of input-outputmeasurements of the system, it is shown how to estimatethe parameters and the corresponding uncertainty of aneuron using the so-called bounded-error approach (Mi-lanese et al., 1996; Walter and Pronzato, 1997). As a re-sult, a tool that is able to generate an adaptive thresholdwas obtained. The methodology developed for parameterand uncertainty estimation of a neuron makes it possibleto formulate an algorithm that allows obtaining a neuralnetwork with relatively small modelling uncertainty. Allthe hard computations regarding the design of the GMDHneural network are performed off-line and hence the prob-lem regarding the time consuming calculations is not ofparamount importance. The approach can also be ex-tended for dynamic systems by using the dynamic neuronstructure (Mrugalski and Witczak, 2002).

It is well known that the reliability of such fault di-agnosis schemes is strongly dependent on model uncer-tainty, i.e., the mismatch between a neural network andthe system being considered. Thus, it is natural to mini-mize model uncertainty as far as possible. This can be re-alized with the application of Optimum Experimental De-sign (OED) theory (Atkinson and Donev, 1992; Uci nski,2005; Walter and Pronzato, 1997). Some authors haveconducted active investigations in this important researcharea. White (1989), MacKay (1992), and Cohn (1994)showed the attractiveness of the application of OED toneural networks. Fukumizu (1996; 2000) developed theso-called statistical active learning technique, which isbased on the general theory of OED. Recently, Witczakand Pretki (2005) developed a D-optimum experimentaldesign strategy that can be used for training single-outputneural networks. They also showed how to use the ob-tained network for robust fault detection with an adaptivethreshold. In (Witczak, 2006), the author showed how toextend this technique to multi-input multi-output neural

networks. He also proposed a sequential experimental de-sign algorithm that allows obtaining a one-step-ahead D-optimum input. This algorithm can be perceived as a hy-brid one since it can be used for both training and datadevelopment. Section 3.1 presents selected parts of theabove-described design methodology.

Finally, there are also a large number of approachesthat use neural networks as pattern classifiers (Korbiczet al., 2004) to tackle the FDI problem. Instead of us-ing neural networks as models of the systems being di-agnosed, the networks are trained to recognise differentmodes of the system, i.e., both faulty and non-faulty ones.Examples of using such an approach are: FDI in hydraulicfluid power systems (Le et al., 1997; Le et al., 1998), FDIin machine dynamics and vibration problems (Yen andLin, 2000), sensor fault diagnosis (Zhang et al., 2000),fault diagnosis of chemical processes (Zhao et al., 1998),and fault diagnosis of a two-tank system (Korbicz et al.,2004).

3.1. Experimental Design for Neural Networks

Consider a feed-forward neural network given by the fol-lowing equation (Witczak, 2006):

ym,k = P (l)g(P (n)uk

), (23)

where ym,k ∈ Rm stands for the model output, g(·) =

[g1(·), . . . , gnh(·), 1]T , where gi(·) = g(·) is a non-linear

differentiable activation function,

P (l) =

⎡⎢⎢⎣p(l)(1)T

...

p(l)(m)T

⎤⎥⎥⎦ , P (n) =

⎡⎢⎢⎣p(n)(1)T

...

p(n)(nh)T

⎤⎥⎥⎦ ,

(24)are matrices representing the parameters (weights) of themodel, nh is the number of neurons in the hidden layer.Moreover, uk ∈ R

nr+1, uk = [u1,k, . . . , unr,k, 1]T

where ui,k, i = 1, . . . , nr are system inputs. For thesake of notational simplicity, define the following para-meter vector:

p =[p(l)(1)T , . . . , p(l)(m)T ,

p(n)(1)T , . . . , p(n)(nh)T]T

,

where p ∈ Rnp , np = m(nh + 1) + nh(nr + 1). Conse-

quently, (23) can be written in a more compact form:

ym,k = f (p, uk) , (25)

where f (·) is a non-linear function representing thestructure of the neural network.

M. Witczak94

∂f (p, uk)∂p

=

⎡⎢⎢⎣g(P (n)uk

)T0T

(m−1)(nh+1) pl1(1)g′

(uT

k pn(1))uT

k . . . plnh

(1)g′(uT

k pn(nh))uT

k

......

......

...

0T(m−1)(nh+1) g

(P (n)uk

)Tpl1(m)g′

(uT

k pn(1))uT

k . . . plnh

(m)g′(uT

k pn(nh))uT

k

⎤⎥⎥⎦. (30)

Assume that the system output satisfies

yk = ym,k + εk = f (p, uk) + εk, (26)

where the noise ε is zero-mean, Gaussian, and uncorre-lated in k, i.e., its statistics are

E(εk) = 0, E(εiεTk ) = δi,kC, (27)

where C ∈ Rm×m is a known positive-definite matrix

of the form C = σ2Im, and σ2 and δi,k stand for thevariance and Kronecker’s delta symbol, respectively. Un-der such an assumption, the theory of experimental de-sign (Atkinson and Donev, 1992; Walter and Pronzato,1997) can be exploited to develop a suitable training dataset that allows obtaining a neural network with consid-erably smaller uncertainty than those designed without it.First, let us define the so-called Fisher information matrix,which constitutes a measure of parametric uncertainty of(23):

P−1 =nt∑

k=1

RkRTk , (28)

Rk =(

∂f (p, uk)∂p

)T

p=p

, (29)

and the Jacobi matrix is defined by Eqn. (30), whereg′(t) = dg(t)/dt, p is the least-squares estimate of p,and nt stands for the number of input-output measure-ments. It is easy to observe that the FIM (28) dependson the experimental conditions ξ = [u1, . . . , unt ]. Thus,optimal experimental conditions can be found by choos-ing uk, k = 1, . . . , nt, so as to minimize some scalarfunction of (28). Such a function can be defined in vari-ous ways (Fukumizu, 2000; Ucinski, 2005), and here arethe most popular of them:

• D-optimality criterion:

Φ(ξ) = detP , (31)

• G-optimality criterion:

Φ(ξ) = maxuk∈U

φ(ξ, uk). (32)

where U stands for the set of admissible uk thatcan be used for the system being considered (design

space), and

φ(ξ, uk) = trace(RT

k PRk

)=

m∑i=1

ri,kPrTi,k,

(33)

where ri,k stands for the i-th row of RTk .

A valuable property of the FIM is that its inverseconstitutes an approximation of the covariance matrix forp (Goodwin and Payne, 1977). Thus, D-optimum designminimizes the volume of the confidence ellipsoid approxi-mating the feasible parameter set of (23) (see, e.g., (Atkin-son and Donev, 1992, Sec. 6.2) for further explanations).G-optimum design minimizes the variance of the esti-mated response of (23). The D-optimality criterion hasbeen employed by many authors in the development ofcomputer algorithms for calculating optimal experimentaldesigns. Another important property is that D-optimumdesigns are invariant with respect to non-degenerate lin-ear transformations of the model. It is also importantto underline that, from the practical point of view, D-optimum designs often perform well according to othercriteria (see (Atkinson and Donev, 1992) and the refer-ences therein for more details). For further explanationsregarding D-optimality criteria, the reader is referred tothe excellent textbooks (Atkinson and Donev, 1992; Fe-dorov and Hackl, 1997; Ucinski, 2005; Walter and Pron-zato, 1997).

Since fault diagnosis applications are the primarypurpose, the main objective is to use a design criterionwhich makes it possible to obtain accurate bounds of thesystem output (cf. Fig. 7). Indeed, it is rather pointlessto assume that it is possible to develop a neural networkwith arbitrarily small uncertainty, i.e., to obtain a perfectmodel of the system. A more realistic task is to designa model that will provide reliable knowledge about thebounds of the system output that reflect the expected sys-tem behaviour. The design methodology of such robusttechniques rests on the paradigm that fault diagnosis andcontrol schemes should perform reliably for all kinds ofsystem behaviour that are consistent with output bounds.This is in contradiction with the conventional approaches,where fault diagnosis and control schemes are designed tobe optimal for one single model.

Advances in model-based fault diagnosis. . . 95

yi,k

yMi,k

yNi,k

yi

k

Fig. 7. i-th output of the system and its boundsobtained with a neural network.

The bounds presented in Fig. 7 can be described asfollows:

yNi,k ≤ yi,k ≤ yM

i,k, i = 1, . . . , m. (34)

In (Chryssolouris et al., 1996), the authors developed anapproach that can be used for determining (34) (that formsthe 100(1−α) confidence interval of yi,k) for single out-put (m = 1) neural networks. In (Witczak, 2006), theapproach of (Chryssolouris et al., 1996) was extended tomulti-output models, i.e., it can be shown that yN

i,k andyM

i,k (that form the 100(1−α) confidence interval of y i,k)can be approximated as follows:

yNi,k = yi,k − t

α/2nt−np

σ(1 + ri,kPrT

i,k

)1/2, (35)

yMi,k = yi,k + t

α/2nt−np

σ(1 + ri,kPrT

i,k

)1/2, (36)

i = 1, . . . , m, where tα/2nt−np

is the t-Student distributionquantile, and σ is the standard deviation estimate. Bear-ing in mind the fact that the primary purpose of this workis to develop reliable bounds of the system output, it isclear from (34), (35), and (36) that the G-optimality crite-rion should be selected.

When some experiments are repeated, the numberne of distinct uks is smaller than the total number of ob-servations nt. The design resulting from this approachis called the continuous experimental design and can bedescribed as follows:

ξ =

{u1 u2 . . . une

μ1 μ2 . . . μne

}, (37)

where uks are said to be the support points, andμ1, . . . , μne , μk ∈ [0, 1] are their weights, which satisfy∑ne

k=1 μk = 1.

Thus, the Fisher information matrix can now be de-fined as follows:

P−1 =ne∑

k=1

μkRkRTk . (38)

The fundamental property of continuous experimental de-sign is the fact that optimum designs resulting from theD- and G-optimality criteria are the same (the Kiefer-Wolfowitz equivalence theorem Atkinson and Donev,1992; Walter and Pronzato, 1997; Ucinski, 2005). Finally,in the light of this theorem, the design (37) is D-optimumwhen

φ(ξ, uk) = trace(RT

k PRk

) ≤ np, (39)

where the equality holds for measurements described by(37).

Generally, it is impossible to obtain the design (37) inan analytical way. To tackle this problem, in (Witczak andPretki, 2005) the authors employed the Wynn-Fedorov al-gorithm (Ucinski, 2005; Walter and Pronzato, 1997) toobtain a D-optimum design for a single-output neural net-work, while in (Witczak, 2006) it is shown how to useit for a general multi-output multi-input neural network.The subsequent section presents an illustrative example ofusing the above approach for fault detection of a valve ac-tuator (DAMADICS benchmark).

3.2. Experimental Results

Let us reconsider an example presented in Section 2.3.The main objective of the subsequent part of this sectionis to develop a neural network that can be used for faultdetection of an industrial valve actuator. The above taskwas divided into the following steps (Witczak, 2006):

Step 1: Training of a network based on the nominal dataset.

Step 2: Design of the experiment with the Wynn-Fedorov algorithm described in (Witczak, 2006)based on the network obtained in Step 1.

Step 3: Training of a network based on the data obtainedwith optimal experimental design.

Based on the experience with an industrial valve actu-ator, it was observed that the following subset of mea-sured variables is sufficient for fault detection purposes:u = (CV, P1, 1), y = F .

In Step 1, a number of experiments (the trainingof a neural network with the Levenberg-Marquardt algo-rithm (Walter and Pronzato, 1997)) were performed in or-der to find a suitable number of hidden neurons nh (cf.(23)). For that purpose, nt = 100 data points were gen-erated, for which inputs were uniformly spread withinthe design region U, where 0.25 < u1 < 0.75 and0.6625 < u2 < 0.8375. As a result, a neural model con-sisting of nh = 5 hidden neurons was obtained. Themain objective of Step 2 was to utilize the above model

M. Witczak96

0.30.4

0.50.6

0.7

0.7

0.75

0.8

12

14

16

18

20

U1U2

0.3 0.4 0.5 0.6 0.7

0.68

0.7

0.72

0.74

0.76

0.78

0.8

0.82

U1

U2

Fig. 8. Variance function and the correspondingsupport points.

and the Wynn-Fedorov algorithm in order to obtain D-optimum experimental conditions. First, an initial exper-iment was generated in such a way as to equally dividethe design space of u. Finally, the Wynn-Fedorov al-gorithm was applied. Figure 8 shows the support points(ne = 45) and the variance function (33) for the obtainedD-optimum design. As can be observed, the D-optimalitycondition (39) is satisfied. Based on the obtained contin-uous design, a set consisting of nt = 100 points was ob-tained and used for data generation. The number of repeti-tions of each optimal input uk was calculated by suitablyrounding the numbers μknt, k = 1, . . . , ne (Atkinsonand Donev, 1992; Walter and Pronzato, 1997). It shouldbe strongly stressed that the data were collected in a steadystate because the utilized model (23) was static. Finally,the new data set was used for training the network withthe Levenberg-Marquardt algorithm. Figure 9 presentsthe residual signal obtained with a network trained withthe D-optimum data set as well as an adaptive thresh-old provided by this network (2-nd network). This figurealso presents an adaptive threshold provided by a network

100 200 300 400

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

discrete time

resi

dual

s

residual signalbounds for 1−st netbounds for 2−nd net

390 400 410 420 430 440−0.4

−0.3

−0.2

−0.1

0

0.1

0.2

discrete time

resi

dual

s

residual signalbounds for 1−st netbounds for 2−nd net

Fig. 9. Residual and adaptive thresholdsfor the valve clogging fault.

(1-st network) trained with the data set generated byevenly dividing the design space. It can be observed thatthe neural network obtained with the use of D-optimumexperimental design makes it possible to obtain more ac-curate bounds than those obtained with a neural networktrained in a different manner. Indeed, as can be seen inFig. 9, the valve clogging fault (which, in the light of itsnature, is hard to detect) can be detected with the help ofthe second network while it is impossible to detect it withthe use of the first one. It should be strongly underlinedthat the situation is even worse when the first network isused for residual generation, i.e., in the presented exam-ple it was used for adaptive threshold generation only. Fi-nally, Fig. 10 presents exemplary residuals for flow sensorfaults (cf. the results of Section 2.3). A complete reportregarding fault diagnosis of the DAMADICS benchmarkcan be found in (Witczak, 2006).

4. Concluding Remarks

The rapid development of modern fault diagnosis exposesthe disadvantages of the classical model-based techniques,

Advances in model-based fault diagnosis. . . 97

100 200 300 400−0.1

−0.05

0

0.05

0.1

discrete time

resi

dual

s

residual signalbounds

Fig. 10. Residual and adaptive thresholdsfor the flow sensor fault.

which can be perceived from various angles. In spiteof the fact that a large spectrum of analytical techniquesfor FDI of non-linear systems can be found in the liter-ature, they usually all suffer from the lack of an appro-priate mathematical description of the system being con-sidered. Another difficulty is the fact that the classicalsolutions to the robustness problem may impair the effi-ciency and quality of fault diagnosis, e.g., when the direc-tion of faults is similar to that of an unknown input. Aswas shown in the paper, to settle such a challenging prob-lem, it is possible to employ evolutionary algorithms andneural networks. Indeed, a large amount of knowledgeon using these techniques for model-based fault diagnosishas been accumulated through the literature since the be-ginning of the 1990s. One objective of this paper was tosummarize the advances in this interesting research area.Rather than providing an exhaustive survey on evolution-ary algorithms and neural networks in fault diagnosis, thepaper was aimed at providing a comprehensive accountof the published works that exploit the special nature ofthese techniques. Special attention was paid to techniquesthat integrate the classical and soft computing approachesin such a way as to tackle inevitable non-linearity and ro-bustness problems. In particular, it was shown that theintegration of genetic programming and unknown inputobservers results in a hybrid fault diagnosis technique thatis superior over the classical one. Another example con-cerned the problem of robust fault detection using neuralnetworks. It was shown that the application of the classi-cal experimental design theory makes it possible to obtaina neural network with considerably smaller uncertaintythan the one designed without it. The author hopes that thematerial presented in this paper will encourage engineersto use the discussed techniques in industrial practice.

Acknowledgments

The author would like to express his sincere gratitude tothe referees whose efforts have significantly improved thequality of the paper. The work was supported by the Min-istry of Education and Science in Poland under the grant4T11A01425 Modelling and identification of non-lineardynamic systems in robust diagnostics.

References

Alessandri A., Parisini T. and Zoppoli R (1997): Neural approx-imators for non-linear finite-memory state estimation. —Int. J. Contr., Vol. 67, No. 2, pp. 275–302.

Atkinson A.C. and Donev A.N. (1992): Optimum ExperimentalDesigns. — New York: Oxford University Press.

Bäck T., Hammel U. and Schwefel H.P. (1997): Evolutionarycomputation: Comments on the history and current state.— IEEE Trans. Evolut. Computat., Vol. 1, No. 1, pp. 3–17

Blanke M., Kinnaert M., Lunze J. and Staroswiecki M. (2003):Diagnosis and Fault-Tollerant Control. — Berlin: Springer

Chandra P. and Sing Y. (2004): Feedforward sigmoidal net-works – Equicontinuity and fault tolerance. — IEEE Trans.Neural Netw., Vol. 15, No. 6, pp. 1350–1366.

Chen J. and Patton R. J. (1999): Robust Model-based Fault Di-agnosis for Dynamic Systems. — London: Kluwer.

Chen J., Patton R. J. and Liu G.P. (1996): Optimal residual de-sign for fault diagnosis using multi-objective optimizationand genetic algorithms. — Int. J. Syst. Sci., Vol. 27, No. 6,pp. 567–576.

Chen Z., He Y., Chu F. and Huang J. (2003): Evolutionarystrategy for classification problems and its application infault diagnosis. — Eng. Appl. Artif. Intell., Vol. 16, No. 1,pp. 31–38.

Cohn D.A. (1994): Neural network exploration using optimalexperimental design, In: Advances in Neural InformationProcessing Systems 6 (J. Cowan et al., Eds.). — San Ma-teo: Morgan Kaufman.

Chryssolouris G., Lee M. and Ramsey A. (1996): Confidenceinterval prediction for neural network models. — IEEETrans. Neural Netw., Vol. 7, No. 1, pp. 229–232.

DAMADICS (2004): Website of DAMADICS: De-velopment and Application of Methods for Ac-tuator Diagnosis in Industrial Control Systems.http://diag.mchtr.pw.edu.pl/damadics/.

Delebecque F., Nikoukah R. and Rubio Scola H. (2003): Testsignal design for failure detection: A linear programmingapproach. — Int. J. Appl. Math. Comput. Sci., Vol. 13,No. 4, pp. 515–526.

Fedorov V.V., and Hackl P. (1997): Model-Oriented Design ofExperiments. — New York: Springer.

Fleming P.J. and Purshouse (2002): Evolutionary algorithmsin control systems engineering: A survey. — Contr. Eng.Pract., Vol. 10, No. 11, pp. 1223–1241.

M. Witczak98

Fogel D.B. (1995): Evolutionary Computation: Toward a NewPhilosophy of Machine Intelligence. — New York: Willey-IEEE Press.

Fogel L.J., Owens A.J. and Walsh M.J. (1999): An overview ofevolutionary programming, In: Evolutionary Algorithms(De Jong K., Vose M.D. and Whitley L.D., Eds.). — Hei-delberg: Springer.

Frank P.M. and Köppen-Seliger (1997): New developments us-ing AI in fault diagnosis. — Eng. Appl. Artif. Intell.,Vol. 10, No. 1, pp. 3–14.

Frank P.M., Schreier G. and Garcia E.A. (1999): Non-linear ob-servers for fault detection and isolation, In: New Direc-tions in Non-linear Observer Design (Nijmeijer H., Fos-sen T.I., Eds.). — Berlin: Springer.

Fuente M.J. and Saludes S. (2000): Fault detection and isolationin a non-linear plant via neural networks. — Proc. Symp.Fault Detection, Supervision and Safety for TechnicalProcesses, SAFEPROCESS, Budapest, Hungary, pp. 472–477.

Fukumizu K. (1996): Active learning in multilayer perceptrons,In: Advances in Neural Information Processing Systems(D.S. Touretzky et al., Eds.). — Cambridge: MIT Press,pp. 295–301.

Fukumizu K. (2000): Statistical active learning in multilayerperceptrons. — IEEE Trans. Neural Netw., Vol. 11, No. 1,pp. 17–26.

Galar R. (1989): Evolutionary search with soft selection. —Biol. Cybern., Vol. 60, No. 1, pp. 124–141.

Goodwin G.C. and Payne R.L. (1977): Dynamic Systems Iden-tification. Experiment Design and Data Analysis. — NewYork: Acadamic Presss.

Gray G. J, Murray-Smith D. J., Li Y., Sharman K. C. and Wein-brenner T. (1998): Non-linear model structure identifica-tion using genetic programming. — Contr. Eng. Pract.,Vol. 6, No. 11, pp. 1341–1352.

Guo L.Z. and Zhu Q.M. (2002): A fast convergent extendedKalman observer for non-linear discrete-time systems. —Int. J. Syst. Sci., Vol. 33, No. 13, pp. 1051–1058.

Gupta M.M., Jin L. and Homma N. (2003): Static and DynamicNeural Networks. From Fundamentals to Advanced The-ory. — Hoboken, NJ: Wiley.

Holland J.H. (1975): Adaptation in Natural and Artificial Sys-tems. — Ann Arbor, MI: The University of Michigan Press.

Hunt K.J., Sbarbaro D., Zbikowski R. and Gawthrop P.J. (1992):Neural networks for control systems – A survey. — Auto-matica, Vol. 28, No. 6, pp. 1083–1112.

Ivakhnenko A.G. and Mueller J.A. (1995): Self-organizing ofnets of active neurons. — Syst. Anal. Modell. Simul.,Vol. 20, pp. 93–106.

Janczak A. (2005): Identification of Non-Linear Systems Us-ing Neural Networks and Polynomial Models: A Block-Oriented Approach. — Berlin: Springer.

Karpenko M., Sepehri N. and Scuse D. (2003): Diagnosis ofprocess valve actuator faults using a multilayer neural net-work. — Contr. Eng. Practice, Vol. 11, No. 11, pp. 1289–1299.

Korbicz J., Koscielny J.M., Kowalczuk Z. and Cholewa W.(Eds.) (2004): Fault Diagnosis. Models, Artificial Intelli-gence, Applications. — Berlin: Springer.

Kowalczuk Z., Suchomski P. and Białaszewski T. (1999): Evo-lutionary muti-objective Pareto optimization of diagnosticstate observers. — Int. J. Appl. Math. Comput. Sci., Vol. 9,No. 3, pp. 689–709.

Koza J. R. (1992): Genetic Programming: On the Programmingof Computers by Means of Natural Selection. — Cam-bridge: The MIT Press.

Le T.T., Watton J. and Pham D.T. (1997): An artificial neuralnetwork based approach to fault diagnosis and classifi-cation of fluid power systems. — J. Syst. Contr. Eng.,Vol. 211, No. 4, pp. 307–317.

Le T.T., Watton J. and Pham D.T. (1998): Fault classificationof fluid power systems using a dynamics feature extrac-tion technique and neural networks. — J. Syst. Contr. Eng.,Vol. 212, No. 2, pp. 87–97.

MacKay D. (1992): Information-based objective functions foractive data selection. — Neural Comput., Vol. 4, No. 4,pp. 305–318.

Marcu T. (1997): A multiobjective evolutionary approachto pattern recognition for robust diagnosis of processfaults. — Proc. Symp. Fault Detection, Supervision andSafety for Technical Processes, SAFEPROCESS, Hull,UK, pp. 1183–1188.

Metenidis M.F., Witczak M. and Korbicz J. (2004): A novel ge-netic programming approach to non-linear system mod-elling: Application to the DAMADICS benchmark prob-lem. — Eng. Appl. Artif. Intell., Vol. 17, No. 4, pp. 363–370.

Michalewicz Z. (1996): Genetic Algorithms + Data Structures= Evolution Programs. — Berlin: Springer.

Milanese M., Norton J., Piet-Lahanier H. and Walter E. (Eds.)(1996): Bounding Approaches to System Identification. —New York: Plenum Press.

Miller J.A., Potter W.D., Gandham R.V. and Lapena C.N.(1993): An evaluation of local improvement operators forgenetic algorithms. — IEEE Trans. Syst. Man and Cybern.,Vol. 23, No. 5, pp. 1340–1351.

Mrugalski M. and Witczak M. (2002): Parameter estimation ofdynamic GMDH neural networks with the bounded-errortechnique. — J. Appl. Comput. Sci., Vol. 10, No. 1, pp. 77–90.

Narendra K.S. and Parthasarathy K. (1990): Identification andcontrol of dynamical systems using neural networks. —IEEE Trans. Neural Netw., Vol. 1, No. 1, pp. 1–27.

Patan K. and Parisini T. (2005): Identification of neural dynamicmodels for fault detection and isolation: the case of areal sugar evaporation process. — J. Proc. Contr., Vol. 15,No. 1, pp. 67–79.

Porter L.L. and Passino K.M. (1995): Genetic adaptive ob-servers. — Eng. Appl. Artif. Intell., Vol. 8, No. 3, pp. 261–269.

Advances in model-based fault diagnosis. . . 99

Ruano A.E. (Ed.) (2005): Intelligent Control Systems UsingComputational Intelligence Techniques. — London: TheIEE Press.

Sjoberg J., Zhang Q., Ljung L., Benveniste A., Delyon B., Glo-rennec P. Y., Hjalmarsson H. and Juditsky A. (1995): Non-linear black-box modeling in system identification: A uni-fied overview. — Automatica, Vol. 31, No. 12, pp. 1691–1724.

Sun R., Tsung F. and Qu L. (2004): Combining bootstrap andgenetic programming for feature discovery in diesel enginediagnosis. — Int. J. Ind. Eng., Vol. 11, No. 3, pp. 273–281.

Ucinski D. (2005): Optimal Measurements Methods for Distrib-uted Parameter System Identification. — Boca Raton, FL:CRC Press.

Walter E. and Pronzato L. (1997): Identification of ParametricModels from Experimental Data. — Berlin: Springer

Watton J. and Pham D.T. (1997): An artificial NN based ap-proach to fault diagnosis and classification of fluid powersystems. — J. Syst. Contr. Eng., Vol. 211, No. 4, pp. 307–317.

Weerasinghe M., Gomm J.B and Williams D. (1998): Neuralnetworks for fault diagnosis of a nuclear fuel processingplant at different operating points. — Contr. Eng. Pract.,Vol. 6, No. 2, pp. 281–289.

Witczak M., Obuchowicz A. and Korbicz J. (2002): Genetic pro-gramming based approaches to identification and fault di-agnosis of non-linear dynamic systems. — Int. J. Contr.,Vol. 75, No. 13, pp. 1012–1031.

Witczak M. (2003): Identification and Fault Detection of Non-linear Dynamic Systems. — Zielona Góra: University ofZielona Góra Press.

Witczak M. and Korbicz J. (2004): Observers and geneticprogramming in the identification and fault diagnosis ofnon-linear dynamic systems, In: Fault Diagnosis. Mod-els, Artificial Intelligence, Applications (Korbicz J., Kos-cielny J.M., Kowalczuk Z. and Cholewa W., Eds.). —Berlin: Springer.

Witczak M., Korbicz J., Mrugalski M. and Patton R.J. (2006):GMDH neural network-based approach to robust fault de-tection and its application to solve the DAMADICS bench-mark problem. — Contr. Eng. Pract., Vol. 14, No. 6,pp. 671–683.

Witczak M. and Pretki P. (2005): Designing neural-network-based fault detection systems with D-optimum experi-mental conditions. — Comput. Assist. Mech. Engi. Sci.,Vol. 12, No. 2, pp. 279–291.

Witczak M. (2006): Towards the training of feed-forward neuralnetworks with the D-optimum input sequence. — IEEETrans. Neural Netw., Vol. 17, No. 2, pp. 357–373.

White H. (1989): Learning in artificial neural networks: Astatistical perspective. — Neural Comput., Vol. 4, No. 4,pp. 305–318.

Yen G.G. and Lin K. (2000): Wavelet packet feature extractionfor vibration monitoring. — IEEE Trans. Ind. Electron.,Vol. 47, No. 3, pp. 650-667.

Zhang J., Ma J. and Yan Y. (2000): Assessing blockage of thesensing line in a differential-pressure flow sensor by usingthe wavelet transform of its output. — Meas. Sci. Technol.,Vol. 11, No. 3, pp. 178–184.

Zhao J., Chen B. and Shen J. (1998): Multidimensionalnonorthogonal wavelet-sigmoid basis function neural net-workfor dynamic process fault diagnosis. — Comput.Chem. Eng., Vol. 23, pp. 83–92.

Zolghardi A., Henry D. and Monision M. (1996): Design of non-linear observers for fault diagnosis. A case study. — Contr.Eng. Pract., Vol. 4, No. 11, pp. 1535–1544.

Received: 12 November 2005Revised: 2 February 2006