control of non-linear systems with unkno ics in the presence of disturbances

J. Maztik, A.A.H. Damen, S. Weiland, A.C.P.M. Bach Eindhoven University of Technology,

Measurement and Control Group, Department of Electrical Engineering, P.O. Box 513, 5600 MB Eindhoven, The Netherlands

Tel.: +31 40 473795, Fax: +31 40 434582, E-mail: J.Mazak@ele.tue.nl

Abstract

This paper deals with the design of a state tracking controller for a non-linear dynamical system where the dy- namics are only partly known and where only part of the state vector components are directly measured. The de- sign procedure takes the influence of state disturbances and measurement noise into account, which are naturally present in practical designs of control systems. The design of a non-linear controller is based on the estimation of a simulation model of the process in state space form. A non-linear filter gain is estimated to build up a state ob- server to be used for the optimization of a non-linear static state feedback controller. All non-linear maps are approx- imated by neural networks trained by a mixed stochastic- deterministic optimization procedure. The performance of the proposed design procedure is demonstrated by means of a simulation example.

1. Introduction

This paper addresses the state-tracking problem for a non-linear process whose states are corrupted by random disturbances. The measured output signals are also cor- rupted by noise. This sort of problems occurs in prac- tice when operating points of a process change while the changeover trajectory of the system is specified in advance.

In this approach we assume that the only available data for the controller design are the input and output measure- ments taken from the process. Besides this, we assume to have a priori knowledge about relations among some states of the process, e.g. one state is a derivative of another and we know how the states are mapped to the outputs. Never- theless, we allow for a set of completely unknown states to be represented by neural net configurations, that are only characterized by not having a direct influence on the out- puts (just via the previously indicated "physically" defined states).

The proposed state-tracking controller contains two components. The first component is the non-linear state observer producing an estimation of the state vector. An essential part of the observer design is the identification of

a non-linear state space simulation model of the given pro- cess. Based upon this model the previous output is used for minimizing the variance of the one-step ahead output prediction. The second component is the non-linear static state feedback gain which is designed such that a defined cost functional is minimized.

All the non-linearities in the control system are mod- eled by neural networks and the synaptic weights of the networks are optimized by a mixed stochastic-deterministic optimization procedure.

The problem of identification of non-linear process dy- namics corrupted by random disturbances using neural net- works was recently treated in [l]. In this paper we extend these results for a non-linear state-tracking controller de- sign considering a possibility that the full state vector is not available.

2. Problem Statement

Let a non-linear process P have the following discrete- time state-space representation

(1)

where x E X C R", U E U C Rm, y E Y C Bp are the state, control and output signals, respectively, w E W C R" and v E V C RJ' are respectively the state disturbance and measurement noise. Bounded sets X, U, W and V determine operating ranges of the non-linear pro- cess (1). The non-linear mapping h is fully known and is assumed to be invertible on the bounded set X. The non- linear mapping f is partially given and partially should be represented by a tuned neural network. It is assumed that f is continuous and differentiable on the set X. With respect to the previous knowledge the state is partitioned into two components x = ($), where x1 E R"1, 7 ~ 1 5 a. The first component X* represents the non-measured part of the state vector or those state components, which are left free for modeling purposes. The second component X' represents the measured part of the state vector or those states which are directly related to the measured states

4270

by an exactly known relation. The reason behind this con- struction is that in practice we measure physical quantities like position, flow, temperature, angles etc. and these have either fully or partly known relations to speed, accelera- tion, pressure, heat flow, rotation speed etc., while more complicated dynamics of the system under study may be completely hidden in an analytical description.

The disturbances w and w are mutually independent tic processes and supposed to be zero mean white

noises with unknown covariances. The instantaneous val- ues ofw(k) , w(k) are correlated neither with ~ ( k ) nor with

Let the cost function estimating the quality of the con- % ( I C ) .

trol be given by the following expectation

E { ( z l ( k ) - T(k))TWz(Z1(k9 - ?-(IC))

+ .(~)TWU.(k)) (2)

where W, and W,, are symmetric non-negative definite weighting matrices and T E R" is the reference signal to be tracked by the component z1 of the state vector. We will assume, that the initial state d ( 0 ) is known and given by E R"1. The weighting matrices must be chosen such, that U will remain in the set U. In the following we consider as the reference signal a bounded sequence of random steps to conform to switching phenomena.

Problem. Find an optimal state feed-back control strategy u ( k ) E U for IC = 0, . . . , N, such that the cost function (2) is minimized for a bounded set of step-like reference signals r ( k ) , k = 0, . . . , N .

The direct minimization of the cost function (2) is a very difficult task because of the non-convex stochastic character of this problem. The solution presented in this patper is based on the separation of a state estimation and au optimum control strategy calculation based on the state estimate. The simplified block diagram of the control loop is shown in Figure 1.

Figure 1: Considered closed loop system

The estimator is a Kalman like state space observer yielding an estimation of the state vector

% ( I C ) = E { Z ( k ) 1 u(k - 1), y(k - l), . . . } It contains the state space simulation model of the process and a non-linear static gain factor to correct for the influ- ence of the state disturbance. The identification of a state space simulation model for the process (1) can be done by

4271

a non-linear output error least squares estimation leading to the minimization of the following expectation

E{llh-'[d - m) (3) The design of the statetracking controller will be con-

sidered for multiple step bounded reference signals with a priori given upper bound 7 E Rn1 and lower bound - T E R"' . The availability of an estimated state dows us to calculate a non-linear static state feedback controller K of the following form

U@) = KIW, ml (4)

where the multivariate non-linear gain function K is cd- culated such that the cost function (2) i3 minimized b r a sufficiently long reference signal T. Sufficiency here means an accurate estimate of the non-linear controller gain K while for a linear system it would be sufficient to compute the controller for just one step giving global optimality of the controller. This is not true for non-linear systems and the whole state space X must be tested.

All the non-linear functional relations both in the ob- server and in the controller are approximated by multilayer neural networks [2] containing only feed-forward connec- tions. Required dynamics are included into the neural net- work by an outer feed-back of delayed outputs resulting in a recurrent neural network.

3. Identification of the process dynamics

The identification scheme used for estimation of the process dynamics is the output error identification shown in Figure 2. This allows us to obtain a simulation model of the process, i.e. it will predict the output of the process for a long time horizon.

The available input/output data measurements are de- noted by

ID, = {u(IC),y(IC)} for IC = I,. . . , N

where N is the length of the data set. The control in- put U is assumed to be a bounded white noise signal from the set U persistently exciting the process dynamics in the required ranges of the state space X. Let the model be parameterized by the following state space description

i ( k + 1) = F [ i ( k ) , ~ ( k ) , 0 ~ ] ( 5 )

$(k) = h[G1(k)] (6)

B = (;:) where F is the state evaluation map described by a static neural network including the known part of f in equation (1) and 0 F is the parameter vector consisting of all synaptic weights of the neural network. The neural network com- putes the next sample of 2 which is then fed back to the input of the neural network through a one step delay de- vice together with the process input sample. The output map h is known and defines the state space basis for the state component x1. The basis for states Bo, which are not

I Network

I

I I

I I

Y > +

Y U

'Model

Figure 2: State space identification set-up

measured directly, is left free. For these states the neural network will choose a basis during the optimization of the parameters of the neural network model.

For the chosen complexity of the neural network, the weights OF are estimated such, that the cost function

is minimized. This cost function represents the variance of the error between the uniquely defined states through mea- surements and the corresponding simulated states. The optimization problem stated above is a non-convex non- linear programming problem and therefore it is in general very difficult to find the best approximation in the sense that the global minimum of (7) is found. In practice we can use for the optimization some of the local gradient optimization techniques or a global stochastic search tech- nique. This will be more precisely described in one of the following sections.

The complexity of the model should be estimated using a validation data set Vv different from Ve. For different complexities of the neural network the value of the cost function (7) is computed on the validation data set and the model with the smallest cost is chosen as the best a p proximation of the process dynamics. This process is time consuming and difficult to perform in a systematic way, but prevents from over-parameterized models.

4. Non-linear state observer design

The state observer is used to reconstruct the state of the process in the situation where a random disturbance is acting on the process state components. The idea of designing a state observer is to use the actual output mea- surements y to improve the state estimation P resulting from the previously identified simulation model. Hypo- thetically an estimate $(k) of the state ~ ( k ) at the time instance k is a non-linear function of the state estimate at

time instance k - 1 and the error in the expected measured data $(k - 1). This approach is similar to the Kalman fil- ter for linear systems with minimum error variance of the process state estimates.

The equations of the non-linear estimator are proposed as follows

where G is some non-linear static gain of the observer again represented by a neural network. Note that the assumption of whiteness of v allows us to define a static G here. In case of colored noise on the states we would have to define extra states in G to describe this effect. Here, 6)G should be chosen such that the variance of the process state estimate is minimal. That is, we wish to minimize the expectation

E { ( Z ( k ) - ? ( k ) ) T ( Z ( k ) - ?(k))} (10)

where 5 denotes the true state vector explained in the b* sis of the identified simulation model ( 5 ) . The whole true state vector Z(k) is in general not available and the min- imization of the previous cost function can not be done directly. But we can still design a static gain factor G of the observer such that we further improve the vaxiance of the output error e ( k ) = y(k) - c(k) obtained during the identification of the model (5)-(6). The objective for the observer gain parameters calculation is the minimization of the cost function

while the parameters of the process model (see Figure 3).

remain fixed

5. Non-linear state-feedback design

The availability of estimated states allows us to design a static state-feedback controller to track the given reference signal. The controller is chosen static and non-linear and its non-linearity is parameterized by a neural network K as follows

u(k) = K[g(k) ,r(k) ,eK] (12)

where the parameters OK are chosen such that the following cost function is minimized

M N - d - 1

where d stands for a delay to be chosen with respect to system dynamics and ~j represents a single step of length N samples.

Recall, that the reference trajectory can only be spec- ified for the measured states z1 or for the states whose physical meaning is determined in advance by putting spe- cific restrictions to the non-linearity F in the state equation

4272

of the model (5). However, the rest of the state vector so is also used in the control action computation, but can not he tracked in a physical meaning.

Figure 3 shows the final control loop containing the identified model of the process, the non-linear filter gain ffactor G and the non-linear controller gain K .

Y >

II

Figure 3: Detailed scheme of the control loop

6. Neural network training

The estimation of the weighting parameters of the neu- ral network while minimizing a cost functions (7), (11) or (13) is a non-convex optimization problem and a global optimization algorithm should be used to avoid getting trapped in local minima. Our training procedure is a com- bination of a quasi-Newton algorithm [4] and a stochastic Optimization algorithm [5]. First, the stochastic search is performed for certain number of iterations and from the point with lowest cost the quasi-Newton optimization is initialized to find a local minimum. This still may not be a trivial task for a quasi-Newton algorithm, which often runs into numerical difficulties. In such a situation the stochas- tic search is again restarted for the specific number of iter- ations and the optimization is followed by a quasi-Newton algorithm. This procedure is repeated until a reasonable solution is found.

7. Numerical example

Let an example process be described by the following state space equations

so(k) 1 + zl(k)zl(k)

+ u(k) + wl(k) (14) so@+ 1) =

xo(k)xl(k) +wz(k) (15) 1 + s l (k ) s l (k )

Z ' (k+l ) = 1+

y(k) = d ( k ) + v ( t ) where w1, w2 and v are zero mean uniformly distributed noises with maximal amplitude 0.1. As a testing signal U we used a uniformly distributed noise, zero mean with maximum amplitude 1. For the identification of the non- linear model described in Section 3 we generated an es- timation data set and a validation data set both of 1000

input/output pairs { ~ ( k ) , y(k)} by simulation of the sys- tem equations (14)-(16).

Equation (16) is supposed to be known but equations (14) and (16) have been approximated by a one hidden layer network for which the number of nodes varied be- tween 2 and 8. For each configuration we estimated the neural network weights and checked the performance on the validation set. The results of these experiments are shown in Table 1 and a good choice for the complexity of the neural network model of the process dynamics could be a configuration with 4 hidden nodes.

Nn I Nw I ve I v v 2 I 14 I 1.585e-02 I 1.668e-02

44 8 50

Table 1: Output error identification results

The above estimated simulation model was then used to improve the variance of the state estimates by estimation of a non-linear filter gain G as described in Section 4. The non-linear gain of the filter was approximated by a one hid- den layer network containing 4 nodes and 22 weights. By optimization of the cost function of type (13) we reached the value 8.7699e-03 at which there was an increase of the cost for the validation data set. The optimal performance on the validation data set was found to be 1.1771e-02. Having a look at the system equations, we can not decrease the effect of noise v and very little of the effect of noise 202

which appears in fact directly in the output. The system has very short impulse responses and therefore the correc- tion for the noise 202 is a difficult task. We can correct the state estimates more easily for the noise wl, but note, that its effect is corrupted in the output by the noises w2 and v and therefore makes the filter design still difficult.

Figure 4a) shows the spectrum of the output error sig- nal for the identification together with the spectrum of the validation error. Because of the non-linear character of our problem we do not know what should be the exact spec- trum of the error signal. But we can compare the spectra of our identification results and observe certain similari- ties. In our case this was done for the configuration of the network with four hidden nodes.

From these plots we can also see the decreased per- formance for the validation data and therefore a better model could be possibly found. However, this effort can be too time consuming in practice. The filter design gave a flat spectrum of the output error signal shown in the Figure 4b), but it can very well be that in this case the filter takes care not only about the noise compensation, but also about the misfit in the simulation model estim* tion and fitting particular noise samples. The non-linear static state feedback controller was optimized for the esti-

4273

-:*a ii 4(&, 1%- -<-I

a) b)

igure 4: Error spectra: a) solid line - estimation output error; dashed line - validation output error b) filter validation error

: State tracking controller: solid line - state ref- erence signal; dashed line - a) controller opti- mization, b) controller validation; dotted line - control signal

mated simulation model in a noise free situation and then validated on the true description of the process given by (14-16) including noise. To approximate the non-linearity of the controller gain we have chosen the complexity of the network to consist of 4 hidden nodes and 21 weights. Other complexities of the neural network for the controller gave similar or worse performance. As reference signal we used a sequence of random steps uniformly distributed in the interval (0.5,1.5) of 500 samples. The amplitude was decreased because of the expected worse approximation of the system dynamics at the bounds of the testing signal used in the identification. The weighting factors w, and

were chosen as follows: W, = 1 and W, = 0.1. The value of W, was estimated from previous experiments to bound the amplitude of the control function. The noise in the system was the same as described in the beginning of this section. The delay factor d in (13) was chosen 2.

The results of the controller design are plotted in Fig- ure 5. The steady state error in the tracking of the state component x1 is due to the penalty we put on U to limit its range. However, from a practical point of view the control values can be unacceptably high and larger value of the control penalty is then required. The steady state errors where experimentally removed by including one state into the controller neural network and optimizing parameters of the following non-linear dynamic controller

simulating an integral action in the closed loop. The plot of these results shows similar controller behavior as shown

in Figure 5a) except that the steady state error is zero. The performance of the controller on the original example shown in Figure 5b) is still good compared to the one ob- tained during the controller estimation. The steady state errors are slightly increased for the reference signal values close to -0.5 possibly due to the worse approximation of the system dynamics in these ranges.

8. Conclusions

The numerical experiment showed all the difficulties in designing and computing a non-linear stochastic state tracking controller. The main difficulties arise in identi- fying a good simulation model of the! underlying process. This is a difficult task when data is corrupted by dis- turbances. The main difficulty lies im the limitations of the optimization procedure due to the non-convex char- acter of the underlying optimization problem and finite length of the data set used for identific:ation. The proposed stochastic-deterministic optimization procedure which re- quires an initialization from many initial points turns out to be a time consuming process. However, good results can be obtained when the performance of the estimated model is always evaluated on the validation data set. In this way we can avoid the fitting of the noise in the data and the over-parameterization of the models. This procedure gives also a reasonable neural network complexity for the simu- lation model. The non-linear Kalman-like filter design can be used for the reconstruction of non-measurable states and can improve the variance of the state estimation in presence of the state or measurement disturbances.

The non-linear state feedback controller was able to fol- low the given reference signal with good dynamic response. The steady state errors could be removed by introducing an integral action into the control loop by means of dynamic state feed-back controller. The validation of the controller on the real plant, in general, results in a worse performance caused by modeling errors.

References [I] 0. Nerrand, P. Roussel-Ragot, D. Wrbani, L. Per- sonnaz, and Dreyfus G. Training recurent neural networks: Why and how? an illustration in dynamical process mod- eling. IEEE 13-ans. on Neural Networks, 5(2):178-184, 1994. [2] K. S. Narendra and K. Parthasatrathy. Identification and control of dynamical systems using neural networks. IEEE Trans. on Neural Networks, 1(1):4-27, 1990. [3] Robert Hecht-Nielsen. Neurocomputing. Amster- dam: Addison-Westley, 1990. 141 The Numerical Algorithms Group Limited. The NAG Fortran Library Manual, July 1988. [5] H. J.M. Telkamp and A.A.H. Damen. Neural network learning in nonlinear system identification and control de- sign. In Proc. of the European Contnol Conf., Groningen, The Netherlands, 1993.

4274