multivariate controller performance analysis: methods, … · 2010. 7. 15. · multivariate...

Multivariate Controller Performance Analysis:Methods, Applications and Challenges

Sirish L. Shah∗, Rohit Patwardhan† and Biao HuangDepartment of Chemical and Materials Engineering

University of AlbertaEdmonton, Canada T6G 2G6

AbstractThis paper provides a tutorial introduction to the role of the time-delay or the interactor matrix in multivariate minimumvariance control. Minimum variance control gives the lowest achievable output variance and thus serves as a usefulbenchmark for performance assessment. One of the major drawbacks of the multivariate minimum variance benchmark isthe need for a priori knowledge of the multivariate time-delay matrix. A graphical method of multivariate performanceassessment known as the Normalized Multivariate Impulse Response (NMIR), that does not require knowledge of theinteractor, is proposed in this paper. The use of NMIR as a performance assessment tool is illustrated by applicationto two multivariate controllers. Two additional performance benchmarks are introduced as alternatives to the minimumvariance benchmark, and their application is illustrated by a simulated example. A detailed performance evaluation ofan industrial MPC controller is presented. The diagnosis steps in identifying the cause of poor performance, e.g. as dueto model-plant mismatch, are illustrated on the same industrial case study.

KeywordsMultivariate minimum variance control, Time delay, Normalized multivariate impulse response, Model predictive control,Model-plant mismatch

Introduction

The area of performance assessment is concerned withthe analysis of existing controllers. Performance assess-ment aims at evaluating controller performance fromroutine data. The field of controller performance as-sessment stems from the need for optimal operation ofprocess units and from the need of getting value fromimmense volumes of archived process data. The fieldhas matured to the point where several commercial al-gorithms and/or vendor services are available for processperformance auditing or monitoring.

Conventionally the performance estimation procedureinvolves comparison of the existing controller with a the-oretical benchmark such as the minimum variance con-troller (MVC). Harris (1989) and co-workers (1992; 1993)laid the theoretical foundations for performance assess-ment of single loop controllers from routine operatingdata. Time series analysis of the output error was usedto determine the minimum variance control for the pro-cess. A comparison of the output variance term withthe minimum achievable variance reveals how well thecontroller is doing currently. Subsequently Huang et al.(1996; 1997) and Harris et al. (1996) extended this ideato the multivariate case. In contrast to the minimumvariance benchmark, Kozub and Garcia (1993), Kozub(1997) and Swanda and Seborg (1999) have proposeduser defined benchmarks based on settling times, risetimes, etc. Their work presents a more practical methodof assessing controller performance. A suitable referencesettling time or rise time for a process can often be cho-sen based on process knowledge.∗[email protected]†Matrikon Consulting Inc., Suite 1800, 10405 Jasper Avenue,

Edmonton, Canada , T5J 3N4

The increasing acceptance of the idea of process andperformance monitoring has also grown from the aware-ness that control software, and therefore the applica-tions that arise from it, should be treated as capitalassets and thus maintained, monitored and revisited rou-tinely. Routine monitoring of controller performance en-sures optimal operation of regulatory control layers andthe higher level advanced process control (APC) appli-cations. Model predictive control (MPC) is currentlythe main vehicle for implementing the higher level APClayer. The APC algorithms include a class of modelbased controllers which compute future control actionsby minimizing a performance objective function over a fi-nite prediction horizon. This family of controllers is trulymultivariate in nature and has the ability to run the pro-cess close to its limits. It is for the above reasons thatMPC has been widely accepted by the process indus-try. Various commercial versions of MPC have becomethe norm in industry for processes where interactionsare of foremost importance and constraints have to betaken into account. Most commercial MPC controllersalso include a linear programming stage that deals withsteady-state optimization and constraint management.A schematic of a mature and advanced process controlplatform is shown in Figure 1. It is important to notethat the bottom regulatory layer consisting mainly ofPID loops forms the typical foundation of such a plat-form followed by the MPC layer. If the bottom layerdoes not perform and is not maintained regularly thenit is futile to implement advanced control. In the samevein, if the MPC layer does not perform then the benefitsof the higher level optimization layer, that may includereal-time optimization, will not accrue.

The main contribution of this paper is in its general-

190

Multivariate Controller Performance Analysis: Methods, Applications and Challenges 191

Planning and Scheduling

Optimization

Model PredictiveControl

PID Loops

Days

Hours

Minutes

Seconds

Optimization


PID Loops

Subsystem 1 Subsystem 2

Planning and Scheduling

Optimization


PID Loops

Days

Hours

Minutes

Seconds

Optimization


PID Loops

Subsystem 1 Subsystem 2

Figure 1: The control hierarchy.

ization of the univariate impulse response (between theprocess output and the whitened disturbance variable)plot to the multivariate case as the ‘Normalized Mul-tivariate Impulse Response’ plot. A particular form ofthis plot, that does not require knowledge of the pro-cess time-delay matrix, is proposed here. Such a plotprovides a graphical measure of the multivariate con-troller performance in terms of settling time, decay ratesetc. Simple time and frequency domain measures suchas multivariate autocorrelation and spectral plots areused to illustrate the interactions arising in multivari-able control systems. Two relatively new multivariateperformance evaluation ideas are also explored in de-tail: (1) the use of LQG as a benchmark based on theknowledge of the open loop process and noise modelsfor the soft-constrained performance assessment problem(Huang and Shah, 1999) and (2) the use of the designperformance as a benchmark (Patwardhan, 1999; Pat-wardhan et al., 2001). Both of these benchmarks can beapplied to any type of controller. The LQG benchmarkapplies to all class of linear controllers, irrespective of thecontroller objective function, and is of use when inputand/or output variance is of concern. The LQG bench-mark represents the ‘limits of performance’ for a linearsystem, is more general and has the minimum varianceas a special case. However, it needs a model of the linearprocess. The design objective function based approachcan be applied to constrained MPC type controllers andis therefore a practical measure. However, it does nottell you how close the performance is relative to the low-est achievable limits. Issues related to the diagnosis ofpoor performance are discussed in the context of MPCcontrollers. Performance assessment of the general MPCis as yet an unresolved issue and presents a challengingresearch problem. A constrained MPC type controlleris essentially a nonlinear controller, especially when op-erating at the constraints. Conventional MVC or lin-ear controller benchmarking is infeasible and alternative

techniques have to be developed. The development ofnew MPC performance monitoring tools thus representsan area of future challenges. The challenges associatedwith MPC performance evaluation are illustrated by con-sidering an industrial case study of a 7 × 6 problem.

This paper is organized as follows. The next sectionprovides a tutorial introduction to the concept of thetime-delay matrix or the interactor. This is an impor-tant entity, particularly if one wants to evaluate MPCperformance using multivariate minimum variance as abenchmark. The following two sections, respectively, dis-cuss the tools required in the analysis of multivariatecontrol loops such as the normalized multivariate im-pulse response, and alternative benchmarks for multi-variate performance assessment. Applications are usedto demonstrate the proposed techniques in each section.A discussion on the challenges in performance analysisand diagnosis and issues in MPC performance evalua-tion are outlined in the penultimate section, followed bya detailed industrial case study of an industrial MPCevaluation.

The Role of Delays for Univariate andMultivariate Processes

Time delays play a crucial role in performance assess-ment particularly when the minimum variance bench-mark is used. The concept of the multivariate delay isexplained below in a tutorial manner by first defining theunivariate delay term and then generalizing this notionto the multivariate case.

Definition of a Delay Term for a Univariate Pro-cess:

The time-delay element in a univariate case is character-ized by several different properties. For example, it rep-resents the order of the first, non-zero (or non-singular)impulse response coefficient (also characterized by thenumber of infinite zeros of the numerator portion of thetransfer function). It is important to fully understandthe definition of a delay term for the univariate case inorder to generalize the notion to a multivariate system.From a system theoretic point, the delay for a univariatesystem is characterized by the properties listed below.Consider a plant with the discrete transfer function oran impulse response model given by:

G(q−1) =q−dB(q−1)

A(q−1)

= 0q−1 + 0q−1 + · · · + 0q−d+1

+ hdq−d + hd+1q

−d−1 + hd+2q−d−2

+ · · ·

The delay term for such a univariate system is definedby:

192 Sirish L. Shah, Rohit Patwardhan and Biao Huang

• the minimum integer r such that

limq−1→0

qr

(q−dB(q−1)

A(q−1)

)= k 6= 0

(i.e. a non-singular coefficient) which in the caseconsidered above, for r = d gives:

limq−1→0

qr

(q−dB(q−1)

A(q−1)

)= hd 6= 0

Note that for the univariate case, the number ofinfinite zeros of the process as obtained by settingthe numerator of the process transfer function tozero, i.e. q−dB(q−1) = 0, also yields d infinite zerosand n finite zeros given by the roots of B(q−1) = 0.

• the static or steady-state value of the delay termshould be equal to 1, i.e. at steady-state (whenq−1 = 1), q−d = 1.

Definition of a Delay Matrix for a MultivariateProcess:

Analogous to the univariate case, it is possible to factor-ize the open-loop transfer matrix into two elements: thedelay matrix, D(q−1)−1 containing all the infinite zerosof the system, and the ‘delay-free’ transfer-function ma-trix, T ∗(q−1), containing all the finite zeros and poles.

T (q−1) = D(q−1)−1 · T ∗(q−1)

= H1q−1 + H2q

−2 + · · ·+ Hdq

−d + Hd+1q−d−1 + · · ·

where Hi are the impulse response or Markov matricesof the system parallel to the definition of the univariatedelay, the multivariate delay matrix is defined by:

• Fewest number of linear combinations of the impulseresponse matrices that give a nonsingular matrix,i.e. (a finite and nonsingular matrix)

limq−1→0

D(q−1)(D(q−1)−1 · T ∗(q−1)) = K

(a finite and nonsingular matrix)

Unlike the univariate case, a nonzero Hi may notnecessarily indicate the delay order. Instead, for themultivariate case, it is the fewest linear combinationof such non-zero Hi to give a non-singluar K thatdefines the delay matrix, D(q−1)−1. Applying thisidea to the univariate case will reveal that K = hd,which is the first or leading non-zero coefficient ofthe impulse response or the Markov parameter ofthe scalar system. Such an interpretation makes thechoice of D(q−1)−1, as a multivariate generalizationof the univariate delay term, a very meaningful one.Note that det(D(q)) = cqm, where m is the numberof infinite zeros of the system and c is a constant.

For the multivariate case the number of infinite zerosmay not be related to the order d of the time-delaymatrix.

• DT (q−1)D(q) = I (As compared to the univariatecase where q−dq = 1). This is known as the uni-tary interactor matrix. This unitary property pre-serves the spectrum of the signal, which leads usto the result that the variance of the actual out-put and the interactor filtered outputs are the same,i.e. E(Y T

t Yt) = E(Y Tt Yt), where Yt = q−dDYt (see

Huang and Shah, 1999).

Example. Consider the following transfer functionmatrix and its impulse response or Markov parametermodel:

T (q−1) =

q−2

1 − q−1

q−3

1 − 2q−1

q−2

1 − 3q−1

q−3

1 − 4q−1

=[

0 00 0

]q−1 +

[1 01 0

]q−2

+[

1 1.33 1

]q−3 +

[1 0.5

0.109 0.25

]q−4

+ · · ·

Note that even though H2 6= 0, a linear combination ofH1 and H2 does not yield a nonsingular matrix. In thisexample, at least three impulse response matrices arerequired to define the delay matrix for this system. Thedelay matrix that satisfies the properties listed above, isgiven by:

D(q−1)−1 =[−0.707q−2 −0.707q−3

−0.707q−2 0.707q−3

]The order of the delay is 3, i.e. a linear combination ofat least 3 impulse response matrices is required to havea non-singular K.

Remark 1. The interactor matrix D(q) can be oneof the three forms as described in the sequel. If D(q) isof the form: D(q) = qdI, then the process is regardedas having a simple interactor matrix. If D(q) is a di-agonal matrix, i.e., D(q) = diag(qd1 , qd2 , . . . , qdn), thenthe process is regarded as having a diagonal interactormatrix. Otherwise the process is considered to have ageneral interactor matrix.

To factor the general interactor matrix, one needs tohave a complete knowledge of the process transfer func-tion or at least the first few Markov matrices of the mul-tivariate system. This is currently the main drawback inusing this procedure. Huang et al. (1997) have provideda closed loop identification algorithm to estimate the firstfew Markov parameters of the multivariate system andthus compute the unitary interactor matrix. However,


this rank determination procedure is prone to errors asit requires one to check if a linear combination of ma-trices is of full rank or not. Ko and Edgar (2000) havealso proposed the use of the first few Markov matricesfor multivariate performance assessment based on theminimum variance benchmark. The factorization of thediagonal interactor matrix requires only time delays be-tween each pair of the input and output variables. Adiagonal interactor matrix by no means implies that theprocess has a diagonal transfer function matrix or thatthe process has no interaction. But the converse is true,i.e. a diagonal process transfer function matrix or a sys-tem with weakly interacting multivariate loops (a diago-nally dominant system) does have a diagonal interactormatrix. In fact, experience has shown that many ac-tual multivariable processes have the structure of the di-agonal interactor, provided the input-output structuringhas been done with proper engineering insight. This factgreatly simplifies performance assessment of the multi-variate system.

The Multivariate Minimum VarianceBenchmark

In a univariate system, the first d impulse response coef-ficients of the closed loop transfer function between thecontrol error and the white noise disturbance term deter-mine the minimum variance or the lowest achievable per-formance. In the same way, the first d impulse responsematrices of the closed loop multivariate system are use-ful in determining the multivariate minimum variancebenchmark, where d denotes the order of the interactor.

Performance assessment of univariate control loops iscarried out, by comparing the actual output variancewith the minimum achievable variance. The latter termis estimated by simple time series analysis of routineclosed-loop operating data and knowledge of the pro-cess time delay. The estimation of the univariate min-imum variance benchmark requires filtering and corre-lation analysis. This idea has been extended to mul-tivariate control loop performance assessment and themultivariate filtering and correlation (FCOR) analysisalgorithm has been developed as a natural extension ofthe univariate case (Huang et al., 1996, 1997; Huang andShah, 1999). Harris et al. (1996) have also proposed amultivariate extension to their univariate performanceassessment algorithm. Their extension requires a spec-tral factorization routine to compute the delay free partof the multivariate process and thus estimate the multi-variate minimum variance or the lowest achievable vari-ance for the process. The FCOR algortihm of Huanget al. (1996), on the other hand, is a term for term gen-eralization of the univariate case to the multivariate caseand also requires the knowledge of the multivariate time-delay or interactor matrix. Figure 2 summarizes thesteps required in computing the mutivariate performance

index. A quadratic measure of multivariate control loopperformance is defined as:

J = E(Yt − Y spt )T (Yt − Y sp

t )

(where Yt represnts an n dimensional output vector).Thelower bound or the quadratic measure of the multivariatecontrol performance under minimum variance control isdefined as

Jmin = E(Yt − Y spt )T (Yt − Y sp

t )|mv

It has been shown by Huang et al. (1997) that thelower bound of the performance measure Jmincan be es-timated from routine operating data. In Huang et al.(1997), the multivariate performance index is defined as

η =Jmin

J

and is bounded by 0 ≤ η ≤ 1. In practice, one mayalso be interested in knowing how each individual out-put (loop) of the multivariate system performs relativeto multivariate minimum variance control. Performanceindices of each individual output are defined as ηY1

...ηYn

=

min(σ2y1

)/σ2y1

...min(σ2

yn)/σ2

yn

= diag(ΣmvΣ−1Y )

where Σmv = diag(Σmv) and ΣY = diag(ΣY ); ΣY is thevariance matrix of the output Yt and Σmv = min(ΣY )is the covariance matrix of the output Yt under multi-variate minimum variance control. It has been shown byHuang et al. (1997) that Σmv can also be estimated usingroutine operating data, and knowledge of the interactormatrix.

To summarize, a multivariate performance index is asingle scalar measure of multivariate control loop perfor-mance relative to multivariate minimum variance con-trol. Individual output performance indices indicate per-formance of each output relative to the loop’s perfor-mance under multivariate minimum variance control. Ifa particular output index is smaller than other outputindices, then some of the other loops may have to bede-tuned in order to improve this poorly tuned loop.

Alternative Methods for PerformanceAnalysis of Multivariate Control Systems

Autocorrelation Function:

The autocorrelation function (ACF) plots may be used toanalyze individual process variable performance. A typi-cal example of the ACF plots for the two output variablesof the simulated Wood-Berry (Wood and Berry, 1973)column control system is shown in Figure 3. A decen-tralized control system comprising two PI controllers was


Figure 2: Schematic diagram of the multivariate FCOR algorithm.

used on the Wood-Berry column. The diagonal plots areautocorrelations of each output variable, while the off-diagonal plots are cross-correlation plots. The diagonalplots typically indicate how well each loop is tuned. Forexample, a slowly decaying autocorrelation function im-plies an under-tuned loop, and an oscillatory ACF typi-cally implies an over-tuned loop. Off-diagonal plots canbe used to trace the source of disturbance or the inter-action between each process variables. Figure 3 clearlyindicates that the first loop has relatively poor perfor-mance while the second loop has very fast decay dy-namics and thus good performance. Interaction betweenthe two loops can also be observed from the off-diagonalsubplots. Note that the autocorrelation plot of the mul-tivariate system is not necessarily symmetric.

Spectral Analysis:

Frequency domain plots provide alternative indicators ofcontrol loop performance. They may be used to assessindividual output dynamic behavior, interactions and ef-fects of disturbances. For example, peaks in the diagonalplots typically imply oscillation of the variables due toan over-tuned controller or presence of oscillatory distur-bances. Frequency domain plots also provide informa-tion on the frequency ranges over which the oscillationsoccur and the amplitude of the oscillations. Like timedomain analysis, off-diagonal plots provide one with in-formation on the correlation or interaction between theloops. Figure 4 is the power spectrum and the cross-power spectrum plot of the simulated Wood-Berry col-umn. The first diagonal plot indicates that there is aclear mid-frequency harmonic in the 1st output. Thiscould be due to an overtuned controller. Off-diagonalplots show a peak in the cross-spectrum at the same fre-quency. The poor performance of loop 1 can then beattributed to significant interaction effects from loop 2to loop 1. In other words, the satisfactory or good perfor-mance of loop 2 could be at the expense of transmitting

0.0

0.5

1.0

0 5 10 15 20 25 30 35 40

Auto Correlation of Tag1.pv

AC

F

Lag

-0.10

-0.05

0.00

0.05

0.10

0.15

0 5 10 15 20 25 30 35 40

Cross-Correlation: Tag1.pv vs. Tag2.pv

CC

F

Lag

-0.4

-0.3

-0.2

-0.1

0.0

0.1

0.2

0 5 10 15 20 25 30 35 40

Cross-Correlation: Tag2.pv vs. Tag1.pv

CC

F

Lag

-0.10.00.1

0.20.30.40.50.6

0.70.80.91.0

0 5 10 15 20 25 30 35 40

Auto Correlation of Tag2.pv

AC

FLag

Figure 3: Autocorrelation function of multivariateprocess.

disturbances or upsets to loop 1 via the interaction term..The power spectrum plots of a multivariate system aresymmetric.

Normalized Multivariate Impulse Response(NMIR) Curve as an Alternative Measure ofPerformance:

As shown in Figure 2, the evaluation of the multivari-ate controller performance has to be undertaken on theinteractor filtered output and not on the actual output.The reason for this is that the interactor filtered outputvector, Yt = q−dDYt, is a special linear combination ofthe actual output, lagged or otherwise, and this fictitiousoutput preserves the spectral property of the system andfacilitates simpler analysis of the multivariate minimumvariance benchmark.

This new output ensures, as in the univariate case,that the closed loop output can be easily factored intotwo terms, a controller or feedback-invariant term anda second term that depends on the controller parame-ters. In the ensuing discussion, we consider an alter-


0

25

50

75

100

125

150

175

10 -2 10 -1

Power Spectrum of Tag1.pv

Mag

nitu

de

Frequency

-10

-5

0

5

10

10 -2 10 -1

Cross Spectrum : Tag2.pv vs. Tag1.pv

Mag

nitu

de

Frequency

5.0

7.5

10.0

12.5

15.0

10 -2 10 -1


Mag

nitu

de

Frequency

0

25

50

75

100

125

150

175

10 -2 10 -1


Mag

nitu

de

Frequency

-10

-5

0

5

10

10 -2 10 -1


Mag

nitu

de

Frequency

5.0

7.5

10.0

12.5

15.0

10 -2 10 -1


Mag

nitu

de

Frequency

0

25

50

75

100

125

150

175

10 -2 10 -1


Mag

nitu

de

Frequency

0

25

50

75

100

125

150

175

10 -2 10 -1


Mag

nitu

de

Frequency

-10

-5

0

5

10

10 -2 10 -1


Mag

nitu

de

Frequency

-10

-5

0

5

10

10 -2 10 -1


Mag

nitu

de

Frequency

5.0

7.5

10.0

12.5

15.0

10 -2 10 -1


Mag

nitu

de

Frequency

Figure 4: Frequencey response of multivariate pro-cess.

native graphical measure of multivariate performance asobtained from the interactor filtered output. So unlessspecified otherwise, the reader should assume that theoperations elucidated below are on the interactor filteredoutput, Yt.

An impulse response curve represents dynamic rela-tionship between the whitened disturbance and the pro-cess output. This curve typically reflects how well thecontroller regulates stochastic disturbances. In the uni-variate case, the first d impulse response coefficients arefeedback controller invariant, where d is the process time-delay. Therefore, if the loop is under minimum vari-ance control, the impulse response coefficients should bezero after d − 1 lags. The Normalized Multivariate Im-pulse Response (NMIR) curve reflects this idea. Thefirst d NMIR coefficients are feedback controller invari-ant, where d is the order of the time-delay matrix or theinteractor. If the loop is under multivariate minimumvariance control, then the NMIR coefficients should de-cay to zero after d − 1 lags. The sum of squares underthe NMIR curve is equivalent to the trace of the covari-ance matrix of the data. In fact the NMIR is a graphicalrepresentation of the quadratic measure of the outputvariance as given by:

E(Y Tt Yt) = E(Y T

t Yt)

= tr(F0ΣaFT0 ) + tr(F1ΣaFT

1 ) + · · ·

where

Yt = F0at + F1at−1 + · · · + Fd−1at−d+1 + Fdat−d + · · ·

is an infinite series impulse response model of the inter-actor filtered output with respect to the whitened distur-bance and matrices Fi represent the estimated Markovmatrices of the closed loop multivariate system. In thenew measure, the first NMIR coefficient is given by√

tr(F0ΣaFT0 ), the second NMIR coefficient is given by

√tr(F1ΣaFT

1 ), and so on. The multivariate performanceindex is then equal to the ratio of the sum of the squaresof the first d NMIR coefficients to the sum of squares ofall NMIR coefficients (see top plot in Figure 5). Care hasto be taken when interpreting the normalized impulse re-sponse curve. The NMIR represents a compressed scalarmetric for a multi-dimensional system. It is a graphi-cal representation of the weighted 2-norm multivariateimpulse response matrix and provides a graphical inter-pretation of the multivariate performance index in muchthe same way as the univariate impulse response givesan indication of the level of damping afforded to a unitimpulse disturbance.

The NMIR as outlined above and first described byHuang and Shah (1999) requires a priori knowledge ofthe interactor matrix. Since this NMIR curve is suitablefor obtaining a graphical measure of the overall closed-loop response, we suggest an alternative measure thatdoes not require knowledge of the interactor. We proposeto use a similar normalized multivariate impulse curvewithout interactor filtering to serve a similar purpose.The NMIR without interactor filtering is calculated asbefore by computing the correlation coefficients betweenthe pre-whitened disturbance and the actual output withlags 0, 1, 2, . . . , d − 1, d, d + 1, . . ..

E(Y Tt Yt) = tr(E0ΣaET

0 ) + tr(E1ΣaET1 ) + · · ·

where

Yt = E0at + E1at−1 + · · · + Ed−1at−d+1 + Edat−d + · · ·

Note that unlike the original NMIR measure as pro-posed by Huang and Shah (1999), the new measure pro-posed here does not require interactor filtering of theoutput, i.e. an explicit knowledge of the interactor isnot required in computing the new graphical and quali-tative measure. From here onwards this new measure isdenoted as NMIRwof .

In the new measure, the first NMIRwof coefficient isgiven by

√tr(E0ΣaET

0 ), the second NMIRwof coefficientis given by

√tr(E1ΣaET

1 ), and so on. Note that theNMIRwof measure (without interactor filtering) is sim-ilar to NMIR with interactor filtering in the sense thatboth represent the closed-loop infinite series impulse re-sponse model of the output with respect to the whiteneddisturbance, one for the actual output and the other onefor the interactor filtered output respectively. The newlyproposed NMIRwof is physically interpretable, but doesnot have the property that the first d coefficients arefeedback control-invariant. The main rationale for usingthe newly proposed NMIRwof is that the following twoterms are asymptotically equal:

limn→∞

{tr(E0ΣaET

0 ) + tr(E1ΣaET1 ) + · · · + tr(EnΣaET

n )}

={tr(F0ΣaF T

0 ) + tr(F1ΣaF T1 ) + · · · + tr(FnΣaF T

n )}


This follows from the equality: E(Y Tt Yt) = E(Y T

t Yt)(Huang and Shah, 1999). It is then clear that the NMIRcurves with and without filtering will coincide with eachother for n sufficiently large. Alternately, the cumulativesum of the square of the impulse response coefficients canalso be plotted and as per the above asymptotic equality,one would expect that the two terms will coincide for asufficiently large n. These curves are reproduced here forthe illustrative Wood-Berry column example. The ordi-nate in the bottom plot in Figure 5 gives the actual out-put variance when the curves converge for a sufficientlylarge n. Note that, unlike the NMIR curve with interac-tor filtering (solid line in Figure 5), the NMIRwof curve(dashed line in Figure 5) can not be used to calculatea numerical value of the performance index. However,it has the following important properties that are usefulfor assessment of multivariate processes:

1. The new NMIRwof represents the normalized im-pulse response from the white noise to the true out-put.

2. The new NMIRwof curve reflects the predictabilityof the disturbance in the original output. If theimpulse response decays slowly, then this is clear in-dication of a highly predictable disturbance (e.g. anintegrated white noise type disturbance) and rela-tively poor control. On the other hand a fast decay-ing impulse response is a clear graphical indicationof a well-tuned multivariate system (Thornhill et al.,1999).

3. The new NMIRwof also provides a graphical mea-sure of the overall multivariate system performancewith information regarding settling time, oscillation,speed of response etc.

NMIRs with and without interactor filtering are calcu-lated for the simulated Wood-Berry distillation columnwith two multiloop PI controllers. With sampling period0.5 second, the interactor matrix of the process is foundto have a diagonal structure and is given by

D(q) =[

q3 00 q7

]Since the order of the interactor is 7 sample units, thefirst 7 NMIR coefficients are feedback control invariantand depend solely on the disturbance dynamics and theinteractor matrix. The sum of squares of these 7 coef-ficients is the variance achieved under multivariate min-imum variance control. In fact as shown in Figure 6,the scalar multivariate measure of performance is equalto the sum of squares of the first 7 NMIR coefficientsdivided by the total sum of squares. Notice that for suf-ficiently large n or prediction horizon, the two curves,as expected, coincide with each other. In this simula-tion example, observe that the NMIR and the NMIRwof

curves decay to zero relatively quickly after 7 sampleunits, indicating relatively good control performance.

Industrial MIMO Case study 1: CapacitanceDrum Control Loops at Syncrude Canada Ltd.Capacitance drum control loops of Plant AB in Syn-crude Canada Ltd. were analyzed for this study. Theprimary objective of Plant AB is to further reduce thewater in the (Plant A) diluted bitumen product priorto it reaching the next plant (Plant B) storage tanks.As the grade of the feed entering plant A reduces, thewater required to process the oilsands increases propor-tionately. A large portion of this excess water ends upin the Plant A froth feed tank and ultimately increasesthe % volume of water in the Plant A product. Asidefrom degrading the quality of the product, the increasedvolume of water means reduction in the amount of bi-tumen that can be piped to the diluted bitumen tanks.In addition, the higher water content means more of thechloride compound present in the oilsands is dissolvedand finds its way to the diluted bitumen tanks and even-tually to plant B. The higher chloride concentration in-creases the corrosion rate of equipment in the Upgradingunits. Plant AB was developed as a means of reducingthe water content, and ultimately, the chlorides sent toUpgrading. This reduction is achieved by centrifugingthe Plant A Product.

All product from plant A is directed to the Plant ABfeed storage tank. The IPS portion of the product isrouted through 5 Cuno Filters prior to entering the PlantAB feed storage tank. Feed from the feed storage tank isthen pumped through the feed pumps to the Alfa Lavalcentrifuges. The Alfa Laval centrifuges remove water anda small amount of solids from the feed. Each centrifugehas its own capacitance drum and product back pressurevalve. This arrangement allows for individual centrifuge“E-Line” control and a greatly improved product quality.Heavy phase water from plant A is used as Process Waterin plant AB.

The cap drum pressure controller controls the capaci-tance drum pressure by adjusting the nitrogen flow intothe drum. The Cap Drum Primary level controller main-tains the cap drum water level by adjusting water ad-dition into the drum. Control of these two variablesis essential to maintain the E-Line in the centrifuges.Currently these two loops are controlled by multiloopPID controllers. The two process variables, pressure andlevel, are highly interacting. The objective of the perfor-mance assessment is to evaluate the existing multiloopPID controllers’ performance, and to identify opportuni-ties, if any, to improve performance by implementing amultivariate controller

Discussion of Performance Analysis. Process datawith a 5-second sampling interval are shown in Figure 7 .These are typical (representative) process data encoun-


0 5 10 15 20 25 30 35 40 45 500

1

2

3

4

time (samples)

NM

IRNormalized Impulse Response Curve

with fi lteringwithout filtering

0 5 10 15 20 25 30 35 40 45 505

10

15

20

25

30

Time (samples)

Cum

ulat

ive

sum

of s

quar

es

Cumulative sum of squares versus sample time

with filtering

actual process variancewithout filtering

Figure 5: NMIR and NMIRwof curves (top plot) andthe cumulative sum of squares plots of the impulseresponse coefficients (bottom plot).

1

0.0

0.5

1.0

1.5

2.0

2.5

Impu

lse

Res

pons

e, f

i

Lag

5 10 15 20 25 30 35 40 45 50

∑+∑

∑= ∞

+==

=

1

2

1

2

1

2

dii

d

ii

d

ii

ff

fη

Interactor order

Figure 6: Normalized multivariate impulse response.

tered in this process. By assuming that both pressureand level loops have no time delay except for the delayinduced by the zero-order-hold device, a scalar multi-variate performance index was calculated as 0.022 andindividual output indices are shown in Figure 8 . Basedon these indices, one may conclude that controller per-formance is poor and may be improved significantly byre-tuning the existing controllers or re-designing a multi-variate controller. However, since the exact time delaysfor these loops are unknown, further analysis of perfor-mance in both time domain and frequency domain isnecessary. For example, the NMIRwof response shownin Figure 9 does indicate that the disturbance persistsfor about 50 samples before it is compensated by thefeedback controllers. Overall the decay in the NMIRwof

curve is rather slow indicating a predictable disturbanceand generally ineffective regulatory control in dealingwith such disturbances. This is equivalent to a settlingtime of 4 minutes for the overall system. To check whichloop causes such long settling time, one can look at theauto- and cross-correlation plots.

The individual loop behavior can be observed from theauto and cross correlation plots shown in Figure 10 . Itis observed that the pressure response does not settledown even after 40 samples. This is clearly unaccept-able for a pressure loop. In addition some oscillatory re-sponse is observed in the level response as evident fromthe spectrum plot shown in Figure 11. Notice that thepeak (oscillation) appears in both the pressure and levelresponses as well as in the cross spectrum plot. This in-dicates that both loops interact and oscillate at the samefrequency. Thus, this analysis indicates that 1) the exist-ing multiloop controller has relatively poor performanceprimarily due to the long settling time and oscillatorybehavior or presence of oscillatory disturbances; 2) thetwo loops are strongly interacting and a multivariate con-troller may be able to improve performance significantly.

The final recommendation for this system was thatperformance of the two loops individually as well as amultivariate system is relatively poor. For predictabledisturbances, there is insufficient integral action in thepressure loop resulting in a slowly decaying ACF plot asnoticeable in the top left hand plot in Figure 10 . Theperformance is poor mainly due to interaction betweenthe two loops. Because of the interaction, the multiloopretuning exercise may be futile. If however only a simplecontrol solution is desired then the level loop can be de-tuned and the pressure loop can have larger gains withsmaller integral action to reduce oscillations. If the sys-tem warrants, then a multivariate control loop could bedesigned.

As would be evident from the above discussion andcase study, there remains much to be desired in obtainingpractically meaningful measures of multivariate controlperformance. The minimum variance control is a usefulbenchmark as it requires little a priori information aboutthe process. If however, more detailed performance mea-sures are desired then, as would be expected, more pro-cess information is needed. For example, it would be de-sirable to include the control ‘cost’ or effort in the perfor-mance evaluation of a controller or answers to questionssuch as the following may be required: What is the bestavailable control subject to soft constraints on the con-troller output variance. Two relatively new benchmarksare presented next as alternative measures of practicalmultivariate controller performance.

LQG Benchmarking

Preliminary results on the LQG benchmark as an alter-native to the minimum variance benchmark were pro-posed in Huang and Shah (1999). These results are re-viewed here and a new benchmark that takes the controlcost into account is proposed. The main advantage ofthe minimum variance benchmark is that other than thetime-delay, it requires little process information. On theother hand if one requires more information on controllerperformance such as how much can the output variance


Figure 7: Pressure and level data with sampling in-terval 5 seconds.

be reduced without significantly affecting the controlleroutput variance then one needs more information on theprocess. In short it is useful to have a benchmark thatexplicitly takes the control cost into account. The LQGcost function is one such benchmark. This benchmarkdoes not require that an LQG controller be implementedfor the given process. Rather the benchmark providesthe ‘limit of performance’ for any linear controller interms of the input and output variance. As remarkedearlier, it is a general benchmark with the minimum vari-ance as a special case. The only disadvantage is that thecomputation of the performance limit curve as shown inFigure 12 requires knowledge of the process model. ForMPC type controllers these models may be readily avail-able. Furthermore the benchmark cannot handle hardconstraints, but it can be used to compare the perfor-mance of unconstrained and constrained controllers (seeFigure 13).

In general, tighter quality specifications lead to smallervariations in the process output, but typically requiremore control effort. Consequently one may be interestedin knowing how far away the control performance is fromthe “best” achievable performance with the same effort,i.e., in mathematical form the resolution of the followingproblem may be of interest:

Given that E(u2) ≤ α, what is the lowestachievable E(y2)?

The solution is given by a tradeoff curve as shown inFigure 12. This curve can be obtained by solving theLQG problem (Kwakernaak and Sivan, 1972), where theLQG objective function is defined by:

J(λ) = E(y2) + λE(u2)

By varying λ, various optimal solutions of E(y2) andE(u2) can be calculated. Thus a curve with the optimaloutput variance as ordinate, and the incremental manip-ulative variable variance as the abscissa can be plottedfrom these calculations. Boyd and Barratt (1991) haveshown that any linear controller can only operate in theregion above this curve. In this respect this curve de-fines the limit of performance of all linear controllers, asapplied to a linear time-invariant process, including theminimum variance control law. If the process is modelled

0.0000

0.00250.00500.00750.0100

0.01250.01500.01750.02000.0225

Multivariate performance indices

PI

PVs Level PressurePVsPI 0.022 0.012

Figure 8: Individual output performance indices.

0.03

0.04

0.05

0.06

0.07

0.08

0.09 Normalized Multivariate Impulse Response

Imp

uls

e

Re

sp

on

se

Lag5 10 15 20 25 30 35 40 45 50

Figure 9: Normalized multivariate impulse responsewithout interactor filtering.

-0.1

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

0 5 10 15 20 25 30 35 40

Auto Correlation of Level

ACF

Lag

-0.2

-0.1

0.0

0.1

0.2

0.3

0.4

0.5

0 5 10 15 20 25 30 35 40

Cross-Correlation: Pressure . vs. Level

CCF

Lag

-0.50

-0.25

0.00

0.25

0.50

0.75

1.00

0 5 10 15 20 25 30 35 40

Auto Correlation of Pressure

ACF

Lag

-0.3

-0.2

-0.1

0.0

0.1

0.2

0.3

0.4

0 5 10 15 20 25 30 35 40

Cross-Correlation: Level . vs. Pressure

CCF

Lag

Figure 10: Auto and cross-correlation of process out-put.

as an ARIMAX process then the resulting LQG bench-mark curve due to the optimal controller will have anintegrator built into it to asymptotically track and rejectstep type setpoints and disturbances respectively. Fiveoptimal controllers may be identified from the tradeoffcurve shown in Figure 12. They are explained as follows:

• Minimum cost control: This is an optimal controlleridentified at the left end of the tradeoff curve. Theminimum cost controller is optimal in the sense that


0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

10 -2 10 -1

Power Spectrum of Level

Magnitude

Frequency

-0.05

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

0.45

10 -2 10 -1

Cross Spectrum : Pressure vs. Level

Magnitude

Frequency

0.000

0.025

0.050

0.075

0.100

0.125

0.150

0.175

0.200

0.225

10 -2 10 -1

Power Spectrum of Pressure

Magnitude

Frequency

Figure 11: Frequency domain analysis of process out-put.

it offers an offset-free control performance with theminimum possible control effort. It is worthwhilepointing out that this controller is different fromthe open-loop mode since an integral action is guar-anteed to exist in this controller.

• Least cost control: This optimal controller offersthe same output error as the current or existingcontroller but with the least control effort. So ifthe output variance is acceptable but actuator vari-ance has to be reduced then this represents the low-est achievable manipulative action variance for thegiven output variance.

• Least error control: This optimal controller offersleast output error for the same control effort as theexisting controller. If the input variance is accept-able but the output variance has to be reduced thenthis represents the lowest achievable output variancefor the given input variance.

• Tradeoff controller: This optimal controller can beidentified by drawing the shortest line to the tradeoffcurve from the existing controller; the intersection isthe tradeoff control. Clearly, this tradeoff controllerhas performance between the least cost control andthe least error control. It offers a tradeoff betweenreductions of the output error and the control effort.

• Minimum error (variance) control: This is an op-timal controller identified at the right end of thetradeoff curve. The minimum error controller is op-timal in the sense that it offers the minimum pos-sible error. Note that this controller may be dif-ferent from the traditional minimum variance con-troller due to the existence of integral action.

The challenges with respect to the LQG benchmarklie in the estimation of a reasonably accurate processmodel. The uncertainty in the estimated model thenhas to be ‘mapped’ onto the LQG curve, in which case

*o

o

o

o

M inimum V ar iance Contr ol

L east Er r or Contr ol

Cur r ent Contr ol

L east CostContr ol

Tr adeoffContr ol

o

M inimum CostContr ol

2|||| yy sp −

2|||| u∆

Figure 12: The LQG tradeoff curve with several op-timal controllers.

it would become a fuzzy trade-off curve. Alternately theuncertainty region can be mapped into a region aroundthe current performance of the controller relative to theLQG curve (see Patwardhan et al., 2000).

An Alternative Method for Multivariate Perfor-mance Assessment Using the Design Case as aBenchmark

An alternative approach is to evaluate the controller per-formance using a criterion commensurate with the actualdesign objective(s) of the controller and then comparethe achieved performance. This idea is analogous to themethod of Kammer et al. (1996), which was based onfrequency domain comparison of the achieved and designobjective functions for LQG. For a MPC controller witha quadratic objective function, the design requirementsare quantified by:

J(k) =

p∑i=1

(ysp(k + i|k) − y(k + i|k))T

Γk,i(ysp(k + i|k) − y(k + i|k))

+

M−1∑i=1

∆u(k + i − 1)T Λ∆u(k + i − 1)

where

y(k+i|k) is the i-step ahead predictor of the outputsbased on the process modelysp(k + i|k) is the setpoint trajectory∆u(k + i − 1) are the future moves of the inputsΓk,i are the output weightings that, in general, candepend upon the current time and the predictionhorizon

Details of the model predictive control calculations canbe found in any standard references (Garcia et al., 1989;Mayne et al., 2000; Qin and Badgwell, 1996). Here we re-strict ourselves to the performance assessment aspects.The model predictive controller calculates the optimal


control moves by minimizing this objective function overthe feasible control moves. If we denote the optimal con-trol moves by ∆u∗(k), the optimal value of the designobjective function is given by

J∗(k) = J(∆u∗(k))

The actual output may differ significantly from thepredicted output due to inadequacy of the model struc-ture, nonlinearities, modeling uncertainty etc. Thus theachieved objective function is given by:

J(k) =p∑

i=1

(ysp(k + i|k) − y(k + i|k))T

Γk,i(ysp(k + i|k) − y(k + i|k))

+M−1∑i=1

∆u(k + i − 1)T Λ∆u(k + i − 1)

where y(k) and ∆u(k) denote the measured values of theoutputs and inputs at corresponding sampling instantsappropriately vectorized. The inputs will differ from thedesign value in part due to the receding horizon natureof the MPC control law. The value of the achieved objec-tive function cannot be known a priori, but only p sam-pling instants later. A simple measure of performancecan then be obtained by taking a ratio of the design andthe achieved objective functions as:

η(k) =J∗(k)J(k)

This performance index will be equal to one whenthe achieved performance meets the design requirements.The advantage of using the design criterion for the pur-pose of performance assessment is that it is a measure ofthe deviation of the controller performance from the ex-pected or design performance. Thus a low performanceindex truly indicates changes in the process or the pres-ence of disturbances, resulting in sub-optimal control.The estimation of such an index does not involve anytime series analysis or identification. The design ob-jective is calculated by the controller at every instantand only the measured input and output data is neededto find the achieved performance. The above perfor-mance measure represents an instantaneous measure ofperformance and can be driven by the unmeasured dis-turbances. In order to get a better overall picture thefollowing measure is recommended:

αk =

k∑i=1

J∗(i)

k∑i=1

J(k)

α(k) is the ratio of the average design performance tothe average achieved performance up to the current sam-pling instant. Thus α(k) = 1 implies that the designperformance is being achieved on an average. α(k) < 1means that the achieved performance is worse than thedesign. This alternative metric of multivariate controllerperformance has been applied towards the evaluation ofa QDMC and another MPC controller. Further detailson the evaluation of this algorithm can be found in (Pat-wardhan, 1999).

The motivation for a lumped performance index is thatthe MPC controllers in the dynamic sense, attempt tominimize a lumped performance objective. The lumpedobjective function and subsequently the performance in-dex, therefore does reflect the true intentions of the con-troller. The motivation for this idea was to have a per-formance statistic for MPC that is commensurate withits constrained and time-varying nature. The idea ofcomparing design with achieved performance has beencommon place in the area of control relevant identifica-tion (also known as iterative identification and control,identification for control)—see the survey by Van denHof and Schrama (1995). Performance degradation ismeasured as a deviation from design performance andbecomes the motivation for re-design/re-identification.

Simulation Example: A Mixing Process. Theabove approach was applied to a simulation example.The system under consideration is a 2×2 mixing process.The controlled variables are temperature (y1) and waterlevel (y2) and the manipulated inputs are inlet hot water(u1) and inlet cold water (u2) flow rates. The followingmodel is available in discrete form,

P (z−1) =

0.025z−1

1 − 0.8607z−1

−0.1602z−1

1 − 0.8607z−1

0.2043z−1

1 − 0.9827z−1

0.2839z−1

1 − 0.9827z−1

A MPC controller was used to control this process inthe presence of unmeasured disturbances. The controllerdesign parameters were:

p = 10, , m = 2, , λ = diag([1, 4]), Γ = diag([1, 2])

White noise sequences at the input and output with co-variance equal to 0.1I served as the unmeasured distur-bances. First the LQG benchmark was found, and theperformance of a constrained and unconstrained MPCwas evaluated against this benchmark (see Figure 13).Constraints on input moves were artificially imposed inorder to activate the constraints frequently. The uncon-strained controller showed better performance, comparedto the constrained controller, with respect to the LQGbenchmark.

A plot of the LQG objective function compared to theachieved objective function is shown in Figure 14. A per-formance measure of Jlq/Jach = 0.467/0.71 = 0.6579 was


0 0.1 0.2 0.3 0.41.2

1.4

1.6

1.8

2

2.2

2.4

||Var(u)||/||Var(w)||

||Var

(y)||

/||V

ar(w

)||Constrained MPC

Unconstrained MPC

Figure 13: MPC performance assessment using LQGbenchmark.

J lq

Jach Jlq/Jach=0.6597

0 2 4 6 8 100.4

0.45

0.5

0.55

0.6

0.65

0.7

0.75

0.8

lamda

Figure 14: Comparison of the achieved performancewith the LQG objective function.

LQG α(k)Unconstrained 0.6579 0.8426Constrained 0.4708 1.00

Table 1: Effect of constraints on MPC performance.

obtained for the unconstrained controller. Performanceassessment of the same controller using the design casebenchmarking approach yields contrasting results (Table1). For the unconstrained controller a performance in-dex 0.8426 revealed satisfactory performance while theimposition of constraints led to a performance index of1. The constrained controller showed improvement ac-cording to one benchmark and deterioration with respectto the LQG benchmark. The design case approach in-dicates that the controller is doing its best under thegiven constraints while the LQG approach which is basedon comparison with an unconstrained controller shows adegradation in performance.

-0.02-0.01 0 0.01 0.02

-0.01

0

0.01

du2

du1

Figure 15: The input moves for the constrained con-troller during the regulatory run.

Figure 15 shows the input moves during the regula-tory run for the constrained controller. The constraintsare active for a large portion of the run and are limitingthe performance of the controller in an absolute sense(LQG). On the other hand the controller cannot do anybetter due to design constraints as indicated by the de-sign case benchmark.

Challenges in Performance Analysis andDiagnosis: General Comments and Issuesin MPC Performance Evaluation

A single index or metric by itself may not provide allthe information required to diagnose the cause of poorperformance. Considerable insight can be obtained bycarefully interpreting all the performance indices. Forexample, in addition to the minimum variance bench-mark performance index, one should also look at thecross-correlation plots, normalized multivariate impulseresponse plots, spectrum analysis, etc., to determinecauses of poor performance. As an example, if the pro-cess data is ‘white’ then the performance index will al-ways be close to 1 irrespective of how large the varianceis. On the other hand, if the data is highly correlated(highly predictable), then the performance index will below irrespective of how small the output variance is. Inthis respect the performance index plus the impulse re-sponse or the auto-correlation plot would provide a com-plete picture of the root cause of the problem. (Theauto-correlation plot would have yielded information onthe predictability of the disturbance). In summary then,each index has its merits and its limitations. Therefore,one should not just rely on any one specific index. Itwould be more appropriate to check all relevant indicesthat reflect performance measures from different aspects.

As mentioned earlier, the multivariate extension of


the minimum variance benchmark requires a knowledgeof the time-delay or the interactor matrix. This re-quirement of a priori information on the interactor hasbeen regarded by many as impractical. However, fromour experience this benchmark, when applied with care,can yield meaningful measures of controller performance.Yet, many outstanding issues remain open before onecan confidently apply MIMO assessment techniques for awide-class of MIMO systems. Some of the issues relatedto the evaluation of multivariate controllers are listedbelow:

1. To calculate a general interactor matrix, one needsto have more a priori information than just the timedelays. However, experience has shown us that asignificant number of MIMO processes do have thediagonal interactor structure. In fact, a properly de-signed MIMO control structure will most likely havea diagonal interactor structure (Huang and Shah,1999). A diagonal interactor depends only on thetime delays between the paired input and outputvariables.

2. Since models are available for all MPC based con-trollers, a priori knowledge of the time delay matrixis surely not an issue at all.

3. A more important issue in the analysis and diag-nosis of control loops is the accuracy of the mod-els and their variability over time. How does themodel uncertainty affect the calculation of perfor-mance index? This question has not been answeredso far. It is a common problem in both MIMO andSISO performance assessment. Therefore, one of themany outstanding issues remaining is the robustnessof performance assessment, i.e. how to transfer themodel uncertainty onto the uncertainty in the cal-culation of the performance index? This issue hasbeen addressed to some extent in Patwardhan (1999;2001), where SISO and MIMO examples, relatingmodeling uncertainty to uncertainty in performancemeasures, are given.

Industrial MPC is a combination of a dynamic partand a steady state part, which often comprises a linearprogramming (LP) step. The dynamic component con-sists of unconstrained minimization of a dynamic costfunction, comprising of the predicted tracking errors andfuture input moves, familiar to academia. The steadystate part focuses on obtaining economically optimal tar-gets, which are then sent to the dynamic part for trackingas illustrated in Figure 16. This combination of the dy-namic and steady state parts and constraint handling viathe linear programming or the LP step renders the MPCsystem as a nonlinear multivariate system. Patward-han et al. (1998) have illustrated the difficulties causedby the LP step on an industrial case study involving ademethanzier MPC (see Figure 17). In that particular

y

+

+

d

Dynamic Control uProcess

P

Model

P

+

-

desJMinu

y

+

-

Steady StateModel

Steady StateOptimization

K

ssy

ssy

Jss

max,uss

ssu

Figure 16: Schematic of a typical commercial MPCwith a blended linear programming module that setstargets for the controlled and manipulative variables.

application as in a number of other MPC applications,when the LP stage is activated fairly routinely, signifi-cant correlation exists between the LP targets and themeasurements that the LP relies upon. In such instanceswhen the LP stage is activated at the same frequenciesas the control frequencies, the controller structure is nolonger linear. Situations such as these preclude the use ofconventional performance assessment methods such theLQ or the minimum variance benchmark. We believethat the variable structure nature of industrial MPC canbe captured by the objective function method since ittakes into account the time varying nature of the MPCobjective. Patwardhan (1999) has applied this methodsuccessfully on an industrial QDMC application. Eventhough one may argue that QDMC is devoid of the LPstep, it is a variable structure MIMO controller that al-lows different inputs and outputs to swap into active andinactive states relative to the active constraint set. Inthis respect, the lumped objective function and subse-quently the performance index proposed here does ‘mea-sure’ the true intentions of the controller relative to thedesign case. It thus provides a useful performance met-ric. The only limitation being that the access to theactual design control objective has to be available in theMPC vendor software.

Establishing the root causes of performance degrada-tion in industrial MPCs is indeed a challenging task. Po-tential factors include models, inadequately designed LPin that the LP operates at the control frequencies, in-appropriate choice of weightings, ill-posed constraints,steady-state bias updates etc. In practice, these factorscombine in varying degrees to give poor performance.Thus the issues and challenges related to the diagno-sis aspects of MPC performance assessment are many.Some of these issues are listed below and one ‘quan-tifiable’ diagnosis issue related to model-plant mismatchis discussed. The diagnosis stage for poor performanceinvolves a trial and error approach (Kesavan and Lee,


Figure 17: An example of the interaction betweenthe steady state optimization and the dynamic layerin industrial MPC. Note that setpoints have highervariation compared to controlled variables!

1997). For example, the diagnosis or the decision supportsystem has to investigate the cause of poor performanceas being due to:

• Poor or incorrect tuning.

• Incorrect controller configuration, e.g. choice ofMVs may not be correct.

• Large disturbances, in which case the sources ofmeasured disturbances have to be identified and po-tential feedforward control benefits should be inves-tigated.

• Engineering redesign , e.g. is it possible to reduceprocess time delays?

• Model-plant mismatch in the case of MPC con-trollers and how does model uncertainity affect thecalculation of the performance index (e.g. if theindex has been obtained from an uncertain interac-tor).

• Poor choice of constraint variables and constrainedvalues.

Some of the above referenced issues have been alreadydealt with in the literature, e.g. (ANOVA analysis toinvestigate the need for feedforward control by Desbor-ough and Harris (1993), Vishnubhotla et al. (1997), andStanfelj et al. (1993); others are open problems. Thediagnosis issues related to the model-plant mismatch isbriefly discussed below in a theoretical framework andillustrated on an industrial MPC evaluation case studythat follows. A discussion of the poor performance diag-nosis steps leading to guidelines for tuning and controllerdesign issues is beyond the scope of this paper.

Model-Plant Mismatch

MPC controllers rely heavily on process models. In par-ticular an accurate model is required if the process is

Controller Plant

Model

dysp

e

-

-

Σ Σ

Σ

Figure 18: Schematic of a closed loop system inwhich the prediction error is monitored.

to be regulated very tightly. On the other hand perfor-mance can be detuned in favour of robustness if an accu-rate model is not available or should the process changeover a period of time. The extent of MPM can not beeasily discerned by simply examining the closed loop pre-diction error. As shown in Figure 18, the prediction errorunder closed loop conditions is a function of the MPM,setpoint changes and measured and unmeasured distur-bances. Thus the cause of large prediction errors maynot necessarily be attributed to a large MPM. ConsiderFigure 18, where the prediction error is denoted as e.

Under open loop conditions, the prediction errors is:e = (P − P )u + d. Under closed loop control the predic-tion error expression is:

e =

((P − P )C1 + CP

)ysp +

(1 + CP

1 + CP

)d

It is clear from the above expression that a largeprediction error signal could be due to a large MPMterm, or a large disturbance term or setpoint changes.Thus the question of attributing a large prediction er-ror as being due to model-plant-mismatch needs carefulscrutiny. Huang (2000) has studied the problem of de-tecting significant model plant mismatch or process pa-rameter changes in the presence of disturbances.

Industrial MIMO Case study 2: Analysisof Cracking Furnace Under MPC Control

This section documents the results of the controller per-formance analysis carried out on an ethane cracking fur-nace. The control systems comprises of (1) a regulatorylayer and (2) an advanced MPC control layer. The firstpass of performance assessment revealed some poorlyperforming loops. Further analysis revealed that theseloops were in fact well tuned but were being affected byhigh frequency disturbances and setpoint changes. Thefurnace MPC application considered here, however, isunlike conventional MPC applications. The steady statelimits were set in such a way that the setpoints for thecontrolled variables were held constant, i.e. the focus ofthe evaluation was on the models and the tuning of thedynamic part.

The MPC layer displays satisfactory performance lev-


els when there are no rate changes. During rate changes,the MPC model over predicts thus causing poor perfor-mance. Re-identification of the model gains was foundto be necessary to improve MPC performance.

Control Strategy Overview

The purpose of the Furnace MPC controllers is to main-tain smooth operation, maximize throughput and mini-mize energy consumption to the furnaces while simulta-neously honoring all constraints. There is one MPC con-troller per furnace. The conversions, the total dry feed,the wet feed bias, the steam/feed ratio and the oxygen tofuel ratio is controlled by MPC. These variables are ma-nipulated by moving the north and south fuel gas dutysetpoints, the north and south wet feed flow setpoints,the steam pressure setpoint, the fan speed controller set-point and the induced fan draft. Thus there are 6 pri-mary controlled variables and 7 manipulated variables.There are a number of secondary controlled variables,which MPC is required to maintain within a constraintregion. These secondary CVs include valve constraints,constraints on critical variables such as the coil averagetemperatures (COTS). Considering the degrees of free-dom (MVs) available, it may not always be possible tosatisfy all the constraints. In such cases, a ranking mech-anism decides what constraints are least important andcould be let go.

A critical component of the MPC controllers is themodel describing the relationships between the MVs andthe CVs—primary as well as secondary. These mod-els are developed on the basis of open loop tests. Stepresponse curves are used to parameterize the models.These models are used by MPC to predict the futureprocess response. A portion of the step response modelmatrix is shown in Figure 19.

Performance analysis of the furnace control loops wasconducted in two stages. Phase one loop analysis wasperformed on the lower regulatory layer. With the ex-ception of a few loops, the first pass of performanceassessment revealed satisfactory performance of mostloops. The higher level MPC performance assessmentcommenced next.

Multivariable Performance Assessment for MPC. Ta-ble 2 summarizes the performance statistics. A diagonalinteractor was used, based on the knowledge of the pro-cess models.

The performance metrics indicated satisfactory perfor-mance on all the variables except for tags 4 and 6. Theclosed loop settling time is approximately the same asthe open loop settling time, which indicates a conserva-tively tuned application. Part of the reason for the slowresponse in control of tags 4 and 6 could be the presenceof a measurement delay. During January 2000, the ser-vice factor for MPC was low due to some communicationissues, which have been since resolved. This meant thatthere was an opportunity to compare the furnace per-

Description PI

Closed-Loop

SettlingTime (min) Status

Tag 1 0.95 8Tag 2 0.94 1Tag 3 0.76 15Tag 4 0.44 15 LOWTag 5 0.71 15Tag 6 0.49 15 LOWTag 7 0.88 10Multivariable PI 0.71

Table 2: Summary of MPC performance.

formance with and without MPC. Based on data fromJan 14-16 when MPC was shut off for part of the time,performance metrics were obtained to compare the twocontrol systems—MPC and conventional PID controls.The statistics indicate that the overall control is onlyslightly better with MPC turned on.

MPC Diagnostics

Is MPC doing its best? Can we improve the currentperformance levels of the furnace MPC controller? Thesequestions lead us to two issues that are closely related toeach other:

1. How good are the models used for predicting theprocess response?

2. How well tuned is the multivariable controller?“Tuning” includes a whole range of different ofparameters—weightings, horizons, constraints,rankings . . .

We will try to illustrate a case where the model pre-dictions can mislead the controller and hence cause poorperformance. This Furnace was showing poor MPC per-formance, especially during rate changes. This moti-vated us to look more closely at the model predictionaccuracy.

Before evaluating the current predictions, we establisha baseline when the open loop tests were conducted. Fig-ures 21–23 compare the conversion predictions for theopen loop case as conducted before commissioning theMPC. The model accuracy is reasonable. The averageprediction error, for this data set was 3.18.

AveragePrediction

Error=

1N

N∑k=1

Ny∑i=1

{yi(k + 1) − yi(k + 1|k)}2

The predictions for other variables—COTs, Feed Flowand Bias, S/F ratio also fared well. The remaining vari-ables are not shown for the sake of brevity.


0 500123

x 10-3

0 50

0246

x 10-4

0 50

-15-10

-5

0 500

0.1

0.2

0 50

0246

x 10-4

0 500123

x 10-3

0 50

-15-10

-5

0 500

0.1

0.2

0 50

0.20.4

0.60.8

0 50

0.20.4

0.60.8

0 50-0.02

-0.01

0 50

0.5

1

0 50

-1

-0.5

0 50-14-12-10

-8-6-4-2

x 10-3

0 50-14-12-10

-8-6-4-2

x 10-3

0 50

0.51

1.52

2.5

x 10-3

Figure 19: A portion of the step response models used by MPC.

The model accuracy can then be compared with thecurrent predictions. There were two rate changes duringthis period, due to furnace decokes.

An average prediction error of 175.7 was observed dur-ing this period. The tags 3 to 6 predictions were muchworse than the rest. Taking a closer look at the tags 3-6predictions revealed that the models were over predictingby a factor of 2. Based on a combination of statisticalanalysis and process knowledge it was decided that themodel gains were incorrect.

Improving MPC Performance

The main stumbling block to improving MPC perfor-mance was its model accuracy. The models causing thelarge mismatch were identified. One of the reasons forthe model plant mismatch is the fact that open loop testswere carried out in a operating region which is quitedifferent from the current operating conditions (higherrates, conversions, duties). Fairly routine plant test,were used to identify the suspected changes in steadystate gains. These tests were conducted under closed

0

0.2

0.4

0.6

0.8

1

1.2

Tag 1 Tag 2 Tag 3.

Tag 4 Tag 5 Tag 6 Tag 7

MPC ON

MPC OFF

Figure 20: Comparison of performance, with andwithout MPC.

loop conditions and the gain mismatch has now beenfixed with satisfactory MPC performance. The newlyidentified gains were indeed found to be significantly dif-ferent from the earlier gains.

To illustrate the effect of the MPM on cracking effi-ciency, conversion control on the 3 furnaces was com-


Figure 21: The predictions (green) and the Northconversion measurements (blue) are shown in the topgraph. The bottom graph shows the changes in fuelduty during this period.

Figure 22: The scaled conversion predictions for Feb11-15.

pared and is shown in Figure 24. Furnace 3 with theolder models takes almost 45 minutes more to settle out,compared to furnaces 1 and 2, which have the updatedmodels. The overshoot for Furnace 3 is 2.5%, as opposedto 1.5% on Furnace 1 and 2 (a significant 40% decrease).Thus the performance and subsequent diagnosis analysisof this MPC application illustrates the value in routinemonitoring and maintenance of MPC applications.

Concluding Remarks

In summary, industrial control systems are designed andimplemented or upgraded with a particular objective inmind. We hope that the new controller loop performanceassessment methodology proposed in the literature andillustrated here, will eventually lead to automated andrepeated monitoring of the design, tuning and upgrading

Figure 23: The scaled conversion predictions—apparent gain mismatch.

Figure 24: Comparison of Furnace control with up-dated models.

of the control loops. Poor design, tuning or upgradingof the control loops will be detected, and repeated per-formance monitoring will indicate which loops should bere-tuned or which loops have not been effectively up-graded when changes in the disturbances, in the processor in the controller itself occur. Obviously better design,tuning and upgrading will mean that the process will op-erate at a point closer to the economic optimum, leadingto energy savings, improved safety, efficient utilizationof raw materials, higher product yields, and more con-sistent product qualities. Results from industrial appli-cations have demonstrated the applicability of the multi-variate performance assessment techniques in improvingindustrial process performance.

Several different measures of multivariate controllerperformance have been introduced in this paper andtheir applications and utility have been illustrated bysimulation examples and industrial case studies. Themultivariate minimum variance benchmark allows one tocompare the actual output performance with the mini-mum achievable variance. However it requires knowledge


of the process time-delay matrix or the interactor. Onthe other hand the newly proposed NMIRwof measure ofperformance provides a graphical ‘metric’ that requireslittle or no a priori information about the process andgives a graphical measure of mutivariate performance interms of settling time, rate of decay etc.. The challengesrelated to MPC performance evaluation are illustratedby an industrial case study of an ethylene cracker. It isshown how routine monitoring of MPC applications canensure good or ‘optimal’ control. The lumped objectivefunction based method of monitoring MPC performanceis shown to work well on the industrial case study. Thestudy illustrates how controllers, whether in hardwareor software form, should be treated like ‘capital assets’;how there should be routine monitoring to ensure thatthey perform close to the economic optimum and thatthe benefits of good regulatory control will be achieved

Acknowledgments

The authors of this paper wish to express sincere grati-tude and appreciation to the following persons who haveprovided assistance by participating in industrial evalu-ation studies at their respective work sites: Jim Krestaand Henry Tse (Syncrude Research), Janice Vegh-Morinand Harry Frangogiannopoulus (both of TDCC), GenichiEmoto (Mitsubishi Chemical Corporation, Japan) andAnand Vishnubhotla (Matrikon Consulting) Their feed-back and industrial insight were helpful in enhancing thepracticality and utility of the proposed methods to makeit better suited to the process industry.

Financial support in the form of research grants fromthe Natural Science and Engineering Research Councilof Canada and Equilon Enterprises (Shell-USA) is grate-fully acknowledged.

References

Boyd, S. and C. Barratt, Linear Control Design. Prentice Hall,Englewood Cliffs, New Jersey (1991).

Desborough, L. and T. Harris, “Performance Assessment Mea-sures for Univariate Feedback Control,” Can. J. Chem. Eng.,70, 1186–1197 (1992).

Desborough, L. D. and T. J. Harris, “Performance AssessmentMeasures for Univariate Feedforward/Feedback Control,” Can.J. Chem. Eng., 71, 605–616 (1993).

Garcia, C. E., D. M. Prett, and M. Morari, “Model PredictiveControl: Theory and Practice—A Survey,” Automatica, 25(3),335–348 (1989).

Harris, T. J., F. Boudreau, and J. F. MacGregor, “PerformanceAssessment of Multivariate Feedback Controllers,” Automatica,32(11), 1505–1518 (1996).

Harris, T. J., “Assessment of Control Loop Performance,” Can. J.Chem. Eng., 67, 856–861 (1989).

Huang, B. and S. L. Shah, Performance Assessment of ControlLoops. Springer-Verlag, London (1999).

Huang, B., S. L. Shah, and K. K., How Good Is Your Con-troller? Application of Control Loop Performance AssessmentTechniques to MIMO Processes, In Proc. 13th IFAC WorldCongress, volume M, pages 229–234, San Francisco (1996).

Huang, B., S. L. Shah, and E. K. Kwok, “Good, Bad or Optimal?Performance Assessment of Multivariable Processes,” Automat-ica, 33(6), 1175–1183 (1997).

Huang, B., S. L. Shah, and H. Fujii, “The Unitary InteractorMatrix and its Estimation Using Closed-Loop Data,” J. Proc.Cont., 7(3), 195–207 (1997).

Huang, B., “Model Validation in the Presence of Time-variant Dis-turbance Dynamics,” Chem. Eng. Sci., 55, 4583–4595 (2000).

Kammer, L. C., R. R. Bitmead, and P. L. Barlett, Signal-basedtesting of LQ-optimality of controllers, In Proc. of the 35thIEEE CDC, pages 3620–3624, Kobe, Japan (1996).

Kesavan, P. and J. H. Lee, “Diagnostic Tools for MultivariableModel-Based Control Systems,” Ind. Eng. Chem. Res., 36,2725–2738 (1997).

Ko, B. and T. F. Edgar, Performance Assessment of MultivariableFeedback Control Systems (2000). Submitted to Automatica.

Kozub, D. J. and C. E. Garcia, Monitoring and Diagnosis of Au-tomated Controllers in the Chemical Process Industries, AIChEAnnual Meeting, St. Louis (1993).

Kozub, D., Controller Performance Monitoring and Diagnosis Ex-periences and Challenges, In Kantor, J. C., C. E. Garcıa, andB. Carnahan, editors, Chemical Process Control—V, pages 156–164. CACHE (1997).

Kwakernaak, H. and R. Sivan, Linear Optimal Control Systems.John Wiley and Sons, New York (1972).

Mayne, D. Q., J. B. Rawlings, C. V. Rao, and P. O. M. Scokaert,“Constrained Model Predictive Control: Stability and Optimal-ity,” Automatica, 36(6), 789–814 (2000).

Patwardhan, R. S. and S. L. Shah, Issues in Performance Diag-nostics of Model-based Controllers, Accepted for publication, J.Proc. Control (2001).

Patwardhan, R. S., G. Emoto, and S. L. Shah, Model-based Pre-dictive Controllers: An Industrial Case Study, AIChE AnnualMeeting, Miami (1998).

Patwardhan, R., S. L. Shah, and K. Qi, Techniques for Assess-ment of Model Predictive Controllers, Submitted for publication(2000).

Patwardhan, R., S. L. Shah, and G. Emoto, Experiences in Per-formance Annalysis of Industrial Model Predictive Controllers,Submitted for publication (2001).

Patwardhan, R. S., Studies in the Synthesis and Analysis of ModelPredictive Controllers, PhD thesis, University of Alberta (1999).

Qin, S. J. and T. A. Badgwell, An Overview of Industrial ModelPredictive Control Technology, In Kantor, J. C., C. E. Garcia,and B. Carnahan, editors, Fifth International Conference onChemical Process Control—CPC V, pages 232–256. AmericanInstitute of Chemical Engineers (1996).

Stanfelj, N., T. E. Marlin, and J. F. MacGregor, “Monitoringand Diagnosing Process Control Performance: The Single-LoopCase,” Ind. Eng. Chem. Res., 32, 301–314 (1993).

Swanda, A. P. and D. E. Seborg, Controller Performance Assess-ment Based on Setpoint Response Data, In Proceedings of theAmerican Control Conference, pages 3863–3867, San Diego,California (1999).

Thornhill, N. F., M. Oettinger, and P. Fedenczuk, “Refinery-widecontrol loop performance assessment,” J. Proc. Cont., 9, 109–124 (1999).

Van den Hof, P. M. J. and R. J. P. Schrama, “Identification andcontrol—closed-loop issues,” Automatica, 31(12), 1751–1770(1995).

Vishnubhotla, A., S. L. Shah, and B. Huang, Feedback and Feed-forward Performance Analysis of the Shell Industrial ClosedLoop Data Set, In Proc. IFAC Adchem 97, pages 295–300,Banff, AB (1997).

Wood, R. K. and M. W. Berry, “Terminal Composition Control ofa Binary Distillation Column,” Chem. Eng. Sci., 28, 1707–1717(1973).

multivariate controller performance analysis: methods, … · 2010. 7. 15. · multivariate...

Documents