Real-Time Computational Methods for SDRE Nonlinear Control of Missiles*

P. K. Menon, T. Lam, L. S. Crawford, V. H. L. ChengOptimal Synthesis Inc.

4966 El Camino Real, Suite 108Los Altos, CA 94022

* Research supported under Air Force Contract No. F08630-99-C-0060, with Dr. J. R. Cloutier serving as theTechnical Monitor.

AbstractComputational speed and performance issues

arising in the practical implementation of the statedependent Riccati equation (SDRE) technique arediscussed. A software package for real-timeimplementation of the SDRE technique wasdeveloped during the present research. The executionof this software at speeds up to 2 kHz sample rates onproblems of the size commonly encountered inmissile flight control applications is thendemonstrated on commercial off-the-shelf processors.

1. IntroductionState Dependent Riccati Equation (SDRE)

method is a recently developed technique for thecontrol of nonlinear dynamic systems [1 – 4]. Thispaper discusses the development of numericalalgorithms for the practical implementation of theSDRE control technology in missile flight controlproblems. The SDRE control technology hasextensive applications in the design of advancedflight control systems, and also in numerousautomotive and process control applications. Themain difficulty encountered by the users attemptingto use the SDRE technology is that of developingreal-time computational algorithms that can beimplemented on commercially available processors.This is due to the fact that the SDRE controltechnique demands advanced numerical algorithmsand significantly more computational resources thanconventional control algorithms. This paperaddresses the speed and performance issues that arisein the practical implementation of the SDRE controltechnique. Commercial, off-the-shelf computertechnology such as the Windows [5] operatingsystem running on Pentium II/III processors [6] areemployed in the present work.

The following lists the accomplishments of theresearch:1. Developed fast, C language-implementations of

two algorithms for the solution of algebraicRiccati equations that form the centralcomponent of the SDRE design technique. Thefirst is a direct method based on Schurdecomposition of the Hamiltonian matrix [7, 8].The second approach is an iterative algorithmthat refines an initial guess of the Riccati

solution using the Kleinman algorithm [9, 10].These software packages use extensively re-worked code components from the public-domain linear algebra package LAPACK [11].Numerical results from these algorithms havebeen compared with those from softwarepackages such as MATLAB[12].

2. Solution speeds of these algorithms have beenassessed on Intel Pentium II, 300 MHz andPentium III 450 MHz computers running theWindowsNT operating system, and on a real-time Motorola 604e PowerPC board (DS1103)from dSPACE Inc [13]. The dSPACE PowerPCboard is used extensively in automotive controlsystem development. A five-state nonlineardynamic system representing a missile flightcontrol system was used in these evaluations.The maximum speed obtained for this problem isaround 2 kHz.Present research shows that these numerical

algorithms are capable of delivering computationalspeeds far in excess of that required for implementinghigh-performance weapon flight control systems.Since the main computational burden inimplementing the SDRE technique is in the solutionof the algebraic Riccati equation, the present researchdemonstrates the feasibility of implementing theSDRE technique on commercial, off-the-shelfprocessors in real-time. Moreover, analysis hasrevealed that it may be possible to gain an additional25% speed improvement by further refining the Ccode.

The next section will present an overview of theSDRE technique. Section 3 will discuss an approachfor numerically constructing the state dependentcoefficient form (SDC form) of the system dynamicsfrom a numerical simulation of the system dynamics.Approaches for solving the algebraic matrix Riccatiequation will be discussed in Section 4. Approximateoperation counts for each of the computationalalgorithms will also be given in that section. Section5 will discuss code development and speedevaluation issues. Conclusions will be given inSection 6.

The present research clearly demonstrates thatthe SDRE technique can be implemented in real-time

232

2

for a realistic, high-order multivariable system usingcommercial, off-the-shelf computing subsystems.Real-time SDRE computations for a problem ofdimensions comparable to missile flight controlsystem is used as an example.

2. An Overview of the SDRE TechniqueState Dependent Riccati Equation (SDRE)

method [1 - 4] is a recently emerged nonlinearcontrol system design methodology for directsynthesis of nonlinear feedback controllers. Using aspecial form of the system dynamics, this approachpermits the designer to employ linear optimal controlmethods such as the LQR methodology and the H8design technique for the synthesis of nonlinearcontrol systems.

The SDRE design technique requires thedynamic model of the system to be placed in the statedependent coefficient (SDC) form. The SDC formhas the structure:

( ) ( )uxBxxAx +=&In its more general form, the SDRE technique allowsthe inclusion of disturbance terms [4]. If desired, thedesigner can augment this system by introducingintegral states and dynamic compensators to improvethe tracking and disturbance rejection characteristicsof the SDRE controller.

Note that the SDC form has the same structure asa linear dynamic system, but with the system matrixA and the control influence matrix B being functionsof the state variables. Reference 2 has shown that theSDC form can be derived for most nonlinear dynamicsystems using simple algebraic manipulations. Anapproach for numerically constructing the SDC formfrom a given nonlinear simulation model will bediscussed in the next section.

The matrices A(x) and B(x) evaluated at allvalues of the state vector x are assumed to be suchthat the system dynamics is controllable. Althoughthe original theory discussed in Reference 2 does notimpose this requirement, the need for fullcontrollability arises from the numericalconsiderations.

The second ingredient of the SDRE designtechnique is the definition of a quadratic performanceindex in state dependent form:

( ) ( )[ ]∫ +=∞

0t

TT dtuxRuxxQx21

J

The state dependent weighting matrices Q(x) andR(x) can be chosen to realize the desired performanceobjectives. In order to ensure local stability, thematrix Q(x) is required to be positive semi-definitefor all x and the matrix R(x) is required to be positivedefinite for all x.

Next, a state dependent algebraic Riccatiequation:

( )( ) 0)x(Q)x(P)x(B)x(RxB)x(P

)x(A)x(P)x(PxAT1

T

=+−

+−

is formulated and is solved for a positive definitestate dependent matrix P(x). The nonlinear statevariable feedback control law is then constructed as:

x)x(P)x(B)x(Ru T1−−=An excellent overview of the SDRE design techniquecan be found in Reference [4].

It may be observed that the crucial part of thecontrol law computation is the solution of the state-dependent Riccati equation. In rare situations, thisRiccati equation may be solvable in closed-form. Inmost problems, however, this equation will have tobe numerically solved at each sample instant.

A flowchart illustrating the steps involved in thecomputation of the SDRE control laws is given inFigure 1. At each sample, the state vector obtainedfrom feedback sensors or estimators are used tocompute the SDC matrices, which are then used tofind the state dependent gains. The product of thestate dependent gains and the state vector then yieldsthe control variables.

Thus, the two main steps in the SDRE nonlinearcontrol system design method are the computation ofthe SDC matrices A(x) and B(x), and the solution ofan algebraic matrix Riccati equation for P(x). Theremaining steps involve matrix inversion andmultiplication.

State FeedbackControl Law:u = - K(x) x

ComputeModel Matrices

A(x), B(x)

Formulate and SolveThe StateDependent

Riccati Equation for P(x) >0

ComputeState-Dependent Gain

K (x) = R-1(x) BT(x) P(x)

StateVector: xFrom the

Dynamic System

Off-Line Selection ofState & Control

Weighting MatricesQ(x), R(x)

ControlVector: u

To theDynamic System

Fig. 1. SDRE Control Law Computations

If the nonlinear system dynamics is given insymbolic form, it is possible to derive the SDC modelusing algebraic manipulations. However, in manypractical problems where the system model is givenas a computer simulation rather than in an analyticalform, numerical approaches will have to be employedto construct the SDC form. Next section will discussa numerical approach for constructing an SDC formfrom a given simulation model.

While the user may have a choice in how thesystem dynamics are transformed to the SDC form,the solution of the Riccati equation almost always

233

3

requires numerical approaches. Several reliablealgorithms have been advanced in the literature [8]for the solution of algebraic Riccati equations, two ofwhich will be reviewed in Section 3. Thesealgorithms require the use of numerical linearalgebraic methods. Computer codes for implementinglinear algebraic algorithms available in the publicdomain [11] formed the starting point for the presentwork. Highly optimized versions of these codecomponents were developed to assemble the SDREcontrol software.

3. Numerical Approach for Obtaining the SDCModel

It will be assumed that the dynamic system isgiven in the standard form:

( ) ( )u xgxfx +=&

Here, x is the state and u is the control vector. Notethat the control vector appears linearly in the systemdynamics. If the control variables appear nonlinearlyin the system dynamics, an input dynamiccompensator can be introduced to transform themodel into standard form [2, 4]. At any given valueof x, finding the SDC form

( ) ( )u xgx xAx +=&from the given nonlinear dynamic system requires thesolution of a system of n equations:

( ) ( )xfx xA =

The matrix A(x) has n×n elements. Since only nequations are available, additional relations must befound to solve for the elements of A(x). If the vectornonlinear function f(x) is available in analytical form,algebraic manipulations can be used to construct thematrix A(x). Reference 2 has advanced severalapproaches for the construction of the A(x) matrix.

However, if the model is specified in the form ofa computer simulation, a numerical approach willneed to be set up. Instantaneous SDCparameterization can be obtained by evaluating thevector nonlinear function f(x) using a set of linearlyindependent probe vectors n2 ......,, ζζ . As a practical

matter, since the behavior of the systemnonlinearities far from the current states are notexplicitly known, it is wise to choose probe vectorsthat are close to the current state vector.

The probe vectors can be constructed by addingsmall magnitude perturbation vectorsσ2, σ3, σ4,....σn, to the nominal state vector to yield aset of linearly independent vectors:

nn433222 x......,,x,x,x σζσζσζσζ +=+=+=+=The nonlinear function f(x) is next evaluated usingthese linearly independent vectors. Assemble thematrix equation

[ ] ( ) [ ]n2n2 .........x xA)(f.......)(f)x(f ζζζζ =At any given value of x, this linear matrix equationcan be solved for the elements of A(x). Since theprobe vectors and the state vector are linearlyindependent, this equation is well conditioned, andcan be solved using well-known linear algebraicmethods.

Note that the foregoing computations will haveto be carried out at every sample. The SDC matrixA(x) from these computations can next be used toformulate and solve the SDRE control problem. Asan aside, it is interesting to examine the relationshipbetween the numerical construction of the SDCmodel and the conventional Taylor seriesapproximation. If the perturbation vectorsσ2, σ3, σ4,....σn, are small, it can be found that:

0xat,xf

A =∂∂

≅

Note that this corresponds to the Taylor serieslinearization of the system dynamics about the origin.Thus, the present methodology for constructing theSDC model automatically reverts to Taylor serieslinearization of the system dynamics near the originof the state space. For the constant control influencematrix case, the present SDC parameterizationscheme preserves the controllability properties of thedynamic system near the origin.

Since the only restriction on the probe vectors isthat they be linearly independent, it is possible toconstruct an infinite variety of SDCparameterizations for a given dynamic system.

4. Efficient Algorithms for Solving AlgebraicRiccati Equations

As discussed in Section 2, main computationalsteps in the implementation of the SDRE techniqueare the computation of the SDC form of the systemdynamics and the solution of a high-dimensionalalgebraic matrix-Riccati equation.

The following subsections will review twonumerical techniques for solving Riccati equationsthat have been found to be useful in applications [7 -10]. These are:1. Solution using Schur-decomposition ofHamiltonian matrix2. Kleinman algorithm

The first is a direct method while the second isiterative. Generally speaking, direct methods arecomputationally faster than iterative methods,especially in poorly conditioned problems and incases where a good quality initial guess is notavailable. On the other hand, the computation andstorage requirements for a direct method can be morethan twice as much as that of an iterative method

234

4

because the former operates on a nn 22 ×Hamiltonian matrix for a Riccati equation of ordern . Iterative techniques can outperform directmethods if the starting solution is close to the finalsolution.

Techniques not examined in this paperinclude discrete-time transformation approach,spectral factorization, doubling algorithm,Chandrasekhar algorithm, square root algorithm,information filter algorithm, and the matrix signfunction algorithm. Some of these are discussed inReference 8.

4.1. Direct Solution Using the Hamiltonian Matrix(Schur Algorithm)

The solution of Riccati equation of order n canbe obtained in terms of the solution of a linearHamiltonian equation of order n2 . The Hamiltonianmatrix corresponding to the algebraic matrix Riccatiequation is defined as:

−−−=

−

T

T1

AQBBRAM . Define

−

=0II0

J .

The Hamiltonian matrix satisfies the condition:1TT JJMJJMM −−== .

M is assumed to have no imaginary eigenvalues.Given the stabilizability and detectability of thedynamic system, if λ is an eigenvalue of theHamiltonian matrix, so is λ− . Thus, there exists areal W such that

=−

2

11

00

MWWΛ

Λ

and 1Λ , 2Λ are real Jordan matrices such that thereal parts of all eigenvalues are respectively negativeand positive. Under these assumptions, the solution tothe matrix-Riccati equation can be written as

11121WWP −= .

Since 111111

T1 WWPBBRA −− −=− Λ , the eigenvalues of

1Λ represent the closed loop system modes. Note

that the matrix

=

2221

1211

WWWW

W consists of the

eigenvectors of the Hamiltonian matrix .MIt is numerically preferable to use a Schur

form

UL0LL

UM22

1211T

=

where L is a Schur form of M with 11L possessing

all negative real part eigenvalues, and the matrix Uis orthogonal. Then, the solution to Riccati equationis given by

T111

T12 )U(UP −= .

Since 1T1111

T11

T1 )U(LUPBBRA −− −=− , the

eigenvalues of 11L represent the closed loop system

modes.As is evident from the above discussion, two

distinct steps are involved in the Schurdecomposition approach. The first is the reduction ofa nn 22 × Hamiltonian matrix to an ordered realSchur form; the second is the solution of an nn ×linear matrix equation. An approximate estimate ofoperation counts required to solve the Riccatiequation using the Schur method is given inReference 8. That work indicates that the solution

requires approximately 375n floating pointoperations (FLOPS), where n is the dimension of thesystem. Missile autopilots typically involve fivestates and three controls [14]. Thus, about 10,000floating-point operations per second (10 MFLOPS)will be required to implement the SDRE techniquefor a missile autopilot at about 1 kHz sample rate.Implementing more advanced integrated guidance-control algorithms [15, 16] will require additional 2MFLOPS.

4.2. The Kleinman AlgorithmThe Kleinman recursive algorithm [9, 10] uses

an initial guess of the closed gain to obtain thesolution to the algebraic Riccati equation. Thisalgorithm is implemented as follows. Let 0K be such

that closed-loop system )BKA( T0+ has all

eigenvalues with negative real parts. Define iP , iKrecursively as:

QRKKP)BKA()BKA(P Tiii

TTi

Tii −−=+++

11

−+ −= BRPK ii

Then 1ii PP +≥ and PPlim ii

=∞→

. Further,

2i1i PPcPP −≤−+ , 0c > , implying that the

convergence of the algorithm is quadratic. Initialguess of the closed loop system gain can be generatedusing any available technique. For instance, poleplacement can be used as the starting point.Alternately, the Schur algorithm discussed in theprevious section can be used to obtain a first estimate

0P of P, and then the algorithm can be initialized

with 100 BRPK −−= .

The Kleinman algorithm requires approximately6n3 floating point operations per iteration. Dependingupon the initial guess, 10 or more iterations may berequired to obtain the solution. Thus, the Riccatiequation solution can be accomplished in 60n3

235

5

floating point operations. A better quality initialguess can produce faster convergence of thealgorithm. It has been shown [9, 10] that thealgorithm will converge to the true solution, if thestarting guess of the gain matrix provides a stableclosed-loop system.

In the case of SDRE computations, even if agood quality guess is not available initially, thequality of initial guess will improve as thecomputations proceed.

5. Coding and Evaluation of ARE SolversThe algorithms for Riccati equation solution

described in Sections 4.1 and 4.2 formed the basis forthe development of the software for implementing theSDRE control laws. The Schur direct method and theKleinman iterative method were coded in ANSI-C.Basic numerical algebraic routines required for theimplementation of these methods were obtained fromthe LAPACK [11] software package, and wereoptimized for improved performance.

LAPACK can solve systems of linear equations,linear least squares problems, eigenvalue problemsand singular value problems. It can also handle manyassociated linear algebraic computations such asmatrix factorizations or estimating conditionnumbers. LAPACK software supercedes the well-known LINPACK and EISPACK software packages.This software forms the basis for several well-knowncommercial computer programs.

Algebraic Riccati equation solution codes weredeveloped using Microsoft Visual C/C++ compiler,Version 6.0 for WindowsNT operating system. Twoversions of the software were assembled. The firstversion is for use in standalone speed evaluations,while the second version is for implementation in theMATLAB-Simulink® environment. The latter wasused to compare the accuracy of the presentimplementations with those from MATLAB, and wasalso useful for algorithm evaluation on hardwareplatforms such as the dSPACE-DS1103 PowerPC604e machine.

5.1. Performance EvaluationThe Riccati solution algorithms discussed in the

foregoing section were next evaluated for accuracyand speed. The accuracy of Riccati solutions wasascertained by comparing the results from the presentimplementation to those obtained from MATLAB.

Controllable systems of several sizes were usedin these evaluations. In every case, the direct solutionmethod produced results that were indistinguishablefrom the MATLAB solution. The accuracy of theiterative solution is essentially a function of thespecified tolerances. However, in all cases, five to six

iterations were sufficient to produce resultscomparable to the direct method.

Next, in order to evaluate the speed of executionof these algorithms, a missile autopilot examplecontaining five states and three control variables wasselected. This problem size is comparable to therealistic flight control system for an advancedweapon system. Executable programs generated fromthe code described in the previous sections were thenused to solve this problem on three differentmachines. The first two employed commercialversions of the WindowsNT operating system, whilethe third employed a proprietary operating systemfrom dSPACE Inc. The WindowsNT evaluationswere conducted on a Pentium II 300 MHzworkstation, and a Pentium III 450 MHz workstation.The dSPACE proprietary operating systemimplementation was evaluated on a PowerPC 604eprocessor (DS1103) from dSPACE Inc [13]. ThedSPACE PowerPC board is used extensively inautomotive control system development. Speedcomparisons for the Riccati solutions on these threemachines are presented in Table 1.

Processor/Algorithm/Operating Sys. ProcessorP-II, 300MHz/Schur/NT 2.38 msP-III, 450MHz/Schur/NT 1.57 msPPC 604e/Schur/dSPACE 1.56 msP-II, 300 MHz/Kleinman/NT 1.46 msP-III, 450 MHz/Kleinman/NT 0.91 msPPC 604e/Kleinman/dSPACE 0.476 ms

Table 1. Execution Times for theTwo Riccati Solution Methods

Computing times are given for one full-cycle of thesealgorithms. The computing time for the Kleinmanmethod is given for three iterations. Note that thecomputational times deliver speeds far in excess ofthose required for implementing high-performancemissile flight control systems. Since the maincomputational burden in implementing the SDREtechnique is in the solution of the algebraic Riccatiequation, these speed evaluations demonstrate thefeasibility of implementing the SDRE technique oncommercial, off-the-shelf processors in real-time.Moreover, analysis of the C code has revealed that itmay be possible to gain an additional 25% speedimprovement by further refining the code.

6. ConclusionsThe present research was motivated by the

application potential of the SDRE nonlinear controltechnique in practical nonlinear control problems.The state dependent Riccati equation (SDRE)technique is a recently developed methodology for

236

6

designing control laws and estimation algorithms forgeneral nonlinear dynamic systems. Although itspotential is well recognized, the industry acceptanceof the technique has been slow. The main reasons forthis are: a) Unlike linear control technology, theSDRE approach requires advanced numericalmethods for its implementation, which are notcurrently available off-the-shelf, b) the perceptionthat this technique may not be computationallyfeasible for real-time implementation on commercialoff-the-shelf processors.

Using software based on the Schur algorithm andthe Kleinman method, present research showed thatSDRE control laws can be implemented at speeds upto 2 kHz sample rates on problems of the sizecommonly encountered in missile flight controlapplications, using commercial, off-the-shelfprocessors. The computational accuracy of thepresent implementations has been shown to becomparable to other commercial software packages.

References[1] Cloutier, J. R., “Adaptive Matched AugmentedProportional Navigation”, Proceedings of the AIAAMissile Sciences Conference, Monterey, CA,November 1994.[2] Cloutier, J. R., D’Souza, C. N., and Mracek, C.P., “Nonlinear Regulation and Nonlinear H8 ControlVia the State-Dependent Riccati EquationTechnique”, Proceedings of the InternationalConference on Nonlinear Problems in Aviation andAerospace, Daytona Beach, FL, May 1996[3] Mracek, C.P. and Cloutier, J. R., “MissileLongitudinal Autopilot Design using the StateDependent Riccati Equation Method”, Proceedingsof the 1997 American Control Conference, June 4 - 6,Albuquerque, NM.[4] Cloutier, J. R., “State-Dependent Riccati EquationTechniques: An Overview”, Proceedings of the 1997American Control Conference, June 4 - 6,Albuquerque, NM.[5] http://www.microsoft.com/windows/embedded/.[6]http://developer.intel.com/technology/. [7] Laub, A. J., “A Schur Method for SolvingAlgebraic Riccati Equations”, IEEE Transactions onAutomatic Control, Vol. AC-24, No. 6, December1979.[8] Anderson, B. D. O., and Moore, J. B., OptimalControl: Linear Quadratic Methods, Prentice Hall,1990.[9] Kleinman, D. L., “On an Iterative Technique forRiccati Equation Computation”, IEEE Transactionson Automatic Control, Vol. AC-13, No. 1, February1968, pp. 114-115. [10] Anderson, B. D. O., “Second-order convergentalgorithms for the steady-state Riccati equation”,

International Journal of Control, Vol. 28, No. 2,1978.[11] Anderson, F., et al, LAPACK User’s Guide,Society for Industrial and AppliedMathematics(SIAM), Philadelphia, PA, August 1999.(http://www.netlib.org/lapack)[12] Anon, MATLAB User’s Manual, TheMathWorks, Inc., Natick, MA, 1998.[13] Anon, Solutions for Control-dSPACE Catalog1999, dSPACE Inc., 22260 Haggerty Road, Suite120, Northville, MI 48167. [14] Zarchan, P., Tactical and Strategic MissileGuidance, American Institute of Aeronautics andAstronautics, Reston, VA, 1997.[15] Menon, P. K., Iragavarapu, V. R., andOhlmeyer, E. J., "Integrated Design of Agile MissileGuidance and Control System”, 1998 MissileSciences Conference, November 17-19, Monterey,CA.[16] Menon, P. K., Dewell, L. D., and Sweriduk, G.D., “Integrated Guidance-Autopilot Designs for Anti-Jam Operation of Precision Munitions”, Phase ISBIR Project Final Report being Prepared under AirForce Contract No F08630-99-C-0040, December1999.

237