optimal control a review of theory and practice

AIAA JournaVOLUME 3 NOVEMBER 1965 NUMBER 11

Optimal Control: A Review of Theory and Practice

BERNARD PAIEWONSKYInstitute for Defense Analyses, Arlington, Va.

I. Introduction

Scope of Review

THE objectives of this review are to summarize and toevaluate the present state of knowledge concerning opti-

mal control synthesis. The primary areas of interest areflight mechanics and flight control.

The design of optimal stabilization and control systems,the determination of optimal flight paths, and the calculationof optimal orbital transfers have a common mathematicalfoundation in the calculus of variations; this review is limitedto problems requiring a variational treatment. The selectionof optimum vehicle shapes and the determination of optimumstaging and propellant loadings for rockets are not includedin the scope of this work although they are variational prob-lems.

A general discussion of optimization techniques drawsupon the accumulated experience and research results ofmany workers in different branches of engineering and appliedmathematics. In recent years, there has been a confluenceof control research in aeronautical, electrical, mechanical,and chemical engineering. The research efforts in specializedtechnical fields have produced results of general interest, butthe splintering of the literature according to old traditionshas sometimes retarded the application of the results.

A principal task of this review is to discuss the synthesisof closed-loop optimal controllers. This review will notmake a sharp distinction between optimal guidance andoptimal control because the mathematical problems are nearlyidentical even though the time scales of the system beingcontrolled may differ by several orders of magnitude. A

synthesis procedure for an optimal system is a set of designrules whose application results in a closed feedback loopcomprised by the system being controlled and a computerto process the output measurements and to determine theoptimal control law. This is illustrated in Fig. 1, which is ablock diagram encompassing a very wide range of diverseproblems extending from the optimal correction of inter-planetary trajectories to optimal flight controllers forairplanes.

These two apparently dissimilar technical problems arenevertheless nearly identical from a system synthesis view-point. The same basic design steps are required in each case.These include the selection of a performance criterion, theestimation of the statistical properties of noise and randominputs, the determination of the system state from noisydata, and the computation of the optimal control law. Eachof these items will be taken up again later on.

Scope of the Literature Survey

The survey is oriented towards aeronautical or astronauti-cal applications, although papers from other branches of thegeneral engineering and scientific literature are included.

Several texts have been published which give comprehen-sive coverage to certain aspects of this subject. Much es-sential material is concentrated there, and the problem ofsearching for individual papers in a large and widely scatteredliterature is reduced. The reader seeking more detaileddescriptions of the material contained in this review shouldconsult the books by Bellman,1 Chang,2 Lawden,3 Leitmannet al.,4 Miele,5 Pontriagin et al.,6 as well as conference pro-ceedings such as those given in Refs. 7 and 8.

Bernard Paiewonsky received a B.Sc. in mathematics from the Massachusetts Institute ofTechnology, an M.A. in mathematics from Indiana University, and an M.S.E. and Ph.D. inAeronautical Engineering from Princeton University. Dr. Paiewonsky came to the Institutefor Defense Analyses in 1964 as a member of the Senior Technical Staff. Prior to this he wasa consultant at Aeronautical Research Associates of Princeton, Inc., which he joined in 1958.He was a Legendre Fellow at Princeton in 1957-1958 and was a Lieutenant in the U. S. AirForce at the Flight Control Laboratory at Wright Field from 1955 to 1957. Dr. Paiewonskywas a senior member of the American Rocket Society and is a member of the AIAA, B.I.S., andthe AIAA Astrodynamics Committee. He was an associate editor of the ARS Journal andAIAA Journal from 1959 to 1964.

Received June 11, 1965; revision received September 14, 1965. A portion of the work on which this paper is based was carried outwhile the author was at Aeronautical Research Associates of Princeton (ARAP). The work at ARAP was supported in part by theU. S. Air Force Flight Dynamics Laboratory under contract AF 33(657)7781 and in part by the U. S. Navy (BuShips) Contract Nonr3703(00) administered by the David Taylor Model Basin (DTMB). I am happy to acknowledge the valuable assistance of Peter Woodrowof ARAP and John Mclntyre, formerly at Princeton University and now at North American Aviation, in collecting, examining, anddiscussing the material used in parts of this report. This review draws on the results of a 1963 survey presented in AF TDR-63-239and on the results of research on The Bounded State Space Problem (ARAP Report #60, prepared for DTMB). Thanks are due toPatricia Maines of the Institute for Defense Analysis (IDA) who typed the manuscript.

1985

Dow

nloa

ded

by B

eiha

ng U

nive

rsity

(C

NPI

EC

- X

I'AN

BR

AN

CH

) on

May

12,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/3.3

307

1986 B. PAIEWONSKY AIAA JOURNAL

Disturbances

Commands

Estimated state variables

Fig. 1 Closed-loop optimal control.

Section II contains a brief historical account of optimizationresearch in areas of aeronautical and astronautical interest.These areas include 1) orbital flight mechanics and rocketsteering, 2) aircraft performance, 3) flight controllers andautomatic stabilization systems in general, and 4) optimalcontrollers.

In Sec. Ill, there is a discussion of certain mathematicalaspects of optimization. The discussion emphasizes theproblems of inequality constraints on the control and on thestate variables. This is followed by Sec. IV, on stochasticoptimization, e.g., problems with random inputs, noisy meas-urements, and imperfectly specified system parameters.

Variational problems usually lead to differential equationswith split boundary conditions. Section V on two-point bound-ary problems describes methods of successive approximationsfor the usual situation where exact solutions in closed formare unavailable.

Methods for solving the two-point boundary problems fre-quently require the solution of an ordinary minimizationproblem with several independent variables. Section VI isdevoted to this, and some examples are given. In Sec. VII, thedesign of an optimal control closed-loop system is discussed.The final section is a list of the cited articles.

II. Historical Aspects and Areas of InterestThe history of optimization in flight mechanics and con-

trol theory is long, and the literature is extensive. Thissection outlines some of the problem areas already explored.It is not intended to be a complete catalog of all past work.It should be viewed as a selection of research papers chosento show the relations between adjacent areas of engineeringanalysis.

Rocket Steering Problems

The question of the optimal mode of operation for rocketswas raised over 45 years ago by Goddard,9 who was inter-ested in maximizing the altitude of sounding rockets. Thisproblem, and other versions of it, have been treated subse-quently by Malina,10 Tsien and Evans,11 Ross,12 Miele,13

Leitmann,14 and Garfinkel.15 There are several cases ofimportance, and these differ from each other mainly in theways in which the drag is assumed to vary with velocity andthe atmospheric density varies with altitude.

The general problem of optimal interorbital transfer of arocket is still unsettled. In 1925 Hohmann16 discussedorbital transfers and arrived at the conclusion that for co-planar circular orbits the cotangential ellipse with impulsiverocket thrust is optimal with respect to fuel. Other in-vestigators in subsequent studies conjectured that", in prob-lems of this type (i.e., chemical rockets with high thrustcapability), the thrust level was either a maximum or a mini-mum, with no extremal arcs of intermediate thrust levels.Barrar17 has shown that the Hohmann transfer is the mini-mum two-impulse transfer.

Lawden,18 however, has found a spiral extremal arc em-ploying intermediate thrust levels. Kopp and Mover19

have proved that the spiral arcs are nonoptimal in the time-open case. McCue and Bender20 have made numericalstudies and observed that, in the cases considered, the char-acteristic velocity for the spiral arc exceeds that for an im-pulsive transfer with the same boundary conditions.

When an impulsive mode of transfer is postulated, theoptimal locations, directions, and magnitudes of the im-pulses must be determined. There exists a very large bodyof references on this problem. Some of these results can befound in the papers of Lawden,21~27 Edelbaum,28'29 Municket al.,30 Altman and Pistiner,31 Hoelker and Silber,32 Fimple,33

Gobetz,34 Horner,35 and Templeman,36 among others.37"43

The determination of optimal steering and thrusting pro-grams for separately powered rockets is also a problem of con-tinuing interest. (Edelbaum44 has examined the use of highand low thrust in combination.) The optimal operation ofnuclear rockets has been studied by Leitmann,45 Bussard,46

Irving and Blum,47 and Wang et al.48 Optimal trajectoriesfor interplanetary operations with continuous thrust systemshave been investigated by Melbourne,49 Melbourne andSauer,50 Fox,51 Kelley,52 Hinz and Moyer,53 and MacKay.54

The important problems of optimal rocket boosting instrong gravitational fields has been studied by Hibbs,55

Fried,66 Bryson and Ross,57 Ross,58 Miele,59 Stancil andKulakowski,60'61 Breakwell,62 Leitmann,63 and many others.The optimal spacing of corrective thrusts on interplanetaryflights has been studied by Lawden,64'65 Breakwell and Strie-bel,66 and Denham and Speyer.67 The field of optimal rockettrajectories has been reviewed by Lawden,68 Leitmann,69

Miele,70 and Bergqvist.71

Fuel-optimal and time-optimal rendezvous maneuvershave been studied by Goldstein et al.,72 Kelley and Dunn,73

Hinz,74 Mclntyre arid Crocco.75 The thrust and fuel con-straints were applied separately. Paiewonsky and Woodrow76

treat a linearized time-optimal rendezvous in three dimen-sions with bounded thrust and limited fuel by application ofNeustadt's method.77 Fuel-optimal lunar landings arestudied by Meditch78 and Hall et al.79 Neustadt has recentlyprepared a general treatment of minimum-fuel space trajec-tories which includes many of the foregoing special problems.80

Aircraft Performance

Airplane performance is the subject of many optimiza-tion studies. The objective of the optimization is the im-provement in the operational capabilities of the aircraft withrespect to range, takeoff and landing distances, time-to-climb, etc. The minimization of the time-to-climb wastreated originally by a quasi-steady analysis based on poweravailable and power required.81 Lippisch82 modified thequasi-steady analysis by including the effect of accelerationsalong the flight path. The time-to-climb problem was sub-sequently treated by Lush,83 Garfinkel,84 Miele,85 and Cicalaand Miele.86 In 1949 Hestenes87 formulated a very generalproblem in the calculus of variations and applied it to two-and three-dimensional time-optimal climbs. This work in-cluded an analysis of inequality constraints, e.g., bounds onthe angle of attack. Mengel,88 using Hestenes7 work as astarting point, recast the equations in a form better suitedto computational techniques and carried out a series of studieson an analog computer. The endpoints of the computedpaths were observed to be very sensitive to variations in theinitial choices of Lagrange multipliers, and it was very diffi-cult to satisfy the boundary conditions of the problem.

Kelley studied the optimal climb problem using in onecase a classical variational approach,89 and more recently agradient method.90 The latter study includes bounds on thestate variables as well as on the controls. Bell has reviewedatmospheric flight optimization for the 1945-1960 period.91

The maximization of aircraft range was also treated origi-nally by quasi-steady techniques, e.g., the well-known Breguet

Dow

nloa

ded

by B

eiha

ng U

nive

rsity

(C

NPI

EC

- X

I'AN

BR

AN

CH

) on

May

12,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/3.3

307

NOVEMBER 1965 OPTIMAL CONTROL: A REVIEW OF THEORY AND PRACTICE 1987

formulas.81 The maximization of range for hypervelocityvehicles has been studied by Arens92 and Bell93 and Miele.94

Energy management, range control, and aerodynamic heat-ing associated with space vehicles returning to the earth havebeen investigated by Bryson et al.,95 Levinsky,96 and others.97

Bryson et al.98 have also described the application of an optimalterminal controller to the tasks of tracking nominal re-entrypaths. Optimal ship routing and optimal aeronavigationare the subjects of many fundamental investigations. Zer-melo's minimum-time steering problem,99 is well known andis often used today as a prototype or example for controlsynthesis ideas.100-101

Minimum-time steering including the action of winds orocean currents also has been studied by Von Mises,102 Levi-Cevita,103 Faulkner,104 and Haltiner et al.105

An exceptionally fine account of this subject appears inDeJong's monograph.106 This 1956 monograph presents aninteresting development of the variational problem from theviewpoint of isochronal wave fronts and the Hamilton-Jacobitheory. It is regrettable that this work did not receivewider circulation at the time of its publication, as it certainlywould have stimulated research along fruitful lines in controltheory and flight mechanics.

Flight Controllers

Flight controllers, autopilots, and stability augmentationsystems for manned aircraft are ordinarily designed usinglinearized analyses. Feedback gains or adjustable param-eters are manipulated to obtain satisfactory system transientresponse to control inputs and to gust inputs. Linear systemssubject to deterministic inputs are frequently optimized withrespect to transient response, and auxiliary criteria basedon transient rise time, peak overshoot, settling time, band-width, etc., are often used.107"109 These characteristics of thesystem response depend upon the locations of the poles andzeros of the system transfer function. Kalman et al.110'111

point out that a transfer function is not always adequate todescribe the behavior of a system. The design of optimallinear feedback controllers has been thoroughly explored,and comprehensive reviews of this aspect of control theoryhave already appeared.112-113

The design of an optimal controller requires the selection ofa performance criterion. In many aeronautical applications,the selection of a performance criterion is based on realphysical considerations, e.g., payload, final velocity, total heatabsorbed, etc. There are studies of optional controllersthat are not based on extremizing physically meaningfulquantities; the result may nevertheless be a good controlsystem from the standpoint of the pole and zero locations ofthe transfer function.114-115 Sometimes it is possible to givea physical meaning to these criteria, but these appear to beartificial. The minimization of these functionals may beviewed as a helpful device, which selects' a set of~ systemparameters producing a stable system with a good transientresponse, even though the physical significance of the optimi-zation criterion is not obvious at all.

The ultimate decision on the quality of the flight-controlsystem in the case of manned aircraft is, of course, deter-mined by pilot opinion. The relationship between general-ized performance criteria for optimal controllers, aircrafthandling qualities, and pilot opinion is still an open questionfor all but the simplest systems and even then only for certainspecial piloting tasks. The greatest difficulty in designingan optimal controller for a manned vehicle lies in finding outwhat the pilot really wants. A realistic mathematical modelfor the pilot has yet to be developed, although research on thecharacteristics of human pilots has been carried on for manyyears.116 A recent review117 shows that progress has beenachieved in making a servomechanistic model for the pilotin certain tracking and pursuit tasks. In these studies, thehigher cognitive processes are not considered at all; the pilot

is treated as a quasi-linear dynamical element in a closed-loop servomechanism. This approach has been appliedsuccessfully to the elimination of pilot-induced oscillations118

and to the improvement of pilot-aircraft systems, e.g., carrierlandings.119

Nonlinear autopilots that are not optimal but whose de-signs are based on the ideas of optimal control have beendescribed in the literature. (See, for example, the works ofPassera,120 Schmidt, et al.,121 and Flugge-Lotz.122)

There have also been a number of studies of nonoptimalpredictive terminal controllers. The predictive controllersbear a strong resemblance to the optimal controllers underdiscussion because two-point boundary problems occur ineach case. Studies have been made of the application ofthese controllers to re-entry and automatic landing prob-lems. 123~129

The design of optimal autopilots for aircraft and missilessubjected to random inputs has received considerable atten-tion. The minimum mean square error criterion is oftenused, but this is not the only one that has been considered.113

If the system is linear and if the noise has stationary sta-tistical properties and is Gaussian, then the mean squareerror is easily found in terms of the input power spectraldensity or correlation function. Minimization problems ofa similar nature occur in communication research, and anextensive theory of optimal filtering and prediction is avail-able based on the work of Weiner130 and subsequent investi-gators.131

An approach frequently used in autopilot design postulateslinear feedback of the state variables and seeks values of thefeedback gains which minimize the penalty criterion, e.g.,expected value of rms loads. The omission of control de-flection limitations or a cost-of-control term from the problemformulation leads to a trivial result.

The optimization of nonlinear control systems with randominputs and the design of optimal linear systems subjected tonon-Gaussian random inputs requires an analysis of theprobability distributions of the output variables. If theoutput variables form a Markov process, then the Fokker-Planck partial differential equation can be used to generatethe time-varying probability densities.132 This techniquehas been used by Ruina and Van-Valkenberg133 in a studyof radar tracking systems. Kalman and Bucy134 havestudied the combined problems of optimal control and optimalfiltering. The results of these studies are being applied tothe guidance of space vehicles.135"137 This technique hasalso been investigated by Battin.138

The stochastic optimization of a terminal controller witha saturating element has been examined by Booton.139 Thedesign of optimal autopilots subjected to random inputs wasstudied by Schwartz140 and others. Schwartz appliedBooton's technique to the terrain clearance problem for low-altitude aircraft flight with a g limit.

The loads on aircraft in flight through turbulent air can beameliorated by the use of control manipulations that changethe aircraft's response characteristics. These correctivecontrol motions may be due to the pilot or to an automaticcontrol system. Measurements of atmospheric turbulencehave been made, and the general shapes of the power spectraare known, although the level of intensity depends upon thelocal meteorological situation. The problem of high tran-sient loadings due to turbulence is particularly severe for high-speed, elastic aircraft flying near the ground. Some opti-mization studies have been done in this area,141-142 but theproblem is not at all solved; in fact, the surface has hardlybeen scratched.

A closely related problem is the control of large flexiblebooster rockets flying through strong wind shear.143"145 Thedifficulty here is the limited control moment available for loadrelief and artificial stabilization. This is often aggravatedby the coupling of the elastic body bending modes and therigid body modes through the control system and the sensors.

Dow

nloa

ded

by B

eiha

ng U

nive

rsity

(C

NPI

EC

- X

I'AN

BR

AN

CH

) on

May

12,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/3.3

307


The optimal attitude control of a tumbling satellite isusually treated in two steps. The body is first brought torest by application of an optimal control that stops the rota-tion; the correct orientation is then sought by control ac-tions (not necessarily optimal) that turn the body from oneattitude to another. The optimization of the combinedtasks (the rotational rates brought to zero at the proper ori-entation with minimum fuel or in minimum time from anyinitial condition) is very difficult, and the synthesis problemhas not yet been solved. Athans146 and Lee147 have produceduseful results on the problem of stopping the tumbling.

Automatic stabilization systems (nonoptimal) for earthsatellites and space vehicles have been designed based onapproximations derived from optimal control theory. Thecontrols try to keep the attitude errors within specified boundsand reduce the fuel or power expenditure.148"150 Thesecontrol systems may be operating with a limit cycle close tothe error null position, and component imperfections, e.g.,relay and valve hysteresis, gear backlash, propellant ignitiondelays, friction, deadzones, etc., become important and mustbe taken into account. Satellite attitude control techniquesare reviewed in detail by Ergin et al.150 and Roberson.151 Ashorter review by Ergin has recently been published.152

Single axis attitude regulation is reviewed and discussed inRef. 153.Optimal Controllers

The discussion to this point has drawn mainly on aero-nautical sources. This section outlines the development ofoptimal control research in other areas.

The historical development of control theory in engineeringemphasized stability. Although stability is always a con-sideration, it is not the main concern here. Optimizationproblems in the engineering literature are frequently treatedseparately, aside from stability considerations, and areordinarily formulated in terms of calculus of variations.

The optimal control papers in the electrical and mechanicalengineering literature can be placed into two broad categories.The first category consists of abstract studies not directedtowards any specific system or device. In the second cate-gory are studies of more concrete applications, e.g., high-speed servos, relay controllers,154"156 and space vehicleattitude controllers.157'158 The latter group may containexperimental tests, whereas papers in the first categoryare usually devoid of hardware considerations. The scopeof the studies in both categories includes time-optimal con-trol,159"161 optimal regulation,162"164 optimal sampled datasystems,165'166 and optimal stochastic systems.167"169 Ap-plication of optimal controllers to chemical processes hasalso been studied extensively.170"172

The problem of finding optimal locations of static operatingpoints or "set-points" for chemical processes led to researchon "optimalize^."173-174 These devices (analog or digitalcomputers) are mechanized hill-climbers that. search formaxima of functions of several variables in a systematicway. The algorithms used to speed up the hill climbingprocesses are of interest, but static optimization will not betreated here at all. We note in passing that early originalwork on optimalizers was done by Draper and Li175 and ap-plied to the control of reciprocating engines for aircraftThis work is also described in Tsien's book.176

Early work on engineering problems with bounded con-trols assumed a relay-type operation, and a large engineeringliterature exists for bang-bang or relay controllers. Therelay, a deceptively simple device, can be employed easilyas a logical element in an automatic controller. Some engi-neering studies177"179 of relay controllers begin with thestipulation that a two-position or three-position control beused. This requirement is imposed prior to any optimalityconsiderations. The relay switching function can be a linearcombination of state variables, or it can be a nonlinear (notnecessarily optimizing) function of the state variables.

Many studies have been carried out to determine thebehavior of closed-loop systems containing relays.180-181

The emphasis has been on the control of time-invariant linearprocesses with relays employing linear switching laws; e.g.,

Xj + (1)

The over-all behavior of the closed-loop system depends onthe slope of the switching hyperplane (determined by the c/)and the elements of the matrix A and vector b. The func-tion "sgn" is discontinuous, and consequently the system doesnot possess the qualities needed to insure existence of smoothsolutions from a mathematical standpoint. The solutionsof this system of equations can exhibit peculiar behaviorknown as "chattering" and "endpoint." The behavior ofsystems with relay-like discontinuities has been studied ex-tensively by Flugge-Lotz,182 Andre and Seibert,183-184 Ayzer-man and Gantmacher,185 Filippov,186-187 and Tsypkin.188

The fact that actual relays have hysteresis, dead zones,and time delays prevents the endpoint phenomenon frombeing observed, but in its place, there is a chattering phe-nomenon and a limit cycle about the error null position.These results are fairly well known, and the fundamentalideas are summarized in Tsien's book176 and in more recenttexts.

The development of high-speed switching amplifiers andhigh-speed relays has reduced the effects of componentimperfections, but the use of bang-off-bang jet reaction con-trols with valve dead zones, time delays, and hysteresis hasreintroduced these problems.189"191

Many engineering studies of relay controllers are directedat second-order systems because the phase-plane permitsgraphical synthesis techniques to be developed easily. Thegraphical phase-space approach has been extended to somethird- and fourth-order systems by studying the projectionsof the phase motions onto coordinate planes.192 The re-sulting techniques are relatively complicated, and onlylimited applications have been attempted. Bass193 pointedout that synthesis of stable relay controllers for systems oforder higher than two could be accomplished via Liapunov'ssecond method. Kalman and Bertram discuss this in detailand give examples.194

Some effort has been directed at physically realizing non-linear switching functions by means of diodes and otherdevices.195-196 Attempts to improve the time response ofservomechanisms by adding nonlinear elements to the feed-back circuit have been made by McDonald,197 Schieber,198

Ku,199 and many others.The 1961 article by Flugge-Lotz200 provides an excellent

review of relay controllers; her monograph201 provides acomprehensive and detailed account of a class of discontinu-ous controllers.

An older review by Kazda202 provides a critical survey ofthis area up to 1957. The work on second-order systems be-ginning with McDonald's 1950 paper203 is thoroughly dis-cussed.

The optimization of linear systems with bounded controlsand limited control effort is important to the engineer aswell as to the mathematician because the linearized versionsof many physical problems can be easily forced into thisgeneral formulation.

Time-optimal control is one of the more interesting ofthese problems because it is actually possible to get solutionsin some cases, and these solutions do provide insight intomore general aspects of optimization. The literature ontime-optimal control was reviewed in 1963 by Kreindler.204

The problem is usually stated as follows. Given a dynami-cal system of nth order described by a linear time-varyingvector differential equation,

Xi = AH (t)xj + Bik (f)uK (2)find a piecewise continuous control vector w(t), which

Dow

nloa

ded

by B

eiha

ng U

nive

rsity

(C

NPI

EC

- X

I'AN

BR

AN

CH

) on

May

12,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/3.3

307


satisfies the following requirements: 1) u lies in U, where Uis a compact, convex set containing the origin as an interiorpoint, e.g., Uk\ < 1, and 2) the system is transferred froman initial state Xi (0) to a desired terminal state #»• (T) inthe least possible time, by the application of u°(T). Thecontrol may also be required to satisfy an integral constraintof the form

<p(u)k-dt < I

in addition to, or in place of, requirement 1.Bushaw205 obtained solutions for the second-order time-

optimal regulator problem [x(T) = 0], u\ < 1 by usinggeometrical constructions in the phase plane. These resultsprovided a rational synthesis procedure, for the first time,for systems with oscillatory characteristics. Nonoscilla-tory, stable, second-order systems can be brought to restat the origin from any initial phase point by a bang-bangscalar control having at most one change of sign. Oscilla-tory systems may require a large number of switches; thisdepends on the size of the initial errors in velocity and posi-tion. It may not always be possible to bring the state to theorigin if the system is unstable.

Bellman, Glicksberg, and Gross published a topologicalapproach to the time-optimal regulation problem in n di-mensions.206-207 The main results of their study can be sum-marized as follows: If all solutions of x = Ax go to zeroas t -*- co? then there is a time-optimal control vector u°(t),and it is bang-bang, i.e., Uk° = ±1. If all of the character-istic roots of A are real, distinct, and negative, then thereexists a minimizing control vector u°, uk° = 1, and uk°changes sign at most (n — 1) times where n is the order ofthe system. These results are very important, yet they donot say anything about systems with complex eigenvalues orreal nonnegative eigenvalues. Bushaw did not impose theserestrictions, but he was looking only at second-order systems.

LaSalle208 extended these results to nth-order time-varyingsystems and showed that the control is bang-bang andunique (a.e.) if the system (2) is normal, i.e., no componentof the expression n-X"1^) B(2) vanishes over an interval offinite length for any vector n.* If the system is normal thenthe optimal control as a function of time is u° = — sgn [tiX"1^)B(£)]. Similar results were obtained by Gamkrelidze209

for the autonomous time-optimal control problem. Thisproblem was treated via the maximum principle by Pontri-agin et al.6 and Rozonoer.211 Krasovskii,212 Kulikowski,213

and others214"216 applied functional analysis to the time-optimal problem and obtained formal solutions. Kulikowskiobserved that some optimal control problems can be solvedby application of the theory of approximations217; e.g., solu-tions for the switching times were obtained as zeros ofTschebychev polynomials.

The time-optimal problem was formulated by Desoer218

using the calculus of variations. The results of the differentformulations are nearly identical. In each case, the time-optimal control of Eq. (1) is bang-bang, and the switchingtimes for the components of the control vector are the zerocrossing times of a function formed from adjoint equations, i.e.,u° = — sgnnX~1(OB0). This fact was observed earlier byBass.219 The initial conditions for the adjoint, in effect,determine the switch times; the boundary conditions aresatisfied by applying the appropriate initial conditions to theadjoint. These initial conditions are generally unknown andmust be found for each initial state point. The adjointvariables (also called costate variables) correspond to theLagrange multipliers of the variational formulation. Neu-stadt,77 Gamkrelidze,221 Krasovskii,222 Eaton,223 Knudson,162

Ho,224 and many others have proposed methods for findingthe initial conditions of the adjoint for the bang-bang prob-lem. Neustadt's method appears to be the best known of

* This rules out singular subarcs.

these, and it has been applied to rocket steering,225 rendez-vous,76 midcourse guidance,226 and lunar landings.78 Neu-stadt220 has extended this method to include minimum effortcontrol where the effort is defined by an integral of a func-tional of the control, i.e.,

<p(u)dt < I

Time-optimal problems with fuel constraints as well as mini-mum fuel problems can be solved in this way.

The synthesis procedures listed so far are based on con-tinuous models of the process being controlled. There aremany situations in which the nature of the problem makes itdesirable or necessary to use a discrete model. The modelmay be based on sampling a continuous system, or it mayarise from a process that is properly described by differenceequations. This is an important subject area because it isquite likely that major applications of closed-loop optimalcontrol in flight mechanics will appear in a sampled data ordiscrete form. The use of digital and hybrid analog-digitalcomputers leads in a natural way to a discrete formulation.

Closed-loop optimal controllers that repetitively solve anopen-loop continuous problem between sampling instantsobviously fall into the class of discrete optimal systems.

The optimal spacing of interplanetary corrective thrust,mentioned earlier, is a problem in this category. Discretetime-optimal problems have been studied by Desoer andWing,227 Neustadt,228 Polak,229 and others. There is a largeliterature on sampled data systems which will not be re-vie wed here.230-231

A complete and consistent theory of discrete optimalstochastic feedback systems has not yet been developed,although considerable progress has been made in solvingprerequisite deterministic problems. Some aspects of thisare discussed in the section on stochastic optimal control.

III. Mathematical Treatment of Optimization

Introduction

Mathematical formulations of optimal control problemshave been made by Hestenes,87 Warga,232 Marcus and Lee,210

Breakwell,62 Krasovskii,222 Bellman,1 LaSalle,208 Neustadt,220

Pontriagin,260 and many others. Many of the papers aredirected at the time-optimal problem for linear systems withbounded controls. Hestenes233 has recently reviewed thevariational formulations of optimization theory.

Optimization problems containing a state space inequalityconstraint are by no means new. Weierstrass' lectures, re-ported by Bolza,234 develop the usual necessary conditionsplus the "corner" conditions for the problem of Lagrange intwo dimensions. A fourth necessary condition (analogousto the Jacobi conjugate point condition) was developed bysubsequent investigators and used in a sufficiency proof byBliss235 for the problem of Lagrange in two dimensions andby Bliss and Underhill236 for the three-dimensional Lagrangeproblem. Sufficiency proofs were developed under the as-sumption that the problem is "regular"; this condition re-quires that the extremal be smooth. Later, Mancill237 wasable to remove the "regular" condition, thereby allowingsolutions that are not tangent at the boundary surface andboundary surfaces that have corners. The problem wasalso discussed by De Jong.106

In more recent treatments of the problem, sufficiency con-siderations have been virtually ignored. The emphasis hasbeen placed on computational aspects and also on methodsfor including control constraints that restrict the admissiblecontrol to a bounded control space. One of the earliestworks on a bounded control problem was by Valentine238 whotreated a general problem in calculus of variations. Morerecently, Isaacs239 considered bounds on the state variablesin his papers on differential games. The constraints in state

Dow

nloa

ded

by B

eiha

ng U

nive

rsity

(C

NPI

EC

- X

I'AN

BR

AN

CH

) on

May

12,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/3.3

307


variables are taken care of by imposing a special restrictionon the class of allowable controls. That is, whenever thesystem state is about to strike a boundary, the range of pos-sible control strength is suitably reduced. This is to bedone in such a way that the system becomes incapable ofviolating the restrictions. The problem, in principle, maythen be treated by modifications of known methods.

Berkovitz240 rewrote the boundaries as an additional systemof first-order differential equations and applied Valentine'stechniques. Chang241 and Kipiniak242 each apply a penaltyto the performance criterion if the boundaries are violated.The penalty function is made large and "sharp-edged,"and then a limiting process is invoked. In addition, Bryson95

and Levinsky96 have treated variational problems of re-entryflight.

The main contribution in the Russian literature on thisproblem, to date, is a paper by Gamkrelidze.243 AdditionalLagrange multipliers are introduced, which may be discon-tinuous, and additional Euler equations are required.

Gamkrelidze, Chang, and Berkovitz deal solely with thetheoretical aspects of the problem, developing necessaryconditions that the solution must satisfy under the additionalrestriction that the control u be in a specified set U. Boththeoretical and computational aspects are considered inpapers by Dreyfus,244 Denham,245 Bryson et al.,246 and Brysonand Denham,247 with the computational procedure beingessentially an extension of the gradient or steepest-ascentmethod frequently used in unconstrained problems.Kahne,248 Ho and Brentani,249 and Kelley et al.250 deal ex-clusively with the computational aspects of the problem whiledeveloping methods of solution which are rather indirect.Garfinkel and McAllister251 point out some unusual caseswhere an extremal striking the boundary and satisfying thecorner conditions may have no possible continuation.

Necessary Conditions: Maximum Principle

The bounded state space problem can be stated mathe-matically as follows: We are given the system

Xi = fi(x, t, U) 1, . . . n (3)

where x is an ^-dimensional state vector, and u is an r-dimensional control vector. We must find a piecewise con-tinuous control u in the set U such that the function of thefinal state

'8 = S(xf,tf) (4)

takes on its minimum value subject to the condition that xstays within a specified region of the state space given by theinequality

0 (5)The set U is assumed to be compact, convex, and to containthe origin as an interior point. The functions /, S, and Gare assumed to be of class C" with respect to all tneir argu-ments. This assumption on the smoothness of G(x) can berelaxed to allow for the occurrence of corners on the boundarysurface.6 It should be noted that G(x) is not explicitly de-pendent on the control action.

The optimal trajectory is composed of two types of seg-ments: interior segments and boundary segments. It mayhappen that there are no boundary segments or no interiorsegments, but in the general case both types will be present.

An interior segment satisfies the same necessary conditionsas an optimal trajectory in the unconstrained problem. Theunconstrained problem [i.e., without inequality (5)] may beset up in several different ways.

The maximum principle requires the formation of a func-tion H, called the Hamiltonian:

H = #(p,x,u) = (6)

and where the pi, called adjoint variables, costate variablesor multipliers, satisfy the differential equations

(7)

In order for the function S in Eq. (4) to be a minimum, thecontrol action u must be selected so that

#(p, x, u«) > # (p, x, u) (8)at each point along the trajectory. The symbol u° denotesthe optimal control, and u denotes any other control, bothof which are contained in the set U.

It is important to remember that the maximum principleand the necessary conditions obtained from classical calculusof variations produce, not optimal trajectories, but extremals.It is still required to show that the extremal is optimal.

A singular arc or subarc occurs whenever the inequality(8) becomes an equality over a finite interval of time, andH becomes independent of u. If, for example, H is linearin Uk, then there may be a singular subarc whenever the co-efficient of u^ ftHfbuk vanishes over a finite interval. Themaximum principle fails to select the optimum control in thiscase. The singular control is found by using the requirementthat the system remain on a path such that bH/buk — 0.

Singular arcs occur in the Goddard problem,15 linear prob-lems with quadratic criteria,252'255 orbital transfers,18'19 andin terrestrial flight mechanics.5 The difficult part of theanalysis lies in determining whether a singular arc is optimal.The singular arc problem has been studied by Kelley253 andHermes254 and discussed by Johnson and Gibson.252 Miele5

has used Green's theorem successfully on problems with twostate variables. Kopp and Moryer have obtained a testfor determining the optimality of a singular arc.

A formulation along the classical variational lines is pos-sible if it is assumed that the constraint, that the control ulie in the set U, can be expressed analytically by an inequalityof the form

0 (9)

where the boundary curve, 4>(u) = 0, is piecewise differenti-able. For example, Valentine's method applied to con-straints on the control of the form a < u < ft leads to thefunction (u — a) (ft — u) — 772, where rj is real.

The functions F and / are then formed as in the Mayerproblem,256'257 where

E

,)+ f t f F d tJ to

Setting 5J = 0, we obtain the Euler equations

(10)

(11)

t = l,r (12)

The control action u and the multiplier A are determinedfrom Eq. (12) and the constraint

X^Cw) = 0 (13)

The Weierstrass necessary condition must also be satisfied.This yields an equation identical to Eq. (8) in the MaximumPrinciple formulation. Application of the Legendre-Clebschcondition using the device of Valentine provides the additionalinequality

X >0 (14)

An interior segment of a bounded state space problem mustsatisfy the foregoing necessary conditions.

Along a boundary segment, the inequality (5) becomes anequality, and the optimization takes place subject to the

Dow

nloa

ded

by B

eiha

ng U

nive

rsity

(C

NPI

EC

- X

I'AN

BR

AN

CH

) on

May

12,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/3.3

307


constraint

G(x) = 0 (15)

over the entire segment. Many classical problems in thecalculus of variations contain constraints of this form (e.g.,geodesies on surfaces), and the optimization procedures arewell known. In fact there are two equivalent procedures fortreating this problem.257 The first and more direct approachconsists in adjoining the constraint G to the unconstrainedversion of the problem with an additional multiplier /*i.

The second method consists in adjoining the total deriva-tive of the constraint dG/dt to the unconstrained problemwith a multiplier u%. If the function G(x) vanishes at theinitial point of a boundary segment, then adjoining the totalderivative and making it vanish along the segment is equiva-lent to adjoining the constraint itself. It has been shown258

that /t2 = — Ai, and ̂ < 0.It has been pointed out that an optimal trajectory in a

bounded state space problem consists of two types of seg-ments, boundary segments and interior segments, and theoptimization procedure for each type segment is well known.It still remains to determine the manner in which the indi-vidual segments join together to form the composite optimaltrajectory. A point at which an interior segment joins aboundary segment is called a corner point. The conditionsthat must be satisfied so that the resulting path is optimalare called the corner conditions.

It has been shown6'258 that at a corner point the multi-pliers satisfy the relation

+ v (16)

where z>o is a constant that depends on the end conditions.The Hamiltonian is continuous, i.e.,

, x, (17)

The derivation of this condition assumes that the vari-ations in the position and time of the corner (dxt and dt)are arbitrary. There are two cases of interest for which thisassumption is not valid, and the corner conditions appearin a different form. The first case occurs when the nthderivative of the constraint (dnG/dtn) is the first derivativethat contains the control u explicitly. In order to satisfythe constraint G < 0, it is required to make the first (n — 1)derivatives vanish at the junction point. The conditionsnow are

(18)

(19)n-l 5

£**where VQ and vi are constants whose values are to be deter-mined from the boundary conditions, and u° denotes theoptimal control.

If the first derivative dG/dt does not depend explicitlyupon the control, then a trajectory striking the boundarymay go past it and violate the constraint. An example ofthis is given by Dreyfus.244 For example, in trajectory op-timization problems with a constraint on the maximum alti-tude, the rate of change of altitude does not depend explicitlyon the control (which in this case is the thrust vector). Thevehicle will pass the altitude limit unless the rate of climb iszero at that altitude.

The second case occurs when the optimal trajectory doesnot lie in a sufficiently rich neighborhood of extremals.For example, suppose the system is

Time optimal switching curve

Switching curve x2

/ / \Optimal

trajectory\

Neighboring \trajectories

G>0

/ /

G <0

b)Fig. 2 Minimum time bang-bang problem with a velocity

limit.

with the control constraint \u < 1, and the state constraint

G = x2 - k < 0 (21)

The problem is to take the system from the initial point andbring it to the origin in a minimum time. The problem with-out the state variable constraint has been treated in theliterature with the optimal trajectory shown in Fig. 2. Theconstrained solution takes the form shown in Fig. 2b withthe corner points denoted by 1 and 2. Note that all neigh-boring trajectories lie to the right of the optimal as indicatedin Fig. 2b. There is no trajectory in the small neighborhoodof the optimal which falls to the left of the corner point 1.Furthermore, all the neighboring trajectories strike theboundary at a time later than the original optimal. Thus,the differentials dxi and dt, at £ = h, satisfy the conditions

i < 0 dt > 0 (22)

= X2 X 2 = U (20)

The corner conditions for this case given by Eqs. (16) and(17) do not necessarily hold.

There is no differential condition or local test for determin-ing when or where to leave the boundary and to return to theinterior of the domain. It is necessary to make successiveapproximations to find the junction points and satisfy the

Dow

nloa

ded

by B

eiha

ng U

nive

rsity

(C

NPI

EC

- X

I'AN

BR

AN

CH

) on

May

12,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/3.3

307


boundary conditions. Optimal control in a bounded statespace is discussed in more detail in Ref. 258.

IV. Stochastic Problems

Statement of the Problem

Major theoretical problems arise as a result of feedbackconsiderations in stochastic control systems. Some of theseproblems are discussed in a recent review by Kushner.263

The literature dealing with optimization of stochastic sys-tems, in contrast to the deterministic case, is rather sketchy;certain points are still obscure, and much work remains tobe done. The published papers, for the most part, presenteither carefully selected simple problems having closed-formsolutions or theorems of moderate generality which are toocomplicated for routine engineering applications. Thereare, however, some noteworthy exceptions, particularly theproblem of optimizing linear systems with quadratic per-formance criteria.

The difficulty of problem formulation is caused in part bythe fact that the transition from deterministic to stochasticperformance criteria is not direct. A meaningful problemmay result from transforming the minimization of quantityQ in the deterministic case to the minimization of the ex-pected value, or mean value, of quantity Q in the stochasticcase, but in other cases this transition provides a poor indexfor measuring system performance.

The design of an "optimal controller" is frequently ob-tained by treating the problem from an open-loop point ofview. The initial state of the system and its governing equa-tions are specified, and a control law is calculated whichtransfers the system from its initial state to a terminal statein an optimum manner with respect to the performancecriterion. The process is assumed to be completely deter-ministic, and the effects of uncertainties in process dynamicsand random disturbances are neglected. The control lawobtained under these assumptions unfolds only as a functionof the initial state without recognition of the subsequentstates of the process.

In actual practice, however, the system will operate, noton an open-loop basis, but on a closed-loop basis: the systemstate being monitored either continuously or discretely withthe control, depending on the estimates of the state variables.The control law in this case unfolds in time as the randomprocess itself unfolds, and the control tends to counteractand smooth the effects of random disturbances.

The question now arises as to how an optimum control lawshould be designed to include the effects of feedback in sto-chastic systems. One approach is to recompute continuallythe open-loop control law based on a deterministic perform-ance criterion but using the observed or estimated values ofthe state variables as new initial conditions. The controllaw can be computed as a function of the initial conditions,with the initial conditions themselves updated and the timeorigin adjusted to match the existing state of the system.Such an approach is proposed by Kelley351 and Breakwell,Speyer, and Bryson,352 along with a second-order theory tofacilitate adjustments in the control law.

Problem Formulation and Literature Survey

Figure 1 illustrates a typical model of a stochastic systemunder control. The process is assumed to be governed bythe vector differential equation

x = F(x, u, £, t) (23)

where x is the state vector, u the control, and £ a randomvector representing dynamic disturbances and/or uncer-tainties in process parameters. The relation between theobserved and actual state vector is given by the equation

where s is the observation vector, and 97 is a second randomvector denoting input and/or measurement errors. Thedimensions of s and x need not be equal.

Many papers on optimal stochastic control do not treatobservation errors; that is, the system is assumed to beeither perfectly observable with rj == 0 and s == x, or per-fectly unobservable where no observations are made and noobservation equation exists. An important contrast is thecase of partial observability when 77 ̂ 0, and Eq. (24) holds.It is convenient to treat these two cases separately. Uncer-tainties in system parameters can also be included in thisformulation.

Perfectly observable or perfectly unobservable systems

The state of the process is given by

x = F(x, u, £, 0 (25)

along with specified initial conditions at time t = t0 anda probability distribution for the random variable f . It isdesired to minimize the expected value of some function ofthe terminal state <j>[x(T)}. The control u is to be chosensuch that the integral

= fJ*

P[x(T), (26)

is minimized. In this equation, exp£ is the expectationoperator corresponding to the random variable f, and P[x(T),T] denotes the probability of the vector x having a valuebetween x(T) — \dx and x(T) + \dx at time T, the finaltime.

The meaning of the term P[x(T), T] must be examined inthe light of the particular problem. For example, theprobability density could mean the probability density condi-tioned on the state having the specified initial condition x(to)at time to. For this case, the probability density function is

P[x(T), T] = P[x(T), \ tQ] (27)

= G(x, 77) (24)

The probability density may be conditioned not only by theinitial state x(to) but by observations made on the state attimes between to and T] e.g.,

P[x(T), T] = P[x(T), T\x(t,\ t,' x(tj, ti; . . . ] (28)where ^, fe, . . . are the observation times. Knowledgeof the state at times subsequent to the initial time is used incomputing the optimal control law. In contrast, Eq. (27)represents the feed-forward or open-loop type of control.

The classical approach, which results in the control lawu = u[x(to), t], applies to the feed-forward case, where noobservations are made on the system. Kipiniak242 utilizesa variational approach for a class of problems in which thevariational and expectational operators commute.

Formulations utilizing the functional equation techniqueof dynamic programing treat the stochastic system as aclosed-loop or perfectly observable system with a P functionin the form of Eq. (28). The dynamic programing formula-tion of the stochastic control problem has been used manytimes.261"266 The most complete exposition of the methodis given by Bbllman.267 For the most part, the problemstreated by dynamic programing have linear dynamics withquadratic cost. The stochastic analogue of the bang-bangregulator is treated by Aoki.268 Krasovskii269 included theeffects of time lag in the system, and Zadeh and Eaton270

consider the stochastic problem in the absence of a statetransition equation. Some overlapping of the areas of opti-mal and adaptive control occurs in the cases where thecontroller is used to optimize and to probe the system forthe purpose of improving knowledge of system param-eters.271-272 Lass259 and Drenick and Shaw260 have treatedoptimizations with random parameters.

The disturbances are assumed to possess rather simplestatistical descriptions in most of the cases studied. The

Dow

nloa

ded

by B

eiha

ng U

nive

rsity

(C

NPI

EC

- X

I'AN

BR

AN

CH

) on

May

12,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/3.3

307


values of the random process £ are usually considered to beindependent in nonoverlapping intervals of time. Thissimplifying assumption allows the system to be treated as aMarkoff process where the state at time t + dt depends on itspast history only through its dependence on the state attime t. An elegant exposition of dynamic programing solu-tions of Markoffian processes is given by Howard.273 Drey-fus274 has recently discussed the problem of closed-loop opti-mal control. This paper provides a clear account of theadvantages and disadvantages of dynamic programing, vis-a-vis other approaches.

The formulations based on the classical variational meth-ods usually approach the problem from an open-loop or"perfectly unobservable" point of view. If the systemstate is perfectly unobservable, then it is not possible to usefeedback, and the control is necessarily open-loop. Severalopen-loop optimal synthesis procedures are based on the ideaof using the control to modify the probability density func-tion P[x(T), Tx(to), t0] of Eq. (27). In order to do this, itis necessary to determine the relation between P[x(T),T\x(to), to] and u(t), t0 <t <T. We assume that the systemwithout control is disturbed by additive noise. The equa-tions describing the process are

(29)

where the £»• are the random disturbances as before.The probability that x(t) belongs to a given region of the

state space 0 at time t is

P(x, t) dx

If the initial conditions are known precisely, then

where the 5?s indicate Dirac 5-functions. In the case wherethe initial conditions are random, the appropriate statisticalinformation must be supplied.

If the process is a Markoff process, then

P(x, t + dt) dx

is determined uniquely by the state of the system given atan earlier time r.

In that case, it can be shown275 that P[x} t\x(0)] satisfiesthe following second-order parabolic partial differentialequation, known as the Fokker-Planck equation

(30)

where the coefficients «»• and ba can be derived from Eq.(29) and the statistical properties of £;. This means thatEq. (30), together with a description of the initial state,contains all of the information about the future of the processthat we can get from the system (29).

The evolution in time of the probability density of thestate variables depends on the action of the control, and thisrelation must be incorporated into the formulation of theoptimal control problem. Several investigators276"278 havetaken the approach of using the Fokker-Planck equation or,alternatively, the "backward" equation or Kolmogorovequation.

Katz279 and Florentin280 derive the Fokker-Planck equa-tion for the probability density diffusion in space and time(assuming the process to be Markoffian) and employ theWeierstrass condition to determine the optimal control law.The difference between the feed-forward and feed-back ap-proach is discussed in,detail by Katz, and some examples aregiven.

A stochastic version of the Hamilton-Jacobi equation anda corresponding maximum principle have been formally de-

rived by dynamic programing by Kushner281 and by Won-ham.282 Krasovskii and Lidski283 and Kramer284 employ ageneralized Liapunov function for optimizing stochasticsystems. These papers demonstrate the difficulty en-countered in formulating the optimal feed-forward problem.

The problem is sometimes simplified and made easier tosolve by eliminating the time dependence of P and assumingthe existence of a steady state. The details of the transientsolution are lost in this simplification, and, in many problemswith finite time of action (e.g., missile interceptions), thetransient solution contains the interesting information.-The question of the value of asymptotic control applied toproblems of finite duration has been studied285 but has notbeen completely settled, i.e., what happens when a "steady-state" control is applied to a transient situation?

It can easily be appreciated, from what has already been-said, that the open-loop stochastic problem is equivalent to-a multidimensional optimization problem with a partial dif-ferential equation as a side condition. Constraints of thiskind, particularly partial differential equations of parabolictype (e.g., the heat conduction equation), appear in theoptimization of industrial and chemical processes with dis-tributed parameters, and a maximum principle has alreadybeen derived.286 A solution or an advance in computationaltechniques in either one of these areas (stochastic problemsor distributed parameter problems) should find rapid applica-tion in the other.

Partially observable systems

A partially observable system is one governed by theequations

x = F(x, u, £,t) s = G(x, 77) (31)

where s is the observation vector contaminated by noise77. Research is being done on the individual problems oflinear optimal filtering and optimal control as well as on theproblem of optimizing the combined filter and controller.The problem of synthesizing a combined optimal nonlinearfilter and control is very difficult, and fewer results are avail-able than in the linear case. A consistent general theory ofoptimizing partially observable nonlinear systems has notyet been developed.

A paper by Florentin,287 on the optimization of partiallyobservable systems, considers a system with linear dynamics,quadratic cost function, independent Gaussian observationnoise 77, and disturbances £. The observation vector s isa linear function of the state x and of dimension less thanor equal to that of x. The method centers on the use ofBayes' Theorem to update the probability distribution of thefuture behavior of the system, based on prior knowledge ofprobability distributions and present observations. In thisway, the control law unfolds as a function of all the pastobservations.

Another approach to the partially observable system in-volves filtering the observation signal to subtract out asmuch as possible of the unwanted noise. The problem ofdesigning an optimal filter is well known and one for whicha consistent theory exists, with the restriction that the filterbe linear and that the input signal and noise have stationarystatistical properties.288"291 The design of nonlinear filtersis studied by Bose,292 and the state space approach to linearfiltering has been developed by Kalman.293'134 Kalman'swork is quite well known and has been applied in many differ-ent circumstances.

There is a very large body of literature reporting work onthe optimal estimation of system state variables and param-eters. The optimal estimation problem has been reviewedelsewhere294-295 and will not be treated here.

Kalman and Bucy134 have investigated optimal controllersfor linear systems and obtained solutions to the combined

Dow

nloa

ded

by B

eiha

ng U

nive

rsity

(C

NPI

EC

- X

I'AN

BR

AN

CH

) on

May

12,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/3.3

307


optimal control and filtering problem. They study the sys-tem

dx/dt = Fz + Gu (32)

The measurable output is y = H#. The deterministic per-formance criterion is an integral of a quadratic form

27 = \im{[x(T),Sx(T)] + fT [\\y\\*Q + \\u\\Wr} (33)r-*-oo •/ towhere \\y\\2Q is the quadratic form y'Qy.

The optimal regulatory control is shown to be linear inthe state, i.e.,

u°(t) = — (R-Kr'P):*; = —K(t)x (34)The matrix P in Eq. (34) is the solution of the following ma-trix equation:

~dP/dt = F'P + PF - PGR-1 G'P + H'QH (35)Suppose the system is perturbed by noise

x = F# + Gu + NT; (36)

The deterministic procedure is still optimal if 77 is whiteGaussian noise in the sense that it will minimize the expectedvalue of the deterministic performance index. This formula-tion does not take into account saturation of the control,although the control manipulations can be reduced by put-ting a large penalty on the control via the matrix R.

The term "certainty equivalence" has been applied to theclass of control laws which optimize both the deterministicproblem and the stochastic version formed by taking the ex-pectation of the deterministic performance criterion.296 Thisnotion of certainty equivalence can be extended to includemore general questions, e.g., the relations between optimalcontrols for the same dynamic system and different probabil-ity density distributions for additive noise. Some work hasbeen done in this area, and also in the related area of theinverse problem of optimal control297 (e.g., for what class ofproblems is a given control optimal?), but these questionshave not yet been completely and thoroughly explored.

V. Two-Point Boundary-Value Problems

Linear Problems

The optimization problem formulated earlier contains anumber of unspecified parameters, which must be adjustedso that the boundary conditions are met. In a few cases thisis not difficult to do, but, in the great majority of cases, it isan extremely difficult task. In the rare event that both thesystem equations and the adjoint equations are linear, andif the control is also linear in the state and adjoint variables,and if the range of independent variable is prescribed, thena method based on Green7s functions can be applied. Thisis described in detail in Courant-Hilbert.298

Kalman has studied the problem of optimal control oflinear systems with quadratic cost criteria and has developeda synthesis procedure described in Sec. IV. It has recentlybeen applied to aircraft autopilots by Rynaski et al.114 andTyler.299 The existence of an optimal control in this casedepends upon complete controllability; uniqueness requirescomplete observability.

Optimal control of linear systems with quadratic criteriaand bounded controls has also been studied extensive-ly. 255,300-302 This problem is interesting because there is abang-bang mode, a linear feed-back mode, and also a singularsolution.Direct Integration

A method commonly used for solving differential equationswith two-point boundary conditions is direct integration ofthe system equations and adjoint equations. The solutionsare usually obtained in one of the following two ways.

1) A flooding technique is used, whereby a great numberof optimal trajectories are obtained emanating from somepoint of interest, either forwards or backwards in the inde-pendent variable. Some examples are available in the workof Kipiniak. It appears that this procedure is being usedin chemical or industrial process control. The generation ofswitching surfaces by backward integration is in effect a flood-ing technique. The flooding techniques have received onlylimited attention in the aeronautical literature because thewide range of initial and terminal conditions usually en-countered makes it necessary to provide for the storage ofmany paths. In some special cases, e.g., rocket boost andinjection into orbit, it is possible to limit the range of endconditions and use flooding technique to generate the optimalcontrol as a function of the state variables and time-to-go.This has been studied by Miner303 and others304""306 underthe name of path adaptive guidance. It is necessary to storeone or more functions of several variables in a manner thatallows rapid readout of the control. It is possible to com-press the information by using suitable approximations, andsome specific suggestions are available in the literature.307

2) A trial and error search technique, whereby an initialguess at the unknown parameters is made and successive re-finements taken until the errors in the boundary conditionsare reduced to tolerable values.308"312

The general idea of the direct integration method is fairlysimple. The initial (or terminal) conditions of the adjointare to be found by successive approximations such that theboundary conditions are satisfied when the coupled systemequations and adjoint equations are integrated. In somecases, it is more convenient to integrate the adjoint equa-tions backwards, but suppose that both equations are beingintegrated forward. The end conditions are observed aftereach run, a correction is made to the initial values of theadjoint, and the equations are integrated again. In orderfor this to work, it is necessary that small changes in initialconditions produce changes in the terminal conditions. Ex-amples taken from elementary problems show that this doesnot always occur.312 What is more, the end conditions areusually extremely sensitive functions of the initial condi-tions, making the search very difficult.

The direct integration method has found application inflight mechanics, particularly in trajectory optimizations.Breakwell,62 in an excellent paper in 1959, presented examplesof trajectory optimization, including the maximum rangeboost of a ballistic missile, boost to maximum velocity, andminimum time-to-climb. Both forward and backwardintegrations are discussed, and some numerical tests aredescribed. The equations are integrated numerically by afirst-order predictor-corrector routine, and adjustments ofthe guessed values are made using linear interpolation. Theprocedure converged, but it was pointed out that some edu-cated guessing was required before this occurred. A second-degree interpolation scheme was suggested to reduce the"educated" guessing, but no numerical results of this werepresented.

An interesting collection of trajectory optimization papersis contained in Ref. 313. A technique employed in severalof these papers is direct integration with a gradient methodused to drive the adjoint variables (or Lagrange multipliers)to the correct value, f It was reported that success de-pended, to a very large extent, on the initial guess used tostart things off and on the procedure used to adjust the stepsize during a descent. For the most part, the range of initialand terminal conditions studied was limited so that, once oneoptimal trajectory was obtained, some neighboring trajecto-ries, if any existed, might be found by perturbing the initialconditions. An extensive computation facility was requiredin all cases.

t The gradient methods mentioned here are not the gradientmethods in function space described in the next part of this sec-tion.

Dow

nloa

ded

by B

eiha

ng U

nive

rsity

(C

NPI

EC

- X

I'AN

BR

AN

CH

) on

May

12,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/3.3

307


Studies of optimal rocket boost trajectories using directintegration have been made by Stancil and Kulikowski.60'61

Other studies of optimal trajectories are listed in the bib-liography. Atmospheric flight problems are particularlydifficult because the presence of dissipative force causes theEuler equations to be unstable.

A study of optimal climb trajectories was carried out byMengel88 on an analog computer. However, as mentionedearlier, the extreme sensitivity of the end conditions of thetrajectory to the initial conditions of the adjoint made itdifficult to satisfy the boundary conditions.

The literature of the chemical and process industries con-tains accounts of attempts to optimize operations170'171 bydirect integration of the equations. Brunner314 presentsan iterative method designed for analog or hybrid analog-digital computation.

Smith et al.315 apply the Newton-Raphson method andmethod of steepest descent to optimization problems. Theypoint out several of the difficulties involved. Partial deriva-tives are evaluated numerically after integrating the coupledequations, and because of this the integration errors mustbe taken into account. If the changes in the initial valuesof the adjoint are too small, then integration errors willmake the resulting partials meaningless. If the incrementsare too large, then the partial derivatives are not correct.A compromise must be sought between the numerical integra-tion errors and the numerical differentiation errors.

The conclusion drawn from the results of these studies,and from studying simple examples, is that direct integra-tion of the system and adjoint equations and searching forthe correct adjoint values can be done, but success requires1) good starting values for the adjoint, and 2) a reliable meansof finding an extreme value of a function of several variables.

The difficulty in doing this obviously depends on the par-ticular problem and also on the particular initial and terminalconditions. It has been found that some places are harderto reach than others. Descriptions of unsuccessful attemptsrarely appear in print.316 This observation can ajso beapplied to the flooding technique and the "switching surface"technique.Gradient Methods in Function Space

The method of gradients or, as it is also called, the methodof steepest descent, has been applied in the field of optimalcontrol problems. The basic idea in the method of gradientsis to search for the optimizing function in a class of functionswhich satisfy the boundary conditions. In some cases, therequirement that the boundary conditions be satisfied canbe relaxed and an additional term included in the perform-ance criterion to account for errors in the end conditions.The searching proceeds by successive approximations takenin the direction of steepest descent in the function space.

The idea can be grasped by looking at an elementary case.For example, the simplest problem of the calculus of vari-ations seeks the extremum of the functional

(37)

(38)

} = fL(x,y,y'}dxThe gradient of the functional F[y(x) ] is

((d/dx)F,r - F,]|,wand the Euler equations are obtained by requiring the func-tional to be stationary, i.e., the gradient vanishes when(d/dx)(Fy

r) — Fy = 0. Instead of solving the Euler equa-tions, suppose that we guess a solution y°(x) and evaluategradient [F], i.e.,

(d/dx)Fy' - Fy\yo(x) (39)Suppose a maximum of F is desired. An improvement8F in the value of F is sought by taking a small step dy in thedirection of the gradient so that

It is necessary to include an explicit statement that the stepsize be small, e.g., fdy*(x)dx < 1. The computational tech-niques, although not conceptually difficult, require carefuldevelopment, and the reader is referred to the literature inRefs. 95,245-247,250,317,318 for details.

The method was developed and applied to aeronauticalproblems by Kelley250 and Bryson and Denham.95 A tutorialaccount by Kelley with several examples is contained in Ref.317. The method works well, although in some cases theconvergence of the iterations near the optimum is reportedto be slow as the minimum is approached. A generalizedNewton-Raphson method, which is reported to give rela-tively rapid convergence, has been successfully applied toaeronautical problems by McGill and Kenneth.318

Direct Methods

The objective of the direct methods is to change the vari-ational problem into a problem of ordinary maxima andminima. The solution of the variational problem and thetwo-point boundary-value problem is obtained by approxi-mating the unknown function by a truncated series andperforming an ordinary maximization over the coefficients ofthe series. The techniques most commonly used are theRayleigh-Ritz and Galerkin methods.298'319

The Rayleigh-Ritz method is based on the idea of approxi-mating the minimizing function by a linear combination offunctions <t>i(x) over [a, 6]; <£;(a) = 0»(6) = 0, i = 1 . . . n:

(41)

where <po(x) satisfies the boundary conditions. The func-tional to be minimized, e.g.,

X bF(x, yn, yn')dxi

= f[vF]-dy(x)dx (40)

is a function of the coefficients c». A minimization of thisfunction with respect to the c» yields a set of equations to besolved for the coefficients. If the functional being extremizedand the <l>i(x) satisfy certain conditions (e.g, form a completeset), then the limit as n -> <», if it exists, will be an exactsolution. The question of the convergence of the approxi-mations is very important and should not be taken forgranted. For example, the fact that a minimizing sequenceof allowable functions converges does not mean that thelimit function lies in the class of allowable functions.

The Rayleigh-Ritz method has been applied to the calcula-tion of optimal interplanetary trajectories by Saltzer andFetherorT.320 They consider a power limited rocket operat-ing at the maximum power level and seek to maximize thepayload. They assume that no coasting arcs will occur.A gradient method is used to solve the nonlinear equationsfor the coefficients of the approximating series.

In Galerkin's method, the equations are first integrated byparts, and then the homogeneous boundary conditions areapplied. Galerkin7s method has been applied to optimalcontrol problems by Boyanovitch.321'322

Ho and Brentani249 have investigated a successive approxi-mation procedure based on discretization of the problem.Another direct method applied to much simpler problemswithout inequality constraints is described in Courant-Hilbert298 and Elsgolc.350

In all of these direct methods, the problem becomes aminimization of a function of many variables.

There are many other attempts at finding satisfactoryapproximate methods for optimal control problems and two-point boundary problems. Some of these are listed in thereferences, but none will be discussed in any detail here. Aplan that is often suggested solves a simplified version of theproblem and uses the resulting initial values for the Lagrangemultipliers to start the integration of the Euler equations forthe original problem. The success ;of this scheme depends

Dow

nloa

ded

by B

eiha

ng U

nive

rsity

(C

NPI

EC

- X

I'AN

BR

AN

CH

) on

May

12,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/3.3

307


on how well the gradients of the payoff surfaces of the originalproblem correspond to the simplified versions. Some workalong these lines has been done by Gragg,355 who applied hisresults to systems with quadratic performance criteria andorbital transfers. It will be interesting to see how well theinitial adjoint vectors for optimal impulsive tranfers serve asstarting values for low thrust orbital transfer iterations.

Optimal Evolution

The concept of optimal evolution of a dynamic system andthe notion of attainable sets provide a convenient frameworkfor examination of closed-loop optimal systems. The dis-cussion in this section is limited to the nonstochastic case.At present, optimal synthesis based on these notions can beachieved only for a certain class of problems for linear sys-tems; nevertheless, the basic idea is extremely powerful.Studies of optimal processes in terms of the principle ofoptimal evolution have been made by deJong,106 Halkin,323

Roxin,324 Isaacs,239 Contensou,325 and Leitmann and Bla-quiere.326 Consider the optimization problem stated previ-ously. The system equations are

Xi = fi(x, i = 0, . . . n (42)

The control u is contained in convex compact set U and ispiecewise continuous. Find u(t) such that 1) there existsT > 0 and x(T) = x1} etc., and 2)

fIJ o L(xi, uh)dt

Is minimized. The space t X xn+1 is of dimension n + 2.Define a reachable point as a point of t X xn+l (t, XQ, xiy

. . . xn) attained by application of an allowable control andtime t > 0, subject to initial conditions z(0) = [0, #0(0), . . . ,&n(0)]. We call this set of reachable points R[t, x3-(Q)], j =1... n.

The reachable set is the set of all points that can be reachedby the system using all permissible controls in a given time.The reachable sets are defined with respect to the initialpoint.

The principle of optimal evolution states simply that eachevent of an optimal trajectory belongs to the boundary ofthe reachable set. The referenced articles show that theadjoint vector p employed in the definition of the maximumprinciple corresponds to the normal to a support plane atthe boundary of the reachable sets. The synthesis problemcan be changed into the problem of generating the reachablesets by solving a partial differential equation.

The evolution or growth of the reachable set can be thoughtof in terms of a generalized Huygens principle. That is,elementary wavelets originating from points on the boundaryof the reachable set at time t possess an envelope, and, at timet + dt, this envelope in turn becomes the new boundary ofthe reachable set, and the process of growth or evolutioncontinues.

We state without proof that the "motion" of the boundaryof the reachable set in n + 1 dimension (x, T) space can bedescribed by the following partial differential equation(Hamilton-Jacobi) if the derivatives exist. Let s be the opti-mal value of the payoff:

bs maxueU [Vs-x(x,u, t ) ] . (43)

The relationships between the Hamilton-Jacobi equation,optimization problems, the maximum principle, and Liapu-nov stability have been discussed by Leitmann and Bla-quiere,326 Pontriagin et al.,6 APbrekht,327 Geiss et al.,322

Aoki,329 and Kalman and Bertram.194 Discrete forms ofthis equation resemble the main equations of the method ofdynamic programing. It is important to remember thatthe payoff surfaces, i.e., surfaces of s = const, are not alwayssmooth and that sharp corners and edges may occur.312

Limiting processes whereby dynamic programing equationsformulated in discrete terms are carried over into partialdifferential equations should be handled with great care.

Pines330 points out that, finding all the constants of motionof a general dynamical system, is equivalent to solving theHamilton-Jacobi equation. He has applied this observa-tion to the solution of an orbital transfer problem for whichhe has found the constants of motion.

Although the description given here is in terms of timerunning forwards, it is possible to develop this idea withtime running backwards and the sets nested about the targetpoint. This approach is also directly related to the ideasof dynamic programing and differential game theory.

An early study with time reversed can be found in the re-port by Anderson.328 Studies of the geometry of the optimalpayoff surfaces and their relation to the control law for thetime-optimal problem have been made by Paiewonsky,312

Fliigge-Lotz and Halkin,331 and Leitmann and Blaquiere.326

The convexity of the set U and the fact that the originis an interior point of U are important because the con-vexity of the reachable set, the piecewise continuity of thecontrol, and the attainability of the extremum depend onthese properties.

A number of interesting examples illustrating the effect ofnonconvexity of U can be found in Isaac's Rand studies.239

The usual practice is to replace the set U by the smallestconvex set encompassing it and to solve the problem as ifthis were the set of admissible functions; this provides abound on the payoff function. More recently, Neustadt332

has treated the optimal control problem in the absence ofconvexity conditions.

VI. Minimizing Functions of Several Variables

The methods for solving two-point boundary-value problemsdepend to a large extent on being able to find extreme valuesof functions of several variables. The minimization of afunction of several variables is hardly ever an easy task,and in many cases it presents a formidable challenge. Thefunctions are often troublesome to evaluate, and the deriva-tives may be unavailable except by numerical approxima-tion. It is important to keep in mind the distinction betweenthe problems of finding the minimum value accurately andfinding the location of the minimum.

In Neustadt's method, for example, the location of themaximum of a certain function determines the optimalinitial conditions for the adjoint equations. The functionbeing maximized is rather flat, and it is difficult to get anaccurate estimate of the location of the maximum point,although the value of the maximum is easy to obtain.

The problem we discuss is a minimization, but, clearly,finding maxima can be treated by making obvious modifica-tions to the methods. Furthermore, the methods describedhere find local minima only.

A minimum value of a function f(xi, x2, . . . Xrj) is to befound, subject perhaps to constraints of the form •&•(#») = 0,j = 1, . . . a, hk(xi) < 0, k = a + 1, . . . 77 — 1.

The inequality constraints are sometimes appended to theproblem in the form of a verbal description of the domain ofdefinition of the function /. It is assumed that, whereverderivatives of f(x) exist, they are available to a computer,either by evaluating an explicit expression for d//d#», or bynumerical differentiation. The function / is assumed to becontinuous so the minimum is actually obtained. Minimamay occur on the boundaries of the domain of definition, atinterior points where bF/dxi = (b/cteO (/ + A^) = 0, andthe constraints </;fe) = 0 are satisfied simultaneously, anddet d^F/dxibX] is > 0, or at points where the derivatives donot exist; e.g., cusps and sharp edges. The necessary condi-tions provide no guarantee that there will be an absoluteminimum.

Dow

nloa

ded

by B

eiha

ng U

nive

rsity

(C

NPI

EC

- X

I'AN

BR

AN

CH

) on

May

12,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/3.3

307


The solution of the simultaneous equations for the pointswhere the gradient vanishes is generally difficult to obtain,and the method seems practical for only the simplest prob-lems. Even in the case where the equations are linear, it isstill a nontrivial task to solve for the minimizing x when thenumber of equations is large. The problem of solving simul-taneous linear equations is sometimes transformed into aminimization problem just to avoid the difficulties associatedwith matrix inversions.

These remarks indicate that it is often impractical tofind the minimum of a function analytically, and, because ofthis, methods have been developed to do this computationally.The problem is to find an algorithm for a computer such thata good approximation to the minimum is obtained after areasonable number of steps.

We concentrate attention on minimizing a continuous anddifferentiate function F(x) by iterative procedures based onselecting an initial choice x°, and then seeking x1 such thatF(x1) < F(x°) and so on. In these iterative methods, aninitial point x° = (XIQ, x2°, . . ., Xr,Q) is chosen, and the algorithmdetermines a vector gk (the direction of the step) and a realnumber hk (the size of the step) . The (k + 1) 'st point in theiteration is xk+1 = xk + hk gk. Sometimes a good guess forthe starting value, or the first point tested, can be madebased on intuition or some general properties of the problem.For example, in applying Neustadt's method to time-optimalcontrol of linear systems, the convexity property of the opti-mal isochrones indicates that a satisfactory rule for startingthe iterations is to select the direction of initial adjoint vectoropposite to the system state vector.

The simplest of the iterative methods is the univariatemethod, where the components of xk are varied one at a time.Suppose dF/dXi(xk) > &F/dxj(x*) for j = 1, . . . , i - 1,t + 1, . . . , n, i.e., the change in F in the direction of the ithcoordinate is greater than along any other coordinate axis.Then we let gk = (0, 0, . . . , 0, -1, 0, . . . , 0), where -1 isin the ith place. To determine hk, we write

and minimize f(xk+1) with respect to hk:

d/dhkF(xk+l) = 0This gives

(xk)£i OXj

(44)

(45)

(46)

The scheme requires b^/ctoi2 to be nonvanishing. Theconvergence properties of the method are believed to beweaker than methods based on the gradient, and it is notrecommended.3 3 3

The terms "gradient method" and "steep descent'' methodas used here apply to minimization techniques that takesuccessive steps in directions obtained by linear transforma-tions of the local gradient vector.* The method of steepestdescent is the special case wherein steps are always taken inthe direction opposite to the local gradient vector. It is wellknown that the convergence of steep descent iterations canbe slow. Special techniques have been developed to speedup the rate of convergence and eliminate time consumingoscillations about ridge crests. These accelerated gradientmethods determine the direction of successive steps fromobservations of the gradient vectors at several points insteadof using only the local gradient (for specific examples, seeRefs. 334-337).

There are many modifications of the idea of steep descent,and accounts of these are given in reviews by Edelbaum,338

Spang,339 Woodrow,333 Wilde,340 Shah et al.,341 Tompkins,342

and Courant.343 We shall briefly sketch the general ideasunderlying some of the very many convergence accelerationschemes for gradient methods. More detailed accountsare contained in the references.

The variations of the steep descent idea differ accordingto the rules used to generate the step hk and the directionof the step gk. A simple rule for choosing the step size isto make it a constant. This rule, although simple to apply,may lead to difficulties in obtaining convergent iterations.We know that F(xk+l) will be less than F(xk) at the kth step[xk+l = xk — hgkty*)], if h is sufficiently small. On the otherhand, the process will move very slowly if h is too small, anda great many steps will be required to bring x close to theminimum.

A technique in common use in overcoming problems of thiskind is to choose a step size at each stage according to valuesof the function and its gradient evaluated at several trialpoints. One of the simplest step-changing routines makesan initial choice of h and, subsequently, doubles or halvesthe step size depending on the improvement obtained inF(x). As steps are taken in a given direction, say in thedirection of — VF, the values of F(x) (in that direction) willdecrease initially but may begin to increase again for a largeenough step away from the initial point. The step hk*corresponding to the point where the rate of change of F(x)along the line of march vanishes is called the optimum step.That is, the optimum step corresponds to finding the firstlocal minimum along a line whose direction is specified by therule of the gradient method being used. An obvious way tocompute the optimum step is to choose some small quantitydh and successively compute F in the direction given by gk atthe points xk + dh gk, xk + 2 dh gk, . . . . Sooner or later Fwill increase; if this happens at the point xk + m-dh gk, goone step back and halve dh, and try the point xk + (m —%)-8h-gk), etc. The scheme is time consuming.

Booth335 has developed a method that reduces the amountof computation but at the cost of decreased accuracy. Theidea is to pass a quadratic polynomial through F(xk) and twoother points along the line and to find the minimum of thepolynomial analytically.

One of the best available methods to find 'hk* for steepestdescent and other gradient methods makes use of the fact thatthe derivative of the function along the line of march,

d/dh{F[xk + h vF(xk)}} = vF[xk +h V F ( x k ) ] - v F ( x k ) (47)

vanishes at hk ~ hk*. We have to solve the equation

VF[xk + hk*'VF(xk)]-vF(xk) = 0 (48)for hk*, the optimal step size.

March out from the starting point xk in the direction AFas shown in Fig. 3. As we move along the line, the directionof the local gradient is found at successive points. The

VF[xk+h*VF(xk)]

VF(xk + hVF(xk))

J These methods are similar to, but not the same as, the gradi-ent methods in function space developed by Bryson,95 Kelley,90

and others.

Starting point

Contour line

Fig. 3 The gradient direction method for finding theoptimum step.

Dow

nloa

ded

by B

eiha

ng U

nive

rsity

(C

NPI

EC

- X

I'AN

BR

AN

CH

) on

May

12,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/3.3

307


arrows show the angle between ^F [xk + hkVF(x)] and V^(xk) at a typical point. The point xk where f(xk) is perpendic-ular to the line of march is the desired stopping point. Thesolution of Eq. (48) can be obtained by trial and error,Newton's method, or any other method for finding a zeroof a function of a single variable. The dot product in Eq.(47) is required to be differentiable with respect to hh, andthere are simple examples in which the derivative along theline of march does not exist.

The presence of grossly unequal eigenvalues of the quad-ratic form obtained from the second-order terms of the Taylorseries expansion of F(x) (a ridge or ravine) may cause veryunpleasant oscillations in the iterations. Gel'fand336 hasdeveloped a method for handling this, but reports of compu-tational experience are scarce. The oscillations in the valueof F(x) at successive steps may occur even when optimumstep routines are employed if the other aspects of numericalanalysis are not treated with care. Step-size adjustmentschemes based on comparisons of F(x) at successive trialsmay fail to work simply because the magnitude of the differ-ence between the observed values of the function is on thesame order as the noise. In this case, the sign and value ofthe scalar product V[F(XI + hvF(xi)]-VF(xi) are usefulguides for selecting the step size.

A glance at the literature will show that a good deal ofeffort has been spent on accelerating the convergence ofsteep-descent methods and reducing the amount of com-putation necessary to get close to the minimum point. Inparticular, very powerful methods have been developedfor minimizing quadratic polynomial functions, and con-vergence proofs and estimates of the number of steps requiredto reach the minimum have been obtained for the ideal caseof a quadratic form in n variables. The hope is that similarbehavior will be obtained when the method is applied to otherfunctions. In many cases of interest, the function beingminimized can be approximated satisfactorily in the vicinityof the minimum point by a quadratic form in n variables. Aconsiderable simplification in the analysis is now possible, asthe gradient of the approximating polynomial is a linearfunction of the coordinates, and the Hessian matrix H =b^F/dxifrXj is constant. The minimum point xmin for aquadratic function can be found in one step if the elementsof the Hessian are known. Specifically,

£min — x = — ^Ffbxi ^Xj]~1-vF(x) = — [H~l]yF (49)This formula can be used to generate an iterative method forfinding the minimum point of a nonquadratic function byusing an optimum step routine to find the minimum in thedirection of — H~1(xk) - VF(xk), recomputing the Hessian,and repeating the procedure until the magnitude of ^F(x)is below a preselected value.

The Hessian of a general function is not a constant, butis a function of position, and the successive approximationroutine must provide for the computation of H~1 at eachstep. Furthermore, a closed form analytical expressionfor the Hessian may not be available, and numerical differ-entiation may be needed.

The computation may be simplified by substituting apositive definite symmetric matrix in place of the unknownHessian. For example, the choice of the identity or unitmatrix as a replacement for the Hessian in Eq. (49) givesthe method of steepest descent, i.e.,

Xk+i _ Xk = -vp (50)The successful application of these iterative methods de-pends on the location of the initial point being fairly close tothe minimum. If the initial guess is very bad, the methodmay not converge. Many of the gradient and acceleratedgradient methods are sensitive to scale changes, and thepossibility of improving the convergence rates by coordinatetransformations should not be overlooked. Some conver-gence acceleration methods do, in fact, generate coordinate

transformation as part of their successive approximationcalculations.344

The inverse of the Hessian plays such a role in Eq. (49)if we view H~l as a coordinate transformation, which mapsthe quadratic form into an n sphere. The minimum is easilyfound by taking one step of length R = [H~l]-\^F\ in thedirection opposite the gradient.

In a method originally due to Davidon345 and later modi-fied by Fletcher and Powell,346 the inverse of the Hessian isobtained by successive approximations. The process isstable, and the minimum of a quadratic form in n variablesis obtained in n iterations.

Powell347 has described an effective method, which is alsoguaranteed to converge to the minimum in the ideal case ofa quadratic, positive definite polynomial in n variables.Powell's method is closely related to the parallel tangent or'Tar-Tan," methods that are reviewed in the previouslymentioned article by Shah et al.

Some of the more important technical details involved inthe programing of Powell's method for a computer aredescribed in reports by Woodrow333 and Paiewonsky etal.348'76 A comparison of convergence rates for Po well'smethod, Davidon-Fletcher-Powell method, and steepestascent applied to Neustadt's synthesis method appears in areport by Paiewonsky and Woodrow.76 A hybrid analog-digital mechanization of Neustadt's method using a modi-fied steepest ascent is described in Ref. 349.

VII. Closed-Loop Optimal Control

The discussion of closed-loop systems in this section ap-proaches the problem from the viewpoint of recomputing theoptimal control at successive instants of time, and perhapstaking into account the possibility of more disturbancesoccurring later on. It is possible to treat stochastic problemsfrom the extremal field viewpoint, and discussions haveappeared using dynamic programing to formulate functionalequations, but very few tangible results have been published.

The notion of a field of extremals is essential to under-standing closed-loop optimal controllers. The rigorousformulation of the properties of fields of extremals can befound in Bliss,256 Bolza,234 or Elsgolc.350

Suppose we have a domain D in an n-dimensional space.If there exists one and only one curve of a family througheach point, then we call this family a proper field in D. Acurve is a member of a family if its defining equations con-tain one or more parameters. If all the curves pass througha common point (i.e., form a pencil, or bundle of rays), thenthey do not form a proper field in D. They form what iscalled a central field, provided that there are no other pointsof intersection within the domain. At all other points ofthe domain except the center of the pencil, the bundle ofrays form a proper field. The size of the domain is relatedto the question of sufficiency of the conditions for extrema.

A field (proper or central) composed of a family of ex-tremals of a variational problem is called a field of extremals.If one extremal of this field x°(t) is known, we would like tofind other neighboring members of this family of extremalsin order to develop an optimal guidance scheme.

Suppose that an optimal path has been obtained to adeterministic optimization problem. Disturbances may beexpected to occur which will force the system off its nominalpath in state space; we seek neighboring optimal pathsthat will enable the requirements of the problem to be met.It is clear that we may not want to return to the originalpath but might prefer to find a new optimal path startingfrom the current state.

Recent papers by Kelley,351 Breakwell, Speyer, andBryson,352 and Merriam353 present a second-order varia-tional theory allowing the development of neighboring pathguidance schemes. Dreyfus100-274 and Rang354 have alsostudied the problem of closed-loop optimal control. The

Dow

nloa

ded

by B

eiha

ng U

nive

rsity

(C

NPI

EC

- X

I'AN

BR

AN

CH

) on

May

12,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/3.3

307


connection between the payoff surfaces and the optimal con-trol law has already been mentioned. If we regard optimalprocesses in terms of the maximum principle, we will needto find new initial conditions for the adjoint correspondingto each new position in state space.

Let the system be displaced from the nominal path by anamount fe(0- Tne new initial conditions for the adjointand hence the new optimal control law are obtained fromthe gradient of the optimal payoff curve through the pointx + dx. A particular problem may possess a very narrowfield of extremals, and a perturbation may put the initialpoint outside the domain of admissible trajectories. Sup-pose that we attempt to represent the payoff surfaces sin the vicinity of the original optimal path by an expansionin terms of the perturbations, e.g., a Taylor series or perhapsan expansion in orthogonal polynominals. The new initialconditions for the adjoint then determine the optimal controlthrough the requirement that the Hamiltonian

H =i=Q

be maximized (or minimized) .Let the payoff surfaces be approximated in the vicinity of

the nominal path bys(x (52)

The partials bs/dx; are known already as functions of timealong the nominal optimal path. A "second-order" theorythen requires the knowledge of the partials ^s/bxidx^ orbP;/dx? along the nominal optimal trajectory. If time doesnot appear explicitly in a problem, then the optimal controlcan be computed as a function only of the location of thesystem in the state space. This can be an important con-sideration because it means that it may not be necessary tocompare the observed or estimated state with the nominalstate at each instant. This depends on how well the in-stantaneous state can be estimated. In an autonomousideal, perfectly observable case, errors in time measurementand errors due to disturbances causing translations alongthe nominal path in state space would not require correctivecontrol action.

The solution to the Hamilton-Jacobi equation can providethe solution to the guidance problem wherever the payoffsurfaces are smooth. It actually may provide more thanwe may need because the Hamilton-Jacobi equation can giveall the smooth payoff surfaces over the entire attainableregion of the state space, and we may want them only in thevicinity of a special curve. The papers by Kelley,351 Break-well et al.,352 and Merriam353 show how the partial deriva-tives can be generated along a nominal optimal path. Thecomputation and storage of neighboring optimal paths hasalso been studied by Vance307 and others for guidance of largerocket boosters.

Ellert and Merriam356 and Bailey357 have applied theseideas to optimal aircraft landing, and guidance and controlof a rocket booster.

It is important to keep in mind that deterministic problemsformulated in the manner just described may blow up whenrandom disturbances are added, because it may become im-possible to meet the terminal point conditions. This is afamiliar problem in terminal control design, and there aremethods that attempt to avoid it. The terminal point canbe replaced by a terminal set, the feedback gains can beallowed to saturate or be reduced as the range to target de-creases, an alternative mode for the controller may be acti-vated at a specified minimum range, and so forth.

In those instances where there is an effort constraint,it may be very undesirable to make a closed-loop optimalcontrol from a succession of deterministic open-loop solutionsalong the lines of a predictive terminal controller. Certainorbital transfers, for example, may have a rocket thrustprogram with some very short periods of burning. The

times when these pulses occur are determined by estimatesof the system state; the introduction of noise into the systemmay produce spurious pulses with the result that the effortconstraint boundary is reached, or the propellant exhausted,before the terminal conditions can be satisfied.

Behavior of this kind may be avoided by filtering, separat-ing the steering program from the thrust program, and per-haps introducing an auxilliary controller to steer about thenominal optimal path.

Kelley101 has investigated terminal accuracies achievableusing several different guidance laws. He presents someexamples for steering about solutions to Zermelo's problemwhich show that feedback based on errors transverse to thenominal optimal path is very attractive from the standpointof ease of generation of the control as well as reduction ofterminal errors. It may be that a dual-mode controller willoffer a practical solution to those situations where a closed-loop optimal control is desired. Nominal optimal pathscould be generated from time to time as needed, and a second-ary mode of continuous control could be employed for steeringabout these paths.

Another approach is to formulate the optimal controlproblem from the beginning in stochastic terms. The diffi-culties in solving stochastic optimization problems have al-ready been pointed out, and ad hoc solutions may have to dotemporarily until the studies of stochastic problems reachthe stage where they allow reasonable computational solu-tions.

Very little work has been published in the open literatureto date on the computational and operational aspects ofclosed-loop optimal controllers. Some studies have beenpublished which do treat the computational problems anda few comparisons of the effectiveness of algorithms areavailable.344'349'358-360

The emphasis in the publications seems only now to beshifting from the problems of finding individual optimalsolutions to the new task of applying these ideas to opti-mizing closed-loop controllers with noise, random disturb-ances, and imperfectly known parameters.

This area of engineering research will become increasinglyimportant because the advance in computer technology(from the standpoint of size, weight, power, reliability, andcost) offered by micro-circuitry361'362 brings the level ofachievable computer performance close to that required foraeronautical and astronautical application of closed-loopoptimal control.

References1 Bellman, R., Adaptive Control Processes, A Guided Tour

(Princeton University Press, Princeton, N. J., 1961); alsoBellman, R. and Stuart, E. D., Applied Dynamic Programming(Princeton University Press, Princeton, N. J., 1962).

2 Chang, S. S. L., Synthesis of Optimal Control Systems (Mc-Graw Hill Book Co., Inc., New York, 1961).

3 Lawden, D. F., Optimal Trajectories for Space Navigation(Butterworth's Mathematical Texts, London, 1963).

4 Leitmann, G., Optimization Techniques with Applicationsto Aerospace Systems (Academic Press Inc., New York, 1962).

5 Miele, A., Flight Mechanics: Theory of Flight Paths(Addison-Wesley Publishing Co., Inc., Reading, Mass., 1962),Vol. I.

6 Pontriagin, L. S., Boltryanskii, V. G., Gamkrelidze, R. V.,and Mishchenko, E. F., The Mathematical Theory of OptimalProcesses (John Wiley and Sons, Inc., New York, 1962).

7 Proceedings of the 5ih Joint Automatic Control Conference(Stanford University Press, Stanford, Calif., 1964).

8 Proceedings of the 1st IF AC Congress (Moscow, 1960); alsoProceedings of the 2nd IF AC Congress (Butterworths ScientificPublications, London, 1963).

9 Goddard, R. H., "A method of reaching extreme altitudes/7

Smithsonian Inst. Misc. Collections 71, no. 2(1919).10 Malina, F. J. and Smith, A. M. O., "Flight analysis of the

sounding rocket," J. Aeronaut. Sci. 5, 199-202 (1938).11 Tsien, H. S. and Evans, R. C., "Optimum thrust pro-

gramming for a sounding rocket," ARS J. 21, 99-107 (1951).

Dow

nloa

ded

by B

eiha

ng U

nive

rsity

(C

NPI

EC

- X

I'AN

BR

AN

CH

) on

May

12,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/3.3

307


12 Ross, S., "Minimality for problems in vertical and hori-zontal rocket flight," Jet Propulsion 28, 55-56 (1958).

13 Miele, A., "Generalized variational approach to the opti-mum thrust programming for the vertical flight of a rocket,"Purdue Univ., School of Aerospace Engineering A-57-1 (March1957); also Air Force Office of Scientific Research-TN-57-173(1957).

14 Leitmann, G., "A calculus of variations solution of God-dard's problem," Astronaut. Acta II, 55-62 (1956).

15 Garfinkel, B., "A. solution of the Goddard problem," SIAMJ. Series A: Control I, 349 (1963); also "Errata: A solutionof the Goddard problem," SIAM J. Series A: Control 2,135(1964).

16 Hohmann, W., Die Erreichbarkeit der Himmelskorper (Olden-bourg, Munich, 1925); also NASA translation TT-F-44, (1960).

17 Barrar, R. B., "An analytic proof that the Hohmann-typetransfer is the true minimum two-impulse transfer," Astronaut.Acta IX, 1(1963).

18 Lawden, D. F., "Optimal intermediate-thrust arcs in agravitational field," Astronaut. Acta VIII, no. 2, p. 106 (1962).

19 Kopp, R. E. and Moyer, H. G., "Necessary conditions forsingular extremals," AIAA Paper 65-63 (January 1965).

20 Bender, D. F. and McCue, G. A., "Numerical demonstra-tion of the non-optimality of the Lawden spiral," llth TechnicalMeeting, Space Flight and Guidance Theory, NASA MarshallSpace Flight Center (December 1962); also AIAA J. (submittedfor publication).

21 Lawden, D. F., "Minimal trajectories," J. Brit. Interplanet.Soc. 9,179-186 (July 1950).

22 Lawden, D. F., "The determination of minimal orbits,"J. Brit. Interplanet. Soc., II, 216-224 (September 1952).

23 Lawden, D. F., "Minimal rocket trajectories," ARS J.23,360-365(1953).

24 Lawden, D. F., "Fundamentals of space navigation," J.Brit. Interplanet, Soc. 13, 87-101 (March 1954).

25 Lawden, D. F., "Stationary rocket trajectories," Quart.J. Mech. Appl. Mech. 7, 488-504 (December 1954).

26 Lawden, D. F., "Optimal programming of rocket thrustdirection," Astronaut. Acta I, 41-56 (1955).

27 Lawden, D. F., "Optimum launching of a rocket into anorbit around the earth," Astronaut. Acta I, 185-190 (1955).

28 Edelbaum, T. N., "Some extensions of the Hohmann trans-fer maneuver," ARS J. 29, 864 (1959).

29 Carstens, J. P. and Edelbaum, T. N., "Optimum maneuversfor launching satellites into circular orbits of arbitrary radiusand inclination," ARS J. 31, 943-949 (1961).

30 Munick, H., McGill, R., and Taylor, G. E., "Minimiza-tion of characteristic velocity for two-impulse orbital transfer,"ARS J. 30, 638-639 (1960).

31 Altman, S. P. and Pistiner, J. S., "Minimum velocity incre-ment solution for two-impulse coplanar orbital transfer," AIAAJ. 1,435-442(1963).

32 Hoelker, R. F. and Silber, R., "The bi-elliptical transferbetween circular coplanar orbits," Dept. of the Army Tech.Memo 2-59, Army Ballistic Missile Agency (1959).

33 Fimple, W. R., "Optimum midcourse plane changes forballistic interplanetary trajectories," AIAA J. 1, 430-434(1963).

34 Gobetz, F. W., "Optimum transfers between hyperbolicasymptotes," AIAA J. 1, 2034-2041 (1963).

35 Horner, J. M., "Optimum impulsive orbital transfers be-tween coplanar orbits," ARS J. 32, 1082-1089 (1962).

36 Templeman, W. H., "Comparison of minimum energypaths and apogee designation paths," ARS J. 29, 865-868 (1959).

37 Progress Reports on Studies in the Fields of Space Flightand Guidance Theory (George C. Marshall Space Flight Center,Huntsville, Ala.)

38 Fosdick, G. E. arid Anthony, M. L., "3-d pulse optimiza-tion for vehicles disorbiting from circular orbits," Astronaut.Acta VIII, 343 (1962).

39 Eckel, K., "Optimum transfer between non-coplanar ellipti-cal orbits," Astronaut. Acta. VIII, 177 (1962).

40 Godal, T., "Method for determining the initial velocitycorresponding to a given time of free flight transfer betweengiven points for a simple gravitational field," Astronaut.(Svenska)2,183(1961). '

41 Long, R. S., "Transfer between non-planar elliptic orbits,small eccentricities," Astronaut. Acta. VI, 167 (1960).

42 Wang, W., "Minimum time transfer between co-planar,

circular orbits by two impulses and the propulsion requirements,"Astronaut. Acta IX, 12 (1963).

43 Ting, L., "Optimum orbital transfer by several impulses,"Astronaut. Acta VI, 256 (1960).

44 Edelbaum, T. N., "The use of high- & low-thrust propulsionin combination for space missions," J. Astronaut. Sci. IX, no. 2(1962).

45 Leitmann, G., "Some remarks on the optimum operationof a nuclear rocket," 10th International Astronautical Congress,London (Springer Verlag, Berlin, 1959).

46 Bussard, R. W., "Some topics in nuclear rocket optimiza-tion," Optimization Techniques (Academic Press Inc., New York),Chap. 14.

47 Irving, J. H. and Blum, E. K., "Comparative performanceof ballistic and low thrust vehicles for flight to Mars," Vistas inAstron. (Pergamon Press, New York, 1959).

48 Wang, C. J., Anthony, G. W., and Lawrence, H. R.,"Thrust optimization of a nuclear rocket of variable specificimpulse," ARS J. 29, 341 (1959).

49 Melbourne, W. G., "Three-dimensional optimum thrusttrajectories for power limited propulsion systems," ARS J.31,1723(1961).

50 Melbourne, W. G. and Sauer, C. G., Jr., "Optimum thrustprograms for power-limited propulsion systems," Astronaut.Acta VIII (Institute of Aerospace Sciences, New York, 1962).

51 Fox, R. H., "Powered trajectory studies for low thrustspace vehicles," ARS J. 31, 28 (1961).

52 Kelley, H. J., "Successive approximation techniques fortrajectory optimization," Proceedings of the I AS Symposiumon Vehicle Systems Optimization (Institute of Aerospace Sciences,New York, 1961), p. 10.

53 Hinz, H. K. and Moyer, H. G., "Three-dimensional lowthrust interplanetary trajectory optimization," Proceedings,2nd IF AC Congress (Basle, 1963).

54 MacKay, John S., "A variational method for the optimiza-tion of interplanetary round-trip trajectories," NASA TN D-1660.

55 Hibbs, A. R., "Optimum burning program for horizontalflight," ARS J. 22, 204-212 (1952).

56 Fried, B. D., "On the powered flight trajectory of an earthsatellite," Jet Propulsion 27, 641-643 (1957).

57 Bryson, A. E. and Ross, S. E., "Optimum rocket trajectorieswith aerodynamic drag," Jet Propulsion 28, 465-469 (1958).

58 Ross, S., "Composite trajectories yielding maximumcoasting apogee velocity," ARS J. 29, 843-848 (1959).

59 Miele, A., "Mathematical theory of the optimum trajec-tories of a rocket—final report," Purdue Univ., School Aero-nautical Engineering Rept. A-58-10, p. 35 (November 1958);also Rept. AD-206 361; also Air Force Office of Scientific Re-search TR 58-154.

60 Kulakowski, L. J. and Stancil, R. L. "Rocket boost tra-jectories for maximum burnout velocity," ARS J. 30, 612-618(1960).

61 Stancil, R. T. and Kulakowski, L. J., "Rocket boost vehiclemission optimizations," ARS J. 31, 935 (1961).

62 Breakwell, J. V., "The optimization of trajectories," J.Soc. Ind. Appl. Math. 7, 215-247 (June 1959).

63 Leitmann, G., "Optimal thrust direction for maximumrange," J. Brit. Interplanet. Soc. 16, 503-507 (1958).

64 Lawden, D. F. and Long, R. S., "The theory of correctionalmaneuvers in interplanetary space," Astronaut. Acta VI, 48(1960).

65 Lawden, D. F., "Optimal program for correctional maneu-vers," Astronaut. Acta VI, 195 (1960).

66Striebel, C. T. and Breakwell, J. V., "Minimum effortcontrol in interplanetary guidance," I AS Paper 63-80 (January1963).

67 Denham, W. F. and Speyer, J. L., "Optimal measurementand velocity correction programs for midcourse guidance,"AIAA J. 2, 896 (1964).

68 Lawden, D. F., "Interplanetary rocket trajectories"Advances in Space Sciences, edited by F. I. Ordway III (AcademicPress, New York, 1959), p. 1053, Vol. 1.

69 Leitmann, G., "The optimization of rocket trajectories,"Progress in the Astronautical Sciences (North Holland PublishingCo., Amsterdam, Holland, 1962), Vol. I.

70 Miele, A., "Some recent advances in the mechanics of ter-restrial flight," Jet Propulsion 28, 581-587 (1958).

71 Bergqvist, B., "The optimization problem for rocket vehiclessubjected to medium and high accelerations; a literature survey,"

Dow

nloa

ded

by B

eiha

ng U

nive

rsity

(C

NPI

EC

- X

I'AN

BR

AN

CH

) on

May

12,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/3.3

307


Svenska Interplanetariska Sallskapet Astronautik 1, 101 (1959).72 Goldstein, A. A., Greene, A. H., and Johnson, A. J., "Fuel

optimization in orbital rendezvous/7 AIAA Progress in Astro-nautics and Aeronautics: Guidance and Control II, edited byR. C. Langford and C. J. Mundo (Academic Press, New York,1964), Vol. 13, pp. 823-844.

73 Kelley, H. J. and Dunn, J. C., Proceedings, 2nd IF ACCongress on Automatic Control (Butterworths Scientific Publica-tions, London, September 1963).

74 Hinz, H. K., "Optimal low-thrust near-circular orbitaltransfer," AIAA J. 1, 1367-1371 (1963).

75 Mclntyre, J. and Crocco, L., "Optimal transfer between closecircular orbits," AIAA Paper 64-29 (January 1964).

76 Paiewonsky, B. H. and Woodrow, P. J., "Study of timeoptimal rendezvous in three dimensions," Wright Patterson AirForce Base, Ohio, TR-65-20, Vol. I (August 1964).

77 Neustadt, L. W., "Synthesizing time optimal control sys-tems," J. Math. Anal. Appl. 1, 4 (December 1960).

78 Meditch, J. S., "On the problem of optimal thrust pro-gramming for a lunar soft landing," Inst. Elec. Electron. Engrs.,Trans. Auto. Control AC-9, 477 (1964).

79 Hall, B. A., Dietrich, R. G., and Tiernan, K. E., "A mini-mum fuel vertical touchdown lunar landing guidance," AIAAProgress in Astronautics and Aeronautics; Guidance and ControlII, edited by R. C. Langford and C. J. Mundo (Academic Press,New York, 1964), Vol. 13, pp. 965-994.

80 Neustadt, L. W., "A general theory of minimum-fuel spacetrajectories," University of Michigan TR06181-1-T (November1964).

81 Perkins, C. D. and Hage, Airplane Performance, Stabilityand Controls (John Wiley and Sons, Inc., New York, 1949).

82 Lippisch, A., "Performance theory of airplanes with jetpropulsion," Headquarters, Air Material Command, Transporta-tion Rept. F-TS-685-RE (1946).

83 Lush, K. J., "Optimum climb theory and techniques ofdetermining climb schedules from flight tests," NATO AGARDFlight Test Manual (1954), Vol. I, Pt. I, Chap. 7; also AFFTC-TN-56-13(1956).

84 Garfinkel, B., "Minimal problems in airplane perfor-mance," Quart. Appl. Math. 9, no. 2 (1951).

85 Miele, A., "On the non-steady climb of turbo-jet aircraft,"J. Aeronaut. Sci. 21, 781-783 (November 1954).

86 Cicala, P. and Miele, A., "Brachistochronic maneuvers of avariable mass aircraft in a vertical plane," J. Aeronaut. Sci.22, 577-578 (August 1955).

87 Hestenes, M. R., "A general problem in the calculus ofvariations with applications to paths of least time," RANDRM 100, Rand Corp., Santa Monica, Calif. (March 1949).

88 Mengel, A. S., "Optimum trajectories," Proceedings of Proj-ect Cyclone Symposium I on RE AC Techniques (1951).

89 Kelley, H. J., "An investigation of optimal zoom-climbtechniques," J. Aerospace Sci. 26, 794-802, 824 (1959).

90 Kelley, H. J., "Gradient theory of optimal flight paths,"ARSJ. 30, 947-954 (1960).

91 Bell, D. J., "A review of flight path optimization . . . in theperiod 1945-1960," J. Roy. Aeron. Soc. 67, 119 (February 1963).

92 Arens, M., "Some requirements for the efficient attain-ment of range by airborne vehicles," ARS J. 30, (August 1960).

93 Bell, D. L, "The determination of maximum range for anunpowered atmospheric vehicle," J. Roy. Aeron. Soc. 68, 111(February 1964).

94 Miele, A., "On the flight path of a hypervelocity gliderboosted by rockets," Proceedings of the 10th International Astro-nautical Congress (Springer Verlag, Berlin, 1959).

95 Bryson, A. E., Denham, W. F., Carroll, F. J. and Mikami,M., "Determination of lift or drag programs to minimize re-entry heating," J. Aerospace Sci. 29 (April 1962).

96 Levinsky, E. S., "Application of inequality constraints tovariational problems of lifting re-entry," I AS Paper 61-21(January 1961); also Wright Air Development Div. Rept. TR60-369 (July 1960).

97 "Flight control study of a manned re-entry vehicle,"General Electric Co., Wright Air Development Div. TR 60-695(July 1960), Vol. 1.

98 Bryson, A. E. and W. F. Denham, "Multivariable terminalcontrol for minimum mean square deviation from a nominalpath," Raytheon Co. .Rept. No. BR-1333 (September 1960);also Proceedings of the I AS Symposium on Vehicle SystemsOptimization (New York, 1961).

99 Zermelo, E., "Uber das navigations Problem bei ruhenderoder veranderlicher wind Verteilung," Z. Angew Math Mech11,115(1931).

100 Dreyfus, S. E. and Elliott, J. R., "An optimal linear feed-back guidance scheme," J. Math. Anal. Appl. 8, 364-386 (1964).

101 Kelley, H. J., "An optimal guidance approximation theory,"Inst. Elec., Electron. Engrs. Trans. Auto. Control AC-9, 375(October 1964).

102 yon Mises, R., "Zum navigations Problem der Luftfahrt,"Z. Angew. Math. Mech. 11, 373 (1931).

103 Levi-Cevita, T., "Uber Zermelo's luftfahrt Problem,"Z. Angew. Math. Mech. 11, 314A (August 1931).

104 Faulkner, F. D., "Optimum ship routing," Navigation(1963-1964).

105 Haltiner, G. J., Hamilton, H. D., and Arnason, G., "Mini-mal-time ship routing," J. Appl. Meteorol. 1, 1-7 (March 1962).

106 DeJong, H. M., "Theoretical aspects of aeronavigation andits application in aviation meteorology," Koninkl. Ned. Meteorol.Inst., no. 64(1956).

107 Rekasius, Z. V., "A general performance index for analyticaldesign of control systems," Inst. Radio Engrs., Trans. Auto.Control AC-6 (May 1961).

108 Zaborsky, J., "The development of performance criteriafor automatic control systems," Am. Inst. Elec. Engrs. Inst.Tech. Groups Auto. Control 1, no. 2 (1962).

109 Gibson, J. E., et. al., "A set of standard specifications forlinear automatic control systems," Am. Inst. Elec. Engrs.Trans., pt. 11(1961).

110 Kalman, R. E., Ho, Y. C., and Narendra, K. S., "Con-trollability of linear dynamical systems," Contributions to Differ-ential Equations (The Macmillan Co., New York, 1961), Vol. 1.

111 Kalman, R. E., "Mathematical description of lineardynamical systems," Research Institute for Advanced Studies,TR 62-18 (November 1962).

112 Walkovitch, J., Magdaleno, R. E., McRuer, D. T., Graham,F. D., and McDonnell, J. D., "Performance criteria for linearconstant-coefficient systems with deterministic inputs," WrightPatterson Air Force Base, Ohio, Aeronautical Systems Div.TR 61-501 (1961).

113 Magdaleno, R. and Walkovitch, J., "Performance criteriafor linear constant-coefficient systems with random inputs,"Wright Patterson Air Force Base, Ohio, Aeronautical SystemsDiv. ASD-TDR-62-470 (1962).

114 Reynolds, P. A. and Rynaski, E. G., "Application of optimallinear control theory of the design of aerospace vehicle controlsystems," Proceedings of the ASD Optimal System SynthesisConference (1962).

115 Rynaski, E. G., Reynolds, P. A., and Shed, W. H., "Designof linear flight control systems using optimal control theory,"Wright Patterson Air Force Base, Ohio, Aeronautical SystemsDiv. ASD-TDR-63-376 (April 1964).

116 Sheridan, T. B., "The human operator in control instru-mentation," Progress in Control Engineering (Heywood and Co.,London, 1962), Vol. 1, pp. 143-197.

117 McRuer, D. T. and Graham, D., "Pilot-vehicle controlsystem analysis," AIAA Progress in Astronautics and Aero-nautics: Guidance and Control—II, edited by R. C. Langfordand C. J. Mundo (Academic Press Inc., New York, 1964), Vol.13, pp. 603-621.

118 Ashkenas, J. L., Jex, H. R., and McRuer, D. T., "Pilot-induced oscillations; their cause and cure," Systems Tech-nology Inc., Rept. TR-239-2 (June 1964); also A'Harrah, R. C., J.Aircraft 1(1964).

119 Cromwell, C. H., Ill, and Ashkenas, J. L., Systems Tech-nology Inc., TR-124-1 (June 1962); also Durand, T. S. and Jex,H. R., Wright Patterson Air Force Base, Ohio, Air Systems Div.,ASD-TR-62-507 (May 1962).

120 Passera, A. L. and Willoch, R. G., Jr., "An optimum switch-ing criterion for a third order contactor acceleration controlsystem," NACA TN 3743 (August 1956).

121 Schmidt, S. F. and Harper, E. V., "The design of feedbackcontrol systems containing a saturation type nonlinearity,"NASA TN D-324 (September 1960).

122 Flugge-Lotz, I., Taylor, C. F., and Lindberg, H. E., "Investi-gation of a nonlinear control system," NACA Rept. 1391 (1958).

123 Ziebolz, H. and Paynter, H. M., "Possibilities of a two-time scale computing system for control and simulation ofdynamic systems," Proc. Nat. Electron. Conf. 9, (1954).

124 Coales, J. F. and Noton, A. R. M., "A non-off servomecha-

Dow

nloa

ded

by B

eiha

ng U

nive

rsity

(C

NPI

EC

- X

I'AN

BR

AN

CH

) on

May

12,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/3.3

307


nism with predicted change-over," Proc. Inst. Elec. Engrs. (Lon-don) 103 (July 1955).

125 Chestnut, H., Sollecito, W. E., and Troutman, P. H.,"Predictive control application," Am. Inst. Elec. Engrs. Paper61-12 (July 1961).

126 Dow, P. C., Fields, D. P., and Scammell, F. H., "Automaticre-entry guidance at escape velocity," AIAA Progress in Astro-nautics and Rocketry: Guidance and Control, edited by R. E.Roberson and J. S. Farrior (Academic Press Inc., New York,1962), Vol. 8, pp. 271-308.

127 Morris, R. V. and Steeg, C. W., "Multicondition terminalcontrol systems for aircraft," J. Aerospace Sci. 712 (1960).

128 O'Hern, E. A. and Smyth, R. K., "Terminal control sys-tem applications," Inst. Radio Engrs., Trans. Auto. ControlAC-6 (May 1961).

129Wingrove, R. C. and Coate, R. E., "Piloted simulatortests of a guidance system which can continuously predictlanding point of a low L/D vehicle during atmosphere re-entry,"NASA TN D 787 (March 1961); also Wingrove, R. C., "Surveyof atmosphere re-entry guidance and control methods," AIAAJ. 1,2019-2029(1963).

130 Weiner, N., The Extrapolation, Interpolation and Smoothing ofStationary Time Series (John Wiley and Sons, Inc., New York1949).

131 Zadeh, L. and Ragazzini, J., "An extension of Weiner'stheory of prediction," J. Appl. Phys. 21, 645-655 (1950).

132 Bharucha-Reid, A. T., Elements of the Theory of MarkovProcesses and Their Applications (McGraw-Hill Book Co., Inc.,New York, 1960).

133 Ruina, J. P. and Van Valkenburg, M. E.; "Stochastic analy-sis of automatic tracking systems," International Federation onAutomatic Control Conference, Moscow, 1960 (Butterworths Scien-tific Publication, London, 1962).

134 Kalman, R. E. and Bucy, R. S., "New results in linearfiltering and prediction theory," J. Basic Eng. 95 (March 1961).

135 Stewart, E. C., "An explicit linear filtering solution forthe optimization of guidance systems with statistical inputs,"NASA TN D-685 (February 1961).

136 McLean, J. D., Schmidt, S. F., and McGee, L. A., "Opti-mal filtering and linear prediction applied to a midcourse navi-gation system for the circumlunar mission," NASA TN D-1208(March 1962).

137 Tung, F., "Linear control theory applied to interplanetaryguidance," Inst. Elec. Electron. Engrs. Trans. Auto. ControlAC-9, 82 (January 1964).

138 Battin, R., "Optimizing a statistical navigation procedurefor space flight," ARS J. 32, 1681-1696 (1962).

139 Booton, R. C., "The analysis of non-linear control sys-tems with random inputs," Proceedings of the Symposium onNonlinear Circuit Analysis (Polytechnic Press, Brooklyn, N. Y.,1953), pp. 402-411.

140 Schwartz, L., "Optimum filter technique for terrain avoid-ance under G-limiting constraint," Wright Air DevelopmentDiv. TR 60-709 (October 1960).

141 Swaim, R. L., Inst. Elec. Electron. Engrs., Trans. Auto.Control AC-9, 508 (October 1964); also Houbolt, J. C., Steiner,R., Pratt, K, G., NASA TR R-199, p. 115 (June 1964).

142 Quinlivan, R. P., Tye, G., Westerhold, H. H., "Analyticalinvestigation of control requirements for high speed low altitudepenetration, final report brief," Flight Dynamics Lab., Researchand Technology Div., Wright Patterson Air Force Base, Ohio,FDL-TDR-64-104 (1964).

143 Moore, F. B. and Brooks, M., "Saturn ascending phaseguidance and control techniques," ARS Preprint 2458-62 (July1962).

144 Geissler, E., "Problems in attitude stabilization of largeguided missiles," Aerospace Eng. 19 (October 1960).

145 Burke, H. H., "Aerodynamic load reduction techniques forlarge elastic launch vehicles," Inst. Elec. Electron. Engrs.,Trans. Auto. Control AC-9, 565 (October 1964).

146 Athans, M., Falb, P. L., Lacoss, R. T., "Optimal controlof self-adjoint systems," Inst. Elec. Electron. Engrs., Trans.Appl. Ind. 83 (May 1964); also see Ref. 160,

147 Lee, E. B., "Discussion of satellite attitude control," ARSJ. 32, 981 (1962).

148 Suddath, J. H. and Carney, T. M., 'Technique for syn-thesis of constant linear dynamical systems with a bang-bangcontroller," NASA TR R-200, p. 33 (August 1964).

149 Pistiner, J. S., "On-off control system for attitude stabiliza-tion of a space vehicle," ARS J. 29, 283 (April 1959).

150 Ergin, E. L, Norum, V. D., and Windeknecht, T. G., "Tech-niques for analysis of nonlinear attitude control systems forspace vehicles," Aeronautical Systems Div. ASD-TDR-62-208,Vols.I-IV(1962).

151 Roberson, R. E., "Methods for the control of satellitesand space vehicles," Wright Air Development Div. TR-60-643(July 1960).

152 Ergin, E. L, "Current status of progress in attitude con-trol," AIAA Progress in Astronautics and Aeronautics: Guidanceand Control-II, edited by R. C. Langford and C. J. Mundo(Academic Press, New York, 1964), Vol. 13, p. 7.

153 "Single axis attitude regulation of extra atmosphericvehicles," Aeronautical Systems Div., ASD TR 61-129 (February1962).

154 Weiss, H. K., "Analysis of relay servomechanisms," J.Aeronaut. Sci. 13, 364 (July 1946).

155 Wang, P. K. C., "Analytical design of electrohydraulicservomechanisms with near time-optimal response," Inst.Elec. Electron, Engrs., Trans. Auto. Control AC-8 (January1963).

156 Sawaragi, Y., Hajime A., and Tsuyoshi O., "On-off controlsystems operating on sampled data," Bull.-Japan Soc. Mech.Engrs. 4, 489-499 (August 1961).

157 Flugge-Lotz, I. and Marbach, H., "The optimal control ofsome attitude control systems for different performance cri-teria," Proceedings of the Joint Automatic Control Conference(1962).

158 Meditch, J. S., "On minimal-fuel satellite attitude controls,"Inst. Elec. Electron. Engrs., Trans. Appl. Ind., no, 71. p. 120(March 1964).

159 Friedland, B., "A minimum response-time controller foramplitude and energy constraints," Inst. Radio Engrs., Trans.Auto. Control AC-7 (January 1962).

160 Athans, M., Falb, P. L., and Lacoss, R. T., "Time-, fuel-,and energy-optimal control of nonlinear norm-invariant sys-tems," Inst. Elec. Electron. Engrs., Trans. Auto. Control AC-8(July 1963).

161 Chang, S. S. L., "Minimal time control with multiplesaturation limits," Inst. Elec. Electron. Engrs., Trans. Auto.Control AC-8 (January 1963).

162 Knudson, H. K., "An iterative procedure for computingtime optimal controls," Inst. Elec. Electron. Engrs., Trans.Auto. Control AC-9, 23 (January 1964).

163 Kalman, R. E., "The theory of optimal control and thecalculus of variations," Research Institute for Advanced Studies,Baltimore, Md., TR 61-3 (1961).

164 Lee, E. B., "Geometric properties and optimal controllersfor linear systems," Inst. Elec. Electron. Engrs., Trans. Auto.Control AC-8, 379 (October 1963).

168 Kalman, R. E. and Koepche, R. W., "Optimal synthesis oflinear sampling control systems using generalized performanceindices," Trans. Am. Soc. Mech. Engrs., p. 1120 (November1958).

166 Katz, S., "A discrete version of Pontriagin's maximumprinciple," J. Electron. Control XIII, 179 (August 1962).

167 Hopkin, A. M. and Wang, P. K. C., "Further studies ofrelay-type feedback control systems designed for random in-puts," International Federation on Automatic Control Conference,Moscow (Butterworths Scientific Publications, London, 1960);also Hopkin, A. M. and Wang, P. K. C., "A relay type feedbacksystem control design for random inputs," Trans. Am. Inst. Elec.Engrs. 78, pt. II 228, (September 1959).

168 Ho, Y. C., "On the stochastic approximation method andoptimal filtering theory," J. Math. Anal. Appl. 6, 152-154 (1962).

169 Aoki, M., "On minimum of maximum expected deviationfrom an unstable equilibrium position of a randomly perturbedcontrol system," Inst. Radio Engrs., Trans. Auto. ControlAC-7 (March 1962).

170 Aris, R., The Optimal Design of Chemical Reactors (AcademicPress Inc., New York, 1961).

171 Lupfer, D. E. and Johnson, M. L., "Automatic control ofdistillation column to achieve optimum operation," Proceedingsof the 1963 Joint Automatic Control Conference (1963), p. 145;also Kipiniak, W. and Gould, L. A., Am. Inst. Elec. Engrs.Trans., pt. I, 79 (January 1961).

172 Perry's Chemical Engineering Handbook (McGraw-HillBook Co., Inc., New York), 4th ed., pp. 22-51 to 22-60.

173 Chestnut, H., Duresch, R. R., and Gaines, W. M., "Auto-matic optimizing of a poorly defined process, part I," JointAutomatic Control Conference, Paper 8-1 (1962); also "Auto-

Dow

nloa

ded

by B

eiha

ng U

nive

rsity

(C

NPI

EC

- X

I'AN

BR

AN

CH

) on

May

12,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/3.3

307


matic optimizing of a poorly defined process, part II," Proceed-ings of the 1963 Joint Automatic Control Conference (1963), p. 54.

174 Box, G. E. P. and Wilson, K. B., "On the experimentalattainment of optimum conditions," J. Royal. Statistical Soc.B. B. 1 (1951); also Box, G. E. P., "The exploration & exploita-tion of response surfaces: Some general considerations & ex-amples," Biometrics 10,16 (1954).

175 Draper, C. and Li, Y., "Principles of optimizing controlsystems and an application to the internal combustion engine,"Mech. Eng. 74, 145 (February 1952).

176 Tsien, H. S., Engineering Cybernetics (McGraw-Hill BookCo., Inc., New York, 1954).

177 Rose, N. J., "Optimum switching criteria for discontinuousautomatic controls," Institute of Radio Engineers ConventionRecord, pt. 4, p. 61 (1956).

178 Fuller, A. T., "Relay control systems optimized for variousperformance criteria," Proceedings of the International Federationon Automatic Control Congress (Butterworths Scientific Publica-tions, Ltd., London, 1960).

179 Flugge-Lotz, I. and Lindbert, H. E., "Studies of secondand third order contactor control systems," pt. I, TR 116,Div. of Engineering Mechanics, Stanford Univ. (1959).

iso King, R. E., "Passive optimization of a relay control systemsubjected to random inputs," J. Electron. Control XV, 617(December 1963).

181 Izawa, K. and Weaver, L., "Relay type feedback controlsystems with dead time and sampling," Trans. Am. Inst. Elec.Electron. Engrs., p. 49 (May 1959).

182 Flugge-Lotz, L, Taylor, C. F., and Lindberg, H. E., "In-vestigation of a nonlinear control system," NACA Rept. 1391(1958).

183 Andre, J. and Seibert, P., "The local theory of piecewisecontinuous differential equations," Contributions to the Theoryof Nonlinear Oscillations (Princeton University Press, Princeton,N.J.,1960),Vol.5.

184 Andre, J. and Seibert, P., "After endpoint motions of gen-eral discontinuous control systems and their stability proper-ties," Proceedings of the International Federation on AutomaticControl Congress (Butterworths Scientific Publications, Ltd.,London, 1960).

iss Ayzerman, M. A. and Gantmacher, F. R., "Some aspectsof the theory of a nonlinear automatic control system with dis-continuous characteristics," Proceedings of the International Fed-eration on Automatic Control (Butterworths Scientific Publica-tions, Ltd., London, 1960).

186 Filippov, A. G., "Application of the theory of differentialequations with discontinuous right-hand sides to non-linearproblems in automatic control," Proceedings of the InternationalFederation on Automatic Control Congress (Butterworths ScientificPublications, Ltd., London, 1960).

187 Filippov, A. G., "Differential equations with discontinuousright hand sides," Rec. Math. 51 (1960).

188Tsypkin, la. Z., "VIIIANIE Sluchainykh Pomekh NaPeriodicheskii Rezhim V Releinykh Avtomaticheskikh Siste-makh," USSR, An, Dokl. 139, 570-573 (July 21, 1961).

189 Molitor, J. H., "Ion propulsion system for stationary-satellite control," J. Spacecraft Rockets 1, 170-175 (1964).

190 Brown, S. C., "Predicted performance of on-off systems forprecise satellite attitude control," NASA TN-D-1040 (July1961).

191 Abzug, M. J., Martin, L. W., and Ohgi, F. T., "A time opti-mal attitude control system designed to overcome certain sensorimperfections," AIAA Progress in Astronautics and Aeronautics:Guidance and Control—II, edited by R. C. Langford and C. J.Mundo (Academic Press Inc., New York, 1964), pp. 231-259.

192 Kuba, R. E. and Kazda, L. F., "A phase-space method forthe synthesis of nonlinear servomechanisms," Am. Inst. Elec.Engrs. Trans., Appl. Ind., p. 282 (November 1956).

193 Bass, R. W., "Discussion of a paper by A. M. Letov, proc.Heidelberg conf. on automatic control," Regelungstechnik: Mod-erne Theorien und ihre Verwendbarkeit (R. Oldenbourg, Munich,1957), pp. 209-210.

194 Kalman, R. E. and Bertram, J. E., "Control system analysisand design via the 'second method' of Liapunov," J. BasicEng., pt. I, p. 371 (June 1960).

195 Rauch, L. L. and Howe, R. M., "A servo with linear opera-tion in a region about the optimum discontinuous switchingcurve," Proceedings of the Symposium on Nonlinear Circuit Analysis(Polytechnic Press, Brooklyn, N. Y., 1956).

196 Doll, H. G. and Stout, T. M., "Design and analog computer

analysis of an optimum third order nonlinear servomechanism,"Trans. Am. Soc. Mech. Engrs. 79, p. 513 (1957).

197 McDonald, D. C., "Intentional non-linearization of servo-mechanisms," Proceedings of the Symposium on Nonlinear CircuitAnalysis (Polytechnic Press, Brooklyn, N.Y., 1953), pp. 402-411.

198 Schieber, L., "Variably damped servomechanisms," Am.Inst. Elec. Engrs. Trans., Appl. Ind., p. 414 (January 1957).

199 Ku, Y. H., "Analysis of servomechanisms with nonlinearfeedback control," Trans. Am. Inst. Elec. Engrs., Appl. Ind.(January 1957).

200 Flugge-Lotz, L, "Discontinuous automatic control," Appl.Mech. Rev. 14, p. 581 (1961).

201 Flugge-Lotz, I., Discontinuous Automatic Control (PrincetonUniversity Press, Princeton, N. J., 1953).

202 Kazda, L., "Control system optimization using computersas control system elements," Proceedings of Computer in ControlSystems Conference (American Institute of Electrical Engineers,New York, 1958).

203 McDonald, D., "Nonlinear techniques for improving servoperformance," Proc. Nat. Electron. Conf. 6, 400-421 (1950).

204 Kreindler, E., "Contributions to the theory of time-optimalcontrol," J. Franklin Inst. 275 (April 1963).

205 Bushaw, D., "Optimal discontinuous forcing terms,"Contributions to the Theory of Nonlinear Oscillations (PrincetonUniversity Press, Princeton, N. J., 1958), Vol. IV.

206 Bellman, R., Glicksberg, L, and Gross, O., "On the bang-bang control problem," Quart. Appl. Math. 14, 11-18 (1956).

207 Bellman, R. F., Glicksberg, L, and Gross, O. A., "Someaspects of the mathematical theory of control processes," RandRept. R-313 (January 1958).

208 LaSalle, J. P., "The time optimal control problem," Con-tributions to the Theory of Nonlinear Oscillations (PrincetonUniversity Press, Princeton, N. J., 1960), Vol. V.

209 Gamkrelidze, R. V., "The theory of time optimal processesin linear systems," Bull. Acad. Sci. USSR English Transl. 2,449-474(1958).

210 Markus, L. and Lee, E. B., "On the existence of optimalcontrols," J. Basic Eng. 84, 13-20 (March 1961); also Arch Ra-tional Mech. Analysis 8, 36-58 (1961).

211 Rozonoer, L. L, "L. S. Pontriagin's maximum principlein the theory of optimum systems, I, II, III," Avtomat. i Tele-meh. 20, 1320-1334, 1441-1458, 1561-1578 (October, November,December 1959); Automation Remote Control 20, 1288-1302,1405-1421, 1517-1532 (June, July, August 1960); transl.

212 Krasovskii, N. N., "On the theory of optimum regulation,"Automation Remote Control 18, 1005 (November 1957).

213 Kulikowski, R., "Optimizing processes and synthesis ofoptimizing automatic control systems with nonlinear invariableelements," Proceedings of the International Federation on Auto-matic Control Congress (Butterworths Scientific Publications,Ltd., London, 1960).

214 Kranc, G. M. and Sarachik, P. E., "An application of func-tional analysis to the optimal control problem," J. Basic Eng.143 (June 1963).

215 Lee, E. B., Inst. Radio Engrs., Trans. Auto. ControlAC-5, 283 (1960).

216 Balakrishnan, A. V., SIAM J. Control 12, 109-127 (1963).217 Achieser, N. L, Theory of Approximation (Frederick Ungar

Publishing Co., New York, 1956).218 Desoer, C. A., "The bang-bang servo problem treated by

variational technique," Inform. Control 2, 333-348 (December1959).

219 Bass, R. W., "Equivalent linearization—nonlinear circuitsynthesis, and the stabilization and optimization of controlsystems," Proc. Symp. Nonlinear Circuit Analysis, PolytechnicInst. of Brooklyn 16,163-198 (1956).

220 Neustadt, L. W., "Minimum effort control systems," Soc.Ind. Appl. Math. J. Control 1, (1963).

221 Gamkrelidze, R. V., "Theory of time-optimal processesin linear systems," Izv. Akad. Nauk K. SSR, Ser. Mat. i Mek-han. 22, 449 (1958); Rept. 61-7, Univ. of California, LosAngeles (January 1961); transl.

222 Krasovskii, N. N., Prikl. Mat Met. 23, 625-639 (1959).223 Eaton, J. H., "An iterative solution to time-optimal con-

trol," J. Math. Anal. Appl. 5, 329-344 (1962).224 Ho, Y. C., "A successive approximation technique for

optimal control systems subject to input saturation," J. BasicEng. 84, 33-40 (March 1962).

225 Paiewonsky, B. H. and Woodrow,~P. J., "The synthesis

Dow

nloa

ded

by B

eiha

ng U

nive

rsity

(C

NPI

EC

- X

I'AN

BR

AN

CH

) on

May

12,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/3.3

307


of optimal controls for a class of rocket steering problems/'AIAA Paper 63-224 (June 1963).

226 Meditch, J. S. and Neustadt, L. W,, "An application ofoptimal control to midcourse guidance," Proceedings of the 2ndInternational Federation on Automatic Control Congress (1963).

227 Desoer, C. A. and Wing, J., "A minimal time discrete sys-tem," Inst. Radio Engrs., Trans. Auto. Control AC-6 (May1961).

228 Neustadt, L. W., "Discrete time optimal control systems,"Proceedings of Office of Scientific Research-Research Institute forAdvanced Studies Symposium (1961); also Nonlinear DifferentialEquations and Nonlinear Mechanics (Academic Press, Inc.,New York, 1963).

229 Polak, J., "On the equivalence of discrete systems in timeoptimal control," J. Basic Engr. 205 (June 1963); also Polak,E., "An application of discrete optimal control theory," J.Franklin Inst. 276, 118 (August 1963).

230 Clement, P. R., "On the bang-bang regulator problem forlinear sampled-data systems," Inst. Elec. Electron. Engrs.,Trans. Auto. Control AC-8 (April 1963).

231 "Sampled-data control systems and Z transform applica-tions," (Bibliography) Inst. Radio Engrs., Trans. Auto. Control4, 28-30 (March 1958); also Jury, E. I., "Synthesis and criticalstudy of sampled-data control systems," Am. Inst. Elec. Engrs.,Trans. Appl. Ind. 141 (July 1956);, also Gunckel, T. L., Ill andFranklin, G. F., "A general solution for linear sampled datacontrol," J. Basic Eng. 197 (June 1963).

232 Warga, J., "Relaxed variational problems," J. Math.Anal. Appl. 4,111-128 (1962).

233 Hestenes, M. R., "Variational theory and optimal controltheory," Computing Methods in Optimization Problems, editedby A. V. Balakrishnan and L. W. Neustadt (Academic Press,Inc., New York, 1964).

234 Bolza, O., Lectures on the Calculus of Variations (DoverPublications, Inc., New York, 1961).

235 Bliss, G. A., "Sufficient conditions for a minimum withrespect to one-sided variations," Trans. Am. Math. Soc. 5,477-492(1904).

236 Bliss, B. A. and Underbill, A. L., "The minimum of adefinite integral for unilateral variations in space," Trans. Am.Math. Soc. 15, 291-310 (1914).

237 Mancill, J. D., "The minimum of a definite integral withrespect to unilateral variations," Contributions to the Calculus ofVariations, 1933-1937 (University of Chicago Press, Chicago,111., 1937).

238 Valentine, F. A., "The problem of Lagrange with differ-ential inequalities as added side conditions," Contributions to theCalculus of Variations, 1933-1937 (University of Chicago Press,Chicago, 111., 1937).

239 Isaacs, R., "Differential games I, II, III, IV," Rand Corp.Research Memo. RM-1391, RM-1399, RM-1411, RM-1486(1954).

240 Berkovitz, L. D., "On control problems with bounded statevariables," J. Math. Anal. Appl. 5 (December 1962).

241 Chang, S. S. L., "An extension of Ascoli's theorem and itsapplications to the theory of optimal control," New York Univ.,TR 400-51 (January 1962).

242 Kipiniak, W., Dynamic Optimization and Control: A Vari-ational Approach, (Technical Press, Massachusetts Institute ofTechnology, Cambridge, Mass, and John Wiley and Sons, Inc.,New York, 1961).

243 Gamkrelidze, R. V., "Time optimal processes with boundedphase coordinates," Dokl. Akad. Nauk. SSSR 125, 475-478(1959); also Pontriagin, L. S., Boltyanskii, V. G., Gamkrelidze,R. V., and Mishchenko, E. F., "The mathematical theory ofoptimal processes," transl. by K. N. Trirogoff and L. W. Neu-stadt (Interscience Publishers, Inc., New York, 1962), Chap. 6.

244 Dreyfus, S., "Variational problems with state variableinequality constraints," Rand Corp. Rept. P-2605 (July 1962).

245 Denham, W. F., "Steepest-ascent solution of optimalprogramming problems," Raytheon. Co., Bedford, Mass., Rept.BR-2393 (April 1963); also Bryson, A. E. and Denham, W. F.J. Appl. Mech. 29 (June 1962).

246 Bryson, A. E., Denham, W. F., and Dreyfus, S. E., "Opti-mal programming problems with inequality constraints I:necessary conditions for extremal solutions," AIAA J. 1, 2544-2550(1963).

247 Denham, W. F. and Bryson, A. E., "Optimal programmingproblems with inequality constraints II: Solution by steepest-ascent," AIAA J. 2, 25-34 (1964).

248 Kahne, S. J., "On direct fixed-time optimization of invert-ible systems," Rept. R-162, Coordinated Science Lab., Univ. ofIllinois, Urbana, 111. (June 1963).

249 Ho, Y. C. and Brentani, P. B., "On computing optimalcontrol with inequality constraints," Rept. 1529 TR-5, Minne-apolis-Honeywell, Boston, Mass. (March 1962); also, SIAM J.Control 1,319 (1963).

250 Kelley, H. J., Falco, M., and Ball, D. J., "Air vehicle flightpath optimization," Rept. AFOSR/DRA-62-4, Vol. I, Air ForceOffice of Scientific Research, Holloman Air Force Base, N. M.(February 1962).

251 Garfinkel, B. and McAllister, G. T., "Singularities in avariational problem with an inequality," Rept. 1207, BallisticResearch Labs., Aberdeen Proving Ground, Md. (July 1963).

252 Johnson, C. D. and Gibson, J. E., "Singular solutions inproblems of optimal control," Inst. Elec. Electron. Engrs.,Trans. Auto Control AC-8, 4-74 (January 1963).

253 Kelley, H. I., "A transformation approach to singular sub-arcs in optimal trajectory and control problems," SIAM J. Con-trol 2, 234(1964).

254 Hermes, H., "Controllability and the singular problem,"SIAM J. Control 2, 241 (1964).

255 Wonham, W. M. and Johnson, C. D., Proceedings of the1963 Joint Automatic Control Conference (1963).

256 Bliss, G. A., Lectures on the Calculus of Variations (Univer-sity of Chicago Press, Chicago, 111., 1946), p. 296.

257 Bliss, G. A., "The problem of Lagrange in the calculus ofvariations," Am. J. Math. 52, 673-744 (1930).

258 Mclntyre, J. and Paiewonsky, B. H., "On optimal controlwith bounded state variables," Aeronautical Research Associatesof Princeton Inc. Rept. 60 (July 1964).

259 Lass, H., Jet Propulsion Lab. Space Program Summary37-25, Vol. IV, pt. F. (1963).

260 Drenick, R. F. and Shaw, L., "Optimal control of linearplants with random parameters," Inst. Elec. Electron. Engrs.Trans. Auto. Control AC-9, 236 (1964).

261 Orford, R. J., "Optimal stochastic control systems," J.Math. Anal. Appl. 6 (June 1963).

262 Aoki, M., "Dynamic programming and numerical experi-mentation as applied to adaptive control systems," Univ. ofCalifornia, Los Angeles, Rept. 60-16 (February 1960).

263 Kushner, M. J., "Some problems and recent results instochastic control," Inst. Elec. Electron. Engrs. InternationalConvention Record, pt. 6 (1965).

264 Bellman, R., "On the foundations of a theory of stochasticvariational processes," Rand Corp. Rept P-1903 (February1960).

265 Aoki, M., "Dynamic programming and numerical experi-mentation as applied to adaptive control systems," Univ. ofCalifornia, Los Angeles, Engineering Dept. Rept. 60-16 (February1960).

266 Bellman, R., "Dynamic programming and stochastic con-trol processes," Info. Control I (September 1958).

267 Bellman, R., "On the application of the theory of dynamicprogramming to the study of control processes," Proceedings ofthe Symposium on Nonlinear Circuit Analysis (Polytechnic Press,Brooklyn, N. Y., 1956), pp. 199-213.

268 Aoki, M., "Stochastic time optimal control systems,"Am. Inst. Elec. Engrs. Appl. Ind. (May 1961).

269 Krasovskii, N. N., "On optimum control in the presence ofrandom disturbances," Trans. PMM J. 24, No. 1 (1960).

270 Eaton, J. H. and Zadeh, L. A., "Optimal pursuit strategiesin discrete state probabilistic systems," Paper 61-JAC-ll,Proceedings of the Joint Automatic Control Conference (June1961).

271 Florentin, J. J., "Optimal probing; adaptive control of asimple bayesian system," J. Electron. Control XIII, 165 (August1962).

272 Ho, Y. C. and Lee, R. C. K., "A Bayesian approach toproblems in stochastic estimation and control," Inst. Elec.Electron. Engrs. Trans. Auto. Control AC-9, 333 (October 1964).

273 Howard, R. A., Dynamic Programming and Markovian Proc-esses (Technical Press, Massachusetts Institute of Technology,Cambridge, Mass., 1960).

274 Dreyfus, S. E., "Some types of optimal control of stochasticsystems," SIAM J. Series A: Control 2, 120 (1964).

275 Wax, N. (ed.), Selected Papers on Noise and StochasticProcesses, (Dover Publications New York, 1954); also WangM. C. and Uhlenbeck, G. E., Rev. Mod. Phys. 17, 323-342(April-July 1945).

Dow

nloa

ded

by B

eiha

ng U

nive

rsity

(C

NPI

EC

- X

I'AN

BR

AN

CH

) on

May

12,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/3.3

307


276 Blackmail, N. M., "On the effect of noise in a non-linearcontrol system," Paper IFAC Conference, Moscow (1961).

277 Barrett, J. F., "Application of Kolmogorov's equation torandomly disturbed automatic control systems," Proceedings ofthe International Federation on Automatic Control Congress (IFAC)(Butterworth Scientific Publications, Ltd., London, 1960).

278 Chuang, K. and Kazda, L. F., "A study of non-linearsystems with random inputs," Trans. Am. Inst. Elec. Engrs.,Appl. and Ind. 42,100-105 (1959).

279 Katz, S., "On the best endpoint control of noisy systems,"J. Electron. Control XII (April 1962).

280 Jlorentin, J. J., "Optimal control of continuous timeMarkovian Stochastic systems," J. Electron. Control (June1961).

281 Kushner, H. J., "On the dynamical equations of conditionalprobability density functions, with applications to optimalstochastic control theory," J. Math. Anal. Appl. 8 (April 1964).

282 Wonham, W. M., "Stochastic analysis of a class of nonlinearcontrol systems with random inputs," J. Basic Eng., p. 223(June 1963).

283 Krasovskii, N. N. and Lidskii, E. A., "Analytical controldesign in systems with random properties," Automation andRemote Control (September-October-November 1961).

284 Kramer, J. D. R., "On control of linear systems with timelags," Inform. Control (December 1960).

285 Bellman, R. and Bucy, R., "Asymptotic control theory,"SIAM J. Series A: Control 2, 11 (1964).

286 Butkovskii, A. G., "The maximum principle for optimumsystems with distributed parameters," Automation and RemoteControl 22 (March 1962); also Wang, P. K C. and Tung, F.,"Optimum control of distributed parameter systems," JointAutomatic Control Conference, Minneapolis, Minn. (June 1963).

28? Florentin, J. J., "Partial observability and optimal con-trol," J. Electron. Control XIII, 263 (September 1962).

288 Peterson, E. L., Statistical Analysis and Optimization ofSystems (John Wiley and Sons, Inc., New York, 1961).

289 Davenport, W. B. and Root, W. L., An Introduction to theTheory of Random Signals and Noise (McGraw-Hill Book Co.,Inc., New York, 1958).

290 Laning, J. H. and Battin, R. H., Random Processes in Auto-matic Control (McGraw-Hill Book Co., Inc., New York, 1956).

291 Solodovnikov, V. V., Introduction to Statistical Dynamics ofAutomatic Control Systems (Dover Publications, Inc., New York,1960).

292 Bose, A. G., "A theory of nonlinear systems," Massachu-setts Institute of Technology Research Lab. of Electronics TR309 (May 1956).

293 Kalmari, R. E., "A new approach to linear filtering andprediction problems," J. Basic Eng. (March 1960).

294 Rideout, V. and Rajarman, V., "A digest of adaptivesystems," Proc. Am. Inst. Elec. Engrs.-Inst. Tech. GroupsAuto. Control I, 10-27 (1962); also Aseltine, J., Mancini, A.,and Sarture, C., "A survey of adaptive control systems," Inst.Radio Engrs. Trans. Auto. Control AC-6 102 (December 1958);also Stromer, P., Inst. Radio Engrs. Trans. Auto. Control AC-4(May 1959).

295 Cox, H., "On the estimation of state variables and param-eters for noisy dynamic systems," Inst. Elec. Electron. Engrs.Trans. Auto. Control AC-9, 5 (January 1964).

296 Adorno, D. S., "Optimum control of certain linear systemswith quadratic loss," Inform. Control 5, 1-12 (March 1962);also Adorno, D. S., "Studies in the asymptotic theory of con-trol systems, II," Jet Propulsion Lab. TR 32-99 (June 1, 1961);also Adorno, D. S., "Studies in the asymptotic theory of controlsystems: I. Stochastic and deterministic N-stage processes,"Jet Propulsion Lab. TR 32-21 (May 1960).

297 Kalman, R. E., "When is a linear control system optimal?"Proceedings of the Joint Automatic Control Conference (1963).

298 Courant, R. and Hilbert, D., Methods of Mathematical Phys-ics, Vol. I (Interscience Publishers, Inc., New York, 1953).

299 Tyler, J. S., Jr., "The characteristics of model-followingsystems as synthesized by optimal control," Proceedings of theJoint Automatic Control Conference (1964), p. 41.

300 Krasovskii, N. N. and Letov, A. M., "The theory of analyticdesign of controllers," Automation and Remote Control 23(June 1962).

301 Rekasius, Z. V. and Hsia, T. C., "On an inverse problemin optimal control," Inst. Elec. Electron. Engrs., Trans. Auto.Control AC-9 (October 1964).

302 Johnson, C. D. and Gibson, I. E., "Optimal control with

quadratic performance index and fixed terminal time," Inst.Elec. Electron. Engrs., Trans. Auto. Control AC-9 (October1964).

303 Miner, W. E. and Schmeider, D. H., "The path adaptivemode for guiding space vehicles," ARS Preprint 1944-61 (August1961).

304 Schmeider, D. H. and Braud, N. J., "Implementation ofthe path-adaptive guidance mode in steering techniques forSaturn multi-stage vehicles," ARS Paper 195-61 (August 1961).

305 Wheeler, R. E., "A mathematical model for an adaptiveguidance mode system," On studies in the fields of space flightand guidance theory, NASA Progress Rept. 4, MTP AERO-63-65(1963).

306 Hoelker, R. F., "Theory of artificial stabilization of missilesand space vehicles with exposition of four control principles,"NASA TND-555 (June 1961).

307 Vance, R. L, "Implementation of an optimal adaptiveguidance mode," AIAA J. 3, 141-142 (1965).

308 Hestenes. M. R., "Numerical methods of obtaining solu-tions of fixed endpoint problems in the calculus of variations,"Rand Rept. RM-102 (August 1949).

309 Knapp, C. H. and Frost, P. A., "Determination of optimalcontrol and trajectories using the maximum principle in associ-ation with a gradient technique," Proceedings of the Joint Auto-matic Control Conference (1964), p. 222fL

310 Jurovics, S. A. and Mclntyre, J. E., "The adjoint methodand its application to trajectory optimization," ARS J. 32,1354(1962).

311 Greenley, R. R., "Comments on 'the adjoint method andits application to trajectory optimization,' " AIAA J. 1, 1463(1963).

312 Paiewonsky, B. H., "A study of time optimal control,"Aeronautical Research Associates of Princeton Rept. 33 (June1961); also Proceedings of International Symposium on Non-linear Differential Equations and Non-linear Mechanics, editedby J. P. LaSalle and S. Lefshetz (Academic Press Inc., New York,1963), pp. 333-365.

sis Proceedings of the Vehicle Systems Optimization Sympo-sium, Garden City, New York (November 28-29, 1961).

314 Brunner, W., "An iteration procedure for parametric modelbuilding and boundary value problems," Joint IRE-AIEE-ACMComputer Conference, Los Angeles, Calif., (May 9-11, 1961).

315 Smith, F. B., Jr. and Lovingood, J. A., "An application oftime-optimal control theory to launch vehicle regulation," Pro-ceedings of the Optimum System Synthesis Conference (Septem-ber 11-13, 1962); also Aeronautical Systems Div. ASD-TDR-63-119 (February 1963), p. 99; also Scharmack, D. K., "Theequivalent minimization problem and the Newton-Raphsonoptimization method," Proceedings of the Optimum SystemSynthesis Conference (September 11-13, 1962); also Aeronau-tical Systems Div. ASD-TDR-63-119 (February 1963), p. 119.

316 Rang, E. R., "An unsuccessful nonlinear control designprocedure," Inst. Elec. Electron. Engrs., Trans. Auto. Control8 (January 1963).

317 Kelley, H., "Optimization techniques," Method of Gradi-ents, edited by G. Leitman (Academic Press, Inc., New York,1962), Chap. 6.

318 McGill, R. and Kenneth, P., "Solution of variational prob-lems by means of a generalized Newton-Raphson operator,"AIAA J. 2,1761(1964).

319 Kantorovich, L. V. and Krylov, V. I., Approximate Methodsof Higher Analysis (Interscience Publishers, Inc., New York andP. Noordhoff Ltd., Groningen, The Netherlands, 1958).

320Saltzer, C. and Fetheroff, C. W., "A direct variationalmethod for the calculation of optimum thrust programs forpower limited interplanetary flight," Astronaut. Acta 7, 8(1961).

321 Boyanovitch, D., "The use of Galerkin's method in theoptimal stabilization of control systems," Grumman AircraftEngineering Co. RN-161 (July 1963).

322 Geiss, G. R., "The analysis and design of nonlinear con-trol systems via Liapunov's direct method," RTD-TDR-63-4076, Wright-Patterson Air Force Base, Ohio (August 1964);also D. Boyanovitch, Appendix VII, p. 285.

323 Halkin, H., "The principle of optimal evaluation,"Contributions to the Theory of Differential Equations (AcademicPress, Inc., New York, 1962), Vol. III.

324 Roxin, E., "A geometric interpretation of Pontriagin'smaximum principle, Research Institute for Advanced StudiesTR 61-15 (December 1961).

Dow

nloa

ded

by B

eiha

ng U

nive

rsity

(C

NPI

EC

- X

I'AN

BR

AN

CH

) on

May

12,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/3.3

307


325 Contensou, P., "Theoretical study of optimal trajectoriesin a gravitational field. Application in the case of a single centreof attraction," Astronaut. Acta, Grumman Research Translation,TR-22 (August 1962).

326 Blaquiere, A. and Leitmann, G., "On the geometry ofoptimal processes—part I," Rept. AM-64-10, Institute ofEngineering Research, Univ. of California, Berkeley, Calif.(June 1964); also Leitmann, G., "Some geometrical aspects ofoptimal processes," SIAM J., Series A: Control 3 (1965);also Blaquiere, A., "Further investigation into the geometry ofoptimal processes," SIAM J., Series A: Control 3 (1965).

327 Al'brekht, E. G., "On the optimal stabilization of nonlinearsystems," J. Appl. Math. Mech. 25 (1961).

328 Anderson, C. A., "Switching criteria for certain contactorservos with arbitrary inputs," Univ. of Michigan, Ph.D. Thesis(1958).

329 Aoki, M., "Synthesis of optimal controllers for a class ofmaximization problems," Automatica I, 69 (January-March1963); also Aoki, M., "On a successive approximation tech-nique in solving some control system optimization problems,"J. Basic. Engr. 177 (June 1963).

330 pmes> g>? "Constants of the motion for optimum thrusttrajectories in a central force field," AIAA J. 2, 2010 (November1964).

*33i Flugge-Lotz, I. and Halkin, EL, "Pontriagin's maximumprinciple and optimal control," Stanford Univ. TR 130; alsoAir Force Office of Scientific Research TN 1489 (September1961).

332 Neustadt, L. W., "The existence of optimal controls in theabsence of convexity conditions," J. Math. Anal. Appl. 7, 110(August 1963).

333 Woodrow, P. J., "Methods for locating the extrema ofmultivariate functions," Princeton Univ., Dept. of ElectricalEngineering TR 4 (November 1962).

334Hestenes, M. R. and Stiefel, E., "Method of conjugategradients for solving linear systems," J. Res. Natl. Bur. Std.49,409-436(1952).

335 Curry, H. B., "The method of steepest descent for non-linear minimization problems," Quart. Appl. Math. II (October1944).

336Gel'fand, I. M. and Tsetlin, M. L., "Nonlocal searchprinciple in problems of automatic optimization," Dokl. Akad.Nauk. SSSR137,295-298 (1961); English transl. US JPRS 4687.

337 Levenberg, K., "A method for the solution of certain non-linear problems in least squares," Quart. Appl. Math. II (July1944).

338 Edelbaum, T. N., "Theory of maxima and minima,"Optimization Techniques, edited by G. Leitmann (AcademicPress Inc., New York, 1962), Chap. L

339 Spang, H. A., Ill, "A review of minimization techniques fornonlinear functions," SIAM Review 4, (October 1962).

340 Wilde, D. J., Optimum Seeking Methods (Prentice-Hall,Inc., Englewood Cliffs, N. J. 1964).

341 Shah, B. V., Buehler, R. J., and Kempthorne, p., "Somealgorithms for minimizing a function of several variables," J.SIAM 12, 74 (March 1964).

342 Tompkins, C. B., "Methods of steep descent," ModernMathematics for the Engineer, edited by E. F. Beckenbach(McGraw-Hill Book Co., Inc., New York, 1956), Chap. 18.

343 Courant, R., "Variational methods for the solution of

problems of equilibrium and vibrations," Bull. Am. Math. Soc.49 (January 1943).

344 Fadden, E. I. and Gilbert, E. G., "Computational aspectsof the time-optimal control program," Computing Methods inOptimization Problems, edited by A. V. Balakrishnan and L,W. Neustadt (Academic Press Inc., New York, 1964).

345 Davidon, W. C., "Variable metric method for minimiza-tion," Atomic Energy Commission Research and Develop-ment Rept. ANL-5990, Argonne (1959).

346 Fletcher, R. and Powell, M. J. D., "A rapidly convergentdescent method for minimization," Computer J. (July 1963).

347 Powell, M. J. D., "An iterative method for finding station-ary values of a function of several variables," Computer J.(July 1962).

348 Paiewonsky, B. H., Woodrow, P., Terkelsen, F., and Mc-Intyre, J., "A study of synthesis techniques for optimal con-trol," Aeronautical Systems Div. ASD-TDR-63-239, Wright-Patterson Air Force Base, Ohio (June 1964).

349 Paiewonsky, B., Woodrow, P., Brunner, W., and Halbert,P., "Synthesis of optimal controllers using hybrid analog-digitalcomputers," Computing Methods in Optimization Problems,edited by A. V. Balakrishnan and L. W. Neustadt (AcademicPress Inc., New York, 1964).

350 Elsgolc, L. E., Calculus of Variation (Addison-WesleyPublishing Company, Inc., Reading, Mass., 1962).

351 Kelley, H. J., "Guidance theory and extremal fields,"Inst. Radio Engrs., Trans. Auto. Control AC-7 (October 1962).

352 Breakwell, J. V., Speyer, J. L., and Bryson, A. E., "Opti-mization and control of. nonlinear systems using the secondvariation," SIAM J. Series A: Control 1, p. 193 (1963).

353 Merriam, C. W., Ill, "Use of a mathematical errorcriterion in the design of adaptive control systems," Am. Inst.Elec. Engrs. Trans., Appl. Ind. 78, 506-12 (1959) (January1960 Section); also "An algorithm for the iterative solution of aclass of two-point boundary value problems," SIAM J. SeriesA: Control 2,1(1964).

354 Rang, E. R., "On error analysis for closed-loop optimalguidance systems," Honeywell, MP 691, Rept. 1546-TR1 (June15,1964).

355 Gragg, B. B., "Computation of approximately optimalcontrol," Stanford Univ. SUDAER 179 (January 1964).

356 Ellert, F. J. and Merriam, C. W., Ill, "Synthesis of feed-back controls using optimization theory—an example," Inst.Elec. Electron. Engrs. Trans., Auto. Control AC-8 (April 1963).

357 Bailey, F. B., "The application of the parametric expansionmethod of control optimization to the guidance and controlproblem of a rigid booster," Inst. Elec. Electron. Engrs., Trans.Auto. Control 9 (January 1964).

358 Schwartz, L., "Optimization techniques—a comparison,"U. S. Air Force Flight Dynamics Lab. TDR-64^21, Wright-Patterson Air Force Base, Ohio (March 1964).

359 Peterson, E. L. and Remond, F. X., "Investigations ofdynamic programming methods for flight control problems,"U. S. Air Force Flight Dynamics Lab. TDR-64-11, Wright-PattersonAir Force Base, Ohio (March 1964).

360 Balakrishnan, A. V. and Neustadt, C. W., "Computingmethods in optimization problems," Proceedings of the January1964 UCLA Conference (Academic Press Inc., New York, 1964).

361 Miller, B., "Microcircuitry production growth outpacesapplication," Aviation Week, 76-87 (November 16, 1964).

362 "Integrated circuitry: I," Interavia XX, 409-413 (Marck1965).

Dow

nloa

ded

by B

eiha

ng U

nive

rsity

(C

NPI

EC

- X

I'AN

BR

AN

CH

) on

May

12,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/3.3

307

This article has been cited by:

1. Marc Rouff. 1997. Analytical computation of differential equations involved in dynamical nonlinear optimal problems.Computer Methods in Applied Mechanics and Engineering 142:1-2, 153-163. [CrossRef]

2. Gene H. Golub, Dianne P. O’Leary. 1989. Some History of the Conjugate Gradient and Lanczos Algorithms: 1948–1976. SIAM Review 31:1, 50. [CrossRef]

3. M. L. Audu, D. T. Davy. 1988. A Comparison of optimal control algorithms for complex bioengineering studies. OptimalControl Applications and Methods 9:1, 101-106. [CrossRef]

4. Marc Rouff, Françoise Lamnabhi-Lagarrigue. 1986. A new approach to nonlinear optimal feedback law. Systems & ControlLetters 7:5, 411-417. [CrossRef]

5. F. L. Chernous'ko, V. B. Kolmanovskii. 1979. Computational and approximate methods of optimal control. Journal ofSoviet Mathematics 12:3, 310-353. [CrossRef]

6. A.G.J. Macfarlane. 1972. A survey of some recent results in linear multivariable feedback theory. Automatica 8:4, 455-492.[CrossRef]

7. R. O. ROGERS, D. D. SWORDER. 1970. Approximate minimum energy control. AIAA Journal 8:12, 2242-2244.[Citation] [PDF] [PDF Plus]

8. I. Flügge-Lotz. 1970. The importance of optimal control for the practical engineer. Automatica 6:6, 749-753. [CrossRef]9. J. LOOMISOptimal longitudinal control for obstacle clearance on takeoff . [Citation] [PDF] [PDF Plus]

10. J. R. DOLL, F. W. GOBETZ. 1969. A survey of impulsive trajectories. AIAA Journal 7:5, 801-834. [Citation] [PDF][PDF Plus]

11. A. C. ROBINSON. 1969. Survey of dynamic analysis methods for flight control design. Journal of Aircraft 6:2, 81-98.[Citation] [PDF] [PDF Plus]

12. References 59, 231-233. [CrossRef]13. A. E. SABROFF. 1968. Advanced spacecraft stabilization and control techniques. Journal of Spacecraft and Rockets 5:12,

1377-1393. [Citation] [PDF] [PDF Plus]14. Menahem Sidar. 1968. An iterative algorithm for optimum control problems. International Journal of Non-Linear

Mechanics 3:1, 1-16. [CrossRef]15. A. E. BRYSON, JR.. 1967. Applications of optimal control theory in aerospace engineering. Journal of Spacecraft and

Rockets 4:5, 545-553. [Citation] [PDF] [PDF Plus]16. Eustratios Kounias. 1966. Optimal bounded control with linear stochastic equations and quadratic cost. Journal of

Mathematical Analysis and Applications 16:3, 510-537. [CrossRef]

Dow

nloa

ded

by B

eiha

ng U

nive

rsity

(C

NPI

EC

- X

I'AN

BR

AN

CH

) on

May

12,

201

3 | h

ttp://

arc.

aiaa

.org

| D

OI:

10.

2514

/3.3

307

http://dx.doi.org/10.1016/S0045-7825(96)01126-7

http://dx.doi.org/10.1137%2F1031003

http://dx.doi.org/10.1002/oca.4660090110

http://dx.doi.org/10.1016/0167-6911(86)90060-5

http://dx.doi.org/10.1007/BF01098370

http://dx.doi.org/10.1016/0005-1098(72)90105-7

http://dx.doi.org/10.2514/3.6094

http://arc.aiaa.org/doi/pdf/10.2514/3.6094

http://arc.aiaa.org/doi/pdfplus/10.2514/3.6094

http://dx.doi.org/10.1016/0005-1098(70)90022-1

http://dx.doi.org/10.2514/6.1970-963

http://arc.aiaa.org/doi/pdf/10.2514/6.1970-963

http://arc.aiaa.org/doi/pdfplus/10.2514/6.1970-963

http://dx.doi.org/10.2514/3.5231



http://dx.doi.org/10.2514/3.44013



http://dx.doi.org/10.1016/S0076-5392(08)62650-8

http://dx.doi.org/10.2514/3.29492



http://dx.doi.org/10.1016/0020-7462(68)90021-8

http://dx.doi.org/10.2514/3.28907



http://dx.doi.org/10.1016/0022-247X(66)90159-4

optimal control a review of theory and practice

Documents