denn optimization by variational methods

OPTIMIZATION BYVARIATIONAL METHODSMORMON M. DENN

OPTIMIZATION BYVARIATIONAL METHODS

MORTON M. DENN

Associate Professor of Chemical EngineeringUniversity of Delaware

Preface

The development of a systematic theory of optimization since the mid-1950s has not been followed by widespread application to the design andcontrol problems of the process industries. This surprising and disap-pointing fact is largely a consequence of the absence of effective communi-cation between theorists and process engineers, for the latter typically donot have ,sufficient mathematical training to look past the sophisticationwith which optimization theory is usually presented and recognize thepractical significance of the results. This book is an attempt to present alogical development of optimization theory at an elementary mathematicallevel, with applications to simple but typical process design and controlsituations.

The book follows rather closely a course in optimization which Ihave taught at the University of Delaware since 1965 to classes made upof graduate students and extension students from local industry, togetherwith some seniors. Approximately half of the students each year havebeen chemical engineers, with the remainder made up of other types ofengineers, statisticians, and computer scientists. The only formal

vii

rill PREFACE

mathematical prerequisites are a familiarity with calculus throughTaylor series and a first course in ordinary differential equations, togetherwith the maturity and problem awareness expected of students at thislevel. In two sections of Chapter 3 and in Chapter 11 a familiarity withpartial differential equations is helpful but not essential. With this back-ground it is possible to proceed in one semester from the basic elements ofoptimization to an introduction to that current research which is of directprocess significance. The book deviates from the course in containingmore material of an advanced nature than can be reasonably covered inone semester, and in that sense it may be looked upon as comprising inpart a research monograph.

Chapter 1 presents the essential features of the variational approachwithin the limited context of problems in which only one or a finite num-ber of discrete variables is required for optimization. Some of thismaterial is familiar, and a number of interesting control and designproblems can be so formulated. Chapter 2 is concerned with the parallelvariational'development of methods of numerical computation for suchproblems. The approach of Chapter 1 is then resumed in Chapters 3to 5, where the scope of physical systems analyzed is gradually expandedto include processes described by several differential equations withmagnitude limits on the decisions which can be made for optimization.The optimal control of a flow reactor is a typical situation.

In Chapters 6 and 7 much of the preceding work is reexamined andunified in the context of the construction of Green's functions for linearsystems. It is only here that the Pontryagin minimum principle, whichdominates modern optimization literature, is first introduced in its com-plete form and carefully related to the more elementary and classicalmaterial which is sufficient for most applications.

Chapter 8 relates the results of optimization theory to problems ofdesign of practical feedback control systems for lumped multivariableprocesses.

Chapter 9 is an extension of the variational development of prin-ciples of numerical computation first considered in Chapter 2 to the morecomplex situations now being studied.

Chapters 10 and 11 are concerned with both the theoretical develop-ment and numerical computation for two extensions of process signifi-cance. The former deals with complex structures involving recycle andbypass streams and with periodic operation, while the latter treatsdistributed-parameter systems. Optimization in periodically operatedand distributed-parameter systems represents major pertinent efforts ofcurrent research.

Chapter 12 is an introduction to dynamic programming andHamilton-Jacobi theory, with particular attention to the essential

PREFACE ix

equivalence in most situations between this alternate approach and thevariational approach to optimization taken throughout the remainderof the book. Chapter 12 can be studied at any time after the first halfof Chapters 6 and 7, as can any of the last five chapters, except thatChapter 9 must precede Chapters 10 and 11.

Problems appear at the end of each chapter. Some supplement thetheoretical developments, while others present further applications.

In part for reasons of space and in part to maintain a gonsistentmathematical level I have omitted any discussion of such advancedtopics as the existence of optimal solutions, the Kuhn-Tucker theorem,the control-theory topics of observability and controllability, andoptimization under uncertainty. I have deliberately refrained fromusing matrix notation in the developments as a result of my experiencein teaching this material; for I have found that the very concisenessafforded by matrix notation masks the significance of the manipulationsbeing performed for many students, even those with an adequate back-ground in linear algebra. For this reason the analysis in several chaptersis limited to two-variable processes, where every term can be convenientlywritten out.

In preparing this book I have incurred many debts to colleagues andstudents. None will be discharged by simple acknowledgement, butsome must be explicitly mentioned. My teacher, Rutherford Aris, firstintroduced me to problems in optimization and collaborated in thedevelopment of Green's functions as the unifying approach to variationalproblems. J. R. Ferron, R. D. Gray,_Jr., G. E. O'Connor, A. K. Wagle,and particularly J. M. Douglas have been most helpful in furthering myunderstanding and have permitted me to use the results of our jointefforts. The calculations in Chapter 2 and many of those in Chapter 9were carried out by D. H. McCoy, those in Section 10.8 by G. E. O'Con-nor, and Figures 6.1 to 6.4 were kindly supplied by A. W. Pollock. Myhandwritten manuscript was expertly typed by Mrs. Frances Phillips.For permission to use copyrighted material I am grateful to my severalcoauthors and to the following authors and publishers:

The American Chemical Society for Figures 4.2, 5.7, 5.8, 9.1, 9.2, 9.7 to9.13, 10.5 to 10.15, 11.8 to 11.12, which appeared in Industrial andEngineering Chemistry Monthly and Fundamentals Quarterly.

Taylor and Francis, Ltd., for Figures 11.1 to 11.7 and Sections 11.2 to11.7, a paraphrase of material which appeared- in InternationalJournal of Control.

R. Aris, J. M. Douglas, E. S. Lee, and Pergamon Press for Figures 5.9,5.15, 9.3, and Table 9.8, which appeared in Chemical EngineeringScience.

x PREFACE

D. D. Perlmutter and the American Institute of Chemical Engineers forFigures 8.1 and 8.2, which appeared in AIChE Journal.

Several of my colleagues at the University of Delaware have shapedmy thinking over the years about both optimization and pedagogy, andit is my hope that their contributions to this book will be obvious at leastto them. J. M. Douglas and D. D. Perlmutter have kindly read theentire manuscript and made numerous helpful suggestions for improve-ment. For the decision not to follow many other suggestions fromstudents and colleagues and for the overall style, accuracy, and selectionof material, I must, of course, bear the final responsibility.

MORTON M, DENN

Contents

Preface

Introduction

vii

OPTIMIZATION AND ENGINEERING PRACTICE 1

BIBLIOGRAPHICAL NOTES 2

Chapter 1 OPTIMIZATION WITH DIFFERENTIAL CALCULUS

1.1 Introduction 41.2 The Simplest Problem 41.3 A Variational Derivation 7

1.4 An Optimal-control Problem: Formulation 10

1.5 Optimal Proportional Control 12

1.6 Discrete Optimal Proportional Control 13

1.7 Discrete Optimal Control 15

1.8 Lagrange Multipliers 18

1.9 A Geometrical Example 21

A

X11 CONTENTS

1.10 Discrete Proportional Control with Lagrange Multipliers 23

1.11 Optimal Design of Multistage Systems 241.12 Optimal Temperatures for Consecutive Reactions 27

1.1'3 One-dimensional Processes 291.14 An Inverse Problem 301.15 Meaning of the Lagrange Multipliers 321.16 Penalty Functions 34

APPENDIX 1.1 Linear Difference Equations 36BIBLIOGRAPHICAL NOTES 38

PROBLEMS 40

Chapter 2 OPTIMIZATION WITH DIFFERENTIAL CALCULUS: COMPUTATION 44

2.1 Introduction 442.2 Solution of Algebraic Equations 452.3 An Application of the Newton-Raphson Method 462.4 Fibonacci Search 49

2.5 Steep Descent 52

2.6 A Geometric Interpretation 532.7 An Application of Steep Descent 552.8 The Weighting Matrix 57

2.9 Approximation to Steep Descent 59APPENDIX 2.1 Optimality of Fibonacci Search 62APPENDIX 2.2 Linear Programming 65BIBLIOGRAPHICAL NOTES 68

PROBLEMS 70

Chapter 3 CALCULUS OF VARIATIONS 73

3.1 Introduction 733.2 Euler Equation -73

3.3 Brachistochrone 77

3.4 Optimal Linear Control 793.5 A Disjoint Policy 913.6 Integral Constraints 843.7 Maximum Area 853.8 An Inverse Problem 863.9 The Ritz-Galerkin Method 883.10 An Eigenvalue Problem 903.11 A Distributed System 91

3.12 Control of a Distributed Plant 93BIBLIOGRAPHICAL NOTES 96

PROBLEMS 97

CONTENTS xUI

Chapter 4 CONTINUOUS SYSTEMS-I 100

4.1 Introduction 100

4.2 Variational Equations 101

4.3 First Necessary Conditions 105

4.4 Euler Equation 108

4.5 Relation to Classical Mechanics 1094.6 Some Physical Equations and Useful Transformations 1104.7 Linear Feedback Control 114

4.8 An Approximate Solution 117

4.9 Control with Continuous Disturbances 121

4.10 Proportional Plus Reset Control 124

4.11 Optimal-yield Problems 125

4.12 Optimal Temperatures for Consecutive Reactions 128

4.13 Optimal Conversion in a Pressure-controlled Reaction 130BIBLIOGRAPHICAL NOTES 130PROBLEMS 132

Chapter 5 CONTINUOUS SYSTEMS-II 135


5.2 Necessary Conditions 136

5.3 A Bang-bang Control Problem 138

5.4 A Problem of Nonuniqueness 142

5.5 Time-optimal Control of a Stirred-tank Reactor 144

5.6 Nonlinear Time Optimal Control 150

5.7 . Time-optimal Control of Underdamped Systems 152

5.8 A Time-and-fuel Optimal Problem 155

5.9 A Minimum-integral-square-error Criterion and Singular Solutions 159

5.10 Nonlinear Minimum-integral-square-error Control 163

5.11 Optimal Cooling Rate in Batch and Tubular Reactors 165

5.12 Some Concluding Comments 169


PROBLEMS 171

Chapter 6 THE MINIMUM PRINCIPLE 175


6.2 Integrating Factors and Green's Functions 176

6.3 First-order Variational Equations 180

6.4 The Minimization Problem and First Variation of the Objective 181

6.5 The Weak Minimum Principle 184

6.6 Equivalent Formulations 187

6.7 An Application with Transversality Conditions 189

xtr CONTENTS

6.8 The Strong Minimum Principle 191

6.9 The Strong Minimum Principle: A Second Derivation 1946.10 Optimal Temperatures for Consecutive Reactions 197

6.11 Optimality of the Steady State 199

6.12 Optimal Operation of a Catalytic Reformer 2026.13 The Weierstrass Condition 2076.14 Necessary Condition for Singular Solutions 2076.15 Mixed Constraints 2096.16 State-variable Constraints 211

6.17 Control with Inertia 2126.18 Discontinuous Multipliers 2146.19 Bottleneck Problems 2176.20 Sufficiency 220

APPENDIX 6.1 Continuous Dependence of Solutions 221BIBLIOGRAPHICAL NOTES 222

PROBLEMS 226

Chapter 7 STAGED SYSTEMS 228

7.1 Introduction 2287.2 Green's Functions 2297.3 The First Variation 2317.4 The Weak Minimum Principle 2327.5 Lagrange Multipliers 2337.6 Optimal Temperatures for Consecutive Reactions 2347.7 The Strong Minimum Principle: A Counterexample 2377.8 Second-order Variational Equations 2387.9 Mixed and State-variable Constraints 242


PROBLEMS 245

Chapter 8 OPTIMAL AND FEEDBACK CONTROL 247

8.1 Introduction 2478.2 Linear Servomechanism Problem 2488.3 Three-mode Control 2508.4 Instantaneously Optimal Relay Control 2548.5 An Inverse Problem 2578.6 Discrete Linear Regulator 262

APPENDIX 8.1 Liapunov Stability 265BIBLIOGRAPHICAL NOTES 266

PROBLEMS 269

CONTENTS xv

Chapter 9 NUMERICAL COMPUTATION 271


9.2 Newton-Raphson Boundary Iteration 271

9.3 Optimal Temperature Profile by Newton-Raphson Boundary Iteration 2749.4 Steep-descent Boundary Iteration 278

9.5 Newton-Raphson Function Iteration: A Special Case 283

9.6 Newton-Raphson Function Iteration: General Algorithm 288

9.7 Optimal Pressure Profile by Newton-Raphson Function Iteration 2909.8 General Comments on Indirect Methods 293


9.10 Steep Descent: Optimal Pressure Profile 2999.11 Steep Descent: Optimal Temperature Profile 301

9.12 Steep Descent: Optimal Staged Temperatures 3049.13 Gradient Projection for Constrained End Points 308

9.14 Min H 311

9.15 Second-order Effects 314

9.16 Second Variation 315

9.17 General Remarks 321


PROBLEMS 325

Chapter 10 NONSERIAL PROCESSES 326


10.2 Recycle Processes 327

10.3 Chemical Reaction with Recycle 329

10.4 An Equivalent Formulation 331

10.5 Lagrange Multipliers 332

10.6 The General Analysis 333

10.7 Reaction, Extraction, and Recycle 337

10.8 Periodic Processes 348

10.9 Decomposition 355


PROBLEMS 358

Chapter 11 DISTRIBUTED-PARAMETER SYSTEMS 359


11.2 A Diffusion Process 359

11.3 Variational Equations 360

11.4 The Minimum Principle 362

11.5 Linear Heat Conduction 364


xvi CONTENTS

11.7 Computation for Linear Heat Conduction11.8 Chemical Reaction with Radial Diffusion11.9 Linear Feedforward-Feedback Control11.10 Optimal Feed Distribution in Parametric Pumping11.11 Concluding Remarks

BIBLIOGRAPHICAL NOTES

PROBLEMS:

366371377381

387387389

Chapter 12 DYNAMIC PROGRAMMING AND HAMILTON-JACOBI THEORY 392

12.1 I ntroduction 39212.2 The Principle of Optimality and Comnutation 39312.3 Optimal Temperature Sequences 394

12.4 The Hamilton-Jacobi-Bellman Equation 39812.5 A Solution of the Hamilton-Jacobi-Bellman Equation 40012.6 The Continuous Hamilton-Jacobi-Bellman Equation 40212.7 The Linear Regulator Problem 405

BIBLIOGRAPHICAL NOTES 407PROBLEMS 407

Name Index 411

Subject Index 415

Introduction

OPTIMIZATION AND ENGINEERING PRACTICE

The optimal design and control of systems and industrial processes haslong been of concern to the applied scientist and engineer, and, indeed,might be taken as a definition of the function and goal of engineering.The practical attainment of an optimum design is generally a conse-quence of a combination of mathematical analysis, empirical information,and the subjective experience of the scientist and engineer. In the chap-ters to follow we shall examine in detail the principles which underlie.theformulation and resolution of the practical problems of the analysis andspecification of optimal process units and control systems. Some of theseresults lend themselves to immediate application, while others providehelpful insight into the considerations which must enter into the specifi-cation and operation of a working system.

The formulation of a process or control system design is a trial-and-error procedure, in which estimates are made first and then infor-mation is sought from the system to determine improvements. When a

2 OPTIMIZATION BY VARIATIONAL METHQDS

sufficient mathematical characterization of the system is available, theeffect of changes about a preliminary design may be obtained analyti-cally, for the perturbation techniques so common in modern appliedmathematics and engineering analysis have their foundation in linearanalysis, despite the nonlinearity of the system being analyzed. Whethermathematical or experimental or a judicious combination of both, pertur-bation analysis lies at the heart of modern engineering practice.

Mathematical optimization techniques have as their goal the devel-opment of rigorous procedures for the attainment of an optimum in asystem which can be characterized mathematically. The mathematicalcharacterization may be partial or complete, approximate or exact,empirical or theoretical. Similarly, the resulting optimum may be afinal implementable design or a guide to practical design and a criterionby which practical designs are to be judged. In either case, the optimi-zation techniques should serve as an important part of the total effort inthe design of the units, structure, and control of a practical system.

Several approaches can be taken to the development of mathemati-cal methods of optimization, all of which lead to essentially equivalentresults. We shall adopt here the variational method, for since it isgrounded in the analysis of small perturbations, it is the procedure whichlies closest to the usual engineering experience. The general approach isone of assuming that a preliminary specification has been made and thenenquiring about the effect of small changes. If the specification is in factthe optimum, any change must result in poorer performance and the pre-cise mathematical statement of this fact leads to necessary conditions, orequations which define the optimum. Similarly, the analysis of the effectof small perturbations about a nonoptimal specification leads to compu-tational procedures which produce a better specification. Thus, unlikemost approaches to optimization, the variational method leads rathersimply to both necessary conditions and computational algorithms byan identical approach and, furthermore, provides a logical framework forstudying optimization in new classes of systems.


An outstanding treatment of the logic of engineering design may be found in

D. F. Rudd and C. C. Watson: "Strategy of Process Engineering," John Wiley &Sons, Inc., New York, 1968

Mathematical simulation and the formulation of system models is discussed in

A. E. Rogers and T. W. Connolly: "Analog Computation in Engineering Design,"McGraw-Hill Book Company, New York, 1960

R. G. E. Franks: "Mathematical Modeling in Chemical Engineering," John Wiley &Sons, Inc., New York, 1967

INTRODUCTION 3

Perturbation methods for nonlinear systems are treated in such books as

W. F. Ames: "Nonlinear Ordinary Differential Equations in Transport Processes,"Academic Press, Inc., New York, 1968

"Nonlinear Partial Differential Equations in Engineering," Academic Press,Inc., New York, 1965

R. E. Bellman: "Perturbation Techniques in Mathematics, Physics, and Engineering,"Holt, Rinehart and Winston, Inc., New York, 1964

W. J. Cunningham: "Introduction to Nonlinear Analysis," McGraw-Hill BookCompany, New York, 1958

N. Minorsky: "Nonlinear Oscillations," I). Van Nostrand Company, Inc., Princeton,N.J., 1962

Perhaps the most pertinent perturbation method from a system analysis viewpoint, theNewton-Raphson method, which we shall consider in detail in Chaps. 2 and 9, isdiscussed in

R. E. Bellman and R. E. Kalaba: "Quasilinearization and Nonlinear BoundaryValue Problems," American Elsevier Publishing Company, New York, 1965

1

Optimization with DifferentialCalculus

1.1 INTRODUCTION

A large number of interesting optimization problems can be formulatedin such a way that they can be solved by application of differential calcu-lus, and for this reason alone it is well to begin a book on optimizationwith an examination of the usefulness of this familiar tool. We have afurther motivation, however, in that all variational methods may be con-sidered to be straightforward extensions of the methods of differentialcalculus. Thus, this first chapter provides the groundwork and basicprinciples for the entire book.

1.2 THE SIMPLEST PROBLEM

The simplest optimization problem which can be treated by calculus is thefollowing: S(Xl,x2, . . . is a function of the n variables xl, x2, . . . ,

x,,. Find the particular values x,, xz, . . . , x which cause the function& to take on its minimum value.4

'IMIZATION WITH DIFFERENTIAL CALCULUS S

We shall solve this problem in several ways. Let us note first thatthe minimum has the property that

g(xl,x2j . . . ,xn) - g(xl,x2. . . . ,xn) 0 (1)

Suppose that we let xl = x, + 8x1, where Sxl is a small number in abso-lute value, while x2 = x2, x3 = z3, . . , X. = L. If we divide Eq. (1)by 8x1, we obtain, depending upon the algebraic sign of 8x1,

&(xl + 8x1, x2, . . . , xn) - 6(x1,x2, . . . xn) > 0 8x1 > 0 (2a)8x1

or

g\xl + Ext, 22, xn) - g(xl,x2, . . .

ax,n))<0 Sxl < 0 (2b)

The limit of the left-hand side as Sxi - 0 is simply ag/axi, evaluated at21, x2, . . . , xn. From the inequality (2a) this partial derivative isnonnegative, while from (2b) it is nonpositive, and both inequalities aresatisfied only if a&/ax, vanishes. In an identical way we find for all xk,k=1,2,...,n,

ag = 0 (3)axk

at the minimizing values ti, x2, . , 2n We thus have n algebraicequations to solve for the is unknowns, xl, x2, . . . , xn-

It is instructive to examine the problem somewhat more carefullyto search for potential difficulties. We have, for example, made a ratherstrong assumption in passing from inequalities (2a) and (2b) to Eq. (3),.namely, that the partial derivative in Eq. (3) exists at the values x1,x2; . . . , xn. Consider, for example, the function

g(xt,x2, ,x,,) = Ix1I +x21 + ... + IX-1 (4)

which has. a minimum at xl = x2 = = x,,,= 0. Inequalities (1)and (2) are satisfied, but the partial derivatives in Eq. (3) are not definedat xk = 0. If we assume that one-sided derivatives of the. function &exist everywhere, we must modify condition (3) to -

limag < 0 (5a)

x,,-,pk- axk

limag > 0

Zk-*ik* axk(5b)

with Eq. (3) implied if the derivative is continuous.In problems which describe real situations, we shall often find that

physical or economic interpretations restrict the range of variables we

6 OPTIMIZATION BY VARIATIONAL METHODS

may consider. The function

8 = Y2 (x + 1) 2 (6)

for example, has a minimum at x = -1, where Eq. (3) is satisfied, but ifthe variable x is restricted to nonnegative values (an absolute tempera-ture, or a cost, for example), the minimum occurs at the boundary x = 0.In writing the inequalities (2a) and (2b) we have assumed that we werefree to make Sxa positive or negative as we wished. But if xa lies at alower bound of its allowable region, we cannot decrease xa any more, sothat inequality (2b) is inadmissable, and similarly, if za lies at an upperbound, inequality (2a) is inadmissable. We conclude, then, that zk eithersatisfies Eq. (3) for inequalities (5a) and (5b)] or lies at a lower boundwhere a8/axk >- 0, or at an upper bound where a8/axk < 0; that is,

< 0 zk at upper bounda8 = 0 xk between bounds [or (5a) and (5b)] (7)

axk > 0 xk at lower bound

Thus, to minimize Eq. (6) subject to 0 < x < ao we find a8/ax - 1 > 0at z = 0.

Several more restrictions need to be noted. First, if we were seek-ing to maximize 8, we could equally well minimize -8. But from Eq. (3),

a(-8) = - a8 a8axk axk axk

which is identical to the condition for a minimum, so that if fk is aninterior point where the derivative is continuous, we have not yet founda way to distinguish between a maximizing and minimizing point.Indeed, a point at which Eq. (3) is satisfied need be neither a maxi-mum nor a minimum; consider 6 = (XI)2 - (x2)2 Furthermore, whileinequalities (1) and (2) are true for all xa, x2, ... , x, from the mean-ing of a minimum, and hence define a global minimum, when we allowax, to become small and obtain Eqs. (3), (5), and (7), we obtain con-ditions which are true for any local minimum and which may have manysolutions, only one of which leads to the true minimum value of 6.

These last points may be seen by considering the function

6(x) = Y3x' - x + 3 (g).

shown in Fig. 1.1. If we calculate d8/dx = 0, we obtain

dx=x2-1=0 (9)

or

x= ±1 (10)

OPTIMIZATION WITH DIFFERENTIAL CALCULUS

Fig. 1.1 A function with a local mini-mum and maximum: E(x) = ?sx= -x+3's.

7

and we see by inspection of the figure that & is a minimum at x = +1and a maximum at x = -1, with 8(1) = 0. But for x < -2 we have8(x) < 0, so that unless we restrict x to x > -2, the point x = 1 is onlya local minimum. If, for example, we allow -3 < x < oo, then at theboundary x = -3 we have

d&

dx z--3 = (-3)2 - 1 = 8 > 0 (11)

and so the point x = -3 also satisfies Eq. (7) and is in fact the globalminimum, since

g(-3) = -1s3 < a(+1) = 0 (12)

1.3 A VARIATIONAL DERIVATION

In order to motivate the development in later chapters, as well as toobtain a means of distinguishing between maxima and minima, we shallconsider an alternative treatment of the problem of the previous section.For simplicity we restrict attention to the case n = 2, but it will be obvi-ous that the method is general. We seek, then, the minimum of thefunction 3(x1,x2), and for further simplicity we shall assume that thefunction has continuous first partial derivatives and that the minimumoccurs at interior values of x1, x2.

If g has bounded second partial derivatives, we may use a two-dimensional Taylor series to express &(x1 + &x1, x2 + &x2) in terms ofS(x1,x2), where bx3, bx2 are any small changes (variations) in x1, x2 whichare consistant with the problem formulation:

6(x1 + bx1, x2 + bx2) - 3(x1,x2) =a&i

6x1 +as ax2

ax1 T ax2+ o(max I6xl1,I6x2) (1)

where the partial derivatives are evaluated at x1 = x1, x2 = x2 and the

! OPTIMIZATION BY VARIATIONAL METHODS

notation o(e) refers to a term which goes to zero faster than e; that is,

lim o(E) = 0 (2)r_o E

In Eq. (1),

o(max I6x,I,1Sx21) =2 [ax? (axl)e + 2 ax

zaxzEx, 6x2

+az

2(Sxz)' (3)

where the second partial derivatives are evaluated at some point

[Pxi + (1 - P) (xa + Sxi), P12 + (1 - P)42 + Sxz) ] 0 < P < 1

If za, x2 are the minimizing values then, from Eq. (1) and Eq; (1) of thepreceding section,

ar,ax

Sx, + Sx'ax2 + o(max ISxiI,f Ox2I) >_ 0 (4)

Now ax, and 5X2 are arbitrary, and Eq. (4) must hold for any vari-ations we choose. A particular pair

Sxa = -E 6x2 = -Eax, aX2

(5)

where a is a small positive constant and the partial derivatives are againevaluated at z,, x2. If these partial derivatives both vanish,. inequality(4) is satisfied trivially; if not, we may write

-E Ka&), + + o(e) > 0 (6)

or, dividing by e and taking the limit as a with Eq. (2),

()2(,3g)2 + < 0 (7)

But a sum of squares can be nonpositive if and only if each term in thesum vanishes identically; thus, if 21'and x2 are the minimizing values,it is necessary that

a3 = 38 =0ax, ax2

(8)

We may now obtain a further condition which will allow us to dis-tinguish between maxima and minima by assuming that C is three times


differentiable and carrying out the Taylor series an additional term:

8(x1 + ax,, 22 + axe) - 8(!1,x2) _ 1 ax, + r 6x2

E

2 z z

+ 2 ax (ax,) 2 + 2ax49

, ax2ax, 5x2 + a22 (0x2) 2

+ o(max I6x,{2,10x212) >- 0 (9)

The terms with arrows through them vanish by virtue of Eq. (8). If wenow set

ax, = Ea, axz = Eat (10)

where e is an infinitesimal positive number and a,, az are arbitrary finitenumbers of any algebraic sign, Eq. (9) may be written, after dividingby E2,

1

V2 2

828 o(E2)

2(11)a`a'

ax; az + E2> 0

s-1;1 ,

and letting c ---' 0, we find that for arbitrary a;, a; it is necessary that2 2

828a;a; > 0

; 1 f Iax; ax;

The collection of terms

[826 a28

ax, ax, ax, ax2492& 828

ax, ax2 ax2 ax2

(12)

is called the hessian matrix of 8, and a matrix satisfying inequality (12)for arbitrary a,, az is said to be positive semidefinite. The matrix is saidto be positive definite if equality holds only for a,, a2 both equal to zero.

The positive semidefiniteness of the hessian may be expressed in amore convenient form by choosing particular values of a, and a2. Forexample, if at = 1, a2 = 0, Eq. (12) becomes

a2s > 0ax, ax,

(13)

and similarly for 828/(ax2 ax2). On the other hand, if a, = - 828/((jx, ax2),a2 = 192s/(49x, ax,), then Eq. (12) becomes the more familiar result

> o0x,2 0x22 a2:a2s a2s --( a2s 12 (14)


Thus we have proved that, subject to the differentiability assumptionswhich we have made, if the minimum of S(x1,x2) occurs at the interiorpoint z,, x2, it is necessary that the first and second partial derivatives of &satisfy conditions (8), (13), and (14). ' It is inequality (13) which dis-tinguishes a minimum from a maximum and (14) which distinguishesboth from a saddle, at which neither a minimum nor maximum occursdespite the vanishing of the first partial derivatives.

Let us note finally that if Eq. (8) is satisfied and the hessian ispositive definite (not semidefinite) at x1, 22, then for sufficiently smallI6x1J, 16x2I the inequality in Eq. (9) will always be satisfied. Thus, thevanishing of the first partial derivatives and the positive definiteness ofthe hessian are sufficient to ensure a (local) minimum at x1, x2.

1.4 AN OPTIMAL-CONTROL PROBLEM: FORMULATION

The conditions derived in the previous two sections may be applied toan elementary problem in optimal control. Let us suppose that we havea dynamic system which is described by the differential equation

dy = F(y,u) (1)

where u is a control variable and y is the state of the system (y might bethe temperature in a mixing tank, for example, and u the flow rate ofcoolant in a heat exchanger). The system is designed to operate at thesteady-state value y,,, and the steady-state control setting is,, is foundby solving the algebraic equation

0 (2)

If the system is disturbed slightly from the steady state at timet = 0, we wish to guide it back in an optimal fashion. There are manycriteria which we might use to define the optimal fashion, but let us sup-pose that it is imperative to keep the system "close" to y,,. If we deter-mine the deviation from steady state y(t) - y at each time, then aquantity which will be useful for defining both positive and negativedeviations is the square, (y - y,,)2, and a reasonable measure of the totaldeviation will be

foe

(y - y,.)2 dt, where 0 is the total control time. Onthe other hand, let us suppose that we wish to hold back the controleffort, which we may measure by foe (u - u..)' dt. If p2 represents therelative values of control effort and deviation from steady state, then ourcontrol problem is as follows.

Given an initial deviation yo and a system defined by Eq. (1), find

OPTIMIZATION WITH DIFFERENTIAL CALCULUS 11

the control settings u(t) which minimize

!o [(y - y..)2 + a2(u - u..)2] dt (3)

For purposes of implementation it is usually helpful to have the controlfunction u not in terms oft but of the state y. This is called feedbackcontrol.

In later chapters we shall develop the necessary mathematics forthe solution of the control problem just stated. At this point we mustmake a further, but quite common, simplification. We assume that thedeviations in y and u are sufficiently small to permit expansion of thefunction F(y,u) in a Taylor series about y,,, u retaining only linear terms..Thus we write

F(y..,u..) +IT

(y - y..) + Bu v-v.. (u - u..) (4)usu.. u-u..

Letting

x -y - y w = -(u-u..)and noting that i = y and 0, we have, finally,

Ax + w x(0) = xo (5)

min & = 2 to (x2 + c2w2) dt (6)

It is sometimes convenient to consider control policies which areconstant for some interval; that is,

w(t) = w = const (n - 1)A < t < n0 (7)

Equation (5) then has the solution

x(nA) = x[(n - 1),&]eA° - eAA)

A

and Eq. (6) may be approximated byN

min 8 = 7220 [x2(nA) + c2wn2]

(8)

(9)

Letting

x = x(nQ) u =w a A

c2A 2C2 a=eAo

(1 _. eA.A)2

eAA)

(10)


we obtain the equivalent control problem

x = u xo givenN

min S =2

(x,,2

+C2un2)

n-1

(11)

(12)

where, since 0 is a constant, we have not included it in the summationfor 6.

1.5 OPTIMAL PROPORTIONAL CONTROL

The most elementary type of control action possible for the systemdefined by Eqs. (5) and (6) of the preceding section is proportional con-trol, in which the amount of control effort is proportional to the devi-ation, a large deviation leading to large control, a small deviation to little.In that case we set

w(t) = Mx(t) (1)z = (A + M)x x(O) = xo (2)

and we seek the optimal control setting M:

min 6(M) = 12(1 + c2M2) fo x2 dt (3)

The solution to Eq. (2) is

x(t) = xoe(A+M)t (4)

and Eq. (3) becomes

min 6(M) = x02(1 + c2M2) foe e2(A+.N)t dt (5)

or

min 6(M) = x02 (1 + MMC2) (e2(A+M)e _ 1) (6)4 A+MSince x cannot be allowed to grow significantly larger because of the con=trol action, it is clear that we require for stability that

A + M < 0 (7)

This is the concept of negative feedback. The optimal control setting isthen seen from Eq. (6) to depend on A, c, and 0 but not on xo, which issimply a multiplicative constant. Since we shall usually be concernedwith controlling for long times compared to the system time constantIA + MI-1, we may let 0 approach infinity and consider only the problem

min 6(M) IA+ M 2(8)


We obtain the minimum by setting the derivative equal to zero

dg _ 2Mc2 1 + M2c2dM

A+M+(A+M)2=0

or

M = -A ± 1 +A 2C2c2

13

(9)

(10)

Condition (7) requires the negative sign, but it will also follow from con-sidering the second derivative.

For a minimum we require d2g/dM2 > 0. This reduces to

-2 1 + A2c2 > 0 (11). (A+M)$-

which yields a/minimum for the stable case A + M < 0, or

w = - CA + 1 cA2c2) x (12)

1.6 DISCRETE OPTIMAL PROPORTIONAL CONTROL

We shall now consider the same problem for the discrete analog describedby Eqs. (11) and (12) of Sec. 1.4,

xn = ax.-1 + u. (1)N

min F. = I (xn2 + C2un2) (2)n-1

We again seek the best control setting which is proportional to the stateat the beginning of the control interval

u = mxn-1 (3)

so that Eq. (1) becomes

xn = (a + (4)

which has a solution

xn = xo(a + m)n (5)

It is clear then that a requirement for stability is

ja+ml < 1 (6)

For simplicity in the later mathematics it is convenient to substi-tute Eq. (4) into (3) and write

un =a

+ mxn = Mxn (7)


and

x = 1 a M xn-1= (1 a

/

M )nxo (8)

From Eq. (2) we then seek the value of M which will minimizeN

a )3(M)

x02(1 + M2C2) 1 - M1nil

or, using the equation for the sum of a geometric series,

(9)

2!1 2C22 1 - 1

M)21](10)&(M) 2 xo2 (1 M)2 -

Ma a

As in the previous section, we shall assume that the operation is suf-ficiently long to allow N --p -, which reduces the problem to

min &(M) =1 1 + M2C2 (11)2(1-M)2-a2

Setting d3/dM to zero leads to the equation

M2-I-(a2-1-} 12)M-C2=0or

(12)

M 2C2 [C2(1 - a2) - 1 ± {[C2(1 - a2) - 1]2 + 4C21] (13)

The positivity of the second derivative implies

M< C2(1 - a2) - 12C2

(14)

so that the negative sign should be used in Eq. (13) and M is negative.The stability condition, Eq. (6), is equivalent to

1 - M > dal (15)

From Eq. (14),

1-M> 2

2C+1a2>-y2(1+a2)> al

where the last inequality follows from

a2 - 21a1 + 1'= (Ial - 1)2 > 0

Thus, as in the previous section, the minimizing control is stable.

(16)

(17)


In Sec. 1.4 we obtained the discrete equations by setting

a=eAA u-wnI

C2c2A2

= - eAA

6AA.

A

If we let A become infinitesimal, we may write

a --- 1 + A4 u -Equation (12) then becomes

2

M o

1 CA2c2 ) xn

C2 C2

Q2

which is identical to Eq. (12) of Sec. 1.5.

1.7 DISCRETE OPTIMAL CONTROL

We now consider the same discrete system

1s

(18)

(19)

(20)

(21)

(22)

xA = u (1)

but we shall not assume that the form of the controller is specified.Thus, rather than seeking a single proportional control setting we wishto find the sequence of settings u1, u2, . . . , ui that will minimize

n-1(2)

If we know all the values of x,,, n = 1, 2, . . . , N, we can calculate u,;from Eq. (1), and so we may equivalently seek

min 8(xl,x2, . . . ,x1V)Ex~2

+ C2(x - axw-1)2)nil

(3)

We shall carry out the minimization by setting the partial derivatives ofC with respect to the x equal to zero.

1$ OPTIMIZATION BY VARIATIONAL METHODS

Differentiating first with respect to xN, we obtain

ac 'axN

or

= xN ± C'(xN - axN-I) = xN + C uN = U

1UN = - C,= xN

Next,

a&

axN-1xN-1 + C2(XN-1 - axN-Y) - aC2(XN

= xN-1 + C'u,v-1 - aC2 UN = 0

and combining Eqs. (1), (5), and (6),

8xN_I= -TN-1 + C2UN-1 + 1 + C2 XN_1 = 0

or

axN-1)

(4)

(5)

(6)

(7)

1 1 + C2 + a2C2X 'V-1UN-1 C' 1 + C2 N_I

In general,

X. + C'(x. - ax,.-1) - aC'(x.+1 - ax.) 0n 1, 2, .. . , N - 1 (9)

and the procedure of Eqs. (4) to (8) may be applied ip order from nto n = 1 to find the optimal control,settings ul, u2, . . . , UN.

We note from Eq. (9) that the optimal control will always be aproportional controller of the form

-1aM1 xa-i

with, from Eq. (5),

MN= -1

If we substitute into Eq. (9), we obtain

x.+C'M.x. -a'C'1 M"M*M,+1 x.

or, if x.00,

=0

(10)

(12)

M.+1-M.-I = 0 (13)M.M.+1+1 C2


with Eq. (11) equivalent to the condition

MN+1 = 0 (14)

Equation (13) is a Riccati difference equation, and the unique solu-tion can be obtained by applying Eq. (13) recursively from n = N ton = 1. An analytical solution can be found by the standard techniqueof introducing a new variable

1

M. - (1/2C2)[1 + a2C2 - C2 ± _01 + a2C2 - C2)2 + 4C2)

where either the positive or negative sign may be used, and solving theresulting linear difference equation (see Appendix 1.1 at the end of thechapter). The solution satisfying Eq. (14) may be put in the form

M = 1 - 2a2C2 [1 + C2 + a2C2 - (1 + a2C2 - C2)2 + 4C2] -in [1 + C2 + a2C2 - 'V(1 + a2C2 - C2)2 + 4C 2]n

+ a[l + C2 + a2C2 + (1 + a2C2 - C2)2 + 4C21n-'

+ all + C2 + a2C2 + \/(I + a2C2 - C2)2 + 4C2]n

where

a x C 1 + C2 + a2C2 -V/(1 + a2C2 - C2)2+4 C2 1N

L 1 + C2 + a2C2 + /(1 + a2C2 - C2)2 + 4C21 + C2 - a2C2 - (1 + a2C2 - C2)2 + 4C21 + C2- aC2 + V (1+ a2C2 - C2)2 +4C2

If we take the control time to be infinite by letting N --> oo, weshall obtain lim xN -' 0, so that Eq. (4) will be automatically satisfiedwithout the need of the boundary condition (11). We find that

lim a = 0N_.

and Mn becomes a constant.

Mn= a2-1+C2+ V(a2+C2/z

+ 4 (17)

which is the optimal proportional control found,in/Sec. 1.6.The result of this section, that the optimal control for the linear sys-

tem described by Eq. (1) and quadratic objective criterion by Eq. (2) is ,aproportional feedback control with the control setting the solution of a Riccatiequation and a constant for an infinite control time, is one of the impor-tant results of modern control theory, and it generalizes to larger sys-tems. We shall return to this and related control problems in Sec. 1.10and in later chapters.


1.8 LAGRANGE MULTIPLIERS

In the previous section we were able to substitute the equation describ-ing the process, Eq. (1), directly into the objective function, Eq. (2), andthus put the problem in the framework of Sec. 1.1, where we needed onlyto set partial derivatives of the objective to zero. It will not always bepossible or desirable to solve the system equations for certain of the varia-bles and then substitute, and an alternative method is needed. One suchmethod is the introduction of Lagrange multipliers.

For notational convenience we shall again restrict our attention toa system with two variables, x, and x2. We seek to minimize 8(xl,x2),but we assume that xl and x2 are related by an equation

g(xl,x2) = 0 (1)

It is convenient to consider the problem geometrically. We can plot thecurve described by Eq. (1) in the x1x2 plane, and for every constant wecan plot the curve defined by

g(x,,x2) = const (2)

A typical result is shown in Fig. 1.2.The solution is clearly the point at which the line g = 0 intersects

the curve of constant 8 with the least value, and if both c%rves are con-tinuously differentiable at the point, then they will be tangent to eachother. Thus, they will possess a common tangent and a common normal.Since the direction cosines for the normal to the curve g = 0 are propor-tional to ag/ax; and those of g = const to a6/ax; and for a common nor-mal the direction cosines must coincide, it follows then that at the opti-mum it is necessary that 8g/ax; be proportional to 8g/8x;; that is, a

Fig. 1.2 Contours and a constraintK, curve.


necessary condition for optimality is

ax + ax

T1 +Xax o

U

(3a)

(3b)

We shall now obtain this result somewhat more carefully and moregenerally. As before, we suppose that we somehow know the optimalvalues x,, 22, and we expand £ in a Taylor series to obtain

3(91 + 6x1, xs + bx2) - &(x,,x2) = ax axl + 8x2axe

+ o(max Iaxil,Iax2I) ? 0 (4)

Now, however, we cannot choose axe and axe in any way that we wish,for having chosen one we see that the constraint, Eq. (1), fixes the other.We can, therefore, freely choose one variation, ax, or ax2, but not both.

Despite any variations in x, and x2, we must satisfy the constraint.Thus, using a Taylor series expansion,

9(x1 + ax,, x: + axe) - 9(x,,xs) = ax ax, + a y axe

+ o(max IaxiI,lax2l) = 0 (5)

If we multiply Eq. (5) by an arbitrary constant X (the Lagrange multi-plier) and add the result to Eq. (4), we obtain

(+X)Oxt+(+X)Oxta& cig a&

+ o(max I3x,l,I6x21) >_ 0 (6)

We ate now free to choose any X we please, and it is convenient to defineX so that the variation we are not free to specify vanishes. Thus, wechoose X such that at x,, 22

+ X x = 0ax, a

which is Eq. (3b). We now have

(7)

(az + a a g) axl + o (max l axl 1, l ax:I) ? 0

and by choosing the special variation

(8)

ax,= -e -+Xc3x) (9)


it follows, as in Sec. 1.3, that at the minimum of C it is necessary that

g=0+X a1ax (10)

which is Eq. (3a). Equations (1), (7), and (10) provide three equationsfor determining the three unknowns x1, f2, and X. We note that by includ-ing the constraint in this manner we have introduced an additional varia-ble, the multiplier X, and an additional equation.

It is convenient to introduce the lagrangian 2, defined

£(x1,x2,X) = 3(xl,x2) + X9(x1,x2) (11)

Equations (7) and (10) may then be written as

a.e aye0=

8x1 = axe

while Eq. (1) is simply

aye=0TX

(13)

Thus, we reformulate the necessary condition in the Lagrange multiplierrule, as follows:

The function & takes on its minimum subject to the constraint equation(1) at a stationary point of the lagrangian C.

(A stationary point of a function is one at which all first partial deiava-tives vanish.) For the general case of a function of n variablesS(x1,x2, . . . and m constraints

g;(x1,x2, . . . 0 i = 1, 2, . . . , m < n (14)

the lagrangiant is written

2(21,x2, . . . ,x,,,X1iX2, . . . ,X.,) = 6(x1,x2, . . . ,x,)

+ X g,(x1,x2, . . . ,x,,) (15):-1

t We have not treated the most general situation, in which the lagrangian must bewritten

Act + A,&. 1

So-called irregular cases do exist in which No - 0, but for all regular situations No maybe taken as unity without loss of generality.


and the necessary condition for a minimum is

ax~0 i=1,2,...,naca,=0 i=1,2,...,m

(16a)

(16b)

It is tempting to try to improve upon the multiplier rule by retain-ing second-order terms in the Taylor series expansions of & and g andattempting to show that the minimum of 3 occurs at the minimum of £.This is often the case, as when g is linear and 8 is convex (the hessian of gis positive definite), but it is not true in generat, for the independentvariation cannot be so easily isolated in the second-order terms of theexpansion for 2. It is easily verified, for example, that the function

3 = x1(1 + x2) (17)

has both a local minimum and a local maximum at the constraint

x1 + (x2)2 = 0 (18)

while the stationary points of the lagrangian

.C = x1(1 + x:) + Xxi + X42), (19)

are neither maxima nor minima but only saddle points.The methods of this section can be extended to include inequality

constraints of the form

g(x1,x2) >_ 0 (20)

but that would take us into the area of nonlinear programming and awayfrom our goal of developing variational methods of optimization, and wemust simply refer the reader at this point to the specialized texts.

1.9 A GEOMETRICAL EXAMPLE

As a first example in the use of the Lagrange multiplier let us considera problem in geometry which we shall find useful in a later discussion ofcomputational techniques. We consider the linear function

g = ax1 + bx2 (1)

and the quadratic constraint

g,= a(xl)e + 2$x1xi + y(x2)s - 0= = 0 (2)

where

a>0 ay-$2>0 (3)


Fig. 1.3 A linear objective with ellipticconstraint.

The curve g = 0 forms an ellipse, while for each value of 3 Eq. (1) definesa straight line. We seek to minimize & subject to constraint (2); i.e., asshown in Fig. 1.3, we seek the intersection of the straight line and ellipseleading to the minimum intercept on the x2 axis.

As indicated by the multiplier rule, we form the lagrangian

E = ax, + bx: + Xa(x1)' + 2a$x1x: + Xy(x2)2 - XQ2 (4)

We then find the stationary pointsat =J721

a + 21%axi + 27$x2 = 0 (5a).

a z = b + 2\8x1 + 2a7xs - 0 (5b)ax,

while 8.c/ax = 0 simply yields Eq. (2). Equations (5) are easily ^olvedfor x1 and x2 as follows:

x1 = - ay - bf(6a)2X(ay-0')

ba -a#X2= - (6b)2X(ay - $2)

X can then be obtained by substitution into Eq. (2), and the final result isay - b$X1 = f

(ory - $')(ya2 + ab2 - 2ab$B)(7a)

zs =ba - a{3 (7b)

(a'y - $') (ya2 + ab2 - 2ab$)


The ambiguity (±) sign in Eqs. (7) results from taking a square root, thenegative sign corresponding to the minimum intercept (point A in Fig.1.3) and the positive sign to the maximum (point B).

1.10 DISCRETE PROPORTIONAL CONTROL WITH LAGRANGE MULTIPLIERS

We shall now return to the problem of Sec. 1.7, where we wish tominimize

S2

(xn2 + C2un2)n-1

and we write the system equation

(1)

x - axn-1 - un - 9n(xn,xn-1,un) = 0 (2)

Since we wish to find the minimizing 2N variables x1, x2, . . . , xN, U1,U2, . . . , UN subject to the N constraints (2), we form the lagrangian

N N

(xn2 + C2un2) + I Xn(xn - axn-1 - un) (3)n-1n-1Taking partial derivatives of the lagrangian with respect to x1,

x2, xN, u1, U2i . . . , UN, we find

cle2-A N=0 n=1xC2 (4a),...,,

ccun una-

= xN + aN = 0 (4b)axNaL =xn+Xn-aXn+1=0 n=1,2, ...,N-1 (4c)axn

Equation (4b) may be included with (4c) by defining

XN+1 = 0 (5)

We thus have simultaneous difference equations to solve

xn - axn-1 - C2 = 0 xo given (6a)

xn+Xn-axn+1=0 XN+1=0 (6b)

with the optimal controls un then obtained from Eq. (4a).Equations (6a) and (6b) represent a structure which we shall see

repeated constantly in optimization problems. Equation (6a) is a differ-ence equation for xn, and (6b) is a difference equation for X,,; but they are,coupled. Furthermore, the boundary condition xo for Eq. (6a) is givenfor n = 0, while for Eq. (6b) the condition is given at n = N + 1. Thus,

N

24

our problem requires us to solve coupled difference equations with splitboundary conditions.

For this simple problem we can obtain a solution quite easily. If,for example, we assume a value for Xo, Eqs. (6) can be solved simultane-ously for X1, xi, then X2, x2, etc. The calculated value for XN+l will proba-bly differ from zero, and we shall then have to vary the assumed value Xountil agreement is reached. Alternatively, we might assume a value forxN and work backward, seeking a value of xN which yields the required x0.Note that in this latter case we satisfy all the' conditions for optimalityfor whatever xo results, and as we vary xN, we produce a whole family ofoptimal solutions for a variety of initial values.

We have already noted, however, that the particular problem weare considering has a closed-form solution, and our past experience, aswell as the structure of Eq. (6), suggests that we seek a solution of theform

OPTIMIZATION BY VARIATIONAL METHODS

x. = C2Mnxn (7a)4or

U. = M.X. (7b)

We then obtain

xn+l - axn - Mn+lxn+l = 0x. + C2Mnxn - aC2Mn+lxn+1 = 0

and eliminating xn and x.+l, we again get the Riccati equation

M.M.+l+1 Ca2CM.+1-M.-C2=0

with the boundary condition

MN+1 = 0

(8a)

(8b)

(9)

(10)

for a finite control period. Note that as N - w, we cannot satisfy Eq.(10), since the solution of the Riccati equation becomes a constant, butin that case XN - 0 and the boundary condition AN+1 = 0 is still satisfied.

1.11 OPTIMAL DESIGN OF MULTISTAGE SYSTEMS

A large number of processes may be modeled, either as a true represen-tation of the physical situation or simply for mathematical convenience,by the sequential black-box structure shown in Fig. 1.4. At each stagethe system undergoes some transformation, brought about in part by thedecision u. to be made at that stage. The transformation is describedby a set of algebraic equations at each stage, with the number of equa-


Fig. 1.4 Schematic of a multistage system.

26

tions equal to the minimum number of variables needed in addition tothe decision variable to describe the state of the process.

For example, let us suppose that two variables, z and y, describethe state. The variable x might represent an amount of some material,in which case one of the equations at each stage would be an expressionof the conservation of mass. Should y be a temperature, say, the otherequation would be an expression of the law of conservation of energy.We shall use-the subscript n to denote the value of x and y in the processstream leaving stage n. Thus, our system is described by the 2N alge-braic equations

4'a(xn,xn-17yn,yn-17un) = 0 n = 1, 2, . . . , N (la)>rn(xn,xn-1,ynlyn-17un) = 0 n = 1, 2, . . . , N (1b)

where presumably xo and yo are given. We shall suppose that the stateof the stream leaving the last stage is of interest, and we wish to chooseu1, u2, ... , uN in order to minimize some function S(xN,yN) of thatstream.

This problem can be formulated with the use of 2N Lagrange multi-pliers, which we shall denote by Xi, X2, . . . , AN, A1, A2, . . . , AN. Thelagrangian is then

N

' - &(xN,yN) +n-1

N

+ I An#n(xn,xn-l,yn,ya-l,un) (2)n-l

and the optimum is found by setting the partial derivatives of £ withrespect to xn, yn, un, Xn, and An to zero, n = 1, 2, . . . , N. At stage Nwe have

+XNxN+AN-=oa a aN

+ XX ONyN

+ ANayN

= 0yN a

XNauN

+ ANaUN

= 0

(3a)

(3b)

(3c)


or, eliminating XN and SAN from (3c) with (3a) and (3b),

as a#N a*N 4 N aON C18 aON 491PN 49ikN 4N = 0 (4)ayN {auN azN auN azN axN auN ayN UN ayN>

For all other n, n = 1, 2, . . . , N - 1, we obtain

n a n+ Xn+1

aon+1 + A. a0ft + 1 An+l a_n+l - 0axn aX n axn axn

w a n+ Xn+1

aW + A. aOn+ An+1 04'n+1 - oay. ayn ay. ayn

ayXn

n-1- An au" = 0

(5a)

(5b)

(5c)

Again we find that we have a set of coupled difference equations,Eqs. (la), (lb) and (5a), (5b), with initial conditions xo, yo, and final con-ditions AN, AN given by Eqs. (3a) and (3b). These must be solved simul-taneously with Eqs. (3c) and (5c). Any solution method which assumesvalues of Ao, Ao or xN, yN and then varies these values until agreementwith the two neglected boundary conditions is obtained now requires asearch in two dimensions. Thus, if v values of xN and of yN must be con-sidered, the four difference equations must be solved v' times. For thisparticular problem the dimensionality of this search may be reduced, asshown in the following paragraph, but in general we shall be faced withthe difficulty of developing efficient methods of solving such boundary-value problems, and a large part of our effort in later chapters will bedevoted to this question.

Rewriting Eq. (5c) for stage n + 1,

au+lXn+1

n

+ °n+1 a,"unn+l = o (5d)

we see that Eqs. (5a) to (5d) make up four linear and homogeneousequations for the four variables X., X.+1, An, An+1, and by a well-knownresult of linear algebra we shall have a nontrivial solution if and only ifthe determinant of coefficients in Eqs. (5a) to (5d) vanishes; that is,


or

a# a4.. aor

LAa#,+1 a-0.+1 a-0.+1 a41-+1

ax* au ~ axn au ayw sun+1 ay,,

a4* a4* a,kn+1c10-+1'94"+1 _ 0 (7)

ay au ay,. au.) k ax. ax. au.+1

[Equation (7) has been called a generalized Euler equation.]Equations (la), (1b), (4), and (7) provide 3N design equations for

the optimal x1, yn, and decision variables. One particular method ofsolution, which requires little effort, is to assume u1 and obtain x1 and y1from Eqs. (1a) and (1b). Equations (1a), (ib), and (7) may then besolved simultaneously for x2, y2, u2 and sequentially in this manner untilobtaining x,v, yNv, and uNv. A check is then made to see whether Eq.(4) is satisfied, and if not, a new value of u1 is chosen and the processrepeated. A plot of the error in Eq. (4) versus choice of u1 will allowrapid convergence.

1.12 OPTIMAL TEMPERATURES FOR CONSECUTIVE REACTIONS

As a specific example of the general development of the preceding sec-tion, and in order to illustrate the type of simplification which can oftenbe expected in practice, we shall consider the following problem.

A chemical reaction

X -+ Y --> products

is to be carried out in a series of well-stirred vessels which operate in thesteady state. The concentrations of materials X and Y will be denotedby lowercase letters, so that x, and y., represent the concentrations inthe flow leaving the nth reaction vessel, and because the tank is wellstirred, the concentration at all points in the vessel is identical to theconcentration in the exit stream.

If the volumetric flow rate is q and the volume of the nth vessel V,,,then the rate at which material X enters the vessel is gxA_l, the rate atwhich it leaves is qx,, and the rate at which it decomposes iswhere r1 is an experimentally or theoretically determined rate of reactionand u is the temperature. The expression of conservation of mass is then

qxn +

In a similar way, for y*,

qy,,_1 = qyn + V%r2(x,,y.)

(1)

(2)

where r2 is the net rate of decomposition of y and includes the rate offormation, which is proportional to the rate of decomposition of x,,. In


particular, we may write

ki(u.)F(x,,) (3a)r2(x,,,y,,,u,,) = (3b)

where k, and k2 will generally have the Arrhenius form

k;o exp `(-!) (4)

Defining the residence time 9 by

9 =4n

(5)

we may write the transformation equations corresponding to Eqs. (1a)and (1b) of Sec. 1.11 as

On = x - B (6a),yn = y + 9 ks(un)G(y,.) (6b)

We shall assume that Y is the desired product, while X is a valu-able raw material, whose relative value measured in units of Y is p. ' Thevalue of the product stream is then pxN + yN, which we wish to maximizeby choice of the temperatures u in each stage. Since our formulationis in terms of minima, we wish to minimize the negative of the values, or

& (XN,YN) = - PXN - yN (7)

If Eqs. (6a) and (6b) are substituted into the generalized Eulerequation of the preceding section and k, and k2 are assumed to be ofArrhenius form [Eq. (4)], then after some slight grouping of terms weobtain

1 + v9 ks(u,.)G'(y,.)1 + 9«ks(u,)G'(y,) E'ki(u»)F(x*) + 1 +

ER iEzks(u.+,)G(y+.+,)

(8)

where the prime on a function denotes differentiation. Equation (8)can be solved for in terms of y,.+,, xp, y,,, and u.:

ex1 ) ^ Jkio [ks(u,.) 1 +

kso ki(u,1) 1 +El ks(uw)G'(yw)F(x*+,) llcsi-a

+ 19E2'11 + O ks(u+.)G'(y,)]G(y,.+,)]

[B(xn,x 1,y (g)


and upon substitution Eqs. (6a) and (6b) become

xn-1 - xn - gnk10[B(xn-I)xn,yn-l,yn,un-1)]S,'1($,'-8,')F(xn) = 0(10a)

yn-1 - yn + v(xn-1 - xn)- Bnk30[B(xn-I,xn,yn-I,yn,un-1)]$"/(&;-B,')G'(yn) = 0 (10b)

The boundary condition, Eq. (4) of Sec. 1.11, becomes

k1(uN)F(xN)(p - ') + E, k2(ulv)G(yiv)[1 + BNk1(uN)F'(ZN)]1

+ p8Nk1(uN)ka(uN)F(xN)G'(yN) = 0 (11)

The computational procedure to obtain an optimal design is thento assume u1 and solve Eqs. (6a) and (6b) for x1 and yl, after whichEqs. (10a) and (10b) may be solved successively for n = 2, 3, . .. , N.Equation (11) is then checked at n = N and the process repeated for anew value of u1. Thus only 2N equations need be solved for the opti-mal temperatures and concentrations.

This is perhaps an opportune point at which to interject a note ofcaution. We have assumed in the derivation that any value of U. whichis calculated is available to us. In practice temperatures will be bounded,and the procedure outlined above may lead to unfeasible design specifi-cations. We shall have to put such considerations off until later chap-ters, but it suffices to note here that some of the simplicity of the aboveapproach is lost.

1.13 ONE-DIMENSIONAL PROCESSES

Simplified models of several industrial processes may be described by asingle equation at each stage

xn - ,fn(xn_1,un) = 0 n= 1, 2, . . . , N (1)

with the total return for the process the sum of individual stage returns

8 = 1 6tn(xn-1,u,,) (2)ft-1

The choice of optimal conditions u1, u2, . . . , uN for such a process iseasily determined using the Lagrange multiplier rule, but we shall obtainthe design equations as a special case of the development of Sec. 1.11.

We define a new variable yn by

yn - yn-1 - 61n(xn_1)un) = 0 yo = 0 (3)

It follows then that

8= YN (4)


and substitution of Eqs. (1) and (3) into the generalized Euler equation[Eq. (7), Sec. 1.111 yields

air*-1au,.1 a6i,.of*_1/sun-1 + ax.-1- 0f/au,1

n= 1,2, . . . ,N (5)

with the boundary condition

acRrv - 0 (6)

atN

The difference equations (1) and (5) are then solved by varying ul untilEq. (6) is satisfied.

Fan and Wangt have collected a number of examples of processeswhich can be modeled by Eqs. (1) and (2), including optimal distributionof solvent in cross-current extraction, temperatures and holding times incontinuous-flow stirred-tank reactors with a single reaction, hot-air allo-cation in a moving-bed grain dryer, heat-exchange and refrigeration sys-tems, and multistage gas compressors.

We leave it as a problem to show that Eq. (5) can also be usedwhen a fixed value xN is required.

1.14 AN INVERSE PROOLEr

It is sometimes found in a multistage system that the optimal decisionpolicy is identical in each stage. Because of the simplicity of such apolicy it would be desirable to establish the class of systems for whichsuch a policy will be optimal.' A problem of this type, in which thepolicy rather than the system is given, is called an inverse problem. Suchproblems are generally difficult, and in this case we shall restrict ourattention to the one-dimensional processes described in the previous sec-tion and, in fact, to-those systems in which the state enters linearly andall units are the same.

We shall consider, then, processes for which Eqs. (1) and (2) ofSec. 1.13 reduce to

x = ff(x,._,,u,.) = A(u.)xR-I + B(un) (1)

w(xn_l,uw) Ci(u,.)xn-1 + D(un) (2)

Since we are assuming that all decisions are identical, we shall simplywrite u in place of u,,. The generalized Euler equation then becomes

C'x,.2 + D' D') = 0 (3)A'xn-2 + B' + C - A'xn-1 + B'

where the prime denotes differentiation with respect to u.After substi-

t See the bibliographical notes at the end of the chapter.

OPTIMIZATION WITH DIFFERENTIAL CALCULUS >Rt

tution of Eq. (1) and some simplification this becomes

(AA'C' + AA'2C - AMA'C')x.-22

+ (A'BC' +'B'C' + A'2BC + AA'B'C+ A'B'C - AA'BC' - A2B'C')x._2

+ (A'D'B + B'D' + A'BB'C + B'2 - ABB'C' - AB'D') = 0 (4)

Our problem, then, is to find the functions A(u), B(u), C(u), andD(u) such that Eq. (4) holds as an identity, and so we must require thatthe coefficient of each power of x._2 vanish, which leads to three coupleddifferential equations

AABC' + A'C - AC') = 0 (5a)A'BC' + B'C' + A'2BC + AA'B'C + A'B'C

- AA'BC' - A2B'C' - 0 (5b)A'D'B + B'D' + A'BB'C + B'2 - ABB'C' - AB'D' = Or (Sc)

If we assume that A (u) is not a constant, Eq. (5a) has the solution

C(u) = a(A - 1) (6)

where a is a constant of integration. Equation (5b) is then satisfiedidentically, while Eq. (5c) may be solved by the substitution

D(u) = aB(u) + M(u) (7)

to give

M'(A'B + B' - AB') = 0 (8)

or, if M is not a constant,

B(u) = B(A - 1) (9)

where {4 is a constant of integration. 11M(u) and A(u) are thus arbi-trary, but not constant, functions of u, and the system must satisfy theequations

x. = A(u.)z.-1 + 11 (10)61(x.-i,u,.) = a[A(u.) - 11x._1 + aB(u.) + M(u.)

= a(x. - xn-1) ± (11)

Thus, the most general linear one-dimensional system whose opti-mal policy is identical at each stage is described by Eq. (10), with theobjective the minimization of

N N

S _ I n = a(XN - x0) + I (12)n-1 n-1

N

This objective function includes the special cases of minimizing I M(u.)R-1

a OPTIMIZATION BY VARIATIONAL METHODS

for fixed conversion xN - xo, in which case a is a Lagrange multiplier, tand maximizing conversion for fixed total resources, in which case

N

M(un) u, - U)nil

(13)

with X a Lagrange multiplier and U the total available resource. Multi-stage isentropic compression of a gas, the choice of reactor volumes forisothermal first-order reaction, and multistage cross-current extractionwith a linear phase-equilibrium relationship are among the processeswhich are described by Eqs. (10) and (11).

1.15 MEANING OF THE LAGRANGE MULTIPLIERS

The Lagrange multiplier was introduced in Sec. 1.8 in a rather artificialmanner, but it has a meaningful interpretation in many situations, twoof which we shall examine in this section. For the first, let us restrictattention to the problem of minimizing a function of two variables,g(xi,x2), subject to a constraint which we shall write

q(xl,xs) - b = 0 (1)

where b is a constant. The lagrangian is then

2 = Xg(xl,x2) - Xb (2)

and the necessary conditions are

a& +a

a9 = 0 (3a)ax, ax,

+ a=

0 (3b)axe xaLet us denote the optimum by fi* and the optimal values of xl and

xs by x*, x2*. If we change the value of the constant b, we shall cer-tainly change the value of the optimum, and so we may write S* as afunction of b

&* = E (b) (4a)

and

xi = xi (b) _xs= xi (b) (4b)

hT us

d8' _ as dxi + as dx2(5)WY - \axl/=, __,. db ax2J 1... 1 db

t Recall that the lagrangian is minimized for a minimum of a convex objective func-tion if the constraint is linear.


and, differentiating Eq. (1) as an identity in b,

499 dxi + ag dxq 1 = 0(al)=,__, A 49x2)=,-=, dbxe x z3 - ze

or

dx*1 - (499/49x2)1;_;; dx2 /db

1 _db - (499/ax,)=,-_,.

Combining Eqs. (3a), (5), and (7), we obtain

dS* - x + Adx2

L \496) +

()]db_

axe =__,. 49x2 ,_=,z,_zt zi-Z,

and from Eq. (3b),

33

(6)

(7)

(9)

That is, the Lagrange multiplier represents the rate of change of theoptimal value of the objective with respect to the value of the constraint.If E has units of return and b of production, then X represents the rate ofchange of optimal return with production. Because of this economicinterpretation the multipliers are often referred to as shadow prices orimputed values, and sometimes as sensitivity coefficients, since they repre-sent the sensitivity of the objective to changes in constraint levels.

A second related. interpretation may be developed in the context ofthe one-dimensional processes considered in Sec. 1.13. We seek theminimum of

N

S = n(xn-1,un) (10)n-1

by choice of ul, u2, . . . , UN, where

xn = f*(xn_1,un) (11)

Now we can find the optimum by differentiation, so thataS = 0 n = 1, 2, . . . , N (12)au,.

or, since x,. depends on all u;, i < n, by the chain rule,aS

496tH+1 afn of-+1 afeaun

_aun + 49x,, aun

+0x,+1 axn aun +

+ MIN . . . afn am. + / a'?-+I + of-+1 +axN-1 au,. aun ( 49x axn+1 ax,.

+ MIN. at-+1 afn = 0 (13)T axN-1 49x49 aun


and similarly,as a6t a6tn+1 af.

aun_1 = ax._, + axn ax._,

a8 a6tN = oOUN 49UN

+ aGIN ... __anOXN_1 ax.-1 au.-1

= 0 (14)

If we define a variable X. satisfying

an + afnaxn_1 ax.-1

N=0we see that Eqs. (13) to (15) may be written

618 a6tn+ Xn afn = 0aua aun au

(15)

(16a)

(16b)

(17)

which, together with Eq. (16), is the result obtainable from the Lagrangemultiplier rule. The multiplier is then seen to be a consequence of thechain rule and, in fact, may be interpreted in Eq. (17) a-

X, =

az

(18)

the partial derivative of the objective with respect to the state at anystage in the process. This interpretation is consistant with the notionof a sensitivity coefficient.

1.16 PENALTY FUNCTIONS

An alternative approach to the use of Lagrange multipliers for con-strained minimization which is approximate but frequently useful is themethod of penalty functions. Let us again consider the problem of mini-mizing 8(x1,x2) subject to

9(x1,x2) = 0 (1)

We recognize that in practice it might be sufficient to obtain a small butnonzero value of g in return for reduced computational effort, and we areled to consider the minimization of a new function

9(x1,x2) = 8(x1,x:) + 3. Kl9(x1,x2))Z (2)

Clearly, if K is a very large positive number, then a minimum is obtain-able only if the product Kg2 is small, and hence minimizing 9 will beequivalent to minimizing a function not very different from F. while ensur-ing that the constraint is nearly satisfied.

OPTIMIZATION WITH DIFFERENTIAL CALCULUS n

Before discussing the method further it will be useful to reconsiderthe geometric example of Sec. 1.9

s=axi+bx2 (3)g = a(xi)2 + 2$xix2 + 7(x2)2 - 02 = 0 (4)a>0 ay-,B2>0 (5)

Equation (2) then becomes

R= axi + bx2 + 4K[a(xl)2 + 20x1x2 + y(x2)2 - a,2jt (6)

with the minimum satisfying

ai;= a + 2K(axi + Nx2)[a(xi)2 + 2j6xlx2 + 7(x2)2 - 421 = 0 (7a)

axiQb + 2K($xa + 7x2)[a(xi)2 + 2$xix2 + 7(x2)2 - A21 = Q (7b)

axe

Equations (7) give

x, a$ - baxi(8)b,8 - ay

and we obtain, upon substitution into Eq. (7a),

a + 2Kxi a + # # bP--a-y) I ( X [a + 2# bft -- ay

\ + 7 (bo - ay)2J A j - 0 (9)

Equation (9) may be solved for xi in terms of K,, with x2 thenobtained from Eq. (8). Since we are interested only in large K, how-ever, our purposes are satisfied by considering the limit of Eq. (9) asK - oo. In order for the second term to remain finite it follows thateither xi =rr0, in which case the constraint equation (4) is not satisfied, or

(xi)2[a+2#b/3-ay+7(bfl 0 (10)

This last equation has the solutionJJJJ

- Mxi = ±A a yqq q (Ila)

(a7 - N2) (ya2 + ab2 - 2ab#)

and, from Eq. (8),

x2 = ±Ap ba - as

(11b)(a7 - 02) (ya2 + ab2 - 2abfi)

which are the results obtained from the Lagrange multiplier rule.In practice we would consider a sequence of problems of the form of


Eq. (2) for K(l), K(2), Krz,

9(n) = 8(x1,x2) + 12Kcn>[9(x1,x2)]2 (12)

where K(n+1) > K(.,) and lim -a -. It might be hoped that asn_0

becomes arbitrarily large, the sequence [1(n) } will approach a finitelimit, with g vanishing, and that this limit will be the constrained mini-mum of E(x1,x2). It can in fact be shown that if the sequence converges,it does indeed converge to the solution of the constrained problem. Ingeneral the sequence will be terminated when some prespecified toler-ance on the constraint is reached.

Finally, we note that the particular form 12Kg2 is only for con-venience of demonstration and that any nonnegative function whichvanishes only when the constraint is satisfied would suffice, the particu-lar situation governing the choice of function. Consider, for example,an inequality constraint of the form

1x11 < X1 (13)

The function [x1/(X1 + t)]2N will be vanishingly small for small e andlarge N when the constraint is satisfied and exceedingly large when it isviolated. It is thus an excellent penalty function for this type of con-straint. Other functions may be constructed to "smooth" hard con-straints as needed.

APPENDIX 1.1 LINEAR DIFFERENCE EQUATIONS

We have assumed that the reader is familiar with the method of solutionof linear difference equations, but this brief introduction should sufficeby demonstrating the analogy to linear differential equations. The nth-order homogeneous linear difference equation with constant coefficientsis written

an-lxk+n-1 + . . . + alxk+l + aoxk = 0 (1)

with x specified at n values of the (discrete) independent variable k.If, by analogy to the usual procedure for differential equations, we seeka solution of the form

xk = e"" (2)

Eq. (1) becomesn

e'nk 1 ae"'P = 0 (3)n-o


or, letting y = em and noting that e,nk 5 0, we obtain the characteristicequation

n

1, a,,yP = 0 (4)

pe0

This algebraic equation will have n roots, y1, Y2, - . , yn, which weshall assume to be distinct. The general solution to Eq. (1) is then

xk = Ciylk + C2y2k + . . - + Cnynk (5)

where C1, C2, . . . , Cn are evaluated from the n specified values of x.Consider, for example, the second-order difference equation

xn+2 + 2axn+l + $xn = 0

The characteristic equation is

(6)

y2+2ay+$=0 (7)

or

y= -a±1/a2_# (8)

The general solution is then

xk = C1(-a + v a2 - $)k + C2(-a - 2 - Q)k (9)

If initial conditions xo and x1 are given, the constants CI and C2 areevaluated from the equations

xo = Cl + C2 (10a)

X1 = (-a + 1"a2 - $)C1 - (a + a2 - Q)C2 (10b)

The modification for repeated roots is the same as for differentialequations. If the right-hand side of Eq. (1) is nonzero, the ge eral solu-tion is the sum of a particular and homogeneous solution, the standardmethods of finding particular solutions, such as undetermined coefficientsand variation of parameters, carrying over from the theory of ordinarydifferential equations. For instance, if our example were of the form

Xn+2 + 2axn+1 + $xn = n (11)

the solution would be of the form

xk = C1(-a + a2 - Y)k + C2(-a - Va%_-0) k + X,, (p) (12)

The particular solution xk(r) can be found from the method of undeter-mined coefficients by the choice

xk(v) = A + Bk (13)


Substituting into Eq. (11) gives

A + B(n + 2) + 2a[A + B(n + 1)] +,6(A + Bn) = n (14)

or, equating coefficients of powers of n on both sides,

n°: (1 + 2a + 19)A + 2(1 + a)B = 0 (15a)n': (1 + 2a + $)B = 1 (15b)

Thus,

1B = 1+2a+,62(1 +a)A (1+2a+Y)2

(16a)

(16b)

and the solution to Eq. (11) is

xk=C,(-a+ a2-j)k2(1 + a) k

(1 +2a+6)2+1 +2a+$ (17)

The constants C, and C2 are again evaluated from the boundary con-ditions. If, for example, x° and x, are given, Ci and C2 are found from

xo = C, + C2 - 2(1 +.a) (18a)(1+2a+Q)2x,=C,(-a+ a2

(12+2+$)2+ 1 +2a+$ (18b)


Sections 1.8 and 1.3: The elementary theory of maxima and minima is treated in allbooks on advanced calculus. The fundamental reference on the subject is

H. Hancock: "Theory of Maxima and Minima," Dover Publications, Inc., NewYork, 1960

Useful discussions in the context of modern optimization problems may be found in

T. N. Edelbaum: in G. Leitmann (ed.), "Optimization Techniques-with Applicationsto Aerospace Systems," Academic Press, Inc., New York, 1962

G. Hadley: "Nonlinear and Dynamic Programming," Addison-Wesley PublishingCompany, Inc., Reading, Mass., 1964

D. J. Wilde and C. S. Beightler: "Foundations of Optimization," Prentice-Hall, Inc.,Englewood Cliffs, N.J., 1967

Sections 1.4 to 1.7: We shall frequently use problems in control as examples of applica-tions of the optimization theory, and complete references are given in later chapters.A useful introduction to the elements of process dynamics and control is


D. R. Coughanowr and L. B. Koppel: "Process Systems Analysis and Control,"McGraw-Hill Book Company, New York, 1965

The demonstration that optimal feedback gains are computable from the solution of aRiccati equation is an elementary special case of results obtained by Kalman.References to this work will be given after the development of the required mathematicalgroundwork.

Section 1.8: The references on the theory of maxima and minima in Secs 1.2 and 1.8 arealso pertinent for Lagrange multipliers and eaastrained minimization. The gen-eralization of the Lagrange multiplier rule to include inequality constraints is basedon a theorem of Kuhn and Tucker, which is discussed in the books by Hadley andby Wilde and Beightler. There is a particularly enlightening development of theKuhn-Tucker theorem in an appendix of

R. E. Bellman and S. E. Dreyfus: "Applied. Dynamic Programming," PrincetonUniversity Press, Princeton, N.J., 1962

See also

H. P. Kunzi and W. Krelle: "Nonlinear Programming," Blaisdell Publishing Company,Waltham, Mass., 1966

An interesting application of Lagrange multiplier-Kuhn-Tucker theory with processapplications, known as geometric programming; -i(di.cussed in the text by Wildeand Beightler (cited above) and in

R. J. Duffin, E. L. Peterson, and C. Zener: "Geometric Programming," John Wiley &Sons, Inc., New York, 1967

C. D. Eben and J. R. Ferron: AIChE J., 14:32 (1968)

An alternative approach, taken by some of these authors, is by means of the theory ofinequalities.

Sections 1.11 to 1.14: The generalized Euler equations were derived in

M. M. Denn and R. Aris: Z. Angew. Math. Phys., 16:290 (1965)

Applications to several elementary one-dimensional design problems are contained in

L. T. Fan and C. S. Wang: "The Discrete Maximum Principle," John Wiley & Sons,Inc., New York, 1964

Section 1.15: The interpretation of Lagrange multipliers as sqp'sitwity coefficients followsthe books by Bellman and Dreyfus and Hadley. The chain-rule development forone-dimensional staged processes is due'to

F. Horn and R. Jackson: Ind. Eng. Chem. Fundamentals, 4:487 (1965)

Section 1.16: The use of penalty functions appears to be due to Courant:

R. Courant: Bull. Am. Math. Soc., 49:1 (1943)

The theoretical basis is contained in supplements by H. Rubin and M. Kruskal (1950)and J. Moser (1957) to

R. Courant: "The Calculus of Variations," New York University Press, New York,1945-1946


See also

A. V. Fiacco and G. P. McCormick: Management Sci., 10:601 (1964)H. J. Kelley: in G. Leitmann (ed.), "Optimization Techniques with Applications

to Aerospace Systems," Academic Press, Inc., New York, 1962

Appendix 1.1: Good introductions to the calculus of finite differences and differenceequations may be found in

T. Fort: "Finite Differences and Difference Equations in the Real Domain," OxfordUniversity Press, Fair Lawn, N.J., 1948

V. G. Jenson and G. V. Jeffreys: "Mathematical Methods in Chemical Engineering,"Academic Press, Inc., New York, 1963

W. R. Marshall, Jr., and R. L. Pigford: "The Application of Differential Equationsto Chemical Engineering Problems," University of Delaware Press, Newark,Del., 1947

H. S. Mickley, T. K. Sherwood, and C. E. Reed: "Applied Mathematics in ChemicalEngineering," 2d ed., McGraw-Hill Book Company, New York, 1957

PROBLEMS

1.1. The chemical reaction X --+ Y - Z, carried out in an isothermal batch reactor,is described by the equations

U-kix - k,yz + y + z - coast

If the initial concentrations of X, Y, and Z are ze, 0, and 0, respectively, and the valuesper unit mole of the species are cx, cyy, and cz, find the operating time 8 which maidmizesthe value of the mixture in the reactor

IP - Cx(x(8) - xe] + CYy(8) + czz(8)

12. For the system in Prob. 1.1 suppose that ki and k, depend upon the temperatureu in Arrhenius form

ki .N kie exp (B,)

For fixed total operating time 8 find the optimal constant temperature. Note thedifference in results for the two cases E, < E, (exothermic) and E, < E, (endothermic).U. The feed to a single.etage extractor contains a mass fraction xe of dissolved solute,and the extracting solvent contains mass fraction ye. The mass fraction of solute inthe effluent is x and in the exit solvent stream is y. Performance is approximatelydescribed by the equations

x + oy - xe + ayey - Kx

where K is a constant (the distribution coefficient) and a in the solvent-to-feed ratio.The cost of solvent purification may be taken approximately as

C, -dy - ye)Ye


and the net return for the process is the value of material extracted, P(xo - x), lessthe cost of solvent purification. Find the degree of solvent purification yo whichmaximizes the net return.1.4. The reversible exothermic reaction X Y in a continuous-flow stirred-tankreactor is described by the equations

0 - cf - c - er(c,cf,T)0 = Tf - T + OJr(c,cf,T) - eQ

where c denotes the concentration of X, T the temperature, and the subscript f refersto the feed stream. r is the reaction rate, a function of c, cf, and 7', a the residencetime, J a constant, and Q the normalized rate of heat removal through a cooling coil.For fixed feed conditions find the design equations defining the heat-removal rate Qwhich maximizes the conversion, cf - c. Do not use Lagrange multipliers. (Hint:First consider c a function of T and find the optimal temperature by implicit differentiation of the first equation. Then find Q from the second equation.) Obtain anexplicit equation for Q for the first-order reaction

r a k,o exp 1 7x`1 (Cl - c) - k2o exp ( T ° c

1.5. A set of experimental measurements y,, y2, ... , yn is made at points x,, x2,... x., respectively. The data are to be approximated by the equation

y = af(x) + R9(x)

where f(x) and g(x) are specified functions and the coefficients a and 0 are to be chosento minimize the sum of squares of deviations between predicted and measured valuesof y

min a - [af(xi) + 59(xi): - yi]2

Obtain explicit equations for a and d in terms of the experimental data. Generaliseto relations of the form

N

y = akfk(x)

Find the best values of a and d in the equation

y -a+fxfor the following data:

x 0 1 2 3 4 5 6

y 0 4.5 11.0 15.5 17.0 26.5 30.5

1.6. A sequence of functions 0,(x), ¢2(x), . . . , is called orthogonal with weight-ing p(x) over an interval (a,b] if it satisfies the relation

b

f. dx = 0 i 0 j


A given function y(x) is to be approximated by a sum of orthogonal functions

N

11(x) = In-l

Find the coefficients c,, c2, ... , cN which are best in the sense of minimizing theweighted integral of the square of the deviations

N

mine = f ab P(x)[y(x) - Z C.O.(x)dxn-l

Show that the sequence sin x, sin 2x, sin 3x, ... is orthogonal over the interval0 < z < r with weighting unity. Find the coefficients of the first four terms forapproximating the functions

(a) y=1 0<x<r(b) y =x 0 <x <T

(c) y =x 0 < x <

r

- 2T-x 2<x<

Compare the approximate and exact functions graphically.1.7 The cost in dollars per year of a horizontal vapor condenser may be approxi-m4ated by

C - 0,N-36D-1L-ss + #:N_6.2D0.'L-' + S,NDL + j94N-1."D-4 L

where N is the number of tubes, D the average tube diameter in inches, and L thetube length in feet. 01, Bt, 03, and 04 are coefficients that vary with fluids and con-struttion costs. The first two terms represent coat of thermal energy; the third, fixedcharges on the heat exchanger; and the.fourth, pumping costs: Show that for allvalues of the coefficients the optimal cost distribution is 43.3 percent thermal energy,53.3 percent fixed charges, and 3.33 percent pumping cost.

Show that the optimal value of the cost can be written

C = C \f=) 0)r, \f,\f,

where f,, f:, f,, f4 are respectively the fractions of the total cost associated with thefirst, second, third, and fourth terms in the cost. [Hint: If A - aC, B - OC, anda + A - 1, then C - (A/a)-(B/p)s.) Thus obtain explicit results for N, D, and L interms of the 0;. Solve for 01 - 1.724 X 106, 02 - 9.779 X 104, P = 1.57, Y43.82 X 10-1, corresponding to a desalinatign plant using low-pressure steam. (Theseresults are equivalent to the formalism of geometric programming, but in this casethey require only the application of the vanishing of partial derivatives at a minimum.The problem is due to Avriel and Wilde.)1.8. For the minimization of a function of one variable, &(x), extend the analysis ofSec. 1a3 to obtain necessary and sufficient conditions for a minimum when both thefirst and second derivatives vanish. Prove that a point is a minimum if and only ifthe lowest-order nonvanishing derivative is positive and of even order.


13.. Prove the converse of Eqs. (13) and (14) of Sec. 1.3, namely, that a quadraticform

ax' + 20xy + 'yy'

is positive definite if a > 0, a8 > y'.1.10. For the system described in Prob. 1.4 suppose that the cost of cooling is equal topQ. Find the design equation for the rate of heat removal which maximizes conversionless cost of cooling. Lagrange multipliers may be used.UL Prove that when a is convex (the hessian is positive definite) the minimum of Esubject to the linear constraints

ai1xi - bi i - 1,2, .. . ,m <n

occurs at the minimum of the lagrangian with respect to x,, x=, . . . , x,,.

1.12. Obtain the results of Sec. 1.13 by direct application of the Lagrange multiplierrule rather than by specialization of the results of Sec. 1.13. Extend the analysis toinclude the following two cases:

(a) xN specified.(b) Effluent is recycled, so that x, and xN are related by an equation xo - g(xN).

1.13. The reversible reaction A B is to be carried out in a sequence of adiabaticbeds with cooling between beds. Conversion in the nth bed follows the relation

_. dEf8w-r(T,E)

where 8,, is the holding time, xw the conversion in the stream leaving the nth bed, andr( T, the reaction rate. In an adiabatic bed the temperature is a linear function ofinlet temperature and of conversion. Thus the conversion can be expressed as

8w - _,,dE F(xw-1xwf T,,)

1=. i R(T,,E)

where T. is the temperature of the stream entering the nth bed. Obtain designequations and a computational procedure for choosing 8w and T. in order to maximizeconversion in N beds while maintaining a fixed total residence time

N9- 8w

n-1

This problem has been considered by Horn and Huchler and Aria.)

2Optimization with DifferentialCalculus: Computation

2.1 INTRODUCTION

The previous chapter was concerned with developing algebraic con-ditions which must be satisfied by the optimal variables in minimizingan objective. The examples considered for detailed study were some-what extraordinary in that the solutions presented could be obtainedwithout recourse to extensive numerical calculation, but clearly this willrarely be the case in practice. In this chapter we shall consider severalmethods for obtaining numerical solutions to optimization problems ofthe type introduced in Chap. 1. An entire book could easily be devotedto this subject, and we shall simply examine representative techniques,both for the purpose of introducing the several possible viewpoints andof laying the necessary foundation for our later study of more complexsituations. Two of the techniques which we wish to include for com-pleteness are not conveniently derived from a variational point of view,so that in order to maintain continuity of the development the detailsare included as appendixes.µ

OPTIMIZATION WITH DIFFERENTIAL CALCULUS: COMPUTATION 45

2.2 SOLUTION OF ALGEBRAIC EQUATIONS

The condition that the first partial derivatives of a function vanish atthe minimum leads to algebraic equations for the optimal variables, andthese equations will generally be highly nonlinear, requiring some iter-ative method of solution. One useful method is the Newton-Raphsonmethod, which we shall derive for the solution of an equation in oneunknown.

We seek the solution of the equation

f (x) = 0 (1)

If our kth approximation to the solution is x(h) and we suppose that the(k + 1)st trial will be the exact solution, we can write the Taylor seriesexpansion of f (x(k+I)) about f(x(k)) as

f(x(kFI)) = 0 = f(x(k)) -+' f(x(k))(x(k+i) - xu)) + (2)

Neglecting the higher-order terms and solving for x(k+i), we then obtainthe recursive equation

x(k+I) = X(k) ._. f (x(j (3)

As an example of the use of Eq. (3) let us find the square root of 2by solving

f(x) = x2 - 2 = 0 '(4)


x(k+1) = x(k) x(02 - 2 x(k)2+22x(k) 2z(k) . (5)

If we take our initial approximation as x('),= 1, we obtain x(2) = 1.5,x(') = 1.4167, etc., which converges rapidly to the value 1.4142. On theother hand, a negative initial, approximation.veill converge to - 1.4142,while an initial value of zero will diverge immediately.

When it converges, the Newton-Raphson method does so rapidly.In fact, convergence is quadratic, which means, that the error Ix(k+l)xj is roughly proportional to the square of the previous error, rxk) -xi. Convergence will generally not occur, however, without a goodfirst approximation. The difficulties which tobe anticipated can bevisualized from Fig. -2.1. The Newton-Raphson procedure is one of esti-mating the function by its tangent at the point x(k). Thus, as shown,the next estimate, x(k+I), is closer to the root, x(k+r) closer still, etc. Note,however, that convergence can be obtained only when the slope at xtk)has the same algebraic sign as the slope at the root. The starting point


'(4

0

'(K)

_ i a II

,r(k),(R+2),,(k+3) (k+) F),. 21 Successive- iterations of the,, Newton-Raphson method.

x(m, where f (x) is different in sign from f (f), will result in divergencefrom the solution.

For an optimization problem the function f (x) in Eq. (1) is thederivative 6'(x) of the function 8(x) -which is being minimized. Equa--tion (3) then has the form

(k)z(k+l) = (k) - e (6)X (z(k)

At the minimum, 6" > 0, so that convergence is possible (but not guar-anteed!) only if 8" is positive for each approximation. For the minimi-sation. of a function of several variables, 3(x1,x2, ... x.), the iterationformula analogous to Eq. (6) can be demonstrated by an equivalentdevelopment to be

-Ft(k+1) = z1(k) - V tnei (7)

where wq is the inverse of the hessian matrix of F, defined as the solutionof the n linear algebraic equations

w826(x1(k),x!(k), . . . ,xn(k)) _ 1 i =wi; - a'' - {

p(8)

J- ax, x, o i p

Z3 AN APPLICATION OF THE NEWTON-RAPHSON METHOD

As a somewhat practical example of the application of the. Newton-Raphson method to an optimization problem we shall consider the con-secutive-reaction sequence described in Sec; 1.12 and seek the optimaltemperature in a single reactor. The reaction sequence is

X - Y -- products

and taking the functions F and G as linear, the outlet concentrations of


species X and Y are defined by the equations

xo - x - 6kloe-B1'I"x = 0 (la)yo - y + 6kloe-Bi I"x - 6k2oe-BiI"y = 0 (2a)

or, equivalently,

x= x0

1 + 6k10e-B,'I"_ 0k10e-B''"Lx0

Y 1 + OkzOe-E= lu + (1 + elope E; l")(1 + Ok20e E''1")

The object is to choose u to maximize y + px or to minimize

(2b)

-y -px= - Yo

1 + 6k20e`R' ""Okloe-E''I"xo pxo

(3)(1 + Bkloe-81`)°)(1 + Bk2oe81'J") - 1 + 0kioe-E,'1u

For manipulations it is convenient to define a new variable

v = e-81'1" (4a)

u =In v (4b)

so that the function to be minimized may be written

S(v) _ _ Yo - Okloyzo _ pxo (5)1 + 6kzovB (1 + Ok10v)(1 + 60200°) 1 + 6k10v

where fl is the ratio of activation energies

?E1

The iteration procedure, Eq. (6) of the previous section, is then

v(k+1) = v(k) _. V (y(k))&"(v(k) )

(6)

(7)

We shall not write down here the lengthy explicit relations for 6' and V.For purposes of computation the following numerical values were

used:

x0= 1 yo=0k10 =5.4X 1010 k20=4.6X 1017Ei = 9,000 Eq = 15,000p=0.3 6= 10

The first estimate of u was taken as 300, with the first estimate of v thencalculatectfrom Eq. (4a). Table 2.1 contains the results of the iteration

4a OPTIMIZATION BY VARIATIONAL METHODS

Table 2.1 Successive approximations by the Newton-Raphsonmethod to the optimal temperature for consecutive reactions

Iteration V X 1012 u -6 -6' 6" X 10-44

Initial 9.3576224 X 10-4 300.00000 0.33362777 3.4131804 X 1011 37.249074

1 1.0098890 325.83690 0.53126091 1.2333063 X 1011 14.815753

2 1.8423181 333.08664 0.59275666 3.4447925 X 10" 7.4848782

3 2.3025517 335.85845 0.60156599 5.5884200 X 10' 5.1916627

4 2:4101939 336.43207 0.60187525 2.3492310 X 10' 4.7608371

5 2.4151284 336.45780 0.60187583 4.6704000 X 104 4.7418880

6 2.4151382 336.45785 0.60187583 8.6400000 X 104 4.7418504

7 2.4151382 336.45785 0.60187583 8.6400000 X 104 4.7418504

based on Eq. (7), where an unrealistically large number of significantfigures has been retained to demonstrate the convergence. Startingrather far from the optimum, convergence is effectively obtained by thefourth correction and convergence to eight significant figures by the sixth.

It is found for this example that convergence cannot be obtainedfor initial estimates of u smaller distances to the right of the optimum.The reason may be seen in Figs. 2.2 and 2.3, plots of g and S' versus u,respectively. There is an inflection point in & at u 347, indicating achange in sign of C", which shows up as a maximum in S'. Thus, caremust be taken even in this elementary case to be sure that the initialestimate is one which will lead to convergence.

0

-0.1

-0.4

- 0.5

-0.6

300 320 340 360 380 400u

Fig. 2.2 Objective function versustemperature for consecutive reactions.

OPTIMIZATION WITH DIFFERENTIAL CALCULUS: COMPUTATION

Fig. 2.3 Derivative of objective versustemperature for consecutive reactions.

4!

I I 1 1 1 1 1 1

300 320 340 _360 380 400U

2.4 FIBONACCI SEARCH

The use of the Newton-Raphson method assumes that the function andits derivatives are continuous and, most importantly, that derivativesare easily obtained. Convergence is not assured, and it may be incon-venient to evaluate derivatives, as would be the case if the function .8were not available in analytical form but only as the outcome of a physi-cal or numerical experiment. Under certain assumptions an appealingalternative is available for functions of a single variable, although wemust change our point of view somewhat.

We shall restrict our attention to functions 8(x) which are unimodal;i.e., they must possess a single minimum and no maximum in the regionof interest, but they need not be continuous. An example is shown inFig. 2.4, where the region of interest is shown as extending from x = 0to x = L. An important feature of such functions is the fact that giventwo observations 8(x1) and 8(x2) at points x, and x2, respectively, we maysay unequivocally that if 8(x1) > 8(x2), the minimum lies somewhere inthe interval x1 < x < L, while if 8(x2) > 8(x1), the minimum lies in theinterval 0 < z < X2. Note that this is true even if x1 and x2 both lie onthe same side of the minimum. The Fibonacci search procedure exploits

so OPTIMIZATION BY VARIATIONAL METHODS

this property of unimodal functions to eliminate in a systematic mannerregions of the independent variable in which the minimum cannot occur.After N such eliminations there remains an interval of uncertainty, whichmust contain the minimum, and the procedure we shall describe here isthe one requiring the minimum number of measurements (evaluations)of the function in order to reach a given uncertainty interval.

The algorithm requires that the function be measured at twosymmetric points in the interval of interest, 0.382L and 0.618L. If8(0.382L) > 8(0.618L), the region to the left of 0.382L is excluded,while if 8(0.618L) > &(0.382L), the region to the right of O.&18L isexcluded. The process is then repeated for the new interval. Part ofthe efficiency results from the fact that one of the two previous measure-ments always lies inside the new interval at either the 38.2 or 61.8 pkr-cent location, so that only one new measurement need be made-at thepoint an equal distance on the other side of the midpoint from the pointalready in the interval. The proof that this is the best such algorithm inthe sense defined above is straightforward, but of a very different naturefrom the variational analysis which we wish to emphasize, and so webypass it here and refer the interested reader to Appendix 2.1.

To demonstrate the Fibonacci search algorithm we shall again con-sider the reactor example of the previous section, the minimization of

_ yo _ Okioe-$ "uxoS(u)

1 + 6k2oe-E, i° (1 + ekloe-$''!")(1 + ek2oe`R' /u)

-azo (1)1 + ekloe-Srru

for the values of parameters used previously. The initial interval ofinterest is 300 _< u < 400, and we have already seen that Newton-Raphson will converge from starting values in no more than half theregion.

In this case L = 100, and the points at 0.382L and 0.618L areu = 338.2 and u = 361.8, respectively. Here,

8(338.2) = -0.599 8(361.8) = -0.193

6(x)

6(x2)

X Fig. 2.4 A unimodal function.


300 320 340u

360 380

51

400

Fig. 2.5 Successive reductions of the interval of uncertaintyfor the optimal temperature by Fibonacci search.

so that the region 361.8 < u < 400 is excluded. The remaining point isu = 338.2, which is at 61.8 percent of the new interval 300 < u < 361.8,and the point which is symmetric at 38.2 percent is u = 323.6076. Theprocess is then repeated, leading this time to the elimination of the regionon the far left. The first several eliminations are shown graphically inFig. 2.5, and the full sequence of calculations is shown in Table 2.2,where many extra significant figures have again been retained. The

Table 2.2 . Successive Iterations of the Fibonacci search method to the finalInterval of uncertainty of the optimal temperature for consecutive reactions

No. ofComputa-tlona Int .1Val 140.888E g(u0.088L) u0.818L &(u0.818L)

1,2 300.000-400.000 338.20000 0.59912986 361.80000 0.193054883 300.000-361.800 323.60760 0.50729013 338.20000 0.599129864 323.608-361.800 338.20000 0.59912986 347.21050 0.492502405 323.608-347.211 332.62391 0.59023933 338.20000 0.599129866 332.624-347.211 338.20000 0.59912986 341:63842 0.576475037 332.624-341.638 336.06745 0.60174375 338.20000' 0.599129868 332.624-338.200 334.75398 0.59949853 336.06745 0.601743759 334.754-338.200 336.06745 0.60174375 336.88362 0.6017159210 334.754336.884 335.56760 0.60119983 336.06745 0.6017437511 335.568-336.884 336.06745 0.60174375 336.38086 0.6018706512 336.067-336.884 336.38086 0.60187065 336.57184 0.6018644413 336.067-336.572 336.26013 0.60184180 336.38086 0.6018706514 336.260-336.572 336.38086 0.60187065 336.45277 0.6018758115 336.381-336.572 336.45277 0.60187581 336.49889 0.6018743516 336.381-336.499 336.42595 0.60187494 336.45277 0.60187581

S2 OPTIMIZATION BY VARIATIONAL METHODS

rapid convergence can be observed in that 7 evaluations of C are requiredto reduce the interval of uncertainty to less than 10 percent of the origi-nal, 12 to less than 1 percent, and 16 to nearly 0.1 percent.

2.5 STEEP DESCENT

The Fibonacci search technique does not generalize easily to more thanone dimension, and we must look further to find useful alternatives tothe solution of the set of nonlinear equations arising from the necessaryconditions. - The method of steep descent, originated by Cauchy, is onesuch alternative, and it, too, is a technique for obtaining the solutionwithout recourse to necessary conditions.

Our starting point is again the variational ;equation

M = C(x1 + 6x1, X2 + Ox:) - 8(x1,x2) =8x1

Oxl +8 Ox: + °(E) (1)

where the partial derivatives are evaluated at 91, x2, but we now supposethat xi and 22 are not the values which cause 8(xi,x2) to take on its mini-mum. Our problem, then, is to find values Ox1, ax: which will bring gcloser to its minimum or, in other words, values Ox1, ax: which will. ensurethat

at < 0 (2)

A choice which clearly meets this requirement is

Ox1

Ox2 -W2I aS87X2)11.12

(3b)

where w1 and w2 are sufficiently small to allow o(e) to be neglected inEq. (1). We then have

08 _ -w1 (az) - W2 u-) < 0 (4)

which satisfies Eq. (2), so that if w1 and w2 are small enough, the newvalue x1 + Sxl, x2 + u2 will be a better approximation-of the minimumthan the old.

An obvious generalization is to choose

e& as (5a)Ox1 = - w11 axl - wl2ax2

ax2 = - w21 az - w22 a = (5b)


where the matrix array [w1 W12] is positive definite and may dependW21 W22

on the position xl, z2, guaranteeing that

M aM

i - 1

2 2

i-1(6)

for small enough w;,. All we have done, of course, is to define a set ofdirections

s

ag6x1 = - L WI4 ax;i-1

(7)

which will ensure a decrease in the function 6 for sufficiently small dis-tances. We have at this point neither a means for choosing the properweighting matrix of components w;, nor one for determining how far totravel in the chosen direction. We simply know that if we guess valuesx1i x2, then by moving a small distance in the direction indicated by Eq.(7) for any positive definite matrix, we shall get an improved value.

2.6 A GEOMETRIC INTERPRETATION

It is helpful to approach the method of steep descent from a somewhatdifferent point of view. Let us suppose that we are committed to mov-ing a fixed distance 4 in the xlx, plane and we wish to make that move insuch a way that & will be made as small as possible. That is, neglectingsecond-order terms, minimize the linear form

bS = [ a ax2 (1)>h.3, ax2 2..i.

by choice of x1i x2, subject to the constraint

(axl)e + (ax2)2 - 42 = 0 (2)

This is precisely the problem we solved in Sec. 1.9 using Lagrange.multi-pliers and in Sec. 1.16 using penalty functions, and we may identify termsand write the solution as

ax: 4 as/ax;2

i = 1, 2 (3)((a&/ax1) + (as/ax2) ]

That is, in Eqs. (3) of See. 2.5 w, is defined as

i= 1,2 (4)


where A is a step size and w; is the same for each variable but changes invalue at each position.

The ratios

as/ax;[(as/ax,), + (as/ax,),)i

will be recognized as the set of direction cosines for the gradient at thepoint and Eq. (3) is simply a statement that the most rapidchange in 6 will be obtained by moving in the direction of the negativegradient vector. A potential computational scheme would then be asfollows:

1. Choose a pair of points (21,22) and compute g and a6/ax,, as/axe.2. Find the approximate minimum of 3 in the negative gradient direc-

tion; i.e., solve the single-variable problem

mXnE,ax, ! - aa,Call the minimizing point the new (f,,,) An approximate valueof a might be obtained by evaluating 8 at two or three points andinterpolating or fitting a cubic in X. A Fibonacci search could beused, though for an approximate calculation of this nature the num-ber of function evaluations would normally be excessive.

3. Repeat until no further improvement is possible. In place of step 2it might sometimes be preferable to use some fixed value w; = wand then recompute the gradient.. If the new value of S is not lessthan s(x,,fi,), the linearity assumption has been violated and w istoo large and must be decreased.

It must be noted that this gradient method will find only a singleminimum, usually the one nearest the surfing point, despite the possibleexistence of more than one. Thus, the process must be repeated severaltimes from different starting locations in order to attempt to find all theplaces where a local minimum exists.

A far more serious reservation exists about the method derivedabove, which might have been anticipated from the results of the previ-ous section. We have, as is customary, defined distance by the usualeuclidean measure

A= = 0x,), + (ox,)' (5)

Since we shall frequently be dealing with variables such as temperatures,'concentrations, valve settings, flow rates, etc., we must introduce nor-malizing factors, which are, to, a certain extent, -arbitrary. The proper


distance constraint will then be

A2 = a(axl)2 + y(ax2)2

55

(6)

and the direction of steep descent, and hence the rate of convergence, willdepend on the scale factors a and y. Moreover, it is quite presumptu-ous to assume that the natural geometry of, say, a concentration-valve-setting space is even euclidean. A more general definition of distance is

A2 = a(axi)' + 2$ axl 6x2 + 'y(ax2)2 (7)

Here, the matrix array 10 $] is known as the covariant metric tensor

and must be positive definite, or a > 0, ay - $2 > 0. It follows, then,from the results of Sec. 1.9 that the coefficients w;j in Eq. (7) of Sec. 2.4aret

Aywll= D

W12 = w21 = - D

where

2aFi ' a&)2 p aFi aFi

D = {(ay - #) [y

(.axl) + a (WX2- 2Saxl ax2] c

(8a)

(8b)

(8c)

(Sd)

There is no general way of determining a suitable geometry for agiven problem a priori, and we have thus returned to essentially the samedifficulty we faced at the end of the previous section. After an examplewe shall explore this question further.

2.7 AN APPLICATION OF STEEP DESCENT

As an example of the use of steep descent we again consider the consecu-tive-reaction sequence with linear kinetics but now with two reactorsand, therefore, two temperatures to be chosen optimally. The generalrelations are

xn-1 - x - Ok1(u,)x,. = 0 n = 1, 2, . . . , N (1)y.-i - y,. + Bkl(u,.)xn - Ok2(un)yn = 0 n = 1, 2, ... , N (2)

t The matrix w,, consists of a constant multiplied by the inverse of the covariantmetric tensor. The inverse of an array a;1 is defined as the array b;, such that

I aubt, 11 -jk l0 iOj

Si OPTIMIZATION BY VARIATIONAL METHODS

where

k,oe a,-ru, i = 1, 2 (3)

and the objective to be minimized by choice of ul and u2 is.

yN - PxN (4)

For N = 2 this can be solved explicitly in terms of ul and u2 as

1 Yo Okl(u,)xo= 1 + 9k2(u2) 1 + 9k2(ul) + [1 + ekl(ul)][1 + 9k2(u1)]

_ 9ki(u2)xo[1 + 9k1(u2)][1 + 9k2(u2)][1 + 9k1(ul)]

Pxo[1 + 9kl(u2)] 1'+ 9k1(ul]

or, for computational simplicity,

1 Yo eklovixo1 + Bk2ovza 1 + 9k2ov1D + (1 + 9klovl) (1 + 9k2ovio)

_ 9k1oy2xo

(1 + 9k1ov2)(1 + 9k2ov?)(1 + 9klovl)Pxo

(1 + Bk1ov2)(1 + 9klov1)

where

v = eZ111U.

(5)

(6)

(7)

and

0E2

(8)1

Though they are easily computed, we shall not write down the cumber-some expressions for a6/avl and a8/av2.

The values of the parameters were the same as those used previ-ously in this chapter, except that 0 was set equal to 5 in order to main-tain comparable total residence times for the one- and two-reactorproblems. The simplest form of steep descent was used, in which thecorrection is based on the relation [Eq. (3) of Sec. 2.5]

vlmw = vlold - W1 a&i(9a)

av1

V2n.w = v2c1d - W2 a : I (9b)

Since the relative effects of vi and v2 should be the same, no scale factor isneeded and wl and w2 were further taken to be the same value w. Based


Table 2.3 Successive approximations to the optimal temperaturesIn two reactors for consecutive reactions by steep descent

57

Iteration V1 X 107, u, ei X 101, YZ -G -dG/Bo, -8E/8v,

Initial 9.358 X 10-0 300.0 9.358 X 10'' 300.0 .3340 1.749 X 1011 1.749 X 1011

1 1.843 333.1 1.842 333.1 0.6360 2.877 X 1010 2.427 X 1010

2 2.131 334.6 2.085 334.6 0.6472 1.771 X 1010 1.354 X 1010

3 2.308 335.9 2.220 335.4 0.6512 1.192 X 1010 8.158 X 10,4 2.427 336.5 2.302 335.9 0.6530 8.434 X 100 5.067 X 1006 2.511 337.0 2.353 336.1 0.6538 6.159 X 100 3.168 X 1006 2.573 337.3 2.384 336.3 0.6542 4.602 X 100 1.959 X 1007 2.619 337.5 2.404 336.4 0.6544 3.500 X 100 1.175 X 1008 2,654 337.6 2.416 336.5 0.6546 2.701 X 100 6.623 X 1009 2.681 337.8 2.422 336.5 0.8546 2.111 X 100 3.286 X 10010 2.702 337.9 2.426 336.5 0.6547 1 867 X 100 1.139 X 10011 2.719 338.0 2.427 336.5 0.6547 1.329 X 100 -2.092 X 101

upon the values of derivatives computed in the example in Sec. 2.3, thisweighting w was initially taken as 10-27, and no adjustment was requiredduring the course of these particular calculations. The initial estimatesof u1 and u2 were both taken as 300, corresponding to vi and V2 of 9.358 X10-14. The full sequence of calculations is shown in Table 2.3.

Examination of the value of the objective on successive iterationsshows that the approach to values of I; near the optimum is quite rapid,while ultimate convergence to the optimizing values of v1 and v2 is rela-tively slower. Such behavior is characteristic of steep descent. Sometrial and error might have been required to find an acceptable startingvalue for w had a good a priori estimate not been available, and someimprovement in convergence might have been obtained by estimatingthe optimal value of w at each iteration, but at the expense of morecalculation per iteration.

This is an appropriate point at which to interject a comment onthe usefulness of the consecutive-chemical-reaction problem as a compu-tational example, for we shall use it frequently in that capacity. Theexamples done thus far indicate that the objective is relatively insensi-tive to temperature over a reasonably wide range about the optimum,a fortunate result from the point of view of practical operation. Thisinsensitivity is also helpful in examining computational algorithms, for itmeans that the optimum lies on a plateau of relatively small values ofderivatives, and computational schemes basing corrections upon calcu-lated derivatives will tend to move slowly and have difficulty finding thetrue optimum. Thus, codlpetitive algorithms may be compared underdifficult circumstances. '

2.8 THE WEIGHTING MATRIX

We can gain some useful information about a form to choose for theweighting matrix to,, in steep descent by considering the behavior of the

sa OPTIMIZATION BY VARIATIONAL METHODS

function 8 near its minimum, where a quadratic approximation may suf-fice. Since there is no conceptual advantage in restricting 8 to dependon only two variables, we shall let 8 be a (unction of n variables xi,z:, . . . , xn and write

ft

'Fip (xl,x4, ,xn) = 6(422, . . . ,. tn) +88

bxiiaxicc

Z

+ 8x 8x.axiax;+ ... (2)i-i,-1

or, for compactness of notation, we may denote the components of thegradient 88/8x by Gi and the hessian 8'8/(8x; 8x;) by H1,, so that

8(xl,x2, ,xn) _ 6(z 1,x2, ,fin) + Gi bxi

+2 I I Hi; ax 6x;+ (2)i-i f--1

If we now minimize 8 by setting partial derivatives with respect toeach bxi bo zero, we obtain

n

Gi + Hi;bx; = 0 (3)i-1

Since the hessian is presumed positive definite at (and hence near) theminimum, its determinant does not vanish and Eq. (3) can be solved byCramer's rule to give

n

bxi = - wiG; (4)

j-1

where the weighting matrix satisfies the equationsn 1

WikHk, = ail = i 0k-1

i=ji j

(5)

That is, the proper weighting matrix is the inverse of the hessian. Thisis equivalent to the Newton-Raphson method described in Sec. 2.2. Notethat it will not converge if the hessian fails to be positive definite at thepoint where the calculation is being made. Thus it will be of use only"near" the solution, but even here its use will require the calculation ofsecond derivatives of the function 8, which may often be inconvenient oreven difficult. It will, however, yield the minimum in a single step fora truly quadratic function and give quadratic convergence (when it con-verges) for all others.

OPTIMIZATION WITH DIFFERENTIAL CALCULUS: COMPUTATION SI

Several methods have been devised which combine some of thesimplicity of the most elementary steep-descent procedure with the rapidultimate convergence of the Newton-Raphson method. This is done bycomputing a new weighting matrix w,, at each iteration from formulasrequiring only a knowledge of the first derivatives of S, starting at thefirst iteration with the simple weighting w,, = wS;;. As the optimum isapproached, the weighting matrix w;; approaches that which would beobtained using the Newton-Raphson method but without the necessityof computing second derivatives. In many practical situations such aprocedure is needed to obtain satisfactory convergence. A discussion ofthe basis of the computation of new w;; would be out of place here, how-ever, and we refer the reader to the pertinent literature for details.

2.9 APPROXIMATION TO STEEP DESCENT

In situations where it is inconvenient or impossible to obtain analyticalexpressions for the derivatives of the function 8 required in steep descentsome form of numerical approximation must be used. The amount ofcomputation or experimentation required to obtain accurate estimatesof the gradient at each iteration is generally excessive in terms of theresulting improvement in the optimum, so that most techniques use crudeestimates. A number of such procedures have been developed and tested,and the bibliographical notes will provide a guide for those interested ina comprehensive study. The one technique we shall discuss here is con-ceptually the simplest yet, surprisingly, one of the most effective.

The procedure can be motivated by reference to Fig. 2.6, wherecontours of constant S are drawn for a two-variable problem. . The tri-angle ABC provides data for crudely estimating the gradient, and if A is

Fig. 2.6 Approximation to steep descentby reflection of triangles.

I[,


the worst of the three points, the line with an arrow from A through thecentroid of the triangle provides a reasonable first approximation. Thus,we can simply reflect the triangle about the line BC to obtain a new tri-angle B1C in a region of lower average value of &. With only a single newcalculation the worst point can again be found and the triangle reflected,leading to the triangle B21. The process is continually repeated, asshown.

There are several obvious difficulties with this overly simple pro-cedure. First, continuous reflection is not adequate, for it will result intoo slow convergence far from, the optimum and too much correction andoscillation near the optimum. Thus, the point reflected through the cen-troid should be moved a fractional distance a > 1 on the other side, andif the new point is also the worst, the distance moved should then bereduced by a fractional factor of 1/r < 1. Hence, the triangle will bedistorted in shape on successive iterations. In some cases the distortionwill cause the triangle to degenerate to 'a line, so that more than threestarting points will usually be needed. For n variables the number ofpoints would then be greater than n + 1. (The coordinate of the cen-troid of N points is simply the sum of the individual coordinates dividedby N.)

380

370

360

350

U2

340

330

320

310

P_

I I _ I I I 1

300 310 320 330 340 350 360 370 380u,

Fig. 2.7 Successive approximations to optimal temperatures intwo reactors by the complex method.


Table 2.4 Successive approximations to the optimal temperaturesin two reactors for consecutive reactions by the complex method

Itera-

tion It, [t2 It t U2t2 -c

M

Initial 35:3.0 :330.0 0.0657 322.0 358.0 0.384-1 878.0 370.0 0.0222

1 383.0 330.0 0.0657 322.0 358.0 0.3844 338.9 330.1 0.64312 302.4 3-51.6 0.5018 322.0 358.0 0.3844 3:38.9 1330.1 0.64313 302.4 351.6 10.5018 319.9 331.7 0.5620 :3:35.9 33(1. 1 1 0.64314 343.5 319.9 110.6055 319.9 331.7 0.5620 338.9:330.1 1 0.64315 343.8 319.9 10.6055 336.4 :326.6 0.6229 335.9 3:30.1 1 0.64316 :334.3 3:3'2.9 0.6402 336.4 326.6 0.6229 33,`5.3) 330.1 0.64317 334.3 332.9 336.7 334.1 0.65(18 33,5.9 330.1 0.6431S 339.7 331.7 0.6450 336.7 334.1 0.11508 338.9 330.1 10.64319 339.7 331.7 0.6480 336.7 334.1 0.6508 337.9 3:34.4 0.6529

10 336.0 i 335.6 0.6518 336.7 334.1 0.6608 :337.9 334.4 0.652911 336.0 335.6 0.6518 337.0 1 335.5 0.6534 :337.9. 334.4 0.652912 338.2 I :3:34.6 0.6534 337.0 335.5 0.6534 337.9 334.4 0.652913 1338. 2 3:34.6 0.6534 337.0 335.5 10.6534 337.5 335.4 10.6538

14 338.2 334.6 10.6534 338.3 334.7 0.6537 3:37.5 { 335.4 1 0.6.538

The geometrical figure made up by joining N + 1 points in anN-dimensional space is called a simplex, and the basic procedure is oftencalled the simplex method, a name which can cause unfortunate confusionwith the unrelated simplex method of linear programming (Appendix 2.2).Box has done a detailed study of the modifications required for systemswith constraints and has called the resulting procedure the complexmethod. His paper is highly recommended for computational details.

As an example of the simplex-complex approach the optimal tern-perature problem for two stages was again solved, the function & repre-sented by Eq. (5) of Sec. 2.7, and the same parameters used as for steepdescent. For simplicity only three points were used, and the startingvalues were chosen using random numbers in the interval 300 < ui,u2 < 400. The reflection parameter a was taken as 1.3 and r as 2. Thefirst nine iterations are shown graphically in Fig. 2.7, where the sequenceof triangles is ABC, AC1, A12, 213, 314, 145, 561, 671, 781, 798; thefirst 14 are listed in Table 2.4, with the worst of each set of three pointsin boldface. The consequence of random starting values is an initialscan over the entire region of interest, followed by rapid approach to oneregion and systematic descent to the neighborhood of the optimum. Itwill be observed that convergence is moderately rapid; the final regioncomputed is a small one containing the single value found previously bysteep descent and the worst value of the objective close to the minimumfound previously.


APPENDIX 2.1 OPTIMALITY OF FIBONACCI SEARCH

In this appendix we wish to prove that the Fibonacci search algorithmdescribed and applied in Sec. 2.4 is optimal for unimodal functions in thesense that for a given final interval of uncertainty it is the sequence ofsteps requiring the fewest function evaluations. We shall actually provean equivalent result, that for a given number of function evaluations theFibonacci algorithm places the minimum within the smallest possibleinterval of uncertainty or, measuring length in units of the final intervalof uncertainty, that it allows the largest possible initial interval [0,L]such that after a given number of observations the minimum can belocated within an interval of unit length. We develop the computationalscheme by first proving the following result.

Let L. be any number with the property that the minimum of theunimodal function s(x) on the interval 0 < x < Lp can be located withinan interval of unit length by calculating at most n values and makingcomparisons. If we define

F. = sup L.

then

(1)

F. = F._2 n > 2 (2)Fo = FI = 1 (3)

The notation sup in Eq. (1) stands for supremum, or least upper bound.We use this instead of "maximum" because while L. will be able toapproach the upper bound F. arbitrarily closely, it will never in fact beable to take on this value. Our development follows that of Bellmanand Dreyfus quite closely.

Clearly if we have made no observations, we can place the mini-mum within a unit interval only if we have started with a unit interval,so that Fo = 1, and since a single observation is of no use whatsoever inplacing the minimum, one observation is no better.than none and F1 = 1.For n = 2 the minimum may lie either in the interval [0,x2] or [x1,L2], andso neither of these may exceed unity. It is obvious, however, that eachof these intervals may be set equal to unity and the value of L2 maxi-mized by placing x1 and x2 equal distances from the center of the intervaland as close together as possible; that is,

x1-I-e x2=1

in which case

(4)

L2=2- (5)


where E is as small as we wish, and hence

F2 = sup L2=2=Fo+F1 (6)

The remainder of the development proceeds by induction. Weassume that

Fk = Fk_1 + Fk_2 k = 2, 3, . . . , n - 1 (7)

and then prove that this implies Eq. (2) for k = n, completing the proof.Referring to Fig. 2.4, setting L = L,,, we see that if 6(x1) < g(X2), thisimplies that the minimum is in the interval [J,x:1. Since we now haveonly n - 2 trials in which to locate the minimum in a unit interval, wemust be left with a smaller interval than the largest possible havingn - 1 trials. Hence

x2 < F,.-1 (8)

Similarly, it is possible that the minimum might in fact be in the inter-val [0,x11, but an additional trial would be necessary to estabJi h this,leaving n - 3 trials. Thus,

x1 < (9)

On the other hand, if 6(X2) < &(x1), the minimum would lie in theinterval [x1,L ], and identical reasoning requires

L. - xl < Fn-1 (10)

Combining the inequalities (9) and (10), we find

L. < F,._1 + Fa_2 (11)

orF. = sup L < Fn-1 + Fr_s (12)

Suppose we pick an interval

L. = (1 - 2) F,._2) (13)

and place x1 and x2 symmetrically,

xs = (1 - 2} (14)

so that the interval remaining after these two trials is as close as possibleto the largest possible interval with n - 2 evaluations left. SucF a place-ment of points is consistent with our induction hypothesis. It follows,then, that

F. = sup L. > (15)


and combining the inequalities (12) and (15), we obtain the desired result

F. = F.-, + F._2 (2)

Let us note, in fact, that the placement of starting points in Eqs.(14) is in fact the optimum, for we have always an optimum interval

Lk=(1-E)Fk 2 < k < n (16)

and an optimum placing of one point, in the position (1 - E/2)Fk_l.The procedure is then as follows.

Choose the two starting points symmetrically, a distancefrom each end of the interval 0 < x < L, and make each successive obser-vation at a point which is symmetric in the remaining interval with theobservation which already exists in that interval. After n observationsthe minimum will then be located in the smallest interval possible.

One drawback of this procedure is that it requires advance knowl-edge of the number n of experiments. This is easily overcome by notingthat the sequence defined by Eqs. (2) and (3), known as the Fibonaccinumbers, may be found explicitly by solving the difference equation (2)by the method outlined in Appendix 1.1. The solution with initial con-ditions defined by Eq. (3) is

+1 + (17)F. 2 i/5 ' 2 ) 2 /5 \ 2

and for large n this is well approximated by

F. (18)` J2 -VV/

25

Thus

2 = 0.618Fn .-1+V5 (19)

and a near-optimum procedure is to place the first two points symmetri-cally in the interval 0 < x < L a distance 0.618L from each end and thenprocede as above. In this way the interval of uncertainty for the loca-tion of the minimum can be reduced by a factor of nearly 10,000 withonly 20 evaluations of the function &(x). It is this near-optimum pro-cedure, sometimes referred to as the golden-section search and indistin-guishable from the true optimum for n greater than about 5, which isdescribed and applied in Sec. 2.4.

OPTIMIZATION WITH DIFFERENTIAL CALCULUS: COMPUTATION iS

APPENDIX 2.2 LINEAR PROGRAMMING

A large number of problems may be cast, exactly or approximately, inthe following form:

n

Min 8 = c1x1 + C2 t2 + + Cnxn = c;x;-r

with linear constraints

-1x;>0 j= 1,2, . . . n

(1)

(2)

(3)

where the coefficients c; and a;, are constants and the notation (<, =, > )signifies that any of the three possibilities may hold in any constraint.This is the standard linear programming problem. We note that if thereshould be an inequality in any constraint equation (2), then by defininga new variable y; > 0 we may put

a,,x; ± y; = b; (4)-1

Thus, provided we are willing to expand the number of variables byintroducing "slack" and "surplus" variables, we can always work withequality constraints, and without loss of generality the standard linearprogramming problem can be written

n

Min 8 = I c;x; (1)i-1

n

a;x;=b; i=1, 2, m<n (5)i-1

Q j= 1, 2, . . . , n (3)

In this formulation we must generally have m < n to prevent the exist-ence of a unique solution or contradictory constraints.

The m linear algebraic equations (5) in the n variables it, x2,xn will generally have an infinite number of solutions if any exist at all.Any solution to Eqs. (5) satisfying the set of nonnegative inequalities[Eq. (3)1 will be called a feasible solution, and if one exists, an infinitenumber generally will. A basic feasible solution is a feasible solution toEqs. (5) in which no more than m of the variables x1, x2, . . . , x;, arenonzero. The number of basic feasible solutions ii finite and is boundedfrom above by n!/[m!(n - m)!], the number of combinations of n varia-bles taken m at a time. It can be established without difficulty, althoughwe shall not do so, that the optimal solution which minimizes the linear

66 OPTIMIZATION BY VARIATIONAL METHODS'

form 6 subject to the constraints is always a basic feasible solution.Thus, from among the infinite number of possible combinations of varia-bles satisfying the constraints only a finite number are candidates for theoptimum. The simplex method of linear programming is a systematicprocedure for examining basic feasible solutions in such a way that S isdecreased on successive iterations until the optimum is found in a finitenumber of steps. The number of iterations is generally of order 2m.Rather than devoting a great deal of space to the method we shall demon-strate its operation by a single example used previously by Glicksman.

Let

S = -5x - 4y - 6z (6)x+y+z<100 (7a)3x + 2y + 4z < 210 (7b)3x + 2y < 150 (7c)x,y,z>0 (7d)

We first convert to equalities by introducing three honnegative slackvariables u, v, w, and for convenience we include S as a variable in thefollowing set of equations:

x+ y+ z+u = 100 (8a)3x + 2y + 4z + v = 210 (8b)

3x + 2y + w = 150 (8c)

5x + 4y + 6z + S = 0 (8d)x, y) z, u, v, w > 0 (8e)

The solid line is meant to separate the "convenience" equation (8d)from the true constraint equations. A basic feasible solution to Eqs.(8a), (8b), and (8c) is clearly u = 100, v = 210, w = 150, with x, y, and zall equal to zero, in which case S = 0.

Now, computing the gradient of S from Eq. (6),

8S5

8S -4 8S - -6 (9)8x-5

8y 5z

so that improvement in S can be obtained by increasing x, y, and/or z.Unlike most steep-descent procedures we choose here to move in only.asingle coordinate direction, and since the magnitude of the gradient inthe z direction is greatest, we shall arbitrarily choose that one. . Fromthe point of view.of a general computer program this is simply equivalentto comparing the coefficients in Eq. (8d) and choosing the most positive.Since we choose to retain x and y as nonbasic (zero), Eqs. (8a) and (8b)become

z + u = 100 (10a)4z + v = 210 (10b)


(Equation (8c) does not involve z or it, too, would be included.) Aswe are interested only in basic feasible solutions, either u or v must goto zero (since z will be nonzero) while the other remains nonnegative.If u goes to zero, z = 100 and v = -190, while if v goes to zero, z = 52.5and u = 48.5. That is,

z = min (109/1,21%) = 52.5 (11)

and v is to be eliminated as z is introduced. Again, the required calcula-tion for a general program is simply one of dividing the right-hand columnby the coefficient of z and comparing.

The Gauss-Jordan procedure is used to manipulate Eqs. (S) sothat the new basic variables, u, z, and w each appear in only a singleequation. This is done by dividing the second equation by the coefficientof z, then multiplying it by the coefficient of z in each of the other equa-tions, and subtracting to obtain an equivalent set of equations. Thesecond equation is chosen because it is the one in which v appears. Theresult of this operation is

34x + 32y + U - 14v = 4712 (12a)34x+32y+z + %v = 5212 (125)3x + 2y + w = 150 (12c)

32x + y - 32v + F. = -315 (12d)

The basic feasible solution (x, y, v = 0) is u = 4732, z = 52;2, andw = 150. From Eq. (12d)

g.= -315-32x-y+%v (13)

so that the value of 3 in the basic variables is -315.Repeating the above procedure, we now find that the largest posi-

tive coefficient in Eq. (12d) is that of y, and so y is to be the new basic(nonzero) variable. The variable to be eliminated is found from.exa_m-ining the coefficients of the basic variables in Eq. (12) in the form

Y =min (477 52,- 150 _ 75\72

72,

2(14)

which corresponds to eliminating w. Thus, we now use the Gauss-Jordanprocedure again to obtain y in the third equation only, the result being

-3'zx +u-34v- -%W = 10 (15a)Z + 34v - 34w 15 (15b)

2x + y + 32w = 75 (15c)-x -%v-Y2w+g= -390 (15d)

The basic feasible solution is then x, v, w = 0, u = 10, z = 15, y = 75,

a OPTIMIZATION BY VARIATIONAL METHODS

and the corresponding value of & - -390. There are no positive coeffi-cients in Eq. (15d), and so this is the minimum.. In terms of the originalvariables only, then, x = 0, y = 75, z = 15, and only two of the threeoriginal inequality constraints are at equality.

It should be clear from this example how a general computer codeusing only simple algebraic operations and data comparison could beconstructed. The details of obtaining the required starting basic feasiblesolution for the iterative process under general conditions, as well as otherfacets of this extensive field, are left to the specialized texts on the subject.The interested reader should establish for himself that the one-at-a-timesubstitution used in the simplex method is the required result from steepdescent when, instead of the quadratic-form definition of distance,

42 (16)a

a sum-of-absolute-value form is used,

(17)

Linear programming can be used to define directions of steep descentin-constrained nonlinear minimization problems by linearizing constraintsand objective at each iteration and bounding the changes in the variables.The solution to the local linear programming problem will then providethe values at which the linearization for the next iteration occurs. Sincegradients must be calculated for the linearization, this is essentiallyequivalent to finding the weighting. matrix in a constrained optimization.The MAP procedure referred to in the bib'iographical notes is such amethod.


Section t..5: Discussions of the convergence properties of the Newton-Raphson andrelated techniques may be found in such books on numeripal analysis as _

C. E. Froberg: "Introduction to Numerical Analysis," Addison-Wesley PublishingCompany, Inc., Reading, Mass., 1965

F. B. Hildebrand: "Introduction to Numerical Analysis," McGraw-Hill BookCompany, New York, 1956

Section 2.4 and Appendix t.1: The derivation of the Fibonacci search used here isbased on one in

R. E. Bellman and S. E. Dreyfus: "Applied Dynamic Programming," PrincetonUniversity Press, Princeton, N.J., 1962

OPTIMIZATION WITH DIFFERENTIAL CALCULUS: COMPUTATION a

which contains references to earlier work of Kiefer and Johnson. An alternative approachmay be found in

D. J. Wilde: "Optimum Seeking Methods," Prentice-Hall, Inc., Englewood Cliffs,N.J., 1964

and C. S. Beightler: "Foundations of Optimization," Prentice-Hall, Inc.,Englewood Cliffs, N.J., 1967

Sections 2.5 and 2.6: The development of steep descent was by Cauchy, althougA hispriority was questioned by Sarru8:

A. Cauchy: Compt. Rend., 25:536 (1847)F. Sarrus: Compt. Rend., 25:726 (1848)

Some discussion of the effect of geometry may be found in the books by Wilds cited above;see also

C. B. Tompkins: in E. F. Beckenbach (ed.), "Modern Mathematics for the Engineer,vol. I," McGraw-Hill Book Company, New York, 1956

T. L. Saaty and J. Bram: "Nonlinear Mathematics," McGraw-Hill Book Company,New York, 1964

Section 9.8: The most powerful of the procedures for computing the weighting matrixis probably a modification of a method of Davridon in

R. Fletcher and M. J. D. Powell: Computer J., 6:163 (1963)G. W. Stewart, III: J. Adsoc. Comp. Mach., 14:72 (1967)W. I. ZangwilI: Computer J., 10:293 (1967)

This has been extended to problems with constrained variables by

D. Goldfarb and L. Lapidus: Ind. Eng. Chem. Fundamentals, 7:142 (1968)

Other procedures leading to quadratic convergence near the optimum are discussed inthe books by Wilde and in

H. H. Rosenbrock and C. Storey: "Computational Techniques for Chemical Engi-neers," Pergamon Press, New York, 1966

Many of the standard computer codes use procedures in which the best direction fordescent at each iteration is obtained as the solution to a linear programming problem.The foundations of such procedures are discussed in


One such technique, known as MAP. (method of approximation programming) is developedin

R. E. Griffith and R. A. Stewart: Management Sci., 7:379 (1961)

MAP has been applied to a reactor design problem in

C. W. DiBella and W. F. Stevens: Ind. Eng. Chem. Process Design Develop., 4:16(1965)


Section 2.9: The references cited for Sec. 2.8 are pertinent for approximate proceduresas well, and the texts, in particular, contain extensive references to the periodicalliterature. The simplex-complex procedure described here is a simplified versionfor unconstrained problems of a powerful technique for constrained optimizationdevised by

M. J. Box: Computer J., 8:42 (1965)

It is an outgrowth of the more elementary simplex procedure, first described in this sec-tion, by

W. Spendley, G. R. Hext, and F. R. Himaworth: Technometrics, 4:441 (1962)

Appendix 2.2: A delightful introduction to linear programming at a most elementarylevel is

A. J. Glicksman: "Introduction to Linear Programming and the Theory of Games,"John Wiley & Sons, Inc., New York, 1963.

Among the standard texts are

G. B. Dantzig: "Linear Programming and Extensions," Princeton University Press,Princeton, N.J., 1963

S. I. Gass: "Linear Programming: Methods and Applications," 2d ed., McGraw-HillBook Company, New York, 1964

G. Hadley: "Linear Programming," Addison-Wesley Publishing Company, Inc.,Reading, Mass., 1962

Linear programming has been used in the solution of some optimal-control problems; see

G. I)antzig: SIAM J. Contr., A4:56 (1966)G. N. T. Lack and M. Enna: Preprints 1987 Joint Autom. Contr. Conf., 474H. A. Lesser and L. Lapidus: AIChE J., 12:143 (1966)Y. Sakawa: IEEE Trans. Autom. Contr., AC9:420 (1964)H. C. Torng: J. Franklin Inst., 278:28 (1964)L. A. Zadeh and B. H. Whalen: IEEE Trans. Autom. Contr., AC7:45 (1962)

An e..:ensive review of applications of linear programming in numerical analysis is

P. Rabinowitz: SIAM Rev., 10:121 (1968)

PR03LEMS

2.1. Solve Prob. 1.3 by both Newton-Raphson and Fibonacci search for the followingvalues of parameters:

K=3 a=1 x0=0.05 C - 0.01P

2.2. Obtain the optimal heat-removal rate in Prob. 1.4 by the Fibonacci search method,solving the nonlinear equation for T at each value of Q by Newton-Raphson for thefollowing rate and parameters:

r = 2.5 X 101 exp '0"') 2.0 X 10Tex\p (_ 40,000c

J = 10' B = 10-2


2.3. Derive Eqs. (7) and (8) of Sec. 2.2 for the multidimensional Newton-Raphsonmethod.2.4. The following function introduced by Roscnbrock is frequently used to testcomputational methods because of its highly curved contours:

& = 100(x2 - y')2 + (1 - x)2

Compare the methods of this chapter and that of Fletcher and Powell (cited in thebibliographical notes) for efficiency in obtaining the minimum with initial valuesx = -1, y = -1.

Solve each of the following problems, when appropriate, by steep descent, Newton-Raphson, and the complex method.

2.5. Solve Prob. 1.7 numerically and compare to the exact solution.2.6. Using the data in Prob. 1.5, find the coefficients a and t4 which minimize both thesum of squares of deviations and the sum of absolute values of deviations. Comparethe former to the exact solution.2.7. The annual cost of a heavy-water plant per unit yield is given in terms of the flowF, theoretical stages N, and temperature T, as

300F + 4,000NA + 80,00018.3(B - 1)

where

A =2+3exp(16.875- T)\ 14.4

B =

0

4(1 - s)0.6(1 - $)(c 8 - 1) + 0.40

=a (as)N+t+0-1

0F

1,400

a = exp (508 - 0.382)

Find the optimal conditions. For computation the variables may be bounded by

250 <- F < 5001 <N < 20

223 < T < 295

(The problem is due to Rosenbrock and Storey, who give a minimum cost of S =1.97 X 10'.)2.8. Obtain the optimal heat-removal rate in Prob. 1.4 by including the system equa-tion for temperature in the objective by means of a penalty function. The parametersare given in Prob. 2.2.2.9. Using the interpretation of Lagrange multipliers developed in See. 1.15, formulhtea steep-descent algorithm for multistage processes such as those in See. 1.13 in whichit is not necessary to solve explicitly for the objective in terms of the stage decisionvariables. Apply this algorithm to the example of Sec. 2.7.


2.10. Solve the following linear programming problem by the simplex method:

min E = 6x1 + 2x2 + 3x330x3 + 20x2 + 40x3 2!34

x1+x2+x2=110x, + 70x2 < 11

X1, x2, x3 > 0

Hint: Find a basic feasible solution by solving the linear programming problem

min & = z1 + z230x1 + 20x2 + 40x3 - w1 + 21 a 34

X1 + x7 + x$ + Z2 - 110x1 + 70x2 + w3 - 11

xl, x2, x3, w1, W3, zl, ZS > 0

You should be able to deduce from this a general procedure for obtaining basic feasiblesolutions with which to begin the simplex method.2.11. Formulate Prob. 1.4 for solution using a linear programming algorithm iteratively.

3Calculus of Variations

3.1 INTRODUCTION

Until now we have been concerned with finding the optimal values of afinite (or infinite) number of discrete variables, x1, x2, . . . , x,,, althoughwe have seen that we may use discrete variables to"approximate a func-tion of a continuous variable, say time, as in the optimal-ebnitrol problemconsidered in Sees. 1.4 and 1.6. In this chapter we shall begin consider-ation of the problem of finding an optimal function, and much of theremainder of the book will be devoted to this task. The determinationof optimal functions forms a part of the calculus of variations, and weshall be concerned in this first chapter only with the simplest problemsof the subject, those which can be solved using the techniques of thedifferential calculus developed in Chap. 1.

3.2 EULER EQUATION

Let us consider a continuous differentiable function x(t), where t has therange 0 < t < 0, and a function 9 which, for each value of t, depends

73


explicitly on the value of x(t), the derivative ±(t), and t; that is,

5 = a(x,x,t)

For each function x(t) we may then define the integral

s[x(t)] = f oB S(x,x,t) dt

(1)

(2)

The number &[x(t)] depends not on a discrete set of variables but on anentire function, and is commonly referred to as a functional, a functionof a function. We shall seek conditions defining the particular function2(t) which causes s to take on its minimum value.

We shall introduce an arbitrary continuous differentiable functionn(t), with the stipulation that if x(0) is specified as part of the problem,then n(0) must vanish, and similarly if x(6) is specified, then 17(6) mustvanish. If we then define a new function

x(t) = x(t) + en(t) (3)

we see that &[X(t) + s, (t)] depends only on the constant e, since 2(t) andn(t) are (unknown) specified functions, and we may write

t`i(E) = 10 ff(x + En, ± + En, t) dt (4)

Since 2 is the function which minimizes 9, it follows that the functions(e) must take on its minimum at e = 0, where its derivative must vanish,so that

d& = 0 (5)T,.-0We can differentiate under the integral 'sign in Eq. (4), noting that

dT al; dx Off dx

de = T. de + ax ae

ax '1 + ax ''(6)

anddP- fde Jo [()x_5 n +

I dt = 0 (7)

We integrate the second term by parts to obtaint

e 496 . 85 0 e d agfo azndt = axnlo - fo n dtaxdt (8)

t Note that we are assuming that off /a± is continuously differentiable, which will notbe the case if t(t) is not continuous. We have thus excluded the possible cases ofinterest which have cusps in the solution curve.

CALCULUS OF VARIATIONS 75

where we have dropped the notation ( )Z_, for convenience but it isunderstood that we are always evaluating functions at x. We thus have,from Eq. (7),

10 (ax - itd ai} ,(t) dt + c?aff

z't 100 (9)

for arbitrary functions ,7(t).If x(0) is specified, then since ,1(0) must be zero, the term (af/ax),,

must vanish at t = 0, with an identical condition at t = 0. If x(0) orx(0) is not specified, we shall for the moment restrict our attention tofunctions 71(t) which vanish, but it will be necessary to return to con-sider the effect of the term"(aff/ai),q f o. Thus, for arbitrary differentiablefunctions ,t(t) which vanish at t = 0 and B we must have

Jo ax it ax)t)dt = 0 (10)

If we choose the

(TX

particular function

n(t) = w(t)aT - d af

dt

where w(0) = w(0) = 0 and w(t) > 0, 0 < I < 0, we obtain

Io

z

w(t)ax ax ax) dt = 0 (12)

The only way in which an integral of a nonnegative function over apositive region can be zero is for the integrand to vanish idlintically,and so we obtain

of da9=0 0<t<0ax dt ax

which is known as Euler's differential equation.If we now return to Eq. (9), we are left with

(13)

axy (ax n1,_o = 0(14)

If x(9) is not specified, ,,(B) need not be zero and we may choose a func-tion such that

n(0) = E1 I- (15)

where el > 0 is nonzero only when x(0) is not specified. Similarly, ifx(0) is not specified, we may choose n(t) such that

Carlaz ,_o (16)


Thus, for these particular choices, we have

a5f= 0 (17)

(a5)=ax 1 ax e_o -E'

1 +and it follows that a5/ax must vanish at an end at which x(t) is notspecified.

We have thus obtained a condition for the minimizing functionequivalent to the vanishing of the first derivative for the simple calculusproblem: the optimal function 2(t) must satisfy the Euler second-order differ-ential equation

d a5 a25 a=5 a2S a5 ( )at ax = at ax + ax ax x + ax= x = ax 18

with the boundary conditions

At t = 0: x(0) specified or az = 0

At t = 0: x(0) specified or a = 0

(19a)

(19b)

It is sometimes convenient to use an alternative formulation for theEuler equation. We consider

d a5 _ as . as d asdt

5 - i ax) at +aaxs

+asax

x - x ai - x at az (20)

and substituting Eq. (18),

dT (5 - x ax) a (21)

if if does not depend explicitly on t, then a5/at is zero and we may inte-grate Eq. (21) to obtain a first integral of the Euler equation

if - x ax = const (22)

Finally, if we seek not one but several functions xl(t), xs(t), ... ,x (t) and if is a function 5(xl,x=, . . . ,x,,,il,tt, . . . ±.,t), then the Eulerequations (13) and (18) are written

d a5 - as = 0 i = 1 2 n. . . (23)I ax; ax;, , ,

- x,a- =.Const (24)

It should be noted that the Euler equation is obtained from thenecessary condition for a minimum, the setting of a first derivative to


zero. Just as in the simple problems of differential calculus, we cannotdistinguish from this condition alone between maxima, minima, and otherstationary values. We shall refer to all solutions of the Euler equationas extremals, recognizing that not all extremals will correspond to minima.

3.3 BRACHISTOCHRONE

Solutions of Euler's equation for various functions yield the answers tomany problems of interest in geometry and mechanics. For historicalreasons we shall first consider a "control" problem studied (incorrectly}by Galileo and later associated rith the Bernoullis, Newton, Leibniz, andL'Hospital. We assume that a particle slides along a wire. without fric-tional resistance, and we seek the shape, or curve in space, which willenable a particle acted upon only by gravitational forces to travel betweentwo points in the minimum time. This curve is called a brachistochrone.

The system is shown in Fig. 3.1, where m is the.particle mass, 8 isare length, t time, g the acceleration due to gravity, x the vertical coordi-nate, and z the horizontal. The speed is dx/dt, and the accelerationd2x/dt2. A force balance at any point along the curve then requires

d /x dxmd=mg sin2

« mgt (1)

Dividing out the common factoi m and multiplying both sides by ds/dt,we find

1 d ds 2 dx2 dt dt) - g dt (2)

or, integrating,

dsdt = v2g(x - c) (3)

X,

!-arc length

X2

mgFly. 31 Motion of a particle under theinfluence of gravity along a frictionlesswire between two points.


where the constant c is determined from the initial height and velocity as

C = x(0) - 2-(d),

o (4)

We note that

ds2 = dx2 + dz2

or

(5)

ds = 1 +()2

dz (6)

Thus, substituting Eq. (6) into (3) and integrating, we obtain the totq ltime T to travel the path

2g T = 1 + (dx/dz) 2dz (7)I-= V x - c-

This is the integral to be minimized.Identifying Eq. (7) with our notation f Sec. 3.2, we find

1+ x2

x - c (8)

Because S is independent of t (or, as we have used it, z) we may use thefirst integral of the Euler equation [Eq. (22) of Sec. 3.2], which becomes

1 = const = (1 (9)(x-c)(1+x2) 2b

Equation (9) is a first-order differential equation for x, with the con-stant b to be determined from the boundary conditions. If we presumethat the solution is available and b is known, the control problem issolved, for we have an expression for the steering angle, arctan z, as afunction of the height x. That is, we have the relation for an optimalnonlinear feedback controller. We do seek the complete solution, how-ever, and to do so we utilize a standard trick and introduce a new varia-ble i' such that

sins'x (10)

1 +cos

It then follows from Eq. (9) that

x=c+b(1+cosr) (11)

anddz _ dz dx = b(1 + cos t) (12)

or

dr dx dr

z = k+b(1-+sinr) (13)


Equations (11) and (13) are the parametric representations of thecurve of minimum time, which is a cycloid. This curve daps out thelocus of a point fixed on the circumference of a circle of radius b asthe circle is rolled along the line x = c, as shown in Fig. 3.2. When theboundary conditions are applied, the solution is obtained as

x=x1+(x2-xl) cosh 0< <rf (14a)1 - COS tf

z=zl'+(z2-zl) r+sini' 0<<3'f (14b)_f+sinl'f

where 3'f is the solution of the algebraic equation

1 x2-xi+ sin if z2 - zI

The; existence of such solutions is demonstrated in texts on the calculusof variations.

3A OPTIMAL LINEAR CONTROL

As a second and perhaps more useful example of the application of theEuler equation we shall return to the optimal-control problem formu-lated in Sec. 1.4. We have a system which varies in time according tothe differential equation

i = Ax + w (1)

where A is a constant and w a function of time which we are free tochoose. The optimal-control problem is to choose w(t) so as to minimizethe integral

rC

2 Jo(x2 + c2w2) dt (2)

I1 IZ

X=C

Fig. 3.2 Construction of a cycloid.

so OPTIMIZATION BY VARIATIONAL METHODS

If. we solve for w in Eq. (1) and substitute into Eq. (2), we have

g jo [x2 + c2(± - Ax)2] dt (3)

orif = Y2[x2 + c2(± -- Ax)2] (4)

The Euler equation is then

dc2(Z-Ax) =x-Ac2(t-Ax) (5a)

or

c (1 + A2c2)x(5b)

x(O) is given. Since x(O) is not specified, the second boundary conditionis obtained. from a9/ft = 0 at t = B, or

x-Ax=O att=B (6)

Equation (5) may be solved quite easily to obtain x(t), and hencei(t) and the optimal control policy w(t). For purposes of implemen-tation, however, it is convenient to have a feedback controller, wherew is a function of the present state x, for such a result will be independ-ent of the particular initial condition x(0). Motivated by the resultsobtained in Chap; 1, we seek a solution in the form of a proportionalcontroller

w(t) = M(t)x (7)

and Eq. (1) becomes

x = (A + M) x (8)

Differentiating once, we have

.t = Az + Mt + Mx = Mx + (A + M) 2x (9)

and this must equal the right-hand side of Eq. (5b). For x 96 0, then,M must satisfy the Riccati ordinary differential equation

M + 2AM + M2 - = 0 (10)

with the.boundary condition from Eq. (6)

M(0) = 0 or x(9) = 0 (11)

The Riccati equation is solved by use of the standard transformation

M(t) = y(t) (12)y(t)


with the resulting solution

11 (l) _ - A + 1 + A2C2 1 - k exp [2 (1 + A2c2)/c2 t]4 c2 1 + k exp [2 1/(1 + A2c2)/c2 t]}

(13)

where the constant k is determined from the condition M(B) = 0 as

k = V/1 + A2c2 - Ac eXp

(_2 J1 +A2c2 e (14)

2C2 +Ac

In the limit as0

oo we have k --, 0, and M becomes a constant

M -(A+ /1 +A2c2) (15)

which is the result obtained in Sec. 1.5. Here we do not satisfy M(B) = 0but rather x(B) = 0. We have again found, then, that the optimal con-trol for the linear system with quadratic objective function is a proportionalfeedback control, with the controller gain the solution of a Riccati equation,and a constant for an infinite control time.

3.5 A DISJOINT POLICY

It sometimes happens that a problem can be formulated so that the inte-grand is independent of the derivative, z; that is,

&[x(t)] = fo 5(x,t) dt (1)

The Euler equation then reduces to

0 = ax (2)

In this case, however, a much stronger result is easily obtained withoutany considerations of differentiability or constraints on x.

If t(t) is the function which minimizes &, then for all allowablefunctions x(t)

f 5[x(t),t] dt - fo 3[x(t),t] dt < 0 (3)

In particular we shall take x(t) different from x(t) onlyinterval t, < t < t, + 0, for arbitrary ti. We thus have

and using the mean-value theorem,

5[x(t),t]l < 0


where t is somewhere in the interval ti < t < ti + A. Dividing by Aand then letting A go to zero, we find

ff[z(t),t] < Y[x(t),t] (6)

for all t, since tl was arbitrary. That is, to minimize the integral in Eq.(1) choose x(t) to minimize the integrand at each value of t. Such an opti-mal policy is called disjoint.

An immediate-application arises in the design and control of a batchchemical reactor in which a single reaction is taking place. If, for sim-plicity, we suppose that the reaction is of the form

n,A ± n2B (7)

that is, A reacting to form B with' some reverse reaction, then since thereare no inflow and outflow streams in a batch reactor, the conservation ofmass requires that the rate of change of the concentration of A, denotedby a(t), be equal to the net rate of formation of A. That is

d = -r(a,b,u) (8)

where b is the concentration of B and u is the temperature. Since theamount of B present is simply the initial amount of B plus the amountformed by reaction, we have

b(t) = b(0) +

n=

[a(O) a(t)] (9)

where n,/n2 represents the number of moles of B formed from a mole of A.Thus, we can write

d = -r(a,u) (10)

where it is understood that a(O) and b(0) enter as parameters.Suppose now that we wish to achieve a given conversion in the

minimum time by a programmed control of the reactor temperature u(t).We can solve Eq. (10) formally for the total time as

a(o) da = 9 (11)face) r(a,u) -We wish to choose u(t) to minimize 9, and we note that since a(t) will bea monotonic function of time, when the solution is available, it will besufficient to know a if we wish to know the time t. Thus, we may con-sider u as a function of a and write

F[u(a),a] = r(Qu) (12)

Since r > 0, we shall minimize 1/r by maximizing r. The optimal policy

CALCULUS OF VARIATIONS $3

is then to choose the temperature at each instant to maximize the instanta-neous rate of reaction. This result is probably intuitively obvious.

The kinetics of the reaction will usually be of the form/ F'\r(a,b,u) = plan, exp 1 - J1 } - p2b", exp [ - ' 1/\ u / \ u

plan, exp -ll - p2 (bo 12 ao - 12 a)n, exp 1 u2u n,/

(13)

An internal maximum cannot occur when E,' > Eg (an endothermic reac-tion), since r > 0, and the maximum is attained at the highest value of uallowable. This, too, is perhaps intuitively obvious. The only case ofinterest, then, is the exothermic reaction, E2 > E.

The maximum of Eq. (13) can be found by differentiation to give

Ei - E;U (14)

In p,E' [b(0) + (n2/nI)a(0) - (n2/n,)alnt

n,(PlEf a

By differentiating Eq. (14) we obtain

2> 0 (15)

du u2

da nl(E'2? E>) 'na g + band since a(t) decreases in the course of the reaction, the optimal tem-perature is monotone decreasing.

There will, in general, be practical upper and lower limits on thetemperature which can be obtained, say u* < u < u*, and it can be seenfrom Eq. (14) that a small or zero value of b(0) may lead to a tempera-ture which exceeds u*. The starting temperature is then

'u = min u*, E2' E1(16)

p:E2 b(Q)n'lIn p1Eia(0)n'J

If the starting temperature is u*, this is maintained until Eq. (14) issatisfied within the allowable bounds. In a similar way, the tempera-ture is maintained at u* if the solution to Eq. (14) ever falls below thisvalue. The optimal temperature policy will thus have the form shownin Fig. 3.3. The solution for the concentration a(t) is obtained by sub-stituting the relation for the optimal temperature into Eq. (10) and inte-grating the resulting nonlinear ordinary differential equation. The solu-tions have been discussed in some detail in the literature.

The monotonicity of the function a(t) means that the policy whichprovides the minimum time to a given conversion must simultaneouilly.provide the maximum conversion for a given time. This must be tom,

$4 OPTIMIZATION BY VARIATIONAL METHODS

U*

U(f)

U*

0 Fig. 3.3 Optimal temperature schedulein a batch reactor.

for were it possible to reach a smaller value of a(8) [larger conversiona(O) - a(9)] in the same time 9, the required value of a(O) could havebeen reached in a shorter time, which contradicts the assumption of aminimum-time solution. Since Eq. (10) can be applied to chemicalreaction in a pipeline reactor in which diffusion is negligible when t isinterpreted as residence time in the reactor, we have thus obtained atthe same time the solution to the following important problem: find thetemperature profile in a pipeline reactor of fixed length which will maximizethe conversion.

3.6 INTEGRAL CONSTRAINTS

In applications of the calculus of variations it frequently happens thatthe integral.

8[x(t)] = fo ff(x,x,t) dt (1)

must be minimized subject to a so-called isoperimetric constraint,

fa G(x,x,l) dt - b = 0 (2)

We can obtain a formal solution to this problem in the following way.Let x(t) be the solution, and take

x(t) = x(t) + E1171(t) T E2, 2(t) (3)

where El and t2 are constants and 771(t) is an arbitrary differentiable func-tion. For a given 'n1, E1, and e2, the function 172 will be determined byEq. (2). The minimum of 8 subject to the constraint must then occurat E1 = E2 = 0, and we may write

6(E1,E2) = foe 5(2 + E1711 + E2172, ± + E1711 + E2712, t) dl (4)

g(E1,E2) = f4B

G(x + E1771 + E2712, + E17l1 + E27f2, t) dt - b = 0 (5)

The minimum of 8 subject to the constraint equation (5) is found by

CALCULUS OF VARIATIONS $5.

forming the lagrangian

2(il,t2) = a((1,E2) + Xg(el,E2) (6)

and setting partial derivatives with respect to el and E2 equal to zero atel = E2 = 0, we obtain

Io rax (9 + xG) - at ai (ff + XG)] qi(t) dl + (a + XG)n1 o = 0L

(7a)

fo c3x(5- + XG) - at ax (if + AG)J 'n2(t) dt + az (if + XG)172 to = 0

(7b)

Since ill(t) is arbitrary, we obtain, as.in Sec. 3.2, the Euler equation

d (9dcai(g+XG) -a (if+XG) = 0

with the boundary conditions

(8)

At t = 0: x(0) given or a (if + XG) = 0 (9a)

At t = 0: x(0) given or ax (if + XG) = 0 (9b)

The constant Lagrange multiplier X is found from Eq. (2): For n "func-tions xl(t), x2(t), ... , x,,(t) and m constraints

G.(xl,x2, .fo° ,xn)xl>x2, . . . ,2n,t) dt -- bi = 0

The Euler equation isi = 1, 2, . . . , m < n (10)

dd a" (Y + axe

+ I a1G;) = 0j-1 j-1

k=1,2,...,n (11)It is interesting to note that the Euler equation (8) is the same

as that which would be obtained for the problem of extremalizingj if(x,z,t) dt = const (afterJo G(x,x,t) dt subject to the constraint

0

dividing by the constant 1/X). This duality is of the same type as thatfound in the previous section for the minimum-time and maximum-conversion problems.

3.7 MAXIMUM AREA

The source of the name isoperimetric is the historically interesting prob-lem of finding the curve of fixed length which maximizes a given area.


If, in particular, we choose the area between the function x(t) and theaxis x = 0 between t = 0 and t = 0, as shown in Fig. 3.4, for a curve x(t)of fixed are length L, we then seek to minimize

E = - area = - fa x(t) dt (1)

withAre length =

0(1 + ±2)3`' dt = L (2)

The Euler equation (8) of Sec. 3.6 is then

dt Y7 [-X + X (j + x2),1] = ax [-x + X (I + ±2)%] (3)

or

d i = -1dt 1 -+±2

This has the solution

(4)

(x-E)2+(t-r)2=X2

(5)

which is the are of a circle. The constants t, r, and X can be evaluatedfrom the boundary conditions and Ea. (2).

3.8 AN INVERSE PROBLEM

Th,, second-order linear ordinary differential equation

at p(t)x + h(t)x + f(t) = 0 (1)

appears frequently in physical problems, and, in fact, an arbitrary linearsecond-order equation

a(t)y + b(t)y + c(t)y + d(t) = 0 (2)

may be put in the self-adjoint form of Eq. (1) by introducing the cnange

A

Fig. 3A Area enclosed by a curve oflength L and the t axis.

CALCULUS OF VARIATIONS $7

x(t) = y(t) exp (!u b a a dt (3)

Because of the wide\\\

application of Eq. (1) it is of interest to determineunder what. conditions it corresponds, to the Euler equation for somevariational problem.

We shall write h(t) and f (t) as

h(t) = t(t) - q(t) f(t) = m(t) - n(t) (4)

so that Eq. (1) can be rewritten

d [p(t)t + r(t)x + m(t)] = r(t)± + q(y)x + n(t) (5)

This will be of the form of an Euler equation

doff ofdt ax ax

provided that

497 = p(t)t + r(t)x + m(t)ax

= r(t)x + q(t)x + n(t)TX

(6)

(7b)

Integrating, from Eq. (7a)

3F = Y2[p(i)x= + 2r(t)xx + 2m(t)x] + arbitrary function of x (8a)

while from (7b)

3[q(t)x2 + 2r(t)x-4- + 2n(t)x] + arbitrary function of x (8b)

so that, within an additive constant,

%[p(t)x= + 2r(t)x± + q(t)x!] + m(t)± + n(t)x (9)

with the boundary conditions then becoming

At t = 0 and B: x fixed or px + rx + m = 0 (10),

As shown in the problems, these boundary conditions may be generalizedsomewhat.

In this next section we shall discuss an approximate method fordetermining stationary values for the integral fo ff(x,x,t) dt, and as aconsequence of the result of this section, such a procedure can always beapplied to the solution of a linear second-order differential equation.

U OPTIMIZATION BY VARIATIONAL METHODS

3.9 THE RITZ-GALERKIN METHOD

The Ritz-Galerkin method is an approximate procedure for the solutionof variational problems. Here a particular functional. form is assumedfor the solution with only parameters undetermined, and the integral

fII(x,z,t) dt (1)

which now depends only on the parameters, is minimized (or more gener-ally, made stationary) by choice of the parameters. Suppose, for exam-ple, we seek an approximate solution of the form

N

x(t) 4,o(t) + I C.44.(t) (2)x-1

where ko satisfies any nonzero boundary specifications and the l4. aremembers of a complete set which vanish at any boundary where x(t) isspecified. The minimum of S is then found approximately by choosingthe coefficients C1, C29. . . , CN.

Substituting Eq. (2) into Eq. (1), we obtain

6 x f0 5(#o + FC.O., Oo + t) dt (3)

S is now a function of C1, Cs, . . . , CN, and upon differentiating K withrespect to each of the C. and setting the result to zero

aCf

ax + ax '*) dt = 0 n = 1, 2, ... , N (4)

Here the subscript ap refers to the fact that the partial derivatives areevaluated for the approximate solution. Equation (4) leads to N alge-braic equations for the N coefficients after the integration is carried out,and is generally known as the Ritz method.

An alternative form which requires less computation when dealingdirectly with the solution of Euler differential equations is obtained byintegrating Eq. (4) by parts. Thus

d ax) 4,.(t) dl + az n ofo k az d:(5)

and since either 4.. is zero (x specified) or 8ff/81 vanishes (natural bound-ary condition), we obtain, finally, the Galerkin form

0 BS.D

fdW.P ¢.(t)dt=0 n= 1,2, ... N (6)

o ax ' dt ea~ }Since an exact solution would satisfy the Euler equation,

8R d 8R.3x - dt 8

= 0 (7)

CALCULUS OF VARIATIONS

we may consider the quantity in parentheses in Eq. (6) as the residual ateach value of t which remains when an approximation is used in the left-hand side of the differential equation. Thus, writing

a9;., d as:.P_ R(C, C2, ... Cm,t)ax - it ax

Galerkin's method may be written

foe R(C2,C2, . . . ,Cjv,t)4,.(t) dl = 0 n= 1, 2, ... , N

(8)

(9)

This procedure is often used for the approximate solution of differentialequations which are not Euler equations but without the rigorous justi-fication provided here.

For demonstration, let us consider the approximate solution of thenonh6mogeneous Bessel equation

dwt- )

(t d + tx - t = 0 x(0) finite x(1)=

0 (10)

Comparing with Eq. (1) of Sec. 3.8 this corresponds to

p(t) = t q(t) = -t r(t) = 0 m(t) = 0 n(t) = 1 - (11)

and provided that t(0) remains finite, the boundary conditions of Eq.(10) of Sec. 3.9 are satisfied, the condition at t = 0 being the naturalboundary condition.

We shall use for approximating functions

n=1,2, . . . , N (12a)

which vanish at t = 1 and remain finite at t = 0. A function yf'e is notneeded. Thus, substituting the approximation for x(t) into the left-handside of Eq. (10), the residual is writtenI

C. dt [tdt

to-'(1 - t)+ tn(1 - t)tR(C,,C2, ... ,CX,t)JJ 1

(12b)

and the coefficients are found by solving the linear algebraic equations

fo2

R(C,,C2, .. ,CN,t)tn-'(1 - t) dt = 0n = 1, 2, . . . , N (13)

In particular, for a one-term approximation

N = 1: R(C,,t) = -C, + (C2 - 1)t - Cit2 (14)

and the solution of Eq. (13) is C, = -0.4, or

N = 1: x(t) ;; -0.4(1 --- t) (15)


Table 3.1 Comparison of the first-and second-order approximations by theRitz-Calerkin method to the exact solution

- x(t)

t N=1 N - 2 Exact

0.0 0.40 0.310 0.307

0.1 0.36 0.304 0.303

0.2 0.32 0.293 0.294

0.3 0.28 0.276 0.277

0.4 0.24 0.253 0.255

0.5 0.20 0.225 0.226

0.r) 0.16 0.191 0.192

0.7 0.12 0.152 0.151

0.8 0.08 0.107 0.1060.9 0.04 0.056 0.055

1.0 0.0 0.0 0.0

j, 'or a two-term approximation,

N = 2: R(C1,C2,t) = (C2 - C1) + (C1 - 4C2 - 1)t+ (C2 - C1)t2 - C2 t3 (16)

and Eq. (13) becomes simultaneous algebraic equations for C1 and C2,with solutions C1 = -0.31, C2 = -0.28, or

N = 2: x(t) ..- -0.31(1 - 1) - 0.281(1 - t)The exact solution is

(17)

(t) = 1 _ Jo(t)(18)x

Jo(1)

where Jo is the Bessel function of zero order and first kind. Table 3.1compares the two approximations to the exact solution.

3.10 AN EIGENVALUE PROBLEM

The homogeneous equation

dd1

(t dt) + Xtx = 0 (1)

with boundary conditions

x(0) finite x(1) = 0 (2)

clearly has the trivial solution x(t) = 0, but for certain eigenvalues, orcharacteristic numbers, X, a nontrivial solution can be obtained. These


numbers are solutions of the equation

Jo (.\/-X) = 0 (3)

and the first two values are X1 = 5.30, X2 = 30.47.The Ritz-Galerkin method may be used to estimate the first several

eigenvalues. If we again use the approximationN

x(t) - I 0 (4)

then

n-1

N = 1: R(C1it) = -C1 + xC1t - C1t2

and Eq. (13) of Sec. 3.9 becomes

N= 1: C1(X-G) =0

(5)

(6)

Since if C1 vanishes we obtain only the undesired trivial solution, we cansatisfy Eq. (6) only by

X=6 (7)

Thus, by setting N = 1 we obtain an estimate of the first (smallest)eigenvalue.

Similarly, for a two-term expansion

N = 2: R(C1iC2,t) = (C2 - C1) + (XC1 - 4C2)t+ X(C2 - C1)t2 - )C2t3 (8)

and we obtain the two equations

C1(5X - 30) + C2(2X - 10) = 0 (9a)C1(2X - 10) + C2(X - 10) = 0 (9b)

These homogeneous equations have a nontrivial solution only if thedeterminant of coefficients vanishes, or X satisfies the quadratic equation

(5X - 30) (X - 10) - (2X - 10) 2 = 0 (10)

The two roots are

X1=5.86 A2=34.14

giving a (better) estimate of the first eigenvalue and an initial estimateof the second. In general, an N-term expansion will lead to an Nth-orderpolynomial equation whose roots will approximate the first N eigenvalues.

3.11 A DISTRIBUTED SYSTEM

In all the cases which we have considered thus far the Euler equationshave been ordinary differential equations. In fact, the extension of the


methods of this chapter, which are based only on differential calculus,to the study of distributed systems is straightforward, and this sectionand the next will be devoted to typical problems. We shall return tosuch problems again in Chap. 11, and there is no loss in continuity inskipping this section and the next until that time.

Let us suppose that x is a function of two independent variables,which we shall call t and z, and that x(t,z) is completely-specified whent = 0 or 1 for all z and when z = 0 or 1 for all t. We seek the functionx(t,z) which minimizes the double integral

lg[x(t,z)l = f o1 Jo [

()2

+(+

0(x) i dt dz (1)

where O(x) is any once-differentiable function of x.If we call 2(t,z) the optimum, we may write

x(t,z) _ 2(t,z) + E,t(t,z) (2)

where a is a small number and n(t,z) is a function which must vanish att = 0 or 1 and z = 0 or 1. For a particular function n the integral 8depends only on a and may be written

1 1 1 (a-f an 2 1 az an 28(e) = 0 fo

2 at +Eat) +(az +eaz)+ 4(2 + 2)] dt dz (3)

The minimum of 8 occurs when e = 0 by the definition of 2, and at e = 0the derivative of 8 with respect to a must vanish. Thus,

d3 r1 1 ax an + ax an +0'(2)n dt dz = 0 (4)

aat at fo n dt (5)

and the first term vanishes by virtue of the restrictions on n. A similarintegration may be carried out on the second term with respect to z, andwe obtain

(11 a2z a22

!o fo - ate W2 +,(2)] n(t,z) dt dz = 0 (6)

Since n is arbitrary except for the boundary conditions and obvious differ-entiability requirements, we may set

1 z- 2-

n(t,z) = w(t,x) - ate azz + 0'(x)] (7)


where

w(0,z) = w(l,z) = tv(t,O) = w(t,1) = 0w(t,z) > 0 for t, z * 0, 1

Thus,

fr ((

z- 2- 2

JolJol

w(t,z) at2 + az2 - '(z) dl dz = 0

u

(8)

(9)

and it follows that x must satisfy the Euler partial differential equation

491X 2

ate + az2 -(x) = 0 x specified at t = 0, 1

z=0,1 (10)

The partial differential equation

IX 49 2

ate + azx - kF(x) = 0

arises often in applications, as, for example, in two-dimensional mass orheat transfer with nonlinear generation. The method discussed in Sec. 3.9may then be extended to this case to determine approximate solutions.

3.12 CONTROL OF A DISTRIBUTED PLANT

Many control problems of interest require the control of a system whichis distributed in space by the adjustment of a variable operating only ata physical boundary. The complete study of such systems must awaita later chapter, but we can investigate a simple situation with the ele-mentary methods of this chapter. One of our reasons for doing so is todemonstrate still another form of an Euler equation.

A prototype of many important problems is that of adjusting thetemperature distribution in a homogeneous slab to a desired distributionby control of the fuel flow to the furnace. The temperature in the slab xsatisfies the linear heat-conduction equation

axla2x1 0<t<0Wt-

_az2 0<z<1 (1)

with a zero initial distribution and boundary conditions

ax,0 at z = 1

c3z(2)

ax1 _az P(xl-x,) atz=0

S4 OPTIMIZATION BY VARIATIONAL METHODS

Here x2 is the temperature of the furnace. The first boundary conditionis a symmetry condition, while the second reflects Newton's law of cool-ing, that the rate of heat transfer at the surface is proportional to thetemperature difference. The furnace temperature satisfies the ordinarydifferential equation

2r dt + x2 = u(t) x2(0) = 0 (3a)

where u(t) is the control variable, a normalized fuel feed rate.The object of control is to obtain a temperature distribution x; (z)

in time 6 or, more precisely, to minimize a measure of the deviationbetween the actual and desired profiles. Here we shall use a least-squarecriterion, so that we seek to minimize

ts[u] =fol

[x; (z) - xi(6,z)l2 dz (3b)

In order to use the procedures of this chapter it will be necessary toobtain an explicit representation of s[u] in terms of u. Later we shallfind methods for avoiding this cumbersome (and often impossible) step.Here, however, we may use either conventional Laplace transform orFourier methods to obtain the solution

toxl(8,z) = fK(6 - t, z)u(t) dt (4)

where

=a2 cos a(l - z) ITK(t'z) cos a - (a/p) sin a e

a

+ 2a2 COS (1 - Z)Pi

it (a2 -Oil)

1 + 1 + Pcos +i

with a - 1// and Pi the real roots of

#tan$ = p

Thus,

t;[u] =f01

[xl (z) - fo K(6 - t, z)u(t) dt,2 dz

(5)

(6)

(7)

Now, in the usual manner, we assume that u(t) is the optimum,,(t) is an arbitrary differentiable function, and a is a small number and let

u(t) = u + ert (8)


The function of e which results by evaluating Eq. (7) is

s[n + e>)J = f oI [x, (z)]2 clz - 2 f,' x" (z) fo K(8 - t, z)i (t) clt dz

- 2e fol x; (z) fo K(O - t, z),,(t) dt dz

+ 0f' K(O - t, z)u(t) dt]2 dz

+ 2efot

[ f u K(8 - r, z)u(r) dr] [fo K(O - 1, z)-,(I) dl] dz

+ e2 fot [fo K(8 - t, z), (t) dt]2 dz

ss

(9)

and, evaluating d8/de ate = 0,

de I.so =-2fg x, (r) fo K(8 - t, z)n(t) dt dz

+ 2 Jot [ fo

K(0 - r, z)u(r) dr, [fo K(9 - t, z),7 (t) dt] dz = 0 (10)

There is no difficulty in changing the order of time and space integration,ind the inner two integrals in the second term may be combined to give

fo,7(t) [

fotK(6 - t, z) x,* (z) dz

- fofo' K(O - 1, z)K(8 - r, z)u(r) dz dr] dt = 0 (11a)

or, setting the arbitrary function n(t) proportional to the quantity inbrackets,

foIK(8 - t, z)x, (z) dz

IK(0 - t, z)K(8 - r, z) dz] a(r) dr = 0

fo

B

[ fo

Equation (11) is an integral equation for the function u(i).

(11b)

It issimplified somewhat by defining

ow =101

K(8 - t, z)x; (z) dz (12a)

G(t,r) =0

IK(9 - 1, z)K(O - r, z) dz (12b)

so that the Euler equation becomes

fo G(t,r)u(r) dr = 41(t) (13)

This is a Fredholm integral equation of the first kind. Analytical solu-tions can be obtained in some special cases, or numerical methods maybe used. An obvious approach is to divide the interval 0 < I < 0 intoN even increments of size At and write

G = G(i At, j At) At (14a)u, = u(j at) (14b)O; = 1'(i At) (14c)

!6 OPTIMIZATION BY VARIATIONAL METHODS

We obtain an approximation to u(t) by solving the linear algebraic equa-tions approximating Eq. (13)

N

i = 1,2,...,N (15)-x


Sections 3.2 and 3.3: An extremely good introduction to the calculus of vairations bymeans of detailed study of several examples, including the brachistochrone, is

G. A. Bliss: "Calculus of Variations," Carus Mathematical Monograph, MathematicalAssociation of America, Open Court Publishing Co., La Salle, Ill., 1925

Other good texts on the calculus of variations include

N. I. Akhiezer: "The Calculus of Variations," Blaisdell Publishing Company, Wal-tham, Mass., 1962

G. A. Bliss: "Lectures on the Calculus of Variations," The University of ChicagoPress, Chicago, 1946

0. Bolza: "Lectures on the Calculus of Variations," Dover Publications, Inc., NewYork, 1961

It. Courant and D. Hilbert: "Methods of Mathematical Physics," vol. 1, InteraciencePublishers, Inc., New York, 1953

L. A. Pars: "An Introduction to the Calculus of Variations," John Wiley & Sons,Inc., New York, 1962

Applications specifically directed to a wide variety of engineering problems are found in

It. S. Schechter: "The Variational Method in Engineering," McGraw-Hill BookCompany, New York, 1967

section 3.4: We shall frequently use problems in control as examples of applications ofthe optimization theory, and complete references are given in later chapters. Auseful introduction to the elements of process dynamics and control is

1). R. Coughanowr and L. B. Koppel: "Process Systems Analysis and Control,"McGraw-Hill Book Company, New York, 1965

The reduction of the optimal feedback gain to the solution of a Riccdti equation is anelementary special case of results due to Kalman, which are discussed in detail inlater chapters. An approach to linear feedback control like the one used here, basedon the classical calculus-of-variations formulation, is contained in

P. Das: Automation Remote Contr., 27:1506 (1966)

Section 3.6: The first discussion of the single-exothermic-reaction problem is in

K. G. Denbigh: Trans. Faraday Soc., 40:352 (1944)

A fairly complete discussion is in

It. Aria: "The Optimal Design of Chemical Reactors," Academic Press, Inc., NewYork, 1961


The approach taken here, as applied particularly to the exothermic-reaction problem, iscredited to Horn in

K. G. Denbigh: "Chemical Reactor Theory," Cambridge University Press, NewYork, 1965

where there is discussion of the practical significance of the result. Some useful con-siderations for implementation of the optimal policy when parameters are uncertaincan be found in

W. H. Ray and It. Aris: Ind. Eng. Chem. Fundamentals, 5:478 (1966)

Sections 3.6 and 3.7: The texts cited for Secs. 3.2 and 3.3 are pertinent here as well.

Section 3.8: The general inverse problem may be stated: When is a differential equationan Euler equation? This is taken up in the text by Bolza cited above and in

J. Douglas: Trans. Am. Math. Soc., 50:71 (1941)P. Funk: "V'ariationsrechnung and ihr Anwendung in Physik and Technik," Springer-

Verlag OHG, Berlin, 1962F. B. Hildebrand: "Methods of Applied Mathematics," Prentice-Hall, Inc., Engle-

wood Cliffs, N.J., 1952

Sections 3.9 and 3.10: The approximate procedure outlined here is generally known asthe Ritz or Rayleigh-Ritz method and as Galerkin's method when expressed in termsof the residual. Galerkin's method is one of a number of related procedures knownas methods of weighted residuals for obtaining approximate solutions to systemsof equations. A review of such methods with an extensive bibliography is

B. A. Finlayson and L. E. Scriven: Appl. Mech. Rev., 19:735 (1966)

See also the text by Schechter cited above and others, such as

W. F. Ames: "Nonlinear Partial Differential Equations in Engineering," AcademicPress, the., New York, 1965

L. Collatz: "The Numerical Treatment of Differential Equations," Springer-VerlagOHG, Berlin, 1960

L. V. Kantorovich and V. I. Krylov: "Approximate Methods of Higher Analysis,"Interscience Publishers, Inc., New York, 1938

Sections 3.11 and 3.12: Distributed-parameter systems are considered in some detailin Chap.' 11, with particular attention to the control problem of Sec. 3.12. Thederivation of the Euler equation used here for that process follows

Y. Sakawa: IEEE Trans. Autom. Contr., AC9:420 (1964)

PROBLEMS

3.1. A system follows the equation'

x- -x + uFind the function u(t) which takes x from an initial value xo to zero while minimizing

E =IU

(K + us) dt

!t OPTIMIZATION BY VARIATIONAL METHODS

(time plus cost of control) where 8 is unspecified. Hint: Solve for fixed 0; then deter-mine the value of 0 which minimizes E.3.2. A body of revolution with axis of symmetry in the x direction may be defined asone which intersects all planes orthogonal to the x axis in a circle. Consider such abody whose surface in any plane containing the x axis is described by the curve y(x),

y(0) -0 y(L) -RThe drag exerted by a gas stream of density p and velocity v flowing in the x directionis approximately

dxJ - 4rpvi JOL y dx)

Find the function y(x) passing through the required end points which makes the draga minimum.3.3. S is & function of t, x, and the first n derivatives of x with respect to t. Find theEuler equation and boundary conditions for

min & - foe T(x,z, ... x(R),t) di

3.4. A second-order process described by the equation

I+ax+bx - uis to be controlled to minimize the error integral

rmin E -

oe

(x' + c'u') dt

Show that the optimal control can be expressed in the feedback form

u -Mix +M:xand find the equations for M, and M2.3.5. Obtain the Euler equation and boundary conditions for minimization of

E -2

Jo [p(t)i= + 2r(t)xz + 4(i)x' + 2m(t)x + 2n(t)x] dt + ax'(e) + bxs(0)

and relate

theeee

result to the discussion of Sec. 3.8.3A. Steady-state diffusion with isothermal second-order chemical reaction, as well asother phenomena, can be described by the equations

dz Dx2-0D- - h(x - xo) atz-0

Ddx -0 atz -L

where k, D, h, and xo are constants. Find the parameters in a cubic approximationto the solution.3.7. Using a polynomial approximation, estimate the first two eigenvalues of

Y+ax-0


for the following boundary conditions:

(a)x-0 at t=0,x(b)i-0 att-0x-x-0 at t=w

Compare with the exact values.3.8. Laminar flow of a newtonian liquid in a square duct is described by the equation

a'v+a2v-1APa0 v - Oatx - ta

ax' ay' is L y - toHere v is the velocity, p the viscosity, and AP/L the constant pressure gradient.

(a) Using the Galerkin method, find the coefficients A, B, and C in the approxi-mate form

v - (x' - a')(y' - a')(A + B(x' + y2) + Cx'y')

The solution is most conveniently expressed in terms of the average velocity,

LI -! JaQ

Ias

v(x,y) dx dy

(The form of the approximation is due to Sparrow and Siegal. Numerical values ofthe exact solution are given in the book by Schechter.)

(b) Formulate the flow problem in terms of minimization of an integral and usethe complex method to estimate values for A, B, and C.

4Continuous Systems:

4.1 INTRODUCTION

In the previous chapter we investigated the determination of an entirefunction which would minimize an objective, and we were led to theEuler differential equation for the minimizing function. It is rare thata problem of interest can be formulated in so simple a fashion, and weshall require a more general theory. Consider, for example, a chemicalreactor which we wish to control in an optimal manner by changing cer-tain flow rates as functions of time. The laws of conservation of massand energy in this dynamic situation are represented by ordinary differ-ential equations, and the optimizing function must be chosen consistentwith these constraints.

We shall assume that the state of our system can be adequatelyrepresented by N variables, which we shall denote by xi, x2, . . . , xx.In a chemical system these variables might be concentrations of the per-tinent species and perhaps temperature or pressure, while for a spacevehicle they would represent coordinates and velocities. In addition,100

CONTINUOUS SYSTEMS: I 101

we suppose that certain control or design variables are at our disposalto adjust as we wish, and we shall denote them by ul, u2, ... , UR.These might be flow rates, temperatures, accelerations, turning angles,etc. Finally, we suppose that the state variables satisfy ordinary differ-ential equations

dxi i = 1, 2, .. , N.ti =

dt = fi(xl,x2, . . . )xN,u'1,u2, . . . ,UR) o < t < 8(1)

and we wish to choose the functions Uk(t), k = 1, 2, . . . , R in order tominimize an integral

eC[ul,u2, . . . ,uR) = fo 5(xl,x2, . . . ,XN,UIlu2) . . . UR) dt (2)

We shall generally refer to the independent variable t as time,although it might in fact refer to a spatial coordinate, as in a pipelinechemical reactor. The total operating duration 0 may or may not bespecified in advance, and we may or may not wish to impose conditionson the variables xi at time 0. A typical measure of performance in acontrol problem might be a weighted sum of squares of deviations frompreset operating conditions, so that 5 would be

CJ(xl _ x18)2 + . . . + CN(XN ZNS)2+ CN+1(ul - u18)2 + . . . + CN+R(UR - URS)2 (3)

On the other hand, if the controls were to be set to bring x1, x2, . . . ,

xN to fixed values in the minimum time 8, we would wish to minimize& = 0 or, equivalently,

As we shall see, there is no loss of generality in the choice of the per-formance index, Eq. (2).

4.2 VARIATIONAL EQUATIONS

The development in this section parallels that of Sec. 1.8. For con-venience we shall put N = R = 2, although it will be clear that theresults obtained are valid for any N and R. We thus have the statedefined by the two differential equations

±1 = f1(x1,x2,u1,u:) (14)i2 = f2(xl,x:,u1,u1.) (lb)

me OPTIMIZATION BY VARIATIONAL METHODS

and the performance index

&[u1,u2] = 0ff(xl,x2,ul,u2) dt (2)

Let us suppose that we have specified the decision variables ul(t),u2(t) over the entire interval 0 < t < 0. Call these functions ul(t), u2(t).For given initial conditions x1o, x20 we may then solve Eqs. (1) for xl(t)and 22(t), 0 < t < 0, corresponding to the choices ul(t), 42(t). The valueof the performance index is completely determined by the choice of deci-sion functions, and we may call the result t;[u1iu2].

We now change ul and u2 at every point by small amounts, bul(t)and 6u2(t), where

bol(t)I, Ibu2(t)I < e 0 < t < 0 (3)

and a is a small positive constant. [If x1(0) and x2(0) are not specified,we also make small changes U1(0), bx2(0).] That is,

U1(t) = ul(t) + bul(t) (4a)u2(t) = u2(t) + W2(t) (4b)

and as a result we cause small changes bxl(() and d,z2(t) in xl and x2,respectively, and a change 66 in the performance index. We can obtainexpressions for bxl, 6x2r and 56 by evaluating Eqs. (1) and (2) with (ul,u2)and (ul + bul, u2 + 6u2), successively, and then subtracting. Thus,

at (xl + axl) - at- xl = 6x1

= fl(xl + bxl, 22 + 6x2, ul + aul, u2 + but) - fl(21,22,41,u2)(5a)

it (22 + bx2) - jx2 = bx2

= f2(xl + bxl, x2 + 5x2, u1 + aul, u2 + but) - f2(xl,x2,u1,i72) (5b)3& = g[ul + but, u2 + but] - F,[ul,u2]

= Jobxl, x2 + 6x2, 'ul + aul, u2 + but) - ff(21,22,u'1,i22)] dt

rrg+60

ff(21 + 6x1, x2 + 6x2, ul + bul, u2 + 8u2) dt (6)+ Jd

The last term in Eq. (6) must be added in the event that the total time 0is not specified, and a change in the decisions requires a change in thetotal process time in order to meet some preset final condition. B thenrepresents the optimal duration and 60 the change.

If we expand the functions fl, ff, and T at every t about their.respec-tive values at that t when the decisions ul, u2 are used, we obtain

/ a a a'ai; = fo ` ax,

bxl -}axe

ax, + aul bui +0U2

bug dt

\ + 60 + o(E) (7)

CONTINUOUS SYSTEMS: I

Sz2

afiSxl+ aflSx2+ aflSul+ af26u2+0(e)axl ax2 au, au2af26xl+ af2Sx2+ af2Sul+ af2Su2+0(e)ax1 ax2 aul 8U2

103

(8a)

(8b)

We now multiply Eqs. (8a) and (8b), respectively, by arbitrary continu-ous functions X1(t) and X2(t), integrate from t = 0 to t = B, and add theresult to Eq. (7). Thus,

bS = faJl61 +Xlaxl+X2 XI 311

+1 a Y+Xlz+\2 2) ax2 - X26±2

+ (aul + Xi aul + X2 of l) aul

(a+ + X l Al + X.

af2)

au2 au2 au: sue dt + I _ a Sa + o(e) (9)

Integrating the terms X, ft, and X2 6±2 by parts gives, finally,

SS=fIM-5 +Xlafl+X2azi+Sxl

+ (az + X1 ofy+ X2 auz + ^2) 6x2

+(T11l + Xl aui + X2 aui1 Sul

(C of l

of t)+ au, + l au2 + Z au2sue dt

+5It-S

Se - X1(8) Sxl(6)

- X2(B) 6x2(9) + X1(0) 6xl(0) + X2(0) 6x2(0) + 0(t) (10)

Now, just as in Sec. 1.8, we find ourselves with terms which are atour disposal, namely, the decision changes Sul(t), 5u2(t), and terms whichare not, the state variations Sxl(t) and Sx2(t). We therefore eliminatethese latter terms from the expression for a& by removing some of thearbitrariness from the functions al(t), X2(t) and requiring that they satisfythe differential equations

lafl - aft

(h a)^1 = - axl - a , 2xla axlag afl aft2ax2

Xl aZ2 - X249x2

(11b)

Note that we have not yet specified boundary conditions. The variation


8& is now

b8g a afl afsl

+ Xl au '} X2 au ` 8161= fau1 1

+`a 2+Xiaus+X,/2) 6u2]dl

+5 I1-1 ae - X1() axl(e) - X2(8) 8x2()

+ X1(0) 6xi(0) + X2(0) 5x2(0) + 0(E) (12)

It is necessary at this point to distinguish between cases when thetotal duration 0 is specified and when it is not. If 0 is specified, then58 must be zero. If x1(9) is fixed,'then 8x1(9) is zero and the termX1(8) hl(8) vanishes. If, on the other hand, x1(9) is free to take on anyvalue, we have no'control over 8x1(8), so that we remove it from theexpression for SE by specifying that X1(0) = 0. Similar considerationsapply to the term X2(0) 8x2(9), and we obtain the boundary conditions:

0 specified:x1(9) free X1(9) = 0

x1(9) fixed X1(9) unspecifiedX2(0) free X2(8) = 0 (13)

x2(8) fixed X2(0) unspecified

If a is not specified, the variations 68, 6x1(8), and 5x2(9) are related.In fact,

x1(8 + ae) = x1(6) + axl(e) + fl Il_s 59 + o(e) (14)

and similarly for x2. Thus, if x1 is fixed at the end 'of the process, werequire x1(8 + 69) 21(6), and if x2 is free, the terms

if 69 - X1 5x1 - X2 8x2 = (`if + Xlfl) 58 - X2 5x2 (15)

and similarly for x2 fixed, or both. If neither x1 nor x2 is fixed, of coursewe may choose 89 = 0 and use the previous results. Thus, applying thelogic of the previous paragraph, we obtain the following conditions:

X1(8) = 0 X2(8) = 0

if + X1f1 = 0 X2 = 0 (16)if + X2f2 = 0 X1 = 0

if + Xlfl + X2f2 = 0We now apply the same approach to the terms X1(0) &x1(0) and

X2(0) 5x2(0) and obtain the further boundary conditions:

x1(0) free X140) = 0x1(0) fixed X1(0) unspecifiedx2(0) free X2(0) = 0x2(0) fixed X2(0) unspecified

0 unspecified:X1(0) free x2(8) freex1(8) fixed x2(8) freex1(8) free x2(8) fixedx1(8) fixed x2(8) fixed

(17)

CONTINUOUS SYSTEMS I -ins

For the problem with fixed 9, Eqs. (13) and (17) provide a total of fourboundary conditions for the four differential equations (1a), (1b) and(11a), (11b). When B is variable, conditions (16) and (17) provide fiveconditions, four boundary conditions and a stopping condition.t Withthese conditions we finally obtain an expression for 68 which., dependsonly on the variations in the decision variables,

Al af,+ X= aut) aul- Jo [(au + X1 aut

+ (aus + Claus +_) Susj dt + o(e) (18)

4.3 FIRST NECESSARY CONDITIONS

We now introduce into the discussion, for the first time, the fact thatwe are considering an optimization problem. Thus, if the choices ut(t),til(t) are those which minimize ,8, then for all variations fti(t), 5u2(t) it isnecessary that 8 increase, or S8 > 0. We consider in this section onlysituations in which the optimal decisions are unbounded and we are freeto make any (small) variations we wish. In doing so we exclude a largeclass of problems of interest (indeed, most!), and we shall return shortlyto considerations of constraints on the allowable decisions.

As in Chap., 1, we choose a particular set of variations which makesour task easy. We set .

i9lbut = _E {au + X 1 au, + = aut)85 ' #1 12

OU2 allll Cluj

where e' is a small positive constant. Thus Eq. (18) of the precedingsection becomes

r sS8 = -E' fo

aut + aut +,

= aut)

+ta5au +

X1 au= + a= au:

Z

,] dt + o(E) '- 0 (2)

Since e and e' are of the same order, it follows that

lim o(e) = 0t

t For the case # unspecified and both x,(B) and x2(6) unspecified we require an addi-tional condition. We shall find later that this condition is 5 - 0.

i-e


and dividing by e' and taking the limit in Eq. (2), we obtain

foe

of of l afz = afl af, _

I0Kau, + l aul + = aui) + 1 8U2 + X1 au= + X= au=/ j dt - 0

(4)

Since this is the integral over a positive region of a sum of squares, wecan satisfy the inequality only' if the integrand vanishes identically(except, perhaps, on a set of discrete points). We therefol'e concludethat if ui and u2 are the unconstrained functions which cause.& to take on itsminimum value, it is necessary that

aui + X, aui + a: aui .0. (5a)

except, perhaps, on a set of discrete points. These equations representthe extension of the multiplier rule to the problem of minimizing anintegral subject to differential-equation side conditions. It is importantto note that the optimal functions. ui(t), u2(t) are found from a set ofalgebraic relations to be satisfied at each point.

Let us pause for a moment and contemplate what we have done:In order to obtain conditions for the minimizing functions ui(t) and u=(t)we have had to introduce two additional functions Xi(t) and a=(t) whichalso satisfy a set of differential equations. The four boundary conditionsfor the total of four differential eqtiations are split between the two ends,t = 0 and t = 0, and there is an additional set of algebraic conditionsto be satisfied at each point. This is a rather formidable problem but onewhich we must accept if we are to attack design and control problemsof any significance.

It is often convenient, and always elegant, to introduce the hamil-tonian function H, defined as

H = ff + XJ, + X,f= (6)

The differential equations for xi, x=, ai, and a= can then be written in thecanonical form

aH aHxl =sAi

x= =sa=

(7a)

^1 aH x2= - OH (7b)

while the necessary conditions (5a) and (5b) can be written

aH=0 aH=0aui au2

(8)

CONTINUOUS SYSTEMS: 1 167

Furthermore, the hamiltonian is a constant along the optimal path, fordH aH aH aH . aH . aH .aH

1127t - ax, Z1 +aX2

xz {-aX'

1 -{Xs

)\f -}- aul ul + aus z (9)

which equals zero after substitution of Eqs. (7) and (8).When 8 is unspecified, the constant value of the hamiltonian is

found from the value at t = 0 to be zero. If x1(9) and x2(8) are free,we can simply apply the calculus to find the optimal stopping time

ae " elo 5dt = III = 0 (10)

Together with the first boundary condition [Sec. 4.2, Eq. (16)] this givesf

H = Hl,_, - 0 (11)

On the other hand, if x1j x2, or both are specified, Eq. (11) follows directlyfrom the remaining boundary conditions [Sec. 4.2, Eq. (16)].

We may summarize the results of this section in the followingstatements:

The unconstrained functions ul(t) and u2(t) which minimize 6 makethe hamiltonian stationary$ for all t, 0 < t < 8. The hamiltonian isconstant along the optimal path, and the constant has the value zerowhen the stopping time B is not specified.

This is a very weak form of what has come to be called Pontryagin'sminimum principle.

For the more general problem of N state and R decision variableswe require N multipliers, and the hamiltonian has the form

N

H = 5 + Xnfn(x1)x2, . . ,xv,ul,uz,' .n-l

,us) (12)

with canonical equationsaHx;=a ;°f (13a)

OH

axX.

afnax; (13b)

niland the boundary conditions

X. = 0 x; free (14a)a; unspecified x; fixed (14b)

t This is the miming stopping condition which we noted previously.We shall find later that the hamiltonian is in fact a minimum at these stationary

points.


The hamiltonian is made stationary by all decision variables uk, k = 1, 2,. . . , R, and is a constant, with zero value when 0 is unspecified.

4.4 EULER EQUATION

At.this point it may be useful to reconsider the problem of Sec. 3.2 fromthe more general result of the previous section. We seek the functionx(t) which will minimize the integral

3 =0

a(x,t,t) dt (1)

First, we must put the problem in the form of Eqs. (1) of Sec. 4.2.To do so, we observe that for a given (or optimal) initial condition, x(0),the function x(t) is uniquely determined by its derivative ±(t). Thus,t may be taken as the decision variable. ' Furthermore, the etplicitdependence of 9 on t may be removed by the simple guise of defining anew state variable. That is, if we let x1 denote x, the problem can bereformulated as

S =o

e%(xl,u,x2) dt

withi1 = U

x2 = 1 X2(0) = 0

(2)

(3a)(3b)

so that x2(t) m t. This is precisely the form which we have studied(with the special case of a single decision function).

The hamiltonian is

H=SF+X1u+X2with multiplier equations

aH aif aSaxl =-ax)

aH 05 05ax: axs = - at

Equation (5a) can be integrated to give

al(t) = x1(0) - fax

(4)

(5a)

(5b)

(6)

We generally do not need Eq. (5b), but we should note that it requiresthat X2 be constant if 5 is independent of t.

The condition that the hamiltonian be stationary is

aU=au+a1= 01= 015 +X) (7)


or, combining with Eq. (6),

as asTX = Jo ax

dt + c (8)

where c denotes a constant. This is the equation of du Bois-Reymond.If we differentiate with respect to t, we obtain Euler's equation

d as asdtat -ax (9)

The necessary boundary conditions follow from Eq. (13) of Sec. 4.2 as

x(0) given or ax1 1-0

= 0 (10a)

05x(B) given

1

or = 0 (10b)az ,1-e

Equations (9) and (10) are identical to Eqs. (18) and (19) of Sec. 3.2.It may be that the optimal function x(t) has a "corner" at some

value oft where the derivative is not continuous. The integrand 9(x,z,t)will not be continuous at this point, and the Euler equation (9) will notbe defined. The integral in Eq. (8) will be continuous, however (recallthat the value of an integral is not changed by what happens at a singlepoint), and therefore aS/az must also be continuous at the corner. Thisis the Erdmann-Weierstrass corner condition.

4.5 RELATION TO CLASSICAL MECHANICS

The reader familiar with classical mechanics may wish to relate thehamiltonian defined here with the function of the same name employedin mechanics. It will be recalled that in a system of N particles, withmasses ml, m2i . . . , mN and positions in a three-dimensional space x11,x12, x13, x21, x22, x23, . . . , xNl, xN2, xN3, the physical system is the onewhich minimizes (or makes stationary) the action integral

N 3

(1)L ,fo [ z L-.----V(x11,x12)x1`3, xN1,xN2,xN3..11s-li-1

where the first term is kinetic and the second potential energy. Renum-bering, we may use a single-subscript notation

3N

S = foM2 ,_+,2

- V (x1jx2, . . . ,x3N)J

dtIi-1

(2)

with m1 = m2 = m3, m4 = m5 = m6, etc.

no OPTIMIZATION BY VARIATIONAL METHODS

and

i-Ithen the hamiltonian is

3N 3N

H =Im2

- V (x,,x2, . . . ,xaN) 1- atu;

The multiplier equations are

aH aV' -axi ax;

while the stationary condition is

aH =0=ma-us +X;au,

or

If we take the decision variable as the velocity,

x;=u; i=1,2, ... ,3N

3N

. _ 1 m2 2- V(xl,x2, . . ,x3N)

X;

m;

Defining the momentum p; as - Xi, we find

(3)

(4)

(5)

(6)

(7a)

(7b)

(8)

and

i-i

(9)

which is simply the negative of the usual hamiltonian of mechanics.(The fact that we have obtained the negative stems from an unimpor-tant sign convention.) This is the total energy, which is a constant ina conservative system.

4.6 SOME PHYSICAL EQUATIONS AND USEFUL TRANSFORMATIONS

In this and subsequent chapters we shall often find it useful to relate themathematical development to the particular example of the control of acontinuous-flow stirred-tank chemical reactor. This example is chosenbecause, while simple, it retains the basic features of many practical sys-tems and because the basic equations have received considerable atten-tion in the published literature. In this section we shall develop the

3N

H = pit -- V (x1,x2, . . ,x3N)2m;


equations and consider several, transformations of variables, each of whichwill be useful in various formulations of the control problem. One of ourpurposes is to derive, through the consideration of the reactor example,the general form of a second-order system, and the reader who is notinterested in the particular physical application may wish to begin withEqs. (10).

The reactor system is shown schematically in Fig. 4.1. For sim-plicity we assume a single liquid-phase first-order reaction

A --+ products

The equation of conservation of mass is then

A=V(A, - A) - kA

(1)

(2)

where A denotes the concentration of reactant, the subscript f refers tothe feed stream, V the volume, and q the volumetric flow rate of the feed.k is a temperature-dependent reaction-rate coefficient of Arrhenius form

k = koexp( Tl! (3)

where ko and E' are constants and T is temperature. The equationsimply states that the net rate of accumulation of A in the reactor isequal to the net rate at which it enters less the rate of reaction

Similarly, an energy balance leads to

fi = 4 (T, - T) - UKq` (T - T,) -b (-AH)kA (4)

V VCpp(1 + Kq,) C,p

Here the subscript c refers to the coolant stream, p is the density, Cp the

Flow rote

Feed streamnX

91.

9

Coolant flow rate

IQ,

CoolantO T

Product streamA, T

Fig. 4.1 Schematic of a continuous-flow stirred-tank reactor.


specific heat, U the overall heat-transfer coefficient times the cooling area,AH the heat of reaction, and K a constant, defined as

K = 2CPCPCU (5)

Equations (2) and (4) are made dimensionless by defining a dimen-sionless concentration Z1 and temperatures Z2, Zr, and Z,, as follows:

Z1

Z2

A 1Af

CPpT(-AH)A,

C T

(6a)

(6b)

(6c)

(6d)

1PpZ' _ (-AH)A,CPpTZ (-AH)A1

Thus

Z1 = V (1 - Z1) - kZ1

Z2V

(ZI - Z2)VCPp(K++

(Z2 - Z,) + kZ1

(7a)

(7b)

where k may be written in terms of Eqs. (6) as

k = ko exp - E'CPp 1 (8)(-AH)A, Z2

Let us now assume that the reactor has been designed to operateat a stable steady-state condition Z18, Z25 and that we shall control thereactor in the neighborhood of the steady state by adjusting the flowrates q and q,. Let x1 and x2 denote variations about the steady statein (dimensionless) concentration and temperature, respectively, and uland u2 in feed and coolant flow rates; that is,

x1 = Z1 - Zis X2 = Z2 - Z2s (9a)

ul = q - qs u2 = qc - qcs (9b)

where the subscript S refers to the steady state. Substituting into Eqs.(7) and expanding the right-hand.sides in Taylor series about the steadystate, exactly as in Sec. 1.4, we obtain the following equations for thedynamic behavior of the reactor in a neighborhood of the steady statex1=x2=u1=u2=0:

x1 = a11x1 + a12x2 + b11u1 (10a)z2 = a21x1 + a22x2 + b21U1 + b22U2 (10b)


where the constants are defined as follows:

113

ail = V + ks (lla)

a12

E'CppksZ,s(11b)AH A Z(- ) ! zsz

a21 = -ks (11c)

= qs + UKq,s E'C,pksZlsazz (lld)VC,p(1 + Kq,s) (-AH)A,Z2s2

b11 = V (1 - Z15) (lle)

b21 = V (Z1 - Z25) (111)

b22UK(Z2s - Zc) (llg)= - VC,p(l +

It is clear that Eqs. (10) will apply to a large-lass of systems besides thereactor problem considered here. Values of the parameters which areused for subsequent numerical calculations are collected in Table 4.1 inconsistent (cgs) units.

In some cases we shall choose to hold ui at zero and control onlywith the coolant flow rate. We can simplify the form of the equations bydefining new variables yl and Y2, as follows:

x1 = -ylall _ 1

xz = Y1 Y2ail a12

Substituting into Eqs. (10), we then obtain

yl = y2

y2 = (a12a21 - alia22)yi + (a11 + a22)y2 T (-a12b22u2)

or, with obvious notation,

y1 = y292 = - a2y1 - a1y2 + U

(13a)(13b)

(14a)

(14b)

Table 4.1 Parameters for continuous-flow stirred-tank reactor

V = 1,000 Af - 0.0065 T, = 350T.-340 k0 -7.86X1012 E'-14,000

(--MI) - 27,000 p - 1.0 C, - 1.0U-10 K-0.2 qs-10

As=15.31 X10-1 Ts-460.91 q,s=5


Equations (14) are equivalent to the equation of motion of a solid bodywith viscous drag and a linear restoring force, the general second-ordersystem

y + a,y + azy = u(t) (15)

Hence, Eqs. (15) and (10) are completely equivalent when b11 = 621 = 0.Equations (10) and (15) are also related in another way for this

case. We may wish an equation for concentration alone if the tempera-ture is of little importance to us. Differentiating Eq. (10a), we obtain(with b,1 = b2l = 0)

Zi = a11±1 -4- a,zx2 (16)

and substituting for t2 from Eq. (10b) and x2 from Eq. '(10a),

xl + alxl + azxl = -U(t) (17)

where al, a2, and u have the same meanings as above.Another transformation which we shall find convenient is

xl = Y1 + yz (18a)

X2 all + S1 all + S2yz (18b)=

a12 a12

Here,

231 = -(all + a22) + [(all + a22)2 - 4(alla22 - alzazl)]; (19a)2S2 = -(all + a22) - [(all + a22) 2 - 4(alla22 -. al2a2i)j/ (19b)

For the parameters in Table 4.1, S1 and S2 are negative and real. Equa-tions (10) then become

yl = S1Y1 - M11u1 - M12u2 (20a)y2 = 527,2 + M21u1 + M12u2 (20b)

where

alibi, + alzbzl + S2b11 (21a)S,-StM1, = alzb22

(21b)Sl - SZ

M21 = M11 + bl1 (21c)

4.7 LINEAR FEEDBACK CONTROL

We are now in a position to consider the control of the chemical reactor.We shall suppose that the linearized equations are adequate and that thereactor is to be controlled by adjustment of the coolant flow rate. Equa-


tions (10) of the preceding section are then

z, = a12x2 (la)x2 = a21x, + a22x2 + b22u2 (ib)

where x, and x2 are deviations in concentration and temperature, respec-tively, and u2 the variation in coolant flow rate. The system is initiallydisplaced from the equilibrium values, and our control action will bedesigned to keep fluctuations in all variables small. This objective maybe accomplished by minimizing the integral

g2 f o# + C2x22 + c:u:2) dt (2)

We shall make the transformation

xi = -y,all 1

X2 = - y, - - Y2a,2 a,2

giving

y, = Y202 = - a2y, - a,y2 + u

and the objective

g2

fo (ci,y,2 + 2c,2y,y2 + c22y22 + c::u=) dt

where

a, = - (a + a22)a2 = a,2a2,u = -a,2b22u2

C

C,2

C22

C21

C1 + C2a,22

all-Cl 2a,2

_ C2

a,22

Cs

a,22b2:2

The hamiltonian is then

(3a)

(3b)

(4a)

(4b)

(5)

(6a)

(6b)

(6c)

(6d)

(6e)

(6f)

(69)

H = 2c,2y,y2 + c22y22 + c22u2)

+ X,y2 + X2(- a2y, - a,y2 + u) (7)


and the canonical equations for the multipliers

aH ay= -C11Y1 - C12y2 + a2X21

),2aH ay= -c12y1 - C22y2 - Xi + a1X2

e


X1(9) = X2(9) = 0

The optimality criterion, that the hamiltonian be stationary, is

aH=0=X2+C33Uau

or

X2U= --C33

(8a)

(8b)

Z9)

(10)

Thus, the problem is one of finding the function-X2(t).The optimal control for this system can be found in a rather straight-

forward manner by seeking a solution of the form

X1 = m11y1 + mi2y2 (12a)A2 = mi2yi + m22y2 (12b)

From Eqs. (8), then,

X1 = -Clly, - Clay! + az(muuyi + m22y2)X2 = -C12YL - C22Y2 - (miiy1, + m12y2) + ai(mi2yi + m22y2)

By differentiating Eqs. (12) with respect to time we also obtain

(13a)

(13b)

Xi = miiyi + m1iyi +, m12y2 + m12 r2 (14a)

X2 = m12y1 + mizyl + m22y2 + m22y2 (14b)

and, substituting Eqs. (4),

x1 = m11y1 + mlly¢ + m12Y2

+ m.14 (-a2y1 - aly2 - M12 y1 - T22my2 (15a)

X2 = m12y1 + m12y2 + m22y2/+ m22 l -aiy1 - a1y2 -

m12Cb3 yl -

m22Cs' yz (1 5b)

If a solution of the form of Eq. (12) is to exist, Eqs. (13),and (15) mustbe identical for all yl and y2. That is, the coefficients of yl in Eqs. (13a)and (15a) must be identical, as they must be in Eqs. (13b) and (15b),and similarly for y2. Thus, equating coefficients, we obtain the three


differential equations

m11

*22m12

= m122 -I- 2a2C3311112 - C33C11

m222 - 2c33m12 + 2a1c3391122 - Ca3C22

1n121n22 - C33nL11 + a1C33m12 + a2C33m22 - C3012

(16a)

(16b)

(16c)

Equations (16) are the multidimensional generalization of theRiccati equation, which we have considered several times previously.For finite B a solution can be obtained numerically with boundary con-ditions m11(B) = m12(8) = m22(0) = 0, while as O--- ao, a stable constantsolution exists. We shall consider this case, and setting the time deriva-tives in Eqs. (16) to zero, we obtain

m12 = -c3sa2 + (c$32a22 + C32C11)S4 (17a)m22 = -C33a1 + [C332a12 + C33C22 - 2c332a2

+ c3s(C332a22 + (17b)

Combining Eqs. (11), (12), and (17), we obtain the optimal control

u= [a2 - (a22 +Cc-3. l

+ {ai - a12 - 2a2 - Ci: as + Cu,i] y (18)

This is a multivariable linear feedback controller, whlsre the controllergains depend on both the system parameters and relative weights. Theresulting system is stable, so that yl and y2 go to zero as 0 -> ao, andhence Eq. (12) for X1(8) add 7 2(9) satisfies the zero boundary conditionsof Eq. (9).

The system equations (4a) and (4b), together with the control (18),are linear, and are easily solved and then transformed back to the Z1,Z2 variables defined in the previous section. Figure 4.2 shows the pathsin the Z.1Z2 plane for the linearized reactor equations with the param-eters in Table 4.1, using constants

c1=84.5 c2=6.16 C3=10-' (19)

It is interesting to note that from most starting points the equilibriumpoint is approached along a line in the plane. A weakness of the linear-ized analyses becomes evident by noting that some of the paths cross intonegative concentrations, which are physically meaningless.

4.$ AN APPROXIMATE SOLUTION

The technique used in the previous section to obtain solutions of themultiplier equations is a powerful one when the objective is of the form ofEq. (5), i.e., a quadratic form in the state variables plus a squared cost-

in OPTIMIZATION BY VARIATIONAL METHODS

3.00

2.90

2.80

2.60

2.50

2.40

2.30

2.200

I I I I i 1 1

0.04 0.08 0.12 0.16

Composition z,

Fig. 42 Temperature-concentration paths for the controlledreactor. [From J. M. Douglas and M. M. Denn, Ind. Eng.Chem., 67 (11): 18 (1965). Copyright 1965 by the AmericanChemical Society. Reprinted by permission of the copyrightowner.]

of-control term, but it is restricted to linear systems. It is possible, how-ever, to obtain approximate solutions to nonlinear systems with the sameobjective by similar methods. In order to avoid algebraic complexitywe shall simplify our reactor example somewhat for the time being, butwe shall see later that the result is still of physical as well as mathematicalinterest.

We shall assume for the reactor model described by Eqs. (6) and (7)of Sec. 4.6 that it is possible to choose the temperature Z2 at every instantof time, and hence temperature or, equivalently, the reaction rate k[Sec. 4.6, Eq. (9)] is the control variable. The state of the system isdescribed, then, by the single equation

Z1 =V

(1 - Z1) - U,

and we shall suppose that the objective is again onetions small, that is,

(1)

of keeping flue tua-

3 = 3 (Z1 - Z1a)1 + 3c2(k - ks) 2 (2)

CONTINUOUS SYSTEMS: f U9

It is convenient to define variables

x=Z1-Zis u=k - ks (3)

and so the system equation becomes, after subtracting out the steady-state terms,

i = - (3+ ks) x - (Zis + x)u (4)

= 12 (x2 + c2u2) (5)

Note that we have not linearized. The linearized form of Eq. (4) wouldnot contain the term xu.

The hamiltonian for this nonlinear system is

H = iix2+12c2u2-x(3+ksl x-a(Z15+x)u (6)

and the equation for the multiplier

-aH= -x+lV+ks, X+Xu (7)

The condition that the hamiltonian be stationary isaH

2

au =cu-X(Zis+x)=0 (8)

or

U = c (Zis + x) (9)

so that the problem is again one of finding the solution to the multiplierequation.

For the linear system of the previous section we were able to obtaina solution for X proportional to x. We might look upon this as the leadterm in a series expansion and seek a solution of the form

X = mx + px2 .+. . . . (10)

Again, for simplicity, we shall take 0 ---> so that it will be possible toobtain the solution with ni, p, . . . as constants.

Differentiating Eq. (10), we `find, using Eqs. (4) and (9),

- (m + 2px){(# + ks) x + (Zis c2 x) 2 (mx + px2) ] + .. .

1 (l la)while from Eqs. (7), (9), and (10),

\1s_ -x + V + ks l (mx }px2) + m2x2Z>.

c2+ ... (11b)

The coefficients of each power of x must be equal in these two equations,


so that we obtain from the power x'm2Z1s2 + 2 l 1 0+ k s =m \V

and from the power` x2

(12)

+ mZ1s2 + m2Z1s = 0(q + k 13)sP `` V c2 c2

( )

Equation (12) is easily shown to be the steady-state limit of the Riccatiequation (10) of Sec. 3.4 for the linearized system when the properidentification of coefficients is made, the stable solution being

2 2 ,_

M --Z1s2 I V + k,, - (1 ks) + ZC2 ] (14)

while the solution to Eq. (13) is

m2Z1sPW c2(q/V + ks) + mZ1s2 (15)

Coefficients of x2, x4, etc., in the series for X can be obtained in the samemanner. In terms of the original variables, then, we obtain the optimalnonlinear feedback control

k = ks + Z' [m(Zi - Z15) + p(Z1 - Z1s)2 + .. ] (16)

Had we not let B -- , we would have obtained differential equations form, P, ....

Equation (16) defines a curve in the Z1Z2 plane. If we assumethat we have reached that curve in some way, we can substitute Eq. (16)into Eqs. (7) of Sec. 4.6 to obtain the optimal coolant flow rate qc toimplement this policy, namely, the solution of

q. _ (-1H)VA,Z22 q (1 - Z1) - kZ1 (2Z1 - ZIS)1 + Kq, k UKE'c2 I V

[m -f- P(Z1 - Z1s) + ] + UK (Zf Z2) + kV C P 21 (17)

Hence, had we posed the problem of finding the coolant flow rate tominimize the integralt

_ f [(Z1 - Z1S)2 + c2(k - ks)2] dt

we would have the partial solution that once having reached the linedefined by Eq. (16), the optimal control policy is the nonlinear feedback

t There is no obvious physical reason why k should be any less meaningful in theintegral than Z1, since it is the reaction rate, rather than the temperature, whichaffects conversion.


controller defined by Eq. (17). As we shall see in the next chapter, theoverall policy is the one which we intuitively expect, namely, full coolingor no cooling until the line defined by Eq. (16) is reached and then, non-linear control as defined above.

4.9 CONTROL WITH CONTINUOUS DISTURBANCES

The control problems we have considered thus far have dealt with autono-mous systems only; i.e., the differential equations are assumed to haveno explicit dependence on time. This will be the case, for example, whenthe operating conditions have been changed and the system must bebrought to a new steady state (a set-point change) or when a large pulse-like disturbance has affected the system but no more disturbances areexpected for a time long compared to the system's response time. Inmany control situations, however, disturbances of prolonged durationmay be expected to enter the system and must be included in a rationalcontrol analysis.

For simplicity we shall restrict our attention to linear systems withconstant properties in which the state of the system can be described bya single variable. If the disturbance is a piecewise differentiable func-tion D(t), the system response is described by the differential equation

x = Ax + u + D(t) x(0) = xo (1)

We shall again take the control objective to be the minimization of theintegral

& = 2 Io (x2 + c2u2) dt (2)

If we follow the procedure of Sec. 4.4 and remove the explicit timedependence by definition of a new variable, we can write Eqs. (1) and (2)in autonomous form as

a1 = Ax, + D(x2) + u x1(0)" = xo (3a)x2 = 1 x2(0) = 0 (3b)

2

Ia(x12 + c2u2) dt (4)

The hamiltonian for this system is

H = 3'(x12 + c2u2) + X1(Ax1 + D + u) + X2 (5)

with multiplier equations

1,1=-aH=-xi-AX1 X1(d)=0 (6a)

aH _ `X1dD = -X11 (6b)


where X2(B) will be zero if 0 is unspecified, but unspecified for 0 fixed.The condition of optimality is

aH-5-U

=c2u+X,=0(7a)

or

(7b)

We see, therefore, that x2 and X2 are in fact extraneous, and we maydrop the subscripts on x, and X,.

Equations (3a) and (6a) form a system

z = Ax - + D(t) x(O) = xo (8a)

x = -x - AX X(0) = 0 (8b)

Since, we already know that a homogeneous solution (D = 0) for thissystem can be obtained in the form X proportional to x, we shall seek asolution

X = -c2[M(t)x + L(t)] (9)

From Eq. (8a) we then have

X = -c2M(Ax + D + Mx + L) - c2Mx - c2L (10a)

while from Eq. (8b)

X = -x + Ac2Mx + Ac2L (10b)

Equating coefficients of x in Eqs. (10), we obtain an equation for M(t)

M + 2AM + M2 - c = 0 (Ila)

and therefore L(t) must satisfy

L + [A + M(t)]L + M(t)D(t) = 0 (llb)

Equation (Ila) is, of course, simply the Riccati equation of Sec. 3.4.In order to establish boundary conditions for Eqs. (Ila) and (llb) weshall assume that at some time t < 0 the disturbance vanishes andremains identically zero. During this final period we have the problemwe have already solved, namely, a system offset from x = 0 with nodisturbances, and we know that the solution requires

M(9) = 0 or x(0) = 0 (12a)

The solution for M(t) is thus given by Eqs. (13) and "(14) of Sec. 3.4.It follows then, from Eqs. (8b) and (9), that the proper boundary con-


dition for Eq. (11b) is

L(O) = 0 (12b)--

This boundary condition clearly points up the difficulty we face in deal-ing with nonautonomous systems. While the formal mathematics isstraightforward, we must know the future of the disturbance functionD(t) in order to obtain a solution to Eq. (llb) which satisfies the bound-ary condition [Eq. (12b)] at t = B.

In practice this difficulty, while serious, may be somewhat lesssevere on occasion than it first appears. If we let 0 -- co, the solutionof the Riccati equation (Ila) is

M = -A - 1 le' (13)

and Eq. (lib) is

L` - J1 C2 L = (A + Ji c29 D(t) L(w) =-0 .(14)

which has the solution}

L(t)/

_ - t A + cA2c= f~exp [ - 1

c2c2 (r - t)]

\ D(r) dr (15)

For bounded disturbances the integrand will effectively vanish for futuretimes greater than several time constants, c(1 + A2c2)-, and it is neces-sary only to know (or estimate) the disturbance for that time into thefuture. Indeed, disturbances often have the form of step changes,

D(t) = D = const 0 < t < nc(.1 + A2c2)- (16)

in which case Eq. (15) yields

L(t) _ - l 1 + 1 +AZC= (1 - e-")D (17a)

and, for n greater than 2 or 3,

A 2C2L - 1+ 1+A2c2)D (17b)

That is, the optimal control for a step disturbance is proportional toboth the system state and the disturbance

u(t) - CA + c2A2C2) x(t) - (l+ +2A2c2 D (18)

The term c2u2 in the objective defined by Eq. (2) is meaningless for


many industrial situations, in which a true cost of control is negligible.Such a term might be useful as a penalty function in order to keep thecontrol effort u(t) between bounds, but an immediate disadvantage ofsuet' a practice is evident from the substitution of Eq. (18) into Eq. (1).The optimal response to a step-function disturbance is then found to be

x(t)1+cDc2+(xo+1+A2c2)expI 1+Arest) (19)

or

x(t) _ AclD (20)1 + A 2c2

That is, the optimal control with a cost-of-control term does not returnthe system to x = 0 as long as there is a step disturbance present.This is known as steady-state offset, and is clearly undesirable in manycircumstances.

4.10 - PROPORTIONAL PLUS RESET CONTROL

Very often the serious restriction is not the available control effort butthe maximum rate at which the control setting may be changed. Insuch cases the problem of steady-state offset for step disturbances can beresolved, and the result poiryts up an interesting connection betweentraditional control practice and optimal-control theory.

We shall assume that tale system is at equilibrium for t < 0, witha step disturbance of magnitude D entering at t = 0. We thus have

z=Ax+D+u x(0)=0with the objective

(1)

i; = 2 Jo (x2 + c242) dt (2)

where the term c2u2 is intended as a penalty function to keep the rateof change of control action within bounds. Because D is a Constant, wecan differentiate Eq. (1) once to obtain

z = Ax + it x(0) = 0 z(0) = D (3)

or defining xl=x,x2=z,w=u,21 = x2 x1(0) = 0 (4a)x2 = Arxe + w x2(0) = D (4b)

Fi = 20

(x12 + c2w2) dt (5)

The problem defined by Eqs. (4) and '(5) is precisely the one con--sidered in Sec. 4.7. If we let 0 - ao, then, by relating coefficients. the


optimal solution is obtained from Eq. (18) of Sec. 4.7 as

w(t) = ic(t) = - c x, - (A + X2 (6)

Integrating, the control action is found to be

u(t) = - CA + c) x(t) ^ c fo x(r) dT (7)

That is, the optimal control is proportional to both the offset and theintegral of the offset. The integral mode is often referred to as reset.Most industrial controllers employ proportional and reset modes.

The importance of the reset mode can be clearly seen by substitut-ing the control action into Eq. (3). The system response is determinedby the equation

2+!i+-1X=0 x(0)=0 x(0)=D (8)

which has the solution

2cD e-1/2c sinh t4c

t e < 3/x(t) ,/1 -4 c (9)

-D1 e-111C sin

1/4c __1t c>

2c

That is, the response is overdamped and without oscillation when c < j,underdamped with oscillations for c > Y4, but. always decaying exponen-tially to zero after the initial rise. Thus, there can be no steady-stateoffset.

4.11 OPTIMAL-YIELD PROBLEMS

In many processes the quantity of interest will' not be ,a cumulativemeasure of profit or loss in the form

g = ff(x1,x2,ul,u2) dt (1)

but simply the difference between initial and final values of x, and x2.A particular case would be a chemical-reaction system, where we mightseek to maximize some weighted sum of the conversions, with the profit,say 61, expressed as

19 = c[xi(8) - x1(0)) + [x:(#) - x:(0)l (2)

This is equivalent, however, to writing

6' = fo (cxI + 11) dt =0 (cf, + f2) dt (3)


which is in the form of Eq. (1) if we obtain a minimization problemby letting S = -(P. Hence, we wish to consider the general problem oftwo nonlinear state equations, with 5 defined as

S= -Ch -f= (4)

For algebraic simplicity we shall assume a single decision function u(t).The canonical equations are

21 = f1(xx,x=,u) (5a)

x= = f2(x1,x=,u) (5b)

1= -(X1-c)afl0x1 (5c)

X=-= -(X1-c)ax=-(A2-1)of= (5d)

with boundary conditions X1(9) _ X2(9) = 0 and the hamiltonian

H = fi(A1 - c) + f2(X= - 1)

It is convenient to define new variables 4,1, A2 such that

We then have

f1 C1xjo

aft af=

ax=- tG= ax=

H=#Lf1+ihf=

and the condition of optimality,

aH= 01

af1 + 4,:af= = o

au a a

(6)

(7a)(7b)(7c)

(7d)

(8a)

(8)

(9)

(10)

For this class of problems it is possible to reduce the number ofvariables which need to be considered. Equation (10) is true for alltime, and therefore its derivative with respect to t must vanish. Hence,

jf ( = =h a=f1

d a = IYL au + 1 \8u 0x111 + au ax=f= + ax=_

+ d= au (au ax,f1 + au ax=f= + au= u = 0 (11)


or, substituting Eqs. (8),

aft afl aft afl a2fl 49 2f, 02A4,1 au axt au aX2 + a26 axl fl + au ax2 f2

+ aY42

afl 0,12 -3f2 0_f2 a2f2 a2fl a2f2

+ 412 (- au 5_4 - au axe + au axl f l + au ax2 f 2 + aut is= 0

(12)

Equations (10) and (12) are linear homogeneous algebraic equations for4,t and 4#t, and the condition that they have a nontrivial solution (Jj and1r2 both not identically zero) is the vanishing of the determinant ofcoefficients. Thus

Of, Of, aft aft 012 a2f2 a2ft 02f2T (- au 5Z a x , a x , 49u,

aft 0fl Of, aft Of, 02ft 02fl a2fl-au(rauaxt-au0xs+auaxtft+auaxef2+autu> =0(13)

or, solving for u,

aft 02f, a2ft afl afl aft Oflau'au axt fl + au axe f 2 - U ax au x2)

is =

Of,

(_O'ft02f2 aft aft aft aft1

au au axlfl + au ax2f2 au axl au axtOf, 02f2 aft atfl (14)

auau2 -auau2

Equation (14), which has been called a generalized Euler equation,is an ordinary differential equation for the optimal decision function u(t),which must be solved together with Eqs. (5a) and (5b) for xt and X2-A boundary condition is still required, and it is obtained by evaluatingEq. (10) at t = 0 with the values of ¢l, 02 obtained from Eqs. (7c) and(7d) :

cafutft+a2=0 att=8 (15)

The problem can then be solved numerically by searching over values ofu(0) and solving the three differential equations until the stopping con-dition, Eq. (15), is satisfied. Equivalently, for each, initial value u(0)there is a process running time for which u(O) is optimal, and this cor-respondence is found when Eq. (15) is satisfied.

The reader will recognize that this section parallels Sec. 1.11 fordiscrete systems. In each case it has been possible to eliminate themultipliers by use of the optimality conditions. It should be clear that


the algebraic conditions needed to pass from Eqs. (10) and (12) to (13)require that the number of state variables exceed the number of decisionvariables by no more than 1, so that there will be at least as many Itb'mo-geneous equations as multipliers.


We now make use of the results of the previous section to consider the,continuous analog of the problem of Sec. 1.12. A system of consecutivereactions is assumed to take place,

X1-> X2 -* decomposition products

and the reaction is to be carried out in a batch reactor. X2 is the desiredproduct, and we seek to maximize the increase in value of the contentsof the reactor after an operating period 0 by adjusting the reactor temper-ature u(t) in time.

The equations describing the course of the reaction are

±1 = -k1(u)F(x1) (la)

(that is, the rate of reaction of X, depends only on temperature and con-centration of X1)

x2 = v'kl(u)F(Z1) - k2(u)G(x2) (lb)

(The rate of change of concentration x2 is the difference between therate of formation from Xl and the rate of decomposition. The latter ratedepends only on temperature and concentration of X2.) The coefficientv is introduced to account for changes in reaction stoichiometry (thenumber of molecules of Xl needed to form a molecule of X2). It shouldbe noted that the equations describing the 'course of this reaction in apipeline reactor in which diffusion is negligible are identical if t is inter-preted as residence time, or length into the reactor divided by fluidvelocity. Hence, we may look upon this problem as that of determiningthe best that could be accomplished in a pipeline reactor if it were possibleto specify the temperature at every point, and we shall generally referto the.function u(t) as the optimal temperature profile. In the latter casethe true reactor design problem would be that, of approaching the upperbound represented by the optimum profit with a practical heat-exchangesystem, since in a real ,reactor the temperature cannot be. specified atevery point in space.

The increase in value of the product has precisely the form ofEq. (2) of the preceding section, where c represents the value of the feedXl relative to the desired product X2. Clearly c < 1. Equation (14)


of Sec. 4.11 for the optimal temperature is

in

vF(xl)G'(x2)[kl(u)ks(u) - k'(u)k2(u)}k'(u) (2)u

G(x2)[ki (u)k2(u) - ki(u)kz (u))

where the prime denotes differentiation with respect to the argument, andthe stopping condition, Eq. (15), is

ki(u)F(x,)(v - c) - k'(u)G(x2) = 0 t = 0

More specifically, if the functions k, and k2 are of Arrhenius form

(3)

k; = k;o exp - u` i = 1, 2

where k,o, k2o, E;, and E2 are constants, then

vF(x,)G'(x2) I E'1u -k

(4)

5)io exp u JE2'G(x2)

and the final condition on the temperature is

(

uE2-E,' t=B 6E2kzoG(x2) 1In

( )

- c)E;kioF(xi) J1(y

Equation (5) requires that the optimal temperature always decreasein time or with reactor length. When E2 > Ei, a high temperaturefavors the second reaction with respect to the first, and a decreasingtemperature makes sound physical sense, for it suggests a high tempera-ture initially to encourage the reaction X, -- X2 when there is little X2to react and then a low temperature in the latter stages in order to preventthe decomposition of the valuable product X2. On the other hand, ifE' > Ez, a decreasing temperature profile contradicts the physical intui-tion that since the reaction X, --> X2 is favored with respect to thedecomposition of X2, the highest possible temperature is optimal at alltimes. In Chap. 6 we shall develop a condition analogous to the second-derivative test of Sec. 1.3 which verifies this physical reasoning anddemonstrates that Eq. (5) defines an optimum only when E2 > E.

The procedure for obtaining the optimal temperature profile andoptimal profit is as described at the end of Sec. 4.11. The feed composi-tions x,(0) and x2(0) are presumed known, and a value is assumed foru(0). Equations (1) and (5) are then integrated simultaneously untilEq. (6) is satisfied, and u(0) is varied and the procedure repeated untilthe solution of Eq. (6) occurs at t == 0. Amundson and Bilous havecarried out such solutions for several cases.


4.13 OPTIMAL CONVERSION IN A PRESSURE-CONTROLLED REACTION

As a further example of the use of the generalized Puler equation and forpurposes of reference in our later discussion of computation we shall con-sider a second type of optimal-yield problem, the maximization of inter-mediate conversion in a consecutive-reaction sequence carried out in apipeline reactor where the reaction rate is dependent not upon tempera-ture but upon pressure. The reaction is

X, -> 2X2 -+ decomposition products

where the first reaction is first-order and the second second-order. Con-centrations are denoted by lowercase letters and total pressure by u.

Assuming ideal gases and Dalton's law of additive partial pressures,the state equations may be written

x, _ -2k,u A + x2xj(0) = xio (la)

z

x2 = 4k,u A +x2

- 4k2u2 (A+x2)2

x2(0) = x20 (lb)

where k1 and k2 are positive constants and A = 2x,o + x20. To maxi-mize the conversion of intermediate we have.

(P = x2(0) (2)

in which case the parameter c defined in Sec. 4.11 is zero. Performingthe required differentiations, Eq. (14) of Sec. 4.11 for the optimal pres-sure is then found to be

4uz k2ux2 a+Ax

[k(3), (A + x2)x2(A + x2)2 l

with the boundary condition from Eq. (15) of Sec. 4.11,

k1x,(A + x2)U = at t = 0 (4)2k2x2s

Equation (3) indicates that the optimal pressure decreases withreactor length, and if x20 is small, very steep gradients may be requirednear t = 0. It is, of course, impossible to specify the pressure at 'eachpoint in a pipeline reactor, so that the optimal conversion calculated bythe solution of Eqs. (1), (3), and (4) provides an upper bound for evalu-ating the results of a practical reactor design.


Sections 4.2 and 4.3: The derivation follows

J. M. Douglas and M. M. Denn: Ind. Eng. Chem., 57(11): 18 (1965)


The results obtained here are a special case of much more general ones derived in subse-quent chapters, and a complete list of references will be included later. A funda-mental source for Chaps. 4 to 8 is

L. S. Pontryagin, V. G. Boltyanskii, R. V. Gamkrelidze, and E. F. Mishchenko:"Mathematical Theory of Optimal Processes," John Wiley & Sons, Inc., NewYork, 1962

Section 4.4: Any of the texts on the calculus of variations noted in the bibliographicalnotes for Chap. 3 will contain a discussion of the corner condition.

Section 4.6: This section is based on

L. I. Rozenoer: in Automation and Remote Control, I, Proc.1st IFAC Congr., Moscow,1980, Butterworth & Co. (Publishers), Ltd., London, 1961

The principal of least action is discussed in books on classical mechanics, such as

H. Goldstein: "Classical Mechanics," Addison-Wesley Publishing Company, Inc.,Reading, Mass., 1950

L. D. Landau and E. M. Lifshitz: "Mechanics," Addison-Wesley Publishing Company,Inc., Reading, Mass., 1960

Section 4.6: The model of a stirred-tank reactor and an analysis of its transient behaviorare contained in

R. Aris: "Introduction to the Analysis of Chemical Reactors," Prentice-Hall Inc.,Englewood Cliffs, N.J., 1965

This is also an excellent source of details on other reactor models used as examples through-out this book.

Section 4.7: This section follows the paper by Douglas and Denn cited above. The basicwork i

R. E. Kalman: Bol. Soc. Mat. Mex., 5:102 (1960)

A more general discussion is included in Chap. 8, and an extensive survey of optimal

linear control is contained in

M. Athans and P. Faib: "Optimal Control," McGraw-Hill Book Company, NewYork, 1966

The reader unfamiliar with the conventional approach to process control may wish-toconsult a text such as


'D. D. Perlmutter: "Chemical Process Control," John Wiley & Sons, Inc., New York,1965

J. Truxal: "Automatic Feedback Control System Synthesis," McGraw-Hill BookCompany, New York, 1955

Section 4.8: The expansion technique for obtaining nonlinear feedback controls is dueto Merriam; see


C. W. Merriam: "Optimization Theory and the Design of Feedback Control Systems,"McGraw-Hill Book Company, New York, 1964

A. R. M. Noton: "Introduction to Variational Methods in Control Engineering,"Pergamon Press, New York, 1965

Sections 4.9 and 4.10: The references by Kalman and Athans and Falb cited above arepertinent here also, and the discussion is expanded in Chap. 8. The consequencesof the use of 0 as the cost term in relating optimal-control theory to conventionalfeedback control practice is the subject of research being carried on in collaborationwith G. E. O'Connor; see

G. E. O'Connor: "Optimal Linear Control of Linear Systems: An Inverse Problem,"M. Ch. E. Thesis, Univ. of Delaware, Newark, Del., 1969

Sections 4.11 to 4.15: The generalized Euler equation was obtained in

M. M. Denn and R. Aris: Z. Angew. Math. Phys., 16:290 (1965)

Prior derivations specific to the optimal-temperature-profile problem are in

N. R. Amundson and O. Bilous: Chem. Eng. Sci., 6:81, 115 (1956)R. Aris: "The Optimal Design of Chemical Reactors," Academic Press, Inc., New

York (1961)F. Horn: Chem. Eng. Sci., 14:77 (1961)

Both the optimal temperature- and pressure-profile problems were studied in

E. S. Lee: AIChE J., 10:309 (1964)

PROBLEMS

4.1. The pressure-controlled chemical reaction A = 2B, carried out in a tubularreactor, is described by the equation for the concentration of A

z = -k,u x + k2u!4(xo - x)=

2xo - x (2xo - x)'

where x0 is the initial value of x, u is the pressure, and k, and k2 are constants. x(8)is to be minimized. Obtain an algebraic equation for the theoretical minimum valueof x(8) in terms of e, k,, k2, and x0. For comparison in ultimate design obtain theequation for the best yield under constant pressure. (The problem is due to Van de. Vusse and Voetter.)4.2. Batch binary distillation is described by the equations

z1--ui2 =

u[x2 - F(x2,u)]

x1

with initial conditions x,o, x2o. Here x, denotes the total moles remaining in the still,x2 the mole fraction of more volatile component in the still, u the product withdrawalrate, and F the overhead mole fraction of more volatile component, a known functionof x2 and u which depends upon the number of stages. The withdrawal rate is to befound so as to maximize the total output

emax 61 =

!o

u(t) dt


while maintaining a specified average puritye

o Fu dtF =

u dtIoB

Formulate the problem so that it can be solved by the methods of this chapter andobtain the complete set of equations describing the optimum. Describe a computa-tional procedure for efficient solution. (The problem is due to Converse and Gross.)4.3. Consider the linear system -

x=uX(0) = xa i(0) = yox(8) = 0 x(8) = 0

and the objective to be minimized,

min 8 =1

2a U2(t) dt

(a) Find the unconstrained function u which minimizes 8 for a fixed 8.(b) Examine the nature of the minimum 8 in part (a) as a function of 8 graphi-

cally. Comment on the sensitivity of the solution to changes in 8.(c) Find the unconstrained function u which minimizes 8 for unspecified. B.

Comment on the significance of the solution in terms of the results of part (b). (Theproblem is due to Gottlieb.)4.4. Solve the control problem of Sec. 3.4 by the methoc of Sec. 4.7.4.5. Consider the nonlinear system

i = f(x) + b(x)u

x(0) - xa x(8) - 0where f and b are continuous differentiable functions of x. Show that the optimumunconstrained function u which minimizes

8 = fo [9(x) + c'u=] dt

with c a constant and g a nonnegative continuous differentiable function of x, has thefeedback form

f(x) f(x) '+

[b(x)1'[g(x) + j61u ab(x) b(x) { x x'c'

where d is a constant depending on xa and 8. Suppose 8 is unspecified? Compare thesolution of Sec. 3.4. (The problem is due to Johnson.)4.6. Extend the analysis of Sec. 4.7 to the case

z, = a,zx, + biui, - a21X1 + a:,x,, + b,u

min 8 = 3'z[C,e,1(6) + C2e,'(8) + foe (9 + c,e;' + c,u') dt]

where

el =x, -x,` e2:x2 -x;x,` and x, are some desired values. Obtain equations for the gains in the optimalcontroller.


4.7. The kinetics of a one-delayed-neutron group reactor with temperature feedbackproportional to flux are

it = A (un - ant - pn) + yc

Here n is the neutron density, c the precursor concentration, u the reactivity, and theconstants A, y, a, and 0, respectively, the neutron generation time, the decay constant,the power coefficient of reactivity, and the fraction of neutrons given off but notemitted instantaneously. Initial conditions no and co are given, and it is desired tobring the neutron density from no to ono in time 8, with the further condition that n(8)be zero and the effort be a minimum,

1 eminc =2 fo u'dt

Obtain the equations needed for solution. (The problem is due to Rosztoczy, Sage,and Weaver.)4.8. Reformulate Prob. 4.7 to include the final constraints as penalty functions,

min c a 3j { C,[n(8) - ono)' + C2[c(8) - c']2 + J0 u2 dt}

(What is Obtain the equations needed for solution. Normalize the equationswith respect to ono and obtain an approximate solution with the approach of Sec. 4.8,utilizing the result of Prob. 4.6.4.9. Let I be an inventory, P production rate, and S sales rate. Then

1*-P-SAssuming quadratic marginal costs of manufacturing and holding inventories, theexcess cost of production for deviating from desired, values is

fa IC,[I(t) - !J' + C,[P(t) - PJ'[ dt

where 9 is fixed, I and P are the desired levels, and C1 and CP are constant costs.If the sales forecast S(t) is known, determine the optimal production schedule P(t),0 < t < 8. Would a feedback solution be helpful here? (The problem is due toHolt, Modigliani, Aluth, and Simon.)4.10. Let x denote the CO2 concentration in body tissue and u the pulmonary ventila-tion. An equation relating the two can be writtent

of + (a1 + azu)x + a3ux = a4 + a6u

where the a; are constants. Find an approximate feedback solution for the "control"u which regulates the CO2 level by minimizing

- 1 e& 2 fo (x' + c'u') de

It is commonly assumed that u - a + bx.

t The model equation is due to Grodins et al., J. Appl. Phys., 7:283 (1954).

5Continuous Systems: II

5.1 INTRODUCTION

We now generalize our discussion of systems described by ordinary differ-ential equations somewhat by relaxing the requirement that the optimaldecision functions be unconstrained. Complete generality must awaitthe next chapter, but most situations of interest will be included withinthescope of this one, in which we presume that the optimal decisionfunctions' may be bounded from above and below by constant values.Typical bounds would be the open and shut settings on valves, safetylimitations on allowable temperatures and pressures, or conditionsdescribing the onset of unfavorable reaction products.

We again assume that the state of the system is described by the Nordinary differential equations

.x ,-4 fi(x1/ir4) ,zN,7L1,262, ,26R)

i = 1, 2, N0 < t < 9 (1)

where we wish to choose the R functions ut(t), u4(t), ... , uR(t) in135

W

order to minimize an integral


e8[u1,212, . . . ,UR) = f0 if (xl,x2) . . . ,xN,ul,u2, . . . U R) dt (2)

We now assume, however, that the functions uk(t) are bounded

uk. < Uk(t) < u,' k = 1, 2, . . . , R (3)

where the bounds uk are constants. The absence of a lower boundsimply implies'that uk. -+ - oo,. while the absence of an upper boundimplies that u,' -- + 0o .

5.2 NECESSARY CONDITIONS

For simplicity of presentation we again restrict our attention to the specialcase N = R = 2, although the results will clearly be general. Thus, weconsider the state equations

Zl = f1(x1,x2,u1,u4) (la)x2 = f2(xl,x2,u1,u2) (lb)

with constraints

u1. < u1 < ui (2a)U2* < u2 < u2` (2b)

and objective

Fi[u1,u2) = f o' dt (3)

In Sec. 4.2 we derived an equation for the change in E which resultsfrom small changes in ul and u2. In doing so we were never required to'specify those changes, Sui(t) and bus(t). Thus, as long as we stipulatethat those variations be admissible--i.e., that they not be such that ul oru2 violates a constraint-then Eq. (18) of Sec. 4.2 remains valid, and wecan write

a = fa (au aul + O2 Su_1 dt + o(E) >_ o (4)

Here we have used the hamiltoni/an notation of Sec. 4.3

H = if + Xifl + X2f2 (5)

where the multipliers satisfy the canonical equationsaH aH

x1 = - axl x2 = - axe (6)

with boundary conditions from Eq. (13) or (16) of Sec. 4.2, dependingupon whether or not 0 is specified.

CONTINUOUS SYSTEMS: II 137

We shall presume that in some way we know the optimal functionsul(t), u2(t) for this constrained problem. Each may be only piecewisecontinuous, with segments along upper and lower bounds, as well aswithin the bounds, as, for example, in Fig. 5.1. It is then necessarythat for all allowable variations SS > 0. We first choose Sul = 0 when-ever ul is equal to either ul* or u*, and similarly for Sue. It then followsthat since we are not at a constraint, we may vary ul and u2 either posi-tively or negatively (Fig. 5.1). Thus, whenever H is differentiable, wemay choose the particular variations

Sul = -e HauR aHaul R

for sufficiently small e' and obtain the same result as before:

(7)

When the optimal decision u; (i = 1 or 2) lies between the constraintsu;. and u* and the hamiltonian is differentiable with respect to u;,it is necessary that the hamiltonian be stationary with respect to u,(aH/au; = 0).

Let us now consider what happens when tit = u*. For conveniencewe set Sue = 0 for all t and Sul = 0 whenever ul 96 u,*. Because of theconstraint all changes in ul must be negative (Fig. 5.1), and so we have

Sul < 0 (g)

Let us make the particular choice

Sul=elaH<0 (9)

where we cannot set the algebraic sign of El since we do not know the sign

u,

u, *

_T8

Su, moy onlybe negativewhen u,=u*

-------------1

8u, may bepositive ornegative when

8u, may onlybe positive but > 0when u,=u,*

But 0eui2 j - - bue20

f

Fig. 5.1 Allowable variations about the optimum decisionfunction.


of all/au,. From Eq. (4) we then have

rs

J(3u > dt + o(f) > 0 (10)

Thus, el > 0, and from Eq. (9),

aH,<0aul-

Since u, decreases as we move into the interior of the allowable region, itfollows from the sign of the derivative in Eq. (11) that H is increasingor that the hamiltonian is a minimum relative to u, when u, = u,*. (Inan exceptional case it might be stationary at u; , and the nature of thestationary point cannot be determined.)

We can repeat this analysis for sections where u, = u,., and there-fore &u, > 0 (Figure 5.1) and we obtain the same result. The symmetryof the problem establishes the result for us as well. If u, or us lies alongan interior interval at a constant value where H is not differentiable butone-sided derivatives exist, then an identical proof establishes that at thesepoints also H is a minimum.. Furthermore, the hamiltonian is still aconstant, for whenever the term all/au, (or OH/au:) in Eq. (9) of Sec. 4.3does not vanish along the optimal path for a finite interval, the optimaldecision u, (or us) lies at one of its bounds or at a nondifferentiable pointand is a constant. In that case du,/dt = 0 (or dus/dt) and the productsaH du,

andaH au2ddU2

au, dt-

- always vanish, leading to the result that,iH/dt = 0.tWe summarize the results of this section in the following weak form

of the minimum principle:

Along the minimizing path the hamiltonian is made stationary by an.optimal decision which lies at a differentiable value in the interior of theallowable region, and it is a minimum (or stationary) with respectto an optimal decision which lies along a constraint boundary or at anondifferentiable interior point. The hamiltonian is constant alongthe optimal path, and the constant has the value zero when the stoppingtime 0 is not specified.

5.3 A BANG-BANG CONTROL PROBLEM

As the first example of the use of the necessary conditions derived in theprevious section let us consider the control of a particularly simpledynamical system, described by the equation

1 = u (1)

CONTINUOUS SYSTEMS: II

or, equivalently,

it=x:2==u

in

(2a)(2b)

We shall suppose that the system is initially at some state x1(0), x:(0)and that we wish to"choose the function u(t), subject to the boundednessconstraints

u*=-1<u<+1=u*or

Jul < 1

(3a)

(3b)

in order to reach the origin (x1 = x= = 0) in the minimum time; that is,

6=fu 1dt=0= 1


H = 1 + X1x2 + xsu

(4a)

(4b)

(5)

and since 0 is unspecified, the constant value of H along the optimal pathis zero. The canonical equations for the multipliers are

8H 0=

X2= -aaH= -x1

(6a)

(6b)

and since four boundary conditions are given on the -state variables, theboundary conditions for the multipliers are unspecified. Equation (6a)has the solution

Xi. = cl = const (7a)

and Eq. ON the solution

x= _ -clt - c2 (7b)

where c1 and C2 are unknown constants of integration resulting from theunspecified boundary conditions.

It is of interest first to investigate whether u may take on values inthe interior of the allowable region. In that case the condition for opti-mality is

dH=0=x:=-clt - c: (8)au

Equation (8) cannot be satisfied for any finite interval of time unlessboth c, and Cl, the slope and intercept of the straight line, vanish. In

IQ OPTIMIZATION BY VARIATIONAL METHODS

that case 1\x is also zero [Eq. (7a)] and the hamiltonian, Eq. (5), has thevalue unity. Since we have already noted that the optimal value of Hmust be zero, it follows that the necessary conditions for a minimumcan never be satisfied by a control function which is in the interior ofthe allowable region for any finite time interval.

The only possibilities for the optimum, then, are u = +1 andu = -1. A control system of this type, which is always at one of itsextreme settings, is known as a bang-bang or relay controller. A typicalexample is a thermostat-controlled heating system. We note that the,question of when to use u = + 1 or u = -1 depends entirely upon thealgebraic sign of X2, for when X2 is positive, the hamiltonian is made aminimum by using u = -1 (-11\2 < + 11\2, X2 > 0), while when 1\2 isnegative, the hamiltonian is minimized by u = +1 (+11\2 < -11\2,X2 < 0). Thus, the optimal policy is

u = - sgn X2 = sgn (cxt + c2) (9)

Here the sgn (signum) function is defined as

sgn y_

Iyl = { ±1 y < 0(10)

and is undefined when y = 0.We now have sufficient information to solve the system differential

equations, starting at xx = x2 = 0 and integrating in reverse time, i.e.,calling the final time t = 0 and the initial time - 0. The conditionH = 0 establishes that C2 = 1, and for each value of cl, - co < cx cc,we shall define a trajectory in the xxx2 plane, thus flooding the entireplane.with optimal trajectories and de6ning a feedback control law.: Inthis case, however; the entire, problem can -,be solved more siipply byanalytical methods.

We note first that the argument of the signum function in Eq. (9)can change sign at most once. Thus the optimal solution may switchfrom one extreme value to the other at most onee.t During an intervalin which the optimal control policy is u = +1 the system equations (2)become

±1 = x2 (11a)is = 1 (llb)

or

x2=t+C:'/ 2

xx = %t2 + Cat + C4 = 72(t + c3)2 + (C4 -2

(12a)

(12b)

t It can be demonstrated that for the time-optima! control of an nth-order dynamicalsystem with all real characteristic roots the number of switches cannot exceed 1. lessthan the order of the system.

CONTINUOUS SYSTEMS: 11

X2

Fig. 5.2 Possible responses for u = + 1.

1a,

Thus, a first integral is

x1 = 112 x22 + C (13)

which defines the family of parabolas shown in Fig. 5.2, the arrows indi-cating the direction of motion. Note that the origin can be reached onlyalong the dashed line x1 = /2x22, x: < 0, so that if u = +1 forms thelast part of an optimal trajectory, this must be the path taken. In asimilar way, when u = -1, we obtain the family of parabolas

xi = -12x22 + c (14)

shown in Fig. 5.3, with the only possible approach to the origin along thedashed line x1 = -112x22, x2 > 0.

When the two sets of trajectories are superimposed, as in Fig. 5.4,the optimal policy becomes obvious at once. The approach to the originmust be along the dashed line, which has the equation

x1 + 112x21x21 = 0 (15)

and at most one switch is possible. The only way in which initial statesbelow the dashed line can be brought to the origin in this manner is to

x2

Fig. 5.3 Possible responses for u = -1.

X,

142

XZ


XI

Fig. S.4 Superposition of all possibleresponses with bang-bang control.

use the control u = +1 until the resulting trajectory intersects the lineof Eq. (15) and then to switch to u = -1 for the remainder of thecontrol time. Similarly, initial states above the dashed line are broughtto the origin by employing the control action u = -1 until intersectionwith the dashed line [Eq. (15)] followed by u = -1. This defines theoptimal feedback control policy, and only the switching curve [Eq. (15)]ief required for implementation. The optimal trajectories are then asshown in Fig. 5:5.

S.4 A PROBLEM OF NONUNIQUENESS

The simple dynamical system considered in the previous section may beused to illustrate another feature of solutions employing the minimumprinciple. We now suppose that we wish to solve the minimum-timeproblem to drive xL to zero, bht we do not choose to specify x2(8). Theanalysis is essentially unchanged, but because x=(8) is unspecified, we nowmust invoke the boundary condition

X2(8) = 0

X2

X1

(1)

Fig. S.S Time-optimal paths to the origin.


or, from Eq. (7b) of the preceding section,

X2(O) -CIO - c2 = 0

Thus,

X2(t) = cl(O - t)

143

(2)

(3)

which cannot change algebraic sign, and therefore the optimal controlfunction, defined by Eq. (9) of Sec. 5.3, must always be +1 or -1, withno switching possible.

Figure 5.6 shows the trajectories in the right half-plane. For start-ing values above the dashed line x1 + 112x2Ix2I = 0 the line x1 = 0 canbe reached without switching only by using the policy u = -1. Forstarting points below the line x1 + t3 x2Ix21 = 0, however, the x2 axiscan be reached without switching by using either u = + 1 or u = -1.Thus, even in this simplest of problems, the minimum principle does notlead to a unique determination, and the true optimum must be dis-tinguished between the two candidates by other considerations.

In this case the true optimum can be determined analytically.Setting u = ± 1 and dividing Eq. (2a) of Sec. 5.3 by (2b), we obtain theequation for the tangent to each trajectory passing through a point

tan a = dz2 = x2 u = +1

tan = d22 = -x2 u = -1

Fig. 5.6 Two possible paths to the xZ axissatisfying the necessary conditions.

(4a)

(` b)


Thus, referring to Fig. 5.6, the line segments Q,P2 and Q,P, are equal inmagnitude. But integrating Eq. (2a) of Sec. 5.3,

8 = x2(8) - x2(0) = QOQ2 u = +1 (5a)B = x2(0) - x2(8) = QoQ1 u = - 1 (5b)

and, by inspection,

QOQ2 > QOP2 = QOP1 > Q0Q1 (6)

Thus, u = -1 leads to the shorter time in the entire right-hand plane..By similar reasoning, when x, < 0, the optimal policy is u = +1.

5.5 TIME-OPTIMAL CONTROL OF A STIRRED-TANK REACTOR

We shall now return to the problem of the control of a stirred-tankchemical reactor introduced in Sec. 4.6. The dynamical equations forthe reactor after an upset, linearized about the desired steady-stateoperating conditions, were shown to be of the form of the general second-order system

x1

12

= a11x1 + a12x2 + b11u1= a21x1 + a22x2 + b21u1 + b22u2

where x1 and x2xire the deviations from steady state in reduced concen-tration and temperature, respectively, while u1 and u2 are the variationsin process and coolant flow rates. It was also shown that after a lineartransformation of the dependent variables the system could be repre-sented by the equations

y1 = S1y1 M11u1 - M12u2 (2a)y2 = S2Y2 + M21u1 + M12u2 (2b)

where the parameters S1, S2, M11, M12, and M21 are defined in Sec. 4.6 byEqs. (19) and (21). In this section we shall consider the problem ofreturning the system from some initial state y1(0), 1/2(0) to the steadystate y1 = Y2 = 0 in the minimum time by choice of the functions u1(t),u2(t), subject to the operating constraints on the flow rates

u,. < u1 < u; (3a)U2* < u2 < us (3b)

For the minimum-time problem the function if is equal to unity,and so the hamiltonian is

H= 1 + A1(S1y1 - M11u1 - M12u2) + X2(S2y2 + M21u1 + M12u2)


and the equations for the multipliers areAl _ - 4911 = -S1X1

ay,(5a)

X2 = -aH

= - S2X2aye

(5b)

These last equations may be integrated directly to giveX1(t) = Aloe-s,` (6a)X2(t) = Xye-s,e (6b)

although the initial conditions X1o, X2o are unknown.The possibility that u1 may lie somewhere between its bounds is

considered by setting aH/au1 to zeroaH

= -M11A1 + M21X2 = 0 (7)au1

Substitution of Eqs. (6) and (7) demonstrates that this.equality can holdfor more than a single instant only if S1 = S2, a degenerate case whichwe exclude. Thus, the optimal u1 must always lie at a bound, u1. orui , and the"same may easily be shown true for u2. The coefficient of u1in Eq. (4) is -X1M11 + X2Mf21, so that the hamiltonian is minimized withrespect to u, by setting u1 equal to the smallest possible value when thecoefficient is positive and the largest possible value when the coefficientis negative, and similarly for u2:

uiU, =

M21X2 < 0(8a)u,. -M11A1 + M21A2 > 0

u2 1112(X2 - X1) < 0U2 = (8b)u2 M12(X2 - X1) > 0

For the parameters listed in Table 4.1 we have S1 < S2 < 0,M21 > M11 > 0, M12 < 0. Using Eqs. (6)f we can then rewrite theoptimal control policy in Eqs. (8) as

1 1 >u1X20 (r A

0u1 =

go

X

(9a)

10ecs,-s,I: - 1 1 <u1. X20 (r 0

io1 >- X20 020 0

U2 =

X

(9b)

10 ecs2-s,1t - 1 1 <X20 (U2* 0\X20 //

where

r=Mzi<1 (10)


The structure of the optimal control policy may now be deduced in amanner similar to that in the two previous sections.

If Xlo and 1\2o have opposite signs or have the same sign with1\1o/A20 < 1, the quantities r(A1o/A2o)e($,-s,>r - 1 and tX1o/A2o)e(S1-s,)' - 1

are both always negative, and depending on the sign of X20, it folldwsfrom Eqs. (9) that the optimal control must always be either the pair(u; ,u2.) or (u1*,u2 ), with no switching possible. These pairs may alsooccur when A1o and A2o have the same algebraic sign with A1o/X20 > 1 butonly when t is sufficiently large for (A1o/1\20)e(s; s=)1 to be less than unity.

If 1 < A10/A20 < 1/r, the initial policy is (ul.,u2.) if A20 > 0, fol-lowed by (u1.,ug) if the origin has not been reached after a time 't2 suchthat

A1° 1 = 0X20

which is the criterion of switching in Eq. (9b), or

I Agot2 _S,2 S11n A1o

(11 a)

(llb)

with no further switching possible. Similarly, if A2o <.0, the sequence is(u; ,u; ), (u; ,u2.), with the switching time defined by Eq. (11).

The remaining possibility is that A1o/A2o > 1/r. If A20 > 0, theinitial control policy defined by Eqs. (9) is (ui ,u2.), followed, if the originhas not been reached after a time

1 A2o 1t1 = S2 _ S1

InA1o r

(12)

by the policy (u1.,u2.). A further switch will occur at time t2 definedby Eq. (llb) to the policy (u1.,us ), and no further switching is possible.Thus, the total duration for which the system may be controlled by thepolicy is given as

is - t1 =S2 1 S1

(inA1o

- InA1o r) S2 1 S11n r (13)

which depends only on the system parameters. In a similar way, ifA20 < 0, the sequence of optimal policies is (u1.,us ), (u; ,uz ), and (u2 ,u2.),with the same switching time. This exhausts all possibilities, and itshould be noted that no single control variable may switch more thanonce between limits in this second-order system.

With the sequence of possible control actions available to us we arenow in a position to construct the optimal feedback control policy. Since


the control action must be piecewise constant, an integral of Egs.-(2) is

r yi - (MII/SI)ul - (M12/SJ)uzl s,y1, - (M22/S1)u2J

ys + (M21/S2)ui + (M1:/S:}u2 St (14)Y2, + (M21/S2)u1 -1- (M12/S:)ua

where y1 ys, are values of yi, Y2 somewhere on the path. Thus thepaths leading to the origin may be obtained by setting y1, - yzr - 0and putting the appropriate control policy into Eq. (14).

The line marked y+_ in Fig. 5.7 is a plot of Eq. (14) passing throughthe origin with the-control policy (ui ,uz#). Since we have found thatthis policy must always be preceded by the policy. (ui ,us ), the line y+_must be a switching curve for trajectories with the control (u,*,ui ), for

0.02 \ a04

F10.5.7 Time-optimal paths to the origin in transformed coordinatesfor the controlled reactor. [From J. M. Douglas and M. M. Denn,Ind. Eng. Chem., 57(11):18 (1065). Copyright 1965 by the AmericanChemical Society. Reprinted by permission of the copyright owner.]


otherwise these trajectories could not reach the origin by an. optimalsequence. Similarly, the line y_+, corresponding to (u1.,u2 ), must bethe switching curve for trajectories with control (ul+,u2*)

By choosing points on the y+_ curve and solving Eqs. (2) with theconstant control policy (ui ,u2) for a time interval 1/(S2 - S1) In r weobtain the curve where the optimal control must have switched from thepolicy (u1.,u2 ), the y++ switching curve, and similarly for the y__ curve.We obtain in this way a line for the y++ (or y__) curve which stops shorta finite distance from the origin, for we have seen that we can reach theorigin along an optimal trajectory prior to switching from (us,u= ). Weobtain the remainder of the y++ curve by setting x1, = x2r = 0 and(u1,u2) = (ui ,u=) in Eq. (14), and similarly for y_ _. These switchingcurves may fail to be smooth at the intersection of the two segments.

We have now divided the x1x2 plane into four sections, in each ofwhich the control action is completely specified, with the change in con-trol indicated by reaching a boundary. We have, therefore, by deter-mining the switching curves, constructed the optimal feedback control forthe time-optimal problem. The curves in Fig. 5.7 are calculated for thevalues of the parameters given in Table 4.1, together with the constraints

-8<u1<+10=ui (15a)

u2.= -5<u2<15=?42 (15b)

while Fig. 5.8 shows the trajectories after transformation to the originaldimensionless concentration (Z1) and temperature (Z2) coordinates.Only one switching was required for most trajectories, and initial con-ditions for trajectories requiring more than one switching generally falltoo far frgm the origin in the Z1Z2 plane for a linearized solution to beuseful, in some cases generating trajectories which, lead to negative con-centrations. It is interesting to observe that many of the optimal tra-jectories which approach the y++ and y__ curves do so with a commontangent.

At this point it is useful to note again an alternative method ofsolution which is well suited to automatic digital computation. Wemake use of the fact that for problems with unspecified total operatingtimes the hamiltonian has the value zero. Thus, when the origin hasbeen reached, from Eq. (4),

_ X 1(6)[M11u1(8) + M12u2(B)} - 1X2(e) -

M21u1(e) + M12u2(e)(16)

If we specify the final values u1(6) and u2(8), Eq. (16) definer a uniquerelation between A1(8) and X2(8); for some range of values of A1(O) thisrelation will be consistant with the requirements of Eq. (9) for the choiceof u1i u2. For example, the final policy (ui,us) requires, after some

CONTINUOUS SYSTEMS: (I

3.00

290

2.80

N2.70

2.60aE

r°- 2.50

2.40

2.30

149

2.20 i 1 1 I ( 1

0 0.04 0.08 0.12 O.t6

Composition tti '

R9. 6.E Time-optimal temperature-concentration paths.for the con-trolled reactor. [From J. M. Douglas and M. M. Denn, Ind. Eng.Ch.m., 67(11):18 (1965). Copyright 1965 by the American.ChemieatSociety. Reprinted by permission of the copyright owner.)

algebra, the satisfaction of the two inequalities

M:lu; + M12u, > 0 (17a)

which is a limitation imposed by the physical properties of the system,and

We) > 1(17b)

Similar relations can be found for other policies.For a given X1(8) we then have values X1(6), 12(e), y1(9) = 0,

yz(e) = 0, and we can integrate the four differential equations (2) and(5) in the negative time direction, monitoring the combinations in Eq.(9) at all times. When the sense of an inequality changes, we need sim-ply make the appropriate change is the control and continue. In thisway we shall map out the switching curves and optimal trajectories aswe vary the values of X1(8) . over the range -- = < X1(9) < m. Thisbackward tracing technique will clearly be of the greatest use in non-


linear systems, where the analytical methods of this section cannot beemployed.

5.6 NONLINEAR TIME-OPTIMAL CONTROL

The practical design of a time-optimal control system for the stirred-tank reactor for any but very small upsets will require the use of thefull nonlinear equations, and we shall build on the observations of theprevious section by following a recent paper of Douglas and consideringthis more general problem. The nonlinear equations describing the reac-tor, Eqs. (7) of Sec. 4.6, are

z1= y(1-Z1)-kZl (la)

UKq.Z, =V

(Z1 - Z,)VCDP( +

Kq,) (Z: - Z.) +kZ1 (1b)

where Z, is dimensionless concentration, Z2 dimensionless temperature,and k has the form

k = ko exp ( AH)Af Z=J (2)

The constants are defined in Sec. 4.6, with numerical values given inTable 4.1.

In order to avoid complication we shall assume that the flow rate qis fixed at qs and that control is to be carried out only by varying thecoolant flow rate q. subject to the bounds

qes+u: <qc <_ qcs+U:. (3)

where the numerical values of U2. and u: are the same as those in theprevious section. The hamiltonian for time-optimal control is then

H = 1 + X1 y(1 -Z1) - kZiJ +a: I V (Z, --Z:)

UKga (Z, - Z.) + kZi] (4)VCpp(1 +


1 i - 8Il =.

X: = -

azIaHaz:.

[.1 UKga E'C pkl, 1 X, (5b)+ V + 1'Dp(1 + Kq.) (-LH)AfZ:=

(1 +k) x1-ka,E'C,pkZI

(-AH)A fZ:=


We first consider the possibility of intermediate control by setting8H/8q, to zero:

aq = -a= CK (Z2 - Z.) 1 +1Kq. = 0 (6)

This equation can hold for a finite time interval only if Z, = Z, or X2 = 0.In the former case, Z2 constant implies that Z, is also constant, which isclearly impossible except at the steady state. On the other hand, if Xvanishes for a finite time interval then so must its derivative, which,from Eq. (5b), implies that X, also vanishes. But if X, and X2 are bothzero, then, from Eq. (4), H is equal to unity, which contradicts the neces-sary condition that H = 0 when 8 is unspecified. Thus, the control forthis nonlinear problem is also bang-bang, and we shall have the solutionby construction of the switching surfaces.

The structure of the optimal control function is

qg q.s + u=` (Z: - Z.) X, > 0(7)q.s + us. (Zs - Z.)X2 < 0

Because of the nonlinear nature of Eqs. (1) and (5) analytical solutionsof the type employed in the previous section cannot be used to deter-mine the maximum number of switches or the switching curves. It isto be expected that in a region of-the steady state the behavior of thenonlinear system will approximate that of the linearized system, so thata first approximation to the switching curve can be obtained, by settingq. equal, in turn, to its upper and lower limits and obtaining, respec-tively, the curves y+ and y_, shown in Fig. 5.9. The optimal trajectoriescan then be computed, using the policy us" above the switching curve,us. below, and switching upon intersecting y+ or y_. Because of themanner of construction of these curves no more than one switch will everbe made. It should be noted that many of the trajectories approach they_ switching curve along a common tangent, as'in the linearized solution,although this is not true of y+. No trajectories can enter the region ofnegative concentration.

The verification of this solution must be carried out by the back-ward tracing procedure described in the previous section. When steadystate has been reached, Eqs. (4) and (7) become

H = -1 + X1(e) IV (1 - Zis) - kZ,s] + X2(8) IV (Z1 - Zss)

UK% (Zis - Z.) + kZ,s] = 0 (8)VCpa(1 + Kq.)q0(8) q.s + ui (Z23 - Z.)Xs(6) > 0

;9)1 u2. (Zss - Z.)X2(8) < 0

U2 OPTIMIZATION BY VARIATIONAL METHODS

A choice of X2(8) uniquely determines q,(8) from Eq. (9), while Eq. (8)determines X1(8). Thus, since x1(8) = x2(8) = 0, the four equations (1)and (5) can be integrated simultaneously in the reverse time direction,always monitoring the algebraic sign of (Z2S - Z,)X2. When this signchanges, qe is switched to the other extreme of the range and the processcontinued. This is done for a range of values of X2(e), and the locus ofswitching points is then the switching curve for the feedback control sys-tem. The trajectories and switching curves in Fig. 5.9 were verified byDouglas in this way, except in the region where trajectories approach y_along a common tangent, where extremely. small changes in X2(8) (of theorder of 10-6) are required to generate new trajectories.

It is important to recognize that the nonlinear differential equa-tions (1) may admit more than one steady-state solution; in fact, theparameters listed in Table 4.1 are such that three solutions are possible.Thus, there is a separatrix in the x1x2 plane which is approximately theline Z2 = 2.4, below which no trajectory can be forced to the desiredsteady state with the given control parameters. The design steady stateis controllable only subject to certain bounded upsets, a fact which wouldnot be evident from a strictly linearized analysis.

5.7 TIME-OPTIMAL CONTROL OF UNDERDAMPED SYSTEMS

The systems we have studied in the previous several sections had thecommon property that in the absence of control, the return to the steadystate is nonoscillatory, or overdamped. This need not be the case, and,

3.00

E

r 2.60

2.40'0 0.02 0.04 0.06 0.08 0.10

Composition r,

Ffg. &9 Time-optimal temperature-concentration paths for the non-linear model of the controlled reactor. [From J. M. Douglas, Chem.Eng. Sci., 21:519 (1965). Copyright 1965 by Pergamon Press. Re-printed by permission of the copyright owner.l


indeed, oscillatory behavior is often observed in physical processes. Dif-ferent choices of parameters for the stirred-tank-reactor model would leadto such oscillatory, or underdamped, response, and, since the structure ofthe time-optimal control is slightly changed, we shall briefly examine thisproblem.

The prototype of an oscillatory system is the undamped forcedharmonic oscillator,

2+X=u (1)

ortl=x: (2a)x4 = -xl + U (2b)

We shall suppose that u is bounded

fuI<1 (3)

and seek the minimum time response to the origin. The hamiltonian isthen

H = 1+Xixz-X2x1+X2u (4)


aH = Xzi

aH = -al

(5a)

(5b)

The vanishing of X or X2 over a finite interval implies the vanishingof the derivative, and hence of the other multiplier as well, leading to thecontradiction H = 1 for the time-optimal ease. Thus, intermediate con-trol, which requires X2 = 0, is impossible, and the solution is again bang-bang, with

u= -sgnX2 (6)

The solution of Eqs. (5) may be written

Xz = -A sin (t + 0) (7)

where the constants of integration A and 0 may be adjusted so thatA > 0, in which case Eq. (6) becomes

u = sgn [sin (t + 0)] (8)

That is, rather than being limited to a single switch, the controllerchanges between extremes after each time interval of duration x.

The construction of the optimal feedback control proceeds in thesame way as in Secs. 5.3 and 5.4. When u = +1, the first integral of


Fig. 5.10 Possible responses of anundamped system with bang-bangcontrol.

Eqs. (2) is

u= +1: (x,-1)2+(x2)4=R2 (9a)

a series of concentric circles centered at x, = 1, z2 = 0, while for u = -1the integral is a series of circles centered at x, -1, x2 = 0:

u = -1: (x1 + 1)2 + (x2)2 = R2 (9b)

All trajectories must then lie along segments of the curves shown in Fig.5.10, and since the control action changes after every v time units, a tra-jectory can consist of at most semicireles. Thus, the approach to theorigin must be along one of the dashed arcs, which must also form partof the switching curves, -y+ and -y-.

We can complete the construction of the switching curves by con-sidering, for example, any point on y+. The trajectory leading to itmust be a semicircle with center at -1, and so the corresponding pointon the y_ curve can be constructed, as shown in Fig. 5.11. In this fashionthe switching curve shown in Fig. 5.12 is built up; with some typical tra-jectories shown. Above the switching curve and on y+ the optimal con-trol is u = +1, while below and on y_ the optimum is u = -1.

Fig. 5.11 Construction of the switchingcurve for time-optimal control of anundamped system.


Fig. 5.12 Time-optimal paths to the origin for an undampedsystem.

The general second-order system,

z + alt + a2x = u (10)

which is represented by the chemical reactor, may be thought of as adamped forced harmonic oscillator whenever the parameters al and a2are such that the characteristic equation

m2+aim+a2=0 (11)

has complex roots. In that case a similar construction to the one aboveleads to a switching curve of the type shown in Fig. 5.13 for the time-optimal problem. We leave the details of the construction to the inter-ested reader.

5.t A TIME-AND-FUEL-OPTIMAL PROBLEM

Although rarely of importance in process applications, aerospace prob-lems often involve consideration of limited or minimum fuel expendituresto achieve an objective. If the control variable is assumed to be the

156

X2,


Fig. 5.13 Switching curve for time-optimal control of an underdampedsystem.

thrust, then to a reasonable approximation the fuel expenditure may betaken as proportional to the magnitude of the thrust, so that the totalfuel expenditure is proportional to Io Jul dt. Besides the obvious physi-

cal importance, optimization problems which involve the magnitude ofthe decision function introduce a new mathematical structure, and so weshall consider one of the simplest of such problems as a further exampleof the use of the minimum principle.

The system is again the simple one studied in Sec. 5.3

and we shall assume that the total operating time for control to the originis unspecified but that there is A. premium on both time and fuel. Theobjective which we wish to minimize is then

s = PO + fa lul dL = fa (p + lul) dt (3)

where p represents the relative value of time Sfuel. The hamiltonian is

H = p + Jul + )tiX2 + A2U (4)


l= -aH=0 (5a)

aHxY

ax = -alY

(5b)

or

Al = C1 = const (6a)A2 = -clt + C2 (6b)


As usual, we first consider the possibility of a stationary solutionby setting aH/au to zero, noting that alul jau = sgn u, u 76 0:

aH=sgnu+X21+X2=0(7)au

But if X2 is a constant for a finite interval of time, X2 =-X, is zero, sothat Eq. (4) becomes

H=P+lul -usgnu=p+lul - lul=P>O (8)

which contradicts the necessary condition that H = 0 for unspecified 0.We note, however, that the hamiltonian is not differentiable at u = 0because of the presence of the absolute-value term, and so we may stillhave an intermediate solution if H is minimized by It = 0.

Whenever X2 < -1, the term Jul + a2u is less than zero whenat = + 1, while it is zero for It = 0 and greater than zero for u = -1.Thus the hamiltonian is minimized by It = +1. Similarly, when X2 >+1, the hamiltonian is minimized by u = -1. For - I < X2 < + 1,however, l ui + X214 is zero for u = 0 and greater than zero for u = ± 1.Thus, the optimal solution is

It =

+1 X2 < -10 -1<X2<+1

-1 +1 < X2(9)

Since 1\2 is a linear function of time, the only possible control sequencesare + 1, 0, -1 and -1, 0, + 1, or part of either, with a maximum oftwo switches. This is a bang-coast-bang situation, or a relay controllerwith a dead zone.

We can show further that the final optimal control action must beu +1 or It = -1. Ifu=Oatt= 0, when x, =x2=0,then, again,H = p 96 0, which contradicts a necessary condition for optimality.Thus, the approach to the origin and the switching curve from coastingoperation must be the time-optimal switching curve

xl + i2x2lx21 = 0 (10)

shown as the segments yo+ and yo- in Fig. 5.14. The switching curvesy-o from u = -1 to It = 0 and y+o from It = +1 tp u = 0 can be con-structed analytically in the following way.

Let t2 denote the time that a trajectory intersects the switchingcurve yo+ and t, the prior time of intersection with y_o.(6b) and (9) we write

From Eqs.

A201) = +1 = -c1t1 - c2 (Ila)X2(t2) = -1 = -c1t2 - c2 (1 lb)

158

X2

OPT`MIZATiON BY VARIATIONAL METHODS

Fly. 5.14 Time-and-fuel-optimal switch-ing curves and paths to the origin.

or, solving for c1,

2C1 =

12 - tl

Evaluating the hamiltonian at t = t1, where u = 0,

H = P + clx2(tl) = 0

or, eliminating c1 in Eqs. (12) and (13),

2x2(11)12-t1=-P

(12)

(13)

(14)

In the interval t,1 < t < t2 the solution of Eqs. (1), with u = 0, is

x2(12) = x2(tl) (15a)x1(12) = x1(11) + x2(tl)(12 - tl) (15b)

But, from Eq. (10), the curve yo+ has the equation

x1(12) = 32x22(t2) (16)

and combining Eqs. (14) to (16), we obtain the equation for the switch-ing curve y_o

x1(11) = P 2p 4 x22(tl) (17)

We obtain the y+o curve in a similar way, with the entire switching curverepresented by

xl + P

p4

x2Ix21 = 0 (18)

CONTINUOUS SYSTEMS: II 15I

The two switching curves, together with typical trajectories, are shownin Fig. 5.14.

Note that as p in which case only time is important, thetwo switching curves coincide and the coast period vanishes, giving, asexpected, the time-optimal solution. As p -- 0, the y+o and y_o linestend to the x, axis, which we anticipate will be the fuel-optimal solution(bang-coast). In fact the minimum fuel can be obtained by more thanone control action, and the one obtained here represents the solutionrequiring the minimum time. Furthermore, the reader will note thatwe are asking that the system move at zero velocity between states,which is impossible, so that the limiting solution clearly does not exist,but we can come arbitrarily close to implementing a fuel-optimal solu-tion as p is allowed to become arbitrarily large.

5.9 A MINIMUM-INTEGRAL-SQUARE-ERRORCRITERION AND SINGULAR SOLUTIONS

In Sec. 4.7 we studied the optimal control of the linear second-ordersystem, representing the stirred-tank chemical reactor, for an objectivein the form of an integral of squares of the state deviations and thesquare of the control function, leading to a linear feedback control. Theu2 term in the objective may be rationalized as a penalty function tokeep the control action within bounds, but this goal can also be accom-plished by the methods of this chapter. We thus return to the second-order system with quadratic objective, but we shall now eliminate thecost-of-control term from the objective and include bounds on the con-trol action, the coolant flow rate.

After an appropriate change of variables the reactor equationsbecome Eqs. (4) of Sec. 4.7

yi = Y2 (la):J2 = - a2y1 - a1y2 + u (lb)

and the objective

f (c11y,2 + 2c12y1y2 + c22y22) dt (2)o

where we have set ca, to zero, but we now seek the optimal function usubject to the restriction

u* < u < u* (3)

The hamiltonian may then be written

H = i2(C11y12 + 2c12y1y2 + c22y22)+ X1y2 - a2A2y1 - a1X2y2 + X2u (4)


and the multiplier equations

aHXI = - - = a2X2 - C11y1 - c12ys

ayiaH

s = - - = -XI + aids - c12y1 - Cssy:ay,

(5a)

(5b)

Because of the linearity of H in u the minimum principle then impliesthe optimal solution in part as

U = {u' Xs < 0u. a: >0 (6)

It will now become clear why we; have always been careful toexamine each 'system for the possibility of an intermediate solution.Setting all/au to zero, we obtain

aHasO(7)au

which is the situation not covered by Eq. (6). But if X2 is zero for afinite time interval, so must be its derivative and Eq. (5b) becomes

0=--aI -c1sy>-C::ys (8)

This in turn implies that

X1 = -cisl%i - Cssys = ascssys + (ales: - c>s)ys - C22u (9)

which, when compared to Eq. (5a), leads to a solution for u

u = (as+Ciilyi+a1ys (10)\\ Call

That is, if intermediate control is possible, the form is a linear feedbackcontroller.

We must still satisfy the requirement that the hamiltonian be zeroalong the optimal path. For the intermediate control defined by Eqs.(7), (8), and (10) the hamiltonian, Eq. (4), becomes

H (-c&syl - cssys)ys + i2(ciiyis + 2c12y1y2 + cssyss)_ ,(ciiy>s - cssyss) = 0 (11)

so that the intermediate solution represented by Eq. (10) is possible onlywhen the system lies along one of two straight lines in the ylys state space

Cll yl + css ys = 0 (12a)

Cll yl - Ca ys = 0 (12b)

We can further eliminate the second of these possibilities by substituting


Eq. (10) into Eq. (11), giving

y1 Y2

y2C22

yl

and, differentiating Eqs. (12),

92 ± CC119

222

161

(13q)

(13b)

(14)

where the positive sign corresponds to Eq. (12b). Combining Eqs. (13b)and (14) and solving, we obtain

C ' (t - r) (15)yl = yl(r) exp ± Cft

with the positive exponential corresponding to" Eq. (12b). Thus the con-troller is unstable and clearly not a candidate for the optimum along line(12b), since the objective will grow without bound, while the solution isstable along the line (12a) and, hence, perhaps optimal. At all otherpoints in the y,y2 phase plane except this single line, however, the opti-mal solution must be at one of the extremes, u* or u*. Indeed, only thefinite segment of the line (12a) satisfying

u*< (a2 + e22 y, + ajy2 5 u* (16)

may be optimal.Next we must establish that when the intermediate, or singular,

control defined by Eq. (10) is possible, it is in fact optimal. That is,when 1\2 vanishes at a point yi, y2 on the line segment described by Eqs.(12a) and (16), we do not shift to the other extreme of control but ratheroperate in such a way as to keep X2 identically zero. Because a bang-bang trajectory would again intersect the line segment at some otherpoint y7, y2 , it suffices to show that the singular path between thesetwo points (along the straight line) leads to a smaller value of the objec-tive than any other path. That is, if we let ai(l) and u2(t) denote thevalues of y, and y2 along the singular path, for all paths between thetwo points

v. - vl"Iy,v:--v.' v:"

(e,)yi2 + 2c12y1y2 + c22y22) dtv:- v!'

VI-VI"

2c,20192 4- czsv22) dt > 0 (17)ZYYe

vi

This relation is most easily verified by a method due to Wouham and


Johnson. We note that from l:q. (la)1 d

y1J2 = l Wt(yl2)

The integrand in Eq. (2) may then be written

C11y12 + 2cl2y1y2 + C22y22 = ( C11 yl + C22 Y2)2

(18)

+ (C12 - V Cucz2) dt (y12) (19a)

and, along the singular path, from Eq. (12a),

0110,2 + 2c,2Q1Q2 + 022022 = (012 - 'V C11C22) d (012) (19b)

Integrating Eqs. (19a) and (19b) and subtracting, we make use of thefact that of and yl are identical at both end points (we are comparingthe value of the objective along different paths between the same endpoints) to establish Eq. (17):

y,-y," y1-yt11

Jy, - y," 2

2 y, ' y:"J (Cllul + 201401472(Cllyl + 2c12y,y2 + C22y 2) dt -

y,'y,2

= yt y,y: _y,' Y,=yi

y, - y,"y, - yi'+ 022022) dl = f ( CSI yl + C22 Y2)2 dt > 0 (20)

y, - y,'y=- y1

This proves that whenever it is possible to use the linear feedback con-trol defined by Eq. (10)-i.e., whenever Eqs. (12a) and (16) are satisfied-it is optimal to do so.

The remainder of the optimal policy can now be easily constructedby the backward tracing procedure. At any point on the singular linevalues are known for yl, y2, Xi, and A2

yl = yi (21a)Y2 = Y2 (21b)X1 = - cl2yi - c22y2 (21c)

_A2 = 0 (21d)

Equations (1) and (5) can then be integrated in reverse time for u = u*,checking at all times to be sure that X2 < 0. When A2 again returns tozero, we have reached a point on the y_+ switching curve and u is set tou* and the process continued. In a similar way, setting u initially to u*will generate a point on the y+_ switching curve when X2 returns to zero.By carrying out this procedure for all points on the singular line theentire switching curve can be generated, just as for the time-optimalproblems considered in previous sections.

A final word concerning practical implementation is perhaps in

CONTINUOUS SYSTEMS: II 1u

order. In any real system it will be impossible to maintain the processexactly along the singular line described by Eq. (12a), and careful exami-nation of Eqs. (13) indicates that they are unstable if the system everdeviates from the straight line. Thus the control law of Eq. (10) must bereplaced by one which is completely equivalent when the system is in facton the singular line but which will be stable if slight excursions do occur.This can be accomplished, for example, by subtracting 3[(cl,/c22)y, +

(ci1/c2s)y21, which vanishes along line (12a), from the right-hand sideof Eq. (10), giving

u = Cat - 2 c111yi + (ai - 3 LC22 y2 (22)

with the resulting system equations (1), the stable process

y, = Y2

y' = -2 y1 - 3 J2 y2C22 c2

(23a)

(23b)

5.10 NONLINEAR MINIMUM-INTEGRAL-SQUARE-ERROR CONTROL

A number of the ideas developed in the previous' section can be appliedwith equivalent ease to nonlinear systems. In order to demonstrate thisfact we shall return again to the now-familiar example of the stirred-tankchemical reactor with control by coolant flow rate, having state equations

Z1 =

V

(1 - Z1) - kZ1 (1a)

Z= =

V

(Z, - Z2) - VCP(1 + Kq,) (Z2 - Z.) + kZ, (1b)

where k = k(Z2). Since q/(1 + Kq,) is a monotonic function of qc, wemay simply define a new decision variable w as the coefficient of Z2 - Z.in Eq. (lb) and write

Z2 = y(Z, - Z2) - w(Z2 - Z.) + kZ1 _ (lc)

with constraints derivable from Eq. (3) of Sec. 5.6 as

w* < w < w* (2)

We shall attempt to maintain the concentration Z, and temperature Z:near the respective steady-state values Z15, Z23 by choosing w to mini-mize the integral

2 Jo[(Z1 - Z13)2 + c2(k - k8)2] dt (3)


where ks = k(Z2S). The choice of k, rather than Z2, in Eq. (3) is one ofconvenience, though it can be rationalized by the observation that it isdeviations in reaction rate, not temperature, which adversely affect theproduct composition. This is, of course, the objective chosen in Sec. 4.8.

The hamiltonian for this problem is `

H = 3- (Z1 - Z15)2 + ti2c2(k - k5) 2 + X1 V (1 - Z1) - X1kZ1

+ X2

V

(Zf - Z2) - X2w(Z2 - Zc) + X2kZ1 (4)


1 = -(Z1 - Zls) + V X1 + kX1 - kX2 (5a)

ak ak akqk0 k ZX X- s)(2 = - 1 1 aZ2 + X2 V + X2w - X2Z1aZtaZ2

+ (5b)

and, from the linearity of H in w, the optimum has the form

u' =tu* X2(Z2 -- ZI) > 0w# X2(Z2 - Z,) < 0

(6)

The possibility of a singular solution, for which X2 vanishes over afinite interval, must still be examined. In that case the derivative mustalso be zero, and since ak/aZ2 9d 0, Eq. (5b) reduces to

X1Z1 - c2(k - ks) = 0 (7a)

and Eq. (5a) to\\

x1= -(Z1-Z1s)+(V+k)X1. (7b)

These two equations, together with Eq. (1a),-are identical to Eqs. (1),(7), and (8) of Sec. 4.8, and the approximate solution for the singular lineis given by Eq. (16) and the corresponding singular control by Eq.. (17)of that section. Thus, the singular solution is equivalent. to choosingthe temperature Z2 which minimizes the objective, Eq. (3), provided thatthe resulting flow rate is consistant with Eq. (2).

In the previous section we utilized a mathematical argument toprove the optimality of the singular solution. Here, we simply rely onphysical reasoning. Clearly, if we are in fact free to specify the tem-perature directly, we shall choose to do so, since this is our real physicalgoal, and whenever the choice of flow rate coincides with the optimalchoice of temperature, that choice must also lead 'to the optimal flowrate. Only when the optimal temperature is not accessible by choice of

CONTINUOUS SYSTEMS: it 16S

flow rate need we consider the second in this hierarchy of optimizationproblems, the optimal choice of flow rate itself. Thus, for this reactorproblem, the singular solution, when possible, must be optimal.

the construction of the switching curves can now be carried out bythe backward tracing technique, since values of Z2, Z2, A1, and X2 are allknown along the singular line. We should observe that a closed-formsolution for the nonlinear singular control w (but not for the singular line)is in fact available by eliminating X1 between Eqs. (7a) and (7b).

5.11 OPTIMAL COOLING RATE IN BATCH AND TUBULAR REACTORS

In Sec. 3.5 we briefly considered the optimal specification of the temper-ature program in a batch or plug-flow pipeline chemical reactor in orderto minimize the time of achieving a given conversion, for a single reactionor, equivalently, of obtaining the maximum conversion in a given time.In the batch reactor it is somewhat more realistic to suppose that we canspecify the cooling rate as a function of time, rather than the temper-ature, and in the tubular reactor also the rate of heat removal is some-what closer to actual design considerations than the temperature at eachposition. We are now in a position to study this more realistic problemas an example of the methods of this chapter, and we follow a discussionof Siebenthal and Aris in doing so.

Denoting the reactant concentration as a and temperature as T,the equations describing the state of the reactor are

a -r(a,T) (1a)T = Jr(a,T) - u (1b)

where J is a constant (heat of reaction divided by the product of densityand specific heat) and u, the design or control variable, is the heat removalrate divided by density and specific heat, with bounds

u* = 0 < u < u* (2)

In a batch system u will be a inonotonic function of coolant flow rate.In a tubular reactor the independent variable is residence time, the ratioof axial position to linear velocity. The reaction rate r(a,T) is given byEq. (13) of Sec. 3.5

/ i\ / n2 n2r(a,T) =plan exp E(- P2 bo + 1 ao - nla)n,

exp - T2 t (3)

where ao and bo are the initial values of reactant and product, respec-tively. When E2 > Ei, there is a maximum value of r with respect to

1K OPTIMIZATION BY VARIATIONAL METHODS

T for fixed a satisfying ar/aT = 0, while r will go to zero for fixed T atsome finite value of a (equilibrium). These are shown in rig. 5.15 asthe lines r = 0 and x = rm.z.

We shall first consider the objective of minimizing a (that is, maxi-mizing conversion) while placing some finite cost on operating time (orreactor length), so that the objective is

& - a(e) + pe

or, equivalently, using Eq. (la),

(4a)

s = Jo [p - r(a,T)] dt (4b)

The hamiltonian is

H = p - r(a,T) - Xir(a,T) + X,Jr(a,T) - a2u= p - r(a,T)(1 + ai - JX2).- X2U (5)


as = a (1 + al:- At) a,(e) = 0 (6a)

aH = -& (1 + X1 - JX2) x2(e) = 0 (6b)aT aT

Because of the linearity of H in u the optimal decision function is

11 =u X2 > 00 1\ <0 (7)

with an intermediate solution possible only if X2 vanishes over a finiteinterval.

O

o°

NM FGTemperature T

Fig. 5.15 Reaction paths and switchingcurve for optimal cooling in a batch ortubular reactor. [After C. D. Siebenthaland R. Aris, Chem. Eng. Sci., 19:747(1964). Copyright 1964 by PergamonPress. Reprinted by permission of thecopyright owner.]


Because 0 is not specified, the optimal value of the hamiltonian iszero. At t = 0, using the boundary conditions for X1 and X2, Eq. (5)reduces to the stopping condition

r[a(0),T(0)] = p (8)

That is, as common sense dictates, the process should be terminatedwhen the reaction rate falls to the value p and the incremental return inthe objective changes algebraic sign. The stopping curve r = p is shownin Fig. 5.15.

Let us now consider the possibility that the final operation is inter-mediate. In that case the vanishing of X2 implies the vanishing of itsderivative, and therefore of ar/aT. That is, if the final operation issingular, it must be along the curve ?*,,,ax, terminating at the highest pointof the curve r = p, shown as point A. This is. precisely the policy whichwe found to give the minimum time and maximum conversion operationin Sec. 3.5, so that we may anticipate that singular operation is optimalwhen possible for this combined problem as well, and we shall show thatthis is indeed the case.

We first suppose that an optimal path terminates on the line r = pto the right of rm,x. Here, ar/aT < 0, and it follows from Eq. (6b) thatK2(9) = ar/aT < 0, or 1\2 is decreasing to its final value of zero. Thus,from Eq. (7), the final policy must be u = u*, full cooling. There is,however, a point on the curve r = p, say C, at which a trajectory foru = u* intersects the line r = p from the right; i.e., the slope of the line-da/dT along the trajectory is greater than the slope of the line r = p.From Eqs. (1),

da _ p da ar/aTdT

)u_u*Jp - u* > ( dT)f_P ar/aa

r-o(9)

and, after slight manipulation, the point C is defined by

u* > p J -ar/aa

(10)ar/aT>

Since the integrand in Eq. (4b) is positive whenever the system is to theright of r = p, an optimal policy cannot terminate at r = p to the rightof point C, for smaller values of S could be obtained by stopping earlier.Thus, an optimal policy can terminate to the right of rmax only under thepolicy u = u* on the segment AC of r = p.

In a similar way we find that the system can terminate to the leftof rmax only under adiabatic conditions (u = 0) on the pegment BA ofr = p, where B is defined by J > (ar/aa)/(a)-/aT). Thus, whenever thesystem lies to the right of )'max and to the left of the full-cooling tra-


jectory AF, the optimal policy is full cooling until the intersection withrmax, followed by the intermediate value of u necessary to remain on rmax,for the system can never leave the region bounded by DAF, and this isthe policy which will minimize the term p6 in the objective, a(6) beingfixed. Similarly, to the left of rmax and below the adiabatic line DA thepolicy is adiabatic, followed by the singular operation. Within the regionbounded by FACG the policy is only full cooling, and in EGAD only adi-abatic, but even here some trajectories starting in regions r < p mightgive positive values to the objective, indicating a loss, so that some initialstates will be completely excluded, as will all initial states outside theenvelope EBACG.

By combining Eq. (1) and Eq. (14) of Sec. 3.5 it easily follows thatthe optimal policy on the singular line rmax is

u = r I J + ni[nibo + n2ao - n2a(1 - n2/n,)l T 2> 0a(nlbo + n2ao - n2a)

There will be some point on rmax, say K, at which this value exceeds u*,and a switching curve must be constructed by the usual backward tracingmethod. This switching curve KM will be to the right of the full-coolingline KN, for we may readily establish that KN cannot form part of anoptimal path. By integrating Eq. (6a) along rmax from t = B we findthat X (i) + 1 > 0 at K, in which case, for an approach from the left ofrmax, where ar/aT > 0, X2(0 > 0. Thus, X2 < 0 just prior to reachingK, and only adiabatic approach is possible. Furthermore, along an adi-abatic path both ar/aa and X2 are negative, so that X1 + 1 is alwaysgreater than its value at rmax, which is positive. Hence X2 is always posi-tive, so that X2 can never reach zero on an adiabatic path, and an adi-abatic path cannot be preceded by full cooling. Thus, for starting pointsto the right of the adiabatic line KL the optimal policy is adiabatic tothe switching line KM, followed by full cooling to rmax, then the policydefined by Eq. (11) to the point A.

If we now consider the problem of achieving the maximum con-version in a given duration (p = 0, 0 fixed), clearly the solution is identi-cal, for the hamiltonian now has some nonzero constant value, say -A,and we may write

H+, =A-r(a,T)(1+X1-JX2)-X2u=0 (12)

with the multiplier equations and Eq. (7) unchanged. Furthermore,since a(t) must be a monotone decreasing function, the policy whichminimizes a(6) for fixed 6 must, as noted in Sec. 3.5, also be the policywhich reaches a specified value of a in minimum time (or reactor length).


5.12 SOME CONCLUDING COMMENTS

Before concluding this chapter of applications some comments are inorder. The bulk of the examples have been formulated as problems inoptimal control, largely because such problems admit some discussionwithout requiring extensive computation. From this point on we shallsay less about the construction of feedback control systems and concen-trate on computational methods leading to optimal open-loop functions,which are more suitable to process design considerations, although severalchapters will contain some significant exceptions.

The reader will have observed that the several optimal-control prob-lems which we have formulated for the second-order system, exemplifiedby the stirred-tank chemical reactor, have each led to markedly differentfeedback policies, ranging from relay to linear feedback. The physicalobjective of each of the controls, however, is the same, the rapid elimi-nation of disturbances, and there will often be situations in which it isnot clear that one objective is more meaningful than another, althoughone "optimal" control may be far easier to implement than another.This arbitrariness in the choice of objective for many process applicationswill motivate some of our later considerations.


Section 5.2: The derivation follows

J. M. Douglas and M. M. Denn: Ind. Eng. Chem., 57 (11):18 (1965)

The results are a special case of more general ones derived in Chap. 6, where a completelist of references will be included. A fundamental source is

L. S. Pontryagin, V. G. Boltyanskii, R. V. Gamkrelidze, and E. F. Mishchenko:"Mathematical Theory of Optimal Processes," John Wiley & Sons, Inc., NewYork, 1962

Sections 6.3 to 5.5: The time-optimal control problem for linear systems is dealt with inthe book by Pontryagin and coworkers and in great detail in

M. Athans and P. Falb: "Optimal Control," McGraw-Hill Book Company, NewYork, 1966 it

R. Oldenburger: "Optimal Control," Holt, Rinehart & Winston, New York, 1966

Historically, the bang-bang result was obtained by Bushaw and further developed byBellman and coworkers and LaSalle:

R. Bellman, I. Glicksberg, and O. Gross: Quart. Appl. Math., 14:11 (1956)D. W. Bushaw : "Differential Equations with a Discontinuous Forcing Term,"

Stevens Inst. Tech. Expt. Towing Tank Rept. 469, Hoboken, N.J., 1953; Ph.D.thesis, Princeton University, Princeton, N.J., 1952; also in S. Lefschetz (ed).,"Contributions to the Theory of Nonlinear Oscillations," vol. 4, PrincetonUniversity Press, Princeton, N.J., 1958


J. P. LaSalle: Proc. Natl. Acad. Sci. U.S., 45:573 (1959); reprinted in R. Bellmanand R. Kalaba (eds.), "Mathematical Trends in Control Theory," Dover Publica-tions, Inc., New York, 1964

The calculations shown here for the reactor problem are from the paper by Douglas andDenn. Further considerations of the problem of nonuniqueness are found in

1. Coward and R. Jackson: Chem. Eng. Sci., 20:911 (1965)

A detailed discussion of all aspects of relay control is the subject of

I. Flugge-Lotz: "Discontinuous and Optimal Control," McGraw-Hill Book Company,New York, 1968

Section 5.6: The section is based on

J. M. Douglas: Chem. Eng. Sci., 21:519 (1966)

Other nonlinear time-optimal problems are treated in Athans and Falb and

1. Coward: Chem. Eng. Sci., 22:503 (1966)E. B. Lee and L. Markus: "Foundations of Optimal Control Theory," John Wiley

& Sons, Inc., New York, 1967C. I). Siebenthal and R. Aria: Chem. Eng. Sci., 19:729 (1964)

A simulation and experimental implementation of a nonlinear lime-optimal control isdescribed in

M. A. Javinsky and R. H. Kadlec: Optimal Control of a Continuous Flow StirredTank Chemical Reactor, preprint 9C, 68d Natl. Meeting, A IChE, St. Louis, 1968

Optimal start-up of an autothermic reactor is considered in

R. Jackson: Chem. Eng. Sci., 21:241 (1966)

Section 5.7: See the books by Pontryagin and coworkers and Athans and Falb.

Section 5.8: The book by Athans and Falb contains an extensive discussion of linear andnonlinear fuel-optimal problems.

Section 5.9: The basic paper is

W. M. Wonham and C. D. Johnson: J. Basic Eng., 86D:107 (1964)

where the problem is treated for any number of dimensions; see also

Z. V. Rekazius and T. C. Hsia: IEEE Trans. Autom. Contr., AC9:370 (1964)R. F. Webber and R. W. Bass: Preprints 1987 Joint Autom. Contr. Confer., Phila-

delphia, p. 465

In Chap. 6 we establish some further necessary conditions for the optimality of singularsolutions and list pertinent references. One useful source is

C. D. Johnson: in C. T. Leondes (ed.), "Advances in Control Systems," vol. 2,Academic Press, Inc., New York, 1965

Section 5.10: The use of a hierarchy of optimization problems to deduce an optimalpolicy is nicely demonstrated in


N. Blakemore and R. Aria: Chem. Eng. Sci., 17:591 (1962)

Section 5.11: This section follows

C. D. Siebenthal and R. Aria: Chem. Eng. Sci., 19:747 (1964)

III

Section 5.1P The literature contains many examples of further applications of the prin-ciples developed here, particularly such periodicals as A IA A Journal, Automatica,Automation and Remote Control, Chemical Engineering Science, Industrial andEngineering Chemistry Fundamentals Quarterly, IEEE Transactions on Auto-matic Control, International Journal of Control, Journal of Basic Engineering(Trans. ASME, Ser. D), and Journal of Optimization Theory and Applications.The annual reviews of applied mathematics, control, and reactor analysis in Indus-trial and Engineering Chemistry (monthly) list engineering applications. Someother recent reviews are

M. Athans: IEEE Trans. Autom. Contr., AC11:580 (1966)A. T. Fuller: J. Electron. Contr., 13:589 (1962); 15:513 (1963)B. Paiewonsky : A IAA J., 3:1985 (1965)

Applications outside the area of optimal design and control are growing, particularly ineconomics. A bibliography on recent applications of variational methods to economicand business systems, management science, and operations research is

G. S. Tracz: Operations Res., 16:174 (1968)

PROBLEMS

5.1. Extend the minimum principle of Sec. 5.2 to nth-order systems

xi - fi(x1,x2, . . . xwrul,uir . . . 1 m 1, 2, . . . , n

Show that the hamiltonian is defined by

H-5+ 1xif,

aH as " 8fiaxi axi

7t ' axi

5.2. Solve the optimal-temperature-profile problem of Sec. 4.12 for the case of anupper bound on the temperature. By considering the ratio of multipliers show howthe problem can be solved by a one-dimensional search over initial values of the ratioof multipliers. For consecutive first-order reactions

F(x1) - x, G(x2) _= x,

obtain an algebraic expression for the time of switching from the upper bound to anintermediate temperature in terms of the initial ratio of the multipliers. Discuss thecomputational effect of including a lower bound on temperature as well.5.3. Solve the optimal-pressure-profile problem of Sec. 4.13 for the case of an upperbound on the pressure. By considering the ratio of multipliers show how the problem


can be solved by a one-dimensional search over initial values of the ratio of multipliers.Discuss the computational effect of including a lower bound on the pressure as well.5.4. Consider the system

i, - a,tx, + a12x2 + b,ux2 = a21x, + a2222 + b2uIul < 1

Establish the number of switches possible in the minimum time control to the originfor all values of the parameters and construct the switching curve for the case ofcomplex roots to the characteristic equation.S.S. A system is termed controllable to the origin (xt - 0, z: - 0) if, for each initialstate, there exists a piecewise-continuous control u(t) such that the origin can beattained in some finite time B. We have assumed throughout that the systems withwhich we are dealing are controllable. For the linear system

21 - a12x2 + b,ux2 = a2,21 + a22x2 + b2u

show that a necessary and sufficient condition for controllability is

b,(a21b, + a22b2) - b2(a,,b, + a12b2) # 0

This is a special case of results of Kalman on controllability and the related conceptof observability. Hint: Solve for x, and x2 in terms of a convolution integral involvingu(t) by Laplace transform, variation of parameters, or any other convenient method."Only if" is most easily demonstrated by counterexample, "if" by construction of afunction which satisfies the requirements.5.6. For the nonlinear system

x + g(x) = u Jul < 1,

where g is a differentiable function satisfying tg(.z) > 0, develop the feedback controlfor minimum time to the origin. Show that the optimum is bang-bang and no morethan one switch is possible and indicate the procedure for constructing the switchingcurve. (The problem is due to Lee and Markus.)5.7. A simplified traffic-control problem is as follows.

Let q, and q2 represent arrival rates at a traffic light in each of two directionsduring a rush period of length 9, s, and 82 the discharge rates (st > s2), L the time foracceleration and clearing, and c the length of a cycle. If x, and x2 represent the lengthsof queues and u the service rate to direction 1, the queues satisfy the equations

xt=gt -- uL\ s,2t-q2-,2 1 +-uc Si

with initial and final conditions x,(0) - x2(0) - x,(B)The service rate is to be chosen within bounds

.u,<u<uto minimize the total holdup

- 22(9) = 0.

eE ,. fo (xt + 22).dt


It is helpful to approximate arrival rates as constant during the latter phases of therush period in obtaining a completely analytical solution. (The problem is due toGazis.)5.8. The chemical reaction X Y -+ Z is to be carried out isothermally in a catalyticreactor with a blend of two catalysts, one specific to each of the reactions. Thedescribing equations are

i = u(ksy - kiz)y = u(kix - k2y) - (1 - u)kayx+y+z - const

Here u is the fraction of catalyst specific to the reaction between X and Y and isbounded between zero and 1. Initial conditions are 1, 0, and 0 for x, y, and z, respec-tively. The catalyst blend is to be specified along the reactor, 0 < 1 < 0, to maximizeconversion to Z,

max 61 - z(0) - 1 - x(8) - y(8)

(a) When k, = 0, show that the optimum is a section with u = 1 followed bythe remainder of the reactor with u = 0 and obtain an equation for the switch point.

(b)r When k2 $ 0, show that the reactor consists of at most three compartments,the two described above, possibly separated by a section of constant intermediateblend. Obtain the value of the intermediate blend and show that the time interval(t1,t,) for intermediate operation is defined by

t1 =+ k2

log\1 + k' + k1

k1 ksk,

1

v k:

(The problem is due to Gunn, Thomas and Wood, and Jackson.)5.9. For the physiological system described in Prob. 4.10 find the optimal controlwhen c - 0 with the presumption that u is bounded. Examine possible singularsolutions and describe the process of construction of any switching curves.5.10. Using the device of Sees. 4.4 and 4.9 for treating explicit time dependence,extend the minimum principle of Sec. 5.2 to include problems of the form

i1 = f1(x1,x2,u,t)iz - f2(x1,x2,u,t)

remin t - J a(xa,x2,u,t) dt

0

5.11. The system

i+ai+bx =uis to be controlled to minimize the ITES (integral of time times error squared) criterion,

1 re

Develop the optimal feedback" control. Consider whether intermediate control isever possible and show bow switching curves can be constructed.


5.12. The chemical reactor described by Eqs. (7) of Sec. 4.6 is to be started up frominitial conditions Z,Q, Z20 and controlled to steady state Z,,, Zu by manipulation ofcoolant flow rate q, and feed temperature Z, with bounds

0 < q. < q! Z,. < Z, < Z1

If heating costs are proportional to feed temperature, the loss in profit during start-upis proportional to

&=Io,[(Z1.-Z.)+c(Z,-Z,.)1dd

where 9 is unspecified. Discuss the optimal start-up procedure. (A more complexversion of this problem has been studied by Jackson.)

6The Minimum Principle

6.1 INTRODUCTION

In the preceding chapters we have obtained and applied necessary con-ditions for optimality in a wide variety of optimal-design and optimal-control problems. Greater generality is required, and that is one aimof this chapter. A more serious deficiency of the preceding work,however, is that while the results are certainly correct, the motivationfor several important operations in the derivations is not at all obvious.Thus we have little direction in attacking new problems by these methods,and, indeed, we sense that our ability to devise efficient computationalalgorithms is dependent upon our understanding of the logical steps ina proof of necessary conditions.

There is, in fact, an underlying logic which can be applied to boththe theoretical and computational aspects of variational problems. Thelogic is firmly grounded in the elementary theory of linear ordinarydifferential equations, and the first considerations in this chapter willof necessity be directed to a discussion of this theory. The resulting

175


relations, which we shall apply to modern variational problems, wereused by Bliss in his analysis of problems of exterior ballistics in 1919.

6.2 INTEGRATING FACTORS AND GREEN'S FUNCTIONS

The principle we are seeking is best introduced by recalling the methodof integrating the linear first-order equation. Consider

± = a(t)x + b(t) (1)

In order to integrate this equation we multiply both sides by an inte-grating factor r(t), an arbitrary differentiable function. We are inten-tionally introducing an extra degree of freedom into the problem. Thus,

r(t)x(t) = a(t)r(t)x(t) + r(t)b(t) (2)

or, integrating from t = 0 to any time t,fo'

r(r)±(r) dr = ,,o(r)r(r)x(r) dr +fo'

r(r)b(-r) dr (3)

The term on the left can be integrated by parts to give

r(t)x(t) - r(0)x(o) - fo i'(r)x(r) dr

= fat a(r)r(r)x(r) dr +fot

r(r)b(r) dr (4)

If we now remove most of the arbitrariness from r by defining it tobe the solution of

i' _ - ar (5)or

r(t) = r(0) exp [ -fot

a(>;) d¢] (6)

then Eq. (4) simplifies to

r(t)x(t) exp [ - fo a(s) dE] - r(00)x(o)r

= r(0) f o, [ - .lor a() dt] b(r) dr (7)

Assuming that r(0) is different from zero, we can solve explicitly forx(t)

x(t) = x(O) exp [ fo a(s) d ] + fo exp [Jt

a(t) dA] b(r) dr (8)

The integrating factor r(t) is generally known as a weighting functionor, as we prefer, Green's function.

We can generalize to a system of n linear equations in a straight-

THE MINIMUM PRINCIPLE

forward manner. We consider the system

x, = a,l(t)xl + a12(t)x2 + + aln(t)x, + b,(t)x2 = a21(t)x, + a22(t)x2 + + a2n(t)xn + b2(t)

xn = anl(t)xl + an2(t)x2 + + ann(t)xe + bn(t)

or, equivalently,

177

(9a)

xi = I a)(t)x! + bi(t) (9b)

It is convenient to introduce n2 functions rki(t), k, i = 1, 2,to multiply Eq. (9) by rki and sum over all i. Thus

. , n,

n

rn (`)

T(`n)

I rkixi = I L l kia. jx, + rkibi (10)i-I i-Ii-i i-i

Integrating from t = 0 to any time and integrating the left-hand sideby parts, we obtain

Jogi-1

inn

rki(T)xi(T) dr 1, ki(t)xi(t) - rki(0)xi(0)i=1 i-1

n nn

r10 'ki(r)xi(r) dr = Jo rki(T)aii(T)xi(r) dr

rki(r)bi(r) dr (11)i -i

As for the first-order system, we remove most of the arbitrarinessfrom r by partially defining Fk,(t) as a solution of

n

Pki(t) _ - I rkf(t)a11i-1

equation (12) is called the adjoint of Eq. (9b). Thus,

(12)

n n}

`Rn) (\)rki(t)xi(t) - 4 rki(O)xi(0) = f6 / rki(r)bi(r) dr (13)

i-1

Equation (13) is known as Green's identity. It is the, one-dimensionalequivalent of the familiar relation between volume and surface integrals.The matrix of Green's functions r is known as Green's matrix, the funda-mental matrix, or sometimes the adjoint variables. Unlike the first-ordercase, it is necessary to specify the value of I'ik at some time. It is con-

178 OPTIMIZATION BY VARIATIONA! METHODS

venient to specify r at the time of interest 8 as

=rfk(e) = Sik =

1 i k0 i k (14)

so that Eq. (13) reduces ton n

x:(8) _ r;,(o)x,(0) + fo ri,(t)b,(t) dt (15)

Indeed, for later use it is convenient to note explicitly the moment atwhich condition (14) is specified, and we do this by specifying twoarguments, r;k(8,t), where 8 is the time of interest and t any other time.Thus,

n n

xi(8) = ri;(e,0)x2(o) + fo ri;(e,t)b;(t) dt (16)

where r,,(e,t) satisfies Eq. (12) with respect to t and the condition

rik(e,e) = bik (17)

We shall sometimes be interested in a linear combination of thecomponents x; of x at time 8. We define a vector y(t) with componentsTi(t) by

NO) = Jy;(e)r;i(9,t) (18),-1

where y;(0) are specified numbers; then by multiplying Eqs. (12) and(16) by appropriate components of y(8) and summing we obtain

n n n

yi(9)xi(9) _i

yi(0)xi(0) + fo yi(t)bi(t) dt (19)i-1 -1 i-1

and

Yi = - 4 7,(t)a,i(t) (20)i-1

As a simple illustrative example consider the linear second-ordersystem with constant coefficients

x + a1x + a2x = b(t) (21)

or, equivalently,

z1 = x2 (22a)x2 = -a2x1 _ a1x2 + b2(t) (22b)

THE MINIMUM PRINCIPLE 179

The adjoint system, defined by Eq. (12), with boundary conditions fromEq. (14), is

v11 = a2r12 T11 = 1F12 _ - r11 + a,r12 r12=0F21 = a2r22 r21 = 0V22 _ - r21 + a1r22 r22 = 1

at t = 0 (23a)att=0 (23b)at t = 0 (23c)at t = 0 (23d)

The solution is easily found to beel4« 'u-e)

(ai - a,2 --4a2)2 a12 - 4a2

exp[% a,2-4a2(t- 8))- (a1+ a12-4a2)exp [-Y2 1/x12 - 4a2 (t - 0)) } (24a)

eSSa'(t-e)

r12(0,t) = - {exp [32 a12 - 4a2 (t - B))a,2 - 4a2

- exp [-3z -\ a12 - 4a2 (t - 0))} (24b)14«,cr-e)

a12 - 4a2 (t - B(8,t) = ate { exp [Y2 ))r211/a ,2 - 4az

-exp[-Y2 a12-4a2(t-0))} (24c)

r22(8,t) _ - { (a1 + Va12 - 4a2)2 a,2 - 4a2

exp [32 -V a12 - 4a2 (t - 8)] - (a, - a,2 - 4a2)exp [-% v a12 - 4a2 (t - e))) (24d)

Then, for any time 0,

x(8) = x1(9) = r11(B,O)x(O) + r12(e,O)t(O)

+ foe r12(e,t)b2(t) dt (25a)

±(e) = X2(0) = r21(8,O)x(O) + r22(e,0)x(O)

+ fo r22(e,t)b2(t) dt (25b)

or, substituting just into the equation for x(t),

x(9) -e-«1e12 {[cri a12 - 4a2 x(O) -

1/a12 - 4a2 2

exp(-TL Val2-4a20) - f a1+ 212-4a2-z(O)]

l - fe eSS«,ce-e)exp (32 a12 - 4az 8) }

o ate - 4a2{ exp [3i 1/a12 - 4a2

(1111-

8) )

- exp [-Y2 a,2 - 4a2 (t - 0))}b(t) dt (2&)

150 OPTIMIZATION BY VARIATION* METHODS

6.3 FIRST-ORDER VARIATIONAL EQUATIONS

We now consider a physical system described by the (usually nonlinear)equations

0 < t < 6 (1)x; = f;(x,u)i = 1, 2, , n

If we specify a set of initial conditions to and specify the componentsof u(t) to be particular functions uk(t), k = 1, 2, . . . , R, Eqs. (1) canbe integrated to obtain functions which we shall denote by x;(t), i = 1,2, . . . , n. That is, we define t(t) as the vector whose componentsare solutions of the equations

= f,(t,u) 0 < t < ai = 1, 2, . . . , n (2)

t(0) = t0

Let us now suppose that we wish to specify new functions uk(t)and initial values x;(0) as

uk(t) = uk(t) + auk(t) k = 1, 2, . . . , R (3a)x;(0) = z;0 + bx;0 i = 1, 2, . . . , n (3b)

where, for some predetermined e > 0,

Wk(t) < Ek = 1, 2, . . . , R

(4d)0<t<016x;ol < e i = 1, 2, . . . , n (4b)

If we now solve Eqs. (1) and write the solution as

x,(t) = xi(t) + Sx1(t) (5)

it follows (see Appendix 6.1) that

1 Sx;(t), < Ke (6)

where K depends on 0 but is independent of the variations Su and Sxo.Thus,

i; = ,+O± =ft(t+fix, ft +Su)X(0) = to + Sxo

and, subtracting Eq. (2) from (7),

0<t<0i=1,2, . . . n

(7)

0±, = f,(t + &x, 6 + Su) - f;(t,n) 0<t<0i=1,2,...,n (8)

If f; has piecewise continuous first and bounded second derivatives


with respect to components of x and u, then everywhere except atisolated discontinuities of u we may expand the right-hand side ofEq. (8) in a Taylor series to write

axi= af`axj+ fauk+o(6) .0<lEe,8x;

k lauk i = 1, 2, n-l .j (9

where the partial derivatives are evaluated for It and u and,lence,known functions of t. Comparing the linear equation (9) with Eq. (9`of the preceding section, we identify the terms 8fi/8xj with aij and intro-duce Green's vector y from Eq. (20) of Sec. 6.2

ndlj 0<t<8'Yjaxi i= 1,2, . . . ,n (10)

Green's identity, Eq. (19) of that section, is thenr

`.

-Yi(8) axi(o) y,(O) 6x,0 + JO +I ry'-Lf OUk dt }- U(E)

i I

(11)

To first order, this is the effect on system output of small changes ininitial conditions and decision functions.

6A THE MINIMIZATION PROBLEM AND FIRST VARIATION OF THE OBJECTIVE

It is convenient to formulate the optimization problem in somewhatdifferent form from the previous chapters. Instead of seeking to mini-mize an integr4l we shall suppose that ours concern is with some func-tion S which depends only upon the state of the process at time 0, x(8).We assume that the system is described by the differential equations

-ti = Ji(x,u)0<t<8i= 1, 2, . . . n (1)

that the decision variables uk(t) satisfy inequality constraints

U,(u)>0 p = 1, 2, . . . P

that the initial state xo is subject to equality constraints

q.,(xo) = 0 m = 1, 2, . . . , M

and the final state to equality constraints

g.[x(e)I=0 s= 1, 2, ...,S

(2)

(3)

(4)

122 OPTIMIZATION Bf VARIATIONAL METHODS

The constraint equations (2) to (4) are more general than those con-sidered previously. We shall see later that the choice of the objective&[x(e)] also includes the form of previous chapters as a special case.

We shall first assume that 0 is specified. If we specify u(t) and5u(t) subject to the constraint equation (2) and to and Szo subject toEq. (3), we may write Eq. (11) of Sec. 6.3

i-1 i-1

(5)

Furthermore, q(8o +.Szo) and q(2o) must both equal zero, as mustg[!(0) + Sx(0)] and g[2(8)], so that

n

4m(20 + bao) - Qm(g0) = xaaxi0 + 0(E) = 0

o0

m = 1, 2, . .. , M (6)

9.[!(0) + &z(e)l - 9.[2(0)] 8x, axl(e) + o(E) = 0

s=1,2,...,S (7)

The change in & as a result of decision changes is reflected through achange in x(9)

n

a& = &[:(e) + sa(e)] -.&[t(0)] _ ax:axl(e) + 0(0% (8)

i-1

We have not yet specified the value of the vector y(9). Let uswrite this as

71(e) = ax + .i

From Eq. (8) we may then write

n

&& = I k'i(e) - ')'d axi(B) + 0(E)i-1

and, from Eq. (5),

axl(e) + 'Y,(0) a:Zio + JOB I I yj auk Suk dt

i-1 i=1 i-1 k-1

n n n R

(9)

(10)

+ 0(t) (11)

We can obtain an expression for a& entirely in terms of the variations in


decision btt(t) by choosing particular boundary conditions for 7(t)S

dg,7c = YJ ax

a-1or

and

sat; ag,

49X, Ia-1

183

(12a)

(12b)

M

yi(O) 'hm axm (13)in -I

Here, it and Y are undetermined multipliers which must be found as partof the solution. Using Eqs. (6) and (7), Eq. (11) becomes, upon-substi-tution of the boundary conditions,

x Ra afi

bE =Io yi auk

but dt + o(e) (14)i11 k1

The 2n differential equations for the x; and y; require a total of 2nspecified values. Equations (3) and (4) give a total of if + S conditions.Equation (12b) specifies n - S values and Eq. (13) n - 117, so that thetotal is indeed the required number. The special case in which xi(0) orxio is directly specified is included by setting

g,[x(0)] = x,(0) - x; (0) = 0 (15a)

and

qm(xo) = xmo - X. *O = 0 (15b)

where x; (0) and xw*,a are fixed numbers.

a&Y.(0) = ax + V.

In the former case

(16a)

where P. is unspecified, so that y,(0) is unspecified, while in the latter

'Ym(O) _ 77m (161,)

where i is an unspecified number.When 0 is not fixed, we must allow for the possibility that the

changes bu(t), Sxo will require a change 60 in order to satisfy all the con-ditions on x(0). Let a refer to the interval associated with u(t), to. Then

bs = 6[x(8 + 60)] - s[!(e)] (17a)

I" OPTIMIZATION BY VARIATIONAL METHODS

or, writing E[x(e + ae)] in terms ofn

as = E[x(e)] +as

fi[x(B),u(B)] se + o(se) - 8120)]i -1 axi

=as

s(x(B)] - s[a(e)] + fi[2(6),u(e)] ae + o(ae) + 0(E)i1 ax;

and, finally,n

as = I as

, axi[axi(s) + fi as] + o(E)

where we have included o(SO) as o(E).Similarly,0

(17b)

(17c)

ag. [ax,(e) + fi as] + o(E) = 0 (18)

Defining yi(B) by Eq. (12b), y;(0) by Eq. (13), and substituting Eqs. (5)and (18) into (17c), we obtain

as = n(B)f be + Jo yi auk auk do + o(E) (19)i-1 i-lk-1

Finally, because of the extra degree of freedom resulting from the non-specification of 0 we may set

0 unspecified: y;{9)f;[2(B),n(B)] = 0 (20)i-1

and ag is again represented by Eq. (14). In other words, the unspecifiedstopping time 9 is defined by the additional Eq. (20).

6.5 THE WEAK MINIMUM PRINCIPLE

It is convenient to introduce again the hamiltonian notationn

H = I y; fii-1

(1)

As previously, the system and multiplier equations may be written inthe canonical form

aHayi

aHaxi

(2a)

(2b)


with the boundary conditions Eqs. (3),and the additional condition

(4), (12), and (13) of Sec. 6.4

0 unspecified: H = 0 at t = 8 11 (3)

If we now assume that u(t) is in fact the set of allowable functions whichminimize &, then, from Eq. (14) of the preceding section, we may write

err auk dt + 0aS = Jo 1 a

(4)uk-1

The functions uk(t) must be chosen subject to the restriction

U,(u) > 0 p = 1, 2, . . . , P (5)

If, for a .particular value of k, all constraints are at inequality, we may,choose auk (sufficiently small) as we wish, and it follows (as in Sec 5.2)that

aH-

= 0 uk unconstrained (6)auk

If some constraint U, is at equality for the optimum, it must betrue that

L U" auk + o(e) > 0 (7)aul,1

k-1

In the spirit of Sec. 5.2, we choose all variations but one to be zero andthat one as

a U, aHSuk auk auk (8)

From Eq. (7) auk and aUD/auk must have the same algebraic sign.Substituting into Eq. (4), we obtain

all, -0

(9a)kj)

aauk `auk

or

au°>0

Then, multiplying Eq. (8) by all/auk,\z

H

(9b)

a 0l (10>/kuk

auk auk aor, equivalently,

a)

aukaH>0auk - (10b)

.136 OPTIMIZATION BY VARIATIONAL METHODS

Thus, when a constraint is in force, the hamiltonian is a minimum withrespect to decisions affected by the constraint. (In exceptional casesthe hamiltonian may only be stationary at a constraint.) An equivalentresult holds when the hamiltonian is not differentiable for some f1k.

For the special case considered in Sec. 5.2 we were able to establishthat the hamiltonian is a constant along the optimal path. This result,too, carries over to the more general situation considered here. Wecalculate

lY = 1 aHx;+ Z aHti.+ L aHuk(11)

i-1 ax, ._1 a1 k-1 auk

The first two terms are equal with opposite signs as a result of thecanonical equations (2). When the optimal uk is unconstrained, all/aukvanishes. When at a constraint for a finite interval, however, we mightmove along the constraint surface with changing Uk if more than onedecision enters into the constraint. Thus, we cannot simply set &k tozero at a constraint. However, when the constraint U, is equal to zerofor a finite time, its time derivative must vanish, so that

U' I auk uk = 0 (12)k-1

Furthermore, if H is minimized subject to the constraint U, = 0, theLagrange multiplier rule requires that

D (duk- -X a-. 13)

or, substituting into Eq. (11),

Laukuk (14)H=_Xaukk-I

But from Eq. (12) this is equal to zero. Thus,

H = coast along optimal path (15a)

Furthermore, from Eq. (3),

0unspecified: H = 0 0<t<0 (15b)

We may summarize the necessary conditions as the weak minimumprinciple:

Given C[x(B)] and

Hi-1


where

aH aHz:=ayti ti:=-az;with 2n boundary conditions

o) = 0 m = 1, 2, . . . , i if

9.[z(O)) =0 s 1, 2, . . . S

IV

7; (0) 1 7.aq-az,

i = 1, 2, nm-t

117

Sas Lg.Ti(0)=W i= 1, 2, nax, az,

The decisions uk(t) which minimize 6[z(O)] subject to the restrictions

U,(u) > 0 p = 1, 2, . . . , P

make the hamiltonian H stationary when a constraint is not at equalityand minimize the hamiltonian (or make it stationary) with respectto constraints at equality or at nondifferentiable points. Along theoptimal path the hamiltonian is a constant, and if 0 is unspecified,that constant value is zero.

One final remark should be made. Since an integral is unaffectedby values of the integrand at discrete points, the arguments made hereand in Sec. 5.2 may fail to hold at a set of discrete points (in fact, overa set of measure zero). This in no way affects the application of theresults.

6.6 EQUIVALENT FORMULATIONS

In assuming that the objective function to be minimized depends onlyupon the state z(9) we have written the Mayer form of the optimizationproblem. In previous chapters we have been concerned with theLagrange form, in which an integral of a function of state and decisionis minimized. It is useful to establish that the latter problem, can betreated as a special case of the results of Sec. 6.5.

We consider the system

i, = f1(z,u) i = 1, 2, . . . , n (1)

with appropriately bounded decisions and end points, and we assume thatu is to be chosen in order to minimize

Io !Y(z,u) dt (2)

1!! OPTIMIZATION BY VARIATIONAL METHODS

By defining a new variable, say xo, such that

xo = 3(x,u) xo(0) = 0 (3)

we may write

&=foxodt=xo(9) (4)

Thus, numbering from k = 0 to n, the hamiltonian, Eq. (1) of Sec. 6.5,becomes

H = yog(x,u) + I 7kfk(x,u) (5)

k-1

withaHx fia7;

n (6a)

aH aT C afk-7o ax, - 4 7k ax,ax;k -1

rend

yo=all

-axo=

(6b)

0 (6c)

Furthermore, since xo(O) is completely free, the boundary condition att =.Bfor -yois

7o(9) = axo = 1 (7)

which, together with Eq. (6c), establishes that 7o is identically unity.tThe hamiltonian is therefore .

n

H=+ 7;f: (8)i-I

as in Chaps. 4 and S.It is also useful to consider cases .in which the independent varia-

ble t appears explicitly in the system equations or in the objective.Here we consider systems

zi = f;(x,u,t) i= 1, 2, . . . , n

and objectives

8 = 6[x(9),91.

(9)

(10)

t Any positive constant multiple of an integral of 3 could be minimized, so that yowill be any positive constant. An unstated regularity assumption, similar to the oneused for the Lagrange multiplier rule in Sec. 1.8, prevents y, from becoming zero,and hence there is no loss of generality in taking it to be unity.


The device which we use here is identical to that above.new variable xo such that

xo = 1 xo(O) = 0

in

We define a

in which case xo and t are identical. Equations (9) and (10) now dependexplicitly on xo, x1, . . . , xn but (formally) not on t. Thus, the resultsof Sec. 6.5 must apply, and we write

H=7o+ 7:f:

with

aHn

It-

a&

iYo=-axo -- 7;

0 unspecified7o(B) = axo a8

unspecified xo(O) = 0 specified

(12)

(13a)

(13b)

n

Clearly nothing is changed except that the sum yifi is not equal toi-1

a constant when some fi depend on t, since H = const and 7o varieswith t according to Eq. (13a).

6.7 AN APPLICATION WITH TRANSVERSALITY CONDITIONS

The essential improvement in the form of the weak minimum principledeveloped in the two preceding sections over shat used in Chap. 5 is thegeneralization of allowable boundary conditions for the state variableand the corresponding boundary conditions for the multipliers, or, as wenow prefer to think of them, Green's functions. These latter conditionsare usually referred to as transversality conditions. We can demonstratethe application of the transversality conditions by again considering thesimple second-order system of Sec. 5.3

where we now seek the minimum time control not to the origin but to acircle about the origin of radius R; that is, we impose the final condition

9[z(8)I = x12(8) + x22(0) - R2 = 0 t = 9

Using the equivalent formulation of Sec. 6.6, we have 5 = 1 for


minimum time control, and the hamiltonian is

H = 1 + 71x2 + y2u

where y, and 72 satisfy the canonical equations

yI =

12 =

aH0ax,

aH-axs= -7i

Thus, as previously,

y, = c, = consty2 = -c,t - C2

(2)

(3a)

(3b)

(4a)(4b)

and the optimal control is

u = -sgn 72 = sgn (c,t + c2) (5)

which can switch between extremes at most once. The allowable tra-jectories are again the parabolas

x, x22 + const (6)

Since the objective does not depend explicitly on x,(0) or x2(0),Eq. (12b) of Sec. 6.4 for the boundary conditions of the Green's func-tions becomes

0

11(0) =a$

+ `' ax, = 2yx,(e) (7a)

010

72(0) = $ + I, = 2vx2(0)axor, eliminating the unknown constant v,

yi(0) x:(0)1'2(0) - x2(0)

Thus, evaluating Eqs. (4) at time 0, Eq. (8) becomes

c, _ x,(0)- c,0 - c2 x,(0)

or-C1

C,= 0 + x2(8)/x,(0)

The optimal control, Eq. (5), then becomesr

u =sgn {c2 L1 -_

0 + x2(0)/x1(0)

(7b)

(8)

(9)

(10)


When x2(8) and x1(8) have the same algebraic sign (the first and-third quadrants), the algebraic sign of the argument in Eq. (11) neverchanges (t < 8 and therefore t/[8 + x2(8)/x1(8)] < 1). Thus, all tra-jectories ending on the circle of radius R in the first and third quadrantmust do so without switching between control extremes and must con-sist of one of the parabolas defined by Eq. (6). Inspection of Fig. 5.4clearly indicates that for approaches from outside the circle the optimumis

u= -I x1(8)>0 (12a)x2(8) > 0

U = +1 x1(8) < 0(12b)

X2(0) < 0

For trajectories ending in the fourth quadrant, where x1(8) < 0,x2(0) > 0, there is a switch in the optimal control at a value

1. = 8-}-x2(6) <8 (13)x1(8)

where the argument of the signum function in Eq. (11) passes throughzero. Here the optimal trajectories, which must fi ish with controlu = - 1 in order to intersect the circle (we assume R < 1 to avoid thepossibility of intersection along a line u = +1), can have utilizedu = -1 for a time equal at most to

-=-x1(0)B t, x2(0) > 0 (14)

prior to which the control must have been u = +1. The point on theparabolic trajectory ending at x1(8), x2(8) corresponding to a time0 - t. units earlier is

xl(t.) = R2 - 1x22(8)

x1(0) 2 x12(8)

x2(t.) = x2(0) [x1(8) - 1]x1(0)

which then defines the switching curve as x1(8) runs from zero to -R,x2(0) from R to zero. The switching curve in the second quadrant issimilarly constructed, giving the entire feedback policy.

.

6.8 THE STRONG MINIMUM PRINCIPLE

We have called the necessary conditions derived thus far the weakminimum principle because, with the experience we have now developedand little more effort, a stronger result can be obtained. Specifically,


it is possible to prove that the optimal function u(t) not only makes thehamiltonian stationary but that it makes the hamiltonian an absoluteminimum. This latter result is of importance in some applications.

The essential part of the derivation of the weak minimum principlewas in obtaining an infinitesimal change in x(®) and relating this changeto the hamiltonian and to the infinitesimal change in u for all t. It ispossible to obtain an infinitesimal change in x(O) (and thus S) by makinga change in u of finite magnitude if that change is made over a suffi-ciently small time interval, and the hamiltonian enters in a different way.

Let us suppose that the optimal decision function u(t) and optimalstate vector !(t) are available and that at time t we have effected achange in the state such that

x(tl) = !(t1) + &x(tl) 1sx(tl)j < E (1)

If in the interval t1 < t < 8 we employ the optimal decision 6(t), wemay write

xi = f,(x,u) = fi(2,6) + ,I1 ax, Sx, + 0(t) (2a)

or

Sz; = I ax` ax; + o(e) t, < t < 0 (2b)

s1The equation is of this form because Bu = 0, t1 < t < B. Green'sidentity then becomes

1 70) Sxi(0) = 7i(tl) axi(tl) + 0(E) (3)i-1 i-1

where 7 satisfies the equationsn

'yil ax,-1(4)

Next we presume that for all time earlier than t, - A we makeno changes at all. Thus, x(ti 1(tl 1A), or

5x(t, - A) = 0 (5)

During the interval ti - A < t < t1 we set

u(t) = u(t) + su t1 - A < t < t1

where 1bul is finite. It follows then by direct integration that

(6)

e

x; (t) = ±101 - ) + 1 _o f, (X, u + Su) at t, - A < t 5 t1


while the optimal values may be written

x;(t) = x;(tt - °) + J=' f,(!,u) dt t, - n < t < tl (8)

The variation in the state is then

bx;(t) = x;(t) - -ti(t) = Jto [f;(x, u + &u) - f,(2,u) dttl-°<t<tt

and we may note a useful bound on its magnitude,

lbx,(t)l < max If,(x, u + 5u) - f;(x,u)I(t - t, + °)a-e<t<a

(9)

t,-°<t<t, (10)We are now able to relate the change in the objective to the finite

change 6u. Truncating a Taylor series after one term, we write.

MX, u + &u) = M2, u + &u) + ofi Sx; (11)

ill 8x;

where the partial derivatives are evaluated somewhere between x andx. Because Eq. (10) establishes Sx as being of the same order as °,any integral of fix over an interval ° must be of order °2. Thus

((t,n

CifiJt,-o I' ax

bx; dt = o(°) (12);1

and Eq. (9) becomes, at t = ti,

dx;(ti) =J"

t [f (x, u + Su) - f;(x,u)) dt+ o(i) (13)

Furthermore, if the region t, - ° < t < ti includes no points of dis-continuity of u, the integrand is continuous and we can write the exactrelationship as

bx;(ti) = [f;(x, u + &u) - f;(x,u)J ° + o(°) (k4)

and Eq. (3) asn n

y,(o) bx;(B) u + &x) - y;f;(x,u)J t_t, ° + o(°) (15)

By imposing on y the boundary conditions of Eq. (12) of Sec. 6.4, Eq.(15) becomes

at; _ [y;f,(x, (1 + &u) - y,f;(x,u)] It-t, ° + o(°) (16),_1

Mr OPTIMIZATION BY VARIATIONAL METHODS

relating the finite change in u to the corresponding infinitesimal changein &.

The condition of optimality, 5& > 0, requiresn

A 'Yrf;(=, u + Su) + o(n) >_ n 'Y;f.(=,G) t = tl (17)

or, dividing by the positive number A and taking the limit as A - 0,n

rf,(!, u + Su) > y.f:(=,u) (18)

where tj is any point of continuity of ft. The sum in Eq. (18) is thehamiltonian

n

I tirf; (19);-i

and so we have established that:

The hamiltonian takes on its absolute minimum value when evaluatedfor the optimal decision function u(t).

This result is significantly stronger than the condition that the hamil-tonian be a minimum at boundaries and simply stationary for interiorvalues of the optimal decision, and we call this, together with all theconditions of Sec. 6.5, the strong minimum principle.

63 THE STRONG MINIMUM PRINCIPLE: A SECOND DERIVATION_

Before applying the strong minimum principle to some practical prob-lems, it is useful for future comparisons to consider a second derivationof the strong minimum principle based on Picard's method of the solu-tion of ordinary differential equations. We return to the idea of a con-tinuous infinitesimal variation Su(t), ISul < E, 0 < t < B, and, as in Sec.6.3, we write the equations describing the -corresponding infinitesimalvariation fix. Now, however, we retain terms to the second order in e

z

5z; _axj 5x; +

auk Suk +i ax,

8xk Ex, Sxkj-1 k-1 k-1

n R a2f, 1 R azf

+ I az; auk6x; 6uk +

j au; ,5uj Suk + o(e2) (1)ou,j-1 k-1 .k-1

The essence of Picard's method is first to integrate Eq. (1) by con-sidering the second-order terms as nonhomogeneous forcing terms in alinear equation. In this light we introduce the appropriate Green's func-


tions r;k(t,rr) as defined by Eq. (12) of Sec. 6.2

\G r;k (t,T)

azkk-1 j

rij(t,t) = aij =1

0ii

= j= j

Then Green's identity becomes

az;(t) = r;,(t,0) 8xj(0) + Ia r;j(t,s) auk auk dsj-1 j-1 k-1

Cn R

I+o

+ / r;j(t,s)Zf

but &ul dsauk butj-1 k,1-1

1 /= I a2f+ 2 o

r;j(t's)azk azl

bxk dxt dsj,k1

1!S

. (2a)

(2b)

rt n R a2fj+0

r;j(t,s)OXk aul

axk aul ds + o(e2) 0 < t < 9j,k--I

1

which is an integral equation for &x. We next substitute 6x(t) from Eq.(3) into the last two terms of Eq. (3). For simplicity we set &x(0) tozero, either as a special variation or because of fixed initial values, andwe obtain the explicit representation at t = 0,

6X, (0) =

j-1 kJ-1n

()Ra2j'.

+ jo 1 1 r;j(e,8) auk ax, auk1 k,r-1n R

fo rl,,,(s,v)our

au, do I ds

+ 2 Jo I I r,,(o,s) azk2alxlj,k,l,m,r-11 r,w-1

fo' rkrn(s,a) aum our da] ko rln(s,r) a , 6u dr 1 ds + o(E2) (4)

where all products of variations higher than the second have beenincluded in o(e2).

We now make a special choice of the variation Bu(t). As we havealready established that the hamiltonian is a minimum for noninteriorvalues of the optimal decision, we set au; to zero whenever the corre-sponding decision u; does not correspond to a stationary value of the

0n R af

Jo11 r;j(o,8) aL. auk ds

i-1 k-In R

c 2j

+ 2 Io 11 r`j(B's).au Jau, auk 6u, ds

1% OPTIMIZATION IJY VARIATIONAL METHODS

hamiltonian. Furthermore, we allow Su to be nonzero only over aninfinitesimal interval t1 - A < t < tI

OH

bu,(t) =0

au,0

y,(t)F6 0 t1-A<t<t10 otherwise

(5)

In that case each of the first two terms in Eq. (4) is a single integral andhence of order A, while the third and fourth terms involve products ofintegrals containing Su and are of order W. Thus,

n R

bx;(O) = f I I r,,(o,s) auk yk dsi-1 k-1

n R

rij(B,s) azf' ykyl ds + o(E2) + o(0) (6)+ 2 Je,! o auk au,Jtj-2 k,l-1

If we multiply Eqs. (2) and (6) by a constant, y;(8), and sum over i = 1to n, then, as in Sec. 6.2, we define a vector ?(t) satisfying

n

and

ax;yji-I

0<t<e

,(B) axi(e) °I, o yi auk yk ds

i-1 i-1k-1

n R

+ 2 I°` n yi auza'ut ykyi ds + o(E2) + o(A) (7)-1 k.l-1

We shall assume 0 to be fixed for simplicity. The variation in t;caused by the change fix(O) is then, to second order,

bs axj'bxi(e)ax)(e) + o(f2) (8)= GI ax & (0) + 2 8xi2

But, from Eq. (3),

6xi(8) bxj(@) = 0(0) + o(E2)

so that

bS =as

bxi (9)ax:

+ O (A) +o (e2)

(9)

(10)i-1


Thus, if for simplicity we assume x(8) to be unspecified and write

asti: (e) =

ate;

-197

and introduce the hamiltonian, the combination of Eqs. (10), (11), and(7) becomes

naH 1

a2Has = f 1 y' ds + 21=,-o G au; au; ysY; ds + o(A)

:-1(12)

The first integral vanishes by virtue of the arguments used in thederivation of the weak minimum principle. The nontiegativity of thevariation 53 requires that

na2H

(13)aau; yiy' ? 0t,j-1

for arbitrary (small) y and, in fact, equality in Eq. (13) is an exceptionalcase which oannot occur when the minimum is taken on at discretepoints, so that Eq. (13), taken together with the weak minimum prin-ciple, implies that the hamiltonian is a (local) minimum. If equalityshou'd obtain in Eq. (13), a consideration of higher-order terms wouldlead to the same conclusion.

This derivation leads to weaker results than that of the previoussection. Rather than establishing, as in Sec. 6.8, that the minimizingfunction u(t) causes the hamiltonian to take on its absolute minimum,we have shown here only that the hamiltonian is a local minimum.The extension to constrained end points and variable 0 is straightforward.The particular usefulness of the somewhat weaker result of this sectionwill become apparent in considering sufficiency and in the discussionof discrete systems in the next chapter.


In Sec. 4.12 we applied the weak minimum principle to the problem.of determining the optimal temperature program for the consecutive-reaction scheme

X1-> X2 - decomposition products

We return to that problem now to demonstrate the usefulness of thestrong principle and to impose a limitation on the solution we haveobtained.


The system is described by equations

±1 = k,(u)F(xi) (la)x2 = vk,(u)F(x,) - k,(u)G(x2) (1b)

where F and G must be positive functions for physical reasons. and k,and k2 have the form

kr(u) = k,oe-s,'Iu i = 1, 2

The goal is to maximize x,(9) or, equivalently, to minimizewith x,(0) unspecified. Thus, the hamiltonian is

H = - y,k,F + y2vk1F - y2k2G

with multiplier equations and boundary conditions

y, aH = (yi - 71(9) = 0

'Y

aaH= y2k2G' 72(e) _ - 1

(3)

(4a)

(4b)

Equation (4b) may be integrated to give the important result

y,(t) = - exp (,e k2G' d8) < 0 (5)

If we assume that the optimal temperature function u(t) is uncon-strained, the conditions for $ minimum are

aH8u =

_ y,k'F + ysrk,F - y2k'G = 0 (6)

a2H ki'F - ks G > 0k''F += - (7)y3r yayiaus

Equation (6) can be solved for y, as

k'GIt = P72 - 72 2 (8)k jp

in which case Eq. (7) becomes/k',k

,.y2G ( kiks > 0 (9)

.or, making` use of the negativity of y2 and the positivity of G, a necessarycondition for optimality is

k''ksk='<0 (10)k'


Now, from the defining equation (2) it follows that

k' = k:oE;u2

= k,oE;k`a

a_B,,Iu - 2k.oE;e-se u

U4u,

so that Eq. (10) becomes, after some simplification,

k'oE= a-$j tu(E, - Es) S 0U4

in

(1lb)

(12)

or, eliminating the positive coefficient,

E1 < E2 (13)

Thattis, the solution derived in Sec. 4.12 is optimal only when E1 < .E2.When El > E2, the optimum is the highest possible temperature.

6.11 OPTIMALITY OF THE STEADY STATE

It has been found recently that certain reaction and separation processeswhich are normally operated in the steady state give improved perfor-mance when deliberately forced to operate in a time-varying manner, wherethe measure of performance is taken to be a time-averaged quantity suchas conversion. Horn has shown a straightforward procedure for deter-mining under i erhh.in conditions when improved performance mA.y heexpected in the unsteady state by the obvious but profound observationthat if the problem is posed as one of finding the time-varying operatingconditions to optimize a time-averaged quantity, the best steady statecan be optimal only if it satisfies the necessary conditions for the time-varying problem. As we shall see, the application of this principlerequires the strong minimum principle.

We shall consider an example of Horn and Lin.of parallel chemicalreactions carried out in a continuous-flow stirred-tank reactor. Thereactions are

X2

X1

X,

where the reaction X1 -+ X2 is of order n and the reaction X1--' X,is of first order. X2 is the desired product, and the goal is to choosethe operating temperature which will maximize the amount of X2. The


reactor equations are

z1 = -ux1n - aurxi - xl + 1 (1a)x2 = uxln - x2 (lb)

where x1 and X2 are dimensionless concentrations, r is the ratio of acti-vation energies of the second reaction to the first, u is the decision varia-ble, the temperature-dependent rate coefficient of the first reaction, .andtime is dimensionless.

For steady-state operation the time derivatives are zero, and weobtain a single equation in x2 and u by solving Eq. (16) and substitutinginto (la)

0 = -x2 - xz'In(aur-tin + U -1/n) + 1 (2)

As we wish to maximize x2 by choice of u, we differentiate Eq. (2) withrespect to u to obtain

0 = - ax; - 1x21Jn-1

ax2(aur-1'n + u- 1/n)au n au

- xz1/n

a

nr - 1 1 u-0441thl (3)\ n n

and since 49x2/(9u must be zero at an. internal maximum, solving for uleads to the optimum steady-state value

=a(nr 1- 1)JI,r

(4)

Clearly this requires

nr - 1 > 0 (5)

To establish that Eq. (4) does indeed lead to a maximum we take thesecond derivative of Eq. (2) with respect to u to obtain, after using Eq.(4) and the vanishing of ax2/au,

/0

z

dug t l + x21/n-lu l/nnr r 1) - x21)n u-(I+2n)/n(6)

Since u and x2 are both positive, it follows at once that (92x2/au2 is nega-tive and that a maximum is obtained.

If we formulate the problem as a dynamical one, our goal is tochoose u(t) in the interval 0 < t < 0 in order to maximize the time-average value of x2(t) or, equivalently, minimize the negative of thetime-average value. Thus, i

aJ = - e f x2(t) dt (7)


or, using the equivalent formulation which we have developed,

8(8)

Using Eqs. (la) and (lb), the hamiltonian is then

H = - B + y1(-uxl" - au'xl - xl + 1) + y2(uxi" - x2) (9)

The multiplier equations are

yi = -all

= 71(nuxl°-' + au' + 1) - ny2ux1"-' (10a)axlaH 1

12 = + 72(IX 2 B

and the partial derivatives of H with respect to u are

au = -ylxx^ - ay1ru'-'xl + 72xln02 H

au2 = -ay,r(r - 1)u'-2x,

(10b)

(12)

Since we are testing the steady state for optimality, it follows thatall time derivatives must vanish and the Green's functions y, and y2 mustalso be constant. Thus, from Eqs. (10),

nuxln-'y

1y2=-e

(13a)

(13b)

(Clearly we are excluding small initial and final transients here.) SettingaH/au to zero in Eq. (11) and using Eq. (13) leads immediately to thesolution

[anr'_ 1) (14)

the optimal steady state, so that the optimal steady state does satisfy thefirst necessary condition for dynamic optimality. However, it followsfrom Eq. (13a) that y, is always negative, so that a2H/au2 in Eq. (12)has the algebraic sign of r - 1. Thus, when r > 1, the second deriva-tive is positive and the hamiltonian is a minimum, as required. Whenr < 1, however, the second derivative is negative and the hamiltonian isa local maximum. Thus, for r < 1 the best steady-state operation canalways be improved upon by dynamic operation.

We shall return to this problem in a later chapter with regard to the


computational details of obtaining a dynamic operating policy. We leaveas a suggestion to the interested reader the fruitfulness of investigatingthe relation between steady-state optimality and minimization of thesteady-state lagrangian, with emphasis on the meaning of steady-stateLagrange multipliers.

6.12 OPTIMAL OPERATION OF A CATALYTIC REFORMER

One of the interesting applications of the results of this chapter to anindustrial process has been Pollock's recent preliminary study of theoperation of a catalytic reformer. The reforming process takes a feed-stock of hydrocarbons of low octane number and carries out a dehydro-genation over a platinum-halide catalyst to higher octane product. Atypical reaction is the conversion of cyclohexane, with octane number of77, to benzene, with an octane number of over 100.

A simple diagram of the reformer is shown in Fig. 6.1. The feed iscombined with a hydrogen recycle gas stream and heated to about 900°F,then passed into the first reactor. The dehydrogenation reaction is endo-thermic, and the effluent stream from the first reactor is at about 800°F.This is teheated and passedto the second reactor, where the temperaturedrop is typically half that in the first reactor. The stream is again heatedand passed through the final reactor, where the temperature varies onlyslightly and may even rise. Finally, the stream is passed to a separatorfor recovery of hydrogen gas, and the liquid is debutanized to make the

Hydrogenproduct

FIg. 6.1 Schematic of a catalytic reforming process. (Courtesy of A. W.Pollock.)0


reformate product, which may have an octane number as much as 60greater than the feed.

Numerous side reactions deposit coke on the catalyst, reducing itsefficiency, and it becomes necessary eventually to shut down the reactorand regenerate the catalyst. The problem posed by Pollock, then, wasto determine how to operate the reformer and when to shut down inorder to maximize the profit over the total period, including both oper-ating time and downtime.

The decision variable is taken by means of a simplified model to bethe desired octane number of the product, which is adjusted by anoperator. For convenience we define u as the octane number less 60.Pollock has used estimates from plant and literature data to write theequation for coke accumulation, the single state variable, as

it = b2uO xx(0) = 0 (1)

and the time-average profit as proportional to

W e + r fo (B + u)[1 - (N + b,x,)u2] dt - Q +A9(2)

Here r is the time needed for regeneration, B + u the difference in valuebetween product and feed divided by the (constant) marginal return forincreased octane, 1 - (bo + bix,)u2 the fractional yield of product, Qthe fixed cost of regeneration divided by flow rate and marginal return,and A the difference in value between other reaction products and feeddivided by marginal return. We refer the reader to the original paperfor the construction of the model, which is at best very approximate.The following values of fixed parameters were used for numerical studies:

bo=10-' b,=2X10-8b2=35-a+, r=2

The objective equation (2) is not in the form which we have used,and to convert it we require two additional state variables, defined as

x2 = (B + u)[1 - (bo + b,x,)u2] x2(0) = 0 (3a)xa = 1 x3(0) = 0 (3b)

In that case we can express the objective as one of minimizing

= Q - Ax3(9) - x2(e) (4)xa(a) + r

The hamiltonian is then

H = ylb2u5 + y2(B + u)[1 - (bo + blxi)u2] + ya (5)



aH = -biu2(B + u)y2 710) = - = 0 (6a)aH _ 0 as 1

72 -ax2 y2(0) =axe xa(e) + r (6b)aH as _ Q - Ar - x2(9)

ye = 0 73(9) =2

(6c)-axe axe Ix:(B) rJ

It is convenient to define

Z = yi(O + r)

which, together with Eqs. (3), (5), and (6), leads to

H = -9

+ r'Zb2ua + (B + u)[1 - (bo + bixi)u2l

+Q-Ar-x2(9)16+r J

2 = b,u2(B + u) Z(a) = 0

(7)

(8)

(9)

Eliminating constant terms, minimizing the hamiltonian is equivalentto maximizing

H* = Zb2u' li- (B + u)[1 - (bo + bixl)u2] (10)

The optimal 9 is determined by the vanishing of the hamiltonian

(a + r)(B + u)[1 - (bo + bIx1)u2]= fa (B + u)[1 - (bo + blxl)u2J dt - (Q - Ar) t = 9 (11)

Because of the preliminary nature of the model and the uncertaintyof the parameters, particularly A, B, Q, and S, a full study was notdeemed warranted. Instead, in order to obtain some idea of the natureof the optimal policy, the value of Z at t = 0 was arbitrarily set at-0.1. Equations (1) and (9) were then integrated numerically fromt 0, choosing u at each time instant by maximizing Eq. (10), subjectto constraints

30 < u < 40 (12)

(octane number between 90 and 100). 9 was determined by the pointat which Z reached zero. Equation (11) then determined the values ofQ and A for which the policy was optimal. A complete study wouldrequire a one-parameter search over all values Z(0) to obtain optimalpolicies for all Q and A. Figures 6.2, 6.3, and 6.4 show the optimalpolicies for 0 = 1, 2, and 4, respectively, with values of B from 30 to

THE MINIMUM PRINCIPLE Zos

R°I 83040

50

905 10 15 20 25 30 35 40 45

t

Fig. 6.2 Optimum target octane schedule for 15 - 1.(Courtesy of A. W. Pollock.)

90. The near constancy of the policies for $ = 2 is of particular interest,for conventional industrial practice is to operate at constant targetoctane.

The remarkable agreement for some values of the parametersbetween Pollock's preliminary investigation of optimal policies andindustrial practice suggests further study. We consider, for example,the possibility that constant octane might sometimes represent a rigor-ously optimal policy. Differentiating H* in Eq. (10) with respect tou and setting the derivative to zero for an internal maximum yields,after some rearrangement,

Nb2_-1 Z B - bol(B + 2)u2 + 2Bu]1 _ +(x 13)

2 2b1[(B + 2)u + 2Bu] b1[(B + 2)u + 2Bu]

80 100

2 30U

9840

E 96E 60

9470

92 0

O 90900 5 10 15 20'. '25 30 35 40 45

Fig. 6.3 Optimum target octane schedule for l4 s 2.(Courtesy of A. W. Pollock.)


f

Fig. 6.4 Optimum target octane schedule for B a 4. (Cour-tesy of A. W. Pollock.)

Now, if u is a constant, then differentiating Eq. (13) with respect to time

$b2u8-'x' b,[(B + 2)u2 + 2Bu) Z

(14)

and combining this result with Eqs. (1) and (9), we obtain a value for u

u B($-2) (15)B+2-$We test for a maximum of H* by calculating the second derivative

0211 *

8u2-2(bo + bixi)[(B + 2)u + B] + Zb20($ - OUP-2 (16)

Since Z < 0, it follows that we always have a maximum for u > 0,or, from Eq. (15),

2<0<B+2 (17)

Equation (11) is easily shown to be satisfied for positive 0.For S near 2 and large B (small marginal return), the rigorously

constant policy corresponds to very small u, which is verified by examina-tion of Fig. 6.3. For small values of B (B near R - 2) constant valuesof u of the order of 35 to 40 can be accommodated. For very large$ ($ > 10) and large B corresponding large values of u are also obtained.It is clear from Figs. 6.2 to.6.4 that only minor changes in shape occurover wide ranges of B,, so that essentially constant policies will generallybe expected. Thus, we conclude that within the accuracy of Pollock'smodel the industrial practice of constant-octane operation is close tooptimal for a significant range of parameters.


6.13 THE WEIERSTRASS CONDITION

The minimum principle is a generalization of a well-known necessarycondition of Weierstrass in the calculus of variations. In Sec. 4.4 weconsidered the problem of finding the function x(t) which minimized theintegral

& = Io Y(x,.t,t) dt (1)

where the hamiltonian was found to be

H=iF+x1x-1-X2 (2)

and l is the decision variable. The minimum condition is then

V2,14) + X1± < 5(2,±,t) + X1± (3)

where x denotes the optimal function. From the stationary conditionEq. (7) of Sec. 4.4,

(4)

or

iT(211,t) - ff(z,x;t) ax (I - x) < 0 (5)

which is the Weierstrass inequality.The somewhat weaker condition that the second derivative of the

hamiltonian be positive for a minimum becomes, here,

025 >TX - 0(6)

which is known as the Legendre condition. Both these results are gener-alized to situations with differential-equation side conditions.

6.14 NECESSARY CONDITION FOR SINGULAR SOLUTIONS

In Chap. 5 we considered several applications in which a portion of theoptimal solution lay along a singular curve. . These were internal optimain problems in which the decision appeared linearly, so that first deriva-tives of the hamiltonian were independent of the decision. Consequently

'the second-derivative test for a minimum cannot be applied, and differ-ent conditions are needed. To this end we return to the analysis inSec. 6.9.

Our starting point is Eq. (3) of Sec. 6.9, which we multiply by y;(9)

an OPTIMIZATION BY VARIATIONAL METHODS

and sum over i to obtain

as - 1 o'; a2H axk axj dt + f ° I " axk au dto k axk axj o k axk au

+ 0 (1)

Here we have assumed a single decision variable and made use of thefact that all/au is zero for an optimum and a2H/au2 vanishes for singularsolutions. fix(O) was taken to be zero, and we have deleted the term

a2Saxj(9) axk(e)

axj axk

leaving to the reader the task of demonstrating that for the special vari-ation which we shall choose this term is of negligible magnitude comparedto the terms retained. We now assume the symmetric special variation

au = +E to < t < to + 0 (2)-E to - A<t < to

in which case both integrals in Eq. (1) need be evaluated only over to -A < t < to + O to within the specified order. . Now we expand the inte-grands in Taylor series about their values at to as follows:

a:H axkaxj = a:H + d a2H (t -to) + CAafkax-' ax, [ax. ozj I ax-" OZj au

(CA 0) + CA L'IOU au) (t

+ (CA afjaf'eafk! t - to)+ (3a)au.- au

aHaxkau fe aH. +d: aH (t-to)+ Eoafk

axk au [axk au dt axle au ] [ au

+ 1 eG1 axj au auk) (t - to) + d ED j iclu

f t a) (t 21to) + (3b)

Here all derivatives are evaluated at to, and ax has been calculated fromthe equation

dik = akkax;+au 6U41I f

(4)

The symbol ± denotes + for t > to, - fort < to.


Substituting Eqs. (3) into Eq. (1) and integrating, we obtainE2A3 _a2H af, af,

L ..

+a2H (d af;

683 ax; ax; au au axi au dt

_ af; af;au ` L ax, au

_ (d a2H.) al;] +v o(E203) >'0 {5}

dt ax; au au

in which case the term in square brackets must be nonnegative. Thismay be shown to be equivalent to the statement

a d2 aH <0au dt2 au

(6)

which is, in fact, a special case of the more general relation which weshall not derive

(-1) a(d2k OH)

? 0VU d12k auk = 0, 1, 2, . . . (7)

Consider, for example, the batch- and tubular-reactor temperature-profile problem of Sec. 5.11. The hamiltonian is.

H = p - r(a,T)(1 + X1 - JX2) - X2u (8)

where a and T are the state variables, and the singular arc correspondsto X2 = 0, ar/aT = 0, a2r/aT2 < 0, the line of maximum reaction rate.The state equations are

a = -r(a,T) (9a)T = Jr(a,T) - u (9b)

so that the bracketed term in Eq. (5) becomes

- aT22

(1 + X1 - JX2) > 0 (10)

But X2 = 0, a2r/aT2 < 0, and since the singular are has been shown. tobe a final arc with \,(O) = 0, it easily follows from Eq. (6a) of Sec. 5.11that a, > 0. Thus Eq. (10), is satisfied, and the singular are doessatisfy the further necessary condition.

6.15 MIXED CONSTRAINTS

Our considerations thus far have been limited to processes in which onlythe decision variables are constrained during operation. In fact, itmay be necessary to include limitations on state variables or combina-tions of state and decision variables. The latter problem is easier, andwe shall consider it first.

We suppose that the minimization problem is as described in


Sec. 6.4 but with added restrictions of the form

Qk(x,u) >_ 0 k = 1, 2, ... , K (1)

When every constraint is at strict inequality, Eq. (1) places no restric-tions on variations in the optimal decision, and the minimum principleas derived in Sec. 6.8 clearly applies. We need only ask, then, whatchanges are required when one or more of the constraints is at equalityfor a finite interval.

It is instructive to consider first the case in which there is a singledecision variable u and only one constraint is at equality. Droppingthe subscript on the constraint, we then have

Q(x,u) = 0 (2)

If the state is specified, Eq. (2) determines the decision.in x or u must satisfy, to first order,

I ayax` +

a6l au = 0bQ =G ax: au:-1

while, since the differential equations must be satisfied,

5x;= aiax1+a`bu

-1

Substituting for bu from Eq. (3),

f 1 aQlax; - au (au) ax;, bx;

Any change

(3)

(4)

(5)

Thus, using the results of Sec. 6.2, along this section of the optimalpath the Green's vector must satisfy the equation

1 aa af, f a-Q Q

y' - ax; au au ax; y'(6)

Although the canonical equations are not valid over the constrainedpart of the trajectory we define the hamiltoniai as before, and it iseasily shown that H is constant over the constrained section. If con-tinuity of H is required over the entrance to the constrained section,the multipliers are continuous and H retains the same constant valuethroughout and, in particular, H = 0 when 0 is unspecified.

In the general case of K1 independent constraints at equalityinvolving R1 >_ K1 components of the decision vector we solve the K1


equations

aQkk sui + I aQk axi = 0 k = 1, 2, . . . , Kl (7)aui

for the first K1 components Sui. (It is assumed that aQk/aui is of rankK1.) The equation for the Green's vector is then

af; - Ki8J aQk

is = Spk - 1'i (8)- axi P.I au, Ox,

Here Sk is the inverse of the matrix consisting of the first Kl elementsoQk/aui; that is,

KI

k-iZ S,k auk = a;, i,j=1,2,...Kl (9)

Upon substitution into the variational equations it follows that theR, - K, independent components of u must be chosen to satisfy theweak minimum principle and that H is again constant.

6.16 STATE-VARIABLE CONSTRAINTS

We are now in a position to consider constraints on state variablesonly. Clearly we need only consider the time intervals when suchconstraints are actually at equality. For simplicity we consider hereonly a single active constraint

Q(x) = 0 (1)

The generalization is straightforward.Let us consider an interval tl < t < t2 during which the optimal

trajectory lies along the constraint. Since Eq. (1) holds as an identity,we can differentiate it with respect to t as often as we wish to obtain

Q Q(-) =0 ti<t<t2 (2)

In particular, we assume that the decision vector u first enters explicitlyin the mth derivative. If we then let

Q(m) (x) = Q(x,u) = 0 t, < t < is (3)

we can apply the results of the previous section to the mixed constraint0 in order to obtain the multiplier equations. In addition, however,we now have m additional constraints

QU)=0 j=0,1,2,...,m-1 (4)

which must be satisfied at t = tl.


If we apply Green's identity across the vanishingly small intervalti < t < t1+, we obtain

n n

I yi ax:t. -i-1 i-1

(5)

The Green's vector y(ti-) may be expressed as the sum of m + 1 vectors,the first in being normal to the surfaces defined by Eq. (4). Thus

m-1 aQ(i)yi(t1 ) _ µi ax; + ai (6)

ioor, from Eq. (5),

m-1n

aQ(j) n n

G G µ, a.'xi + 1 Qi ax; _ y; ax; isi-Oi-1 i-I i-1

(7)

But the constraints imply thatn

1aQci)6Q(j)

= axi = 0 j = 0, 1,- (8)8x;

i-1

so that Eq. (7) becomesn n

I Gi axi Its- _ I yi axi It1+ (9)i-1 i-1

Since ax is continuous, Eq. (9) has a solution

d = y(t1+) (10)

and the Green's vector may then have a discontinuity of the formn<-1

a)ye(ti) yi(tl+) + F1i

Qi(11)

C1X-o

This jump con&ion is unique if we require that ?(t2) be continuous, andthe m components of i are found from the m extra conditions of Eq. (4).As before, the hamiltonian has a continuous constant value along theoptimal path.

6.17 CONTROL WITH INERTIA

In all the control problems we have studied we have assumed that instan-taneous switching is possible between extremes. A more realistic approx-imation might be an upper limit on the rate of change of a control setting.Consider again, for example, the optimal-cooling problem of Sec. 5.11.


The state equations are

a = -r(a,T) (la)T = Jr(a,T) - q (lb)

where we have written q for the cooling rate. Now we wish to boundthe rate of change of q, and so we write a third equation

¢ = u (1c)

with constraints

U* < u < u* (2a)0 < q < q* (2b)

and perhaps a temperature constraint

T* <T <T* (2c)

The objective is again to minimize

& = f o' - r(a,T)] dt (3)

We may consider. u to be the decision variable, in which case(2b) arld (2c) denote constraints on the state. The hafniltonian is

H = -'Yir+y2(Jr-q)+yau+p-r

Eqs.

(4)

and if neither constraints (2b) nor (2c) are in effect, the multiplier equa-tions are

7r

tie

_ aH Or'(1+Yr-J72)

as aa

_ aH Or

aT aT(l + yt - Jy2)

(5a)

(5b)

aHYa = -

aq= y2 (5c)

If control is not intermediate, u lies at one of its extremes, so that theoptimum is to change controls as fast as possible. If the constraint sur-gface q = 0 or q = q* is reached, the control appears in the first derivativeand m =.1:

Q(I)= (q.*-q)u=0a-t (6)

Since Q is independent of x, the multiplirr equations are unchanged andwe simply reduce to the problem of Sec. 5.11.

Intermediate control is possible only if ya = 0, which, from Eq.(k), implies 72 = 0. But y2 vanishes only along the line of maximum

!U OPTIMIZATION BY VARIATIONAL METHODS

reaction rate, so that the same intermediate policy is optimal. If a tem-perature constraint is reached, say T = T*, we have

Qu> = dt(T* - T) q - Jr(a,T*) = 0 (7a)

(7b)e(=)

= dt

so that m = 2 and the coolant is varied at a rate

8r(a,T*)' )u = Jr(a,T8a

(8)

It will be impossible to satisfy equality constraints on both temperatureand rate of cooling, as seen by comparing Eqs: (6) and (8), and so someinitial states will be excluded.

We wish to emphasize here the manner in which we have developedthe optimal policy for this problem. In See. 3.5 we studied the optimaltemperature policy. We found in Sec. 5.11 that, when possible, theoptimalSoolant policy reduces to one of choosing the optimal tempera-ture policy. Here we have found that, except when switching, the opti-mal coolant rate of change is one which gives the optimal coolant policy.This hierarchy of optimization problems is typical of applications, andthe physical understanding eliminates some of the need for rigorousestablishment of optimality of intermediate policies.

5.18 DISCONTINUOUS MULTIPLIERS

One of the most striking features of the presence of state-variable con-straints is the'possibility of a discontinuity in the Green's functions. Weshall demonstrate this by a problem of the utmost simplicity, a first-order process,

±1 = U (1)

We assume that xl must he within the region bounded by the straight lines

Q1=xi-(ai--$it)>0Q2 = -xi + (a= - s=t) > 0

and that we choose u to minimize

5--210 (x1'+c'u')dl

while obtaining xi(8) = 0. The reader

(2a)

(2b)

(3)

seeking a physical motivation


might think of this as a power-limited mixing process in which the con-straints are blending restrictions.

It is convenient to define a new state variable

x2 = 3 (x12 + c2u2) x2(0) = 0 (4)

and, because time enters explicitly into the constraints

za = 1 x3(0) = 0 x3(8) = 0 (5)

We then have

g = x2(0)

Q1 = x1 + $1x3 - al 1 0Q2 = -x1 - $2x3 + a2 0

The hamiltonian is

H = Y1u + i2Y2(x12 + c2u2) + 73

and, along unconstrained sections of the trajectory, ,

tit = - Y2x1

y2 = 0 Y2(8)

'Y3=0

(6)(7a)(7b)

(g)

Furthermore, along unconstrained sections the optimal path is defined by

c2x1-xl=0 (12)

From the time t1, when xl intersects the constraint Q1 = 0, until leav-ing at t2, we have

1=0=u+$1 t1 <t<t2 (13)

and since this is independent of x, the multiplier equations are unchanged.In particular, 72 and Y3 are constants with possible jumps at intersectionsof the free curve and the constraint.

The jump condition, Eq. (11) of Sec. 6.16, requires that

Y1(ti) = Y1(t1+) + µ (14a)Y2(t1-) = Y2(t1+) (14b)7301) = Y3(tl+) + µ01 (14c)

and together with the continuity of the hamiltonian it follows that thecontrol is continuous

u(tl) = -$1 = u(t1+) (15)


In the interval 0 < t < t, Eq. (12) must be satisfied, and the solution with

is

(16)

+ XI(0)e_ 1c (17a)

- x1(0) _,1c (17b)ec

Because of the continuous concave curvature of x,(t) the point t, is thefirst at which u = -t,. Thus t, is the solution of

(a, - alt,) cosh t, + Ole sirrh ' - x,(0) = 0 (18)

The solution curve x,(4) leaves the constraint Q, = 0 at t = t2. Inthe interval between constraints Eq. (12) must again be satisfied, and itis a simple consequence of the curvature of the solution of that equationand the continuity of u(t2) that the solution curve x,(t) can never inter-sect the line Q2 = 0 for any t < 0. The solution of Eq. (12) satisfyingx,(!) = a, - $,t2 and x,(0) = 0 is

x,(t) _ (al - a,t2) sinh [(0 - t)/c](19a)sinh [(0 - t2)/c]

u = - (a, - alt2) cosh [(0 - t) /c](19b)

c sinh [(0 - t2)/c]

The contribution to the objective from this final section is made as smallas possible by choosing 0 as large as allowable, 0 = a2/02 EvaluatingEq. (19b) at t2, the continuity of u(t2) requires that t2 be the solution of

(a, - Nlt2) cothat - $2t2 - CO, = 0 (20)

Finally, we return to the multipliers. Since Y2 is continuous, it fol-lows from Eq. (10) that Y2 = 1. Thus, setting all/au to zero, we obtainfor unconstrained sections

Yl = -C2u

In particular, at

tt

= t, and, from Eq. (13), t = t2

YI(tl-) = c2R1 = Yl(t2)

But since Eq. (9) applies along the constraint,

'Y,= -x, =alt - ca, tl <t<t2

x,(t,) = al - Iglti

XI [a, - Olt, - x,(0)e-11'1 sinh (t/c)sinh (t,/c)

u = [al - Olt, - x,(0)e-'d°] cosh (t/c)c sinh (tile)

(21)

(22)

(23)


and integrating with the boundary condition at t2,

71(ti+) = c2$1 + i2Nl(t12 - t22) - al(tl - t2)

Thus, from Eqs. (14a), (22), and (24),

Fu = al(tl - t2) - %ZSl(tl2 - t22)

and 7 is discontinuous at ti.

(24)

(25)

6.19 BOTTLENECK PROBLEMS

Bottleneck problems are a class of control problem typified by constraintsof the form

u < 0(x) (1)

When there-is little of the state variable and .0 is small, a bottleneckexists and little effort can be exerted. When 0 has increased and thebottleneck has been removed, large effort can be used. Many economicproblems are of this type, and a bottleneck model has recently beenemployed to describe the growth of a bacterial culture, suggesting anoptimum-seeking mechanism in the growth pattern.

The example of a bottleneck problem we shall examine has beenused by Bellman to demonstrate the application of dynamic program-ming. We have

xi = alu1 (la)

x2 = a2u2 - U1 (lb)

with

U1iu2>0Q1 = x2 - ul - U2Q2 =xl-u2>0

>0(2)

(3a)(3b)

and we seek to maximize x2(9). In this case the multiplier boundaryconditions are

71(0) = 0 72(0) =

and the hamiltonian is

- 1 (4)

H = ylalul + 72a2u2 - 72u1

At t = 8 the hamiltonian becomes

(5)

H = -a2u2 -I- u1 t = 0 (6)


in which case we require u2 to be a maximum, ul a minimum. Thus,

ul(9) = 0 (7a)

and, from Eqs. (3),

u2(9) = min j x'(8) (7b)11

x2(8)

We shall carry out our analysis for x2(9) > x1(9), in which case

u2(9) = x1(9) (8)

Then Q2 = 0 is in force, and just prior to t = 0 the state satisfies theequations

x1 = 0 (9a)z2 = a2x1 (9b)

or

x1(t) = x1(8) (103)x2(t) = a2x1(8)(t - 0) + x2(9) (104)

The Green's functions satisfy Eq. (5) of Sec. 6.15 when Q2 = 0 is in effect,which leads to

71 = -

'Y2 = -

or

72 = - a272 (11a)

raft afs -L-(,q)' 72=0 (11b)ax2 au2 au2 ax2

[afI afl (-9Q2\-' aQ2ryl

axl au2 au2/ 8xI

aft aft aQ2 -' aQ2Cax1 - au2 (au2) axl

[afl _ afl aQ2 -' aQ271

ax2 au2 Cau2) ax2

72 = 171 = a2(8 - t)

This final interval is preceded by one in which either

(12a)(12b)

72(t) - a71(t) > 0 x2(t) > xl(t) (13a)or

72(t) - a,71(t) < 0 X1(t) > x2(t) (13b)

The former condition ends at time t1, when 72 equals a171, which is, fromEq. (12),

tl = 9 _ 1 (14a)alas


The latter ends at t2, when x, = x2, from Eq. (10)

__ _ I' - 11 1it

x,(8) at0

(14b)

The simpler case encompasses final states for which it > t,. I or timesjust preceding t2i then, the optimal policy is

u1 = 0 (15a)

X1(t) (15b)U2 =min1X2(t)) =

x2

The state equations are

z,=0x2 = a2x2

in which case

x1(t < t2) = x,(t2)x2(t < t2) < x2(t2)

(16a)

(16b)

(17a)

(17b)

so that x2 < x, for all t < it during which this policy is in effect. Butnow Q, = 0 is in effect, leading to multiplier equations

7', = 0 71 = 71(12) = a2(0 - t2)

7'2 = -a272 72 = 72(t2)e-°,(e-y) = e-%(-y)

and

72(t) - a,71(t) > 72(12) - a,7,(12) > 0 t < it

so that the optimal policy is unchanged.for all t < t2.

(18a)

(18b)

(19)

The second possibility is t, > t2. For times just prior to t, thecoefficient of u2 in the hamiltonian is negative, while x2 > x1. If thecoefficient of u1 is zero and u, is intermediate while u2 is at its maximum,the multiplier equations are unchanged and 72 - a7, < 0, which is acontradiction. Thus, the coefficient of u1 is also negative, requiring u1to be at a maximum. The only possibilities compatable with x= > x1 are

u2=x1 ul=x2-x,U2 = 0 u, = x1

(20a)(20b)

The second case corresponds to Q, = 0, Q2 not in effect, in which case itfollows from the previous discussion that the coefficient of u1 is positive,resulting in a contradiction. Thus, for t < t, the policy is Eq. (20a).

It follows directly by substitution into the state and multiplierequations that this policy remains unchanged for all t < t,. Thus, the


optimal policy can be summarized as

1 x2 S XI, u1 = 0, u2 = x20 < t < B -a1a2 x2 > x1, u1 = x2 - x1, u2 = XI

0- 1 <t<0a1a2 U2 = min (x1,x2)

(21)

Bellman has established the optimality of this policy by an interestingduality relation which establishes strong connections between this type oflinear bottleneck problem and a continuous analog of linear programming.

6.20 SUFFICIENCY

In Sec. 1.3 we found that only .a slight strengthening of the necessaryconditions for optimality was required in order to obtain sufficient con-ditions for a policy that was at least locally optimal. Sufficient con-ditions in the calculus of variations are far more difficult to establish,and, in general, only limited results can be obtained.

Our starting- point is Eq. (4) of Sec. 6.9, which, after multiplicationby -yi(6) and summation from i = 1 to n yields

i-1

+ L auk a ; Suky; + 2axza kYIYk A + o(E2) (1)

k-1 i-1

where y'is defined as

y7(3) =IOa

L.l L.l r"(s,t) aukauk(t) dt

i-1 k-1(2)

If we restrict attention to situations in which g is a linear function ofcomponents of a(9) so that the second derivative vanishes identically,Eq. (1) is an expression of 53 for an arbitrary (small) variation in u.We require conditions, then, for which a& is always strictly positive fornonzero 5u.

When a function satisfying the minimum principle lies along aboundary, the first-order terms dominate. Thus we need only considerinterior values at which aH/auk vanishes and the hessian is positivedefinite. The two remaining terms in Eq. (1) prevent us from statingthat bg is positive for any change in u, rather than the very specialchanges which we have considered. A special case of importance is that

ne /i

RaH 1 R` 82H

k

Su; butyi(9) 8xi(B) = Ia1

aukSi6k + .41 49% auk


of minimizing

foeff(x,u) dt

with linear state equationsR

DD

xi Aijxj + I "ikuk

221

(3)

(4)

By introduction of a new state variable this ,is equivalent to a form inwhich S is a linear function of the final state. The hamiltonian is then

n n R'r

+ + Y L riHikuk=1 k=1

and for a function satisfyh: the minimum principle Eq. (1) becomes

r //R R

2SS-1J\1 25 a6%yi

0 au auk 444 axj aukj,k - l j-1 k+1

(5)

+ I- a yjyk j de + o(E2) (6)ax; axk /j,k 1

If if is strictly convex, that is,

R a2 n R a2_p

Ld auk auk ajak + 2 ftj auk Nj«kj,k-1 j-1 k=1

+n

a25Pjlek > 0 (7)

axi axkj,k-1

at all differentiable points for arbitrary nonzero a, g, then 6& is positive.Thus, the minimum principle is sufficient for local optimality in a linearsystem iethe objective is the integral of a strictly convex function. Animportant case is the positive definite quadratic objective, which we haveconsidered several times.

APPENDIX 6.1 CONTINUOUS DEPENDENCE OF SOLUTIONS

The linearizations in this chapter are all justified by a fundamental resultabout the behavior of the solutions of differential equations when smallchanges are made in initial conditions or arguments. We establish thatresult here.

Consider

x = f(x,u) 0 < It < 0 (1)


where u(t) is a piecewise-continuous vector function. We assume that fsatisfies condition with respect to x and u

. If(zi,ut) - f(x2,us)I < Lp([xI - z1I + Iul - u2I) (2)

where Lp is a constant and any definition of the magnitude of a vectormay be used. If we specify a function u(t) and initial condition 2o, Eq.(2) is sufficient to ensure a unique solution 2(t) to Eq. (1). Similarly,if we specify u = u + Bu and xo = 20 + Sao, we obtain a unique solution2 + Sx. We seek the magnitude of Sx at some finite terminal time 9 forwhich both solutions remain finite

Let

16u(t)1, ISxoI < C. (3)

If we integrate Eq. (1) for the two choices of xo and u and subtract, weobtain

. fix (t) = 5xo +fo'

[f(2 + &x, u + &u) - f(2,u)I ds 0 < t < B (4)

Making use of Eqs. (2) and (3), this leads to

Isx(t)I < e + Lp fog (ISxI + 16u() ds 0 < t < e (5)

If ISzI. is equal to the least upper bound of ISxI in 0 < t < 0, then

[fix (t)( <- e(1 + Lpt) +

Substituting Eq. (6) back into Eq. (5), we obtain

I&x(t)I < e(1 + Lpt) + et + Y2Lpt2 + i2LpI5a1mt2

and by continued substitution we obtain, finally,

I5z(t)I < e(e=La' - 1)or

Isz(e)I < Ke

where

K = 2eLD° - 1

and depends on 0 but not on Sz0 or Su(t).

(6)

(7)

(8)

(9)


Section 6.2: Solution proxdures for linear ordinary differential equations are discussedin any good text, such as

E. A. Coddington and N. Levinson: "Theory of Ordinary Differential Equations,"McGraw-Hill Book Company, New York, 1955


The linear analysis used here is covered in

M. M. Denn and R. Aria: Ind. Eng. Chem. Fundamentals, 4:7 (1965)L. A. Zadeh and C. A. Desoer: "Linear System Theory," McGraw-Hill Book Com-

pany, New York, 1963

Green's identity introduced here is a one-dimensional version of the familiar surfaceintegral-volume integral relation of the same name, which is established in any bookon advanced calculus, such as

R. C. Buck: "Advanced Calculus," 2d ed., McGraw-Hill Book Company, New York,1965

Section 8.5: The properties of the adjoint system of the variational equations were exploitedin pioneering studies of Bliss in 1919 on problems of exterior ballistics and subse-quently in control studies of Laning and Battin; see

G. A. Bliss: "Mathematics for Exterior Ballistics," John Wiley & Saps, Inc., NewYork, 1944

J. H. Laning, Jr., and R. H. Battin: "Random Processes in Automatic Control,"McGraw-Hill Book Company, New York, 1956

Sections 6.4 and 6.5: The development is similar to

M. M. Denn and R. Aris: AIChE J., 11:367 (1965)

The derivation in that paper is correct ealy for the case of fixed 0.

Section 6.8: The earliest derivation of a result equivalent to the strong minimum principlewas by Valentine, using the Weierstrass condition of the classical calculus of varia-tions and slack variables to account for inequality constraints; see

F. A. Valentine: in "Contributions to the Theory of Calculus of Variation, 1933-37,"The University of Chicago Press, Chicago, 1937

The result was later obtained independently by Pontryagrin and coworkers under some-what weaker assumptions and is generally known as the Pontryagin maximum (orminimum) principle; see

V. G. Boltyanskii, R. V. Gamkrelidze, and L. S. Pontryagin: Rept. Acad. Sci. USSR,110:7 (1956); reprinted in translation in R. Bellman and It Kalaba (eds.),"Mathematical Trends in Control Theory," Dover Publications, Inc., New York,1964

L. S. Pontryagin, V. A. Boltyanskii, R. V. Gamkrelidze, and E. F. Mishchenko:"The Mathematical Theory of Optimal Processes," John Wiley & Sons, Inc.,New York, 1962

A number of differ.mt approaches can be taken to the derivation of the minimum principle.These are typified in the following references, some of which duplicate each other inapproach:

M. Athans and P. L. Falb : "Optimal Control," McGraw-Hill Book Company, NewYork, 1966

L. D. Berkovitz: J. Math. Anal. Appl., 3:145 (1961)A. Blaquiere and G. Leitmann: in G. Leitmann (ed.), "Topics in Optimization,"

Academic Press, Inc., New York, 1967


S. Dreyfus: "Dynamic Programming and the Calculus of Variations," AcademicPreen, Inc., New York, 1965

H. Halkin: in G. Leitmann (ed.), "Topics in Optimization," Academic Press, Inc.,New York, 1967

1. Hestenes: "Calculus of Variations and Optimal Control Theory," John Wiley &Sons, Inc., New York, 1966

R. E. Kalman: in R. Bellman (ed.), "Mathematical Optimization Techniques," Uni-versity of California Press, Berkeley, 1963

E. B. Lee and L. Markus: "Foundations of Optimal Control Theory," John Wiley &Sons, Inc., New York, 1967

U. Leitmann: "An Introduction to Optimal Control," McGraw-Hill Book Company,New York, 1966

L. Neustadt: SIAM J. Contr., 4:505 (1966); 5:90 (1967)J. Warga: J. Math. Anal. Appl., 4:129 (1962)

The derivation used here is not rigorous for the constrained final condition, but the trans-versality conditions can be obtained by using penalty functions for end-point con-straints and taking limits as the penalty constant becomes infinite. We have assumedhere and throughout this book the existence of an optimum. This important ques-tion is discussed in several of the above references.

Section 6.9: We follow here

M. M. Denn and R. Aris: Chem. Eng. Sci., 20:373 (1965)

The result is equivalent to the Legendre-Clebsch condition of classical calculus of variations.

Section 6.10: This result was obtained by Aris; see

R. Aris: "Optimal Design of Chemical Reactors," Academic Press, Inc., New York,1961

Section 6.11: The example and mathematical development follow

F. Horn and R. C. Lin: Ind. Eng. Chem. Process Design Develop., 6:21 (1967)

A good introduction to the notion of process improvement by unsteady operation is

J. M. Douglas and D. W. T. Rippin: Chem. Eng. Sci., 21:305 (1966)

Section 6.12: The example is from

A. W. Pollock: Applying Pontryagin's Maximum Principle to the Operation of aCatalytic Reformer, 16th Chem. Eng. Conf. Chem. Inst. Can., Windsor, Ontario,1966

Applications to other problems with decaying catalysts have been carried out by

A. Chou, W. H. Ray, and R. Aris: Trans. Inst. ('hem. Engre. (London), 45:T153 (1967)S. Szepe and O. Levenspiel: unpublished research, Illinois Institute of Technology,

Chicago

Section 6.13: The classical Weierstrass and Legendre conditions are treated in any of thetexts on calculus of variations cited for Sec. 3.2.


Section 6.14: See

B. S. Goh: SIAM J. Cont., 4:3091(1966)C. D. Johnson: in C. T. Leondes (ed.), "Advances in Control Systems," vol. 2,

Academic Press, Inc., New York, 1965H. J. Kelley, R. E. Kopp, and H. G. Moyer: in G. Leitmann (ed.), "Topics in Opti-

mization," Academic Press, Inc., New York, 1967

These papers develop the necessary conditions in some detail and deal with several non-trivial applications. A very different approach applicable to certain linear problemsis taken in

A. Miele: in G. Leitmann (ed.), "Optimization Techniques with Applications, toAerospace Systems," Academic Press, Inc., New York, 1962

Sections 6.15 and 6.16: The approach is motivated by

A. E. Bryson, Jr., W. F. Denliam, and S. E. Dreyfus: AIAA J., 1:2544 (1963)

State-variable constraints are included in Valentine's formulation cited above and aretreated in most of the other references for Sec. B.S. See also the review paper

J. McIntyre and B. Paiewonsky: in C. T. Leondes (ed.), "Advances in Control Sys-tems," vol. 5, Academic Press, Inc., New York, 1967

Section 6.17: This problem was solved using other methods by

N. Blakemore and R. Aria: Chem. Eng. Sci., 17:591 (1962)

Other applications of the theory can be found in the references cited for Secs. 6.8 and 6.15and

J. Y. S. Luh, J. S. Shafran, and C. A. Harvey: Preprints 1987 Joint Autom. Contr.Conf., Philadelphia, p. 144

C. L. Partain and R. E. Bailey: Preprints 1987 Joint Autom. Contr. Conf., Philadelphia,p. 71

C. D. Siebenthal and R. Aris: Chem. Eng. Sci., 19:729 (1964)

Section 6.19: The example was solved by other methods in

R. E. Bellman: "Dynamic Programming," Princeton University Press, Princeton,N.J., 1957

An intriguing analysis of batch microbial growth as an optimum-seeking bottleneckprocess is in

C. H. Swanson, R. Aris, A. G. Fredrickson, and H. M. Tsuchiya: J. Theoret. Biol.,12:228 (1966)

Section 8.20: Sufficiency for more general situations is considered in several of the refer-ences cited for Sec. 6.8.

Appendix 6.1: This is a straightforward] extension of a well-known result in differentialequations; see, for example, the book by Coddinglon and Levinson cited above.


PROBLEMS

6.1. Show that the equation

d p(t)E + h(t)i s u(t)

is Self-adjoint in that the adjoint equation is

d p(t)i' + h(t) r - 0

6.2. Obtain the control which takes the second-order system

x+ax+bx - uJul < 1

from some initial state to a circle about the origin while minimizing

(a) time

(b) E-2Io (x'+cu')dt

U. Determine the relation between optimality of the steady state and the characterof stationary points of the steady-state lagrangian. Interpret this result in terms ofthe meaning of steady-state Lagrange multipliers.6.4. Examine the singular solutions of Secs. 5.9 and 5.10 in the context of the necessarycondition of Sec. 6.14.6.5. Goddard's problem of maximizing the height of a vertical flight rocket is describedby

h-vo I (T - D) -

m-T

rh s -c

Here h is the altitude, v the velocity, m the mass, g the acceleration due to gravity,and c the exhaust velocity. D is the drag, a f unction of v and h. The control variableis the thrust T, to be chosen subject to

B<T <T'to maximize the terminal value of h for fixed m. Show that intermediate thrust ispossible only for drag laws satisfying

aD 2a0

av! c 0,Vc=

6.6. Given

=uX(0) - z(1) 0

x(0) - vo > 0 z(l) + -vl < 0Jx(t) I < L


find u(t), 0 < t < 1 to minimize

2 Jou!(t) dt

(The problem is due to Bryson, Denham, and Dreyfus.)6.7. Consider the problem of determining the curve of minimum length between point(x,o,x20) and the origin which cannot pass through a closed circular region. Theequations are

i, = sin uit = Cog U

(xi-a):+xs:>_R'

B

min 6 = (o dt

Find u(t). (The problem is due to Leitmann.)6.8. In some practical problems state variables may be discontinuous at discretepoints (e.g., the mass in a multistage rocket) and the form of the system equationsmight change. Consider the system

xti(") = f (")(x("),u) 4-1 < t < t"

with discontinuities of the form

z("+1%) = lim z(t" + E) = lim x(tn - E) + {« =

where t" is a specified constant and t" is defined by the condition

+y"(z,t} = 0

For specified initial conditions zo and decision and final state constraints

U. <U <uOt[z(N)(1N)1 = 0

find the control which minimizes

H

f :.5[x ")(t),u(t)i dt

nil(The problem is due to Graham.)

7

Staged Systems

7.1 INTRODUCTION

In Chap. 1 we briefly examined some properties of optimal systems inwhich the state is described by finite difference equations, while the pre-ceding four chapters have involved the detailed study of continuous sys-tems. We shall now return to the consideration of staged systems inorder to generalize some of the results of the first chapter and to placethe optimization of discrete systems within the mathematical frameworkdeveloped in Chap. 6 for continuous systems. We shall find that theconditions we develop are conveniently represented in a hamiltonian for-mulation and that many analogies to continuous variational problemsexist, as well as significant differences.

In the study of continuous systems it has often been possible tosuppress the explicit dependence of variables on the independent varia-ble t so that inconvenient notational problems have rarely arisen. Thisis not the case with staged processes, where it is essential to state the'location in space or discretized time precisely. For typographical rea-sons we shall denote the location at which a particular variable is con-sidered by a superscript. Thus, if x is the vector representing the statexn

STAGED SYSTEMS 229

of the system, the state at position n is x", and the ith component ofx" is xi". The systems under consideration are represented by the blockdiagram in Fig. 7.1, and the relation between the input and output ofany stage is

x" = f"(x"-1,u") n = 1, 2, . . . , N (la)or

1 2xi" = fi"(x"-1,un)

n = 1, 2,, N (lb)

S is the number of variables needed to represent the state. The func-tions f" need not be the same for each stage in the process.

7.2 GREEN'S FUNCTIONS

Linear difference equations may be summed in a manner identical to theintegration of linear differential equations by means of Green's functions.We consider the system

S i=1,2, Sx,"= Aij"x,^+bi- n = 1, 2, N (1)

j-1

We now introduce S1 functions rkiN", i, k = 1, 2, . . , 8, at each stage,denoting both the stage number n and the final stage N, with which wemultiply Eq. (1) and sum over n from 1 to N and over i from 1 to S.Thus,

N S N S N SkiNnAijxjn-1 + I 4 rkiNnbi nI I rki Nnxin= 4) r

n-1 i-1 n-1 i.j-1 n-1 i-1

k= 1, 2, . . . S (2)

The left-hand side of Eq. (2) may be rewritten (summed by parts) asN S N S S

I I rkiN"x," = I I rkjN."-'xjn-1 + I rkiNNxiNILLL-I j-1n-lei-1 n i-I

s- I rkiNOXie (3)

{-1

Xa

X'=f1(XO, u1)

Decision u'

X11=f0(X"-1 U") + XN=fN(IN-1, UN)

X" XN-1 INSta e n .. - . NStg . j age

iDecisi on u" Decision uN

Fig. 7.1 Schematic of a staged system.


which yields, upon substitution into Eq. (2),S

i-1

S

1,kiNNxiN - I rkiNOxi0i-1

N S S(`j

N S

1N I t/\ I rkiNnAijn '- 1'A.ji"n-1``J

/ xjn-1 + rkiN nb,nn-1 j-1 i-1 n-1i-1

k = 1, 2, . . , S (4)

Since the goal of summing the equations is to express the output x"'in terms of the input x° and the forcing functions bn, we define the Green'sfunctions by the adjoint difference equations

(SrkjN.n-1 =

L\ rkiNnAijn

i- Ik,j=1,2,...,5n=1,2,...,N

-rkjNN = akj =1 ko k

Thus, Eq. (4) becomes a special form of Green's identity

S N S

xiN = I rjNOxj0 + I I riiNnbjnj-1 n-ij-1

(5)

`6)

(7)

These equations are analogous to Eqs. (12), (14), and (15) of Sec. -6.2,but it should be noted that the difference equation (5) has an algebraicsign opposite that of the differential equation (12) of that section.

We shall generally be interested in a linear combination of the com-ponents xiN. If we define a vector yn with components y,n by

,yin = yjNrj,Nn

i-1i=1,2,...,S (8)

where y; ' are specified numbers, then by multiplying Eqs. (5) to (7) byyiN and summing over i we obtain the scalar form of Green's identity

s S N S

y'NX,N

I 7i0A0 + I I y, nbini-1 i-1 n-1 i-1

and

yin-1 _--

si = 1, 2, . . . , S

j 1n=1,2, .,N

(9)

(10)

STAGED SYSTEMS

7.3 THE FIRST VARIATION

We now turn to the optimization problem.the equations

xin = fin(xn-I,un)i=1,2,...,5n= 1,2, . . . N

Uu

The state is described by

(1)

with the objective the minimization of a function of the final state S(xN).The decision functions u" may be restricted by inequality constraints

Upn(un) > 0 p = 1, 21 Pn=1,2,...,N (2)

and the initial and final states by equality constraints,

qk(x°) = 0 k = 1, 2, . . , K '(3)

gi(XN) = 0 1 = 1, 2, ... , L (4).

We assume that we have chosen a sequence of decisions fin and aninitial state 2°, after which we determine a sequence of states 2" fromEq. (1). We now make small changes consistant with the constraints.

uk" = ukn + &Ukn I auknl _< E

xi° = xio + axi° 15xi°I :5 E(5a)(5b)

for some predetermined t > 0. If the functions f° are continuous, it fol-lows that the outputs from stage N change only to order E. For piece-wise continuously differentiable functions fn the variational equations foraxn are then

ftSn pn

axis = I af 6xn-1 + I aJ, OUkn + o(E)LLL axjn-1 (3Zdknj.l k-I

(6)

where the partial derivatives are evaluated at xn, nn. Equation (6) islinear and has Green's functions defined by Eq. (10) of the precedingsection as

,Yn-I = S yjnL ax n-Ij-i '

in which case Green's identity, Eq. (9), becomes

i-l i-1 n.1 i-1 k-1

S 3 N 3 Rafi"

yiN axiN = yi° axi° + yin aukn

(7)

sukn + 0(E) (8)

For staged systems we need only consider the case where N is fixed,


so that the change in ti brought about by the changes Bu-, r,)xo isS

6& _ ar;SX,N + o(E)aX,N

i-1(9)

If, exactly as in Sec. 6.4 for continuous systems, we define the boundaryconditions for the difference equations for the Green's functions as

N _ as ag1ry N Ylaxi az;N

i-1

K°

Yiaqk° L ?7kax;U

k-I

then Eqs. (8) to (10) combine to giveN 3 R

_ af;"SF,

y," aut"Suk" + 0(t)

n-1 i-1 k-1

(10a)

(10b)

7.4 THE WEAK MINIMUM PRINCIPLE

In order to reinforce the analogy to continuous systems we introduce thestage hamiltonian H" as

sHn = y;"fn

i-1

We then have the canonical equations

OH"xIn

=nay;

7 n-1 =aH"

'a..in-1

and Eq. (11) of Sec. 7.3 for SS may be writtenN R

OHMSs =

auk"Sukn + o(e) > 0

n-1 k-1

(1)

(2a)

(2b)

(3)

where the inequality follows from the fact that the sequence On mini-mizes E. By a proof identical in every respect to that in Sec. 6.5 forcontinuous systems we obtain the following weak minimum principle:

The decisions ukn which minimize 6(XN) subject to the constraints17,"(u") > 0 make the stage hamiltonian H" stationary when a con-straint is not at equality and minimize the hamiltonian (or make it

STAGED SYSTEMS 233

stationary) with respect to constraints at equality or at nondifferentiablepoints.

It should be noted that, unlike continuous systems, it is not truethat H" is a constant for all n or, as we shall subsequently prove, thatH" is a minimum at stationary points. Equivalent formulations of theobjective may be accommodated by defining new state equations as forcontinuous systems.


The only essential generalization other than notation which we haveintroduced beyond the discussion of staged systems in Chap.

.1is the

restriction on the allowable values of the decision. It follows, then, thatin situations in which the optimal decisions are unconstrained the sta-tionary condition on the staged hamiltonian should be derivable from theLagrange multiplier rule. We show here that this is so.

We shall write the system equations and boundary restrictions as

-xi" + fn(xn-',un) = 0 1, 2, . . . , S (1)n 1, 2, N

g1(xN) = 0 1 = 1, 2, . . . , L (2)-qk(x°) = 0 k = 1, 2, ... , K (3)

The choice of positive or negative signs is motivated by a desire to relatethe results directly to those of the previous section. The minimizingvalues of u" for the objective S(xN) are found from stationary values ofthe lagrangian

2=C($N)+N s

n-1 i-1I I Xi"[-xi" + fn(t"-1,u;./J

L K

+ I v,gi(x'') - I 1kgk(x°) (4)1-1 k-1

Here we have introduced a multiplier Xe" for each of the constraint Eqs.(1), a multiplier vi for each Eq. (2), and 'hk for each Eq. (3).

Setting partial derivatives with respect to u;" to zero, we obtains of, n

Yin au n = 0 j = 1, 2, , R (5)

For n 1 the partial derivatives with respect to xi"-' yieldsA+ L x;" of in = 0 2 V 1' 2' '

S03)axe-' n = 2, 3, , N

i=1

LU OPTIMIZATION BY VARIATIONAL METHODS

Partial derivatives with respect to x,N give the equationsL

X iN + / Vly1

061xiNL..I/ ax{Ni-1

while derivatives with respect to x;° give

3 J1 ou' - KI u aax; ax;° =j-l k-1

By defining X,° through Eq. (6) this last relation may be written

Xc°- 1 7,,C9

a=0K

k-1

(7)

(8a)

(8b)

Equations (5) to (8) are identical to the equations for the weakminimum principle when the decisions are unconstrained if the Lagrangemultipliers Xi" are identified with the Green's functions y;". Surprisinglysome authors have failed to recognize this simple relation and havedevoted considerable space in the literature to the complicated deri-vation (or rederivation) through a "minimum principle" of results inunconstrained systems more easily (and frequently, previously) obtainedthrough application of lagrangian methods. Indeed, the interpretationof the multipliers in Sec. 1.15 in terms of partial derivatives of the objec-tive leads directly to the weak minimum principle with constraints, andit is only to simplify and unify later considerations of computation thatwe have adopted the form of presentation in this chapter.


In order to illustrate the difficulties which arise in even the simplestsituations when constraints are imposed on the decisions in a staged sys-tem it is helpful to return to the example of Sec. 1.12. We desire theoptimum sequence of operating temperatures in carrying out the chemi-cal reaction

X --> Y -- products

in N stirred-tank reactors. We shall retain x and y to denote concen-trations and u for temperature, but we now use superscripts to signifythe stage number.

Equations (6) defining the state in Sec. 1.12 are

0 = xn-1 - x" - 8"k1(un)F(xn) (1a)0 = yn-1 - y" + v0nk1(u")F(xn) - B"k2(u")G(y") (lb)

STAGED SYSTEMS

and the goal is to choose u', u2, . . . , uN in order to minimize

- pxI - yrr

us

(2)

In Chap. 1 we found that if u" is not constrained, the design problem isreduced to the simultaneous solution of two coupled nonlinear differenceequations with an iterative search over one initial condition, a rathersimple calculation. We now add the restriction that u" lie within bounds

u* <u" <u* (3)

Equations (1) are not of the form for which we have developed thetheory, which would be

x" = f"(x"-',y"-',u") (4a)

y" = gn(x"-1,y"-1,u") (4b)

Since we need only partial derivatives of f" and g" in order to apply thetheory, however, this causes no difficulty. For example, partially differ-entiating Eq. (la) with respect to x"-' yields

1 - af" - O"k1(u")F'(x")a f"

= 0 (5a)ax"'' ax"-3

or

af" __ 1

ax"'' 1 + 8"kl(un)F'(xn)(5b)

The other partial derivatives are similarly obtained, and we can writethe multiplier equations

af" ag"yl"-i = 'Y1' ax"-1 +

ax"-'

yl"1 + 8"kl(u")F'(x")

l2

+ [1 + O"kl(u")F'(x"))[1 + 8"k2(u")G'(y")]"afn a

"y

y2 (6b)72"-' - 71" ayn_1 + 72"49yn_1 1 + 9"k2(u*)G'(y")


astilN=ay= -p

y2"'=ayN= -1

(u")F'(x")'Y "pO"k

(7a)

(7b)

Z36 OPTIMIZATION BY VARIATIONAL METHODS

Also,

_all- af" nag- yl"9"k1(un)F(x")T-- ` 7i" au- + 72 au" 1 -}- 9"kl(un)F'(x")72np8"kl(u")F(x")

+ [1 + 9"k2(u")G'(y"))[1 + 9"ki(u")F'(x°)]72"9"k2(u")G(y")

(8)1 + 9"k2(u")G'(y")

The equation aH"/au" = 0 will have a unique solution for variables ofinterest, in which case the optimum u" for solutions which lie outside thebounds established by Eq. (3) is at the nearest bound.

Because Eq. (6b) is independent of yin and 72" cannot go to zero,we may divide Eqs. (6) to (8) by 72" and by defining

" 72n (9)

we obtain1 + 9"kl(u")F'(x") p9"kl(u")F'(x")1+ 9"k2(u")G'(y") 1 + 9"k2(u")G'(y") (10)

N = P (11)

with solutions of aHn/au" = 0 at solutions of

9"k1(u")F(x") p9"k; (u")F(x")1 + 9"ki(u")F'(x") + [1 + 9"k2(u")G'(y")][1 + 9"kl(u")F'(x"))

9"k2(u")G(y") = 0 (12)1 + 9"k2(u")G'(y")

The required computational procedure is then as follows:

1. Assume r'.2. Solve Eqs. (1) and (12) simultaneously for x', y', u'.3. If u' exceeds a bound, set u' to the nearest bound and recompute

x', y' from Eqs. (1).4. Compute x2, y:, us, r simultaneously from Eqs. (1), (10), and (12).5. If u2 exceeds a bound, set u2 to the nearest bound and recompute

x2, y2, ?.2 from Eqs. (1) and (10).6. Repeat steps 4 and 5 for n = 3, 4, . . . , N.7. Repeat steps 1 to 6 for changing f' until YN = p.

This is substantially more computation than required for the uncon-strained case. Note that we could have chosen zN and yN and com-puted backward, adjusting xN and yN until matching the specified x°, yo,but that without some rational iteration procedure this requires substan-tially more calculations than a one-dimensional search on f''.

STAGED SYSTEMS

In the special case that

F(xn) = xnG(yn) = yn

the problem simplifies greatly. Defining

y"17

xn

we combine Eqs. (1) to yield

n_1 _ 1 + 9nk2(un) n - vOnk,(un)11 1 + 9nk1(un) 1 + 9nk,(un)

while Eqs. (10) and (12) become, respectively,

n t = I + 9nk2(un) "Bnk,(un)

1 + 9"kl(un) + 1 + Onk,(un)kl(u") k2(un)

1 + 9"k,(un) n I + enk2(un)

(13a)

(13b)

(14)

(15)

(16)

vk,(un) 0 (17)[1 + 9nk1(u")J[1 + 9"k2(u"))

We may now either follow the procedure outlined above or choose 1f N,compute since we are given i-N, then compute 1 N-1, rN-1, um-1, etc.,by means of Eqs. (15) to (17), iterating on ,IN until matching the speci-fied feed ratio n°. The advantage of this latter calculation is that what-ever the , ° corresponding to the chosen 71N, the complete set of necessaryconditions has been used and the policy is optimal for that particular 11°.,Thus a complete set of optimal policies is mapped out in the course ofthe computation, which is not the case in calculations which assume r°.

7.7 THE STRONG MINIMUM PRINCIPLE: A COUNTEREXAMPLE

At the end of Sec. 7.4 we stated that it is not generally true for discretesystems that the stage hamiltonian assumes a minimum at stationarypoints. From the discussion in Sec. 7.5 it may be seen that such aminimization would correspond to minimizing the lagrangian, which weshowed to be incorrect in Sec. 1.8, but because of substantial confusionabout this point in the engineering literature we shall pursue it some-what more. In this section we shall demonstrate a counterexample toa strong minimum principle; in the next section we shall construct someof those situations for which a strong minimum principle does exist.

Consider the system

xln = x1n-1(1 + x2"_1) 12(u")2 x1° _ 74 (la)x2n = 4x1"-1 - 2x2n-1 + u" x2 = +1 (1b)

no OPTIMIZATION BY VARIATIONAL METHODS

where u' and u2 are to be chosen subject to

0<u*<u" n=1,2in order to minimize

S = _x,2

By direct substitution

S = + (u1)21(2 - u') + 12(U2)2

-and the optimal values are u' =+ 1, ult = u*.Now

(2)

(3)

(4)

H" = 'Y1"(x,"-1 + xl"-'x2'-' - 7l('u")zl+ 72"(4x1"'' - 2x,"-' + u") (5)

aH,u' = -U171' + 72'a (6)

a2H'a(u')2 = -y1' (7)

Buts

'rll = axil = -(I + x21) (-2 + 'u') (8a)

72'axaH2

=1=l= -xl1

7'l 11 + (4999(499 (8b)

Thus, all'/au' does indeed vanish at u' = 1. But from Eqs. (7) and(8a),

a'H'a(u1)2

-1 2 = -1 < 0

which corresponds to a maximum of H' rather than a minimum.

(9)

7.8 SECOND-ORDER VARIATIONAL EQUATIONS

We shall now examine circumstances under which a strong minimumprinciple, in which the stage hamiltonian is in fact minimized by theoptimum values of u", can be shown to exist. The significance of sucha result is clear in the light of the equivalent results for continuous sys-tems discussed in the previous chapter, and we shall find subsequentlythat there are other computational implications. Our procedure is againthe application of Picard's iteration method.

We suppose that z° and n° have been specified and that we nowconsider variations Bus, n 1, 2, . . . , N such that I du;"J < E. &x0 will

be taken to be zero. The variational equations corresponding to Eqs.

STAGED SYSTEMS 239

(1) of Sec. 7.1 are then, to second order in e,

sn naxin = I k + I ai,

Suknj-1 axjn-1

k-lS 2f n S R 2n

+ 2 1 axjna' axkn-1 axn-1 axkn-1 + axjaif

Sukn axjn-1

j-1 k=1

I c`)

+3If.n

Sujn bUk n + O/(. L1 all nUO(J) (1)

J,k - 1

If we look upon this as a linear nonhomogeneous system for which theappropriate Green's functions I'ijnm satisfy the equation

r{_n,m-1 = \ rijnm afkm

' G ax n-1kal l

then Green's identity becomes

axis rijnm (a26m auk- +

mmljglk=1 k p=1

a2fjma,ukm aUPm SUkm SUP- }

n S a2f.m

+ 2 rijnm axkm-1 ax m-lS.rjm-1 axpm-1

m - I j,k.p = 1 P

(2)

n S R

32",m+ ./ I rijnm

axk, au,- Sxkm-1 + O(E2) (3)m=1 j,k=1 p-1

Evaluating Eq. (3) for n = m - 1 and substituting back into the right-hand side, we obtain, finally, an explicit representation for Sxv

N S R RrJ

2

. nS2lkn aupnSxiN =

LIrijNn (Li

auk~ + `I auaLn

n-1j+1k-1 pal P

N S2f.. n

n -1 S R{{

Nn _ J7 rk n-l,m a2cvm+ axkn al dxpn-1 Q a2lvmn-I j.k,pI 111q1 v1n-i S R N S R 2 fn{ r \X rp n

r nn-1 j,kml axk allp

n-1 S R

X rjqn-l.ma{19m

auvm) + O(E2) (4)allvm,n-1 =1 v-1q

We shall assume, for simplicity, that the objective to be minimized,S(xN), is linear in the components xiN. By suitable definition of addi-tional variables we can always accomplish this. If we further assume


that xN is completely unconstrained, we obtain

SS = bx,N N ax;Na.Z,N (5)

and multiplying Eqs. (2) and (3) by y,,' and summing we obtain, withthe usual definition of the stage hamiltonian,

I 2 n

SF = I an aukn +2 LI 3unfaukn bu7n auknn=1 k=1 j,k=1 7

N S2

n n-1 S R {m

(n-l,m fi m 1Suy /J+ 8X"al axPn-1 I I rkj

U

v

n=1 k,p=1 m=1 7-1 v=1

M-1 q-1 j-1

Unlike the continuous case, in which At can be made arbitrarily small,there is no general way in which the last two terms can be made to van-ish, and this is the source of the difference between discrete and continu-ous systems. Several special cases, however, can be treated. Becausewe are interested in the character of stationary points, we shall restrictnonvanishing Sukn to those corresponding to interior values of theoptimal f, n.

Linear separable equations have the form

sx;n A,,nxjn-1 + bin(un)

j=1(7)

where the A,," are constants. In that case both the second partial deriv-atives of Hn with respect to components only of xn-1 and xn-1 and unvanish identically, and Eq. (6) reduces to

n 2 nN` R

SS= I I aukn

sukn + 2 L I 8u n aukn SujnSukn + 0(E2) > o

n-1 k=1 n-1 j,k-1 7

(8)

The first term is zero by virtue of the weak minimum principle and theassumption concerning admissable bun. It follows, then, that the hessianmatrix of second derivatives of Hn must he positive definite, correspond-ing to a minimum. Furthermore, since the Sun are arbitrary, Eq. (8)

/`'1

/C T, t CR 2 n

X (I I V 1P4n-l r LL

euzr} + axka

auPnSuynr=l -1 s=1 =1 k-l I

n-1 S R N S'

0 CX I I iTk,,n-I.m

m

qm aujml + o(e2) > 0 (6)7

q n p=n-1 S P

STAGED SYSTEMS 241

establishes the sufficiency of the strong minimum principle for the linearseparable case as well.

Linear nonseparable equations are of the form

sx," = I A;;" (u") x"-' + b;" (u")

j-1

Here the mixed second partial derivatives do not vanish.the special variation, however,

bum _bu* m = n*0 m n*

(9)

By choosing

(10)

for some specified n* the last term in Eq. (6) also vanishes, and we obtain,using the weak minimum principle,

IR asg"*

SE = 2 au "* auk"*Su* Su,* + o(EZ) > 0j I

which establishes the necessity of the strong minimum principle for thiscase also. That it is not sufficient is apparent from Eq. (6).

If there is a single state variable, with the scalar equation

x" = f"(x"-',u") (12)

then, since y" will be nonzero, the vanishing of all"/auk" implies the van-ishing of 49f"/auk". It follows from Eq. (6), then, that the strong mini-mum principle is both necessary and sufficient. Furthermore, f" alwaystakes on an extreme value, and if of"/ax"-' is positive for all n, then y". isalways of one sign. If, in addition, x" must always be positive for physi-cal reasons, the policy is disjoint, which means that minimizing H" inorder to minimize or maximize x" is equivalent to minimizing or maxi-mizing x" at each stage by choice of u". This should recall the discussionof optimal temperature profiles for single reactions in Sec. 3.5, and, indeed,an identical result holds for the choice of temperatures for a single reactionoccurring in a sequence of staged reactors.

We emphasize that the results obtained here are applicable only fora linear objective.t If S is nonlinear, an additional term is required inEq. (6). In particular, a linear system with a nonlinear objective isformally equivalent to a nonlinear system with a linear objective, forwhich the strong minimum principle does not apply. A counterexample

t With the exception of those for a single state variable.


which is linear and separable is

xln = -3x1n-1 - 3- (un) 2 x1° = given (13a)x2n = X2n-1 + un x2° = 0 (13b)Fi = -x12 - (x22)2 (13c)

We leave the details of the demonstration' to the reader.

7.9 MIXED AND STATE-VARIABLE CONSTRAINTS

For staged systems it is not necessary to distinguish between constraintson the state of the system

Qkn(xn) >- 0 (1)

and mixed constraints

Qkn(xn,un) > 0 (2)

Through the use of the state equation both types of constraint can alwaysbe put in the form

Qkn(xn-l,un) > 0 (3)

If the constraint is not violated by the optimal trajectory, the theorydeveloped in Sees. 7.4 and 7.8 remains valid. Thus, we need only con-sider the case where equality must hold in Eq. (3).

As with the continuouY case, we first suppose that there is a singlestage decision variable u^ and only one constraint is at equality. Wethen have

Qn(xn-l,un) = 0 (4)

and the variational relationship

aQn = L aQn Sxcn-l + aQn Sun = 0 (5)axin-1 nt

Coupled with the variational equation,s

axin =afi"

axjn-1 + inin sun44-I axin-1 clu"

. j-iwe may solve for Sun and write

s

af'n - aftn NA-1 aQn 1axjn-1 aun aun

axjn-l

1

(6)

(7)

The decision un is determined by Eq. (4), and the Green's function corre-

STAGED SYSTEMS 243

sponding to the difference equation (7) satisfies

af;n - af,* aQn -1 aQny, "-` - ; [axin-1 au" %aun) axfn-1 yn

(O)

Equation (3) of Sec. 7.4 for 63 remains unchanged provided we excludevalues of n at which Eq. (4) holds, and the weak minimum principlemust be satisfied for all stages where a constraint is not at equality.

In the general case of K1 independent constraints and R1 > K1components of U" we find, as in Sec. 6.15, that the Green's vector mustsatisfy the equation

s

af'nax{n-1

i=1

K, )n aQkn n

aupn Syk axin-1 7ik-1

(9)

p,

where Spkn is the matrix such that

K,a n

Sp/c = atpau;Ik-1

i,p= 1, 2, . . . ,K1 (10)

It follows then that the R1 - K1 independent components of un must bechosen to satisfy the weak minimum principle.


Section 7.1: Finite difference representations might arise either because of time or spacediscretization of continuous processes or because of a natural staging. For the for-mer see, for example,

G. A. Bekey: in C. T. Leondes (ed.), "Modern Control Systems Theory," McGraw-Hill Book Company, New York, 1965

J. Coste, D. Rudd, and N. R. Amundson: Can. J. Chem. Eng., 39:149 (1961)

Staged reaction and separation processes are discussed in many chemical engineeringtexts, such as

R. Aris: "Introduction to the Analysis of Chemical Reactors," Prentice-Hall, Inc.,Englewood-Cliffs, N.J., 1965

B. D. Smith: "Design of Equilibrium Stage Processes," McGraw-Hill Book Company,New York, 1963

Section 7.2: The linear analysis for difference equations used here is discussed in

M. M. Denn and R. Aria: Ind. Eng. Chem. Fundamentals, 4:7 (1965)T. Fort: "Finite Differences and Difference Equations in the Real Domain," Oxford

University Press, Fair Lawn, N.J., 1948L. A. Zadeh and C. A. Desoer: "Linear System Theory," McGraw-Hill Book Com-

pany, New York, 1963


Sections 7.3 and 7.4: The analysis follows the paper by Denn and Aria cited above.Similar variational developments may be found in

S. S. L. Chang: IRE Intern. Conv. Record, 9(4):48 (1961)L. T. Fan and C. S. Wang: "Discrete Maximum Principle," John Wiley & Sons, Inc.,

New York, 1964S. Katz: Ind. Eng. Chem. Fundamentals, 1:226 (1962)

The paper by Katz contains a subtle error and erroneously proves a strong minimumprinciple. This incorrect strong result is cited, but not necessarily required, in severalof the examples and references in the book by Fan and Wang.

Section 7.5: The lagrangian development was used by Horn prior to the work cited above:

F. Horn: Chem. Eng. Sci., 15:176 (1961)

Section 7.6: This reactor problem has been studied in the papers cited above by Horn andDenn and Aris and in the book ,

R. Aria: "The Optimal Design of Chemical Reactors," Academic Press, Inc., NewYork, 1961

A number of other applications are collected in these references and in the book by Fanand'Wang.

Section 7.7: The observation that a strong minimum principle is not available in generalfor discrete systems is due to Rozenoer:

L. Rozenoer: Automation Remote Contr., 20:1517 (1959)

A counterexample to a strong minimum principle was first published by Horn andJackson in

F. Horn and R. Jackson: Ind. Eng. Chem. Fundamentals, 4:110 (1965)

Section 7.8: The development follows

M. M. Denn and R. Aria: Chem. Eng. Sci., 20:373 (1965)

Similar results are obtained in

F. Horn and R. Jackson: Intern. J. Contr., 1:389 (1965)

Several workers, notably Holtzman and Halkin, have examined the set-theoretic foun-dations of the necessary conditions for optimal discrete systems with care. Some oftheir work and further references are contained in

H. Halkin: SIAM J. Contr., 4:90 (1966)J. M. Holtzman and H. Halkin: SIAM J. Contr., 4:263 (1966)

Section 7.9: These results are obtained in the paper by Denn and Aria cited for Sec. 7.8.

STAGED SYSTEMS 245

PROBLEMS

7.1. Consider the necessity and sufficiency of a strong minimum principle for

xi"

S

- I aijxjw-1 + biu"

j-1N

[

jS

min I [ La xi"Qijxj" + P(u")']2 n11 i,j-1

7.2. Denbigh has introduced the pseudo-first-order reaction sequence

ki k,X, X, -, X,Lk, J,k,

Yl Y2

where X, is the desired product. Taking the reaction-rate coefficients as

ki(T) - ki0 exp (:Pi)where T is the temperature, the equations describing reaction in a sequence of stirred-tank reactors are

xin-1 - xl"[l + u1"(1 + U24))

x:"-i - -uj"xl" + xt"L

1 + 0.01uu1" l l+\ .u'w

XI"-1 - -0.01u1"xt" + x""

Here, ul is the product of reactor residence time and k1, and u2 is the ratio k1/k1.Taken together, they uniquely define temperature and residence time. The followingvalues of Denbigh's have been used:

k2 = 10'k1 exp (::-Tom) ks - 10-2k1

Aris has introduced bounds

f 00,k4 -

3,7T 0

0 < T < 394 0 _< ui < 2,100

For initial conditions x° - 1, x2° - x3° - 0 determine the maximum conversion toX3 in a three-stage reactor. Hint: Computation will be simplified by deriving boundson initial values Yi° and/or final values The optimal conversion is 0.549.7.3. Obtain the equations defining an approximate nonlinear feedback control, for thesystem described by the nonlinear difference equation

xw+: = F(xw+l,xw,uw)

Nmin S = - 1 (z,, + Puw')2 nsl


7.4. Repeat Prob. 1.13 using the formalism of the discrete minimum principle. Extendto the case in which T. is bounded from above and below.7.5. Consider the reaction X Y -, Z in a sequence of stirred tanks. This is definedby Eqs. (1) of Sec. 7.6 with the added equation -

X0 + y* + z* - coast

Derive the procedure for maximizing z"'. Specialize to the case

F(x*) - x* G(y*) - y*

and compare your procedure for computational complexity with the one used in thebook by Fan and Wang for studying an enzymatic reaction.

8Optimal and Feedback Control

8.1 INTRODUCTION

Many of the examples considered in the preceding chapters have beenproblems in process control, where we have attempted to use the opti-mization theory in order to derive a feedback control system. Even insome of the simple examples which we have studied the derivation of afeedback law has been extremely difficult or possible only as an approxi-mation; in more complicated systems it is generally impossible. Further-more, we have seen that the optimal feedback control for the same sys-tem under different objectives will have a significantly different form,although the objectives might appear physically equivalent.

In this chapter we shall briefly touch upon some practical attackson these difficulties. We first consider the linear servomechanism prob-lem and show how, for a linear system, the optimization theory com-pletely determines the feedback and feedforward gains in a linear controlsystem and how classical three-mode feedback control may be viewed asa natural consequence of optimal control. The problem of ambiguity of

247'


objective will be similarly attacked by considering another particularlyeasily implementable feedback system and solving the inverse problem,thus defining a class of problems for which a reasonable optimum controlis known.

8.2 LINEAR SERVOMECHANISM PROBLEM

The linear servomechanism problem is the control of a linear systemsubject to outside disturbances in such a way that it follows a prescribedmotion as closely as possible. We shall limit our discussion here to sys-tems with a single control variable and disturbances which are approxi-mately constant for a time much longer than the controlled systemresponse time, and we shall assume that the "motion" we wish to followis the equilibrium value x = 0. Thus, the state is described by

zi = Ai;x; + biu + di (1)

with A, b, and d consisting of constant components, and we shall use aquadratic-error criterion,

2 Io ' xiCi;x; + u2) dt (2)

C is symmetric, with Ci; = C;i. The u2 term may be looked upon as apenalty function or simply a mathematical device in order to obtain thedesired result. Multiplying by a constant does not alter the result, sothat there is no loss of generality in setting the coefficient of u2 to unity.The linear regulator problem is the special case with d = 0.


H = Z xiCi1x; + %u2 + yiAi,x, + q yibi + yid; (3)

with multiplier equations(\

}aH

Cyi = _ G i,x, - I ykAkiOxi

k

The partial derivatives of H with respect to u are

aH = u + L yibiau

a2H=1>0au2

(4)

(5a)

(5b)

Equation (5a) defines the optimum when set to zero

OPTIMAL AND FEEDBACK CONTROL

(6)

while Eq. (5b) ensures a minimum. We have already established in Sec.6.20 that the minimum principle is sufficient for an optimum here..

We seek a solution which is a linear combination of the state varia-bles and forcing functions. Thus,

7i Mijxj + D;,dji

Differentiating,

(7)

Mijx, + M;jxj + Dijdj (8)

and substituting Eqs. (1), (6), and (7),

M;jxj + MijAjkzk

I Mijbj Q Muxkb, + I Dudb,) + I Mijdj + E Aidj , (9a)j k.! kj j j

while Eq. (4) becomes

-ii = - I Cijxj - I Q Mklzt + I Dkd:) Aki (9b)j k I 1

Equating the coefficients of each component of x and of d, we then obtainthe two equations

M;j + I (MikAkj + MkjAk;) - Mikbk)rj'\L btMij) + C,j = 0

k k t

1 (((10)

Aj + I Ak;D:i - ((\ M ,kbk) \ b,D:j) + Mij = 0 (11)k k i

Equation (10) is a quadratic Riccati differential equation, while Eq. (11)is a linear nonhomogeneous differential equation with variable coefficients.It should be observed that the symmetry of C implies a symmetric solu-tion to Eq. (10), Mij = Mj;.

We shall not consider transient solutions of Eqs. (10) and (11),for it can be established that as & -- oo, we may obtain solutions whichare constants. Thus, the optimal control is

u = - b,Miizj - b;D;,dj (12)

!!!@ OPTIMIZATION BY VARIATIONAL METHODS

where M and D are solutions of the algebraic equations

- (M;kAk, + Mk;Ak;) + (I Mikb.) \ M1,b,) = C,, (13)

(Ak: - bk Mob) Dk; M(; (14)- k 1

We have previously solved special cases of these equations.The obvious difficulty of implementing the feedback control defined

by Eqs. (12) to (14) is that the matrix C is not known with certainty.It will often be possible to estimate C to within a scale factor, however(the coefficient of u2!), in which case Eqs. (13) and (14) define the inter-actions between variables and only a single parameter needs to be chosen.

$.3 THREE-MODE CONTROL

An interesting application of the linear servomechanism problem ariseswhen one postulates that it is not the u2 term which should be retainedin the objective but a term 42. This would follow from the fact thatthe rate of change of controller settings is often the limiting factor in adesign, rather than the magnitude of the setting. We might have, forexample, for the forced second-order system

x+bx+ax=u+d (1)

or

zl = x2 (2a)z2= -ax,-bx2+u+d (2b)

the objective

S = 2 fom (C,x12 + C2x22 + u=) dt (3)

By defining u as a new state variable x:, we may rewrite Eqs. (2)and (3) as

ZI = x2 (4a)it = -axe - bx2 + x, + d (4b).is = W (4c)

2 !o(0x14 + C2x2' + w2) dt (5)

where w is taken as the control variable. This is of the form of the linearservomechanism problem, with the degenerate solution

w = -MIIxt - M12x2 - M13x3 - D32d (6)


Equations (13) and (14) of Sec. 8.2 reduce to the nine equations

M

C,1 = M132 + 2bM12 (7a)0 = M13M23 - M11 + aM12 + bM22 (7b)

0 = MIZM33 --- MM12 + bM23 (7c)C2 = 'M232 - 2M12 + 2aM22 (7d)

0 = M23M33 - M13 + aM23 - M22 (7e)

0 = M332 - 2M23 (7f)Mil = bD22 + M13D32 (7g)

M22 = -D12 + aD22 + M23D32 (7h)M23 = -D22 + M33D32 (7i)

The solution may be shown to be

D32 = M33 (8a)

M13 = -bM33 + N C1 (8b)

M23 = i2M332 (Sc)

where M33 is the unique positive solution of the quartic equation

M33' + 4aM333 + 4(a2 + b)M332 + 8(ab - )M33 - 4C2-8a =0 (9)

Thus,

w = is = - (/ - bM3,)x - i2M332i - M33u - Maad (10)

We may substitute for u + d in Eq. (10) from Eq. (1) to obtain

x - (3 M33,2 + aM33)i - M33x (11)

or, integrating,

u(t) v Cl f of x(T) dr - (%M332 + aM3a)x(t) - Msai(t) (12)

This is a three-mode controller, in which the control action is taken as aweighted sum of the offset and the derivative and the integral of theoffset. Such controls are widely used in industrial applications, the inte-gral mode having the effect of forcing the system back to zero offset inthe presence of a persistent disturbance. Standard control notationwould be of the form

u(t) _ -K I x(t) + r Jot x(r) dr + TD adt)

(13)


where

K = 12Ma3(Maa + 2a)

M33(M33 + 2a)TI =

2 VC12

TD = M33 + 2a

These equations can be rearranged to yield

2K = 2(1 - aTD)TD

and the nonnegativity of K then bounds the derivative time

aTD<1

The governing equation for the controlled systemdisturbances is

i'+ (a+KrD)x+ (b+K)±+Kx = 0TI

or, defining the ratio of time constants by .

Tra = -TD

(14a)

(14b)

(14c)

(15)

constant

(16)

subject to step

(17)

(18)

and substituting Eq. (15) into (17), the governing equation becomes

+ [2TD rD] x + [b+_(1 - aTD)] x

+LD3(1 amD)] x = 0 (19)

For a stable uncontrolled system (a, b > 0) the necessary and sufficientcondition that Eq. (19) have characteristic roots with negative real partsand that the controlled system be stable is

(coefficient of t) (coefficient of t) > (coefficient of x) (20a)or

2-arDrb+ 2a(1-aTD)] (20b)TD L TD Q7D

The inequality can be converted to equality by introducing a stabilityfactor o > 1 and writing

2-amDI 2b-}Tt)TD

s

-i r(1-arD)I =°- all-arD)] (21)

art)2

Here o = 1 corresponds to marginal stability.

OPTIMAL AND FEEDBACK CONTROL 253

Equation (21) can be rewritten as \a cubic equation inrD

- 2 rp3+(a2+b)rp2+(a3{arn+i2-aJ=0 (22)

It will often happen in practice that a2 > b. For example, in the reactor-control problem used in Chaps. 4 and 5 and again in a subsequent sectionof this chapter a = 0.295, b = 0.005. In that case Eq. (22) approxi-mately factors to

\\ // \\

CrD2+arp-1arn-2+a)=0 (23)

and the unique root satisfying the inequality arD is

arp=2-- (24)

, .. n 1, :...V..., ..uvu.cll '111LUtS-

al1Ow-The nonnegativity of TD :iliu the upper- tuoullua

able values of u/a to

2 > -° > 1-a(25)

Equation (15) for the gain can now be rewritten in terms of the singleparameter o/a as

K _ 2a(o - a)(26)

P (2a - 0')2

Further restrictions on the allowable controller design parametersare obtained by substituting Eqs. (14), (18), (24), and (26) into thequartic equation (9) and rearranging to solve for C2, yielding

C2 = 4K [_a2 - c + av + (2a 0)2] (27)2 a J

A meaningful optimum requires that the objective be positive definite,or that C2 be nonnegative. Using the approximation b >> a2 for con-sistancy, a and c are then further related by the inequality

a2-ao+a, <0An equivalent form obtained by completing the square is

(a-32u)2+ u(1-%a)<0from which it follows that

(28a)

(28b)

0>4 (29)

That is, optimality requires at least 4 times the minimum stability con-


dition! It further follows, then, from Eq. (25) that a is bounded frombelow

a2i2o>2A sharp upper bound on the allowable ratio of time constants

be obtained by rewriting Eq. (28b) as

(a il42 411or, taking the square root,

a- j2v<2j1 -

(30)

can

(28c)

(31)

Thus, the ratio rr/rn isstrictly bounded by the one-parameter inequality

2<<rp<211-+-(32)For a in the range 4 to 6 the allowable values of rr/rn are restricted

to lie between 2 and 4.7. Standard control practice for settings for three-mode systems is a value of rr/rD of approximately 4.

8.4 INSTANTANEOUSLY OPTIMAL RELAY CONTROL

In order to circumvent the difficulty of constructing a feedback controlsystem from optimization theory a number of investigators have usedad hoc methods to construct feedback systems which are, in some sense,instantaneously optimal, rather than optimal in.an overall sense. Forexample, suppose that the positive definite quadratic form

E = 2 XiQrsx;i.J

(1)

is used as a measure of the deviation from the equilibrium x = 0, beingpositive for ar v offset. Then an instantaneously optimal control wouldbe one which drives E to zero as rapidly as possible, disregarding thefuture consequences of such action.

We shall restrict attention to systems which are linear in a singlebounded control variable

x, = f,(x) + b;(x)u (2)U* < u < u* (3)

(Linearity is essential, but the single control is not.) This form is typi-cal of many processes. The criterion of driving E to zero as rapidly as


possible is equivalent to minimizing the time derivative E

E _ xiQ:ixi = xiQiifi + ( x:Qijb) U (4)_. is +a

(We have made use of the symmetry of Q.) Since Eq. (4) is linear in u,the minimum will always occur at an extreme when the coefficient of u;does not vanish identically

u* I x.Qiibi > 0u = `'' (5)

u* x;Q;ibi < 0.

Thus we immediately obtain a feedback control law. Note that whenb is independent of x, the switching criterion is linear.

This mode of control can be illustrated by returning to the exampleof the stirred-tank chemical reactor, defined in Sec. 4.6.as

d (A - A,) =V

(A, - A) - kA (6)

d (T- T.) = q (7)I V C9P VCDP

where x1 is taken as A - A X2 as T - T control is by coolant flowrate q,, and we have defined u as the monotonic function ('Kq,/(1 + Kq,).All parameters will be identical to those used in Sees. 4.6 and 5.6, in whichcase the bounds on u are

0 < u < 8 (8)

Comparison of Eqs. (6) and (7) with Eq. (2) indicates that theswitching function has the form

xiQT T`

IQ12(A - A.) + Q22(T - T,)l (9).L r rbi = - V C'P

As T will always exceed the coolant temperature T,, the coefficient of thebracketed term in Eq. (9) is always negative and Eq. (5). for the controllaw reduces to

_ 8 (T - T.) + a(A - A,) > 0U

0 (T-T,)+a(A-A,)<0 (10)

where a = Q12/Q22 - This, is a linear switching law for the nonlinear proc-ess. An even simpler result is obtained when cross terms are excludedfrom the objective, in which case a = 0. Then the control is

_ 8 T > T,u-0 T<T,

256 OPTIMIZATION BY VARIA' ONAL METHODS

That is, adiabatic operation when the temperature is below the steady-state value, full cooling when above, irrespective of the relative weight-ing placed on concentration and temperature deviations. It is interest-ing to compare this result with the time-optimal control shown in Fig.5.9, in which the switching curve does not differ significantly from thesteady-state temperature.

Paradis and Perlmutter have computed the response of this systemunder the control equation (11) with an initial offset of T - T. = -20,A - A. = 2 X 10-4. The phase plane in Fig. 5.9 indicates that awayfrom equilibrium the temperature should approach the steady-state valueimmediately, first slowly and then quite rapidly, while the concentrationdeviation should first grow and then approach zero. Figures 8.1 and 8.2show that this is precisely what happens, where curve a is the controlledresponse and curve bthe uncontrolled. The first two switches occur at23.70 and 25.65 sec, after which the controller switches between extremesrapidly. Such "chattering" near the steady state is a common charac-teristic of relay controllers. It should be noted that the system is asymp-totically stable and returns to steady state eventually even in the absenceof control.

In order to avoid chatteringt some criterion must be introduced for

t Chattering may sometimes be desirable, as -discussed in detail in the book byFlilgge-Lots.

7

6

5

0

3Q

2

1Desireoperatinglevel

I I I I I I 1 I I 1

10 20 30 40 50 60 70 80 90 100Time t

Fig. $.1 Concentration response of the controlled anduncontrolled reactor using instantaneously optimal control.[From W. 0. Paradis and D. D. Perlmutter, AIChE J.,12:876 (1966). Copyright 1966 by the American Instituteof Chemical Engineers. Reprinted by permission of thecopyright owner.]


K

460

455

-450

445

440

C

10 20 30 40 50 60 70 80 90Time t

Fig. 8.2 Temperature response of the controlled anduncontrolled reactor using instantaneously optimal con-trol. [Front W. 0. Paradis and D. D. Perlmutter, AIChEJ., 12:876 (1966). Copyright 1966 by the American Insti-tute of Chemical Engineers. Reprinted by permission of thecopyright owner.)

changing from relay to another form of control. In analogy with thetime-optimal case Paradis and Perlmutter simply set u to its steady-state

11value of 5 at the time of the second switch and allowed the natural sta-bility of the system to complete the control. These results are shown ascurve c in Figs. 8.1 and 8.2 and indicate quite satisfactory performance.

8.5 AN INVERSE PROBLEM

The results of the preceding section suggest the fruitfulness from a practi-cal control point of view of pursuing the subject of instantaneously opti-mal controls somewhat further. In particular, since the one serious draw-back is the possibility that rapid return toward equilibrium might causeserious future difficulties,. we are led to enquire whether this ad hoc policymight also be the solution of a standard optimal-control problem, in whichcase we would know precisely what overall criterion is being minimized,if any. To that end we are motivated to study the inverse problem.

We shall restrict our attention for simplicity to linear systems withconstant coefficients

zi = Ai,x1 + b.'u (1)

and we shall take u to be a deviation from steady-state control with sym-metric bounds, in which case b may be normalized so that the hounds are


unity

Jul < 1 (2)

If the bounds on u are not symmetric, the subsequent algebra is slightlymore cumbersome iut the essential conclusions are unchanged. Underthese assumptions the feedback control law of Eq. (5) of the precedingsection is

u = - sgn Q b;Q;;x;)

or

u = - sgn Q a;x;)i

(3a)

(3b)

a linear switching law. We shall suppose that the cost of control isnegligible and consider an overall objective of the form

3 = Io T(x) dt (4)

where

5(x) > 9(0) > 0 x 96 0 (5)

The inverse problem which we wish to solve is for a function F(x) satis-fying Eq. (5) such that a control of the form of Eq. (3) minimizes 6.

The hamiltonian for the linear stationary system described by Eq.(1) and the objective equation (4) is

H = F(x) + I y;A;;x; + y;b;u (6)

and, since u enters linearly, whenever the'coefficient of u does not vanish,the optimal control is

u = - sgn (7)

Equations (3) and (7) will define the same control if (but not only if!)we take

yi = Q;;x; (8)

We need, then, to determine whether such a relation is compatable withthe minimum principle and, if so, what the resulting functions is.

The equations for the Green's functions are

x;ax; -

L. 'A;; (9)y


or, upon substituting Eq. (8),

ag - I xkQkjA;;ax;

j.k

But differentiating Eq. (8),

ti; _ Qiixj = I QiA;kxk - I Qiibi sgn C xkQktbt> (11)ilk j k.t

where we have used Eqs. (1), (7), and (8). The right-hand sides ofEqs. (10) and (11) must be identical, leading to a family of partial differ-ential equations for t(x):

ax; --

(QriAikxk + xkQkiAii) + Qiibj sgn (I xkQktbt}j.k 7 k.t

Integration of Eq. (12) is straightforward, yielding

F(x) - 11 xj(Q;iAjk + Qk,Ai;)xk +c.j.k

I x,Q;;bj I + const (13)

26!

(10)

(12)

If 8 is fixed, the value of the constant is irrelevant. If 0 is unspecified,the condition H = 0 establishes that the constant is zero. We obtain,then,

(x) = 2 x;C;ixi +I

x;Qi;b;i.i

(14)i.,

where

Cij = - (QikAk; + QikAki) (15)k

The absolute-value term in Eq. (14) is a linear combination and canvanish for x 5- 0, so that we ensure satisfaction of Eq. (5) by requiringthat the quadratic form be positive definite. This places an

tointeresting restriction on the uncontrolled system, for the time derivativeof the positive definite quadratic form E = 3 x;Q;ix; without control is

simply

E = - I x,C,;x; (16);.j

which is required to be negative definite. According to Liapunov sta-bility theory, then (see Appendix 8.1), the quadratic form E must be aLiapunov function for the uncontrolled system and the uncontrolled sys-tem must be asymptotically stable. In that case t is also negative defi-nite for the controlled system, and the controlled system is also stable.

M OPTIMIZATION BY VARIATIONAL METHODS

The asymptotic stability of the uncontrolled system is sufficient to ensurethe existence of a positive definite solution Q to Eq. (15) for arbitrarypositive definite C, although the converse is not true.

We now have a solution to the inverse problem, which establishesthat for an asymptotically stable linear stationary system the relay con-troller with linear switching corresponds to an objective which is theintegral of a positive definite quadratic form plus a second positive semi-definite term. The use of a quadratic form as an integrand is, of course,now well established, and we anticipate that the additional term willsimply have the effect of bending the trajectories somewhat. Thus, theinstantaneously optimal policy does correspond to a very meaningfuloverall objective. We are not quite finished, however, for we must stillestablish that the minimum of the integral of Eq. (14) does indeed occurfor the control defined by Eq. (3). We must do this because of the possi-bility that a singular solution may exist or that the minimum principleleads to multiple solutions, another of which is in fact the optimum.

The possibility of a singular solution in which the switching cri-terion vanishes for a finite time interval is most easily dismissed by con-sidering the second-order system

21 = X2

x2 = A21x1 + A22x2 + bu

for which the switching criterion defined by Eq. (3) is

(17a)

(17b)

Q12x1 + Q22x2 = 0 (18)

If this is differentiated with respect to time and Eqs. (17) substituted,the resulting control is

u - - (A21Q222) xl - A22x2 (19)

On the other hand, the vanishing of the switching function means thatthe integrand of the objective is simply 3' (Cllxl2 + 2C12x1x2 + C22x22y,and the criterion for singular control was found in Sec. 5.9 for this caseto be

u = - (A ell\ 21-C22)xl-Az2xz

and

ell X1 + C22 x2 = 0

Thus, singular control is possible and, in fact, optimal if and only if

ell _ Q122

C22 Q222

(20)

(21)

(22)

OPTIMAL AND FEEDBACK CONTROL $1

Together with Eq. (15) this yields only discrete values of the ratioC matrix is at the disposal of the designer, only infinitesimal

changes are needed to avoid the possibility of intermediate control. Thegeneralization to higher dimensions is straightforward and yields thesame result.

Finally, it remains to be shown that there cannot be another con-trol policy which satisfies the minimum principle. Here, for the firsttime, we make use of our wish to avoid chattering and presume thatwithin some neighborhood of the origin we want to switch from the relaycontroller to some other form of control, perhaps linear. We shall choosethat region to be an ellipsoidal surface such that the control effort is toterminate upon some manifold

g(x) = I x;(0)Q;;x;(6) - const = 0 (23):.,

where the final time 0 is unspecified. The boundary condition for theGreen's functions is then

y;(0) = v ax = 2v I Qi,x, (24)

where v is some constant and the condition H = 0 immediately estab-lishes that 2v = I. Hence, for any final condition x(0) a complete set ofconditions is available for the coupled differential equations (1) and (9),and the solution over any interval between switches is unique. The con-trol given by Eq. (3) must, then, be the minimizing control for the objec-tive defined by Eq. (14).

Any asymptotically stable linear system, then, may be controlledoptimally with respect to Eq. (14) by a particularly simple feedbackpolicy. Because of the ambiguity of defining a precise mathematicalobjective it will often be the case that ff as defined here meets all physicalrequirements for a control criterion, in which case the instantaneouslyoptimal policy will provide excellent control to within some predeter-,mined region of the desired operating conditions.

The extension to nonlinear systems of the form

x: = f;(x) + b.(x)u (25)

is trivial, and again the condition for optimality is that the quadraticform E be a Liapunov function for the uncontrolled system and, there-fore, for the controlled system. Now, however, this requirement is morerestrictive than simply demanding asymptotic stability, for the existenceof a quadratic Liapunov function is ensured only in the region in whichlinearization is valid, and large regions of asymptotic stability may existin which no quadratic Liapunov function can be found.


One final comment is required on the procedure used to solve theinverse problem. Equation (8) can be shown to be the consequence ofany linear relation between y; and Q;;x,, but any sign-preserving non-

linear relation would also retain the same control law. Thus, by con-sidering nonlinear transformations for the one-dimensional system

z=Ax+bu (26)

Thau has found that the control

u = - sgn Qbx (27)

is optimal not only for the objective found here

S = Jo (AQx2 + IQbxI) dt (28)

but also for

S = fo (Ab2Q3x4 + Ib=Q=x'I) dt (29)

and

T; = fa (b x sinh bQx + Isinh bQxl) dt (30)

One way in which this equivalence of objectives can be established is touse the relation

sgn bQx = sgn [ k (bQx)2k+11 K2 > Ki > 0 (31)k-Ri J

and then to procede in the manner of this section. The desirability ofextending this approach to multidimensional systems is obvious, as arethe difficulties.

8.6 DISCRETE LINEAR REGULATOR

We have observed earlier that we may often be interested in controllingsystems which evolve discretely in time and are described by differenceequations. The difference-equation representation might be the naturaldescription, or it might represent an approximation resulting from com-putational considerations. The procedures discussed in this chapter canall be extended to the control of discrete systems, but we shall restrictourselves to a consideration of the analog of the problem of Sec. 8.2.

We consider a system described by the linear difference equations

x;" _ , Atiixi"-1 + b,u" (1) .


The coefficients Aij and bi are taken as constants, though the extensionto functions of n is direct. We have not included a disturbance term,though this, too, causes no difficulties, and we seek only to find thesequence of controls Jun 1 which regulates the system following an initialupset or change in desired operating point. The minimum-square-errorcriterion is again used

N`

1S`l l R(u")2] (2)

Unlike the continuous case, we include a coefficient of the (u")2 term inthe objective, for R can be allowed to go to zero for discrete control.

A Lagrange multiplier formulation is the most convenient for solv-ing this problem. The lagrangian is

= 2 [I R(u")2]

+ I../n

ff x4" Q A,,x,n_1 + biun - xi"/ Ji i (3)

Setting partial derivatives with respect to xi" and u" to zero, respectively,we obtain

xinCij - xjn + X;"+'Aii = 0

Run + Xi-bi = 0

(4)

(5)

Equation (4) allows us to define a variable Xi'+' as zero. Unlike theanalysis for the continuous problem, we shall not solve Eq. (5) for u"at this point, for further manipulation will enable us to obtain a feed-back solution which will be valid as R - 0.

If Eq. (4) is multiplied by bj and summed over j, we obtain

I xiiC1jbj - Xi"bi + xin+'Ai7.bj = 0 (6)ij 1,3

Substitution of Eq. (5) then leads to

I xi"Ciibi + Run + I Xin+'Aijbj = 0+.j id

(7)

We now seek to express the multipliers in terms of the state variables as

Xi"+1 = M1k"xkn (8)

k

Since X '+' is zero, Mk"" must be zero, except for N -- oo, in which casexkN - 0. ' Substitution of Eq. (8) into Eq. (7) then leads to the required

264

feedback form

u" = I K1"x "-I

where


(9)

I CikbkAij + I blAklAlki"A,;K." i.l.k (10)

R + I b,Cikbk + b1Ak1:llk,"bii,k

The feedback dependence must be on x"-', since the state at the begin-ning of the control interval is the quantity that can be measured.

We still need a means of calculating 211,," in order to compute thefeedback gains Kj". This is done by first substituting Eq. (9) into Eq. (1)

x," = I (Aij + b,Kj").rj"-1 (11)J

Substitution of Eq. (11)1 into Eq. (4) yields

(I Aik + biltk") Cijx,"-I - il!jk"-Ixk"-Ik k

+ Ai; [1 .1f, " I (AIk + b1Kk")] xk"-' = 0 (12)1 k

If Eq. (12) is to hold for all values of x"-1, the coefficient of xk"-1 must

be identically zero, in which case i1ij" must be a solution to the differenceequation

:11jk"-' _ CijAik +N' Aijifil"AIk + biCij + Aijhlil"bzKk"

1l1;kN = 0 (13)

Equation (13), with Kj" defined by Eq. (10), is the generalization of theRiccati difference equation first encountered in Sec. 1.7. For the impor-tant case that N ---* co, a constant solution is obtained. Clearly there isno difficulty here in allowing R to go to zero.

The instantaneously optimal approach of Sec. 8.5 may be appliedto the discrete system to obtain an interesting result. We define a posi-tive definite quadratic error over the next control interval

E = l xi"Q11xj" + 2 P(u")2 (14)

The control which makes E as small as possible over the following inter-val is found by setting the derivative of E with respect to u" to zero

ax"O1' _x,nQ'j

au." + Pu" = 0(15)

fail" - I


From Eq. (1),

ax;" (16)au" = bi


I biQi;x;" + Pu" = 0 (17)i, j

The required feedback form for u" is obtained by substituting Eq. (1) forx;" into Eq. (17) and solving

u" = I kx;"-' (18)s

I / biQikAkik, c/`

ki

/-I I biQikbk+Pk i

(19)

This linear feedback control can be conveniently compared to thediscrete-regulator solution for N - by defining a new variable

µik = A,kMti (20)

It can be shown from Eqs. (10) and (13) that µik is symmetric(µik = µki).Equation (10) for the regulator feedback gain can then be written

I I bi(Cik + Uik)AkiK 1 = -

k i (21)

11 1 bi(Cik + Iik)bJ + Rk i

The results are identical if we make the following identifications:

Qik = Cik + µikP=R

(22)

(23)

Qik defined by Eq. (22) is symmetric, as it must be. We find, therefore,that the discrete-regulator problem is disjoint and has a solution corre-sponding to an instantaneous optimum over each control cycle. Theparameters in the instantaneous objective will have physical significance,however, only when computed by means of Eq. (22).

APPENDIX 8.1 LIAPUNOV STABILITY

We shall review briefly here the elementary principles of Liapunov sta-bility needed in this chapter. Consider any function V(x) which ispositive definite in a neighborhood of x = 0. The values of V define adistance from the origin, although the closed contours might be quite


irregular. Thus, if the system is displaced from equilibrium to a value x0and there exists any positive definite function V(x) such that V(x) < 0in a region containing both Ko and the origin, the system can never escapebeyond the contour V(x) = V(xo) and the origin is stable. Furthermore,if there exists a function such that V(x) < 0 for x 0, the system mustcontinue to pass through smaller and smaller values of V and ultimatelyreturn to equilibrium. In that case the origin is asymptotically stable.

A function V(x) such as that described above, positive definite witha negative semidefinite derivative, is called a Liapunov function. For asystem satisfying the differential equations

xi = Fi(K) Fi(0) = 0 (1)

the derivative of V is computed by

V(x) _ aVzi =

aV(2)

i axi i axi

If a Liapunov function can be found in some region including the origin,the system is stable with respect to disturbances within that region. IfV is negative definite, the system is asymptotically stable. Construc-tion of a Liapunov function is generally quite difficult.


Section 8.1: The conventional approach to the design of feedback control systems is treatedextensively in texts such as

P. S. Buckley: "Techniques of Process Control," John Wiley & Sons, inc., NewYork, 1964


D. D. Perlmutter: "Chemical Process Control," John Wiley & Sons, Inc., New York,1965

J. Truxal: "Automatic Feedback Control System Synthesis," McGraw-Hill BookCompany, New York, 1957

The design of optimal control systems based on a classical frequency-domain analysisis treated in, for example,

S. S. L. Chang: "Synthesis of Optimum Control Systems," McGraw-Hill BookCompany, New York, 1961

A modern point of view somewhat different from that adopted here is utilized in

C. W. Merriam: "Optimization Theory and the Design of Feedback Control Systems,"McGraw-Hill Book Company, New York, 1964

1

The two approaches are reconciled in our Chap. 12; see also

L. B. Koppel: "Introduction to Control Theory with Applications to Process Control,"i Prentice-Hall, Inc., Englewood Cliffs, N.J., 1968


L. Lapidus and R. Luus: "Optimal Control of Engineering Processes," BlaisdellPublishing Company, Waltham, Mass., 1967

and a forthcoming book by J. M. Douglas for parallel discussions pertinent to this entirechapter.

Section 8.8: The properties of the linear system with quadratic-error criterion have beeninvestigated extensively by Kalman, with particular attention to the asymptoticproperties of the Riccati equation. In particular see

It. E. Kalman: Bol. Soc. Mat. Mex., 5:102 (1960)in It. Bellman (ed.), "Mathematical Optimization Techniques," University

of California Press, Berkeley, 1963: J. Basic Eng., 86:51 (1964)

A detailed discussion is contained in

M. Athans and P. Falb: "Optimal Control," McGraw-Hill Book Company, NewYork, 1966

and some useful examples are treated in the books by Koppel and Lapidus and Luus and

A. R. M. Noton: "Introduction to Variational Methods in Control Engineering,"Pergamon Press, New York, 1965

A computer code for the solution of the Riccati equation, as well as an excellent anddetailed discussion of much of the basic theory of linear control, is contained in

It. E. Kalman and T. S. Englar: "A User's Manual for the Automatic SynthesisProgram," NASA Contractor Rept. NASA CR-475, June, 1966, available fromClearinghouse for Federal Scientific and Technical Information, Springfield,Va. 22151

Numerical solution of the Riccati equation by sucessive approximations is discussed in

N. N. Puri and W. A. Gruver: Preprints 1967 Joint Autorn. Contr. Conf., Philadelphia,p. 335

Though not readily apparent, the procedure used in this paper is equivalent to that dis-cussed in Sec. 9.6 for the numerical solution of nonlinear differential equations.

Section 8.3: The general relationship between the linear servomechanism problem witha us cost-of-control term and classical control is part of a research program beingcarried out in collaboration with G. E. O'Connor. See

G. E. O'Connor: "Optimal Linear Control of Linear Systems: An Inverse Problem,"M.Ch.E. Thesis, University of Delaware, Newark, Del., 1969

Section 8.4: The particular development is based upon

W. O. Paradis and D. D. Perlmutter: AIChE J., 12:876, 883 (1966)

Similar procedures, generally coupled with Liapunov stability theory (Appendix 8.1),


have been applied by many authors; see, for example,

C. D. Brosilow and K. R. Handley: A IChE J., 14:467 (1968)R. E. Kalman and J. E. Bertram: J. Basic Eng., 82:371 (1960)D. P. Lindorff: Preprints 1967 Joint Autom. Contr. Conf., Philadelphia, p. 394A. K. Newman: Preprints 1967 Joint Autom. Contr. Conf., Philadelphia, p. 91

The papers by Lindorf and Newman contain additional references. A detailed dis-cussion of the properties of relay control systems will be found in

1. Flugge-Lotz: "Discontinuous and Optimal Control," McGraw-Hill Book Company,New York, 1968

Section 8.5: This section is based on

M. M. Denn: Preprints 1967 Joint Autom. Contr. Conf., Philadelphia, p. 308; A IChEJ., 13:926 (1967)

The generalization noted by Thau was presented in a prepared discussion of the paper,at the Joint Automatic Control Conference. The inverse problem has been studiedin the context of classical calculus of variations since at least 1904; see

0. Bolza: "Lectures on the Calculus of Variations," Dover Publications, Inc., NewYork, 1960

J. Douglas: Trans. Am. Math. Soc., 60:71 (1941)P. Funk: "Variationarechnung and ihr Anwendung in Physik and Technik," Springer-

Verlag OHG, Berlin, 1962F. B. Hildebrand: "Methods of Applied Mathematics," Prentice-Hall, Inc., Engle-

wood Cliffs, N.J., 1952

The consideration of an inverse problem in control was first carried out by Kalman forthe linear-quadratic case,

R. E. Kalman: J. Basic Eng., 86:51 (1964)

See also

A. G. Aleksandrov: Eng. Cybernetics, 4:112 (1967)P. Das: Automation Remote Contr., 27:1506 (1966)R. W. Obermayer and F. A. Muckler: IEEE Conv. Rec., pt. 6, 153 (1965)Z. V. Rekazius and T. C. Hsia: IEEE Trans. Autom. Contr., AC9:370 (1964)F. E. Thau: IEEE Trans. Autom. Contr., AC12:674 (1967) '

Several authors have recently studied the related problem of comparing performance ofsimple feedback controllers to the optimal control for specified performance indices.See

A. T. Fuller: Intern. J. Contr., 5:197 (1967)M. G. Millman and S. Katz: Ind. Eng. Chem. Proc. Des. Develop., 6477 (1967)

Section 8.6: The Riccati equation for the feedback gains is obtained in a different mannerin the monograph by Kalman and Englar cited for Sec. 8.2, together with a computercode for solution. See also the books by Koppel and Lapidus and Luus and

S. M. Roberts: "Dynamic Programming in Chemical Engineering and ProcessControl," Academic Press, Inc., New York, 1964


W. G. Tuel, Jr.: Preprints 1967 Joint Autom. Contr. Conf., Philadelphia, p. 549J. Tou: "Optimum Design of Digital Control Systems," Academic Press, Inc., New

York, 1963"Modern Control Theory," McGraw-Hill Book Company, New York, 1964

The book by Roberts contains further references. Instantaneously optimal methods havebeen applied to discrete systems by

R. Koepcke and L. Lapidus: Chem. Eng. Sci., 16:252 (1961)W. F. Stevens and L. A. Wanniger: Can. J. Chem. Eng., 44:158 (1966)

Appendix 8.1: A good introductory. treatment of Liapunov stability theory can be foundin most of the texts on control noted above and in

J. P. LaSalle and S. Lefschetz: "Stability by Liapunov's Direct Method with Applica-tions," Academic Press, Inc., New York, 1961

For an alternative approach see

M. M. Denn: "A Macroscopic Condition for Stability," AIChE J, in press

PROBLEMS

8.1. For systems described by the equations

2i - f,(x) + b,(x)u

use the methods of Secs. 8.2 and 4.8 to obtain the linear and quadratic terms in thenonlinear feedback control which minimizes

e (j's

3 2 fo \4 xiC,;z; + u' dt

Extend to the feedback-feedforward control for a step disturbance d which enters as

ii - f,(x) + bi(x)u + g,(x)d

8.2. Extend the results of Sec. 8.3 to a second-order system with numerator dynamics,

z+at+bx-u+eii.+d8.3. Extend the control approach of Sec. 8.4 and the optimization analysis to the casewhen E(x) is an arbitrary convex positive definite function.M. The unconstrained control problem

x = f(x,u,P)

x(0) = xo

smin E a fo 5(x,u,c) di

where p and c are parameters, has been solved for a given set of values of xo, p, and c.Obtain equations for the change du(t) in the optimal control when xo, p, and c arechanged by small amounts Sxo, bp, and Be, respectively. In particular, show that au


may be expressed as

au - I Ki(t) axi + 191k(t) apk + Z 92.0) ac,"i k

where ax is the change in x and Ki(t) and g ,(t) are solutions of initial-value problems.(Hint: The method of Sec. 8.2 can be used to solve the coupled equations for changesin state and Green's functions.) Comment on the application of this result to thefollowing control problems:

(a) Feedback control when the system state is to be maintained near an optimaltrajectory.

(b) Feedback-feedforward control when small, relatively constant disturbancescan enter the system and the optimal feedback control for the case of no disturbancecan be obtained.8.6. The system

x(t) + at(t) + bx(t) - u(t)

is to be regulated by piecewise constant controls with changes in u every r time unitsto minimize

S - 1

2Ioe (z' + ci') dt

Obtain the equivalent form

xl" - 2="-1

x7" -0yl"-1 - axf"-1 + w"H

min s -2

I [(x1")' + C(z:")11n-1

Obtain explicit values for the parameters in the optimal control for N -- co

W. - -K1z1"-1 - Ktx:"-1

Extend these results to the system with pure delay T

2(t) + at(l) + bx(t) = u(t - T)

(The analytical solution to this problem has been obtained by Koppel.)

9Numerical Computation

9.1 INTRODUCTION

The optimization problems studied in the preceding six chapters areprototypes which, because they are amenable to analytical solution orsimple computation, help to elucidate the structure to be anticipated incertain classes of variational problems. As in Chaps. I and 2, however,where we dealt with optimization problems involving only differentialcalculus, we must recognize that the necessary conditions for optimalitywill lead to serious computational difficulties if rational procedures fornumerical solution are not developed. In this chapter we.shall considerseveral methods of computation which are analogs of the techniquesintroduced in Chap. 2. In most cases we shall rely heavily, upon the,Green's function treatment of linear differential and difference equationsdeveloped in Sees. 6.2 and 7.2.

9.2 NEWTON-RAPHSON BOUNDARY ITERATION

The computational difficulties to be anticipated in the solution of a vari-ational problem are best illustrated by recalling the necessary conditions

2n


for minimizing the function 8[x(0)J in a continuous system when x(O) isunconstrained, x(0) = xo is given, and 0 is specified. We must simul-taneously solve the S state equations with initial conditions

- f,(x,u)0 < t < 0

i= 1, 2, Sx(0) = xo

the S multiplier equations with final conditions

saf,

'ax; 0<t<ea& i = 1, 2, .. .

7i(e) = axi

together with the minimization of the hamiltonian at each value of t

smin H = yi f,u(Q i-1

(1)

(2)

(3)

An obvious procedure for obtaining the solution is to assume eitherx(e) or 7(0) and integrate Eqs. (1) and (2), choosing u at each value of tto satisfy the minimum principle. By choosing x(0) we compute anoptimal solution for whatever the resulting value of x(0), but we mustcontinue the process until we find the x(9) which leads to the required xo.Let us suppose that the possible range over which each component xi(0)may vary can be adequately covered by choosing M values, xi(1)(0),xi(2)(9), . . . , xi(M)(0). There are, then, MB possible combinations offinal conditions to be evaluated, and for each the 2S differential equa-tions (1) and (2) must be integrated, or a total of 2SM8 differential equa-tions are to be solved. A very modest number for M might be 10, inwhich case a system described by only three differential equations wouldrequire the numerical integration of 6,000 differential equations in orderto reduce the interval of uncertainty by a factor of 10. For S. = 4 thenumber is 80,000. This exponential dependence of the number of com-putational steps on the dimension of the system is generally referred toas the curse of dimensionality.

For certain types of problems the curse may be exorcised by alinearization procedure for improving upon estimates of x(8). We makea first estimate 8(8) and integrate Eqs. (1) and (2) with respect to t fromt = 0 to t = 0, determining u at each step in the numerical integrationfrom the minimum condition. The valug of x so computed at t = 0,x(0), will generally not correspond to the required value xo. A new valueof x(6) will produce new functions x(t), u(t) in the same way, and the

NUMERICAL COMPUTATION

first-order variational equation must be

bii = ax' bxj + auk aukk-1

273

(4)

where fix = x - 2, &u = u - n, and partial derivatives are evaluatedalong the trajectory determined by u. Defining the Green's functionsrij(B,t) by s

I k (5)ax.k-1

rri(B,B) = b+j = { 0 iyd j

Green's identity, Eq. (15) of Sec. 6.2, may be writtens 3 RVIN Is r. <, a f

j-1 j-1 k-1

(6)

a4Akb?Lk dt (7)

At this point we make the critical assumption that the optimalpolicy does not differ significantly for neighboring values of z(8), in whichcase Bu will be approximately zero. Equation (7) then provides an algo-rithm for determining the new estimate of x(6)

S

z (9) _ fti(®) + r0(8,0)[xjo - .4(0)J (8)

j-1

Equation (8) may be looked upon as an approximation to the first twoterms of a Taylor series expansion, in which case we might interpretr(6,0) as an array of partial derivatives

r:j(9,0) =ax;(B)

(9)axj (0)

which is quite consistent with the notion of an influence function intro-duced earlier. Equation (8) is then analogous to the Newton-Raphsonprocedure of Sec. 2.2, where the function to be driven to zero by choice ofz(8) is z(0) - z0.

This procedure is equally applicable to systems described by differ=ence equations

n = 1, 2, . . . , Nn is n-1 n

The variational equations ares R

ax;" = G adJ, 1

6x`n-1 + 4 a2lk" b1Lk" (11)xj n_j-1 k-1


with Green's functions defined by

St afknriix.n-1 rikNnL/dk-1 1r"

NN

_-a"

j0 ij

The first-order correction to an estimate 2N is thens

xiN = ZiN + rijNo[xjo - x-olj-i

11

(12)

(13)

(14)

As outlined, this Newton-Raphson procedure requires the calcu-lation-of S2 + S Green's functions, the S2 functions rij(B,t)(r;, 'n) neededfor the iteration and the S functions yi(t)(yin) for the optimization prob-lem. By use of Eq. (18) of Sec. 6.2 or Eq. (8) of Sec. 7.2 it follows fromthe linearity and homogeneity of the Green's function equations that ycan be calculated from r

Yi(t) _ 8xjrji(9,t) (15)

with an identical relation for staged systems. Thus, only S2 equationsneed be solved on each iteration.

It is likely that the linearization used in the derivation of the algo-rithm will be poor during the early stages of computation, so that usingthe full correction called for in Eqs. (8) and (14) might not result in con-vergence. To overcome this possibility the algorithm must be written as

sxi(B) = zi(o) +

r1 rij(o,0)[xj, - x(0)) (16)i-1

S

x'N = x'N + rrijNO(xjo - xio) (17)

j-1

where r > 1 is a parameter controlling the step size. r must be takenlarge initially and allowed to approach unity during the final stages ofconvergence.

9.3 OPTIMAL TEMPERATURE PROFILE BY

NEWTON-RAPHSON BOUNDARY ITERATION

As an example of the Newton-Raphson boundary-iteration algorithm weshall consider the problem of computing the optimal temperature profilein a plug-flow tubular reactor or batch reactor for the consecutive-

NUMERICAL COMPUTATION 275

reaction sequence

X, --> X2 - > products

We have examined this system previously in Sees. 4.12 and 6.10 and havesome appreciation of the type of behavior to be anticipated. Takingv = 1, F(x1) = x12 (second-order reaction), and G(x2) = x2 (first-orderreaction), the state is described by the two equations

z, = -k1oe-E,'""x,2 x1(0) = x10x2 = k10e-E"'l"x12 - k20e-E"/' x2 x2(0) = X20

with u(t) bounded from above and below

U. < U < u*

(1a)(lb)

(2)

The objective is the maximization of value of the product stream, orminimization of

-e[xl(9) - x,oj - [x2(9) - x2o1 (3)

where c reflects the value of feed x, relative to desired product x2.The Green's functions r;;(9,t) for the iterative algorithm are defined

by Eqs. (5) and (6) of the previous section, which become

- r,, af, - r,2

af2- -axl ax,

t12 = - r , ,af-t - r 12

af2-axt ax2

1'21 =

1'22 =

af,-r21ax, af2- r22ax,

= 2x,k,oe-E,'I"(r - r,2)

r11(9,9) = I (4a)

= k20e-Ei',"r,2 r12(8,0) = 0 (4b)

r,,(9,9) = 0 (4c)

- Of, - of,= E,-,"r2, r22 - k2oe- r22

ax2 ax2r22(9,9) = 1 (4d)

It easily follows that r12(9,t) =_ 0 and will not enter the computation,although we shall carry it along in the discussion for the sake of generality.

The hamiltonian for the optimization problem is

H = -Y,ktoe-E-'i"xt2 + Y2(ktoe-E,'1"x12 - k2pe-Ei'i"x2) (5)

where the Y; are computed from the r;; by

as71 =

a6W r,, + ax2 r2, _ -crl, - r21 (6a)

as asY2 = ax, r,2 + ax2 r22 = -cr,2 - r22 (6b)


Minimization of the hamiltonian leads to the equation for the optimal u(t)

u* v(t) > u*u(t) = v(t) u* < v(t) < u*

IU* v(t) < u*

where the unconstrained optimum is

E' - E'2 1U (t) =In y2x2k2o

('Y2 - yi)xi2kio

(7)

(8)

The computational procedure is now to choose trial values x1(6),22(6) and integrate the six equations (1) and (4) numerically fromt = 6 to t = 0, evaluating u at each step of the integration by Eq. (7)in conjunction with Eqs. (8) and (6). At t = 0 the computed values21(0), 22(0) are compared with the desired values x10, x20 and a new trialcarried out with values

x1(6) = 21(6) +r

{r11(6,0)[x1° - 21(0)] + r12(6,0)[x20 - 22(0)11

(9a)

X2(0) = 22(6) + {r2,(0,0)[x10 - 21(o)J + r22(e,0)[x2° - 22(0)11

(9b)

In carrying out this process it is found that convergence cannot beobtained when xi and x2 are allowed to penetrate too far into physicallyimpossible regions (negative concentrations), so that a further practicalmodification is to apply Eqs. (9) at any value of t for which a preset'bound on one of the state variables has been exceeded and begin a newiteration, rather than integrate all the way to t = 0.

The values of the parameters used in the calculation are as follows:

k1o=5X1010 k20=3.33X 1017E;=9X103 E2=17X103u* = 335 u* = 355

6=6 c=0.3x1o=1 x20=0

The iteration parameter r was taken initially as 2 and increased by 1each time the error, defined as Ix1(0) - x1ol + 1x2(0) - x2ol, did notdecrease. In the linear region, where the ratio of errors on successiveiterations is approximately I - 1/r, the value of r was increased by 0.5for each iteration for which an improvement was obtained. Startingvalues of x1(6) and x2(6) were taken as the four combinations of


x1(9) = 0.254, 0.647 and x2(0) = 0, 0.746, calculated from the limitingisothermal policies as defining the extreme values.

The approach to the values of x1(9) and x2(9) which satisfy thetwo-point boundary-value problem (x1 = 0.421, x2 = 0.497) are shownin Fig. 9.1, where the necessity of maintaining r > 1 during the earlystages to prevent serious overshoot or divergence is evident. Successivevalues of the optimal temperature profile for the iterations starting fromthe point xl = 0.647, x2 = 0 are shown in Fig. 9.2, where the ultimateprofile, shown as a broken line, is approached with some oscillation. Inall four cases convergence to within 0.1 in u(t) at all t was obtained withbetween 12 and 20 iterations.

As described in this section and the preceding one, the algorithmis restricted to problems with unconstrained final values. This restric-tion can be removed by the use of penalty functions, although it is foundthat the sensitivity is too great to obtain convergence for large valuesof the penalty constant, so that only approximate solutions ,can berealized. The usefulness of the Newton-Raphson method rests'ests in partupon the ability to obtain an explicit representation of the optimaldecision, such as Eq. (7), for if the hamiltonian had to be minimized byuse of the search methods of Chap. 2 at each integration step of everyiteration to find the proper u(t), the computing time would be excessive.

0.7

0.6

0.5

m0.4

0.3

Fig. 9.1 Successive approximations to 0.2final conditions using Newton-Raph-son boundary iteration. [From M. M.Denn and R. Aris, Ind. Eng. Chem. 0.1

Fundamentals, 4:7 (1965). Copyright1965b the American Chemical Societ 6/' "Y y. I I I e i

Reprinted by permission of the copyright 0.3 0.4 0.5 0.6 0.7owner.] x, (8)


Fig. 9.2 Successive approximations to the optimaltemperature profile using Newton-Raphson boundaryiteration. [From M. M. Denn and R. Aris, Ind. Eng.Chem. Fundamentals, 4:7 (1965). Copyright 1965 bythe American Chemical Society. Reprinted by permissionof the copyright owner.]

It is this latter consideration, rather than convergence difficulty, whichhas proved to be the primary drawback in our application of this methodto several optimization problems.

9.4 STEEP-DESCENT BOUNDARY ITERATION

The Newton-Raphson boundary-iteration algorithm convetges well nearthe solution to the two-point boundary-value problem resulting from theminimum principle, but it requires the solution of S(S - 1) additionaldifferential or difference equations for each iteration. An approachwhich requires the solution of fewer equations per iteration, but whichmight be expected to have poorer convergence properties, is solution ofthe boundary-value problem by steep descent. The error in satisfyingthe specified final (initial) conditions is taken as a function of the assumedinitial (final) conditions, and the minimum of this error is then found.

To demonstrate this procedure we shall consider the optimal-pressiire-profile problem for the gas-phase reaction

X1 - 2X2 -* decomposition products

where the conversion to intermediate X2 is to be maximized. Thisproblem was studied in Sec. 4.13, where the system equations werewritten as

xl = -2klu xlA + x2 x1(0) = x10

2

xz = 4klu A +X2

- 4k2u2 (A+ X2)2

X2(0) = x20

(1a)

NUMERICAL COMPUTATION Z!!

Here A = 2x,o + x2o, and the objective is to minimize

g = -X2(0)For simplicity u(t) is taken to be unconstrained.

The hamiltonian for optimization is.

(2)

H = yi x1 + y2 4k,u x, - 4kzu2x22

(_2kiu A + x2) A + xt (A + x2)21

(3)

with multiplier equations -

aH _ 2k,u CM

yl _ - 5x, A + X2 (y1 - 2yz) 7j(e) = ax, = 0 (4a)

aH 2k,,ux, 8k2uzyzAx212 = -

ax2 (A +x2)2 (yl 2X:} + (A '+ x2)3

'at'12(8) _

xz= -1 (4b)

The optimal pressure u(t) is obtained by setting aH/au to zero to obtainA + x2 k,x,(y, - 272) t )U

4 k2y2x225

It is readily verified that yz < 0, in which case a2H/au2 > 0 and thecondition for a minimum is met. We shall attempt to choose initialvalues y,(0), y2(0) in order to match the boundary conditions on y att = 8, and we shall do this by using Box's approximate (complex) steep-descent method, discussed in Sec. 2.9, to find y,(0), y2(0) which minimize

E = (71(8))2 + 112(8) 4- 1)2 (6)

We begin with at least three pairs y,(0), yz(0) and integrate Eqs. (1)and (3), calculating u at each step of the numerical integration fromEq. (5), and determine the value of the error E from Eq. (6) in each case.By reflecting the worst point through the centroid new values are founduntil E is minimized.

The parameters used for computation are

k, = 1.035 X 10-2 k2 = 4.530 X 10-2x10=0.010 x20=0.0028=8.0 A=0.022

For these parameters the solution of the boundary-value problem-is aty,(0) = -0.8201, 72(0) = -0.4563, with a value of x2(8) of 0.0132.Table 9.1 shows a sequence of iterations using a' three-point simplexwith one starting point close to the minimizing point, and convergenceto 1 part in 101 in y(8) is obtained in 17 iterations. Computations `frogother initial triangles are shown in Tables 9.2 and 9.3, and it is evidentthat poor results might be obtained without a good first estimate.

Tab

le 9

.1S

ucce

ssiv

e ap

prox

imat

ions

to th

e In

itial

val

ues

of m

ultip

liers

usi

ng th

e co

mpl

exm

etho

d fo

r st

eerd

esce

nt b

ound

ary

Itera

tion

to m

inim

ize

final

err

or in

bou

ndar

y co

nditi

ons

Iteration

- y,(0)

--y2(0)

E X 106

xs(B)

X10

'- y1(0)

- y:(0)

E X 10'

X2(8)

X10

--Y1(O)

-,Y2(O)

E X 10'

as(a

X10=

0.8000

0.4500

5.2 X 10'

1.130

0.5000

1.0000

3.9 X 1010

0.759

0.2000

0.9000

6.4 X 1010

0.732

10.8000

0.4500

5.2 X 10'

1.130

0.6000

1.0000

3.9 X 1010

0.759

0.8900

0.6317

2.3 X 100

0.955

20.8000

0.4500

5.2 X 10'

1.130

0.8879

0.4837

1.4 X 10'

1.123

0.8900

0.8317

2.3 X 10'

0.965

30.8000

0.4500

5.2 X 10'

1.130

0.8879

0.4837

1.4 X 10'

1.123

0.8470

0.4778

1.1 X 10'

1.129

40.8000

0.4500

5.2 X 10'

1.130

0.8385

0.4685

1.2 X 10'

1.132

0.8470

0.4778

1.1 X 10'

1.129

50.8000

0.4500

6.2 X 10'

1.150

0,8385

0.4685

1.2 X 10'

1.132

0.8045

0.4494

6.3 X 10'

1.132

60.8330

0.4637

7.6 X 10'

1.132

0.8386

0.4885

1.2 X 10'

1.132

0.8045

0.4494

6.3 X 10'

1.132

70.8330

0.4637

7.6 X 102

1.132

0.8082

0.4502

6.0 X 10'

1.132

0.8045

0.4494

6.3 X 10'

1.132

80.8330

0.4637

7.6 X 10'

1.132

0.8082

0.4502

6.0 X 10'

1.132

0.8291

0.4610

133

1.132

90.8110

0.4512

95.6

1.132

0.8082

0.4502

6.0 X 10'

1.132

0.8291

0.4810

133

1.132

10

0.8110

0.4512

95.6

1.132

0.8167 0.4544

22.3

1.132

0.8291

0.4610

188

1.132

11

0.8110

0.4512

95.6

1.132

0.8167

0.4544

22.3

1.132

0:8174

0.4547

8.6

1.132

12

0.8203

0.4563

6.5

1.132

0.4187

0.4544

22.3

1.132

0.8174

0.4547

8.6

1.132

13

0.8203

0.4563

6.5

1.132

0.8200

0:4561

0.14

1.132

0.8174

0.4547

8.6

1.132

14

0.8203

0.4563

6.6

1.132

0.8200

(r:4561

'0,14

.1.132

0.8194

0.4558

0.78

1.132

15

0.8199

0.4561

0.63

1.132

0.8200

0.4561

0.14

1.132

0.8194

0.4568

0.78

1.132

16

0.8199

0.4561

0.63

1.132

0.8200

0.4561

0.14

1.132

b.8202

0.4563

0.04

1.132

17

0.8200

0.4562

0.01

1.132

0.8200

0.4611

0.14

.1.132

0.8202

0.4563

0.04

1.132

Tab

le 9

.2Su

cces

sive

app

roxi

mat

ions

to th

e In

itial

val

ues

of m

ultip

liers

usi

ng th

e co

mpl

exm

etho

d fo

r st

eep-

desc

ent b

ound

ary

itera

tion

to m

inim

ize

fina

l err

or in

bou

ndar

y co

nditi

ons

Iter

atio

n-7

1(0)

-1'2

(0)

E X

106

x2(0)

X10

2-1

''(0)

-1'2

(0)

6.E

X 1

0$2(e)

X10

=-1

',(0)

-72(

0)6'

E X

10

x2(0)

X10

2

0.7000

0.4000

2.5 X 106

1.123

0.6000

0.5000

6.1 X 1010

0.882

1.0000

1.0000

6.8 X 101,

0.834

10.7000

0.4000

2.5 X 105

1.123

0.6000

0.5000

6.1 X 1010

0.882

0,4633

0:1567

7.9 X 106

- 1.916

20.7000

0.4000

2.5 X 10'

1.123

0.5869

0.3414

5.8 X 106

1.111

0.4633

0.1667

7.9 X 10,

-1.916

30.7000

0.4000

2.5 X 105

1.123

0.5869

0.3414

5.8 X 1011,

1.111

0.6014

0.3208

4.4 X 106

1.087

40.7000

0.4000

2-5 X 105

1.123

0.6847

0.3705

2.5 X 105

1.116

0.6014

0.3208

4.4 X 105

1.087

50.7000

0.4000

2.5 X 106

1_123

0.6847

0.3706

2.6 X 106

1.116

0.7409

0.4197

1.2 X 106

1.127

60.7900

0.4000

2.6 X 105

1.123

0.7121

0.4007,. 3,7 X 10'

1.130

0.7409

0.4197

1.2 X 106.

1.127

70.7406

0.4156

2.2 X 10'

1.131

0.7121

0.4007

3.7 X 10'

1.130

0.7409

0.4197

1.2 X 105

1.127

80.4156

2.2 X 10'

1.131

0.7121

0.4007

8.7 X 10'

1.130

0.7186

0.4020

1.4 X 10'

1.131

90.7406

0.4156

2.2 X 10'

1.131

0.7390

0.4131

9.7 X 10'

1.131

0.7186

0.4020

1.4 X 10'

1.131

10

0.7225

0.4032

1.1 X 10'

1.132

0.7390

0.4131

9.7 X 105

1.131

0.7186

0.4020

1.4 X 10'

1.131

11

0.7225

0.4082

1.1 X 10'

1.132

0.7390

0.4131

9.7 X 10'

1.131

0.7372

0.4115

8.1 X 10'

1.132

12

0.7464

0.4171

8.3 X 10'

1.132

0.7390

0.4181

9.7 X 10'

1.131.

0.7372

0.4115

8.1 X 10'

1.132

13

0.7464

0.4171

8.3 X 10'

1.132

0.7433

0.4149

7.2 X 10'

1.132

0.7372

0.4115

8.1 X 10'

1.132

14

0.7370

0.4111

7.9 X 10'

1.132

0.7433

0.4149

7.2 X 10'

1.132

0.7372

0.4115

8.1 X 10'

1.132

15

0.7370

0.4111

7.9 X 10'

1.182

0.7433

0.4149 7.2 X 10'

1.132

0.7417

0.4138

7.2 X 10'

1.132

16

0.7454

0.4161

6.9 X 10'

1.132

0.7433

0.4149

7.2 X 10'

1.132

0.7417

0.4138

7.2 X 10'

1.132

17

0.7454

0.4161

6.9 X 10'

1.132

.0.7437.. 0.415p.

6.9 X 10' 11.132

0.7417

0.4138

7.2 X 10'

1.132

Tab

le 9

.3Su

cces

sive

app

roxi

mat

ions

to th

e in

itial

val

ues

of m

ultip

liers

usi

ng th

e co

mpl

exm

etho

d fo

r st

eep-

desc

ent b

ound

ary

itera

tion

to m

inim

ize

fina

l err

or in

bou

ndar

y co

nditi

ons

Iter

atio

n-y

t(0)

- y,

(0)

E X

10'

=zX10(6) =

-y'(0

)-y

s(U

)E

X 1

08xxX10(6) =

--`(

U)

- y,

(0)

E X

106

x2 (8)

X10

2

0.4000

0.6000

8.4 X 10'

0.780

0.5000

0.8000

1.7 X 1010

0.774

0.7000

0.7000

3.3 X 10'

0.834

10.4000

0.6000

8.4 X 10'

0.780

0.5767

0.5700

2.1 X 10'

0.836

0.7000

0.7000

3.3 X 10'

0.834

20.7654

0.6537

1.2 X 10'

0.874

0.5767

0.5700

2.1 X 10'

0.836

0.7000

0.7000

3.3 X 10'

0.834

30.7654

0.6537

1.2 X 10'

0.874

0.5767

0.5700

2.1 X 10'

0.836

0.6566

0.5648

9.8 X 108

0.871

40.7654

0.6537

1.2 X 10'

0.874

0.7819

0.6302

7.7 X 108

0.894

0. 6-566

0.5648

9.8 X 108

0.871

5.0.6939

0.5675

6.9 X 108

0.889

0.7819

0.6302

7.7.x 108

0.894

0.6566

0.5648

9.8 X 108

0.871

60.6939

0.5675

6.9 X 108

0.889

0.7819

0.6302

7.7 X 108

0.894

0.7818

0.6170

6.2 X 108

0.902

70.6939

0.5676

6.9 X 108

0.889

0.7143

0.5720

6.0 X 108

0.897

0.7818

0.6170

6.2 X 108

0.902

80.7769

0.6089

5.7 X 108

0.905

0.7143

0.5720

6.0 X 108

0.897

0.7818

0.6170

6.2 )( 108

0.902

90.7769

0.6089

5.7 X 108

0.903

0.7143

0.5720

6.0 X 108

0.897

0.7263

0.5763

5.7 X 10'

0.900

10

0.7769

0.6089

5,.7 X 108

0.906

0.7716

0.6035

5.5 X 108

0.906

0.7263

0.5763

5.7 X 108

0.900

11

0.7340

0.5799

5.5 X 10'

0.902

0.7716

0.6035

5.5 X 108

0.906

0.7263

0.5763

5.7 X 108

0.900

12

0.7340

0.5799

5.5 X 108

0.902

0.7716

0.6086

6.6 X 10'

0.906

0.7669

0.6000

5.5 X 108

0.906

13

0.7340

0.5799

6.6 X 108

0.902

0.7392

0.5826

5.5 X 108

0.903

0.7669

0.6000

5.5 X 108

0.906

14

0.7632

0.5974

5.4 X 108

0.906

0.7392

0.5826

6.5 X 108

0.903

0.7669

0.6000

5.5 x 108

0.906

15

0.7632

0.5974

5.4 X 108

0.906

0.7788

0.6072

5.4 X 108

0.908

0.7669

0.6000

6.6 X 108

0.906

16

0.7632

0.5974

5.4 X 108

0.906

0.7788

0.6072

5.4 X 10'

0.908

0.7732

0.6035

5.4 X 108

0.907

17

0,7632

0.5974

5.4 X 10'

0.906

0.7625

0.5968

5:4 X 10'

0.906

0.7732

0.6035

5.4 X 108

0.907


For problems of the specific type considered here, where all initialvalues of the state variables are known and final values unspecified, wecan use a more direct approach to steep-descent boundary iteration.. Thevalue of the objective 6 depends only upon the choice of y(0), for every-thing else is determined from the minimum-principle equations if all initialconditions are specified. Instead of minimizing E, the error in final con-ditions, it is reasonable simply to seek the minimum of S directly by steep-descent iteration on the initial conditions. Tables 9.4 and 9.5 showthe results of such a calculation using the simplex-complex procedure.In neither case are the values of yi(O) = -0.8201, yz(0) _ -0.4563approached, although the same ratio 1.80 of these values is obtained.It is evident from Eq. (5) that only the ratio,-yl/,y2 is required for definingthe optimum, and Eqs. (4) can be combined to give a single equation forthis ratio. Hence the optimum is obtained for any initial pair in theratio 1.80 and, had we so desired, we might have reduced this particularproblem to a one-dimensional search.

9.5 NEWTON-RAPHSON FUNCTION ITERATION: A SPECIAL CASE

We have already seen how a computational' scheme of the Newton-Raphson type can be applied to boundary-value problems by lineari-zation of the boundary conditions. The theory of linear differential anddifference equations is highly developed, and we know that the principleof superposition can be used to solve linear boundary-value problems.Thus, we might anticipate that a Newton-Raphson linearization approachto the solution of differential equations would be practical. Before devel-oping the procedure in general it is helpful in this case to study a specificsimple example.

A convenient demonstration problem of some importance in physicsis the minimization of the integral

(2 [l + (x)11 dt (1)1 x

where the function x(t) is to be chosen in the interval 1 < t < 2 subjectto boundary conditions x(1) = 1, x(2) = 2. This is a special case ofFermat's minimum-time principle for the path of a light ray through anoptically inhomogeneous medium. In the notation of Sec. 3.2 we write

ff(x,x,t) = x-I[1 + (±)2J 4

and the Euler equation

day a3dt ax = ex

(2)

(3)

Tab

le 9

.4S

ucce

ssiv

e ap

prox

imat

ions

to th

e In

itial

val

ues

of m

ultip

liers

usi

ng th

eco

mpl

ex m

etho

d fo

r st

eep-

desc

ent b

ound

ary

Itera

tion

to m

inim

ise

the

obje

ctiv

e

Iter

atio

n-y

,(0)

-,y2

(0)

'Y-Y

12

(0)(

0)-h

X 1

0'-7

1(0)

-Y:(

0)(0)

(0)

Yt

--E

X 1

0'-y

,(0)

-y:(

0)W(O)

ys(0

)(0)

-S X

10'

0.5000

0.5000

1.00

0.834

0.5000

1.0000

0.50

0.759

0.2000

0.9000

0.22

0.732

10.5000

0.5000

1.00

0.834

0.5000

1.0000

0.50

0.759

0.6600

0.6700

0.99

0.831

20.5000

0.5000

1.00

0.834

0.6227

0.3637

1.71

1.107

0.6600

0.6700

0.99

0.831

30.5000

0.5000

1.00

0.834

0.6227

0.3637

1.71

1.107

"0.5087

0.3048

1.67

1.086

40.5739

0.3136

1.83

1.126

0.6227

0.3637

1.71

1.107

0.5087

0.3048

1.67

1.086

50.5739

0.3136

1.83

1.126

0.8227

0.8637

1.71

1.107

0.6460

0.3567

1.81

1.131

60.5739

0.3138

1.83

1.126

0.6129

0.3418

1.79

1.132

0.8468

0.3567

1.81

1.131

70.6591

0.3682

1.79

1.132

0.6129

0.3418

.1.79

1.132

0.6460

0.3587

1.81

1.131

80.6591

0.8682

1.79

1.132

0.6129

0.3418

1.79

1.132

0.6384

0.3544

1.80

1.132

90.6078

0.3381

1.80

-1.132

0.8129

0.3418

1.79

1.132

0.6384

.0.3544

1.80

1.132

10

0.6078

0.3381

1.80

1.132

0.6243

0.3474

1.80

1.132

0.6384

0.3544

1.80

1.132

Tab

le 9

.5S

ucce

ssiv

e ap

prox

imat

ions

to th

e in

itial

val

ues

of m

ultip

liers

usi

ng th

eco

mpl

ex m

etho

d fo

r st

eep-

desc

ent b

ound

ary

itera

tion

to m

inim

ize

the

obje

ctiv

e

Iteration

--y,(0)

--Ys(O)

'Yi(0)

72(0)

- 9 X 10'

-7L(O)

-'YS(O)

n(0)

72(0)

- X 10'

-.1(0)

--Y%(O)

y'(0)

(0)

- S X 10'

0.9000

0.5500

1.64

1.068

0.5000

1.0000

0.50

0.759

0.2000

0.9000

0.22

0.732

10.9000

0.5500

1.64

1.068

0.5000

1.0000

0.50

0.759

0.9667

0.7083

1.36

0.937

20.9000

0.5500

1.64

1.068

0.9873

0.5830

1.69

1.098

0.9667

0.7083

1.36

0.937

30.9000

0.5500

1.64

1.068

0.9873

0.5830

1.69

1.098

0.9408

0.5489

1.71

1.108

40.9982

0.5744

1.74

1.118

0.9873

0.6830

1.69

1.098

0.9408

0.5489

1.71

1.108

50.9982

0.5744

1.74

1.118

0.9600

0.5503

1.75

1.121

0.9408

0.6489

1.71

1.108

6.0.9982

0.5744

1.74

1.118

0.9600

0.5503

1.75

1.121

0.9995

0.5695

1.75

1.124

70.9699

0.5521

1.76

1.125

0.9600

0'.6603

1.76

1.121

0.9995

0.5695

1.75

1.124

80.9699

0.5521

1.76

1.125

0.9979

0.5665

1.76

1.126

0.9996

0.5696

1.75

1.124

90.9699

0.6521

1.76

1.126

0.9979

0.5665

1.76

1.126

0.9756

0.5539

1.76

1.126

10

0.9957

0.5645

1.76

1.127

0.9979

0.5665

1.76

1.126

0.9766

0.5689

1.76

1.126

I


reduces to the nonlinear second-order equation

x2 +(z)+1=0 x(1)=1

By noting thatx(2) = 2 (4)

xx + (x)2 =at

xx 2 it (d x) (5)

Eq. (4) can be integrated directly to obtain the solution satisfying theboundary conditions

x(t) = (6t - t= - 4))s (6)

We wish now to solve Eq. (4) iteratively by a Newton-Raphsonexpansion analogous to that developed for nonlinear algebraic equationsin Sec. 2.2. We suppose that we have an estimate of the solution,x(*) (t), and that the solution is the result of the (k + 1)st iteration,x(k+')(t). The nonlinear terms in Eq. (4) may be written

x(k+1)x(k+1) = x(k)x(k) + x(k) \\(x(k+l) - x(k)) + x(k) (i(k+1) - x(k))

+ higher-order terms (7a)(Z(k+1))2 = (±(k))2 + 21(k)(t(k+1) _, t(k))

+ higher-order terms (7b)

With some rearranging and the dropping of higher-order terms Eq. (5)then becomes a linear ordinary differential equation in x(k+1)

Z(k+1) + ) p+1) + 2(k) (k+1) x(k) + (x(k)x(k) 1xtk)

x(k+1)(1) = 1x(k+1) (2) = 2 (8)

A particularly convenient starting estimate which satisfies bothboundary conditions is

x(o) (t) = t (9)

in which case Eq. (8) for x(l) simplifies to

z(1) + x(n = 0 (10)

This linear homogeneous equation has two solutions, x(') = 1/t andx(1) = 1. Using the principle of superposition, the general solution isa linear combination of the two

x(1) = c1 + C2t-1 (ii)and the constants are evaluated from the boundary conditions at t 1

NUMERICAL COMPUTATION Z$7

andt=2x(1)(1) = 1 = C, + C2 (12a.)x(1)(2) = 2 = C1 + %C2 (12b)

The solution is then

x(') (t) = 3 - 2t-' (13)

Table 9.6 shows the agreement between the exact solution and thesefirst two Newton-Raphson approximations.

The starting approximation need hot satisfy all or any of theboundary conditions, though by use of superposition all subsequentapproximations will. For example, with the constant starting valuex(0) = 1.5 Eq. (8) for x(1) becomes

x(u = -23 (14)

The homogeneous solutions are x(') = 1, x(') = t, while the particularsolution obtained from the method of undetermined coefficients is -%t2.By superposition, then, the general solution is

x(1) = C1 + C2t - /St2 (15)

Solving for the. constants from the boundary conditions,x(1)(1) = 1 = c1 + c2 _- % (16a)

X(1) (2) = 2 = c, + 2c2 - % .(16b)

so that the solution is

x(1) = -2j3 + 2t - lj.3t2 (17)

Table 9.6 Comparison of exactsolution and approximation usingNewton-Raphson function iteration

t x(°)(t) xttl

1.0 1.000 1:000 1.0001.1 1.100 1.182 1.1791.2 1.200 1.334 1.3271.3 1.300 1.462 1.453.

1.4 1.400 1.572 1.5621.5 1.500 1.667 1.6581.6 1.600 1.750 i.7441.7 1.700 1.824 1.819

1.8 1.800 1.889 1.8871.9 1.900 1.947 1.9472.0 2.000 2.000 2.000

2ta

Table 9.7 Comparison of exactsolution and approximation usingNewton-Raphson function iteration

t x(0)(t) x(t)(t) x(t)

1.0 1.500 1.000 1.000

1.1 1.500 1.130 1.179

1.2 1.500 1.253 1.327

1.3 1.500 1.370 1.453

1.4 1.500 1.480 1.562

1.5 1.500 1.5&3 1.658

1.6 1.500 1.680 1.744

1.7 1.500 1.770 1.819

1.8 1.500 1.853 1.887

1.9 1.500 1.930 1.947

2.0 1.500 2.000 2.000


Table 9.7 shows the start of convergence for this sequence of -Newton-Raphson approximations, and considering the crude starting value, theagreement on the first iteration is excellent.

It is helpful to observe here that the principle of superposition canbe used in such a way as to reduce the subsequent algebra. We canconstruct a particular solution satisfying the boundary condition at t = 1,

x('D) = 3 - %J2 (1S)

To this we add a multiple of a nontrivial homogeneous solution whichvanishes at t = I

x(Ih) = t - 1so that the solution is written

x(1) = x(IP) + clx<u) - Y3 - 13t2

(19)

+ cl(1 - t) (20)

The boundary condition at t = 1 is automatically satisfied, and thesingle coefficient is now evaluated from the one remaining boundarycondition at t = 2, leading again to the result in Eq. (17).

9.6 NEWTON-RAPHSON FUNCTION ITERATION: GENERAL ALGORITHM

The example of the preceding section is a graphic demonstration of theuse of a Newton-Raphson linearization of the differential equation toobtain an approximate solution to a nonlinear boundary-value problem,but it is peculiar in that explicit analytical solutions are available forboth the exact and approximate problems. The general variational

NUMERICAL COMPUTATION 2$S

problem in which the decision function u(t) is unconstrained leads to 2Snonlinear first-order equations with S conditions specified at t = 0 (it = 0)and S conditions at t = B (n = N). We now consider the extension ofthe Newton-Raphson approach to this problem. This is frequentlyreferred to as quasilinearization.

We shall suppose that we have a system of 2S differential equations

yi = Fi(y) i=1,2,...,2S (1)

In a variational problem S of the variables would correspond to the statevariables x,(t) and the remaining S to the Green's functions y,(t). It isassumed that the condition all/au = 0 has been used to obtain uexplicitly in terms of x and y. If we number the components of y suchthat the first S components are specified at t = 0, we have the condition

Yi(0)NO i = 1, 2, . . . , Sunspecified i = S + 1, S + 2, . . . , 2S

(2)

Any S variables or combination might be specified at t = 0, dependingupon the nature of the problem.

We now assume that we have an nth approximation to the solutiony(n). If the (n + 1)st result is assumed to be the exact solution, wewrite Eq. (1) as

yi(n+l) = F,(Y(n+l)) (3)

and the right-hand side may be expanded in a Taylor series about thefunction y(n) as

zC

0i(n+1) = Fi(Y(n)) +

s

/aFi(Y(n))

(y1(n+l) - yi(n)) +i_1 ayi

or, regrouping and neglecting higher-order terms, ,

(4)

2SaFi(y(n))

zsaF,(y(n))C a yj(n+n + [F() _ y,(n)1 (5)

is1 y, 1

ay

Eq uation (5) is a set of linear nonhomogeneous ordinary differentialequations to which the principle of superposition applies. Thus, weneed obtain only a single particular solution which satisfies the S boundaryconditions at t = 0 and add to this, with undetermined coefficients, Ssolutions of the homogeneous equation

zsF,(y(n))

cn+n,n = --- y1(n+1).h (g)I yi

2" OPTIMIZATION BY VARIATIONAL METHODS

The S coefficients are then found from the S boundary conditions att = 8.

To be more precise, let us denote by yi(n+').p the solution of Eq. (5)which satisfies the initial condition

yi(n+l).p(0) Yiol

// arbitrary

21,2,...,5i=S+1,S+2, . . . 2S (7)

and by k = 1, 2, . . . , S, the linearly independent solutionsof Eq. (6) with zero initial conditions for i = 1, 2, . . . , S. The generalsolution of Eq. (5) may then be written

sy,(n+l)(t) = yi(n+1).p(t) + I ckyi(n+1).hh(t) i = 1, 2,

k-1

, 2S (8\

and the S constants ck are determined from the S specified conditionsat t = 8. For example, if y1(8), 112(8), and the last S - 2 values arespecified, the ck would be obtained from the S linear algebraic equations

s

yi(e) = yi(n+1).P(8) + Ik-1

i = 1, 3, S + 2, 8 + 3, . . . , 2S (9)

As outlined, the method is confined to variational problems inwhich there are no trajectory constraints on u or x but only specificationsat t = 0 and B. Since u must be completely eliminated by means of thestationary condition for the hamiltonian, constraints on u can be imposedonly if they are convertable to constraints on a and y, which can beincorporated by means of penalty functions. Finally, we note that acompletely analogous treatment can be applied to difference equations.

9.7 OPTIMAL PRESSURE PROFILE BYNEWTON-RAPHSON FUNCTION ITERATION

In order to demonstrate the use of Newton-Raphson function iterationwe shall again consider the optimal-pressure problem. The equationsare given in Sec. 9.4 as

i1 = -2k1uA + xs

x1(0) = x10

2

12 = 4k1u A +X2

- 4k2u2 (A +x2)2

S = -x2(8)x2(0) = x20


The Green's functions satisfy

2k1utit=A+X2(f1-27s) til(e)=02k1ux1 8k2u272A'x2rs = - (A + x2) 2 (11 - 272) + (A + x2)'

and the optimal pressure satisfies

A + x2 kjx&yl - 272)u(t) = _4 k272x22

72(8) = - 1

M

(3a)

(3b)

(4)

Substitution of Eq. (4) into Eqs. (1) and (3) leads to the four equationsin four variables

k12x12 71x1 =

2

_1 + - (5a)

k2x2 272

k12x12 712z2 = 1 - 2 (5b)

ksx22 7

k12x1tit = - 2ksysx22(71 - 272)2 (5c)

k12x1272

=(71 - 272) s (5d)

2x2a2k27

In the notation of the previous section x1; x2, 71i and 72 correspond,respectively, to y1, y2, y,, y4 Values at t = 0 are given for x1 and z2(y1 and y2) and at t = 0 for 71 and 72 (y: and y4). The linearized equa-tions for the iteration become, after some simplification,

k12(x1("))2 2 71(n)

k2(x2(n))2 x1(n) 272(n)

2 71("> 1

X2(ft)-1 + 272(")) x2(n+1) + 272(n)

7i(n+u

k12(x1(n) ) 2 7 i(n)) (6a)71(") 72(n+l) +2(7i(n))2 k2(X2( ( 72W

k12(x1("))2 2 (71(n))2

±2('1+0 = 1 - ] x1(.+,)k2(x2(n))2 x100 4(72("))2

2 - (71(n) 2 (n)

x2%) 1x2(n+l) - 2(y2(n))2 y1(n+1)

(71(n))2

(n+1) }k12(x1(n))2

L

_ (71(n)))

(sb)+ 2(72(n)2 72 +k2(x2(n))2 1

4(72(n) 2 J1


k12x1(n) (7'(n) - 272(n))2k2(x2(n)) 27z(n) 2x1(n)

(71(n) - 2v2(n))2x2(ntl) - (71(n) - 27,(n))71("+1)2X2 (n)

(7i(n))2 - 4(72(n))2

72(n+1)27x(n)

k12x1(n)

- (72k2(x2(n))27s(n)

i(n) - 27z("))2 (6c)

.y2(n+1) k12(x1(n))2 f(7i(n) - 2,,,(n))2xl(n+1)

k2(x2(n))172 (n) xl(n)

2x (n) (71(n) - 2y2')2 x2(n+l) + (71(n) - 272(n)) 71(n+1)

4(72(n))2 - (71(n))2

72(n+1)27x(")

k12(xl(n))22k2(x2(n))i72(n)

(71(n) - 272("))2 (6d)

The particular solution, with components x1(n+li.P, x2(n+1).P, 71(n+1).P,

72(n+l).P, is the solution of Eqs. (6) with initial conditions xlo, x20, 0, 1.(The latter two are arbitrary.) The homogeneous equations are obtainedby deleting the last term in each of Eqs. (6), and the homogeneous solu-tions, with components xl(n+l).hk, x2(n+l).Ak, 71(n+1).h,, 7:(n+1),hk, k = 1, 2,are solutions of the homogeneous equations with initial conditions 0, 0,1, 0 and 0, 0, 1, 1. (The latter two are arbitrary but must be inde-pendent for the two solutions and not identically zero.) The generalsolution can then be written

xl("+1)(t) = x1('+I).P(t) + c1x1(n+l).A,(1) + C2x1(n+1).k(t)x2(n+1)(t) = x2(n+1).P(t) + Cax2(n+l).A,(t) -4- c2x2(n+l).A,(t)

71(n+1)(t) = 7t(n+1),P(J) + C171(n+1).A,(t) + c271(n+l).k0)

72(ntll(t) = .2(n+1).P(t) + C172(n+1).A,(t) + C272(n+1).)y(t)

(74)(7b)(7c)(7d)

The initial conditions on xl and x2 are automatically satisfied, and thecoefficients cl and c2 are evaluated from the boundary conditions on 71and 72 at t = 0

71(8) = 0 = 71(n+l>.P(0) + c171(n+1).4,(0) + C271(n+l)Jy(0) (8a)72(0) = -1 = 72(n+1).P(0) + C172(n+l).A,(0) + C272 (0) (8b)

We show here some calculations of Lee using the parameters inSec. 9.4 with the following constant initial choices:

xl(0) = 0.01 x2(0) = 0.0171(0) = 0 72(0) = -.1.0


Convergence of xi and x2 is shown in Fig. 9.3, and the results of the firstfive iterations are listed in Table 9.8.. The rather poor starting valuesresult in rapid convergence, though with some oscillation, and it can,in fact, be established that when convergence occurs for this Newton-Raphson procedure, it is quadratic.

9.8 GENERAL COMMENTS ON INDIRECT METHODS

The computational techniques discussed thus far make use of the neces-sary conditiops for the optimal decision function at each iteration andare often called indirect methods. Reasonable starting estimates canoften be obtained by first assuming a function u(t) and` then solvingstate and multiplier equations to obtain functions or boundary con-ditions, as required, or by the use of the direct methods to be developedin the following sections. .

A common feature of. indirect methods is the necessity of solvingstate and Green's function equations simultaneously, integrating both

Fig. 9.3 Successive approximations to optimal concen-tration profiles using Newton-Raphson function itera-tion. [From E. S. Lee, Chem. Eng. Sci., 21:183 (1966).Copyright 1966 by Pergamon Press. Reprinted by per-mission of the copyright owner.]

$4 OPTIMIZATION BY VARIATIONAL METHODS

Table 9.8 Successive approximations to optimal concentration andmultiplier profiles using Newton-Raphson function Iteration t

i x((0) X 10' XI(" X 10' x;(') X 10' X 102 Xs(') X 10' x,(') X 10'

0 1.0000 1.0000 1.0000 .1.0000 1.0000 1.00002 1.0000 0.6434 0.6372 0.6620 0.6634 0.66344 1.0000 0.6370 0.5119 0.5332 0.5342 0.53436 1.0000 0.6607 0.4393 0.4486 0.4493 0.44938 1.0000 0.6602 0.3927 0.3860 0.3864 0.3864

i x:(0) X 10' x:(') X 10' x:(') X 10' x2(') X 102 xs(`) X 10' x:(') X 10'

0 1.0000 0.2000 0.2000 0.2000 0.2000 0.20002 1.0000 0.8471 0.8060 0.7795 0.7771 0.77704 1.0000 1.0215 0.9757 0.9610 0.9597 0.95976 1.0000 1.1011 1.0648 1.0651 1.0643 1.06438 1.0000 1.1392 1.1147 1.1324 1.1320 1.1320

t -ys(0) -yl(s) -yl(s) -71(3) - In(4)

0 0 1.2736 0.8555 0.8188 0.8202 0.82022 0 0.8612 0.6945 0.6785 0.6780 0.67804 0 0.5696 0.4868 0.4911 0.4913 0.49146 0 0.3032 0.2687 0.2648 0.2654 0.26548 0 0 0 0 0 0

I -ys(o) -y:(') -yea) -y:(') -y:(') -72(6)

0 1.0000 1.0908 0.5492 0.4625 0.4563 0.45822 1.0000 1.2665 0.7399 0.6647 0.6650 0.66504 1.0000 1.2592 0.8636 0.7915 0.7911 0.7911

6 1.0000 1.1320 0.9345 0.9002 0.8997 0.89978 1.0000 1.0000 1.0000 1.0000 1.0000 1.0000

t From E. S. Lee, Chem. Eng. Sci., 21:183 (1966). Copyright 1966 by PergamonPress. Reprinted by permission of the copyright owner.

from either t = 0 to 0 or 0 to 0. A potentially serious computationalproblem not evident in the examples can be observed by considering thesimplest of systems

z=ax+u

for which the Green's function equation is

y = -ay

(1)

(2)

NUMERICAL COMPUTATION 2!S

Solving both equations,

x(t) = x(0)e°' + f o' e°('-')u(r) dr (3a)

-y(t) = y(O)e-' (3b)

Thus, when either equation has decreasing exponential behavior, theother must be an increasing exponential. It is: for this reason that thestatement is often made that the optimization equations are unstable.In any indirect method extreme sensitivity and poor convergence willresult whenever 0 is significantly greater than the smallest time constantof the system.

9.9 STEEP DESCENT

The difficulties associated with the solution of the two-point boundary-value problem may be circumvented in many instances by adopting adirect approach which generalizes the steep-descent technique developedin Sees. 2.5 and 2.6. The necessary conditions for optimality are notused, and instead we obtain equations for adjusting estimates of thedecision function in order to improve the value of the objective. For sim-plicity we shall carry out the development with only a single decisionvariable.

The system is described by the ordinary differential equationswith known initial values

z; = f:(x,u) x1(0) = xco

where u(t) may be bounded from above and below

u* < u(t) < u*

(1)

(2)

The goal is to minimize 8[x(0)1, and for the present we shall assume.thatthe values of x(0) are entirely unconstrained.

If we choose any function u(t) which satisfies the upper- and lower-bound constraint, we may integrate Eq. (1) and obtain a value of 8.The effect of a small change ou(t) in the entire decision function is thendescribed to first order by the linear variational equations

ax` ax, + au` au ax1(0) = 0 (3)

and the corresponding first-order bhange in 8 is

ax,(0)8x;

ti

(4)

2!6 OPTIMIZATION BY VARIATIONAL' METHODS

All partial derivatives are evaluated along the solution determined byu. The Green's function for Eq. (3), as discussed in Sec. 6.2, mustsatisfy the adjoint equation

af;yJax;

Green's identity is then, noting that 6x;(0) = 0,

I y,(9) ax;(e) = fo`-Y.

L su dt

By defining boundary conditions for Eq. (5) as

-Mo) =asax;

we can combine Eqs. (4) and (6) to write

a6 = fy. u' au dto

(6)

(7)

(8)

Until this point the approach does not differ from the developmentof the weak minimum principle in Sec. 6.5. Now, as in Sec. 2.5, we makeuse of the fact that u(t) is not the optimum, and we seek au(t) so that tis made smaller, or S8 < 0. An obvious choice is

au(t) = -w(t) y; aut w(t) > 0 (9)

where w(t) is sufficiently small to avoid violation of the linearity assump-tion. Then

af;zSS = - fo w(t)y` au

dt < 0 (10)

At a bound we must take w equal to zero to avoid leaving the allowableregion.

It is helpful to examine an alternative approach which followsthe geometrical development in Sec. 2.6. We define distance in thedecision space as

nZ = fo g(t)[au(t)]2 dt g(t) > 0


and we seek on which minimizes (makes as negative as possible)

as =Io

y; u' au (t) dt (12)

This is an isoperimetric problem in the calculus of variations (Sec. 3.6)for which the Euler equation is

(au) y d ii au + xg(su)'] = 0 (13)

Here X is a constant Lagrange multiplier. .

Differentiating and solving for au, we obtain

1au = - 2xg(t) y'Of,T

;

From Eq. (11), then,

A2 = 12 r,9 l af, 2

dt4, o y` au

or, substituting back into Eq. (10),

G(t) y; af;/au

fr

(;au = -A[J0 G(r) \ y'

af,/au)2dr

(14)

(15)

(16)

where G(t) is the inverse of g(t) and the positive square root has beenused. For later purposes it is helpful to introduce the notation

\2rEE = f G(r) y; a') dr (17)

in which case

G(t) I y; af;/auau = -A _

1.", (18)EEE

Thus,

w(t)G(

t (19)1881,

where G is the inverse of the metric (weighting function) of the space.If we agree to define the hamiltonian

H = y;f: (20)


for any decision function, not just the optimum, the improvements inu may be conveniently written

Su = -zv(t)au (21)

In the case of more than one decision it easily follows that

5u, wi;61H

au

For systems described by difference equations

xtn = fn (x n-',un)

where S(XN) is to be minimized, an analogous development leads to

C1Hftaw - winau;n

with

Hn = \ 1tinliaLs

y'n-1 = yinaf"

axtp;-1 7tiNas

= aXN

In that case the equivalent to Eq. (19) is

GnZwn = - 0

It.6 4N

(22)

(23)

(24)

(25)

(26)

(27)

2 (20)I E I Gn (

The significant feature of steep descent should be noted here.The boundary-value problem is completely uncoupled, for the stateequations are first solved from t = 0 to 9. (n = 0 to N) and then themultiplier equations solved with known final condition from t = 9 tot = 0. If the state equations are stable when integrated from t = 0 inthe forward direction, the multiplier equations are stable in the directionin which they are integrated, from 0 to zero. One practical note ofprecaution is in order. When stored functions are used for numericalintegration, a means must be provided for interpolation, for most numeri-cal-integration algorithms require function evaluation at several loca-tloqs between grid points.

NUMERICAL COMPUTATION 2!!

9.10 STEEP DESCENT: OPTIMAL PRESSURE PROFILE

The method of steep descent is the most important of the computationaltechniques we shall discuss, and we demonstrate its use with severalexamples. The first of these is the optimal-pressure problem for con-secutive reactions studied previously in this chapter. The pertinentequations are

it = -2k,uA

X1

xl(1a)

2

xz = 4k,u A+

x2- 4kzu2

(A + x2)2(lb)

S = -X2(0) (2)

2k,u(Y1 - 2Y2) Y1(0) = 0 (3a)A+x2

2k,ux1 8k2u2Y2Ax2'Y2 = - (A + x2) 2 (y I - 272) + (A + X2)' 72(0) _ -1 (3b)

aH -2k, XI x22

T A + X2(-I, - 2Y2) - 8k2 (A + x2)2 u (4)

The values of the parameters are those used for previous calcu-lations. Following a choice of u, a(t), the new decision function is caleu-lated from the equation

unew(t) = u - W au

0.7

0.6

0.5

0.4

0.3

0.2

0.1

where Eq. (4) for all/au is evaluated for u and the solutions of Eqs. (1)and (3) are calculated using u. For these calculations w was initiallyset equal to the constant value of 25. The program was written so asto reduce w by a factor of 2 whenever improvement was not obtained,but effective convergence was obtained before this feature was needed.Figures 9.4 and 9.5 show successive pressure profiles computed from con-

Fig. 9.4 Successive approximations tothe optimal pressure profile using steepdescenti, starting from the constantpolicy u = 0.5.

(5)

0

1

2

3

5

0 0 1 2 3 4 5 6 7 8Residence time t

300

2 3 4 5 6Residence time t


Fig. 9.5 Successive approximations tothe optimal pressure profile using steepdescent, starting from the constantpolicy u - 0.2.

stant starting profiles of u = 0.5 and u = 0.2, respectively, with thevertical scale on the latter expanded to avoid cluttering. The dashedline is the solution obtained from the necessary conditions by the indirecttechniques discussed previously. The rapid convergence is shown in Fig.9.6, where the lower points are those for the starting value u = 0.5. Thenear-optimal values of the objective, 0.011305 after seven iterations and0.011292 after four, respectively, compared with an optimum of 0.011318,are obtained with profiles which differ markedly from the optimum overthe first portion of the reactor. This insensitivity would be an aid inultimate reactor design.

I I i I

I 2 3 4 5Iteration number

Fig. 9.6 Improvement of the objectivefunction on successive iterations usingsteep descent.


9.11 STEEP DESCENT: OPTIMAL TEMPERATURE PROFILE

As a second example of steep descent we consider the tubular-reactoroptimal-temperature-profile problem solved by Newton-Raphson bound-ary iteration in Sec. 9.3. The equations for the system are

±1 = -kloe-E,'1ux12 (la)22 = kloe El'I"xl - k2oe E='I"x2 (lb)U* < u < u* (2)s = -c[xl(O) - x1o1 - [x2(e) - x20] (3)

The Green's function equations are then

til = 2xlkloe-E,'1,(y1 - 72) 71(9) = -c72 = k2oe-Ei"/°y2

72(0) = -1

and the decision derivative of the hamiltonian is readily computed as

aH kloEixl2e-E,'tn k2oEZx2e-Ez'I'

au - u2 (71 - 72)u2

y2 (5)

The parameters for computation are the same as those in Sec. 9.3. Thenew decision is calculated from the equation

uuew(t) = u(t) - w(t) aA reasonable estimate of w(t) may be obtained in a systematic way.

Following the geometric interpretation, we may write, for uniformweighting,

w(t) = a (7)

[fa (aIf data are stored at N uniform grid points spaced At apart, then,approximately,

w(t) =A

In', [ a

(tn1

) IZ At135

(4a)(4b)

(6)

or, combining (At)3h into the step size,

w(t) = S(9)

I nI1 [",(tn)]T


If au is to be of the order of one degree in magnitude then

b(aH/au)N

2 ;4[IH

or

(10)

S N34 (11)

For simplicity b is taken as an integer. Equation (6) is written forcomputation as

unew = u - b (t) Nell/au

lI

{ 1,[_5_U (in)ls}1

(12)

where b(t) is a constant b, unless Eq. (12) would cause violation of oneof the constraints, in which case b(t) is taken as the largest value whichdoes not violate the constraint. The constant b is taken initially as thesmallest integer greater than N4 and halved each time a step does notlead to improvement in the value of S. For the calculations shown hereN = 60, and the initial value of b is 8.

Figure 9.7 shows 'successive iterations startijig from an initial con-stant policy of u = u* except over the first integration step, where u islinear between u* and u.. This initial segment is motivated by the

10II64

2

Oi i i

I 2 3 4 5 6Residence time f

Fig. 9.7 Successive approximations to the optimal tem-perature profile using steep descent, starting from the con-stant policy u a 335. [From J. M. Douglas and M. M.Denn, Ind. Eng. Chem., 57(11):18 (1965). Copyright 1965by the American Chemical Society. Reprinted by permissionof the copyright owner.]


355

aE

340

303

3351 I l I 1 I

0 1 2 3 4 5 6Residence time f

Fig. 9.8 Successive approximations to the optimal tem-perature profile using steep descent, starting from the con-stant policy u = 355. [From J. M. Douglas and M. M.Denn, Ind. Eng. Chem., 57(11):18 (1965). Copyright 1965by the American Chemical Society. Reprinted by permissionof the copyright owner.]

solution to the generalized Euler equation in Sec. 4.12, which requiresan infinite initial slope for profiles not at an upper bound. The methodof choosing the weighting w(t) in Eq. (12) is seen to produce changes inu(t) of order unity, as desired. The dashed line is the solution obtainedusing Newton-Raphson boundary iteration. Successive values of theprofit are shown as the upper line in Fig. 9.9, and it can be seen thatconvergence is essentially obtained on the eleventh iteration, though thetemperature profile differs from that obtained using the necessaryconditions.

0

Fig. 9.9 Improvement of the objec-tive function on successive iterationsusing steep descent. [From J. M.Douglas and M. M. Denn, Ind. Eng.Chem., 57(11):18 (1965). Copyright1965 by the American Chemical Society.Reprinted by permission of the copy-right owner. ]

' 0.30

00.25

2 4 6 8 10 12 14Iteration number


Calculations starting from an initial constant policy u = u* areshown in Fig. 9.8 and the lower curve in Fig. 9.9, and the conclusions aresimilar. The two calculations lead to values of the objective of 0.3236and 0.3237, respectively, compared to a true optimum of 0.3238. It istypical of steep descent procedures that convergence is rapid far fromthe optimum and very slow near an insensitive minimum or maximum.Though a frustration in calculation, this insensitivity of the optimum is,of course, an aid in actual engineering design considerations.

9.12 STEEP DESCENT: OPTIMAL STAGED TEMPERATURES

We can examine the use of steep descent for a staged system byconsidering the problem of specifying optimal temperatures in a sequenceof continuous-flow reactors of equal residence time. The reactionsequence is the one considered for the tabular reactor

X1--+ X2 -+ products

with all rate parameters identical to those in Sec. 9.3. We have con-sidered this problem in Secs. 1.12 and 7.6, where,. with F quadratic andG linear, the state equations and objective are written

0 = xIn-1 x1n - 8k1oe-E,'1u"(xln)2 (1a)

0 = x2w-1 - x2" + 8kioe-&,'lu'(x1n)2 - 8k2oe-8'"Ir"x2n (lb)t; = -c(x1N - x10) - (x2'' - x20) (2)U* < un < u* (3)

By implicit differentiation the Green's function equations are shown inSec. 7.6 to be

.Y,.-I = -/In1 + 28k10ee,'/u"x1n

28y2nk1oe 8,'l u'xln

+ (1 + 28k1oe 8,'/u"x1")(1 + e,'lue) y1" = -c (4a)

72N = -1 (4b)ye-1 = y "J,/4-*1 + 8k2oe-

The decision derivative of the stage hamiltonian is

aH"au"

T2"EIk10e 8ilu+(x1")2 - y1#E1kioe-,R,'/v*(X1n)1(1 + ek29e

8 - y2"E=k2oe-a,'l""x2"(1 + 28k1oe-8-'lu"xi%)

(un)2 (1 + 8k2oe-e-'/u")(1 + 28kioe'80"x1")(5)

Following the specification of a temperature sequence {in I and thesuccessive solution of Eqs. (1) and Eqs. (4), the new decision is obtained


from the algorithm of Eqs. (20) and (23) of Sec. 9.9

aH^/au"n NJ

H)2]½

(6)

where we have taken the weighting as uniform. If a bound is exceededby this calculation, that value of u" is simply moved to the bound, asin the previous section. The starting value of A is taken as the smallestinteger greater than N" in order to obtain starting-temperature changesof the order of unity. A is halved each time an improvement does notoccur on an iteration.

The residence time 9 was chosen so that No equals the tubular-reactor residence time of 6 used in the preceding numerical study.Figure 9.10 shows the convergence for a three-stage reactor system(N = 3, 0 = 2) for starting values entirely at the upper and lower bounds.

355

350

E

340

335

0 5 10 15Iteration number

U,

u2

U3

Fig. 9.10 Successive approximations to the optimal tem-perature sequence in three reactors using steep descent.[From M. M. Denn and R. Aria, Ind. Eng. Chem. Funda-mentals, 4:213 (1965). Copyright 1965 by the AmericanChemical Society. Reprinted by permission of the copyrightowner.]


The calculations in Fig. 9.11 show convergence for a 60-stage reactor(N = 60, 0 = 0.1) from the lower bounds, with the discrete profilerepresented as a continuous curve not only for convenience but also to,emphasize the usefulness of a large but finite number of stages to approxi-mate a tubular reactor with diffusive transport. For the latter case 22iterations were required for convergence, defined as a step size of lessthan 10-3.

This process is a convenient one with which to demonstrate theuse of penalty functions for end-point constraints. If we take theobjective as simply one of maximizing the amount of intermediate

g = -x2N (7)

and impose the constraint of 60 percent conversion of raw material

x1N = 0.4 (8)

then the penalty-function approach described in Sec. 1.16 would sub-stitute the modified objective

= -x21 + %K(x1N -0.4) 2 (9)

In the minimization of 9 the multiplier equations (4) and direction ofsteep descent described by Eqs. (5) and (6) do not change, but the

355

335 ___ 1 I I I I I

1 5 10 20 30 40 50 60Stoge numbern

Fig. v.11 Successive approximations to the optimal tem-perature sequence in a 60-stage reactor using steep descent,starting from the constant policy u = 335. [From M. M.Denn and R. Aria, Ind. Eng. Chem. Fundamentals, 4:213(1965). Copyright 1965 by the American Chemical Society.Reprinted by permission of the copyright owner.!


boundary conditions on the multipliers do. We now have

'Y ]Naaat

1N = K(z1N - 0.4)

= ai;72 N - ax2N = - 1

(10a)

(10b)

Following convergence of the iterations for a given value of the penaltyconstant K and a corresponding error in the constraint a new value ofK can be estimated by the relation

x1N - 0.4Knew _ K°Id tolerance

For these calculations the tolerance in the constraint was set at 10-1.Figure 9.12 shows the result of a sequence of penalty-function

calculations for N = 60 starting with K = 1. Convergence was obtainedin 17 iterations from the linearly decreasing starting-temperature sequenceto the curve shown as K = 1, with an error in the constraint x1N = 0.4of 1.53 X 10-1. Following Eq. (11), the new value of K was set at1.53 X 102, and 107 iterations were required for convergence, with aconstraint error of 8.3 X 10-1. A value of K of 1.27 X 10' was thenused for 24 further iterations to the dashed line in Fig. 9.12 and an errorof 10-1. This extremely slow convergence appears to be typical of thepenalty-function approach and is due in part to the sensitivity of theboundary conditions for y to small changes in the constraint error, asmay be seen from Eq. (10a).

355

340K'I.53 X102

5 t0 20 30 40 50 60Stage number n

Fig. 9.12 Approximations to the optimal temperaturesequence with constrained output using steep descent andpenalty functions. (From M. M. Denn and R. Aris, Ind.Eng. Chem. Fundamentals, 4:213 (1965). Copyright 1965by the American Chemical Society. Reprinted by permissionof the copyright owner.]

me OPTIMIZATION BY VARIATIONAL METHODS

9.13 GRADIENT PROJECTION FOR CONSTRAINED END POINTS

By a direct extension of the analysis in Sec. 9.9 we can generalize thesteep-descent procedure to account for constrained end points. We againconsider a system described by the differential equations

z; = f;(x,u) x;(0) = x;o (1)

with bounds on u,

u* <u<u* (2)

The objective is to minimize S[x(6)], but we presume that there is also asingle (for simplicity) constraint on the final state

41 (0)] = 0 (3)

For any function u(t) which lies between the upper and lower boundwe can integrate Eqs. (1) to obtain a value of & and of 0 which willgenerally not be zero. The first-order effect of a small change is thendescribed by the equations

ai; = af' ax, + a' su ax;(0) = 0 (4)

with corresponding first-order changes in g and 0

, ax,(e)aa; _a

ax,

ax,(9)ax;

(6)

We now define two sets of Green's functions for Eq. (4), y and 4, satis-fying the adjoint differential equation but different boundary conditions

axi

af, a11

ax; ax;

(7)

(8)

Using Green's identity, Eqs. (5) and (6) will then become, respectively,

SS = fo y; au au dt

a =Io

E ,y; af' su dt

(9)

(10)


We now choose a distance in the decision space

02 = fo g(t)[Su(t)]2 dt

and ask for the function Su which minimizes M in Eq. (9) for a fixeddistance 0' and a specified correction to the constraint 34. This is an iso-perimetric problem with two integral constraints, and hence two constantLagrange multipliers, denoted by X and is. The Euler equation is

f af;a(Su) I ry a SU au Su + Xg(Su)2 = 0 (12)

or

au - -TX-('Y.+i4)au

where G(t) is the inverse of g(t).We need to use the constraint equations (10) and (11) to evaluate

X and P. By substituting Eq. (13) for Su into Eq. (10) for the fixedvalue 6.0 we obtain

-y, Ta

dr

It is

dr- fo G(r)aui

2

convenient to define three integrals2

IEE = fa G(r) yi au dT

IEm = Ja G(r) ti; a'} ( 'G` au' l dr

I,, = fa G(r) I ', au' dr

(14)

(15a)

(15b)

(15c)

The subscripts denote the appropriate boundary conditions for theGreen's functions used in the integral. Equation (14) then becomes,slightly rearranged,

IcmP

+ A so= -Ioe

(16)

If Eqs. (13) and (16) are now substituted into Eq. (11) for A2, we obtain,after some algebraic manipulation,

1 - ± 1I L2 - (6.0)2

TX-L

1Immlee - J 2 J

(17)


Finally, then, Eq. (13) for bu becomes

bu = -G(t) I loaf e&-(I&m2 1 (y' I## #') au

+ G(t) 1o #i a '` (18)

The presence of the square root imposes an upper limit on the correction30 which can be sought, and a full correction cannot generally be obtainedin a single iteration. The procedure outlined here is geometrically equiv-alent to projecting the gradient onto the subspace defined by the con-straint. Identical equations are obtained for systems with differenceequations if integrals are replaced by sums and continuous variables bytheir discrete analogs.

If the gradient-projection procedure is applied to the reactor-tem-perature problem of the preceding section with ¢(xN) = x1N - 0.4, theonly additional computations required are the recalculation of Eqs. (4)of that section, substituting 4'1", 412" for yl", y2", with boundary conditions1, 0 on *N and 0, -1 on yN, followed by calculation of the sums Is., Is*fand Ii,. Computations were carried out using a maximum correction60 of 0.01 in magnitude for each iteration. Lines I and II in Fig. 9.13

0.25

0.20

0.150

00.10

w-0.05

-0.10 11 1 1 1 1 1


Fig. 9.23 Approach to final value con-straint using steep descent and gradi-ent projection. [From M. M. Dennand R. Aris, Ind. Eng. Chem. Funda-mentals, 4:213 (1965). Copyright 1965by the American Chemical Society. Re-printed by permission of the copyrightowner.]


show the convergence of the constraint for N = 3. 0 = 2 from startingpolicies at the lower and upper bounds, respectively. The correction isseen to be linear. Line III shows the convergence for the 60-stage sys-tem, N = 60, 0 = 0.1, from the starting policy in Fig. 9.12. Twenty-two iterations were required here for convergence to the same optimalpolicy as that shown in Fig. 9.12 for K = 1.27 X 103. In all cases thefinal error in the constraint was less than 10-5.

9.14 MIN H

Steep descent provides a rapidly convergent method of obtaining approxi-mate solutions to variational problems, but it is evident from the examplesthat ultimate convergence to the solution of the minimum-principlenecessary conditions is slow and computationally unfeasible. The min-Hprocedure is one which can be used only in the latter stages of solutionbut which will, with only minor modifications of the computer code,lead to a solution satisfying the necessary conditions.

The system and multiplier equations are

x; = f;(x,u) x;(0) = x;o

y; - - Yi ax, y;(0)ax;

(1)

(2)

For a given decision function f4(t) the solutions to Eqs. (1) and (2) aredenoted as Z(t), Y(t). Writing the hamiltonian in terms of these lattervariables, we have

(3)

The new value of u is chosen so that H is minimized. In that wayconvergence is forced to the solution of the necessary conditions. Inpractice severe oscillations and divergence may occur if the full correc-tion is used, so that if we denote the function obtained by minimizing Hin Eq. (3) as il, the new value of u is obtained from the relation

r (4)

A value of r = 2 has generally been found to be satisfactory. Modi-fications similar to those in Sec. 9.13 can be made to accommodateend-point constraints.

The relation to steep descent may be observed by considering the.special case for which the minimum of H occurs at an interior value.


Then the minimum, t!, satisfies the equation

y' au 5

This can be expanded about u for a Newton-Raphson solution as

at;(31,a `Jt

+ L. ry`a2};(2,u) 0 - u) + .. . = 0 (6)au au2

a

and if the higher-order terms can be neglected, the min-H procedure isequivalent to

Ca2H J aHuOCM u -

au2J au(7)

That is, we get nearly the same result as a steep-descent procedure withthe inverse of a2H/au2 as the weighting factor w. At the minimum thehessian of H is positive, and so Eq. (7) will allow convergence, and thismust be true by continuity arguments in a neighborhood of the minimum.For staged systems we have no strong minimum principle in general, anda2H/au2 need not be positive near the optimum, so that this proceduremight lead to divergence arbitrarily close to the minimum in a stagedsystem.

As an example of the convergence properties of the min-H algorithmwe consider the problem of the optimal tubular-reactor temperatureprofile for the consecutive reactions. The state and multiplier equations(1) and (4) of Sec. 9.11 are solved sequentially to obtain xl, 22, tit, titfor the specified u. Using the necessary conditions in Sec. 9.3, we thenfind il from the relation

tZ =u*v(t)

u(t) > u*u* < v(t) < u* (8)

with

(t)

U* u(t) < u*

E_ - E; (9)u

x k12 2 2oIn

(tit - 'Y1)xl2klo

For these computations the initial profile was taken as the constantu = 345 and r was set at 2. The results are shown in Table 9.9, where eis the integral of the absolute value of the difference between old andnew temperature. The stabilizing factor r = 2 clearly slows convergencefor the small values of t. Convergence is not uniform and, indeed, S isnot uniformly decreasing, but very rapid convergence to a solutionsatisfying the necessary conditions is obtained in this way. Numerical

Tab

le 9

.9S

ucce

ssiv

e ap

prox

imat

ions

to o

ptim

al te

mpe

ratu

re p

rofil

e us

ing

the

min

-H m

etho

d

Iter

atio

n N

umbe

r

t0

12

46

810

1214

0345.00

350.00

352.50

354.38

354.84

354.96

354.99

355.00

3:15.00

0.5

345.00

350.00

352.50

354.38

354.84

354.96

354.99

355.00

335.00

1345.00

350.00

349.88

349.20

348.86

348.76

348.70

348.74

348.74

2'

345.00

345.76

344.61

344.19

344.15

344.16

344.16

344.16

344.16

3345.00

343.89

342.50

342.10

342.11

342.12

342.13

342.13

342.13

4345.00

342.94 ,

341.36

340.89

340.89

340.90

340.91

340.91

340.91

5345.00

342.38

340.67

340.09

340.06

340.07

340.07

340.07

340.07

6345.00

342.05

340.21

339.52

339.46

339.46

339.46

339.46

339.46

x,(6)

0.4147

0.4088

0.4186

0.4196

0.4196

0.4196

0,4196

0.4196

0.4196

x,(6)

0.4952

0.4999

0.4977

0.4973

0.4974

0.4974

0.4974

0.4974

0.4974

-s;

0.3196

0.3225

0.3233

0.3232

0.3233

0.3233

0.3233

0.3233

0.3233

29.7

18.1

4.3

0.70

0.29

0,086

0.022

0.0054

0.0013


values differ somewhat from those indicated in earlier calculations forthis system because of the use of a different numerical-integrationprocedure and a constant value of u over an integration step instead ofusing interpolation.

9.15 SECOND-ORDER EFFECTS

In looking at the convergence properties of steep descent for problems indifferential calculus in Sec. 2.8 we found that by retaining second-orderterms in the steep-descent analysis we were led to a relation with quad-ratic convergence which was, in fact, equivalent to the Newton-Raphsonsolution. We can adopt the same approach for the variational problemin order to find what the pertinent second-order effects are.

It helps to examine the case of a single state variable first. Wehave

x = J(x,u) (1)

and we seek to minimize 6[x(9)]. If we choose u(t) and expand Eq. (1)to second order, we have

bx-axbx+aubu+ax2bx2+axaubxbu+au bu2

bx(0) = 0 (2)

The corresponding second-order change in E is

Sa; = g' bx(9) + 3E" 5x(9)2 (3)

where the prime denotes differentiation. Our goal is to choose bu sothat we minimize bs. The quadratic nature of the problem prevents anunbounded solution, and it is not necessary to specify a step size in thedecision space.

Equations (2) and (3) define a variational problem of the type wehave studied extensively. Here the state variable is ax and the decisionSu, and we wish to choose bu to minimize a function [Eq. (3)] of bx(9).The hamiltonian, which we denote by h, is

of of 1 '92f a2f 1 '92fhax

bx + au Su +2 axe bx2 + ax au bx bu +

2 5u2Sue (4)

where the Green's function is denoted by ¢. For unconstrained bu, then,we can set ah/8(bu) to zero to obtain

(5)GOf 2Jbu = - aJu2) au + axaf

8)The Newton-Raphson approximation to min H, Eq. (7) of Sec. 9.14,would correspond to Eq. (5) without the term a'f/(ax au) Ox, so that

NUMERICAL COMPUTATION 31S

min H is clearly not the complete second-order correction for' steepdescent.

We can carry the analysis somewhat further. The first-order solu-tion to Eq. (2) is

{

bx =ft

exp I j ax (Q) da] au (r) au(,) dr (6)

Equation (5) can then be written as a Volterra-type integral equationfor the correction Bu(t)

Bu(t) + (L\auz_I ax a 1o exp Cllr ax (o)] au (r) bu(r) dr

_ _ a2f -1 of (7)au2> au

There is no computational advantage in doing so, however, for the solu-tion of an integral equation is not, an easy task and Eq. (5), although itcontains the unknown ox, can be used directly. Noting that ax is simplyx - x, it follows that the best second-order correction bu is a knownexplicit function of x, which we denote as bu(x). Equation (1) can thenbe written

x = fix, a + bu(x)] (8)

which can be integrated numerically in the usual fashion. The newvalue of u is then constructed from Eq. (5).

9.16 SECOND VARIATION

A complete second-order approach to the numerical solution of vari-ational problems requires that we generalize the result of the precedingsection to a system of equations. We consider

i, = fi(x,u) (1)

with u(t) to be chosen to minimize s(x(e)]. We shall assume x(O) to becompletely unspecified, though the extension to constrained final valuescan be carried out. Unlike the quasi-second-order min-H procedure, wemust assume here that u is unconstrained. Fdr notational conveniencewe shall use the hamiltonian formulation in what follows, so that we define

H= Yf, (2)

(3)


Now, if we expand Eq. (1) about some choice u(t) and retain termsto second order, we obtain

a2 =

64 =ax;

aui 6U + ,-,OXJ OX

ax; axk +8x,

t9u ox, Ou).k

+1

a

2f,au= azi(o) = 0 (4)

The second-order expansion of the objective is

z 8

as = a axt(e) + - ax ax; ax1(a) az;(B) (5)

This defines a variational problem, for we must choose bu(t), subject tothe differential-equation restriction, Eq. (4), so that we minimize SE inEq. (5). Denoting the hamiltonian for this subsidiary problem by h andthe multiplier by 4, we have

ax; axk44 ax` ax; + I.C u au +z

axaxk0 , i ij.k '

_ A _C -

a(ax;)

a=f; - 1 a=fi+ ax1 au

ax; au + 2-au=

au= (6)

f.

axk 4'i2af ;

- /C,

a , aftalb

Y'` ax; k Y' ax; axk ax; au

a(as) aE+ 12-'8- axk7( ) a(ax;) = ax; k ax, axk

(7)

(8)

By assuming that u is unconstrained we can allow au to take on anyvalue, so that the minimum of h is found by setting the derivative withrespect to au to zero

ah cc of i a=f; cV"'

a=f iau o

ax, au ax' + ==

a(bu) au + 1i jJ t

or

(9)

a=ft

I afta=f

ssu = -

auz au +4'i

ax au ax; j (10)i id

There are notational advantages to using the hamiltonian andmultiplier for the original problem, defined by Eqs. (2) and (3). Wedefine 57 by

5Yi=4',-Y; (11)


From Eqs. (3) and (7),

9vi

a2H a2f;axk - Sy;k az; axk ; k ax; axk

axk

a2H a2fau

ax; au au - ay; ax; au (12)

while the boundary condition is obtained from Eqs. (3) and (8)I a2& I a2&

Sy;(B) =ax; + ax; axk axk - ax; k ax, axk 6xk(9) (13)

Equation (10) for au becomesa2H

au auI Lfi )

2

ay`uLfi +G ax auSx'

1

'

+ I ay`aax;2fau ax;) (14)

.1

We require a solution to the system of Eqs. (1) and (12) to (14) whichcan be implemented for computation without the necessity of solving aboundary-value problem.

The fact that we have only quadratic nonlinearities suggests theuse of the approximation technique developed in Sec. 4.8. We shallseek a solution of the form

ay;(t) = g;(t) + I M;k(t) axk(t) + higher-order terms (15)k

The functions g; and M;k must be obtained, but it is evident from Eq. (13)that they must satisfy the final conditions

g,(8) = 0 (16a)2

M'k(B) = ax, axk (16b)

We can also express Eqs. (4), (12), and (14) to comparable order as

ax; _ I az` ax; + au` au + higher-order terms (17)

: a 2a;S' Iy' ay` ax; - ax; axk axk - ax; au

au

+ higher-order terms (18)

au(a2H)-l aH + ayt aJ; + a2H

au au au ax; au

-- higher-order terms (19)


Substituting Eq. (15) into Eqs. (18) and,.(19), we obtain{ f

a Yi = - 9i M". ax, afi - I a_axk

i kax; k ax; axk

612H ,2H)-1t0H + I 9s afi+ I Mik axk afi

ax; au au au i aui,k

au

+ axk auazk)

(20)

On the other hand it must be possible to differentiate Eq. (15) with respectto t and write

aii=#;+I M;kaxk+I mo,61kk k

or, substituting from Eqs. (15) and (17) to (19),

ox; _ +M;kaxk+\'M;k afk axik k,i ax;

LA=-1 /aH afi afi

+ M", au(̀ au au + 9i au + M`' axpau

k

(21)

I .32H+- axi (22)

i axi au

The coefficients of (&x)° and (6x)1 in Eqs. (20) and (22) must be thesame, leading to the equations

Mii + [r afk(8211)_I

afk a2H

Iafk

axi au= au au axi axjk Mki + Mik

a=H Ma=H 1 ofM- (!)'

au=au au ax; - ( k 'k au) (au=> ( au'ia2 H _ -1 2 :aH aH aH aH =0 Mi;(9) =

a8(23)

+ azi ax; axi au au= axi au axi ax;

_ ( afk (a=H\-' at; _ af, af, a=H 1gi L Mik au au= J au ax; + au= au au axi J 9i

k

of a=H-1 aH H 82H -1 aHau (au=) au - axi

0Hau au=) au - 0 gi(9) = 0

(24)

Equation (23) for the "feedback gain" is simply the Riccati equationobtained for the optimal control problem in Sec. 8.2, and Mi; is symmetric

NUMERICAL COMPUTATION 31!

(M;; = M;;). As convergence occurs and 8H/au goes to zero, the forcingterm in Eq. (24) also goes to zero and the "feedforward gain" vanishes.The correction to the decision, bu(t), is then obtained from Eqs. (19)and (15) as

bu(t) =a 2 /-1 Mau / + g' aut

\+ au' M,:1 (x: _f:) (25)

where x; is the (as yet unknown) value of the state corresponding to thenew decision function.

The computational algorithm which arises from these equationsis then as follows:

1. Choose u(t), solve Eqs. (1) for x and then (3) for 7 in succession, andevaluate all partial derivatives of H.

2. Solve Eqs. (23) and (24), where all coefficients depend upon 2(t),!(t), and Y(t).

3. Compute u(t) from the expression

unew(x,t) = t(t) +T

bu(t) (26)

where bu is defined by Eq. (25) and r is a relaxation parameter,r > 1. Note that unew is an explicit function of x.

4. Solve Eq. (1) in the form

x: _ filx,unew(x,t)J (27)

simultaneously calculating u(t), and repeat until convergence isobtained.

It is interesting to note that after convergence is obtained, Eq. (25)provides the feedback gains for optimal linear control about the optimalpath.

The second-variation method was applied to the optimal-pressure-profile problem studied several times earlier in this chapter. There aretwo state variables, hence five auxiliary functions to be computed,M11, M12, M22, g1, 92- We shall not write down the cumbersome setof specific equations but simply present the results for an initial choiceof decision function u(t) = const = 0.2, one of the starting values usedfor steep descent, with r = 1. Solutions to the Riccati equation areshown in Fig. 9.14, where differences beyond the second iteration weretoo small to be seen on that scale. From the first iteration on, the

3m

3 4 5 6Residence time t


Fig. 9.14 Successive, values of feedback gains using thesecond-variation method.

Table 9.10 Successive approximations to optimal pressure profileand outlet concentrations using the second-variation method

Iteration number

t 0 1 2 4 5

0 0.2000 1.1253 0.5713 0.7248 0.7068 0.7174

0.2 0.2000 0.8880 0.4221 0.3944 0.3902 0.3902

0.4 0.2000 0.4914 0.3527 0.3208 0.3194 0.3196

0.8 0.2000 0.3846 0.3014 0.2798 0.2786 0.2786

0.8 0.2000 0.3189 0.2888 0.2529 0.2522 0.2521

1.0 0.2000 0.2749 0.2421 0.2337 0.2332 0.2332

2.0 0.2000 0.1743 0.1819 0.1829 0.1829 0.1829

3.0 0.2000 0.1358 0.1566 0.1590 0.1590 0.1590

4.0 0.2000 0.1169 0.1421 0.1442 0.1442 0.1442

5.0 0.2000 0.1082 0.1324 0.1339 0.1339 0.1339

6.0 0.2000 0.1056 0.1251 0.1260 0.1260 0.1280

7.0 0.2000 0.1086 0.1191 0.1198 0.1198 0.1198

8.0 0.2000 0.1092 0.1140 0.1148 0.1148 0.1147

x (8) 3-3338 X 10-1 1 3.6683 X 10-1 3.&309 X 10-1 3.8825 X 10-1 3.8624 X 10-+ 3.8622 X 10-11

z1(8) 1.0823 X 10-1` 1.1115 X 10-1 1.1315 X 10-1 1.1319 X 10-1 1.1319 X 10-1 1.1319 X 10-1


functions gl and 92 were at least one order of magnitude smaller than-y, and 72, to which they act as correction terms, and could have beenneglected without affecting the convergence. Successive pressure pro-files are given in Table 9.10, where, following some initial overshootresulting from using r = 1, the convergence is extremely rapid. Withthe exception of the value right at t = 0 the agreement is exact withsolutions obtained earlier using the necessary conditions.

9.17 GENERAL REMARKS

Though we have not enumerated every technique available for thenumerical solution of variational problems, this sketch of computationalmethods has touched upon the most important and frequently usedclasses of techniques. By far the most reliable is the method of steepdescent for improving functional values of u. From extremely poorinitial estimates convergence to near minimal values of the objective areobtained by a stable sequence of calculations. As a first-order method,however, it generally demonstrates poor ultimate convergence to theexact solution of the necessary conditions.

All other methods require a "good" initial estimate of either bound-ary values or one or more functions. Such estimates are the end'productof a steep-descent solution, and a rational computational procedure wouldnormally include this easily coded phase I regardless of the ultimatescheme to be used. The final calculational procedure, if any, mustdepend upon the particular problem and local circumstances. Min H,for example, is more easily coded than second variation, but it lacks thecomplete second-order convergence properties of the latter. Further-more, if second derivatives are easily obtained-and if a general code forintegrating the Riccati equation is available, the coding differences arenot significant. On the other hand, if u(t) is constrained, second vari-ation simply cannot be used. Similarly, steep-descent boundary iter-ation is attractive if, but only if, a highly reliable automatic routine offunction minimization without calculation of derivatives is available,while all the indirect methods require relative stability of the equations.


Section 9.1: We shall include pertinent references for individual techniques as they arediscussed. We list here, however, several studies which parallel all or large partsof this chapter in that they develop and compare several computational procedures:

R. E. Kopp and H. G. Moyer: in C. T. Leondes (ed.), "Advances in Control Sys-tems," vol. 4, Academic Press, Inc., New York, 1966

L. Lapidus: Chem. Eng. Progr., 63(12):64 (1967)L. Lapidus and R. Luus: "Optimal Control of Engineering Processes," Blaisdell

Publishing Company, Waltham, Mass., 1967


A. R. M, Noton: "Introduction to Variational Methods in Control Engineering,"Pergamon Press, New York, 1965

B. D. Tapley and J. M. Lewallen: J. Optimization Theory Appl., 1:1 (1967)

Useful comparisons are also made in papers by Storey and Rosenbrock and Kopp,McGill, Moyer, and Pinkham in

A. V. Balakrishnan and L. W. Neustadt: "Computing Methods in OptimizationProblems," Academic Press, Inc., New York, 1964

Most procedures have their foundation in perturbation analysis for the computation ofballistic trajectories by Bliss in 1919 and 1920, summarized in

G. A. Bliss: "Mathematics for Exterior Ballistics," John Wiley & Sons, Inc., NewYork, 1944

Sections 9,2 and 9.3: The development and example follow

M. M. Denn and R. Aria: Ind. Eng. Chem. Fundamentals, 4:7 (1965)

The scheme is based on a procedure for the solution of boundary-value problems withoutdecision variables by

T. R. Goodman and G. N. Lance: Math. Tables Other Aids Comp., 10:82 (1956)

See also

J. V. Breakwell, J. L. Speyer, and A. E. Bryson: SIAM J. Contr., 1:193 (1963)R. R. Greenly, AIAA J., 1:1463 (1963)A. H. Jazwinski: AIAA J., 1:2674 (1963) -

AIAA J., 2:1371 (1964)S. A. Jurovics and J. E. McIntyre: ARS J., 32:1354 (1962)D. K. Scharmack: in C. T. Leondes (ed.), "Advances in Control Systems," vol. 4,

Academic Press, Inc., New York, 1966

Section 9.4: Little computational experience is available for th4, rather obvious approach.Some pertinent remarks are contained in the reviews of Noton and .Storey andRosenbrock cited above; see also

J. W. Sutherland and E. V. Bohn: Preprints 1966 Joint Autom. Contr. Conf., Seattle,p. 177

Neustadt has developed a different steep-descent boundary-interation procedure forsolving linear time-optimal and similar problems. See papers by Fndden andGilbert, Gilbert, and Paiewonsky and coworkers in the collection edited by Bala-krishnan and Neustadt and a review by Paiewonsky,

B. Paiewonsky: in G. Leitmann (ed.), "Topics in Optimization," Academic Press,Inc., New York, 1967

Sections 9.5 to 9.7: The Newton-Raphson (quasilinearization) procedure for the solutionof boundary-value problems is developed in detail in

R. E. Bellman and R. E. Kalaba: "Quasilinearization and Nonlinear Boundary-value Problems," American Elsevier Publishing Company, New York, 1965


Convergence proofs may be found in

R. E. Kalaba: J. Math. Mech., 8:519 (1959)C. T. Leondes and G. Paine: J. Optimization Theory Appl., 2:316 (1968)It. McGill and P. Kenneth: Proc. 14th Intern. Astron. Federation Congr., Paris, 1963

The reactor example is from

E. S. Lee: Chem. Eng. Sci., 21:183 (1966)

and the calculations shown were done by Lee. There is now an extensive literdture ofapplications, some recorded in the book by Bellman and Kalaba and in the reviewscited above. Still further references may be found in

P. Kenneth and R. McGill: in C. T. Leondes (ed.), "Advances in Control Systems,"vol. 3, Academic Press, Inc., New York, 1966

E. S. Lee: AIChE J., 14:467 (1968)--: Ind. Eng. Chem. Fundamentals, 7:152, 164 (1968)R. McGill: SIAM J. Contr., 3:291 (1965)--- and P. Kenneth: AIAA J., 2:1761 (1964)C. H. Schley and I. Lee: IEEE Trans. Autom. Contr., AC12:139 (1967)It. J. Sylvester and F. Meyer: J. SIAM, 13:586 (1965)

The papers by Kenneth and McGill and McGill use penalty functions to incorporatestate and decision constraints.

Section 9.8: Pertinent comments on computational effort are in

R. E. Kalman: Proc. IBM Sci. Comp. Symp. Contr. Theory Appl., While Plains, N.Y.,1966, p. 25.

Sections 9.9 to 9.13: The idea of using steep descent to solve variational problems origi-nated with Hadamard; see

R. Courant: Bull. Am. Math. Soc., 49:1 (1943)

The first practical application to a variational problem appears to be in

J. H. Laning, Jr., and R. H. Battin: "Random Processes in Automatic Control,"McGraw-Hill Book Company, New York, 1956

Subsequently, practical implementation was accomplished, independently about 1960 byBryson, Horn, and Kelley; see

A. E. Bryson, W. F. Denham, F. J. Carroll, and K. Mikami: J. Aerospace Sci.,29:420 (1962)

F. Horn and U. Troltenier: Chem. Ing. Tech., 32:382 (1960)H. J. Kelley: in G. Leitmann (ed.), "Optimization Techniques with Applications to

Aerospace Systems," Academic Press, Inc., New York, 1962

The most comprehensive treatment of various types of constraints is contained in

W. F. 1)enham: Steepest-ascent Solution of Optimal Programming Problems, RaytheonCo. Rept. BR-2393, Bedford, Mass., 1963; also Ph.D. thesis, Harvard University,.Cambridge, Mass., 1963


Some of this work has appeared as

W. F. Denham and A. E. Bryson: AIAA J., 2:25 (1964)

An interesting discussion of convergence is contained in

D. E. Johansen: in C. T. Leondes (ed.), "Advances in Control Systems," vol. 4,Academic Press, Inc., New York, 1966

Johansen's comment& on convergence of singular controls are not totally in agreement withour own experiences. The formalism for staged systems was developed by Bryson,Horn, Lee, and by Denn and Aris for all classes of constraints; see

A. E. Bryson: in A. G. Oettinger (ed.), "Proceedings of the Harvard Symposium onDigital Computers and Their Applications," Harvard University Press, Cam-bridge, Mass., 1962

M. M. Denn and R. Aris: Ind. Eng. Chem. Fundamentals, 4:213 (1965)E. S. Lee: Ind. Eng. Chem. Fundamentals, 3:373 (1964)F. Horn and U. Troltenier: Chem. Ing. Tech., 35:11 (1963)

The example in Sec. 9.11 is from

J. M. Douglas and M. M. Denn: Ind. Eng. Chem., 57(11):18 (1965)

while that in Secs. 9.12 and 9.13 is from the paper by Denn and Aris cited above. Otherexamples of steep descent and further references are contained in these papers andthe reviews. An interesting use of penalty functions is described in

L. S. Lasdon, A. D. Waren, and R. K. Rice: Preprints 1967 Joint Autom. Cont. Conf.,Philadelphia, p. 538

Section 9.14: The min-H approach was suggested by Kelley, in the article cited above,and by

S. Katz: Ind. Eng. Chem. Fundamentals, 1:226 (1962)

For implementation, including the extension to systems with final constraints, see

R. G. Gottlieb: AIAA J., 5:(1967)R. T. Stancil: AIAA J., 2:I365 (1964)

The examination of convergence was-in

M. M. Denn: Ind. Eng. Chem. Fundamentals, 4:231 (1965)

The example is from

R. D. Megee, III: Computational Techniques in the Theory of Optimal Processes,B. S. thesis, University of Delaware, Newark, Deli, 1965

Sections 9.15 and 9.16: The basic development of a second-vao'iatioh procedure is in

H. J. Kelley, R. E. Kopp, and H. G. Moyer: Progr. Astron. Aero., 14:559 (1964)C. W. Merriam: "Optimization Theory and the Design of Feedback Control Sys-

tems," McGraw-Hill Book Company, New York, 1964

The second variation is studied numerically in several of the reviews cited above; see also

D. Isaacs, C. T. Leondes, and R. A. Niemann: Preprints 1966 Joint Autom. Contr.Conf., Seattle, p. 158


S. R. McReynolds and A. E. Bryson: Preprints 1965 Joint Autom. Contr. Conf.,Troy, N.Y., p. 551

S. K. Mitter: Automatica, 3:135 (1966)A. R. M. Noton, P. Dyer, and C. A. Markland: Preprints 1966 Joint Autom. Contr.

Conj., Seattle, p. 193

The equations for discrete systems have been obtained in

F. A. Fine and S. G. Bankoff: Ind. Eng. Chem. Fundamentals, 6:293 (1967)D. Mayne: Intern. J. Contr., 3:85 (1966)

Efficient implementation of the second-variation technique requires an algorithm for solv-ing the Riccati equation, such as the computer code in

R. E. Kalman and T. S. Englar: "A User's Manual for the Automatic SynthesisProgram," NASA Contractor Rept. 475, June, 1966, available from the Clearing-house for Federal Scientific and Technical Information, Springfield, Va. 22151.

PROBLEMS

9.1. Solve the following problem by each of the, methods of this chapter. If a computeris not available, carry the formulation to the point where a skilled programmer withno knowledge of optimization theory could code the program. Include a detailedlogical flow sheet.

2i1 = - arctan u - x1x

is = xI - x:

is = x2 - xax1(0) = 0 x2(0) _ -0.4 xa(0) = 1.5

min & = 103 1(2x:)sa + xa= + 0.01u'1 dt

Take a = 2 and 10. The latter case represents a penalty function approximationfor Ix,! < 5J. The arctangent is an approximation to a term linear in u with theconstraint Jul < 1. (This problem is due to Noton, and numerical results for somecases may be found in his book.) Repeat for the same system but with

i1 -U-x1Ju l < 1

9.2. Solve Prob. 5.8 numerically for parameter values

k1=ka=1 k2 -109-0.4 and 9-1.0

Compare with the analytical solution.9.3. Solve Prob. 7.2 using appropriate methods of this chapter. (Boundary iterationis difficult for this problem. Why?)9.6. Develop an extension of each of the algorithms of t%is chapter to the case inwhich a is specified only implicitly by a final constraint of the form

#(x(e)l = 0

Note that it might sometimes be helpful to employ a duality of the type described inSec. 3.5.

10

Nonserial Processes

10.1 INTRODUCTION

A large number of industrial processes have a structure in which theflow of material and energy does not occur in a single direction becauseof the presence of bypass and recycle streams. Hence, decisions madeat one point in a process can affect the behavior at a previous point.Though we have included the spatial dependence of decisions in processessuch as the plug-flow reactor, our analyses thus far have been couchedin the language and concepts of systems which evolve in time. Forsuch systems the principle of causality prevents future actions frominfluencing the present, and in order to deal with the effect of feedbackinteractions in spatially complex systems we shall have to modify ourprevious analyses slightly.

The optimization of systems with complex structure involves onlya single minor generalization over the treatment of simple systems inthat the Green's functions satisfy a different set of boundary conditions.There has been some confusion in the engineering literature over this326

NONSERIAL PROCESSES 327

problem, however, and we shall proceed slowly and from several differentpoints of view. The results of the analysis will include not only theoptimization of systems with spatial interactions but also, because ofmathematical similarities, the optimal operation of certain unsteady-stateprocesses.

10.2 RECYCLE PROCESSES

We can begin our analysis of complex processes by a consideration of therecycle system shown in Fig. 10.1. A continuous process, such as atubular chemical reactor, is described by the equations

z; = f;(x,u) 0 < t < 0 (1)

where the independent variable t represents length or residence time.The initial state x(0) is made up of a mixture of a specified feed x1 andthe effluent x(O) in the form

x;(0) = G,[xf,x(8)] (2)

The goal is to choose u(t), 0 < t < 0, in order to minimize a functionof the effluent, 6[x(8)).

We carry out an analysis here identical to that used in previouschapters. A function u(t) is specified, and Eqs. (1) and (2) are solvedfor the corresponding 8(t). We now change u(t) by a small amountSu(t) and obtain first-order variational equations for &x

safi

`Ox, = 0x, + usu (3)1 ax,a

1-I

The corresponding first-order change in the mixing boundary condition,Eq. (2), is

saG,0x;(0) = I ax;(e) 0x1(8) (4)

f-I

Green's identity can be written for the linear equations (3) as

Y6) Ox,(B) y:(0) 0x,(0) + fI 'Y; Su dt_ a ioial i-I :.1

L> x(0)-G[x,,x(8)

t

x(0)x=f(x,u)

(5).

x(8)

Fig. 10.1 Schematic of a continuous recycle process.

32i OPTIMIZATION BY VARIATIONAL METHODS

where the Green's functions satisfy the equations

y` ax; 0<t<0ill (6)

Following substitution of Eq. (4), Green's identity, Eq. (5), becomes

NO) ax;(8) = 1 NO [I aG,0 (0)

ax;(B) ] + f p y: a ' au dt

or

(7)

axi (e)] ax; (9) =aHtoI y:(B) - l 70) aG ;

fo au su do (8)

In Eq. (8) we have substituted the usual hamiltonian notations

H= Iyif:i-l

(9)

The first-order change in the objective, g[x(e)j, as a result of thechange su in decision, may be written

sS& _

a x;axi(B) (10)

This can be related explicitly to su by means of Eq. (8), for if we writeS

(o) 1 (11)y' 9)ax az

Eq. (8) becomes

afi - fo au au dt (12)

If a is the optimal decision, the weak minimum principle follows immedi-ately from Eq. (12) by a proof identical to those used previously. Theonly change resulting from the recycle is that in the pair of boundaryconditions, Eqs. (2) and (11).

An identical analysis can be carried out for a sequence of stagedoperations with recycle of the type shown in Fig. 10.2. The state isdescribed by difference equations

x:^ = f;"(x^-',u^) n = 1, 2, . . . , N (13)

where the input to the first stage is related to the effluent x' and feed

NONSERIAL PROCESSES

if xa= G(xr, x")x0

x,=If t(xa,u')

I

x2=f2(x1,u2)

2

N=fN(xN-t,uN)N

32!

N

J

Fig. 10.2 Schematic of a staged recycle process.

x, by the equation

xie = G,(xi,xN) (14)

and the objective is the minimization of S(XN). The stage hamiltonianis defined by

s

Hn = yinfin

with Green's functions satisfyings r

nafire

y; fz = 1, 2, , Ni-1

ysN , Q,, _ aS^fj ax;N ax:N

i-t

(15)

(16)

(17)

As previously, only a weak minimum principle is generally satisfied.For continuous recycle systems a strong principle is proved in the usualway.

10.3 CHEMICAL REACTION WITH RECYCLE

We can apply the minimum principle to an elementary recycle problemby considering the process shown in Fig. 10.3. The irreversible reaction

X -* Y

Feed X Mixer

Choose temperotur profile

Reactor X -Y

Unreocted X

Separator I Product Y

Fig. 10.3 Schematic of a reactor with recycle of unreacted feed.


is carried out in a tubular reactor, after which unreacted X is separatedand recycled. We wish to find the temperature profile in the reactorwhich will maximize conversion less the cost of operation.

The concentration of X is denoted by x, and the temperature by u.With residence time as independent variable the governing equationfor the reactor is then

x, _ -k(u)F(x,) 0 < t < 0

The mixing condition is

x1(0) _ (1 - p)xj + px,(B)

(1)

(2)

where p is the fraction of the total volumetric flow which is recycled.The cost of operation is taken as the heating, which may be approxi-mated as proportional to the integral of a function of temperature.Thus, we seek to maximize

(P = x1(0) - x,(0) - fo g[u(t)] dt (3)

or, equivalently,

= fo [k(u)F(x1) - g(u)] dt (4)

This is put into the required form by defining a variable x2,

22 = -k(u)F(x1) + j(u) X2(0) = 0 (5)

Then we wish to minimize

S = x2(8) (6)

The hamiltonian is

H = -y,k(u)F(x,) + y2[-k(u)F(x1) + 9(u)] (7)


aH = (y1 + 'v2)k(u)F'(x1) (8a)

aHy2° -ax2=0 (8b)

The prime denotes differentiation with respect to the argument. Theboundary conditions, from Eq. (17) of the preceding section, are

y,(8) = Py,(0) (9a)

72(0) = 1 (9b)

Equations (Sb) and (9b) imply that 72 = 1. If the optimum is taken to


be interior, then

aH = 0 = - (y, + 1)F(xi)k'(u) + g'(u)au

or

(10)

g'(u) = (y, + 1)F(xi) (11)k' (u)

It is convenient to differentiate both sides of Eq. (11) with respectto t to obtain

dg(u)

Wt k,( u )= 1,F(x,) + (71 + 1)F'(x,)x,

_ (y, + 1)k(u)F'(x1)F(x,) + (y, + 1)F'(x,)[-k(u)F(x,)]= 0 (12)

Thus, g'(u)/k'(u), a function only of u, is a constant, so that if the equa-tion g'(u)/k'(u) = const has a unique solution, the optimal temperatureprofile must be a constant. The problem then reduces to a one-dimen-sional search for the single constant u which maximizes (Y.

For definiteness we take F(xi) = xl. Then Eqs. (1) and (2) canreadily be solved for constant u and substituted into Eq. (3) to obtain

xf(1 p)(1 - e-k( )

The maximization of 61 with respect to u can be carried out for givenvalues of the parameters and functions by the methods of Chap. 2.

10.4 AN EQUIVALENT FORMULATION

The essential similarity between the results for recycle systems obtainedin See. 10.2 and those for simple straight-chain systems obtained pre-viously in Chaps. 4 to 7 suggests that the former might be directly deriv-able from the earlier results by finding an equivalent formulation of theproblem in the spirit of Sec. G.G. This can be done, though it is awkwardfor more complex situations. We shall carry out the manipulations forthe continuous recycle system for demonstration purposes, but it isobvious that equivalent operations could be applied to staged processes."

The system satisfies the equations and recycle boundary condition

z,=f;(x,u) 0<t<0 (1)x;(0) = G;[xf,x(0)] (2)

and the objective is the minimization of &[x(0)]. We shall define 8


new variables, yl, yz, . , y as fol lows:

y; = 0 (3)

Y'(0) - x;(0) = 0 (4)y;(9) - G;[x,,x(e)) = 0 (5)

Equations (1) and (3) to (5) define a system of 2S equations with Sinitial constraints and S final constraints but with the recycle conditionformally removed. The transversality conditions, Eqs. (12) and (13)of Sec. 6.4, can then be applied to the multipliers.

For convenience we denote the Green's functions corresponding tothe x; as y; and to the y; as 1';. The y, satisfy the equations

s

ti; I yjA-i 0<t<6 (6)i-1 dx,

The transversality conditions resulting from the constraint equations (4)and (5) are, respectively,

y;(0) (7)

y;(B) = ax; - 4 vi (8)

where the 'n; and v; are undetermined multipliers. The functions r;satisfy the following equations and transversality conditions:

t'; = 0 r, = const (9)

r;(e) = v; (11)

Combining Eqs. (7) to (11), we are led to the mixed boundary conditionfor the y; obtained previously

Sd(i d6W ax,

i-1

(12)

The hamiltonian and strong minimum principle,. of course, carry overdirectly to this equivalent formulation, but fqr the staged problem weobtain only the weak minimum principle.


The Lagrange multiplier rule, first applied in Chap. 1 to the study ofstaged systems, is a particularly convenient tool for a limited class ofproblems. One of the tragedies of the minimum-principle-orientedanalyses of recent years has been the failure to retain historical per-


spective through the relation to Lagrange -multipliers as, for example,in Sec. 7.5. We shall repeat that analysis here for the staged recycleproblem as a final demonstration of an alternative method of obtainingthe proper multiplier boundary conditions.

The stage difference equations and recycle mixing condition may bewritten as.

-x,n + fin(xn_l,un) = 0 n = 1, 2, . . . , N (1)-xi° + Gi(x,,xN) = 0 (2)

For the minimization of F(x") the lagrangian is then writtenN S

£ = & + 4I /-.l X,' (-xtin + ./in(an-l,u°)1n-1 i-1

S

+ Xi°(-xi° + Gi(x,,xN)) (3)i-1

Setting partial derivatives with respect to each of the variables u', u2,. . UN, zi°, xi', . . . , x;N to zero, we obtain

a. _ S :n afin =VU_

oaun

i-1

a.C 1S

ax'°X,n T, Xn+1

ax n= O

i-1

aae as

-X.N +

ill° aG; _ o

aX7°5-ZW I ax:N

(4)

n=0,1,2,...,N-1(5)

(6)

Equations (4) and (5) are the usual weak-minimum-principle equationsfor an unconstrained decision, while Eq. (6) is the multiplier boundarycondition obtained previously for the recycle problem.

10.6 THE GENERAL ANALYSIS

We can now generalize the recycle analysis to include plants with anarbitrary structure of interactions. The essence of the analysis is theobservation that any complex structure can be broken down into a set ofserial, or straight-chain, structures of the type shown in Fig. 10.4, with

Fig. 10.4 Serial subsystems.

334 OPTIMIZATION BY VARIATIONAL*METHODS

mixing conditions expressing the interactions between serial subsystems.For notational purposes we shall denote the particular subsystem

of interest by a parenthetical superscript. Thus x(k) refers to the stateof the kth serial subsystem. If it is continuous, the dependence on thecontinuous independent variable is expressed as x(k)(t), where t rangesfrom zero (inlet) to 9(k) (outlet). In a staged serial subsystem we wouldindicate stage number in the usual way, z(4)", where n ranges from zeroto N(k). It is convenient to express the inlet and outlet of continuoussubsystems in the same way as the discrete, so that instead of z(k)(0)we shall write z()0, and X(k)111 for x(4)(9(4)).

The state of the kth serial subsystem is described by the differenceor differential equations

n = 1, 2, . . . , NW(k)"(k)" (k)w-1(k)" )xi (x=s 'U i = 1, 2, . . . , S(k)

xs(k) = f1(k)(x(k),u(k)) 0 < t < 9(k)i = 1, 2, , S(k)

(la)

(lb)

where S(k) is the number of state variables required in the kth subsystemand a single decision is assumed for convenience. The subsystems arerelated to one another (by mixing equations of the form

x(4)0 = G(k)({z,1,fz(1)N1)

n = 1, 2,

i = 1, 2,

0 < t < 9(k)

Equation (2) is a statement that the input to subsystem k depends insome known manner on the set of all external feeds to the system {z,}and the set of outflow streams from all serial subsystems {x(1)N). Thesimple recycle mixing condition, Eqs. (2) and (14) of Sec. 10.2, is a specialcase of Eq. (2). The objective is presumed to depend on all outflowstreams

8 = &{x(1)N{

and for simplicity the outflows are presumed unconstrained.

(3)

Taking the usual variational approach, we suppose that a sequenceof decisions {u(k)j has been made for all parts of all subprocesses and thatthe system equations (1) have been solved subject to the mixing boundaryconditions, Eq. (2). A small change in decisions then leads to the linearvariational equations

s«,a (k)s (k)"

ax(k)n = 1s sx(k)"-1 +af+ au(k)"

4 - axf(4)w-1 ' a.u(k)"j-1

az,(k) =2M!

axj(k) + aVk) au(k)au

j-1

(2)

N(k)S(k)

i = 1, 2, . . . , S(k)

(4a)

(4b)

NONSERIAL PROCESSES

au(k)n staged subsystem

The corresponding first-order changes in the mixing conditions andobjective are

Sxi(k) 0

So)aG(k)

ax (t' Naxj(1) N

t i-1S())

a8 =axj(1'N

axj(i)N

The summations over 1 indicate summation over all subsystems.

33S

(5)

(6)

The linear variational equations have associated Green's functionswhich satisfy difference or differential equations as follows:

,yi(k)n-1S(k)

i-1S(k)

yi(k)n Y

LL..II(3xi(k)n-1

- 1 77(k)i-1

0f.(k)ax/i(k)

n = 1, 2, . . , N(k)i = 1, 2, . . . , 8(k)

0<t<0(k)i = 1, 2, . .

(7a)

(7b)S(k)

Green's identity for each subsystem is thenS(k) S(k) Ar(k) 3(k) (k)n

.Yi(k)N axi(k)N = \ ,Yi(k)0 &xi(k)0 + I I a '(k)n au(k)n (8a)

S(k) So) S(k)

i(k)N axi(k)N = Yi(k)0 axi(k)0 + rO tt) ( ''i(k) aau(k)(t)dt

.Jf

(8b)

The notation is made more compact by introducing the hamiltonian,So)

H(k)n o ,ii(k)nfi(k)n

i-1

S

H(k) = ,Yi(k) f.(k)

staged subsystem

continuous subsystem

and the summation operator $(k), defined by

oH(k)8(k) au(k) _

au(k) Bo) 8H(k)

(9a)

(9b)

au(k)(t) dt continuous subsystem

(10)

Green's identity can then be written conveniently for staged or continuous


subsystems as8(k)

)i(k).V bxi(k)N

i-1

S(k)

I .yi(k)O Sxi(k)o + s(k)au(k)

)

bu(k) (11L.ri-1

We now substitute Eq. (5) for bx,(k)0 into Eq. (11), which is writtenas

S(k) s(k) S,)) aGi(k>

p aH(k)4 7i(k)N bxi(k)N -

,,i(k)O I I axj(!)Naxi(I)N = cl(k) a(k)u 50)

i-1 i-I I j..1

(12)

Equation (12) is then summed over all subsystems3(k) S(,)S(k)

I y,(k)N axi(k)N - .) (k)o I I ax;(!)N"j"),

k i-1 . k i-1 I j-1++

aH(k)_ J(k) 5U(k) au(k) (13)

k

In the first term the dummy indices for summation, i and k, may bereplaced by j and 1, respectively, and order of finite summation inter-changed in the second such that Eq. (13) is rewritten

S(n Su) S(k)

(i)N axi(!)N ( yf(k)U a(y,11 (1)N

yj ax(I)N/ axj1 j-1 1 j-1 k i-1

OH(k)c7 (k) au(k) a2E(k) (14)

A;

or, finally,Su) S(k )

(ywv yi(k)o aGi(k)1 bx.(°N =all(k)

au(k) (15)- ax (I)N J/ 1 eS<k) au(k)1 j-1 k i-1 k

Comparison with Eq. (6) for & dictates the boundary conditions forthe Green's functions

So)(R,

Yi(1)N - .ri(k)0 6Cii _ dE

azi(pN ax.(I)N

k i-1(16)

This is a generalization of Eqs. (11) and (17) of Sec. 10.2. The first-order variation in the objective can then be written explicitly in termsof the variations in decisions by combining Eqs. (6), (15), and (16)

aH(k)bulk) au(k)59 _ (k)

k

(17)


We may now adopt either of the two points of view which we havedeveloped for variational problems. If we are examining variationsabout a presumed optimum, SS must be nonnegative and analyses identi-cal to those in Sees. 6.5 and 7.4 lead to the weak minimum principle:

The Hamiltonian H(k)n or H(k) in each subprocess is made stationaryby interior optimal decisions and a minimum (or stationary) byoptimal decisions at a boundary or nondifferentiable point.

For continuous subsystems H(k) is a constant, and by a more refinedanalysis of the type in either Sec. 6.8 or 6.9, a strong minimum principlefor continuous subsystems can be established. The necessary conditionsfor an optimum. then, require the simultaneous solution of Eqs. (1) and(7) with mixing conditions defined by Eqs. (2) and (16) and the optimumdecision determined by means of the weak minimum principle. .

Should we choose to develop a direct computational method basedupon steep descent, as in Sec. 9.9, we wish to choose Su(k)(t) and Eu(k)nto make & nonpositive in order to drive S to a minimum. It followsfrom Eq. (171, then, that the changes in decision should be of the form

aH(k)ngu(k)n = _w(k)n (18a)au(k)n

au(k)(t) = _w(k)(t) ask> (18b)

where w(k)11 and w(k)(t) are nonnegative. A geometric analysis leads tothe normalization analogous to Eqs. (12) and (23) of Sec. 9.9.

au (k) n OG(k)n 3H(k)n/49u(k)n

H(k) 2

5.'(k)(t)zG(k)(t) aH(k)/au(k)

S(k)G(k)all(k)

au(k))7'k

(19a)

(19b)

where A is a step size and G(k) n, G(k) (t) are nonnegative weighting functions.

10.7 REACTION, EXTRACTION, AND RECYCLE

The results of the preceding section can be applied to the optimizationof the process shown in Fig. 10.5. The reaction

X1 --" Xz -- products


Solvent

E7

Solventremoval

I II

Solvent

Feedpure x, Mixing

Prod ctExtraction of x,

(choose amount of solvent)

I 2 3

Reaction x,-x2--x3(choose temperatures)

FIg.10.5 Schematic of a reactor system with countercurrent extrac-tion and recycle of unreacted feed. (From M. M. Denn and R. Aris,Ind. Eng. Chem. Fundamentals, 4:248 (1965). Copyright 1965 bythe American Chemical Society. Reprinted by permission of thecopyright owner.]

is carried out in a sequence of three stirred-tank reactors. The productstream is passed through a two-staged countercurrent extraction unit,where the immiscible solvent extracts pure X1i which is recycled to thefeed to the first reactor. The reactor system will be denoted by arabicstage numbers and the extractor by roman numerals in order to avoidthe use of parenthetical superscripts.

The kinetics of the reactor sequence are taken as second ' andfirst order, in which case the reactor material-balance equations arethose used in Sec. 9.12

0 = xin-' - x1n - 6kioe-8"'1-'(x1")2 n = 1, 2, 3 (1a)0 = x2n-' - x1; + 6?k, oe-E,'/"1(x1") 2

- 9k2oe 8i "1x2" n = 1, 2, 3 (1b)

The material-balance equations about the first and second stage of theextractor are, respectively,

x11 -I- u11[4'(x11) - 0111)1 - x18 = 0 (2)X111 + 4111(x111) - x11 = 0 (3)

Here u11 is the ratio of volumetric flow rates of solvent to product streamand #(x1) is the equilibrium distribution between the concentration ofx1 in solvent and reactant stream. Because the solvent extracts X1only, there is no material-balance equation needed for x2 about theextractor, for x23 = x211. The external feed is pure X1i so that the

NONSERIAL PROCESSES

feed to reactor 1 is

x1° = xu + u"I '(xi') (4a)x2° = 0 (4b)

We wish to maximize the production of X2 while allowing for costs ofraw material and extraction, and hence we seek to minimize

8 = -x23 - CxIII + OuII (p)

The temperatures u', u2, u3 and solvent ratio u11 are to be chosen subjectto constraints

U* < u', U2, u3 < u*0<u"I

(6a)(6b)

Equations (1) to (5) are not in the form required for applicationof the theory, for, though we need not have done so, we have restrictedthe analysis to situations in which a decision appears only in one stagetransformation and not in mixing conditions or objective. This iseasily rectified by defining a variablet x3 with

x3' = x3' (7)x31I = x3I + uII (8)

Equations (2) and (3) are then rewritten, after some manipulation,

4,(xII) - xI` = 0 (9)xIII + uII4,(xi"1) - X11 = 0 (10)

where the mixing boundary conditions are now rearranged as

xl _ xIf + x,1 - -xIII (lla)x2° = 0 (llb)

i IIxIs

x1 x3II I (1lc)

23' = 0 (lld)

The system is then defined by Eqs. (1) and (7) to (11), and the objectiveis rewritten

8 = --:X1 a - X1 11 + Ox3II (12)

The structure defining the interactions represented by Eqs. (11) is shownin Fig. 10.6.

The equations for the Green's functions are defined by Eq. (7) of

f The superscript z (for zero) will denote the input to the first stage for the roman-numeraled variables.


1 Hi 2 3 r-1 I II

Fig. 10.6 Structure of the interactions in the reaction-extraction-recycle system. [From M. M. Denn and R.Aris, Ind. Eng. Chem. Fundamentals, 4:248 (1965).Copyright 1965 by the American Chemical Society. Re-printed by permission of the copyright owner.)

Sec. 10.6 and, using implicit differentiation as in Sec. 7.6, are

n-1y y1n1 1 + 20k 10e-E,"lu"x In

2e,Y2nk1pe E,"/u"xin

+ (1 + 28k1oe-E''lu'x1")(1 + 8k20e_E='!

72"-1 = ry2"1 + k620e-E="'U'

yI = ,(xll)171I1

i = 1 + ull#'(xi')yin-1=7'3" n=I,II

n = 1, 2, 3

n = 1,2,3 (13a)

(13b)

(14a)

(14b)

(14c)

The prime denotes differentiation with respect to the argument. Theboundary conditions, obtained from Eqs. (11) and (12) by means of thedefining equation (16) of the preceding section, are

.yiII + x31L + 71 = -c (15a)

YI3- iii -y10=0 (15b)

y23 = -1 (15c)

7Y31I + .I3t1 , (X19 ' or (15d)

Finally, the partial derivatives of the hamiltonians with respect to thedecisions are

Lf72n - Y1"(1 + 9k29e_R="/u")jE'kloe-E,"lu"(xiT

aHn _ 0 - y2nE2k2oe-E="'"X2"(1 + 20kjoe-E'''""x1")

au". (u")2 (1 + 9k2oe-E! I"")(1 + 20k10e-E"''""xl")n = 1, 2, 3 (16a)

OHII 11 0111 11 0311+ 13 auliaull - 71 aull

ll_(x i1) II7i 1

1 + u' I.l" (x i11) + '1' 3

?1

(16b)

NONSERIAL PROCESSES -341

or, substituting Eq. (15d) into (16b),

allil^ uII 4011) 1 +IU U1 y (XIII) (16c)

It is evident that the artificial variables x3', x3II and -y3I, 73 11 are neverneeded in actual computation.

The simultaneous solution of the material-balance relations, Eqs.(1) to (4), and Green's function, Eqs. (13) to (15), with the optimal tem-peratures and solvent ratio determined from the weak minimum principleby means of Eqs. (5) and (16), is a difficult task. Application of theindirect methods developed in Chap. 9 would require iterative solutionof both the state and multiplier equations for any given set of boundaryconditions because of the mixing conditions for both sets of variables, andeven with stable computational methods.this would be a time-consumingoperation. Steep descent, on the other hand, is quite attractive, foralthough the material-balance equations must be solved iteratively foreach assumed set of decisions, the multiplier equations are then linearwith fixed coefficients and can be solved by superposition.

To illustrate this last point let us suppose that the decisions andresulting state variables have been obtained. Addition of Eqs. (15a) and(15b) leads to

VIII + 'v'3 = -c (17)

and so it is evident that a knowledge of yl' is sufficient to solve Eqs.(13a), (14), (15a), (15b), and (15d). We assume a value 9I' and computethe set (91^), n = 1, 2, 3, z, I, If. The correct values are denoted by191" + 3yi" J . From Eq. (17)

Sy1II - by1'

in which case it follows from Eqs. (14a) and (14b) that

67", 4'(x,')[1 + u114"(x1I))

From Eq. (13a),3 1

ay1° = b YI 1 + 29xt'"k3oe-E,'l"

But it follows from Eq. (15b) that

8 a

by1' - uII - b7'I° -113 + uII + '11°

&y1'

(18)

(19)

(20)

(21)


and combining Eqs. (19) to (21), it follows that the correct value of yl' is

11 - "T IC + ION'yl = lla + 3

1+u"I"(x,')[1 + u"IL"(xi)] X111 1 + 28xI kloe-1W1u°

(22)

Thus, once the temperatures and solvent ratio have been specified andthe material-balance equations solved iteratively, the equations for theGreen's functions need be solved only twice with one application of Eq.(22).

The corrections to the values of u', us, u', u"I are calculated usingEqs. (16a) and (16c) from the relations

bun = -A3

Gn aHn/aun(23a)()2 u'Ibull

= -A[ 3 GII aH"/aull(23b)

IIaI (OH-)2G. +G au"'CauII)J

The physical parameters used in the calculations in Secs. 9.3 and 9.12were used here, with 0 = 2 and a total reactor residence time of 6. Thefunction 4,(x,) is shown. in Fig. 10.7 and has the analytical form

(xI) = 2.5xi-- 2(xi)1 0 < xI < 0.6 (24)1.08 - 1.lxi+ (xl)2 0.6 <xl <0.9The cost of extraction o was allowed to range over all values, and GI,Gs, G3 were set equal to unity. Following some preliminary experimen-

0.9

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0 0.2 0.4 0.6 0.8X1

Fig. 10.7 Equilibrium distribution func-tion of feed between solvent and reactantstreams. [From M. M. Denn and R.Aris, Ind. Eng. Chem. Fundamentals,4:248 (1965). Copyright 1965 by theAmerican Chemical Society. Reprintedby permission of the copyright owner.]


tation, G" was taken as 0.005. As in Sec. 9.12 the initial step size awas set equal to 2, with the step size halved for each move not resultingin a decrease in the value of S. The criterion for convergence was takento be a step size less than 10-1. In all calculations the material-balanceequations were solved by a one-dimensional direct search. Figure 10.8shows a typical convergence sequence, in this case for o = 0.30.

The profit -S is plotted in Fig. 10.9 as a function of extraction cost.The horizontal line corresponds to the optimal nonrecycle solution foundin Sec. 9.12, and in the neighborhood of the intersection two locally opti-mal solutions were found. Figure 10.10, for example, shows another setof calculations for a = 0.30 starting at the same initial temperature policy

0.4 344

0.3 343

0.2 342

0.1 341

0 34011

0I 1 1

5 10 15Iteration number

Fig. 10.8 Successive approximations to the optimaltemperature sequence and solvent ratio using steepdescent. [From M. M. Denn and R. Aris, Ind.Eng. Chem. Fundamentals, 4:248 (1965). Copy-right 1965 by the American Chemical Society. Re-printed by permission of the copyright owner.]


as in Fig. 10.8 but at a different solvent ratio, and the resulting tempera-tures are those for the optimal three-stage serial process. The optimaltemperature and solvent policies as functions of o are shown in Figs.10.11 and 10.12. For sufficiently inexpensive separation the tempera-tures go to u1., indicating low conversion and large recycle, while for suf-ficiently costly separation the optimal policy is nonrecycle. Multiplesolutions are shown by dashed lines. The discontinuous nature of theextraction process with separation cost has obvious economic implicationsif capital investment costs have not yet been taken into account in deriv-ing the cost factors for the process.

A mixed continuous staged process with the structure shown in Figs.10.5 and 10.6 is obtained by replacing the three stirred-tank reactorswith a plug-flow tubular reactor of residence time 0 = 6. The material-balance equations for the continuous serial process are then

z1 = -kloe-E,'I"(x,)2 0 < t < 6x2 = ki0e_E,11u(x1)2 - kgpe_E''1ux2 0 < t < 6

with corresponding Green's functions

'y, = 2k,oe-$"'ux1(y1 - 72) 0 < t < 6y2 = k2oe-E"'uy2 0 < t <.6

0.43

0.41

0.39

0.37

0.350a

0.33

0.31

0.29

0.270.15 0.20 0.25 0.30 0.35Cost of extraction o

(25a)(25b)

(26a)(26b)

Fig. 10.9 Profit as a function of costof extraction. [From M. M. Dennand R. Aris, Ind. Eng. Chem. Funda-mentals, 4:248 (1965). Copyright 1965by the American Chemical Society. Re-printed by permission of the copyrightowner. ]

NONSERIAL PROCESSES

0.75 345

0.50 344

0.25 343

0 342I l I I I 1 I I


345

Fig. 10.10 Successive approximations to the opti-mal temperature sequence and solvent ratio usingsteep descent. {From M. M. Denn and R. Aris,Ind. Eng. Chem. Fundamentals, 4:248 (1965).Copyright 1965 by the American Chemical Society.Reprinted by permission of the copyright owner.]

345

340

Fig. 10.11 Optimal temperature se- Equence as a function of cost of extrac- ,!tion. From M. M. Denn and R. Aris,Ind. Eng. Chem. Fundamentals, 4:248(1965). 'Copyright 1965 by the American 335Chemical Society. Reprinted by permis- 0.10 0.15 0.20 0.25 0.30sion of the copyright owner.) Cost of extraction c

346

1.1

1.0

0.9

0.8

0.7

0:60N

-0.50

0.4c00 .0.3

Ea 0.2

0.1

0

0.15 0.20 0.25 0.30Cost of extraction o

0.35


Fig. 10.12 Optimal solvent ratio as afunction of cost of extraction. [FromM. M. Denn and R. Aris, Ind. Eng.Chem. Fundamentals, 4:248 (1965).Copyright 1965 by the American Chemi-cal Society. Reprinted by permission ofthe copyright owner.]

The partial derivative of the continuous hamiltonian with respect to thedecision is

aH 1

au u2[x12k`oe t 11E,(y1 - -12) - xsksoeN-1uE2y2] (27)

If we denote the effluent at t = .6 by the superscript n = 3, Eqs. (2) to(4), (7) to (11), (14), (15), and (16c) remain valid. The Green's functionequations can still be solved with a single correction, but instead of Eq.(22) we must use

y1(6) = 91(6)+

1

91(0) - 91(6) + !ls/u1Ir(e 2x,kloe-B,'!- dl)1 + uI1G (xI,)1 - exp ( - Jo>

(28)

The corrections in temperature and solvent ratio are

su(c) -o G(t) aH/au

[j0G(t) (au)2 dt + G"

(aHIt1)2T,gull G11

OHlI/au11

= -f

[JOB G(t) au)2 dt + G" ( u11121'5

(29a)

(29b)

Following the results of Sec. 9.11, the continuous variables were.stored at intervals of 0.1 with linear interpolation, and an initial step sizeof S was taken. G(t) was set equal to unity and GII to 0.0005. A typi-


Residence time t

Fig. 10.13 Successive approximations to the optimal temperatureprofile in a continuous reactor using steep descent. (From M. M.Denn and R. Aris, Ind. Eng. Chem. Fundamentals, 4:248 (1965).Copyright 1965 by the American Chemical Society. Reprinted bypermission of the copyright owner.)

cal convergence sequence is shown in rigs. 10.13 and 10.14 for o = 0.25.The solid starting-temperature profile is the optimal nonrecycle solutionfound in Sec. 9.3, while the shape of the dashed starring curve was dic-tated by the fact that the unconstrained solution for the temperatureprofile can be shown from the necessary conditions to require an infinite

Fig. 10.14 Successive approximations tothe optimal solvent ratio following acontinuous reactor using steep descent.[From M. M. Denn and R. Aris, Ind.Eng. Chem. Fundamentals, 4:248 (1965).Copyright 1965 by the American Chemi-cal Society. Reprinted by permission ofthe copyright owner.]



355

Z 350

v

c 345CL

E

340

335

F1g.10.15 Optimal temperature profile as a function of costof extraction. [From M. M. Denn and R. Aris, Ind. Eng.Chem. Fundamentals, 4:248 (1965). Copyright 1965 by theAmerican Chemical Society. Reprinted by permission of thecopyright owner.)

slope at t = 0 (compare Sec. 4.12). Convergence was obtained in 9 and11 iterations from the solid and dashed curves, respectively, with somedifference in the ultimate profiles. The apparent discontinuity in slopeat t = 0.1 is a consequence of the use of a finite number of storage loca-tions and linear interpolation.

The profit is shown as a function of o in Fig. 10.9, the solvent allo-cation in Fig. 10.12, and the optimal temperature profiles in Fig. 10.15.As in the staged system there is a region in which multiple solutions werefound. For o = 0.325, for example, the same profit was obtained by thecurve shown in Fig. 10.15 and solvent in Fig. 10.12 and by the nonrecyclesolution, shown as ar = oo. It is evident that multiple solutions must beexpected and sought in complex systems in which the amount of recycleor bypass is itself a decision which has monetary value.

10.8 PERIODIC PROCESSES

The observation that the efficiency of some separation and reaction sys-tems can be enhanced by requiring the system to operate in the unsteadystate, as demonstrated, for example, in Sec. 6.11, has led to substantialinterest in the properties of periodic processes. These are processes inwhich a regular cyclic behavior is established and therefore, in terms oftime-aver6ged behavior, allows the overall operation to 'be consideredfrom a production point of view in steady-state terms. As we shall see,the fact that a decision made during one cycle has an effect on an


earlier part of the successive cycle is equivalent to feedback of infor-mation, and such processes are formally equivalent to recycle processes.

We consider a process which evolve6 in time according to the diff er-ential equations

-ii = f;(x,u)o<t<8i=1,2,...,S (1)

The process is to be operated periodically, so that we have boundaryconditions

xi(0) = xi(8) i = 1, 2, . . . , S (2)

This is clearly a special case of the recycle boundary conditions. Forsimplicity we shall assume that 8 is specified. The decision function u(t)is to be chosen to minimize some time-averaged performance criterion

S = fo 5(x,u) dt (3)

This can be put in the form of a function of the state at t = 8 by defininga new variable

x.+1= 9(x,u) 0<t<8x,+1(0) = 0

In that case

&[x(o)J = x,+1(e)

The hamiltonian for this problem is

sH = y.+1 T + y;fi

i-i

where the multiplier equations are

1 as: a j'Yj i=1,2, ...,5

The boundary conditions, followingEq. (11) of Sec. 10.2, are

(4)

(5)

(6)

(7)

(8a)

(8b)

yi(8) - yi(0) = 0 i = 1, 2, . . . , S (9a)

74+1(8) = 1 (9b)

350

Thus_ we obtain, finally,s

H e £ + Yifi

1 05

1aft

e axi axij-1

'y (0) _ 'yi(B) i = 1, 2,


(10)

i = 1, 2,

.,S

... 'S

(12)

The partial derivative of the hamiltonian with respect to the decisionfunction is, of course,

aH 1 M; afi (13)au au + y` T.i-1

The calculation of the periodic control function u(t) for optimalperiodic operation is easily carried out by steep descent by exploitingthe periodic boundary conditions on x and y. An initial periodic decisionfunction is specified and Eqs. (1) integrated from some initial state untilperiodic response is approached. Next, Eqs. (11) for y are integrated inreverse time from some starting value until a periodic response results.Reverse-time integration is used because of the stability problems dis-cussed in Sec. 9.8. Finally, the new decision is chosen by the usualrelation

Su = -w(t) u 0 < t < B (14)

and,, the process is repeated. Convergence to periodic response mightsometimes be slow using this simulation procedure, and a Newton-Raphson scheme with second-order convergence properties can bedeveloped.

We have already seen for the special case in Sec. 6.11 that all/auvanishes at an interior optimal steady state, necessitating the use of thestrong minimum principle in the analysis of optimality. This is, in fact,true in general, and it then follows that a steady state cannot be used asthe starting value for a steep-descent calculation, for then du in Eq. (14)would not lead to an improvement beyond the optimal steady state.We prove the general validity of this property of the steady state bynoting that in the steady state we seek to minimize ff(x,u) subject tothe restrictions

f;(x,u) = 0 i = 1, 2, . . . , S

The lagrangian is thens

(X' U) + XJ1(x,u)

(15)

(16)i-1

NONSERIAL PROCESSES

and it is stationary at the solutions of the following equations:S ' i

0T au + I aui-1S afax;

= ax; + x; = o

jsli=1,2,...,5

351

(17)

(18)

Identifying X; with B-y;, Eq. (17) is equivalent to the vanishing of all/auin Eq. (13), while Eqs. (15) and (17) are the steady-state equivalents ofEqs. (1) and (11), whose solutions trivially satisfy the periodicity bound-ary conditions.

As a computational example of the development of an optimal peri-odic operating policy using steep descent we shall again consider the reac-tor example of Horn and Lin introduced in Sec. 6.11. Parallel reactionsare carried out in a stirred-tank reactor with material-balance equations

xl = -ux1" - aurxi - x1 + 1 (19a)x2=ux1"-x2 (19b)

x, and x2 are periodic over the interval 0. The temperature-dependentrate coefficient u(t) is to be chosen periodically, subject to constraints

u* <u<u*to maximize the time-average conversion of X2, that is, to minimize

SB fo

x2(t) dt

Then

Bx2

(20)

(21)

(22)

The periodic Green's functions satisfy the differential equations

y, = -y,(nux,"-l + au' + 1) - ny2ux1"-1 (23a)

72 = 8 + 72

The hamiltonian is1

(23b)

H = -B

x2 - yl(uxl" + au'x1 + x1 - 1) + 7'2(uxln - x2) (24)

with a partial derivative with respect to uaHau = - (71x1" + aylru'-1x1 + 72x1")

It is shown in Sec. 6.1 that when

nr-1>0 r<1

(25)

(26)

improvement can be obtained over the best steady-state solution.

3S2 OPTIMIZATION BY VARIATIONAL METHODS

For this particular problem some simplification results. Equation(19a) involves only u and x1 and can be driven to a periodic solution forperiodic u(t) in the manner described above. Equations (19b), (23a),and (23b) can then be solved in terms of u and x1 with periodic behaviorby quadrature, as follows:

x2(t) = x2(0)e-e + foe

t u(r)x1"(r) dr (27a)

(27b)X2(0)1 le-0 fo

dr

72(t) const (28)

71(t) _ 'Y1(O) exp [fo (nuxln-1 + au' + 1) dv]

+ fo exp [f' (nuxln-1 + au' + 1) do] u(T)xln(T) dT (29a)

fo exp [f' (nuxln-1 + au' + 1) dv] u(T)xin(r) dr71(0) = (29b)

1 - exp [fe(nuxln-1 + au' + 1) da]

The adjustment in u(t) is then computed from Eq. (14) using all/au inEq. (25).

The numerical values used are as follows:

n =2 r =0.75a = 1 a =0.1u*=1 u*=5

The optimal steady state, computed from Eq. (15) of Sec. 6.11, isu = 2.5198, with corresponding values

x1 = 0.27144x2(= -&) = 0.18567

71 = X3.1319

The starting value of u(t) for the steep-descent calculation was taken as

u(t) = 2.5198 + 0.40 sin 20irt (30)

The weighting factor w(t) for steep descent was based on the normalizedform and taken as

w(t) = G(t) _

[f0 au} dtj34

(31)

with G(t) equal to 0.1 or the maximum required to reach a bound andhalved in case of no improvement in 8.


0 3VQv0

0

0 0.01 0.02 0.03 0 04 0.05 0 06 0.07 0.08 0.09 0.10Time f

Fig. 10.16 Successive approximations to the optimal peri-odic control using steep descent.

A response for 'x1(t) which was periodic to within 1 part in 1,000was reached in no more than eight cycles for each iteration, using thevalue of x1(9) from the previous iteration as, the starting value. Thiserror generally corresponded to less than 1 percent of the amplitude ofthe oscillation. Successive values of the periodic temperature functionu(t) are shown in Fig. 10.16 and corresponding values of the objective inFig. 10.17. The dynamic behavior of x1 and x2 is shown in Figs. 10.18

Fig.10.17 Time-averaged conversion on 0.18 5' I I 4

0 1 2 3 4 55 6 7 8 9 10 11successive iterations. Iterations number


0 280

1

0270

0 2600 001 0.02 0 03 0.04 0.05 006 0 07 0.08 0.09 0.10

I

Fig. 10.18 Dynamic response of x, on successive iterations to peri-odic forcing.

and 10.19, respectively. Note that following the first iteration the entiretime response of x2(t) lies above the optimal steady-state value. Thecourse of the iterations is clearly influenced by the sinusoidal startingfunction and appears to be approaching a bang-bang policy, though thiswas not obtained for these calculations. Horn and Lin report that theoptimum forcing function for this problem is in fact one of irJiinitely rapidswitching between bounds or, equivalently, one for which the length ofthe period goes to zero. The steep-descent calculation evidently cannotsuggest this result for a fixed value of 0.

In realistic applications the decision function will normally notinfluence the system response directly, as the temperature does in this

0 195

X20)0.190

00 0.01 0.02 003 004 005 0.06 007 008 0.09 0...

t

Fig. 10.19 Dynamic response of x2 on successive iterations to peri-odic forcing.


example, but will do so through a mechanism with which damping will beassociated. Coolant flow-rate variations, for example, must be trans-mitted to the output conversion through the energy equation for tem-perature, and high-frequency oscillations will be'masked and thereforehave the same effect as steady-state operation. An optimal period willtherefore exist, and can most easily be found by a one-dimensional search.

10.9 DECOMPOSITION

The computational procedure which we have developed for the solutionof variational problems in complex structures has depended upon adjust-ing the boundary conditions of the Green's function in order to incorpo-rate the mixing boundary conditions. There is an alternative approachthat has sometimes been suggested which we can briefly illustrate bymeans of the examples used in this chapter.

We consider first the reactor problem from Sec. 10.3.is formulated in terms of two variables

The problem

it = -k(u)F(xl) (1a)x2 = -k(u)F(x1) + g(u) (lb)

The boundary conditions are

x1(0) _ (1 - P)x/ + Px1(0) (2a)

X2(0) = 0 (2b)

with objective

C = x2(0) (3)

If we knew the value of the input x1(0), this would specify the outputx1(8). For any fixed input, say x*, we can find the function u(1) whichminimizes E subject to Eqs. (1) and (2b) and

x1(0) = xl (4a)

x P(0) = P x1 - 1 4bxf1

P( )

This is a problem for a serial process with constrained output. We callthe optimum C(xl ). For some physically realizable value of x,*, & takeson its minimum, and this must then represent the solution to the originalrecycle problem. The sequence of operations is then:

1. Choose x,*. Solve a constrained-output minimization problem to findS(x*).

2. Search over values of x; to find the minimum of S(xi) using, forexample, the Fibonacci method of Sec. 2.4.

384 OPTIMIZATION BY VARIATIONAL MtTNIDDS

In this case the ultimate computation by either this decompositionmethod or the use of recycle boundary conditions for the Green's func-tions is identical, and no saving occurs by use of one method or the other.

A somewhat more revealing example is the plant studied in Sec.10.7. The reactor equations are

0 = x1"-1 - xl" - Okloe-E,'"""(xi")2 n = 1, 2, 3 (5a)0 = x2"-' - x2" + Bk10e-e,'Iu'(x1")2 - n = 1, 2, 3

(5b)The extractor ((eq,//uationsand the boundary conditions are

x11 + u1 [.(x11) - 0(x1i1)] - x13 = 0 (6)X111 + ull0(x111').(- x11 = 0 (7)x10 = x11 + 1613#\x11) (8a)x20 = 0 (8b)

and the objective is

& = -x23 - Cxll + null (9)

The key to the decomposition of this problem is the observation that[Eq. (11a) of Sec. 10.7]

X10 = x11 + x13 - x111 (10)

Thus, by setting x13 and x111 to fixed values x13# and xIII* we completelyfix the feed and effluent of X1 from the reactor system. We have alreadyseen in Sec. 9.13 how to solve the problem of minimizing -x23 subject tofixed inputs and outputs of X1. Call this minimum -x23*(x13*,x11I*).

Furthermore, Eqs. (6) and (7) can be solved for ull in terms of x13* andx111# Call this value uuI*(x13*,xlll*). After carrying out these twooperations we then seek the proper values of x13 and xlll by the operation

min [-x23*(X18*,xlIi*) - Cxi11* + vu1I*(xl3*,xlli*)] (11)x1*a.z,.n

.The decomposition approach appears to offer no advantage in thislatter case. The computation of a single optimum temperature sequencefor a constrained output in Sec. 9.13 requires nearly as extensive a setof calculations as the complete analysis in Sec. 10.7. The subminimiza-tion would have to be carried out for each pair of values xi3, x1II in thesearch for the optimum pair using, for example, an approximation to steepdescent such as that in Sec. 2.9. The approach of breaking a largeminimization problem down into a repeated sequence of smaller problemshas been applied usefully in other areas, however, particularly certaintypes of linear programming problems, and might find application insome problems of the type treated here.

NONSERIAL PROCESSES


Section 10.2: The development here was contained in

M. M. Denn and R. Aris: AIChE J., 11:367 (1965)and : Ind. Eng. Chem. Fundamentals, 4:7 (1965)

A proof of a strong minimum principle for continuous systems is in

M. M. Denn and R. Aris: Chem. Eng. Sci., 20:373 (1965)

A first incorrect attempt to extend results for straight-chain processes to recycle processeswas by

D. F. Rudd and E. D. Blum: Chem. Eng. Sci., 17:277 (1962)

The error was demonstrated by counterexample in

R. Jackson: Chem. Eng. Sci., 18:215 (1963)

A correct development of the multiplier boundary conditions for recycle was also obtained by

L. T. Fan and C. S. Wang: Chem. Eng. Sci., 19:86 (1964)

Section 10.3: The example was introduced by

G. S. G. Beveridge and R. S. Schechter: Ind. Eng. Chem. Fundamentals, 4:2:17 (1965)

Section 10.4: The reformulation to permit use of the straight-chain process result ispointed out in

F. J. M. Horn and R. C. Lin: Ind. Eng. Chem. Process Design Develop., 6:21 (1967)

Section 10.5: The lagrangian approach was taken by Jackson to obtain results applicableto the more general problem of the following section in

R. Jackson: Chem. Eng. Sci., 19:19, 253 (1964)

Sections 10.6 and 10.7: The general development and the example follow

M. M. 1)enn and R. Aris: Ind. Eng. Chem. Fundamentals, 4:248 (1965)

Further examples of the optimization of nonserial processes may be found in

L. T. Fan and C. S. Wang: "The Discrete Maximum Principle," John Wiley & Sons,Inc., New York, 1964

-- and associates: "The Continuous Maximum Principle," John Wiley & Sons,Inc., New York, 1966

J. D. Paynter and S. G. Bankoff: Can. J. Chem. Eng., 44:340 (1966); 45:226 (1967)

A particularly interesting use of the Green's functions to investigate the effects on reactionsystems of local mixing and global mixing by recycle and bypass is in

F. J. M. Horn and M. J. Tsai: J. Optimization Theory Appl., 1:131 (1967)R. Jackson: J. Optimization Theory Appl., 2:240 (1968)

Section 10.8: This section is based on the paper by Horn and Lin cited above. The paperwas presented at a symposium on periodic processes at the 151st meeting of theAmerican Chemical Society in Pittsburgh, March, 1966, and many of the otherpapers were also published in the same issue of Industrial and Engineering Chemis-try Process Design and Development. Another symposium was held at the 60th


annual meeting of the AIChE in New York, November, 1967. An enlighteningintroduction to the notion of periodic process operation is

J. M. Douglas and D. W. T. Rippin: Chem. Eng. Sci., 21:305 (1966)

A practical demonstration of periodic operation is described in

R. H. Wilhelm, A. Rice, and A. Bendelius: Ind. Eng. Chem. Fundamentals, 6:141(1966)

Section 10.9: The basic paper on decomposition of recycle processes is

L. G. Mitten and G. L. Nemhauser: Chem. Eng. Progr., 59:52 (1963)

A formalism is developed in

R. Aris, G. L. Nemhauser, and D. J. Wilde: AIChE J., 10:913 (1964)

and an extensive discussion of strategies is contained in

D. J. Wilde and C. S. Beightler: "Foundations of Optimization," Prentice-Hall, Inc.,Englewood-Cliffs, N.J., 1967

See also

D. F. Rudd and C. C. Watson: "Strategy of Process Engineering," John Wiley &Sons, Inc., New York, 1968

PROBLEMS

10.1. The autocatalytic reaction X, + X, = 2X2 in a tubular reactor is describedby the equations

i - kio exp C-EIJ xIxt - kto expC_Et\

x,2.t, uJ u

x, + xt - coastThe reaction is to be carried out in a reactor of length L with product recycle such that

x,(0) = (1 - p)xi + px,(L)x2(0) - (1 - p)xi + px,(L)

Develop an equation for the maximum conversion to x*(L) when

E,= 2 xj = 1 - E Zq = E

E,

(The problem is due to Fan and associates.)10.2. A chemical reaction system in a tubular reactor is described by the differentialequations

z, = f: (z)x;(0) - xio

with an objective 61[z(B)]. Show that an improvement in tP can be obtained byremoving a small stream at a point ti and returning it at t: only if

[Yi(tt) - Yi(tl)k:(ti) > 0

Obtain the equivalent result for recycle of a stream from t, to ti. (The problem isdu: to Jackson and Horn and Tsai.)

11

Distributed -parameter Systems

21.1 INTRODUCTION

A great many physical processes must be described by functional equations of a more general nature than the ordinary differential and differ-ence equations we have considered thus far. The two final examples inChap. 3 are illustrative of such situations. The procedures we havedeveloped in Chaps. 6, 7, and 9 for obtaining necessary conditions andcomputational algorithms, however, based upon the construction ofGreen's function for the linear variational equations, may be readilyextended to these more general processes. We shall illustrate the essen-tially straightforward nature of the extension in this chapter, where wefocus attention on distributed-parameter systems describable by partialdifferential equation.; in two independent variables.

11.2 A DIFFUSION PROCESS

As an introduction to the study of distributed-parameter systems weshall consider the generalization of the slab-heating problem of Sec.

359


3.12, for the results which we shall obtain will be applicable not only to amore realistic version of that problem but to problems in chemicalreaction analysis as well. The state of the system is describedt by the(generally nonlinear) diffusion-type partial differential equation

ax ax a2x 0 < z < 1at = I x,

az' az2t z,tl 0 < t < e

with a symmetry conditionat z = 1ax=0 atz=Iforalltaz

and a "cooling" condition at z = 0,

axaz = g(x,v) at z = 0 for all t

(1)

(2)

(3)

where v(t) represents the conditions in the su*ronnding environment.The initial distribution is given

x = 0(z) at t = 0 (4)

The environmental condition v(t) may be the decision variable, or itmay be related to the decision variable through the first-order ordinarydifferential equation

r dt = h(v,u) (5)

The former case (lagless control) is included by taking the time constant rto be zero and h as u - v. It is assumed that the purpose of controlis to minimize some function of the state at time B, together with thepossibility of a cost-of-control term; i.e., we seek an admissible functionu(t), 0 < t < 0, in order to minimize

s[u] = f01 E[x(0,z);z] dz + fo C[u(t)] dt (6)

11.3 VARIATIONAL EQUATIONS

We shall construct the equations describing changes in the objectiveas a result of changes in the decision in the usual manner. We assumethat we have available functions u(t), 0(1), and x(t,z) which satisfy Eqs.(1) to (5) of the preceding section. If we suppose that in some way wehave caused an infinitesimal change Sv(t) in v(t) over all or part of therange 0 < t < 0, this change is transmitted through the boundary condi-tions to the function x(t,z) and we can write the variational equations to

t Note that the state at any time t" is represented by an entire function of z, z(t',z),0 < z < 1.

DISTRIBUTED-PARAMETER SYSTEMS 361

first order asf

(bx), = f= ax + (ax), + J=..(ax): (1)(bx), = g= ax + g, av at z = 0 for all t (2)(bx), = 0 at z = 1 for all t (3)ax = 0 when t = 0 for all z (4)

We construct the Green's function y(t,z) for this linear partial differ-ential equation as in Sees. 6.2 and 7.2. We consider first the producty ax and write

(y bx), = y, ax + y (Sx),= 'y ax + yf= bx + 7f=.(ax)= + yJ=..(ax)s= (5)

or, integrating with respect to z,

at fol(y ax) dz = fo' [ye bx + yf= ax + 'Yf:.(ax). + YJ:..(ax)s:] dz

The last two terms can be integrated by parts

fo yf.,(ax). dz = - fl (yJ=)= ax dz + yf=. ax Ioofol

yJ=.,(ax).s dz = fo' (yf=..),, ax dz + [yf.,,(ax)= - ('f=..)= ax]


at f o' (y ax) dz = fog [y, + yJ= - (yJ=,)= + (yJ=ax dz

+ { bx[yf=, - (yJ=..)=] + (bx).yf=..1

(6)

(7a)I

0

(7b)

0(8)

Integrating with >;espect tot from 0 to B and using Eqs. (2) to (4), wethen obtain

foI y(B,z) bx(9,z) dz = fo fo' [ye + 'Yf= - (yf=.)=

+ (yJ=,.)=_] ax dz dt + fo ax[-(f., - dt 1', - 1

fo ax[-ff., + yf=..g= - (yf=.) } dt IL-o

- fo y(t,0)f.,,(t,0)g.(t) av(t) dt (9)

t In accordance with common practice for partial differential equations and in orderto avoid some cumbersome notation we shall use subscripts to denote partial dif-ferentiation in Sees. 11.3 to 11.7. Thus,

ax ax ofx` at

X.- as f=" - a0=x/M)

and so forth. Since we are dealing here with a single state variable and a singledecision variable, and hence have no other need for subscripts, there should be noconfusion.


As before, we shall define the Green's function so as to removeexplicit dependence upon 6x from the right-hand side of Eq. (9). Thus,we require y(t,z) to satisfy the equation

ye = -yf= + (yf:.)= - (l'f=..)::0 < z < 10 < t < B (10)

y(f1, + f." g.) - (yfZ )s = 0 at z = 0 for all t (11)if., -(yf')s=0 at z = 1 for


all t (12)

f01y(O,z) ax(9,z) dz = - fo 8v(t) dt (13)

Finally, we note that the variation in the first term of the objective[Eq. (6), Sec. 11.2] as a.result of the infinitesimal change Sv(t) is

fo E2[x(O,z);z} Ex(9,z) dz

If we complete the definition of 'y with

y=E2 att=0forallzthen we may write

(14)

foI E=[x(9,z);z] ax(9,z) dz = - fo av(t) dt (15)

11.4 THE MINIMUM PRINCIPLE

In order to establish a minimum principle analogous to that obtained inSec. 6.8 we must consider the effects on the objective of a finite perturba-tion in the optimal decision function over a very small interval. If u(t)is the optimal decision, we choose the particular variation

Su(t) = 0 0 < t < t,t1+A<t<tl (1)

Su(t) = finite t, < t < t, + 0where A is infinitesimal. Thus, during the interval 0 < t < t1 the valueof v(t) is unchanged, and Sv(t1) = 0.

During the interval t1 < t < t1 + A it follows from Eq. (5) ofSec. 11.2 that Sv must satisfy the equation

T(bv), = h(v + Sv, u + du) - h(f,u) (2)or

dv(t) = T I [h(v + Sv, u + Su) - h(v,u)] dl (3)

The integral may be expressed to within o(d) as the integrand evaluatedat l1 multiplied by the integration interval. Thus, using the fact that


Wt,) is zero we may write

av(t) =t - t,

{h[v(ti), u(ti) + au(tl)1 - o(o)

t1 < t < t, + A (4)and, in particular,

av(tI + o) = (h[v(t,), u(ti) + bu(rl)] - h[v(t,),u(ts)]I + o(o)T

(5)

Because A is infinitesimal this change in v(t) resulting from the finitechange bu is infinitesimal.

Following time t, + A the variation au is again zero. The varia-tional equation describing the infinitesimal change in v is then, to withino(ff),

T(av), = h,(v,u) av t, + o < t < 0which has the solution, subject to Eq. (5),

(6)

av(t) = [h(i,u) - h(v,u)] exp {T f+ ho[vO,uO] d2 } + o(o)

ti+A<"t<8 (7)where the term in brackets preceding the exponential is evaluated att=t,.

Combining Eq. (7) with Eqs. (6) and (15) of Sees. 11.2 and 11.3,respectively, we obtain the expression for the change in S resulting fromthe finite change in u -

[h(v,u) - h(v,u)] f-f (OW ." (8,0) g' (8)

exp [Tf e

h,(t) dEJ ds + A[C(u) - C(u)} + o(A) > 0 (5)

where the inequality follows from the fact that u(t) is the function whichminimizes &. Dividing by A and then taking the limit as d -- 0 and not-ing that t, is completely arbitrary,'we obtain the inequality

C(u) f y(s,0)f=..(s,0)ge(s) exp [T f d 1 dsT J

> C(u) (e y(s,0)f._(s,0)g.(s) exp [f had) dE] ds (9)T

TJe

That is, it is necessary that the optimal function u(t) minimize

min C(u) exp I1 f d li,(t) dEj dsU(t) T t T t

(10)


ieverywhere except possibly at a set of discrete points. If the final time 9is not specified, we obtain a further necessary condition by differentiatingEq. (6) of Sec. 11.2 with respect to 0 and equating the result to zero toobtain

0 unspecified: foI y(B,z)f(B,z) dz + C[u(9)] = 0 (11)

In the important case of the lagless controller, where

h(v,u) = u - vand r goes to zero, we can utilize the fact that for small r the integrand inEq. (10) is negligibly small for all time not close to t. Taking the limitas r -+ 0 then leads to the result

u = v: min C(u) - uy(t,0)f(t,0)g (t) (12)u(t)

with v equal to the minimizing value of u.

11.5 LINEAR HEAT CONDUCTION

The linear heat-conduction system examined in Sec. 3.12 with a quad-ratic objective has the property that y and x can be obtained explicitlyin terms of u. If we consider

_ xx - 0<z<1 (1)t :` 0<t<0

X. = 0 at z = 1 for all t (2)xz = p(x - v) at z = 0 for all t (3)x=0 att=0forallz (4)rv,u - V 0 <t<0 (5)

min E =fol

[x-(z) - x(B,z)]2 dz + fo C[u(t)] dt (6)

where 0 is specified, 17 a constant, and x*(z) a specified function, Eqs. (10)to (12) and (14) of Sec. 11.3 for y then become

0<z<1y`--y" o<<t<0

(7)

py - y. = 0 at z = 0 for all t (8)y, = 0 at z = 1 for all t (9)y = -2[x*(z) - x(@,z)] at t = 0 for all z (10)

Equations (1) to, (5) and (7) to (10) are solved in terms of u(t) byelementary methods such that the minimum principle, Eq. (10) of thepreceding section, becomes

min C[u(t)] + u(t) [fo G(l,s)u(s) dx - kt)] (11)u(t)


where, as in Sec. 3.12,

41 (0 = foI x"(z)K(0 - t, z) dz (12)

G((s) = foI K(8 - t, z)K(O - s, z) dz (13)

anda2 cos all - z) a-a,tK(t z), Cos a - a/q sin a

+ 2a2Cos (1 - z)/ii 5"te- (14)

.L1 (a2 - t.2)[1/P + (1 + P)/O,2J cos t,

with a = 1/Vr and tic the real roots of

0 tan 0 = p (15)

Thus, for example, if C(u) = %c1u2 and u(t) is unconstrained, theminimum of Eq. (11) is obtained by setting the partial derivative withrespect to u(t) to zero. The optimal control is then found as the solutionof the Fredholm integral equation of the second kind

fo G(t,s)u(s) ds + c2u(t) = 4,(t) (16)

If, on the other hand, C(u) = 0 and u(t) is constrained by limitsu, < u < u', Eq. (11) is linear in u and the optimal control is at alimit defined by

u(t)= u fa G(t,s)u(s) ds - '(t) < 0

(17)G(t,s)u(s) ds - ¢(t) > 0U* f0

An intermediate (singular) solution is possible as the solution of theFredholm integral equation of the first kind

fo G(t,s)u(s) ds = '(t) (18)

Equation (18) is the result obtained as a solution to the simplest problemin the calculus of variations in Sec. 3.12.

11.6 STEEP DESCENT

For determination of the optimum by means of a steep-descent calcu-lation we first choose a trial function u(t) and then seek a small changeft(t) over all t which leads to an improved value of the objective C. Weneed, then, the effect on C of a continuous small variation in u, ratherthan the finite sharply localized variation used in the development ofthe minimum principle.


From Eqs. (6) and (15) of Sees. 11.2 ajid 11.3, respectively, we have,for any small change bu, the first-order expression for the correspondingchange in Is,

bs = fo Cu(t) bu(t) dt - fo y(t,O)f...(t,O)g.(t) bv(t) dt (1)

Here, by can be -related to a continuous small change bu by the vari-ational equation derived from Eq.' (4) of Sec. 11.2

r 6v, = h, by + h. bu (2)or

bv(t) = T fo exp LT f` h,(E) dE] bu(s) ds (3)

Thus, by substituting for by in Eq. (1) and changing the order of inte-gration we obtain the expression for the variation in the objective

SE=fo {C

hu f ° y(s,O)f...(s,O)g.(s) exp [T f ' h,(E) dt] I bu(t) dt (4)

The choice of 'bu(t) which leads to a decrease in S is then

au(t) = -w(t) I C.

- T h f 9

'(s,0)f=..(s,0)g.(s) exp [f.

d>:] ds} (5)

where w(t) > 0 reflects the geometry of the space and is chosen suf-ficiently small to ensure that a constraint is not violated. For r = Ofwith u equal to v, we obtain

bu(t) = -w(t)[C - y(t,0)f (t,0)g.(t)J (6)

11.7 COMPUTATION FOR LINEAR HEAT CONDUCTION

As a first application of the computational procedure to a distributedsystem we return to the linear heat-conduction problem described in Sec.11.5 without the cost-of-control term. The system equations ere

0<z<1_x,-x:: 0<t<0 (1)

xy=0 atz= lfor all t (2)

x, = p(x - v) at z = 0 for all i (3)

x=0 att=0foralIz (4)

rv,=u - V 0 <t<0 (5)


u* <u <u* (6)

E = 12[x*(z) - X(O,Z)]2 (7)

C=0 (8)

0 < z < 1_0<t<0 (9)

py-ye=0 atz=0forallt (10)

y.=0 at z= 1 for all (11)y = -2[x*(z) - x(O,z)) at t = 0 for all z (12)

The necessary conditions for an optimum are given in Sec. 11.5 by Eqs.(17) and (18), while the direction of steep descent is determined by

ou(t) _ -w(t) p f e

y(8,0)e(t )!T ds (13)

The linearity of the system makes it possible to obtain and use ananalytical expression for y(s,0).

The parameters used in the numerical solution are

p=10 r=0.04x*(z) = constant = 0.2

u*=0<u<1=u*Two cases are considered, 0 = 0.2 and 0 = 0.4, with the starting func-tions u(t) taken as constant values of u(t) = 0, 0.5, and 1.0 in each case.Numerical integration was carried out using Simpson's rule, with both zand t coordinates divided into 50 increments. w(t) was arbitrarily set as10, or the maximum less than 10 which would carry u to a boundaryof the admissible region, and halved whenever a decrease in the objectivewas not obtained.

Figure 11.1 shows several of the successive control policies calcu-lated from Eq. (13) for 0 = 0.2 with a constant starting policy of u = 0.5,no improvement being found after 10 iterations. The policy is seen toapproach a bang-bang controller, with the exception of a small timeincrement at the end, and the final policies from the other two startingvalues were essentially the same. The successive values of the objectiveare shown in Fig. 11.2, with curves I, II, and III corresponding, respec-tively, to starting values u(t) = 0, 0.5, and 1. The three temperatureprofiles x(0,z) at t = 0 corresponding to the starting control policies,together with the final profile, are shown in Fig. 11.3, where it can beobserved from curve III' that no control policy bounded by unity canraise the temperature at z = 1 to a value of 0.2 in a time period 0 = 0.2.

Figure 11.4 shows several of the iterations for the optimal controlwith 0 = 0.4, starting with the constant policy u(t) = 0.5, with no.improvement possible after 11 descents. The curve for the second itera


1.0

0.8

00.6

CL

0.40U

0.2

0.4 0.8

Time t0.12 0.16 0.20

Fig. 11.1 Successive approximations to the optimal control policyusing steep descent, 8 = 0.2. [From M. M. Denn, Intern. J. Control,4:167 (1966). Copyright 1966 by Taylor and Francis, Ltd. Re-printed by permission of the copyright owner.]

tion is not shown beyond t = 0.25 because it differs in general by toolittle from the optimal curve to be visible on this scale. The sharp oscilla-tions in the optimal controller at alternate points in the spatial grid werefelt to have no physical basis, and a crude smoothed approximation tothe optimum was then used as a starting function for the steep-descentprogram. This converged to the dashed curve shown in Fig. 11.4, whichhas very nearly the same value of the objective as the oscillating result.

Figure 11.5 shows the reduction in the objective for steep-descent

2 3

I I I

2 4 6 8Iteration number

Fig. 11.2 Reduction in the objective onsuccessive iterations, 8 = 0.2. [FromM. M. Denn, Intern. J. Control, 4:167(1966). Copyright 1966 by Taylor andFrancis, Ltd. ' Reprinted by permissionof the copyright owner.]

DISTRIBUTED-PARAMETER SYSTEMS

0.2 04 0.6Position z

36!

1.0

Fig. 11.3 Temperature profiles at t = 8 using initial andoptimal control policies, 8 - 0.2. ]From M. M. Denn,Intern. J. Control, 4:167 (1966). Copyright 1966 by Taylorand Francis, Ltd. Reprinted by permission of the copyrightowner.l

calculations starting from constant control policies of u(t) = 0, 0.5, and 1,corresponding to curves I, II, and III, respectively, with the temperatureprofile at t = 0 for each of these starting policies shown in Fig. 11.6. Theoptimum is indistinguishable from 0.2 on the scale used. The threeunsmoothed policies are shown in Fig. 11.7. They indicate substantiallatitude in choosing the control, but with the exception of the oscillationsin the later stages, which can be smoothed by the same computer program,all are smooth, intermediate policies, corresponding to approximationsto the singular solution defined by the integral equation (18) of. Sec. 11.5.

1.0

0a60.4c0U

0.2

0.04 0.08 0.12 0.16 0.20 0.24Time t

0.28 0.32 0.36 0.40

Fig. 11.4 Successive approximations to the optimal control policyusing steep descent, 8 - 0.4. [Front M. M. Denn, Intern. J. Control,4:167 (1966). Copyright 1966 by Taylor and Francis, Ltd. Re-printed by permission of the copyright owner.]

IE

10-

to-

10-

10-4 6 8 10 12

Iteration number


Fig. 11.5 Reduction in the objective onsuccessive iterations, 0 = 0.4. [FromM. M. Denn, Intern. J. Control, 4:167(1966). Copyright 1966 by Taylor andFrancis, Ltd. Reprinted by permissionof the copyright owner.)

It is interesting to note the change in the optimal control strategy frombang-bang to intermediate as more time is available. It has sometimesbeen suggested in the control literature that steep descent cannot beused to obtain singular solutions to variational problems, but it is clearfrom this example that such is not the case.

1.0

ti

m 0.8

0.60w 0.4

E

0.2

0 0.2 0.4 0.6

Distance z

0.8 1.0

Pig. 11.6 Temperature profiles at t = 0 using initial andoptimal control policies, 0 - 0.4. [From M. M. Denn,Intern: J. Control, 4:167 (1966). Copyright 1966 by Taylorand Francis, Ltd. Reprinted by permission of the copyrightowner. I


0.04 0.08 0.12 0.16 0.20 0.24 0.28 0.32 0.36 0.40Time f

Fig. 11.7 Unsmoothed intermediate optimal control policies for threestarting functions, 0 - 0.4. [From M. M. Denn, Intern. J. Control,4:167 (1966). Copyright 1966 by Taylor and Francis, Ltd. Reprintedby permission of the copyright owner.]

11.8 CHEMICAL REACTION WITH RADIAL DIFFUSION

We have considered the optimal operation of a batch or pipeline chemicalreactor in Sec. 3.5 with respect to the choice of operating temperature andin Sec. 5.11 with respect to the optimal heat flux. In any realistic con-sideration of a packed tubular reactor it will generally be necessary totake radial variations of temperature and concentration into account,requiring even for the steady state a description in terms of two inde-pendent variables. The study of optimal heat-removal rates in such areactor provides an interesting extension of the results of the previoussections and illustrates as well how an optimization study can be used inarriving at a practical engineering design.

In describing the two-dimensional reactor it is convenient to definex(t,z) as the degree of conversion and y(t,z) as the true temperaturedivided by the feed temperature. The coordinates t and z, the axialresidence time (position/velocity) and radial coordinate, ;espectively, arenormalized to vary from zero to unity. With suitable physical approxi-mations the system is then described by the equations

ax _ A (1 a /z L-[- Darr(x,y)

0 < t < 1 (1)cat Pe zaz` az 0<z<1ay A 1 a z ay + Dairlr(x,y)

0< t< 1at X7 z r az 0< z< 1 (2)

Here A is the ratio of length to radius, Pe and Pe' the radial Peclet num-bers for mass and heat transfer, respectively, Dal and Datii the first and.third Damkohler numbers, and r(x,y) the dimensionless reaction rate.


The boundary conditions are

x=0 att=0forallz (3a)y = 1 at t = 0forallz (3b)axa =

0 at z = 0 for all t (3c)

az = 0 at z = 0 for alit (3d)

ax az=0 atz=lforallt (3e)

ayaz = - u(t) at z = I for all t (3f)

The dimensionless wall heat flux u(t) is to he chosen subject to bounds

u* < u(t) < u* (4)

in order to maximize the total conversion in the effluent

t(1) = 2 foI zx(l,z) dz (5)

41

The generalization of the 'inalysis of Secs. 11.2 to 11.4 and11.6 to include two dependent variables is quite straightforward, and weshall simply use the results. We shall denote partial derivatives explic-itly here and use subscripts only to distinguish between components y,and y2 of the Green's vector. The multiplier equations corresponding toEq. (10) of Sec. 11.3 are

ayi - (Daiyi + Daliiy2)ar + A a yi _ a2y1

(6)at=

ax Pe az z 4922

2at- = -(Dajyi + Daiij72) ay + PP Gz 72 5-,-) (7)

while the boundary conditions corresponding to Eqs. (11), (12), and (14)are

yl = -2z at t = 1 for all z (8a)72 = 0 at t = 1 for all z (8b)y,=y20 atz=0foralf t (8c)

y, azl = y2 - z2 = 0 at z = l for all t (8d)

The minimum principle corresponding to Eq. (12) of Sec. 11.4 is

min - Au(() Pei 720100) (9)

DISTRIBUTED-PARAMETER SYSTEMS .37

The linearity of the minimum criterion requires that the optimalcooling function have the form

u* 72(1,1) > 0U11

u* 72(t,l) < 0(10)

with intermediate cooling possible only if 72(t,l) vanishes for some finiteinterval. This is the structure found for the optimal operation of thereactor without radial effects in Sec. 5.11. Indeed, the similarity canbe extended further by noting that if Eq. (8d) is taken to imply that 12 isapproximately proportional to z near z = 1, then the second parenthesison the right side of Eq. (7) must be close to zero near z = 1. Thus12(1,1) can vanish for a finite interval only if the reaction rate near thewall is close to a maximum; i.e., the singular solution implies

8ray

0 z = 1 (11)

which must be identically true without radial effects. If the parametersare chosen such that u* = 0 and 12(1,1) is initially zero, then since whenu = 0 initially there are no radial effects, the switch to intermediatecooling should be close to the switch for the one-dimensional reactor.

Computations for this system were carried out using steep descent,the direction corresponding to Eq. (6) of Sec. 11.6 being

BU(t) =. w(t)72(t,l) (12)

For the computations the reaction was taken to be first order, in whichcase the dimensionless reaction rate has the form

r(x,y) = eE,'ITo[(1 _ x)e-E,'IYTo - kxe-E,'/VToJ (13)

where To is the temperature of the feed. The physical parameters werechosen as follows:

Pe = 110Ei = 12,000

Pe' = 84.5E' =,25,000

A=50Dal = 0.25 Darn = 0.50To = 620 u* = 0

u* was not needed for the parameters used here. The solution, for thecase without radial effects, following the analysis of See. 5.11, gives aninitially adiabatic policy (u = 0) with a switch to an intermediate policyat t = 0.095.

Solutions of the nonlinear partial differential equations wereobtained using an implicit Crank-Nicholson procedure with various gridsizes. Figure 11.8 shows successive values of the heat flux starting from

374

0.5Axial distance t


Fig. 11.8 Successive approximations tothe optimal heat flux using steep descent.[From M. M. Denn, R. D. Gray, Jr., andJ. R. Ferran, Ind. Eng. Chem. Funda-mentals, 5:59 (1966). Copyright 1966by the American Chemical Society. Re-

0 printed by permission of the copyrightowner.)

a nominal curve arbitrarily chosen so that a reactor without diffusionwould operate at a dimensionless temperature y, of 1.076. w(t) was takeninitially as 0.11f' [y2(t,l)J2 dt and halved when no improvement wasobtained. For these computations a coarse integration grid was usedinitially, with a fine grid for later computations. No improvement couldbe obtained after the fourth iteration, and the technique.-vas evidentlyunable to approach a discontinuous profile, although the sharp peak atapproximately t = 0.075 is suggestive of the possibility'of such a solution.The successive improvements in the conversion are shown by curve I inFig. 11.9.

The next series of calculations was carried out by assuming thatthe optimal policy would indeed be discontinuous and fixing the pointat which a switch occurs from adiabatic to intermediate operation. Thesteep-descent calculations were then carried out for the intermediatesection only. Figures 11.10 and 11.11 show such calculations for theswitch at t = 0.095, the location in the optimal reactor without radialvariations. The starting policy in Fig. 11.10 is such that tOsectionbeyond t = 0.095 in a reactor without radial effects would remain at aconstant value of y of 1.076 [this is the value of y at t = 0.095 for u(t) = 0,i<0.0951. The values of the objective are shown in Fig. 11.9 as curve II,with no improvement after two iterations. The starting policy inFig. 11.11 is the optimum for a reactor without radial effects, with suc-cessive values of the conversion shown as curve III in Fig. 11.9. It issignificant that an optimal design which neglects radial effects when theyshould be included can result in a conversion far from the best possible.The maximum conversion was found to occur for a switch to intermediate


Fig. 11.9 Improvement in outlet con-version on successive iterations. [FromM. M. Denn, R. D. Gray, Jr., andJ. R. Ferron, Ind. Eng. Chem. Funda-mentals, 5:59 (1966). Copyright 1966by the American Chemical Society. Re-printed by permission of the copyrightowner.]

0.16

0.14

0U

0

0

10

0.08

0.061 1 1


375

operation at t = 0.086, but the improvement beyond the values shown-here for a switch at t = 0.095 is within the probable error of the computa-

tionaat-seh eme.One interesting feature of these 'ellealations was the discovery that

further small improvements could be obtained-followcing the convergenceof the steep-descent algorithm by seeking changes in u(0-in-_the directionopposite that predicted by Eq. (12). This indicates that the cumulative

15

Fig. 11.10 Successive approximations tothe optimal heat flux using steep descentwith switch point specified at t = 0.095.[From M. M. Denn, R. D. Gray, Jr., andJ. R. Ferron, Ind. Eng. Chem. Funda-mentals, 5:59 (1966). Copyright 1966 bythe American Chemical Society. Reprint-ed by permission of the copyright owner.]

0.5

0 L ; J

0 0.5Axial distance t

I.0

376

I i

0.5 1.0

Axial distance f


Fig. 11.11 Successive approximations tothe. optimal heat flux using steep descentwith switch point. specified at t e 0.095.[From M. M. Denn, R. D.+Gray, Jr., andJ. R. Ferron, Ind. Eng. Chem. Funda-mentals, 5:59 (1966). Copyright 1906 bythe American Chemical Society. Reprint-ed by permission of the copyright owner.)

error associated with solving Eqs. (1) and (2) followed by Eqs. (6) and (7)was sufficiently large to give an incorrect sign to the small quantityy2(t,1) in the region of the optimum. Such numerical difficulties mustbe expected in the neighborhood of,the optimum in the. solution of varia-tional problems for nonlinear distributed systems, and precise resultswill not be obtainable.

From an engineering point of view the results obtained thus farrepresent a goal to be sought by a practical heat-exchanger design. Thecoolant temperature required to produce the calculated beat-removalrate u(t) can be determined by the relation

ay(t_1) _ -u(t) = n[yJt) - y(t,l)l (14)aZ

where ye is the reduced coolant temperature and +1 a dimensionless overallheat-transfer coefficient times surface area. Using a value of n = 10,the functions y,(t) were calculated for the values of u(t) obtained fromthe steep-descent calculation. The variation of ye as a function of t wasgenerally found to be small, and a final series of calculations was carriedout to find the best constant value of y,,, since an isothermal coolant iseasily obtained in practice. Here the best switch from adiabatic opera-tion was found at t = 0.07i and yc = 0.927, but the results were essen-tially independent of the switch point in the range, studied. For a switchat.t = 0.095, corresponding to the calculations discussed here, the bestvalue of ye was found to be 0.925. The corresponding function u(t),


calculated from Eq. (14), is plotted as the dashed line in Fig. 11.12,together with the results from the steep-descent calculations. The cor-respondence with the results from the rigorous optimization procedureis striking, indicating for this case the true optimality of the conventionalheat-exchanger design, and, in fact, as a consequence of the numericaldifficulties in the more complicated steep-descent procedure, the con-version for the best constant coolant is slightly above that obtained fromthe steep-descent computation.

This example is a useful demonstration of several of the practicalengineering aspects of the use of optimization theory. An optimal designbased on an incomplete physical model, such as neglect of importantradial effects, might give extremely poor results in practice. A carefuloptimization study, however, can be used to justify a practical engineer-ing design by providing a theoretical guideline for required performance.Finally, in complicated systems the application of the theory is limitedby the sophistication and accuracy of the numerical analysis, and allresults in such'situations must be taken as approximate.

11.9 LINEAR FEEDFORWARD-FEEDBACK CONTROL

The two examples of optimization problems in distributed-parametersystems which we have examined thus far have both been situations inwhich the decision function has appeared in the boundary conditions.In a number of important applications the decision function enters thepartial differential equation directly. The simplest such situation, which

Fig. 11.12 Heakflux profiles obtainedusing steep descent from three startingfunctions. The dashed line correspondsto an isothermal coolant. [After M. 111.Denn, R. D. Gray, Jr., and J. R. Ferron,Ind. Eng. Chem. Fundamentals, 5:59(1966). Copyright 1966 by the Ameri-can Chemical Society. Reprinted by per-mission of the copyright owner.)


includes as a special case the regulation of outflow temperature in a cross-flow heat exchanger, would be described by the single linear hyperbolicequation

ax. <T + V a x = Ax + Bu

0 < t a(1)

0<z<1

where the coefficients V, A, and B are constants and u is a function onlyof t. We shall restrict attention to this case, although the generalizationto several equations and spatially varying coefficients and control func-tions, as well as to higher-order spatial differential operators, is direct.

The system is presumed to be in some known state at time t = 0.We shall suppose that disturbances enter with the feed stream at z = 0,so that the boundary condition has the form

x(t,0) = d(t) (2)

and that the object of control u(t) is to maintain a weighted positionaverage of x2 as small as possible, together with a cost-of-control term.That is, we seek the function u(t) which minimizes the cumulative error

S fo [ f of C(z)x2(t,z) dz + u2(t) ] dt (3)

This formulation includes * the special case in which we wish only toregulate x at z = 1, for the special choice C(z) = C6(1 - z), where S(¢)is the Dirac delta, leads to

g = 2 fo [Cx2(t,1) + u2(t) dl] (4)

The necessary conditions for optimality are obtained in the usualmanner, by constructing the Green's function for the variational equa-tions and examining the effect of a finite change in u(t) over an infinites-imal interval. The details, which are sufficiently familiar to be left as aproblem, lead to a result expressible in terms of a hamiltonian,

H = 3zCx2 + 322u2 + y(Ax + Bu) (5)

asCIX axat + V az - 8 = Ax + Bu

ay+VayaHCx-Ayat az ax

(6)

y(O,z) = y(t,1) = 0

H demin0u(t)


The minimum condition in turn implies

u(t) = -B fol y(t,t) dt (10)

with a minimum ensured by requiring C(z) > 0.The system of Eqs. (6), (7), and (10) are analogous to the lumped-

parameter system studied in Sec. 8.2, where we found that the Green'sfunctions could be expressed as a linear combination of the state and dis-turbance variables, provided the disturbances could be approximated asconstant for intervals long with respect to the system response time.We shall make the same assumption here concerning d(t), in which case,interpreting x and y at each value of z as corresponding to one componentof the vectors x and y in the lumped system,'the relation analogous toEq. (7) of Sec. 8.2 is

y(t,z) =101

M(t,z,E)x(i,E) A + D(t,z) d(t) (11)

The boundary conditions of Eq. (8) require

M(t,l,:;) = M(e,z,t) = D(t,l) = D(8,z) = 0 (12)

It follows from the results for the lumped system (and will be establishedindependently below) that M is symmetric in its spatial arguments

M(t,z,) = M(t,E,z) (13)

Substitution of Eq. (11) into the partial differential equation (7)yields

+ V ay = -C(z)x(z) - A f 1 M(z,E)x() dt - AD(z)d (14a)at

Here and henceforth we shall suppress any explicit dependence on t, butit will frequently be necessary to denote the spatial dependence. It isconvenient to rewrite Eq. (14a) in the equivalent form

at + V az = - fo' [C(l:)b(z - ) + AM(z,>;)}x() dt - AD(z)d

(14b)

On the other hand, Eq. (11) requires that

at + V or=

fo1 a]l a(z,E)x(t) dt + foI M(z,E) dt

+ad+fol (15)

which becomes, following an integration by parts and substitution of


Eqs. (2), (6), (7), (11), and (13),ay ay art(z,t aM(z,Eat + V az = Jo at- + ` oz + V at + All1(z,E)

- B2 [Jo1 M(z,o) do][ J' df] } x(t) dt + JaDt

+ V aD

- B2 [J0' M(z,a) do] [Jo D(E) dt] + VM(z 0)} d (16)

Equations (14) and (16) represent the same quantity, and thus thecoefficients of x and d must be identical. We therefore obtain the twoequations

/I

+ v(a + a3fJ+ 2AM

B2 [ fo' M(z,o) do] [ fo' df] + C(t)S(z - E) = 0 (17)

OD + V aZ + AD - B2 [fo' M(z,v) do][Joy D(Z) dE]

+ VM(z,0) = 0 (18)

Any solution of Eq. (17) clearly satisfies the symmetry condition, Eq.(13). In the special case V = 0 Eq. (17) reduces with two integrationsto the usual Riccati equation for lumped systems, Eq. (10) of Sec. 8.2.For the industrially important case of 0 - oo Eqs. (17) and (18), as inthe lumped-parameter analogs, tend to solutions which are independentof time. We shall consider regulation only of the exit stream, so thatC(E) = C6(1 - E), and we let 0 - oo. The optimal control is computedfrom Eqs. (10) and (11) as

u(t) = fo' GFB(z)x(t,z) dz + GFF d(t) (19)

where the feedback (FB) and feedforward (FF) gains are written interms of the solutions of Eqs. (17) and (18) as

GFB(z) = -B f0' M(z,E) dE (20)

GFF = -B fO' D(t) dt (21)

Because of the term 5(1 - ) the steady-state solution to Eq. (17)is discontinuous at the values r = 1, t = 1. By using the method ofcharacteristics a solution valid everywhere except at these points can beobtained in the implicit form

M(z,E) _ - 1 fi-(F-:)s(E-z) e-(2A/V)(z-w)GFB(0)GFB(,7 - z + ) dj

+ ti e(2a(V)(1-t)5(t - z) (22)


Here the Heaviside step function is defined as

S(Z - z) = {1 >z (23)

Integration of Eq. (22) then leads to an etuation for the feedback gain

GFB(z)CB

e(2AIV)(1-0V

+ B j' GFB(,j)e-(2A1V)(t-0 do rL' GFB(t) dt (24)

Equation (24) is in a form which is amenable to iterative solution for theoptimal gain. Equation (18) in the steady state is simply a linear first-order equation which can be solved explicitly for the feedforward gain

GFF _ B Jo' dz e-(AI')( t) d Jfl e-(2A1v)E-,)GF 1(,?)G1B(n - t) do

V - B f ' dz 1',;e-(AIV)(:-e'GFB(t) dl;

(25)

This result can be easily generalized to include nonconstant dis-turbances which are random with known statistical properties, in whichcase the feedforward gain GFF depends upon.the statistics.of the dis-turbance. In particular, if d(t) is uncorrelated ("white") noise, GFF iszero and the optimal control is entirely feedback. The feedback sectionof the optimal control requires the value of x at every position z, which isclearly impossible. The practical design problem then becomes one oflocating a minimum number of sensing elements in order to obtain anadequate approximation to the integral in Eq. (19) for a wide range ofdisturbance states x(t,t).

11.10 OPTIMAL FEED DISTRIBUTION IN PARAMETRIC PUMPING

Several separation processes developed in recent years operate periodi-cally in order to obtain improved mass transfer. One.such process,developed by Wilhelm and his associates and termed parametric pump-ing, exploits the temperature dependence of adsorption; equilibria toachieve separation, as shown schematically in Fig. 11.13. A streamcontaining dissolved solute is alternately fed to the top and bottom ofthe column. When the feed is to the top of the column, which wedenote by the time interval 71, the stream is heated, while during feedto the bottom in the interval ra the stream is cooled. There may beintervals 'r2 and 74 in which no feed enters. A temperature gradient istherefore established in the column. During r1 cold product lean in thesolute is removed from the bottom, and during ra hot product rich in


Hot feed311

Valve openduring r,

Hot product (rich)

during r3

Feed

Cold feed old product (lean)911

Valve open during r,

during r3

Fig.11.13 Schematic of a parametric-pumping separationprocess.

solute is removed from the top. This physical process leads to an inter-esting variational problem when we seek the distribution of hot and coldfeed over a cycle of periodic operation which maximizes the separation.

Denoting normalized temperatures by T and concentrations by c,with subscripts f and s referring to fluid and solid phases, respectively,the mass and energy balances inside the column (0 < z < 1) are

82T, aT, aT, aT.az2 + u(t) a + at + OK = o (la)

aat' + -t (T, - T,) =0 (lb)

a2c, ac, ac, ac,

'' aZ2 + u(t) a + at + X at 0 (lc)

ar+a(c*-c.)=0

Here, c* is the equilibrium concentration of solute which, for the saltsolutions studied by Rice, is related to solution concentration and tem-perature by the empirical equation

C.

c* = 111)$.I G-0. 447.

88

u(t) is the fluid velocity, which is negative during TI (downflow, hot feed),positive during Ta (upflow, cold feed), and zero during 72 and 7 4 (periods


of no flow). The values used for the other physical parameters are

#K 1.10 K=1.38y=200 a=0.3,,n - 10'In order to simplify the model for this preliminary study two

approximations were made based on these values. First, y was takenas approaching infinity, implying instantaneous temperature equilibra-tion. In that case T. = T1 = T. Secondly, dispersion of heat and masswere neglected compared to convection and ' and 71 set equal to zero.This latter approximation changes the order of the equations and wasretained only during the development of the optimization conditions.All calculations were carried out including the dispersion terms. Thesimplified equations describing the process behavior are then

(1 + RK)OT = -u aT (3a)

act ac, ac, (3b)at az ata at- = X (C! - C*) (3c


r,, u < 0: T = 1 cf = 1 at z = 1 (4a)

ra, u > 0: T = 0 cj = 1 at z = 0 (4b)

All variables are periodic in time with period 0. An immediate conse-quence of this simplification is that Eq. (3a) can be formally solved toyield

T 1z - l 1 OK fo u(r) drl = const

Periodicity then requires

ou(r) dr = 0fe

(5)

(6)

The amount of dissolved solute obtained in each stream during onecycle is

Rich product stream: f u(t)c1(t,l) dtT,

(7a)

Lean produet Scream: - f u(t)cj(t,O) dt

Thus, the net separation can be expressed as

(7b)

(P = f I u(t)c;(t,1) (it + f u(l)cf(t,O) dt (8)


The total feed during one cycle is

V =0

ju(t)j dt = f u(t) dt - ff, u(t) dt (9)

The optimization problem is that of choosing u(t) such that (P is maxi-mized (-(P minimized) for a fixed value of V. To complete the specifi-cation of parameters we take 8 = 2, V = 2.

The linearization and construction of Green's functions proceeds inthe usual manner, though it is necessary here to show that terms result-ing from changes in the ranges of rl and r3 are of sicond order and thatEqs. (6) and (9) combine to require

f, au(t) dt =Ifs

3u(t) dt = 0 (10)

The Green's functions are then found to satisfy

(1 + OK) aatl + u az1 - ay, aT = 0

aat2 + u a22 + Xy3 = 0

ay3X73

aC + K 1372 = 0at aC. at

71:

r3:'

y1(t,0) = 0 72(t,0) = -171(t,1) = 0 y2(t,1) _ +1

(11a)

(11b)

(11c)

(12a)

(12b)

All three functions are periodic over 8. The direction of steep descent'for maximizing the separation (P is

1 aT acw(t) I y 4 1 + c/(t,0) - 10 (y1 az + 72 a J) dz in r1

r 1 aT aCf)2U(t) 743 + Cf(0) - f

oy1 az

+y2 az J dz in ra

(13)

Here w(t) > 0, and 741 and 743 are constants chosen to satisfy the restric-tions of Eq. (10) on au.

The initial choice for the feed distribution was. taken to be sinusoidal

u(t) = -0.5 sin t (14)

The periodic solution to the system model was obtained by an implicitfinite difference method using the single approximation T. = T1 = T(y oo) and with dispersion terms retained. The multiplier equationswere solved only approximately in order to save computational time, foran extremely small grid size appeared necessary for numerical stability.

DISTRIBUTED-PARAMETER SYSTEMS 3M

First Eqs. (llb) and (lle) were combined to give a single equation in -Y2,

and the following form was assumed for application of the Galerkinmethod introduced in Sec. 3.9:

72 = (2z - 1) + Clz(1 - z) cos t + C2[$(1 - z) + (1 - $)z] sin t+ C3z(1 - z) cos 2t + C4[$(1 - z) + (1 - $)z] sin 2t (15)

where $ = +I when u > 0, j6 = 0 when u < 0. Evaluating the con-stants in the manner described in Sec. 3.9 shows them to be

C, = - 5.54 C2 = 0.461C3 = 0.056 C4 = -0.046

indicating rapid convergence, and, in fact, the C3 and C4 terms wereneglected for computation. y3 was then found by analytically integratingEq. (llc), treating the equation as a linear first-order equation with vari-able coefficients. -y, was also estimated using the Galerkin method,leading to the solution

y, = 0.0245[z(1 - z) cos t + [0(1 - z) + (1 - Q)z} sin t} (16)

The function Su(t) was computed from Eq. (13) for a constantweighting function w(t) = k, and a function w(t) = k1sin t!. These areshown normalized with respect to k in Fig. 11.14 as a dotted and solidline, respectively. The dashed line in the figure is the function 0.5 u(t).Both weighting functions indicate the same essential features when the

0.2

0.5

sa 0k.0.1

-0.2

0V2

a*2

0

-0.5

-1.02

Fig. 11.14 Normalized correction to velocity. Solid line w(t) = klsint1; dotted linew(t) = k; dashed line 0.5u(t) _ -0.25 sin t.


starting velocity function a(t) is compared with the change indicatedby the optimization theory in the form of Su. Amplification of the flowis required in some regions for improved performance and reduction inothers. The value of the separation obtained for the initial velocity dis-tribution (k = 0) was 61 = 0.150. For sinusoidal weighting and valuesof k = 0.2, 0.4, and 0.8 the values of (Q were, respectively, 0.160, 0.170,and 0.188. For constant weighting the value of (P for k = 0.1 was 0.160.Further increase in k in either case would have required a new computerprogram to solve the 'state equations, which did not appear to be afruitful path in a preliminary study of this kind. The dashed sinusoidin Fig. 11.15 is the initial velocity and the solid line the value for sinus-oidal weighting and k = 0.8.

The trend suggested by the calculated function 6u(t) is clearlytowards regions of no flow followed by maximal flow. Such a flow pat-tern was estimated by taking the sign changes of du for sinusoidal weight-ing to define the onset and end of flow, leading to the rectangular wavefunction shown in Fig. 11.15. The separation is 0.234, an increase of36 percent over the starting sinusoidal separation.

No further study was undertaken because the approximate nature

0.8 r-

0.6

04

0.2

u (f) 0

-0.2

-0.4

-0.6

0I3w2

J2w

Fig. 11.15 Velocity functions. Dashed line u(t) = -0.5 sin t; solid line sinusoidalweighting with k = 0.8; rectangular wave final estimate of optimum.


of the model did not appear to justify the additional computational effort.The preliminary results obtained here do suggest a fruitful area of appli-cation-of optimization theory when physical principles of the type con-sidered here have been carried to the point of process implementation.

11.11 CONCLUDING REMARKS

This concludes our study of the use of Green's functions in the solutionof variational problems.. In addition to the examination of severalexamples which are of considerable practical interest in themselves, thepurpose of this chapter has been to demonstrate the general applicabilityof the procedures first introduced in Chaps. 6, 7, and 9. The variationalsolution of optimization problems inevitably reduces to a considerationof linearized forms of the state equations, whether they are ordinarydifferential, difference, integral, partial differential, difference-differential,differential-integral, or any other form. The rational treatment of theselinearized systems requires the use of Green's functions.

As we have attempted to indicate, in all situations the constructionof the Green's function follows in a logical fashion from the applicationof the timelike operator to the inner product of Green's function andstate function. It is then a straightforward matter to express the varia-tion in objective explicitly in terms of finite or infinitesimal variations inthe decisions, thus leading to both necessary conditions and computa-tional procedures. As long as this application of standard proceduresin the theory of linear systems is followed, there is no inherent difficultyin applying variational methods to classes of optimization problems notexamined in this book.


Sections 111 to 11.7: These sections follow quite closely

M. M. Denn: Intern. J. Contr., 4:167 (1966)

ThePartieular problem of temperature control has been considered in terms of a minimumprinciple for integral formulations by

A. G. Butkovskii: Proc. sd Intern. Congr. IFAC, paper' 513 (1963)

and by classical variational methods by

Y. Sakawa: IEEE Trans. Aulom. Cont., AC9:420 (1964); AC11:35 (1966)

Modal analysis, approximation by a lumped system, and a procedure for approximatingthe solution curves are used by

1. McCausland: J. Electron. Contr., 14:635 (1963)Proc. Inst. Elec. Engrs. (London), 112:543 (1965)

Sit OPTIMIZATION BY VARIATIONAL METHODS

Section 11.8: The essential results are obtained in the paper

M. M. Denn, R. D. Gray, Jr., and J. It. Ferron: Ind. Eng. Chem. Fundamentals,6:59 (1966)

Details of the construction of the model and numerical analysis are in

R. D. Gray, Jr.: Two-dimensional Effects in Optimal Tubular Reactor Design, Ph.D.thesis, University of Delaware, Newark, 1)el., 1965

The usefulness of an isothermal coolant and the effect of design parameters are investigated in

A. R. Hoge: Some Aspects of the Optimization of a Two-dimensional Tubular Reactor,B.S. thesis, University of Delaware, Newark, Del., 1965

J. D. Robinson: A Parametric Study of an Optimal Two-dimensional Tubular ReactorDesign, M.Ch.E. thesis, University of Delaware, Newark, Del., I966

Section 11.9: This section is based on,

M. M. Denn: Ind. Eng. Chem. Fundamentals, 7:410 (1968)

where somewhat more general results are obtained, including control at the boundary z = 0.An equation for the feedback gain corresponding to Eq. (17) for more general spatialdifferential operators was first presented by Wang in his review paper

P. K. C. Wang: in C. T. Leondes (ed.), "Advances in Control Systems," vol. 1,Academic Press, Inc., New York, 1964

A procedure for obtaining the feedback law using modal analysis is given by

D. M. Wiberg: J. Basic Eng., 89D:379 (1967)

Wiberg's results require a discrete spectrum of eigenvalues and are not applicable tohyperbolic systems of the type studied here. An approach similar to that used hereleading to an approximate but easily calculated form for the feedback gain is in

L. B. Koppel, Y. P. Shih, and D. R. Coughanowr: Ind. Eng. Chem. Fundamentals,7:296 (1968)

Section 11.10: This section follows a paper with A. K. Wagle presented at the 1989Joint Automatic Control Conference and published in the preprints of the meeting.More details are contained in

A. K. Wagle: Optimal Periodic Separation Processes, M.Ch.E. Thesis, University ofDelaware, Newark, Del., 1969

Separation by parametric pumping was developed by Wilhelm and coworkirs and reportedin

R. H. Wilhelm, A. W. Rice, and A. R. Bendelius: Ind. Eng. Chem. Fundamentals,5:141 (1966)

R. W. Rolke, and N. H. Sweed: Ind. Eng. Chem. Fundamentals,7:337 (1968)

The basic equations are developed and compared with experiment in the thesis of Rice,from which the parameters used here were taken:


A. W. Rice: Some Aspects of Separation by Parametric Pumping, Ph.D. thesis,Princeton University, Princeton, N.J., 1966

Section 11.11: Variational methods for distributed-parameter systems have receivedattention in recent years. The paper by Wang noted above contains an extensivebibliography through 1964, although it omits a paper fundamental to all that we havedone, namely,

S. Katz: J. Electron. Contr., 16:189 (1964)

More current bibliographies are contained in

E. B. Lee and. L. Markus: "Foundations of Optimal Control Theory," John Wiley &Sons, Inc., New York, 1967

P. K. C. Wang: Intern. J. Contr., 7:101 (1968)

An excellent survey of Soviet publications in this area is

A. G. Butkovsky, A. I. Egorov, and K. A. Lurie: SIAM J. Contr., 6:437 (1968)

Many pertinent recent papers can be found in the SIAM Journal on Control; IEEETransactions on Automatic Control; Automation and Remote Control; and Auto-matica; in the annual preprints of the Joint Automatic Control Conference; and in theUniversity of Southern California conference proceedings:

A. V. Balakrishnan and L. W. Neustadt (eds.): "Mathematical Theory of Control,"Academic Press, Inc., New York, 1967

The following are of particular interest in process applications:

R. Jackson: Proc. Inst. Chem. Engrs. AIChE Joint Meeting, 4:32 (1965)Intern. J. Contr., 4:127, 585 (1966)Trans. Inst. Chem. Engrs. (London), 46:T160 (1967)

K. A. Lurie: in G. Leitmann (ed.), "Topics in Optimization," Academic Press, Inc.,New York, 1967

A minimum principle fyir systems described by integral equations is outlined in the paperby Butkovskii li4fd above. Results for difference-differential equations, togetherwith further refeonces, are in

D. H. Chyung: Pr: prints 1967 Joint Autom. Contr. Conf., p. 470,M. M. Denn and R. Aris: Ind. Eng. Chem. Fundamentals, 4:213 (1965)M. N. Ogflztoreli: "Time-lag Control Systems," Academic Press, Inc., New York, 1966H. R. Sebesta and L. G. Clark: Preprints 1967 Joint Autom. Contr. Conf., p. 326

Results due to. Kharatishvili can be found in the book by Pontryagin and coworkers andin the collection above edited by Balakrishnan and Neustadt.

PROBLEMS

11.1. Obtain the minimum-principle and steep-descent direction used in See. 11.8.11.2. Obtain necessary conditions and the direction of steep descent for the problem inSec. 11.10. What changes result when the objective is maximization of the separation


factor,

I u(t)c1(t,l) dt

f u(i)cj(t,0) dt

11.3. Optimal control of a heat exchanger to a new set point by adjustment of walltemperature is approximated by the equation

ax ax

at + az= P(u - x)

x(0,t) = 0

Obtain the optimal feedback gain as a function of P and C for the objective

[Cx2(l,t) + u2(t)1 dtE - 2 Iu-

Compare with the approximate solution of Koppel, Shih, and Coughanowr,

GFB(z) _ { -C exp (- (2P + PK)(1 - z)] 0 < z < 10 z=1

where K is the solution of a transcendental equation

K =2±K[1 -exp(-2P-PK)]11.4. The reversible exothermic reaction X = Y in a tubular catalytic reactor isapproximated by the equations

ax,x:rkoe [(1 - yo - (yo + x01 0 < z < 1

az

ax, -«ux:at

0<r<1

where x, is the extent of reaction, x2 the catalyst efficiency, u the temperature, r thereactor residence time, and yo the fraction of reaction product in the feed. Boundaryand initial conditions are

x, =0 atz =0forall tx==1 att -Oforallz

The temperature profile u(z) is to be chosen at each time t to maximize averageconversion

tP =fo,

dt

Obtain all the relevant equations for solution. Carry out a numerical solution ifnecessary. Parameters are

rko3X10' Ko-2.3X10-6 yo=0.06E_ = 10,000 E; - 5,000 « = 4 X 10'6

(Th(s problem is due to Jackson.)


11.5. Discuss the minimum time control of the function x(z,t) to zero by adjustingsurroundings temperature u(t) in the following radiation problem:

ax a'x 0<z<1at - aZ2 0 < t < ax=0 att=0forall zax-=0 atz=OforalltaZ

ax =k[u4(t) -x'] atz = 1for all taz

(The problem is due to Uzgiris apd D'Souza.)11.6. Obtain necessary conditions and the direction of steep descent for a systemdescribed by the difference-differential equations

ii(t) =l,fx(t),:(t - r), u(t), u(t - r)]x,(t) = x,o(t) -r < t < 0min E = fo 5[x(t),u(t)] dt

12

Dynamic Programming andHamilton-Jacobi Theory

12.1 INTRODUCTION

The major part of this book has been concerned with the application ofclassical variational methods to sequential decision making, where thesequence was either continuous in a timelike variable or, as in Chap. 7,over discrete stages. Simultaneously with the refinement of these meth-ods over the past two decades an alternative approach to such problemshas been developed and studied, primarily by Bellman and his coworkers.Known as dynamic programming, this approach has strong similarities tovariational methods and, indeed, for the types of problems we havestudied often leads to the same set of equations for ultimate solution.In this brief introduction to dynamic programming we shall first examinethe computational aspects which differ from those previously developedand then demonstrate the essential equivalence of the two approaches tosequential optimization for many problems.

DYNAMIC PROGRAMMING AND HAMILTON-JACOBI THEORY 393

1L2 THE PRINCIPLE OF OPTIMALITY AND COMPUTATION

For illustrative purposes let us consider the process shown schematicallyin Fig. 12.1. The sequence of decisions u1, u2i . . . , uN is to be madeto minimize some function 8(xN),- where there is a known input-outputrelation at each stage, say

x" = f^(x°-',un) n = 1, 2, . . . , N (1)

The dynamic programming approach is based on the principle of opti-mality, as formulated by Bellman:

An optimal policy has the property that whatever the initial state andinitial decision are, the remaining decisions must constitute. an opti-mal policy with regard to the state resulting from the first decision.

That is, having chosen u' apd thug determined x1, the remaining deci-sions u2, u', .. , uN must be chosen so that C(xN) is minimized for thatparticular x'. Similarly, having chosen u', u2, . . . , uN-1 and thusdetermined xN-1, the remaining decision uN must be chosen so that&(xN) is a minimum for that xN-1. The proof of the principle of opti-mality is clear by contradiction, for if the choice u' happened to be theoptimal first choice and the remaining decisions were not optimal withrespect to that x1, we could always make &(xN) smaller by choosing anew set of remaining decisions.

The principle of optimality leads immediately to an interestingcomputational algorithm. If we suppose that we have somehow deter-mined xN-1, the choice of the remaining decision : simply involves thesearch over all values of.uN to minimize C(xN) or, substituting Eq. (1),

min E[fN(xN-1,uN)1UN

(2)

Since we do not know what the proper value of xN`1 is, however, we mustdo this for all values of xN-' or, more realistically, for a representativeselection of values. We can then tabulate for, each XN-1 the minimizingvalue of uN and the corresponding minimum value of E.

We now move back one stage and suppose that we have available

Fig. 12:1 Schematic of a sequential decision process.


xN-2, for which we must find the decisions uN-1 and uN that minimizeS(XN). A specification of uN-1 will determine xN-1, and for any givenxN-1 we already have tabulated the optimal uN and value of C. Thus,we need simply search over UN-1 to find the tabulatedt xN-1 that resultsin the minimum value of S. That is, we search over uN-1 only, and notsimultaneously over UN-1 and uN. The dimensionality of the problem isthus reduced, but again we must carry out this procedure and tabulatethe results for all (representative) values of xN-2, since we do not. knowwhich value is the optimal one.

We now repeat the process for xN-1, choosing uN-2 by means of thetable for xN-2, etc., until finally we reach x°. Since x° is known, we canthen choose u' by means of the table ford This gives the optimalvalue for x', so that u2 can then be found from-Ug table for x2, etc.In this way the optimal sequence u1, U2, . . .. , UN is cdtp d for agiven x° by means of a sequence of minimizations over a single variable.Note that we have made no assumptions concerning differentiability ofthe functions or bounds on the decisions. Thus, this algorithm can beused when those outlined in Chap. 9 might be difficult or inapplicable.We shall comment further on the computational efficiency following anexample.

12.3 OPTIMAL TEMPERATURE SEQUENCES

As an example of the computational procedure we shall again considerthe optimal temperature sequence for consecutive chemical reactionsintroduced in Secs. 1.12 and 7.6 and studied computationally in Chaps.2 and 9. Taking the functions F and G as linear for simplicity, the stateequations have the form

xn-1xn = (1a)

1 + 0k1oe-E''/u*yn-1 + k1oe-E-'/U,(yn-1 + xn-')

n = (1b)y (1 + 8kjoeE,"/°")(1 + 02oe-E:'I"")

where we wish to choose u', u2, . . . , uN in order to minimize

fi= - yN - pxN (2)

We shall use the values of the parameters as follows:

k1o = 5.4 X 1010 k20 = 4.6 X 1011E1 = 9 X 10' E_ = 15 X 10'

0=5 p0.3x°=1 y°=0

t Clearly some interpolation among the tabulated representative values of zN-' willbe required.

DYNAMIC PROGRAMMING AND THEORY

Table 12.1 Optimal decisions and values of the objective atstage N for various Inputs resulting from stage N - 1

3"

XN-1

1/N-1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

340 340 340 340 340 340 340 340 340 3400

0.056 0.112 0.168 0.224 0.281 0.337 0.393 0.449 0.505 0.561

330 334 338 340 340 340 340 340 340 340 3400.1 0.096 0.145 0.199 0.255 0.310 0.367 0.423 0.479 0.535 0.591 0.647

330 330 334 336 338 338 340 340 340 340 3400.2

0.192 0.240 0.291 0.344 0.399 0.454 0.509 0.565 0.621 0.677 0.733

330 330 332 334 336 338 338 338 338 340 3400.3 0.288 0.336 0.385 0.436 0.489 0.543 0.598 0.654 0.709 0.764 0.820

330 330 330 332 334 336 336 338 338 338 3380.4 0.384 0.432 0.481 0.530 0.582 .0.634 0.688 0.742 0.798 0.853 0.908

/ 330 330 330 330 332 334 336 336 336 338 3380.5 0.480 0.528 0.577 0.625 0.675 0.727 0.779 0.833 0.887 0.942 0.997

330 330 330 330 332 334 334 334 336 336 3380.6

0.576 0.624 0.673 0.721 0.770 0.820 0.872 0.925 0.978 1.032 1.086

330 330 330 330 330 332 332 334 334 336 3360.7 0.672 0.720 0.769 0.817 0.866 0.913 0.966 1.018 1.070 1.123 1.177

330 330 330 330 330 330 332 334 334 334 3360.8 0.768 0.816 0.865 0.913 0.962 1.010 1.060 1.111 1.163 1.215 1.268

330 330 330 330 330 330 332 332 332 334 3340.9 0.864 0.912 0.961 1.009 1.058 1.106 1.155 1.205 1.25. .309 1.361

330 330 330 330 330 330 330 330 332 3341E1.0 0.960 1.008 1.057 1.105 1.154 1.202 1.250 1.329 1.354 1.11 1.44

Furthermore, we shall restrict u° by the constraints

330 < u^ < 340 (3)

and, to demonstrate a situation where techniques based on the calculusare not applicable, we shall further restrict each u° to values which areeven integers. That is, we shall restrict the choice to the values 330,332, 334, 336, 338, and 340.

We begin by considering the last (Nth) stage. For each value ofxN-1, yN-1 we can compute xN and yN by means of Eqs. (1) for the sixpossible choices of UN. The particular value of uN which minimizes S'and the corresponding value of -g is recorded in Table 12.1, where it is

3% OPTIMIZATION BY VARIATIONAL METHODS

assumed that increments of 0.1 in xN-' and yN`' are sufficient for purposesof interpolation.

Next we consider the penultimate [(N - 1)stj stage. For example,for xN-2 = 0.7, yN-2 = 0.2, we obtain:

uN-1 xN-1 yN-1 -E

330 0.504 0.380 0.617332 0.481 0.397 0.621334 0.455 0.415 0.626336 0.429 0.430 0.625338 0.403 0.443 0.624

340 0.375 0.452 0.617

where the values of -S are obtained by linear interpolation in Table 12.1.Thus, for this pair of values xN-2, yv-2 the optima, UN-' is 334 with acorresponding value of S of -0.626. In this way Table 12.2 is con-structed for all values of xN-2, yN-2, where, for simplicity, the range hasnow been reduced.

In a similar manner we move on to the second from last stage andconsider values of x'''-3, yN-s, now finding the optimum.by interpolationin Table 12.2. The only entry we note here is xN-3 = 1, yN-3 = 0, withUN-2 = 336 and S = -0.706. When we have finally reached the firststage, we can reconstruct the optimal policy. For example, if N = 1,we find from Table 12.1 that for x° = 1, y° = 0 the optimum is -0.56with u' = 340. For N = 2t we begin with Table 12.2, where we findfor x° = 1, y° = 0 the optimum is -0.662 and u' = 340. Equations (1)then indicate that x' = 0.536, y' = 0.400, and Table 12.1 for the last(second) stage indicates the optimal decision u2 = 336. For N = 3 oursole tabular entry recorded above indicates u' = 336 and S = -0.71.Then, from Eqs. (1), x' = 0.613 and y' = 0.354, and Table 12.2 indicatesthat u2 = 334. Applying Eqs. (1) again, x2 = 0.395, y2 = 0.530, andfrom Table 12.1 we find u3 = 332. Finally, then, x3 = 0.274, y3 = 0.624.This process can be continued for N as large as we wish.

We can now evaluate the strong and weak points of this computa-tional procedure. The existence of constraints increases efficiency, asdoes the restriction to discrete values of the decisions. The former is

t For this case (No = 10) we found in Chap. 2 that u' = 338, u2 - 336.5, andl; - -0.655 when there were no restrictions on u', u2. Since the restricted optimumcannot be better than the unrestricted, we see some small error resulting from theinterpolation.

DYNAMIC PROGRAMMING AND HAMILTON-JACOBI THEORY

Table 12.2 Optimal decisions at stage N - 1and optimal values of the objective forvarious Inputs resulting from stage N - 2

3!7

ZN-4

0.5 0.6 0.7 0.8 0.9 1.0

336 338 338 338 338 3400

0.332 0.404 0.460 0.524 0.590 0.662

334 336 338 336 336 3380.1

0.414 0.475 0.541 0.605 0.672 0.736

0.2334 336 336 334 36 3340.509 0.559 0.626 0.696 0.757 0.818

0.3334 334 336 334 " 336 3340.555 0.648 0.713 0.776 0.840 0.905

334 334 336 334 334 3340.4

0.649 0.735 0.799 0.863 0.926 0.992

0.5330 332 334 334 334 334

0.743 0.826 0.887 0.951 1.014 1.078

not a major limitation on procedures derivable from the calculus, but thelatter effectively eliminates their use. Thus, if such a restriction isphysically meaningful, an algorithm such as this one is essential, whileif the restriction is used simply as a means of obtaining first estimates byuse of a coarse grid, we might expect to obtain first estimates as easilyusing steep descent as outlined in Chap. 9. In reducing the N-dimen-sional search problem to a sequence of one-dimensional searches we havetraded a single difficult problem for a large number of simpler ones, andindeed, we have simultaneously solved not only the problem of interestbut related problems for a range of initial values. If we are truly inter-ested in a range of initial values, this is quite useful, but if we care onlyabout a single initial condition, the additional information is of little useand seemingly wasteful. Dynamic programming does not. eliminatethe ty o-point boundary-value problem which has occupied us to such agreat extent, but rather solves it by considering a complete range of finalvalues.

The fact that the optimum is automatically computed as a feedbackpolicy, depending only upon the present state and number of remainingdecisions, suggests utility in control applications. The storage prob-lem, however, is a serious restriction, for with two state variables we havebeen able to tabulate data in a two-dimensional array, but three variables

Us OPTIMIZATION BY VARIATIONAL METHODS

would require a three-dimensional array, four variables four dimensions,etc. Fast memory capabilities in present computers effectively limitdirect storage to systems with three state variables, and approximationtechniques must be used for higher-dimensional problems. Thus, thealgorithm resulting from direct application of the principle of optimalitywill be of greatest use in systems with many stages but few state variables,particularly when variables are constrained or limited to discrete values.

12.4 THE HAM I LTON-JACOBI-BELLMAN EQUATION

For analytical purposes it is helpful to develop the mathematical formal-ism which describes the tabulation procedure used in the previous section.At the last stage the minimum value of g(xN) depends only upon thevalue of zN-1. Thus, we define a function SN of 1N-1 as

SN(zN-1) min E[fN(xN_i,uN)J (1)UM

which is simply a symbolic representation of the construction of Table12.1 in the example. Similarly with two stages to go we can define

SN`I(XN-2) = min min C(xN) (2)UN-1 UN

or, using Eq. (1),

SN-1(EN-2) = mm SN[fN-1(XN_2,uN-1)]UN-1

(3)

which is the symbolic representation of the construction of Table 12.2.In general, then, we obtain the recursive relation

8"(z"-1) = min S"+'[f"(z"-',u")J (4)U.

Consistent with this definition we can define -

S'N+1(XN) = g(x") (5)

Equation (4) is a difference equation for the function S",. with theboundary condition Eq. (5). It is somewhat distinct from the typeof difference equation with which we normally deal, for it contains aminimization operation, but we can conceive of first carrying 'out theminimization and then using the minimizing value of u" to convertEq. (4) to a classical difference equation in the variable n, with S" depend-ing explicitly upon x"-'. By its definition the optimum must satisfyEq. (4), and furthermore, when we have found the functions S", we have,by definition, found the optimum. Thus, solution of the difference equa-tion (4) with boundary-condition equation (5) is a necessary and suffi-cient condition for an optimum. We shall call Eq. (4) a Hamilton-Jacobi-

DYNAMIC PROGRAMMING AND HAMILTON-JACOBI THEORY 3!!

Bellman equation, for it is closely related to the Hamilton-Jacobi equa-tion of mechanics.

If we assume that the functions S" are differentiable with respect tox»-I and, further, that fn is differentiable with respect to both x"-I and u",

we can relate the Hamilton-Jacobi theory to the results of Chap. 7. Forsimplicity we shall temporarily presume that u" is unconstrained. Then,since S"+' depends explicitly upon xn, the. minimum of the right-hand sideof Eq. (4) occurs at the solution of

asn+l 0 (6)

au" ax" au"

The value of u" satisfying this equation, un, is, of course, a function of theparticular value of x"-', and Eq. (4) can now be written as

Sn(xn-1) = Sn+l[f"(x"-l,u")1 (7)

Equation (7) is an identity, valid for all x"-'. The partial deriva-tives in Eq. (6) can then be evaluated by differentiating both sides ofEq. (7) with respect to each component of x"-', giving

aS" aSn+Ia fn asn+lOf

ax;"-1 - [, Z { ax;"au"au"

ax;. 8x;"-1

n

) ax;"-'

where the second term is required because of the dependence of u" onx"-1. From Eq. (6), however, the quantity in parenthesis vanishes, and

as" v a,3"+1 af;" (9)ax," ax"-'

From Eq. (5), if there are no relations among the components of z' ,

aSN+1 ar, (10)

ax;N ax/'

Now, it is convenient to define a vector y" as

aS"+1y," =ax ^

(11)

that is, as the rate of change of the optimum with respect to the stateat the nth stage. Then Eqs. (9) and (10) may be written

,y,n-I Yin af;" (12)(( ax'n-1

aS?iN o (13)

axiN


and Eq. (6) for the optimum u^

y"af,"=0aul (14)

These are the equations for the weak minimum principle derived inSec. 7.4, and they relate the Green's functions to the sensitivity interpre-tation of the Lagrange multiplier in Sec. 1.15.

Even if u" is bounded, the minimization indicated in Eq. (4) canbe carried out and the minimizing value an found as a function of z"-'.Equation (8) is still valid, and if the optimum occurs at an interior value,the quantity in parentheses will vanish. If, on the other hand, theoptimal u" lies at a constraint, small variations in the stage input willnot cause any change and au"/ax;"'' will be zero. Thus, Eqs. (9) and(10), or equivalently (12) and (13), are unchanged by constraints.The minimum in Eq. (4) at a bound is defined by

()S"+' aS"+' af;^ > 0 u" at lower boundau"

_- L

ti

ax,." au" < 0 u" at upper bound(15a)

or

aS^+1 f ^ "f" =min u^ at bound (15b);ax; y;

Thus we have the complete weak minimum principle. We must reiter-ate, however, that while the Hamilton-Jacobi-Bellman equation is bothnecessary and sufficient, the operations following Eq. (5) merely defineconditions that the solution of Eq. (4) must satisfy and are therefore onlynecessary.

12.5 A SOLUTION OF THE HAMILTON-JACOBI-BELLMAN EQUATION

It is sometimes possible to construct a solution to the Hamilton-Jacobi-Bellman equation directly, thus obtaining a result known to be sufficientfor an optimum. We can demonstrate. this procedure for the case of asystem described by the linear separable equations

x;" _ A;i"x,"-' + b;"(u") (1)

where we wish to minimize a linear function of the final state

(2)

The Hamilton-Jacobi-Bellman equation, Eqs. (4) and (5) of the


preceding section, is

Sn(x"-1) = min S"+I[f"(x"-Ir"u")] (3)U.

SN+1(xN) _ & _ CjxjN (4)IThe linearity of f" and 8 suggest a linear solution

Sn(x"-1) = I yn-1xj"-1 + t"(5)

where _y n-1 and t" are to be determined. Substituting into Eq. (3), then,

yjn--1xj"-1 + c" = min 11y,nAj

nxjn-1 + yinbi"(u") + V +I] (6)..II LL....ll

.JJ

J U. i,j i

or, since only one term on the right depends upon u",

(yjn-1 - yinA,jn) xi"-I = min [G yi"bin(u")J + n+t - t n (7)j i

ui

The left-hand side depends upon x"-1 and the right does not, so that thesolution must satisfy the difference equation

yj"-I yinA ,n (8)

with the optimal u" chosen by

min ytinbi"(u") (9)U.

The variable Z" is computed from the recursive relation

Zn = n+1 + min yi"bin(u")u

i

Comparison of Eqs. (4) and (5) provides the boundary conditions

yiN = citN+1 = 0

(10)

(11)

(12)

Equations (8), (9), and (11) are, of course, the strong. form of theminimum principle, which we established in Sec. 7.8 as both necessaryand sufficient for the linear separable system with linear objective. Wehave not required here the differentiability of b" assumed in the earlierproof. This special situation is of interest in that the optimal policy iscompletely independent of the state as a consequence of the uncouplingof multiplier and state equations, so that Eqs. (8), (9), and (11) may besolved once and for all for any value of x" and an optimum defined onlyin terms of the number of stages remaining.


12.6 THE CONTINUOUS HAMILTON-JACOBI-BELLMAN EQUATION

The analytical aspects of dynamic programming and the resultingHamilton-Jacobi theory for continuous processes described by ordinarydifferential equations are most easily obtained by applying a carefullimiting process to the definitions of Sec. 12.4. We shall take the sys-tem to be described by the equations

ti = fi(x,u) 0 < t < 6 (1)

and the objective as 8[x(6)]. If we divide 0 into N increments of lengthAt and denote x(n At) as xn and u(n At) as un, then a first-order approxi-mation to Eq. (1) is

zi" = zin-1 + Mx"-``',u") At + o(At) (2)

Then Eq. (4) of Sec. 124, the discrete Hamilton-Jacobi-Bellman equa-tion, becomes

S"(x") = min Sn+1[x"-' + f(x"-',u") At + o(At)] (3)

U.

or, writing S as an explicit function of the time variable n At

S"(x"-1) = S(x"-1, n At)

we have

S(x"-1, n At) = min S[x"-' + f(x"-',u") At + o(At), n AL + Qt] (4)u+

For sufficiently small At we can expand S, in the right-hand side ofEq. (4) about its-4alue at xn-1, n At as follows:

S[x"-' + f At + o(At), n At + At] = S(x"-1, n At)

+ ax"

1fi At + a(n At)

At + o(At) (5)

or

S(x"-', n At) = min [S(xut_1, n At) + azas

, fi At4. I

+ a(n At)-Al + o(At)] (6)

Since S(x"-1, n At) does not'depend upon un, it is not involved in theminimization and can be canceled between the two sides of the equation.Thus,

0 = minax

s, fi At +

a (n At)At + 0 (At) ] (7)

As At gets very small, the distinction between x"-' and x", u"-' and u"gets correspondingly small and both x"-1 and x" approach x(t) and simi-


larly for u^. Again letting t = n At, dividing by At, and taking the limitas At -. 0, we obtain the Hamilton-Jacobi-Bellman partial differentialequation for the optimum objective S(x,t)

0 = min aSfi(x,u) + asl(8)U(t) LL. ax, at J

with boundary condition from Eq. (5) of Sec. 12.4

S[x(B),B] = 8[0)J (9)

The partial differential equation can be written in classical form as

as fr(x,u) + at = 0 (10)

where u is an explicit function of x obtained from the operation

m.nas

f;(x,u) (11)tiax;When u is unconstrained or u lies inside the bounds, this minimum isfound from the solution of the equation

as af; 0(12)T, au

_-

Equations (9) to (11) form the basis of a means of computation ofthe optimum, and in the next section we shall demonstrate a direct solu-tion of the Hamilton-Jacobi-Bellman equation. Here we shall performsome manipulations analogous to those in Sec. 12.4 to show how theminimum principle can be deduced by means of dynamic programming.The reader familiar with the properties of hyperbolic partial differentialequations will recognize that we are following a rather circuitous path toarrive at the characteristic ordinary differential equations for Eq. (10).

First, we take the partial derivative of Eq. (10) with respect to thejth component z; of x. Since S depends explicitly only upon x and t andu depends explicitly upon x through Eq. (11) we obtain

j` 2 f as af; au a2sLax ax;f`+L, axa;+LOx,Ouaz;+ax;at=0 (13)

The third term vanishes, because of Eq. (12) if u is interior or because asmall change in x cannot move u from a constraint when at a bound,in which case su/ax; = 0. Thus,

a2S as af; a2Saxax;f`+Gaxax;;+ax;at = o (14)


Next, we take the total time derivative of aS/ax;

d as a2S a2Sat ax; ax; ax; f` + at ax; (15)

Assuming that the order of taking partial derivatives can be interchanged,we substitute Eq. (15) into Eq. (14) to obtain

dasdt ax;

or, defining

aSafiax; ax; (16)

asys

__

Eq. (16) m

axi

ay be written

afi

(17)

Yi - (18)

while Eq. ( 11) bec

ax

omes

min Yifi (19)ti

If 0 is not specified, it is readily established that S is independent of t andEq. (10) is

7ifi(a,u) = 0 (20)

Equations (18) to (20) are, of course, the strong minimum principle forthis system, and Eq. (17) establishes the Green's function y as a sensi-tivity variable.

It is apparent from theseresults and those of Sec. 12.4 that if weso wished, we could derive the results of the preceding chapters from adynamic programming point of view. We have not followed this coursefor several reasons. First, the variational approach which we have gen-erally adopted is closer to the usual engineering experience of successiveestimation of solutions. Second, computational procedures are morenaturally and easily obtained within the same mathematical frameworkby variational methods. Finally (and though much emphasized in theliterature, of lesser importance), the dynamic programming derivationof the minimum principle is not internally consistent, for it must beassumed in the derivation that the partial derivatives aS/axi are con-tinuous in x when in fact the solution then obtained for even the ele-mentary minimum-time problem of Sec. 5.3 has derivatives which arediscontinuous at the switching surface.

DYNAMIC PROGRAMMING AND THEORY 405

12.7 THE LINEAR REGULATOR PROBLEM

To complete this brief sketch of the dynamic programming approach toprocess optimization it is helpful to examine a direct solution to the con-tinuous Hamilton-Jacobi-Bellman equation. This can be demonstratedsimply for the linear regulator problem, a special case of the control prob-lem studied in Sec. 8.2. The system is described by the linear equations

A;ixi + b,u i = 1, 2, . . . , si-1

with the quadratic objective

8 f r.

2f (, x1C;ixi + u2) dt

1J-1or, defining

1 1i.+1 =

2 x;C;ixi + u2 x,+1(0) = 0

we have

C[a(e)) = x,+1(e)

The control variable u is assumed to be unconstrained.The Hamilton-Jacobi-Bellman partial differential equation is

0 = min +1 asl;+

aslu axi at)i-1

or, from Eqs. (1) and (3),

(1)

(2)

(3)

(4)

(5)

' as as 1 as.+

x;C,ixi0 = min A;ixt + L, ax b'u + 2 axY i,7-1axi i-1 ; 1

1 as2 Ls)

+ 2 ax__ u + at(6)

From the minimization we obtain

_ ( as )-1; ' asu =11

biax,+1 axi

(7)

so that, upon substitution, Eq. (6) becomes a classical but nonlinearpartial differential equation

as 1 ' asb,

asb,

axi A`ixi -i2 Gas as 1) \ Ti \ axi /id-1 i-1 i-1

+1 as asxicoxi + at = 0 (8)

id-1

IOi

with boundary condition

S[x(8),8] = &[x(8)] = x,+1(8)


(9)

The function S(x,t) satisfying Eqs. (8) and (9) will be of the form

8(x'0 = x,+1 + i71G

i4L"- I

in which case

M;;(0) = 0 (10)

C'xk 2 \ x,Mik + Mt;x;) k = 1, 2, , s (11)i-1 ,-1


1

2 xix1(MikAkj + AkiMkf)i,,.k-I

2 + x;x; (Mik + Mki)bk] 12 b1(Mii + M11)]+.i. - I L

+ x;C;;x; + x;lll;;x; = 0 (12)

or, since it must hold for all x,

I (MikAk, + Ak;Mk,) - I [2 (Mik + Mk;)bk] [2 b1(Ml, + M;1)]

+ C;; + M;; = 0 M;;(8) = 0 (13)

The solution to Eq. (13) is clearly symmetric, Mij = M;i, so that we maywrite it finally in the form of the Riccati equation (10) of Sec. 8.2

Mi, + I (MikAk, + Mk;Aki) - Q Mikbk) ( btM1j)k k I

+ Ci, = 0 Mi;(8) = 0 (14)

The optimal feedback control is then found directly from Eqs. (7) and(10) as

,

u = - I b;M,,x,i.,-1

(15)

Since this result was obtained in the form of a solution to the Hamilton-Jacobi-Bellman equation, we know that it is sufficient for a minimum,a result also established by other means in Sec. 6.20.

DYNAMIC PROGRAMMING AND THEORY 407


We shall not attempt a survey of the extensive periodical literature on dynamic program-ming but content ourselves with citing several texts. Excellent introductions may befound in

R. Aris: "Discrete Dynamic Programming," Blaisdell Publishing Ccmpany, Wal-tham, Mass., 1964

R. E. Bellman and S. E. Dreyfus: "Applied Dynamic Programming," Princeton Uni-versity Press, Princeton, N.J., 1962


G. L. Nemhauser: "Introduction to Dynamic Programming," John Wiley & Sons,Inc., New York, 1966

Problems of the type considered in Sec. 12.6 are treated in

S. E. Dreyfus: "Dynamic Programming and the Calculus of Variations," AcademicPress, Inc., New York, 1965

and numerous topics of fundamental interest are treated in the original book on the subject:

R. Bellman: "Dynamic Programming," Princeton University Press, Princeton, N.J.,1957

Extensive applications are examined in

R. Aris: "The Optimal Design of Chemical Reactors: A Study in Dynamic Program-ming," Academic Press, Inc., New York, 1961

R. Bellman: "Adaptive Control Processes: A Guided Tour," Princeton UniversityPress, Princeton, N.J., 1961

S. M. Roberts: "Dynamic Programming in Chemical Engineering and Process Con-trol," Academic Press, Inc., New York, 1964

D. F. Rudd and C. C. Watson: "Strategy of Process Engineering," John Wiley &Sons Inc., New York, 1968

J. Tou: "Optimum Design of Digital Control via Dynamic Programming," AcademicPress, Inc., New York, 1963

PROBLEMS

12.1. Develop the Hamilton-Jacobi-Bellman equation for the system

z" = f^(z"-' u")N

min s = E 61"(x"-',u") + F(1N)n-1

Apply this directly to the linear one-dimensional case

x" = A(u^)x"-' + 14(A(u") - 11N

t = a(xN - x°) + E M(u")n-1

and establish that the optimal decision u" is identical at each stage.


12.2. Formulate Prob. 7.2 for direct solution by dynamic programming. Draw acomplete logical flow diagram and if a computer is available, solve and comparethe effort to previous methods used. Suppose holding times u," are restricted tocertain discrete values?12.3. The system

x=u lul <1

is to be taken from initial conditions x(O) = 1, i(0) = 1 to the origin to minimize

E -0

5(x,i,u) dt

Show how the optimum can be computed by direct application of the dynamic pro-gramming approach for discrete systems. (Hint: Write

ij - x: x1(t - A) = xi(t) - x3(t)Ais = u x:(t - A) = X2(t) - u(t)A

and obtain the Hamilton-Jacobi-Bellman difference equation for recursive calculation.)Assume that u can take on only nine evenly spaced values and obtain the solution for

(a)5(b) if = %(xs= + u=)(c) if - 3Vx' + V)

Compare the value of the objective with the exact solution for continuously variableu(t) obtained with the minimum principle.1L4. Using the dynamic programming approach, derive the minimum principle forthe distributed system in Sec. 11.4.

Indexes

Name Index

Akhiezer, N., 96Aleksandrov, A. G., 268Ames, W. F., 2, 97Amundson, N. R., 129, 132, 243Aris, R., 39, 96, 97, 131, 132, 165, 170,

171, 223-225, 243, 244, 322, 324,357, 358, 389, 407

Athans, M., 131, 132, 169-171, 223, 267

Bailey, R. E., 225Balakrishnan, A. V., 322, 389Bankoff, S. G., 325Bass, R. W., 170Battin, R. H., 223, 323Beightler, C. S., 38, 39, 69, 358Bekey, G. A., 243Bellman, R. E., 2, 39, 62, 68, 169, 220,

225, 322, 323, 392, 407Bendelius, A. R., 358, 388Berkovitz, L. D., 223Bertram, J. E., 268Beveridge, G. S. G., 357Bilous, 0., 129, 132Blakemore, N., 171, 225Blaquiere, A., 223Bliss, G. A., 96, 223, 322Blum, E. D., 357Bohn, E."V., 322Boltyanskii, V., 131, 169, 223Bolza, 0., 96, 97, 268Box, M. J., 61, 70Bram, J., 69Breakwell, J. V., 322Brosilow, C. D., 268Bryson, A. E., 225, 322-325Buck, R. C., 223Buckley, P. S., 266Bushaw, D. W., 169Butkovskii, A. G., 387, 389

Carroll, F. J., 323Cauchy, A., 52, 69Chang, S. S. L., 244, 266Chou, A., 224Chyung, D. H., 389Clark, L. G., 389Coddington, E. A., 222, 224Collatz, L., 97Connolly, T. W., 2Coste, J., 243Coughanowr, D. R., 39, 96, 131, 266,

388Courant, R., 39, 96, 323Coward, I., 170Cunningham, W. J., 2

Dantzig, G. B., 70Das, P., 96, 268Denbigh, D. G., 96, 97Denham, W. IF., 225, 323, 324Denn, M. M., 39, 130-132, 169, 170,

223, 224, 243, 244, 268, 269, 322,

324, 357, 387-389

Qesoer, C. A., 223, 243DiBella, C. W., 69Douglas, J., 97, 268Douglas, J. M., 130, 131, 150, 169, 170,

224, 267, 324, 358Dreyfus, S. E., 39, 62, 68, 224, 225, 407Duffin, R. J., 39Dyer, P., 325

Eben, C. D., 39Edelbaum, T. N., 38Egorov, A. I., 389Englar, T. S., 268, 325Enns, M., 70

412

Fadden, R. J., 322Falb, P., 131, 132, 169, 170, 223, 267Fan, L. T., 30, 39, 244, 357Ferron, J. R., 39, 388Fiacco, A. V., 40Fine, F. A., 325Finlayson, B. A., 97Fletcher, R., 69Fliigge-Lotz, I., 170, 256, 268Fort, T., 40, 243Franks, R. G. E., 2Fredrickson, A. G., 225Froberg, C. E., 68Fuller, A. T., 171, 268Funk, P., 97, 268

Gamkrelidze, R., 131, 169, 223Gass, S. I., 70Gilbert, E. G., 322Glicksberg, 1., 169Glicksman, A. J., 66, 70Goh, B. S., 225Goldfarb, D., 69Goldstein, H., 131Goodman, T. R., 322Gottlieb, R. G., 324Gray, R. D., Jr., 388Greenly, R. R., 322Griffith, R. E., 69Gross, 0., 169Gruver, W. A., 267

Hadley, G., 38, 39, 69, 70, 407Halkin, H., 224, 244Hancock, H., 38Handley, K. R., 268Harvey, C. A., 225Hestenes, M., 224Hext, G. R., 70Hilbert, D., 96Hildebrand, F. B., 68, 97, 268Himsworth, F. R., 70Hoge, A. R., 388


Holtzman, J. M., 244Horn, F. J. M., 39, 132, 199, 224, 244,

323, 324, 351, 357Hsia, T. C., 170, 268

Isaacs, D., 324.

Jackson, R., 39, 170, 244, 357, 389Javinsky, M. A:, 170Jazwinsky, A. H., 322Jeffreys, G. V., 40Jenson, V: G., 40Johansen, D. E., 324Johnson, C. D., 162, 170, 225Jurovics, S. A., 322

Kadlec, R. H., 170Kalaba, R. E., 2, 322, 323Kalman, R. E., 39, 96, 131, 132, 224,

267, 268, 323, 325Kantorovich, L. V., 97Katz, S., 244, 268, 324, 389Kelley, H. J., 40, 225, 323, 324Kenneth, P., 323Kgepck, R., 269Kopp, R. E., 225, 321, 322, 324Koppel, L., 39, 96, 131,- 266, 268, 388Krelle, W., 39Kruskel, M., 39Krylov, V. I., 97Kunzi, H. P., 39

Lack, G. N. T., 70Lance, G. N., 322Landau, L. D., 131Laning, J. H., Jr., 223, 323Lapidus, L.,. 69, 70, 267-269, 321,LaSalle, J. P., 169, 170, 269Lasdon, L. S., 324Lee, E. B., 170, 224, 389Lee, E. S., 132, 292, 323, 324

NAME INDEX

Lee, I., 323Lefschetz, S., 269Leitmann, G., 224Leondes, C. T., 323, 324Lesser, H. A., 70Levenspiel, 0., 224Levinson, N., 222, 224Lewallen, J. M., 322Lifshitz, L. D., 131Lin, R. C., 199, 224, 351, 357Lindorff, D. P., 268Luh, J. Y. S., 225Lurie, K. A., 389Luus, R., 267, 268, 321

McCausland, I., 387McCormick, G. P., 40McGill, R., 322, 323McIntyre, J. E., 225, 322McReynolds, S. R., 325Markland, C. A., 325Markus, L., 170, 224, 389Marshall, W. R., Jr., 40Mayne, D., 325Megee, R. D., III, 324Merriam, C. W., 131, 132, 266, 324Meyer, F., 323Mickley, H. S., 40Miele, A., 225Mikami, K., 323Millman, M. G., 268Minorsky, N., 2Mishchenko, E., 131, 169, 223Mitten, L. G., 358Hitter, S. K., 325Moser, J., 39Moyer, H. G., 225, 321, 322, 324Muckler, F. A., 268

Nemhauser, G. L., 358, 407Neustadt, L. W., 224, 322, 389Newman, A. K., 268Nieman, R. A., 324Noton, A. R. M., 132, 267, 322, 325

413

Obermayer, R. W., 268O'Conner, G. E., 132, 267Oguztoreli, M. N., 389Oldenburger, R., 169

Paiewonsky, B., 171, 225, 322Paine, G., 323Paradis, W. 0., 256, 267Pars, L. A., 96Partain, C. L., 225Paynter, J. D., 357Perlmutter, D. D., 131, 256, 266, 267Peterson, E. L., 39Pigford, R. L., 40Pinkham, G., 322Pollock, A. W., 202, 224Pontryagin, L. S., 131, 169, 170, 223,

389Powell, M. J. D., 69Puri, N. N., 267

Rabinowitz, P., 70Ray, W. H., 97, 224Reed, C. E., 40Rekazius, Z. V., 170, 268Rice, A. W., 358, 388, 389Rice, R. K., 324Rippin, D. W. T., 224, 358Roberts, S. M., 268, 269, 407Robinson, J. D., 388Rogers, A. E., 2Rolke, R. W., 388Rosenbrock, H. H., 69, 322Rozenoer, L. I., 131, 244Rubin, H., 39Rudd, D. F., 2, 243, 357, 358, 407

Saaty, T. L., 69Sakawa, Y., 76, 97, 387Sarrus, F., 69Scharmack, D. K., 322Schechter, R. S., 96, 357Schley, C. H., 323

414

Scriven, L. E., 97Sebesta, H. R., 389Shafran, J. S., 225Sherwood, T. K., 40Shih, Y. P., 388Siebenthal, C. D., 165, 170, 171, 225Smith, B. D., 243Spendley, W., 70Speyer, J. L., 322Stancir, R. T., 324Stevens, W. F., 69, 269Stewart, G. W., III, 69Stewart, R. A., 69Storey, C., 69, 322Sutherland, J. W., 322Swanson, C. H., 225Sweed, N. H., 388Sylvester, R. J., 323Szepe, S., 224

Tapley, B. D., 322Thau, F. E., 262, 268Tompkins, C. B., 69Torng, H. C., 70Tou, J. T., 269, 407Tracz, G. S., 171


Troltenier, U., 323, 324Truxal, J. G., 131, 266Tsai, M. J., 357Tsuchiya, H. M., 225Tuel, W. G., Jr., 269

Valentine, F. A., 223, 225

Wagle, A. K., 388Wang, C. S., 30, 39, 244, 357Wang, P. K. C., 388, 389Wanniger, L. A., 269Waren, A. D., 324Warga, J., 224Watson, C. C., 2, 358, 407Webber, R. F., 170Whalen, B. H., 70Wiberg, D. M., 388Wilde, D. J., 38, 39, 69, 358Wilhelm, R. H., 358, 381, 388Wonham, W. M., 161, 170

Zadeh, L. A., 70, 223, 243Zangwill, W. I., 69Zener, C., 39

Subject Index

Absolute minimum (see Globalminimum)

Action integral, 109Adiabatic bed, 43Adjoint, 177, 223

(See also Green's function;Lagrange multiplier)

Algebraic equations, 5, 45Approximate solution, 59, 88, 117,

164, 287, 317Autocatalytic reaction, 358

Bang-bang control, 140, 145, 151,153, 255, 258, 370

Batch reactor, 40, 82, 128, 165, 212,274

Bottleneck problem, 217, 225Bound (see Constraint)Boundary condition (see

Transversality condition)Boundary iteration, 272, 278, 321Boundary-value problem (see

Two-point boundary-valueproblem)

Brachistochrone, 77Bypass, 326

Calculus of variations:isoperimetric problem, 84, 297, 309"simplest" problem, 73, 108(See also Euler equation)

Canonical equations, 106, 136, 184,232, 378

Catalyst, 173, 224, 390-Catalytic reformer, 202Chatter, 256Chemical reaction, 27, 40, 41, 98,

111, 199, 202, 329, 337, 351, 371

Chemical reactor (see Batch reactor;Continuous-flow stirred-tankreactor; Tubular reactor)

Classical mechanics, 109Complex method, 61, 279Condensor, 42Constraint, 6, 18, 34

on decisions, 135, 181, 231on state variables, 181, 209, 211,

231, 242, 308Continuity, 5, 212, 214, 221Continuous-flow stirred-tank reactor,

27, 30, 41, 46, 55, 111, 144, 150,163, 199, 234, 304, 337, 351

Control, 1, 10, 38, 212, 247, 266(See also Feedback control;

Feedforward control; Time-optimal control)

Control variation (see Variation)Convergence, 54, 57, 61, 64, 68, 69,

277-279, 299, 302, 305, 307, 311,312, 319, 321, 343, 347, 353, 367,374

Cooling rate, 165, 212, 371Corner condition, 109Cost-of-control term, 117, 124, 132,

159, 250Curse of dimensionality, 272

Data fitting, 41Decomposition, 355Definite function, 9, 21, 43, 259Design, 1, 2, 24, 128, 130, 165, 235,

300, 304, 371, 376, 386Difference-differential equation, 389Difference equation, 36, 64, 228, 236,

262, 398Differential calculus, 4, 34Differential equation, 176, 221

415 '

416

Diffusion, 98, 359, 371Direct methods, 295, 337Discrete system (see Staged system)Discrete variable, 11, 13, 15, 23, 36,

73, 262, 395Disjoint policy, 82Distance in decision space, 54, 68, 296Distillation, 132Distributed-parameter system, 92,

359, 389Disturbance, 121, 124, 248, 379, 381Drag, minimum, 98Dynamic programming, 217, 392

computation by, 393and the minimum principle, 40Y), 404

Eigenvalue, estimation of, 90Equivalent formulation (see Objective

f unction)Euler equation, 27, 28, 30, 39, 75, 85.

93, 109, 127, 283, 297, 309Existence, 224Extractor, 30, 40, 337Extremal, 77

Feedback control, 11, 17, 80, 114,120, 123, 125, 132, 142, 147, 160,247, 249, 251, 255, 264, 319, 380,406

Feedforward control, 123, 247, 249, 380Fibonacci search, 49, 62, 68, 355Final time unspecified, 104, 105, 184,

364Finite variation, 192, 362First variation, 181, 231Fredholm equation, 95, 365Fuel-optimal control, 159Function iteration, 283, 288, 295, 321Functional, 74Fundamental matrix, 177

Galerkin's method, 88, 97, 385Geometric programming, 39, 42


Geometry, 21, 53, 54, 296Global minimum, 6, 197

(See also Necessary conditions;Sufficient conditions)

Goddard's problem, 226Golden section, 64Gradient, 54, 59, 66

(See also Steel) descent)Gradient projection, 308Green's function, 176, 177, 181, 210,

214, 229, 273, 296, 308, 328, 335,361, 372, 378, 384, 387, 400

(See also Adjoint; Lagrangemultiplier)

Green's identity, 177, 181, 230, 273,296, 308, 328, 335

Hamilton-Jacobi-Bellman equation,398, 400, 402

Hamiltonian, 106, 109, 136, 184, 232,297, 316, 328, 335, 378

Harmonic oscillator, 153Heat conduction, 93, 364, 366Heat exchanger, 30, 376, 378, 390Heavy-water plant, 71Hessian, 9, 21, 46, 57Hierarchy of optimization

problems, 165, 214

Index of performance (seeObjective function)

Indirect methods, 293Instantaneously optimal control,

254, 264(See also Disjoint policy;

Inverse problem)Integral control (see Reset mode)Integrating factor, 176Intermediate solution, 139, 145, 151,

153, 157, 160, 213, 365, 369Interval of uncertainty, 50, 62Inverse matrix, 46Inverse problem, 30, 86, 97, 205, 257

SUBJECT INDEX

Jump condition, 212, 214

Kuhn-Tucker theorem, 39

Lagrange form, 187Lagrange multiplier, 18, 22, 23, 25,

32, 85, 103, 234, 263, 297, 309.333, 400

(See also Adjoint; Green's function)Lagrange multiplier rule, 20, 233, 332Lagrangian, 20, 202Laminar flow, 99Legendre condition, 207Liapunov function (see Stability)Linear programming, 65, 70, 220, 356Linear-quadratic problem (see

Quadratic objective)Lipschitz condition, 222Local minimum, 6, 197

(Sec also Necessary conditions;Sufficient conditions)

MAP, 68, 69Maximum conversion (see

Optimal yield)Maximum principle (see

Minimum principle)Mayer form, 187Metric (see Weighting function)min 11, 311, 314, 321Minimum (see Global minimum;

Local minimum; Necessaryconditions; Sufficient conditions)

Minimum fuel-plus-time control, 155Minimum integral square-error

criterion (see Quadratic objective)Minimum principle:

for complex systems, 328, 337for continuous systems, 107, 138,

186, 194, 304, 328, 337for distributed-parameter systems,

363, 372, 378for staged systems, 232, 237, 400

417

Minimum principle:strong, 194, 197, 223, 237, 238,

312, 329, 350, 401, 404weak, 107, 138, 186, 232, 328, 337,

400Minimum-time control (see Time-

optimal control)Mixing, 215, 327, 334, 357Modal analysis, 387, 388Model, 2, 10, 27, 29, 110, 131, 203,

333, 371, 382Multipliers (see Adjoint; Green's

function; Lagrange multiplier)Multistage system (see Staged system)

Necessary conditions, 5, 10, 20, 76,85, 106, 138, 186, 194, 207, 232,240, 241, 311, 363, 378, 398

(See also Lagrange multiplier rule;Minimum principle)

Negative feedback (see Stability)Newton-Raphson iteration, 3, 45, 57,

68, 273, 283, 288, 312, 314, 321,350

Nonlinear programming, 21, 39, 68Nonlinear system, optimal control of,

118, 131, 150, 163, 170, 254Nuclear reactor, 134Numerical analysis, 70, 298, 367,

373, 376

Objective function, 10, 29, 101, 120,125, 159, 169, 181, 187, 203, 247,250, 258, 268, 331, 349, 384

One-dimensional process, 29, 30, 241Operations research, 171Optimal yield, 28, 46, 84, 125, 128,

130, 275, 278, 351, 374Orthogonal function, 41

Parametric pumping, 381Particular variation (see Special

variation)

418

Penalty function, 34, 124, 224, 248,277,306

Periodic process, 348, 381(See also Steady state,

optimality of)Perturbation, 2, 3

equations (see Variational, equation)(See also Variation)

Picard iteration, 194, 238Pipeline reactor (see Tubular reactor)Pontryagin's maximum principle

(see Minimum principle)Pressure profile, 130, 171, 278, 290,

299, 319Principle of optimality, 393Production scheduling, 134Proportional control, 12, 13, 17, 23,

80, 117, 125Pulmonary ventilation, control of,

134

Quadratic objective, 17, 21, 79, 115,117, 121, 124, 159, 163, 248, 263,364, 378, 405

Quasi linearization, 3, 289

Reaction (see Chemical reaction)Reactor (see Batch reactor;

Continuous-flow stirred-tankreactor; Nuclear reactor;Tubular reactor)

Recycle, 326, 327, 329, 337Reformer, 202Regularity, 20, 188Regulator problem (see Quadratic

objective)Relay control (see Bang-bang

control)Reset mode, 125

(See also Three-mode control)Riccati equation, 17, 24, 80, 117, 122,

249, 264, 318, 321, 380, 406Ritz-Galerkin method, 88


Second-order variational equations,194, 238, 314, 316

Second variation, 315, 321Self-adjoint, 86, 226Sensitivity variable, 33, 34, 400

(See also Adjoint; Green's'function;Lagrange multiplier)

Servomechanism problem, 248Signum function, 140Simplex method:

of linear programming, 66of steep descent, 61

Simulation, 2, 350Singular solution, 160, 164, 207, 213,

260, 324, 365, 369, 370, 373Special variation, 8, 19, 75, 105, 137,

195, 208, 362Stability, 12-14, 163, 252, 259, 265,

295, 2Staged system, 24, 228, 393Steady state:

control of, 10, 112, 144, 152optimality of, 199, 350

Steep descent, 52, 57, 66, 69, 278,295, 308, 311, 321, 337, 341, 350,365, 367, 370, 373, 384

Step size, 53, 57, 301Stopping condition (see Final time

unspecified)Strong minimum principle (see

Minimum principle)Structure, 326, 333, 339, 355Sufficient conditions, 10, 220, 241,

398, 406Switching surface, 142, 147, 151, 154,

157, 162, 168, 191, 255, 258

Taylor series, 7, 11, 19, 45, 102, 112,181, 193, 208, 273, 286, 289

Temperature profile, 27, 46, 55, 84,128, 171, 197, 234, 274, 301, 304,312, 337, 394

Terminal condition (see Tranaversalitycondition)

Three-mode control, 247, 251

SUBJECT INDEX

Time-and-fuel-optimal control, 155Time-optimal control, 77, 82, 139, 144,

150, 153, 168, 189, 322Traffic control, 172Transversality condition, 23, 104, 183,

189, 224, 261, 332, 336Tubular reactor, 84, 128, 130, 165,

274, 344, 371Two-point boundary-value problem,

23, 26, 76, 236, 272, 289, 298,321, 397

Underdamped system, control of, 155'Unimodal function, 49, 62Uniqueness, 143, 170

411

Variation, 7, 102, 136, 180, 192, 194,296

Variational equation, 52, 103, 181,194, 231, 239, 273, 295, 327, 334,360

Variational method, 2, 4, 387, 404

Weak minimum principle (weMinimum principle)

Weierstrass condition, 207, 223Weighting function, 53, 57, 69, 297,

301, 312, 385

Yield (see Optimal yield)

This book was set in Modern by The Maple Press Company, and printed on-permanent paper and bound by The Maple Press Company. The designer wasJ. E. O'Connor; the drawings were done by J. & R. Technical Services, Inc.The editors were B. J. Clark and Maureen McMahon. William P. Weisssupervised the production.

denn optimization by variational methods

Documents