presentazione di powerpointiacoviel/materiale/lecture_4_2018_sito.… · lev semenovich pontryagin...

Optimal Control

Prof. Daniela Iacoviello

Lecture 4

31/10/2018 Pagina 2Prof.Daniela Iacoviello- Optimal Control

THESE SLIDES ARE NOT SUFFICIENT

FOR THE EXAM:YOU MUST STUDY ON THE BOOKS

Part of the slides has been taken from the References indicated below

Pagina 231/10/2018

31/10/2018 Pagina 3Prof.Daniela Iacoviello- Optimal Control

Course outline

• Introduction to optimal control

• Nonlinear optimization

• Dynamic programming

• Calculus of variations

• Calculus of variations and optimal control

• LQ problem

• Minimum time problem

Pagina 331/10/2018

R.F.Hartl, S.P.Sethi. R.G.Vickson, A Survey of the

Maximum Principles for Optimal Control Problems with

State Constraints, SIAM Review, Vol.37, No.2,

pp.181-218, 1995

Prof.Daniela Iacoviello- Optimal Control Pagina 431/10/2018

Pontryagin

Lev Semenovich Pontryagin (3 September 1908 – 3

May 1988) was a Soviet Russian mathematician. He was

born in Moscow and

lost his eyesight in a stove explosion when he was 14.

Despite his blindness he was able to

become a mathematician due

to the help of his mother who read

mathematical books and papers to him.

He made major discoveries in a number of fields of

mathematics, including the geometric parts of topology.

Prof.Daniela Iacoviello- Optimal Control Pagina 531/10/2018

The Pontryagin principle

Problem 1: Consider the dynamical system:

with:

Assume fixed the initial control instant and the initial and final

values :

Define the performance index :

with

( )uxfx ,=

( ) niURCx

ffRUtuRtx n

i

pn ,...,2,1,,)(,)( 0 =

Pagina 631/10/2018Prof.Daniela Iacoviello- Optimal Control

ff

ii xtxxtx == )()(

( ) ( ) ))(()(),(,, f

t

t

f txGduxLtuxJ

f

i

+=

( ) 20 ,,...,2,1,, CGniURCx

LL n

i

=

Determine:

▪ the value ,

• the control

• the state

that satisfy:

✓ the dynamical system,

✓ the constraint on the control,

✓ the initial and final conditions

✓ and minimize the cost index

( ) ,if tt

)(0 RCuo

)(1 RCxo


Hamiltonian function

( ) ( ) ( )uxftuxLuxH T ,)(,,,, 00 +=



Theorem 1 (necessary condition):

Assume the admissible solution is a minimum

there exist a constant

and a n-dimensional vector

not simultaneously null such that :

( )*** ,, ftux

00

*1* , fi ttC

T

x

H*

*

−=

( ) ( )U

ttutxHttxH

,)(,),(),()(,,),( **0

****0

*


0*=H


Remark- If is fixed, the condition

Is substituted by


0*=H

ft

fi tttkH ,,*

=



with:

( )uxfx ,=

( ) niURCx

ffRUtuRtx n

i

pn ,...,2,1,,)(,)( 0 =


Assume fixed the initial control instant and the initial state

while for final values assume:

where is a function of dimension of C1 class.


with

ii xtx =)(

( ) 0)( = ftx

nf

( ) ( ) ))(()(),(,, f

t

t

f txGduxLtuxJ

f

i

+=

( ) 20 ,,...,2,1,, CGniURCx

LL n

i

=

Determine:

▪ the value

▪ the control

▪ and the state

that satisfy




✓ and minimize the cost index .

( ) ,if tt

)(0 RCuo

)(1 RCxo



Consider an admissible solution such that

If it is a minimum

there exist a constant and an n-dimensional vector


Moreover there exists a vector such that:

( )*** ,, ftux

00

*1* , fi ttC

fftdx

drank =

*

)(


T

x

H*

*

−=

( ) ( )U

ttutxHttxH

,)(,),(),()(,,),( **0

****0

*

T

ff

tdx

dt

*

)()(

=

fR

0*=H


Remark- If is fixed condition

Is substituted by


0*=H

ft

fi tttkH ,,*

=



with:

( )tuxfx ,,=

( ) niURCx

ffRUtuRtx n

i

pn ,...,2,1,,)(,)( 0 =


Assume fixed the initial control instant and the initial state

while for final values assume: where

is a function of dimension of C1 class.


with

ii xtx =)(

( ) 0),( = ff ttx

1+ nf

( ) ( )=f

i

t

t

f duxLtuxJ ),(),(,,

( ) niRURCt

L

x

LL n

i

,...,2,1,,, 0 =

Determine:

▪ the value

▪ the control

▪ and the state

that satisfy

the dynamical system,

the constraint on the control,

the initial and final conditions

and minimize the cost index .

( ) ,if tt

)(0 RCuo

)(1 RCxo


Theorem 3: Consider an admissible solution such that

IF it is a minimum there exist a constant and an

n-dimensional vector not simultaneously null

such that :

( )*** ,, ftux

00

*1* , fi ttC

( ) fff ttx

rank =

*

),(


,

**

T

x

H

−= Rkkd

HH

ft

t

=

+ ,

*

**

( ) ( ) UttutxHttxH ,)(,),(),()(,,),( **0

****0

*

Moreover there exists a vector such that: fR

T

ft

T

f

ft

Htx

tf

*

*

*

**

)()(

−=

=



with:

and initial instant and state fixed

( )tuxfx ,,=

( )RURCt

f

x

ffRUtuRtx npn

0,,)(,)(

ii xtx =)(


For the final values assume: where is a

function of dimension of C1 class.

Assume the constraint with


with

( ) 0),( = ff ttx

1+ nf

( ) kduxh

f

i

t

t

= ),(),(

( ) niRURCt

h

tx

hh n ,...,2,1,,

)(, 0 =

( ) ( )=f

i

t

t

f duxLtuxJ ),(),(,,( )RURC

t

L

x

LL n

0,,

Determine

• the value

• the control

• and the state

that satisfy




✓ and minimize the cost index .

( ) ,if tt

)(0 RCuo

)(1 RCxo


Hamiltonian function

( ) ( ) ( ) )),(),((,)(,,,, 00 ttutxhuxftuxLuxH TT ++=




IF it is a local minimum

there exist a constant , ,


( )*** ,, ftux

00 *1* , fi ttC

( ) f

ff ttxrank =

*

),(


,

**

T

x

H

−=

( ) ( )U

ttutxHttxH

,)(,),(),()(,,),( **

0

****

0

*

Moreover there exists a vectorsuch that:fR

T

ft

T

f tH

txT

f

*

*

*

**

)()(

−=

=

RkkdH

H

ft

t

=

+ ,

*

**

R*

The discontinuities of may occur only in the instants

in which u has a discontinuity and in these instants the

Hamiltonian is continuous

*


Remark

If the set U coincides with Rp the minimum

condition reduces to :

0=

u

H


The Pontryagin principle - convex case

Problem 5: Consider the dynamical linear system:

with A and B of function of C1 class; assume fixed the initial

and final instants and the initial state, and

Assume

where U is a convex set.

utBxtAx )()( +=

nfff Rtxorfixedxtx = )()(

fip tttRUtu ,)(



with

L convex function with respect to x(t), u(t) in

per ogni

G is a scalar function of C2 class and convex with

respect to x(tf).

( ) ( ) ( ))(),(),(, f

t

t

txGduxLuxJ

f

i

+=

( ) nittURCt

L

x

LL fi

n

i

,...,2,1,,,, 0 =

URn

fi ttt ,


Determine:

the control

and the state

that satisfy

the dynamical system,

the constraint on the control,

the initial and final conditions

and minimize the cost index .

fio ttCu ,0

fio ttCx ,1


Theorem 5 (necessary and sufficient condition):


It is a minimum normal (i.e.λ0 =1)

if and only if there exists an n-dimensional vector

such that :

( )oo ux ,

fio ttC ,1

( ) f

o

ff ttxrank =

),(


oTo

x

tuxH

−=

),,,(

( ) ( ) UttutxHttxH ooooo ,)(),(),()(,),(

nf Rtx )(

oT

f

fo

tdx

dGt

)()( =Moreover, if

Example 3 (from L.C.Evans)

Control of production and consumption

Consider a factory whose output can be controlled.

Let’s set:

x(t) the amount of output produced at time t , 0 ≤ t.

Assume we consume some fraction of the output at each time

and likewise reinvest the remaining fraction u(t) .

It is our control, subject to the constraint

0 ≤ u(t) ≤ 1


The corresponding dynamics are:

The positive constant k represents the growth rate of our

reinvestment. We will chose K=1.

Assume as cost index the function:

The aim is to maximize the total consumption of the

output

0)0(

0),()()(

xx

ktxtkutx

=

=

( ) −=

ft

dttxtuuJ

0

)()(1))((


We apply the Pontryagin Principle; the Hamiltonian is:

The necessary conditions are:

( ) ( ) xuxuuxH +−= 1,,

( )

( ) ( ) 1)()()()(max)(),(),(

)()()(

0)(1)()(1)(

10−+=

=

=−−−=

ttxtutxttutxH

txtutx

tttut

u

f


1,0),()()()()()()()()()()()( −+−+ txtttxttxtxtuttxtutx

1,0,1)()(1)()( −− ttttu since x(t)>0


=

1)(0

1)(1)(

tif

tiftu

1,0,1)()(1)()( −− ttttu

From the equation of the costate, since ,

by continuity we deduce for t<tf, t close to tf, that

thus for such values of t.

Therefore and consequently:

More precisely so long as

and this holds for:

0)( =ft

0)( =tu

1)( t

1)( −=t ttt f −=)(

1)( t

ff ttt −1

ttt f −=)(


For times with t near tf we have

Therefore the costate equation yields:

1− ftt 1)( =tu

( ) )(1)(1)( ttt −=−−−=

Since we have for all

and over this time interval there are no switchings

1)1( =−ft 1− ftt

1)(1

=−− tt fet


Therefore:

For the switching time

Homework: find the switching time

=

fs

s

tttif

ttiftu

0

01)(*

1−= fs tt

st ft

Optimal solution: we should reinvest all

the output

(and therefore consume nothing)

up to time ts and afterwards we should

consume everything

(and therefore reinvest nothing)

Bang-bang control


presentazione di powerpointiacoviel/materiale/lecture_4_2018_sito.… · lev semenovich pontryagin...

Documents