lecture 17-18 primal methods - solmaz s. kiasolmaz.eng.uci.edu/teaching/mae206/lecture17-18.pdf ·...
TRANSCRIPT
Lecture 17-18Primal methods
Solmaz KiaMechanical and Aerospace Eng. Dept.,
University of California Irvine
Consult: Section 2.3 of Ref[1] and Sections 12.1,12.2 and 12.4 of Ref[2]
Primal Methods for constraint 4timizationWe consider the problem . . ?>7ô
. Iminf(x), LI f’k c
g(x)<O ‘ ‘,— )
I ?,q4ajItQ 1h(x)=O t,‘> P/i
/Y
—:---U::T—; d140By a primal method we mean a search method that works bysearching through the feasible region.First Order Necessary Condition for Optimality: x is a local minimizer then
Vf(x*)TL\x ; 0, for I\x e V(x*)
. Set of first order feasible variations at x
V(x) ={d e R f Vhj,(x)Td = 0, Vg(x)Td 0, j e A(x*)}
. Active inequality constraints at x
A(x) = {j E {1, . . . , r} gj (x) = O}
Primal Methods for constraint optimization
We consider the problem
mm f(x)g(x)Oh(x)=O
By a primal method we mean a search method that works bysearching through the feasible region.
; I •; i
I S
\C’ “ ‘1
Advantages “
. If the process is terminated before reaching the solution, the. . . 4-current point is feasible. V — ‘j
. It can often be guaranteed that if the sequence of pointsconverges, then the limit is a local constrained minimum. 7
• Most of the primal methods do not rely on special structure,such as convexity.
Disadvantages• Needs a feasible starting point.• It can be computationally hard to remain in the feasible region.
Feasible Direction MethodsThe idea of feasible direction methods is the same as with
) L
where dk 5 a feasible direction at xk, and ük 0.
is chosen to minimize f with the restriction that the point xk+1,and the line segment joining xk and xk+1 be feasible.
\>
unconstrained problems:,7)
Xk+1 Xk+kdkS
OPT: x =argmin f(x)xR
x E X (X is the set of constraints)for X = RU (problem becomes unconstrained) /
D RTh is a feasibledirection at x e X for OPTif(x+ocd) Xfor
&
LXEE [O,]
c)
A Feasible Direction Method: Simplified Zoutendijk method4Consider the a problem with linear constraints:
Given a feasible point Xk, let A(Xk) be the set of indices of active constraints,aj1xk b for 1 e A(Xk).
• The other constraints assure that Xk + dk will be feasible for sufficientlysmall k > 0.
minimize f(x)
u[xb, 1=1,••• ,rS t. fr- 4
The direction vector dk is then chosen as the solution of
5r\ )Lc 52.l minimize Vf(xk)Td
ci’
v
s.t.
i e A(xk),ad0,
Idj1=
LL-e
2Th ‘
;
j4CIL)
j=1
I
bounded solution
.-1_ irJ_/(
.,‘\Ot,’ k
c, LII’
• The last equation (which can be converted to linear constraints) ensures a
• The objective function makes d as close to Vf(Xk) as possible.
Feasible Direction Methods
There are two major shortcomings of feasible direction methods inthis form.. For general problems there may not exist any feasibledirections (see figure). ?‘ (‘‘
. They are also vulnerable to ,,jamming”, or ,,zigzagging”, whichwe have also seen with unconstrained problems, but here itmight converge to a point that is not even a constrained localminimum (the algorithmic map is not closed).
Feasibleset
S
c)
IV
:c-
I
S
r V
E r
1
2—.
0
V)
N
Ct
r
—r
f0
ci
Tp
11 0
2UJ
bj4J
9 p
(N
IL I-1-11
I’
(
i-cD
•
-
ii‘s--‘
4
\\-
1
11 S \co
+I, ±
q
+çC
>
F
c1
4-
3
00
1:___
r—cD-
%‘/
&c:s
_%$_J
9%1)
EDp
0
ac0U’
c
b
c_.
‘I
Ii
c)
/
y-
U
U
03
U
: $
±__k
r
k-
%rt-
r.
;:
‘I
4C:
:Lc
0
;,0
:S.
9S
454
r>-‘
..—
9
0
—
S
-11•’-r’O’
p-.
Icc;
‘:,
‘f\
S
r—i
-i--L_J
LJ
0
/
LE
00I-
7II
1ic)c0
0
N
*
cc *
I(C
a—T
‘%—
____
21
Th
--
r
ito
U4:_k c,Lt&(’L
Va
ft
‘I
‘
‘cicLQt
(‘1
0f’
ULI
(—_‘1Th
r
c±l
6
c’c
ra—’
6).
.,
ry__
4s
‘,
c
,;
.
Lc’
!
3_
.
;:3__r
>-,
:h,
s,
r’:r
,,c__
r’i
q:
f‘
%;:
-2—
•
_t-
.‘
3:(].
t?
.c;’.
tV
,‘\
.‘
0’?
\%—.,
.r?
(Cb
::ri
6•.
r’0
‘)—
r4
Lrc’
\‘—‘
S
S
rCI
\r
“
_‘_
-#
‘
r::
a•‘
c__%,
-rs
F-
11•T
?,
LJ
:
F
,
0 1’
T_
zN
(t\
z
0
cc
4-
1 LJ
t
oL ft
\:<c2 I
(%)
: 1 ,‘
cr T
r t.
<0) 3—
-r;
Th t
1
p A U )
Ac
jc (c
rOr
.4-r
(IL
mUI
.÷‘N
5
Cri
r
0 ci
x
1-
V
. (t.
‘
.
(cjJ
‘2
R,
O4-
%-c
J
jj
%)
aI0
n
L‘3
c;4DJ
IcL‘
(Dr
1:I
cJ1
0
Hø—Th:
4
fl4
04
.4
4.c4I’gLi
0
(Th-Jii
-‘
-Th
L
0Th
%
.—0c‘—h
%%J0
oJ
%%4-4
I;:
J‘
.%.-
a’
‘c-%
_)
ksi—
t)o
D,
‘S\_AT—
rL?(
4.-1’a
(Th
C,--
ct
o0
II>
-
‘1
V_
)-
II
“I
V)ì‘\J
_•%
A
U
TD
-Th
i
sJ
1—
L
=‘45
I
r
4
0
C
jp
o
Xk1ke ri hci
L
k().
(
‘:c1 r(L)
-2-
i) 4
c)
Ii 9r
-I’ o5 >
cc q
4
Os
‘
E
9 4
\1 4
\c
‘\I2\s \
c :‘s$\-°
2 L—
4”
Is
E
9 ‘\
11
\
N\‘
it
I1Hz
ii
S
0‘I
—1
>)
4’
F1
> b
11
&\
\\
ii
D
c aI%-
\\‘
‘F[
0
‘Ici.
0r
rCii
Uft
‘%__I
LA
k
__
4\
Csr
‘
Cl)%.
PRACTICAL AUGMENTED LAGRANGIAN METHODS:BOUND-CONSTRAINED FORMULATION
minimize f(x)
h(x)=O, l<x<u
LA (x,;tk) = f(x)+rn , rn
Akh(x)+kh(x)2
Bounded Gradient Lagrangian method
; Ik ) subject to
l<x<u
Ak+l Ak +kh(Xk)
Ak+1 Ak;
k+1 1OO;1 iaOA
11k+1Wk+1 l/&k+1;
(Bound-Constrained Lagrongian Method). I here is the projection operator tor boxed I. , . . . . * . 0 inequality (check your notes on gradient
Choose an initial point x0 and initial multipliers A ; projection method for further details)Choose convergence tolerances i4, and w;Set Co = 10, w = 1/ , and i = if ‘;fork = 0, 1, 2, .
ar0mate solution Xk subproble mm £A(x,Ak;) subject to I x U
IIxk (xkV1CA(xk,A’;Ck),1. <wk;
An efficient technique forsolving the nonlinear programwith bound constraints(forfixed .t andA) is the(nonlinear) gradient projectionmethod (see your notes fromlectures on primal methods)
ifllh(xk)1I 1IkNN
( * test for cwergence )if IIh(xk ) Ii s ‘7* Xk ‘ (Xk VItA (Xk , Ak ; k )‘ 1, u) II <
stop with apprimte solution xk;end(if)(* update multipliers, tighten tokq.ices )Ak+l Ak + kh(xk);
Remember that gradient projection method stopswhen the point generated from Xk by the gradientdescent algorithm gets projected back on Xk
Check the stopping condition of the gradientprojection method in your notes for more details.
else
L.)k+1Io9
Ilk-i-i = TIk/L..’k+i,WkI1 Wk/k+1
(* increase penalty parameter, tighten tolerances
If this condition holds, the penalty parameter is not changed forthe next iteration because the current value of Pk is producing anacceptable level of constraint violation. The Lagrange multiplierestimates are updated according to the update formula and thetolerances Wk and flk are tightened in advance of the nextiteration. If, on the other hand, this condition does nothold, then we increase the penalty parameter to ensure that thenext subproblem will place more emphasis on decreasing theconstraint violations. The Lagrange multiplier estimates are notupdated in this case; the focus is on improving feasibility.
end (if)end (for)
The constants 100, 0.1, 1 appearing hereare to some extent arbitrary;
—-—-———-—-—-—-———————-— other values can be used withoutcompromising theoretical convergenceproperties.