Support Vector Machine

The Lagrange dual problem

min f(x) s.t. gi(x) � bi

@L@x = 0

gi(x) � bi

�i � 0

�i(gi(x)� bi) = 0KKT

Primal problem

Conditions

L(x,~�) = f(x)� P�i(gi(x)� bi)

L⇤(~�) = minx L(x,~�)

max L⇤(~�) s.t. �i � 0

Lagrange dual function

Lagrange function

Lagrange dual problem

Dr. Guangliang Chen | Mathematics & Statistics, San José State University 21/67

Support Vector Machine

The Lagrange dual problem

min f(x) s.t. gi(x) � bi

@L@x = 0

gi(x) � bi

�i � 0

�i(gi(x)� bi) = 0KKT

Primal problem

Conditions

L(x,~�) = f(x)� P�i(gi(x)� bi)

L⇤(~�) = minx L(x,~�)

max L⇤(~�) s.t. �i � 0

Lagrange dual function

Lagrange function

Lagrange dual problem

Dr. Guangliang Chen | Mathematics & Statistics, San José State University 21/67

equal

optimal

objective

values

Consider the following primary problem p :

minimize fax ) subject to gcx ) Zo

[ X ] pZ= text ,

B

G cgcx ) ,fox ) )

•

( yo ,Zo )

.

s y-

- get )O

G- = { Cy ,-2 ) : y = gcx )

,

Z -

- fix ) for some I'

EX }

.

: z = FLI )

y= g CE )

minimize L = f CX ) - Agin ) subject to d Zo

isminimize fox ) - xgcx )

to minimize Z - dy over points c y ,Z ) in G

slopet

z -

- FCI ) Z - ay = a

[ x ] T t pintercept

minimize

•

G cgcx ) ,fix ) )

•

( yo ,Zo )

.

, y-

- g CT )O

.

: z = FCI )

y= g CE )

minimize L = fax ) - Agin ) subject to d Zo

isminimize fox ) - xgcx )

to minimize Z - dy over points c y ,Z ) in G

µZ=fCI) Z - ay =L

[ x ] Tminimize

•

G cgcx ) ,fix ) )

•

•

z - xy=2 20

sloped 70

.

, y-

- g CI )O

• move 2 - ay = 2 parallel to itself as far down

as possible ,while it remains in contact with G

am do is the minimum value of L corresponding

to the given X 70

maximise L 't Cd ) subject to X Zo

[ X ] yrZ= text ,

• Z - x*y=2*G cgcx ) ,fix ) )

•

Z - xy=2 •( Yo ,

Zo )

sloped Zo20

.

, y-

- get )O

Too Have to find the tire with slope A cargo )

to maximize the intercept 2.

Be Such a line has slope A 't and supports the

set G at the point C Do ,Zo )

Bo The solution to the dual problem B X*

and the optimal dual objective value B Zo

[ X ]

qZ=fcI,

• Z - x*y=2*G cgcx ) ,fix ) )

•

Z - ×y=2 •( Yo ,

Zo )

sloped Zo20

.

, y-

- get )O

solutionp ( yo ,

Zo )equal

optimal

objective

Value

solution( yo ,

-20 )