outline - cs229.stanford.educs229.stanford.edu/livenotes2020spring/cs229-livenotes-lecture7.pdf0 e...
TRANSCRIPT
Outlinekernel Methods
SVM
feature mapKd RP
a Ollie
fitting hood Oton using gradientdescentwith 01 uh as features
0 0Loop0 Oth IECgc 80cm loln
Issue 0 Oln CIRP can be very highdimensionalruntime per iteration OCap
Goat improve to 0cm per iteration
key observation0 can be represented as
a EE pi locascalar n variables insteadof p
Proof by inductionAt iteration 00 0 IE O 46
supposeat iteration t O Eh pi 19amNext iteration o y z yet or la Dolce
Epi 19am
0 E patacy Gollum 4cal9new pi
represent 0 C IRP implicitly by 13 Elkworks better p n
Update rule for p 01 Block
Bi pit Nyc OTQue p allaDT
path y E B Gcn 5 love
pied y Ep Ca loca D
atb La bObservations
LOCK OC x 7 As can be precomputed
Often Collum Olsen can be computed fasterthan OcpI
Iid P Ad
25 71 2 EdEdgin z.cziid
l
yXd
10K lolz It 71 12u t.IE eXiXjZi2jtd
EEnXeXsXk2c32qIt CX 27 t Lx
2X 273
Can compute 2041 0677 in old tune
Klee 2 QU 01677 kernelkn IrdXIRD IR
Algocompute kCx x Collie localfor all j e I n
Setp OOP
pi p x y Ep Mx a
preprocessing O n'dEach iteration 0cm per iteration
prediction Gwenn compute 00h seal teh training00cal EE pi love
TOllie example
EE pi Kcnc na new example
OCnd time
Deeper Observation
TheAlgoonly depends on KC e
design of features design of kernel functions
what kernels are valid 7
Flo St Kae 2 local OG
Necessary conditionindata points x 2dm
Kernelmatrix KEIR mKy K za za
matrix K is posceivesemidefinite 4K Z OKE o es 2T K2 70 Ky LolCid locumIT Kz Zi Ky 3 locale 4Cn dad
2 Ollie T01 41 2
E zi deck deck 2g
Ee C ai dela 75 30
K Z O
Th m Mercer K is a valid kernel fn ie Kca2 a Caiolaiff forany ncos and any x Xthe corresponding kernel matrix Kst Kyi Kae red
is positive semidefinite
Other kernels
Mn 2 XTZ t CT Cal lolz
Kfk 2 LxTz ct
Gaussian kernel Kca 2 exp 42221122
Kcr 2 LOCH 01677Infinite dimensional
Xi axe Xi Xi
Protein sequence classificationsequencerof amino acids CA T
AAAAAAAB
204 dem vector
TTT T
Histograms take men and seem up
SV Ms for classification
I n.IE o
O o x wtf n 16 030
SVMgooey n for nowyil E E L I
Warmupifeud w b s t
if y L win c b 70
y z I wt k cb s O
Many such w b
Newgoal Among all w b satisfyingFund web St mw.abx.gyy.my
dirt Cail boundary4
n wheels o
WTH'tbF f N U on positive sidewhats o
max mon y what bw b LEI n
Scale invariant W b 400W toob
wlog we want to fund w bSA mm y Wta b I
iE l n
max I men 11WhAWHz
men11412
St fi y Luta t b 71
men 11Wh men ENKESt the y Ewtn cb 31
Facts optimal soli w dig n for somedisco