home g - machine learningcs229.stanford.edu/livenotes2020spring/cs229-livenotes... · 2020. 4....
TRANSCRIPT
![Page 1: home g - Machine Learningcs229.stanford.edu/livenotes2020spring/cs229-livenotes... · 2020. 4. 16. · Multiclass Classification Cross Entropy minimization U2 g 00 O O 00 Lie K EIRD](https://reader034.vdocuments.mx/reader034/viewer/2022051822/5fec4864aa3beb75fc36d89d/html5/thumbnails/1.jpg)
TopicsPerceptronExponential FamilyGeneralized Linear Models
Softmax RegressionMulticlass Classification
Logistic Regressiona
It
g 2 z GG L 230O Z CO
hope 1home g E'aIt Ota
0 i O T x y how ngdifferent
y how0 algo got it rightL if wrong y L EEEy wrong y 0
EnzoQ
OEl
iO O
O
![Page 2: home g - Machine Learningcs229.stanford.edu/livenotes2020spring/cs229-livenotes... · 2020. 4. 16. · Multiclass Classification Cross Entropy minimization U2 g 00 O O 00 Lie K EIRD](https://reader034.vdocuments.mx/reader034/viewer/2022051822/5fec4864aa3beb75fc36d89d/html5/thumbnails/2.jpg)
want 0 70 but 0Th LOO OtLn tLu5 n On th Nu
Exponential FamiliesPDF
plugin blog exp Thy acne
y data beg exp 2TTG
n natural parameterTcg sufficient statistic
y in classblog Base measure
acq log partition function
y scalar
q vector scalar
TG vector scalar I match
b g scalar
ExampleBernoulli Binary Dataprobability of event
p g lo 494t y
exp log 994 09 exp y tog lo ta g logo a
exp log 9 t logo 0
Tks Fat
![Page 3: home g - Machine Learningcs229.stanford.edu/livenotes2020spring/cs229-livenotes... · 2020. 4. 16. · Multiclass Classification Cross Entropy minimization U2 g 00 O O 00 Lie K EIRD](https://reader034.vdocuments.mx/reader034/viewer/2022051822/5fec4864aa3beb75fc36d89d/html5/thumbnails/3.jpg)
bly ITcg gn log to q Csigmoid
acq logCto log l teenlog Iten
Gaussian w fixed variance or L
ply a En exp HIIe 5k expCpu y Zai
Tk Tk Tanbcg fan expC ETG yq µacne of IzProperties natural paramsMLE w r t 2 is concave
negative laglikelihood NLL is convex
EEg n a IqacnVarly y zigzag
![Page 4: home g - Machine Learningcs229.stanford.edu/livenotes2020spring/cs229-livenotes... · 2020. 4. 16. · Multiclass Classification Cross Entropy minimization U2 g 00 O O 00 Lie K EIRD](https://reader034.vdocuments.mx/reader034/viewer/2022051822/5fec4864aa3beb75fc36d89d/html5/thumbnails/4.jpg)
GLMAssumptions Design Clonolces
c yl K O Exponential familyReal GaussianBinary Bernoullicount PoissonRT Gamma ExponentialDist peta Dirichlet BayesianMystats
Il if 0Th O EIRDk E Rd
cu Test tune output ECyIn o
how IE yl x o
hoax log ply OTH
Framxi Ip
Learning Update RulehoCa
Rest
Oji o t x y hoax ajplug in appropriate holu
Terminologyq natural parameter
µ EE y R g y canonical response for
2 g A canonical link f n
![Page 5: home g - Machine Learningcs229.stanford.edu/livenotes2020spring/cs229-livenotes... · 2020. 4. 16. · Multiclass Classification Cross Entropy minimization U2 g 00 O O 00 Lie K EIRD](https://reader034.vdocuments.mx/reader034/viewer/2022051822/5fec4864aa3beb75fc36d89d/html5/thumbnails/5.jpg)
god znach3 paramatrizations
Modelparameters Natural param CanonicalParamBernoulli0 Y µ ot Gaussian
pi Entg X Poisson
Designgtlearning choice
Logistic Regressionholn Elgin o 9 n teen
Assumption'Legression
ammo A Fundo
Classification
xx xxxx x x0.5FEEI.eefzxxxxaxc xx
a
n1He 2
sigmoidLota of
o
![Page 6: home g - Machine Learningcs229.stanford.edu/livenotes2020spring/cs229-livenotes... · 2020. 4. 16. · Multiclass Classification Cross Entropy minimization U2 g 00 O O 00 Lie K EIRD](https://reader034.vdocuments.mx/reader034/viewer/2022051822/5fec4864aa3beb75fc36d89d/html5/thumbnails/6.jpg)
Softmax RegressionMulticlass Classification
Cross Entropy minimization
U2 g 00OO
00Lie
K E IRDy E 0,13 one hot vector 0 0,40
d
Glass C IRD f oOz
class c o o D 3 kg in i e oooo
IO l
IEnno
OOI ht.ee
l ll l
Goal a
oin
normalize
![Page 7: home g - Machine Learningcs229.stanford.edu/livenotes2020spring/cs229-livenotes... · 2020. 4. 16. · Multiclass Classification Cross Entropy minimization U2 g 00 O O 00 Lie K EIRD](https://reader034.vdocuments.mx/reader034/viewer/2022051822/5fec4864aa3beb75fc36d89d/html5/thumbnails/7.jpg)
PG
t 4
goal i mm distance between 2 dumba
man Cross Entropy pp I ply tog pacyYC 0,0 03log fly
00Thlog e
E eOEn
NgCEEQa o y
Fund Oo Oo 8D Gradient Descent