mathematics for machine learning - henrik bachmann · 2021. 1. 13. · recall: logistic regression...
TRANSCRIPT
![Page 1: Mathematics for Machine Learning - Henrik Bachmann · 2021. 1. 13. · Recall: Logistic regression Logistic regression Hypothesis: Example: Binary classifier (Pass an exam Yes/No,](https://reader036.vdocuments.mx/reader036/viewer/2022071603/613e5ee459df642846167cce/html5/thumbnails/1.jpg)
Mathematics for Machine LearningSpecial Mathematics LectureNagoya University, Fall 2020
https://www.henrikbachmann.com/mml_2020.html
Lecture 11: Neural Networks I
![Page 2: Mathematics for Machine Learning - Henrik Bachmann · 2021. 1. 13. · Recall: Logistic regression Logistic regression Hypothesis: Example: Binary classifier (Pass an exam Yes/No,](https://reader036.vdocuments.mx/reader036/viewer/2022071603/613e5ee459df642846167cce/html5/thumbnails/2.jpg)
Machine learning overview
Remaining plan:• Introduce (simple) Neural Networks (Today)• Understand how to train them (Next time)• Consider more complicated NN.• Use finished implementations, e.g. Tensorflow.
![Page 3: Mathematics for Machine Learning - Henrik Bachmann · 2021. 1. 13. · Recall: Logistic regression Logistic regression Hypothesis: Example: Binary classifier (Pass an exam Yes/No,](https://reader036.vdocuments.mx/reader036/viewer/2022071603/613e5ee459df642846167cce/html5/thumbnails/3.jpg)
Nice to watch
https://www.3blue1brown.com/neural-networks
https://nnfs.io/
3blue1brown
Neural Networks from Scratch
![Page 4: Mathematics for Machine Learning - Henrik Bachmann · 2021. 1. 13. · Recall: Logistic regression Logistic regression Hypothesis: Example: Binary classifier (Pass an exam Yes/No,](https://reader036.vdocuments.mx/reader036/viewer/2022071603/613e5ee459df642846167cce/html5/thumbnails/4.jpg)
Neural Networks
![Page 5: Mathematics for Machine Learning - Henrik Bachmann · 2021. 1. 13. · Recall: Logistic regression Logistic regression Hypothesis: Example: Binary classifier (Pass an exam Yes/No,](https://reader036.vdocuments.mx/reader036/viewer/2022071603/613e5ee459df642846167cce/html5/thumbnails/5.jpg)
Recall: Logistic regression
Logistic regression
Hypothesis:
Example: Binary classifier (Pass an exam Yes/No, Spam email Yes/no)
We learned the correct parameters by maximizing the log-likelihood (by using gradient ascent)
Or mizimizing the negative of the log-likelihood (= cost function) (by using gradient descent)
c galt
S
Oot.fit tOdXd
lXEIRdXo I
oft
![Page 6: Mathematics for Machine Learning - Henrik Bachmann · 2021. 1. 13. · Recall: Logistic regression Logistic regression Hypothesis: Example: Binary classifier (Pass an exam Yes/No,](https://reader036.vdocuments.mx/reader036/viewer/2022071603/613e5ee459df642846167cce/html5/thumbnails/6.jpg)
Rewriting logistic regression as a neural network
Neuron linearpartInput f activation
stürmtlinear 5 activation R
weight Part function
WEIRbFpbias
oldNotationdb Oo
![Page 7: Mathematics for Machine Learning - Henrik Bachmann · 2021. 1. 13. · Recall: Logistic regression Logistic regression Hypothesis: Example: Binary classifier (Pass an exam Yes/No,](https://reader036.vdocuments.mx/reader036/viewer/2022071603/613e5ee459df642846167cce/html5/thumbnails/7.jpg)
Motivation: Image classification
Goal: Check if a picture contains a cati upper index layer ilower index neuron
Picture Z linearpart a Output100
100 D ü dumb
Pixel 3 colorsEtwas 9
A 1001003
![Page 8: Mathematics for Machine Learning - Henrik Bachmann · 2021. 1. 13. · Recall: Logistic regression Logistic regression Hypothesis: Example: Binary classifier (Pass an exam Yes/No,](https://reader036.vdocuments.mx/reader036/viewer/2022071603/613e5ee459df642846167cce/html5/thumbnails/8.jpg)
Motivation: Image classification
Goal: Check if a picture contains a cat or dog
![Page 9: Mathematics for Machine Learning - Henrik Bachmann · 2021. 1. 13. · Recall: Logistic regression Logistic regression Hypothesis: Example: Binary classifier (Pass an exam Yes/No,](https://reader036.vdocuments.mx/reader036/viewer/2022071603/613e5ee459df642846167cce/html5/thumbnails/9.jpg)
Motivation: Image classification. More layers
Goal: Check if a picture contains a cat or dog as Hidden layers
WP
i
iii iii
id
![Page 10: Mathematics for Machine Learning - Henrik Bachmann · 2021. 1. 13. · Recall: Logistic regression Logistic regression Hypothesis: Example: Binary classifier (Pass an exam Yes/No,](https://reader036.vdocuments.mx/reader036/viewer/2022071603/613e5ee459df642846167cce/html5/thumbnails/10.jpg)
(rough) Notation
O
![Page 11: Mathematics for Machine Learning - Henrik Bachmann · 2021. 1. 13. · Recall: Logistic regression Logistic regression Hypothesis: Example: Binary classifier (Pass an exam Yes/No,](https://reader036.vdocuments.mx/reader036/viewer/2022071603/613e5ee459df642846167cce/html5/thumbnails/11.jpg)
Content of a layer
weil Iii
III i am
![Page 12: Mathematics for Machine Learning - Henrik Bachmann · 2021. 1. 13. · Recall: Logistic regression Logistic regression Hypothesis: Example: Binary classifier (Pass an exam Yes/No,](https://reader036.vdocuments.mx/reader036/viewer/2022071603/613e5ee459df642846167cce/html5/thumbnails/12.jpg)
Content of a layer
![Page 13: Mathematics for Machine Learning - Henrik Bachmann · 2021. 1. 13. · Recall: Logistic regression Logistic regression Hypothesis: Example: Binary classifier (Pass an exam Yes/No,](https://reader036.vdocuments.mx/reader036/viewer/2022071603/613e5ee459df642846167cce/html5/thumbnails/13.jpg)
Example of activation functionsThere are several common activiation functions.
https://en.wikipedia.org/wiki/Activation_function
Question: Why not use the identity map as an activation function?F
![Page 14: Mathematics for Machine Learning - Henrik Bachmann · 2021. 1. 13. · Recall: Logistic regression Logistic regression Hypothesis: Example: Binary classifier (Pass an exam Yes/No,](https://reader036.vdocuments.mx/reader036/viewer/2022071603/613e5ee459df642846167cce/html5/thumbnails/14.jpg)
Example of NNLet us consider a NN with 4 layers (2 hidden):
I IRT
ER
amwaz.pk EEtRybIp3a
faftlIIYI Köln b