naïve&bayes - penn...
TRANSCRIPT
![Page 1: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/1.jpg)
Naïve Bayes
![Page 2: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/2.jpg)
Essen,al Probability Concepts • Marginaliza,on:
• Condi,onal Probability:
• Bayes’ Rule:
• Independence:
2
P (A | B) =P (A ^B)
P (B)
P (A ^B) = P (A | B)⇥ P (B)
A??B $ P (A ^B) = P (A)⇥ P (B)
$ P (A | B) = P (A)
P (A | B) =P (B | A)⇥ P (A)
P (B)
A??B | C $ P (A ^B | C) = P (A | C)⇥ P (B | C)
P (B) =X
v2values(A)
P (B ^A = v)
![Page 3: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/3.jpg)
Density Es,ma,on
3
![Page 4: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/4.jpg)
Recall the Joint Distribu,on...
4
alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake
burglary 0.01 0.08 0.001 0.009 ¬burglary 0.01 0.09 0.01 0.79
![Page 5: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/5.jpg)
How Can We Obtain a Joint Distribu,on? Op#on 1: Elicit it from an expert human
Op#on 2: Build it up from simpler probabilis,c facts • e.g, if we knew
P(a) = 0.7 ! !P(b|a) = 0.2 ! ! P(b|¬a) = 0.1" then, we could compute P(a ∧ b)"
Op#on 3: Learn it from data...
5 Based on slide by Andrew Moore
![Page 6: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/6.jpg)
Learning a Joint Distribu,on
Build a JD table for your aTributes in which the probabili,es are unspecified
Then, fill in each row with:
records ofnumber totalrow matching records)row(ˆ =P
A B C Prob 0 0 0 ?
0 0 1 ?
0 1 0 ?
0 1 1 ?
1 0 0 ?
1 0 1 ?
1 1 0 ?
1 1 1 ?
A B C Prob 0 0 0 0.30
0 0 1 0.05
0 1 0 0.10
0 1 1 0.05
1 0 0 0.05
1 0 1 0.10
1 1 0 0.25
1 1 1 0.10
Frac,on of all records in which A and B are true but C is false
Step 1: Step 2:
Slide © Andrew Moore
![Page 7: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/7.jpg)
Example of Learning a Joint PD This Joint PD was obtained by learning from three aTributes in the UCI “Adult” Census Database [Kohavi 1995]
Slide © Andrew Moore
![Page 8: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/8.jpg)
Density Es,ma,on • Our joint distribu,on learner is an example of something called Density Es#ma#on
• A Density Es,mator learns a mapping from a set of aTributes to a probability
Density Es,mator
Probability Input ATributes
Slide © Andrew Moore
![Page 9: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/9.jpg)
Regressor Predic,on of real-‐valued output
Input ATributes
Density Es,ma,on
Compare it against the two other major kinds of models:
Classifier Predic,on of categorical output
Input ATributes
Density Es,mator
Probability Input ATributes
Slide © Andrew Moore
![Page 10: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/10.jpg)
Evalua,ng Density Es,ma,on
Test set Accuracy
?
Test set Accuracy
Test-‐set criterion for es,ma,ng performance
on future data
Regressor Predic,on of real-‐valued output
Input ATributes
Classifier Predic,on of categorical output
Input ATributes
Density Es,mator
Probability Input ATributes
Slide © Andrew Moore
![Page 11: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/11.jpg)
• Given a record x, a density es,mator M can tell you how likely the record is:
• The density es,mator can also tell you how likely the dataset is: – Under the assump,on that all records were independently generated from the Density Es,mator’s JD (that is, i.i.d.)
Evalua,ng a Density Es,mator
P̂ (x | M)
P̂ (dataset | M) = P̂ (x1 ^ x2 ^ . . . ^ xn | M) =nY
i=1
P̂ (xi | M)
dataset
Slide by Andrew Moore
![Page 12: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/12.jpg)
Example Small Dataset: Miles Per Gallon From the UCI repository (thanks to Ross Quinlan) • 192 records in the training set
mpg modelyear maker
good 75to78 asiabad 70to74 americabad 75to78 europebad 70to74 americabad 70to74 americabad 70to74 asiabad 70to74 asiabad 75to78 america: : :: : :: : :bad 70to74 americagood 79to83 americabad 75to78 americagood 79to83 americabad 75to78 americagood 79to83 americagood 79to83 americabad 70to74 americagood 75to78 europebad 75to78 europe
Slide by Andrew Moore
![Page 13: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/13.jpg)
Example Small Dataset: Miles Per Gallon From the UCI repository (thanks to Ross Quinlan) • 192 records in the training set
mpg modelyear maker
good 75to78 asiabad 70to74 americabad 75to78 europebad 70to74 americabad 70to74 americabad 70to74 asiabad 70to74 asiabad 75to78 america: : :: : :: : :bad 70to74 americagood 79to83 americabad 75to78 americagood 79to83 americabad 75to78 americagood 79to83 americagood 79to83 americabad 70to74 americagood 75to78 europebad 75to78 europe
P̂ (dataset | M) =nY
i=1
P̂ (xi | M)
= 3.4⇥ 10�203 (in this case)
Slide by Andrew Moore
![Page 14: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/14.jpg)
Log Probabili,es • For decent sized data sets, this product will underflow
• Therefore, since probabili,es of datasets get so small, we usually use log probabili,es
log
ˆP (dataset | M) = log
nY
i=1
ˆP (xi | M) =
nX
i=1
log
ˆP (xi | M)
P̂ (dataset | M) =nY
i=1
P̂ (xi | M)
= 3.4⇥ 10�203 (in this case)
Based on slide by Andrew Moore
![Page 15: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/15.jpg)
Example Small Dataset: Miles Per Gallon From the UCI repository (thanks to Ross Quinlan) • 192 records in the training set
mpg modelyear maker
good 75to78 asiabad 70to74 americabad 75to78 europebad 70to74 americabad 70to74 americabad 70to74 asiabad 70to74 asiabad 75to78 america: : :: : :: : :bad 70to74 americagood 79to83 americabad 75to78 americagood 79to83 americabad 75to78 americagood 79to83 americagood 79to83 americabad 70to74 americagood 75to78 europebad 75to78 europe
Slide by Andrew Moore
log
ˆP (dataset | M) =
nX
i=1
log
ˆP (xi | M)
= �466.19 (in this case)
![Page 16: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/16.jpg)
Pros/Cons of the Joint Density Es,mator The Good News: • We can learn a Density Es,mator from data. • Density es,mators can do many good things… – Can sort the records by probability, and thus spot weird records (anomaly detec,on)
– Can do inference – Ingredient for Bayes Classifiers (coming very soon...)
The Bad News: • Density es,ma,on by directly learning the joint is trivial, mindless, and dangerous
Slide by Andrew Moore
![Page 17: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/17.jpg)
The Joint Density Es,mator on a Test Set
• An independent test set with 196 cars has a much worse log-‐likelihood – Actually it’s a billion quin,llion quin,llion quin,llion quin,llion ,mes less likely
• Density es,mators can overfit... ...and the full joint density es,mator is the overfikest of them all!
Slide by Andrew Moore
![Page 18: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/18.jpg)
Copyright © Andrew W. Moore
Overfikng Density Es,mators
If this ever happens, the joint PDE learns there are certain combina,ons that are impossible
log
ˆP (dataset | M) =
nX
i=1
log
ˆP (xi | M)
= �1 if for any i, ˆP (xi | M) = 0
Slide by Andrew Moore
![Page 19: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/19.jpg)
Curse of Dimensionality
Slide by Christopher Bishop
![Page 20: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/20.jpg)
The Joint Density Es,mator on a Test Set
• The only reason that the test set didn’t score -‐∞ is that the code was hard-‐wired to always predict a probability of at least 1/1020
We need Density Es-mators that are less prone to overfi7ng...
Slide by Andrew Moore
![Page 21: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/21.jpg)
The Naïve Bayes Classifier
21
![Page 22: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/22.jpg)
Bayes’ Rule • Recall Baye’s Rule:
• Equivalently, we can write:
where X is a random variable represen,ng the evidence and Y is a random variable for the label
• This is actually short for:
where Xj denotes the random variable for the j th feature 22
P (hypothesis | evidence) = P (evidence | hypothesis)⇥ P (hypothesis)
P (evidence)
P (Y = yk | X = xi) =P (Y = yk)P (X1 = xi,1 ^ . . . ^Xd = xi,d | Y = yk)
P (X1 = xi,1 ^ . . . ^Xd = xi,d)
P (Y = yk | X = xi) =P (Y = yk)P (X = xi | Y = yk)
P (X = xi)
![Page 23: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/23.jpg)
Naïve Bayes Classifier Idea: Use the training data to es,mate
Then, use Bayes rule to infer for new data
• Recall that es,ma,ng the joint probability distribu,on is not prac,cal
23
P (X | Y ) P (Y )and . P (Y |Xnew)
P (Y = yk | X = xi) =P (Y = yk)P (X1 = xi,1 ^ . . . ^Xd = xi,d | Y = yk)
P (X1 = xi,1 ^ . . . ^Xd = xi,d)
P (X1, X2, . . . , Xd | Y )
Easy to es,mate from data
Unnecessary, as it turns out
Imprac,cal, but necessary
![Page 24: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/24.jpg)
Naïve Bayes Classifier Problem: es,ma,ng the joint PD or CPD isn’t prac,cal – Severely overfits, as we saw before
However, if we make the assump,on that the aTributes are independent given the class label, es,ma,on is easy!
• In other words, we assume all aTributes are condi-onally independent given Y
• Onen this assump,on is violated in prac,ce, but more on that later…
24
P (X1, X2, . . . , Xd | Y ) =dY
j=1
P (Xj | Y )
![Page 25: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/25.jpg)
Training Naïve Bayes Es,mate and directly from the training data by coun,ng! P(play) = ? ! ! ! ! ! ! P(¬play) = ?"P(Sky = sunny | play) = ? ! ! P(Sky = sunny | ¬play) = ?"P(Humid = high | play) = ?! ! P(Humid = high | ¬play) = ?"... ! ! ! ! ! ! ! ! ! ..."
25
Sky Temp Humid Wind Water Forecast Play? sunny warm normal strong warm same yes sunny warm high strong warm same yes rainy cold high strong warm change no sunny warm high strong cool change yes
P (Xj | Y ) P (Y )
![Page 26: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/26.jpg)
Training Naïve Bayes Es,mate and directly from the training data by coun,ng! P(play) = ? ! ! ! ! ! ! P(¬play) = ?"P(Sky = sunny | play) = ? ! ! P(Sky = sunny | ¬play) = ?"P(Humid = high | play) = ?! ! P(Humid = high | ¬play) = ?"... ! ! ! ! ! ! ! ! ! ..."
26
Sky Temp Humid Wind Water Forecast Play? sunny warm normal strong warm same yes sunny warm high strong warm same yes rainy cold high strong warm change no sunny warm high strong cool change yes
P (Xj | Y ) P (Y )
![Page 27: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/27.jpg)
Training Naïve Bayes Es,mate and directly from the training data by coun,ng! P(play) = 3/4! ! ! ! ! ! P(¬play) = 1/4"P(Sky = sunny | play) = ? ! ! P(Sky = sunny | ¬play) = ?"P(Humid = high | play) = ?! ! P(Humid = high | ¬play) = ?"... ! ! ! ! ! ! ! ! ! ..."
27
Sky Temp Humid Wind Water Forecast Play? sunny warm normal strong warm same yes sunny warm high strong warm same yes rainy cold high strong warm change no sunny warm high strong cool change yes
P (Xj | Y ) P (Y )
![Page 28: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/28.jpg)
Training Naïve Bayes Es,mate and directly from the training data by coun,ng! P(play) = 3/4! ! ! ! ! ! P(¬play) = 1/4"P(Sky = sunny | play) = ? ! ! P(Sky = sunny | ¬play) = ?"P(Humid = high | play) = ?! ! P(Humid = high | ¬play) = ?"... ! ! ! ! ! ! ! ! ! ..."
28
Sky Temp Humid Wind Water Forecast Play? sunny warm normal strong warm same yes sunny warm high strong warm same yes rainy cold high strong warm change no sunny warm high strong cool change yes
P (Xj | Y ) P (Y )
![Page 29: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/29.jpg)
Training Naïve Bayes Es,mate and directly from the training data by coun,ng! P(play) = 3/4! ! ! ! ! ! P(¬play) = 1/4"P(Sky = sunny | play) = 1 ! ! P(Sky = sunny | ¬play) = ?"P(Humid = high | play) = ?! ! P(Humid = high | ¬play) = ?"... ! ! ! ! ! ! ! ! ! ..."
29
Sky Temp Humid Wind Water Forecast Play? sunny warm normal strong warm same yes sunny warm high strong warm same yes rainy cold high strong warm change no sunny warm high strong cool change yes
P (Xj | Y ) P (Y )
![Page 30: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/30.jpg)
Training Naïve Bayes Es,mate and directly from the training data by coun,ng! P(play) = 3/4! ! ! ! ! ! P(¬play) = 1/4"P(Sky = sunny | play) = 1 ! ! P(Sky = sunny | ¬play) = ?"P(Humid = high | play) = ?! ! P(Humid = high | ¬play) = ?"... ! ! ! ! ! ! ! ! ! ..."
30
Sky Temp Humid Wind Water Forecast Play? sunny warm normal strong warm same yes sunny warm high strong warm same yes rainy cold high strong warm change no sunny warm high strong cool change yes
P (Xj | Y ) P (Y )
![Page 31: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/31.jpg)
Training Naïve Bayes Es,mate and directly from the training data by coun,ng! P(play) = 3/4! ! ! ! ! ! P(¬play) = 1/4"P(Sky = sunny | play) = 1 ! ! P(Sky = sunny | ¬play) = 0"P(Humid = high | play) = ?! ! P(Humid = high | ¬play) = ?"... ! ! ! ! ! ! ! ! ! ..."
31
Sky Temp Humid Wind Water Forecast Play? sunny warm normal strong warm same yes sunny warm high strong warm same yes rainy cold high strong warm change no sunny warm high strong cool change yes
P (Xj | Y ) P (Y )
![Page 32: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/32.jpg)
Training Naïve Bayes Es,mate and directly from the training data by coun,ng! P(play) = 3/4! ! ! ! ! ! P(¬play) = 1/4"P(Sky = sunny | play) = 1 ! ! P(Sky = sunny | ¬play) = 0"P(Humid = high | play) = ?! ! P(Humid = high | ¬play) = ?"... ! ! ! ! ! ! ! ! ! ..."
32
Sky Temp Humid Wind Water Forecast Play? sunny warm normal strong warm same yes sunny warm high strong warm same yes rainy cold high strong warm change no sunny warm high strong cool change yes
P (Xj | Y ) P (Y )
![Page 33: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/33.jpg)
Training Naïve Bayes Es,mate and directly from the training data by coun,ng! P(play) = 3/4! ! ! ! ! ! P(¬play) = 1/4"P(Sky = sunny | play) = 1 ! ! P(Sky = sunny | ¬play) = 0"P(Humid = high | play) = 2/3 ! P(Humid = high | ¬play) = ?"... ! ! ! ! ! ! ! ! ! ..."
33
Sky Temp Humid Wind Water Forecast Play? sunny warm normal strong warm same yes sunny warm high strong warm same yes rainy cold high strong warm change no sunny warm high strong cool change yes
P (Xj | Y ) P (Y )
![Page 34: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/34.jpg)
Training Naïve Bayes Es,mate and directly from the training data by coun,ng! P(play) = 3/4! ! ! ! ! ! P(¬play) = 1/4"P(Sky = sunny | play) = 1 ! ! P(Sky = sunny | ¬play) = 0"P(Humid = high | play) = 2/3 ! P(Humid = high | ¬play) = ?"... ! ! ! ! ! ! ! ! ! ..."
34
Sky Temp Humid Wind Water Forecast Play? sunny warm normal strong warm same yes sunny warm high strong warm same yes rainy cold high strong warm change no sunny warm high strong cool change yes
P (Xj | Y ) P (Y )
![Page 35: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/35.jpg)
Training Naïve Bayes Es,mate and directly from the training data by coun,ng! P(play) = 3/4! ! ! ! ! ! P(¬play) = 1/4"P(Sky = sunny | play) = 1 ! ! P(Sky = sunny | ¬play) = 0"P(Humid = high | play) = 2/3 ! P(Humid = high | ¬play) = 1"... ! ! ! ! ! ! ! ! ! ..."
35
Sky Temp Humid Wind Water Forecast Play? sunny warm normal strong warm same yes sunny warm high strong warm same yes rainy cold high strong warm change no sunny warm high strong cool change yes
P (Xj | Y ) P (Y )
![Page 36: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/36.jpg)
Training Naïve Bayes Es,mate and directly from the training data by coun,ng! P(play) = 3/4! ! ! ! ! ! P(¬play) = 1/4"P(Sky = sunny | play) = 1 ! ! P(Sky = sunny | ¬play) = 0"P(Humid = high | play) = 2/3 ! P(Humid = high | ¬play) = 1"... ! ! ! ! ! ! ! ! ! ...""
36
Sky Temp Humid Wind Water Forecast Play? sunny warm normal strong warm same yes sunny warm high strong warm same yes rainy cold high strong warm change no sunny warm high strong cool change yes
P (Xj | Y ) P (Y )
![Page 37: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/37.jpg)
Laplace Smoothing • No,ce that some probabili,es es,mated by coun,ng might be zero – Possible overfikng!
• Fix by using Laplace smoothing: – Adds 1 to each count
where – cv is the count of training instances with a value of v for
aTribute j and class label yk!
– |values(Xj)| is the number of values Xj can take on 37
P (Xj = v | Y = yk) =cv + 1X
v02values(Xj)
cv0 + |values(Xj)|
![Page 38: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/38.jpg)
Training Naïve Bayes with Laplace Smoothing Es,mate and directly from the training data by coun,ng with Laplace smoothing: P(play) = 3/4! ! ! ! ! ! P(¬play) = 1/4"P(Sky = sunny | play) = 4/5 ! P(Sky = sunny | ¬play) = ?"P(Humid = high | play) = ?! ! P(Humid = high | ¬play) = ?"... ! ! ! ! ! ! ! ! ! ..."
38
Sky Temp Humid Wind Water Forecast Play? sunny warm normal strong warm same yes sunny warm high strong warm same yes rainy cold high strong warm change no sunny warm high strong cool change yes
P (Xj | Y ) P (Y )
![Page 39: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/39.jpg)
Training Naïve Bayes with Laplace Smoothing Es,mate and directly from the training data by coun,ng with Laplace smoothing: P(play) = 3/4! ! ! ! ! ! P(¬play) = 1/4"P(Sky = sunny | play) = 4/5 ! P(Sky = sunny | ¬play) = 1/3"P(Humid = high | play) = ?! ! P(Humid = high | ¬play) = ?"... ! ! ! ! ! ! ! ! ! ..."
39
Sky Temp Humid Wind Water Forecast Play? sunny warm normal strong warm same yes sunny warm high strong warm same yes rainy cold high strong warm change no sunny warm high strong cool change yes
P (Xj | Y ) P (Y )
![Page 40: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/40.jpg)
Training Naïve Bayes with Laplace Smoothing Es,mate and directly from the training data by coun,ng with Laplace smoothing: P(play) = 3/4! ! ! ! ! ! P(¬play) = 1/4"P(Sky = sunny | play) = 4/5 ! P(Sky = sunny | ¬play) = 1/3"P(Humid = high | play) = 3/5 ! P(Humid = high | ¬play) = ?"... ! ! ! ! ! ! ! ! ! ..."
40
Sky Temp Humid Wind Water Forecast Play? sunny warm normal strong warm same yes sunny warm high strong warm same yes rainy cold high strong warm change no sunny warm high strong cool change yes
P (Xj | Y ) P (Y )
![Page 41: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/41.jpg)
Training Naïve Bayes with Laplace Smoothing Es,mate and directly from the training data by coun,ng with Laplace smoothing: P(play) = 3/4! ! ! ! ! ! P(¬play) = 1/4"P(Sky = sunny | play) = 4/5 ! P(Sky = sunny | ¬play) = 1/3"P(Humid = high | play) = 3/5 ! P(Humid = high | ¬play) = 2/3"... ! ! ! ! ! ! ! ! ! ..."
41
Sky Temp Humid Wind Water Forecast Play? sunny warm normal strong warm same yes sunny warm high strong warm same yes rainy cold high strong warm change no sunny warm high strong cool change yes
P (Xj | Y ) P (Y )
![Page 42: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/42.jpg)
Using the Naïve Bayes Classifier • Now, we have
– In prac,ce, we use log-‐probabili,es to prevent underflow
• To classify a new point x,
42
P (Y = yk | X = xi) =P (Y = yk)
Qdj=1 P (Xj = xi,j | Y = yk)
P (X = xi)This is constant for a given instance, and so irrelevant to our predic,on
h(x) = argmax
yk
P (Y = yk)
dY
j=1
P (Xj = xj | Y = yk)
= argmax
yk
logP (Y = yk) +
dX
j=1
logP (Xj = xj | Y = yk)
j th aTribute value of x!
h(x) = argmax
yk
P (Y = yk)
dY
j=1
P (Xj = xj | Y = yk)
= argmax
yk
logP (Y = yk) +
dX
j=1
logP (Xj = xj | Y = yk)
![Page 43: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/43.jpg)
43
The Naïve Bayes Classifier Algorithm
• For each class label yk!– Es,mate P(Y = yk) from the data – For each value xi,j of each aTribute Xi!
• Es,mate P(Xi = xi,j | Y = yk)"
• Classify a new point via:
• In prac,ce, the independence assump,on doesn’t onen hold true, but Naïve Bayes performs very well despite it
h(x) = argmax
yk
logP (Y = yk) +
dX
j=1
logP (Xj = xj | Y = yk)
![Page 44: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/44.jpg)
Compu,ng Probabili,es (Not Just Predic,ng Labels)
• NB classifier gives predic,ons, not probabili,es, because we ignore P(X) (the denominator in Bayes rule)
• Can produce probabili,es by: – For each possible class label yk , compute
– α is given by
– Class probability is given by
44
P̃ (Y = yk | X = x) = P (Y = yk)dY
j=1
P (Xj = xj | Y = yk)
This is the numerator of Bayes rule, and is therefore off the true probability by a factor
of α that makes probabili,es sum to 1
↵ =1
P#classesk=1 P̃ (Y = yk | X = x)
P (Y = yk | X = x) = ↵P̃ (Y = yk | X = x)
![Page 45: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/45.jpg)
Naïve Bayes Applica,ons • Text classifica,on – Which e-‐mails are spam? – Which e-‐mails are mee,ng no,ces? – Which author wrote a document?
• Classifying mental states
People Words Animal Words
Learning P(BrainActivity | WordCategory)"
Pairwise Classifica,on Accuracy: 85%
![Page 46: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/46.jpg)
The Naïve Bayes Graphical Model
• Nodes denote random variables • Edges denote dependency • Each node has an associated condi,onal probability table (CPT), condi,oned upon its parents
46
ATributes (evidence)
Labels (hypotheses)
…
Y"
X1" Xi! Xd!…
P(Y)"
P(X1 | Y)" P(Xi | Y)" P(Xd | Y)"
![Page 47: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/47.jpg)
Example NB Graphical Model
47
…
Play
Sky Temp Humid …
Sky Temp Humid Play? sunny warm normal yes sunny warm high yes rainy cold high no sunny warm high yes
Data:
![Page 48: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/48.jpg)
Example NB Graphical Model
48
…
Play
Sky Temp Humid …
Sky Temp Humid Play? sunny warm normal yes sunny warm high yes rainy cold high no sunny warm high yes
Play? P(Play) yes no
Data:
![Page 49: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/49.jpg)
Example NB Graphical Model
49
…
Play
Sky Temp Humid …
Sky Temp Humid Play? sunny warm normal yes sunny warm high yes rainy cold high no sunny warm high yes
Play? P(Play) yes 3/4 no 1/4
Data:
![Page 50: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/50.jpg)
Example NB Graphical Model
50
…
Play
Sky Temp Humid …
Sky Temp Humid Play? sunny warm normal yes sunny warm high yes rainy cold high no sunny warm high yes
Play? P(Play) yes 3/4 no 1/4
Sky Play? P(Sky | Play) sunny yes rainy yes sunny no rainy no
Data:
![Page 51: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/51.jpg)
Example NB Graphical Model
51
…
Play
Sky Temp Humid …
Sky Temp Humid Play? sunny warm normal yes sunny warm high yes rainy cold high no sunny warm high yes
Play? P(Play) yes 3/4 no 1/4
Sky Play? P(Sky | Play) sunny yes 4/5 rainy yes 1/5 sunny no 1/3 rainy no 2/3
Data:
![Page 52: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/52.jpg)
Example NB Graphical Model
52
…
Play
Sky Temp Humid …
Sky Temp Humid Play? sunny warm normal yes sunny warm high yes rainy cold high no sunny warm high yes
Play? P(Play) yes 3/4 no 1/4
Sky Play? P(Sky | Play) sunny yes 4/5 rainy yes 1/5 sunny no 1/3 rainy no 2/3
Temp Play? P(Temp | Play) warm yes cold yes warm no cold no
Data:
![Page 53: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/53.jpg)
Example NB Graphical Model
53
…
Play
Sky Temp Humid …
Sky Temp Humid Play? sunny warm normal yes sunny warm high yes rainy cold high no sunny warm high yes
Play? P(Play) yes 3/4 no 1/4
Sky Play? P(Sky | Play) sunny yes 4/5 rainy yes 1/5 sunny no 1/3 rainy no 2/3
Temp Play? P(Temp | Play) warm yes 4/5 cold yes 1/5 warm no 1/3 cold no 2/3
Data:
![Page 54: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/54.jpg)
Example NB Graphical Model
54
…
Play
Sky Temp Humid …
Sky Temp Humid Play? sunny warm normal yes sunny warm high yes rainy cold high no sunny warm high yes
Play? P(Play) yes 3/4 no 1/4
Sky Play? P(Sky | Play) sunny yes 4/5 rainy yes 1/5 sunny no 1/3 rainy no 2/3
Temp Play? P(Temp | Play) warm yes 4/5 cold yes 1/5 warm no 1/3 cold no 2/3
Humid Play? P(Humid | Play) high yes norm yes high no norm no
Data:
![Page 55: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/55.jpg)
Example NB Graphical Model
55
…
Play
Sky Temp Humid …
Sky Temp Humid Play? sunny warm normal yes sunny warm high yes rainy cold high no sunny warm high yes
Play? P(Play) yes 3/4 no 1/4
Sky Play? P(Sky | Play) sunny yes 4/5 rainy yes 1/5 sunny no 1/3 rainy no 2/3
Temp Play? P(Temp | Play) warm yes 4/5 cold yes 1/5 warm no 1/3 cold no 2/3
Humid Play? P(Humid | Play) high yes 3/5 norm yes 2/5 high no 2/3 norm no 1/3
Data:
![Page 56: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/56.jpg)
Example NB Graphical Model
• Some redundancies in CPTs that can be eliminated 56
…
Play
Sky Temp Humid …
Play? P(Play) yes 3/4 no 1/4
Sky Play? P(Sky | Play) sunny yes 4/5 rainy yes 1/5 sunny no 1/3 rainy no 2/3
Temp Play? P(Temp | Play) warm yes 4/5 cold yes 1/5 warm no 1/3 cold no 2/3
Humid Play? P(Humid | Play) high yes 3/5 norm yes 2/5 high no 2/3 norm no 1/3
![Page 57: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/57.jpg)
Example Using NB for Classifica,on
Goal: Predict label for x = (rainy, warm, normal) 57
…
Play
Sky Temp Humid …
Play? P(Play) yes 3/4 no 1/4
Sky Play? P(Sky | Play) sunny yes 4/5 rainy yes 1/5 sunny no 1/3 rainy no 2/3
Temp Play? P(Temp | Play) warm yes 4/5 cold yes 1/5 warm no 1/3 cold no 2/3
Humid Play? P(Humid | Play) high yes 3/5 norm yes 2/5 high no 2/3 norm no 1/3
h(x) = argmax
yk
logP (Y = yk) +
dX
j=1
logP (Xj = xj | Y = yk)
![Page 58: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/58.jpg)
Example Using NB for Classifica,on
Predict label for: x = (rainy, warm, normal)
58
…
Play
Sky Temp Humid …
Play? P(Play) yes 3/4 no 1/4
Sky Play? P(Sky | Play) sunny yes 4/5 rainy yes 1/5 sunny no 1/3 rainy no 2/3
Temp Play? P(Temp | Play) warm yes 4/5 cold yes 1/5 warm no 1/3 cold no 2/3
Humid Play? P(Humid | Play) high yes 3/5 norm yes 2/5 high no 2/3 norm no 1/3
predict PLAY
![Page 59: Naïve&Bayes - Penn Engineeringcis519/fall2017/lectures/09_NaiveBayes.pdfRecall&the&JointDistribu,on...& 4 alarm ¬alarm earthquake ¬earthquake earthquake ¬earthquake burglary 0.01](https://reader034.vdocuments.mx/reader034/viewer/2022043023/5f3f25a94bfbb400a246f898/html5/thumbnails/59.jpg)
Naïve Bayes Summary Advantages: • Fast to train (single scan through data) • Fast to classify • Not sensi,ve to irrelevant features • Handles real and discrete data • Handles streaming data well
Disadvantages: • Assumes independence of features
59 Slide by Eamonn Keogh