lecture 4a : probability theory review
DESCRIPTION
Advanced Artificial Intelligence. Lecture 4A : Probability Theory Review. Outline. Axioms of Probability Product and chain rules Bayes Theorem Random variables PDFs and CDFs Expected value and variance. Introduction. - PowerPoint PPT PresentationTRANSCRIPT
Lecture 4A: Probability Theory Review
Advanced Artificial Intelligence
Outline
• Axioms of Probability• Product and chain rules • Bayes Theorem• Random variables• PDFs and CDFs• Expected value and variance
Introduction
• Sample space - set of all possible outcomes of a random experiment– Dice roll: {1, 2, 3, 4, 5, 6}– Coin toss: {Tails, Heads}
• Event space - subsets of elements in a sample space– Dice roll: {1, 2, 3} or {2, 4, 6}– Coin toss: {Tails}
• Axioms of Probability– for all
examples
• Coin flip– P(H)– P(T)– P(H,H,H)– P(x1=x2=x3=x4)– P({x1,x2,x3,x4} contains more than 3 heads)
Set operations
• Let and • and
• Properties:
• If then
Conditional Probability
• A and B are independent if
• A and B are conditionally independent given C if
Conditional Probability
• Joint probability:
• Product rule:
• Chain rule of probability:
examples
• Coin flip– P(x1=H)=1/2– P(x2=H|x1=H)=0.9– P(x2=T|x1=T)=0.8– P(x2=H)=?
Conditional Probability
• probability of A given B• Example:– probability that school is closed – probability that it snows– probability of school closing if it snows– probability of snowing if school is closed
Conditional Probability
• probability of A given B• Example:– probability that school is closed – probability that it snows
P(A, B) 0.005
P(B) 0.02
P(A|B) 0.25
𝑃 ( 𝐴∨𝐵 )= 𝑃 ( 𝐴∩𝐵 )𝑃 (𝐵 )
Quiz
• P(D1=sunny)=0.9• P(D2=sunny|D1=sunny)=0.8• P(D2=rainy|D1=sunny)=?• P(D2=sunny|D1=rainy)=0.6• P(D2=rainy|D1=rainy)=?• P(D2=sunny)=?• P(D3=sunny)=?
Joint Probability
• Multiple events: cancer, test result
13
Has cancer? Test positive? P(C,TP)
yes yes 0.018
yes no 0.002
no yes 0.196
no no 0.784
Joint Probability
• The problem with joint distributions
It takes 2D-1 numbers to specify them!
14
Conditional Probability
• Describes the cancer test:
• Put this together with: Prior probability
15
Has cancer? Test positive?
P(TP, C)
yes yes
yes no
no yes
no no
Has cancer? Test positive? P(TP, C)
yes yes 0.018
yes no 0.002
no yes 0.196
no no 0.784
Conditional Probability
• We have:
• We can now calculate joint probabilities
16
Conditional Probability
• “Diagnostic” question: How likely do is cancer given a positive test?
17
Has cancer? Test positive? P(TP, C)
yes yes 0.018
yes no 0.002
no yes 0.196
no no 0.784
Bayes Theorem
• We can relate P(A|B) and P(B|A) through Bayes’ rule:
Bayes Theorem
• We can relate P(A|B) and P(B|A) through Bayes’ rule:
Posterior Probability Likelihood
Normalizing Constant
Prior Probability
Bayes Theorem
• We can relate P(A|B) and P(B|A) through Bayes’ rule:
• can be eliminated with law of total probability:
=
Random Variables
• Random variable X is a function s.t.
• Examples:– Russian roulette: X = 1 if gun fires and X = 0 otherwise and – X = # of heads in 10 coin tosses
Cumulative Distribution Functions
• Defined as such that • Properties:
Probability Density Functions
• For discrete random variables, defined as:
• Relates to CDFs:
• =
Probability Density Functions
• For continuous random variables, defined as:
• Relates to CDFs:
Probability Density Functions
• Example:• Let X be the angular position of freely spinning pointer on
a circle
Probability Density Functions
• Example:• Let X be the angular position of freely spinning pointer on
a circle f(X)
1/2
2
X
Probability Density Functions
• Example:• Let X be the angular position of freely spinning pointer on
a circle f(X)
1/2
2
X
Probability Density Functions
• Example:• Let X be the angular position of freely spinning pointer on
a circle
• for
f(x)
1/2
2
x
F(x)
1
2
x
Probability Density Functions
• Example:• Let X be the angular position of freely spinning pointer on
a circle
• for • =
f(x)
1/2
2
x
F(x)
1
2
x
Expectation
• Given a discrete r.v. X and a function
• For a continuous r.v. X and a function
Expectation
• Expectation of a r.v. X is known as the first moment, or the mean value, of that variable ->
• Properties:• for any constant • for any constant • +
Variance
• For a random variable X, define:
• Another common form:
• Properties:• for any constant • for any constant
Gaussian Distributions
• Let , then
where represents the mean and the variance• Occurs naturally in many phenomena– Noise/error– Central limit theorem