2303 probability bayes f12
DESCRIPTION
It's really good.TRANSCRIPT
INTRODUCTION TO INTRODUCTION TO PROBABILITYPROBABILITY
andandBAYES THEOREMBAYES THEOREM
Introduction to probabilityIntroduction to probability
• Basics of probability• Picturing probability: Venn diagrams, Tree
diagrams• Probability rules
– Complement rule
– Addition rules
– Multiplication rules
– Conditional probability
– Independence
– Reversing the conditioning: Bayes’ Theorem
Dealing with Random PhenomenaDealing with Random Phenomena
• A random phenomenon is a situation in which we know what outcomes could happen, but we don’t know which particular outcome did or will happen.
• When dealing with probability, we will be dealing with many random phenomena
ProbabilityProbability
ProbabilityProbability refers to the chance that a particular refers to the chance that a particular event will occur.event will occur.
The probability of an event will be a value in the range 0.00 to 1.00.
A value of 0.00 means the event will not occur (impossibility).
A probability of 1.00 means the event will occur (certainty).
Anything between 0.00 and 1.00 reflects the uncertainty of the event occurring.
Events and Sample Space:Events and Sample Space:ExperimentExperiment
An experimentexperiment is a process that produces a single outcome whose result cannot be
predicted with certainty.
Events and Sample Space:Events and Sample Space:Elementary EventsElementary Events
Elementary eventsElementary events are the most rudimentary outcomes resulting from a
simple experiment.
Events and Sample SpaceEvents and Sample Space
The sample space (S)sample space (S) is the collection of all elementary outcomes that can result from
an experiment.
Events and Sample SpaceEvents and Sample Space
An eventevent is a collection of elementary events.
Example 1:Example 1:Sample space and eventsSample space and events
• An experiment consists of randomly drawing three components from a batch production and checking whether they are defective or not.
• Let’s define the events:A: acceptable component
D: defective component
• Determine the sample space S for this experiment (tree diagram)
A
D
Example 1Example 1Sample space and eventsSample space and events
1st drawing
A
D
A
D
A
D
1st drawing 2nd drawing
Example 1Example 1Sample space and eventsSample space and events
A
D
A
D
A
D
D
A
A
D
A
D
A
D
1st drawing 2nd drawing 3rd drawing
Example 1Example 1Sample space and eventsSample space and events
A
D
A
D
A
D
D
A
A
D
A
D
A
D
Sample space (S)
AAA
AAD
ADA
ADD
DAA
DAD
DDA
DDD
Eig
ht
ele
men
tary
e
ven
ts1st drawing 2nd drawing 3rd drawing Possible outcomes
Example 1Example 1Sample space and eventsSample space and events
• B and C are two events defined as follows:
B: three components are acceptableB = {(AAA)}
• C: at least one component is acceptableC = {(ADD),(DAD),(DDA),(AAD),(ADA),(DAA),(AAA)}
Example 1Example 1Sample space and eventsSample space and events
Picturing ProbabilitiesPicturing Probabilities
• Tree diagrams can be used to define the sample space of an experiment
• Also, the most common kind of picture to make is called a Venn diagram.
• We will see Venn diagrams in practice shortly…
Mutually Exclusive Events (or Mutually Exclusive Events (or Disjoint Events)Disjoint Events)
Two events are mutually exclusivemutually exclusive if the occurrence of one event precludes the
occurrence of another event.
AS
B
• B: Three components are acceptable
B = {(AAA)}• Let E: at least one component is defective
E = {(AAD),(ADA),(DDA),(ADD),(DAA),(DAD),(DDD)}
• B and E are disjoint or mutually exclusive events
Example 1 (cont.)Example 1 (cont.)Two disjoint eventsTwo disjoint events
Trial, Experiment
Sample space (S) = all possible outcomes
Trial, Experiment
Sample space (S) = all possible outcomes
A combination of outcomes
Event (Ei)
Elementary: one outcome of S
A B
A B
Trial, Experiment
Sample space (S) = all possible outcomes
A combination of outcomes
Event (Ei)
Elementary: one outcome of S
Disjoint = No outcomes in common Non disjoint = can occur together, =
Mutually exclusive outcomes in common
SA SB
Four Types of ProbabilityFour Types of Probability
Marginal
The probability of A occurring
Union
The probability of A or B occurring
Joint
The probability of A and B occurring
Conditional
The probability of A occurring given that B has occurred
BA BA
B
A
)(AP )( BAP )( BAP )|( BAP
Independent and Dependent Independent and Dependent EventsEvents
Two events are independent independent if the occurrence of one event in no
way influences the probability of the occurrence of the other event.
See later Probability Rules 4 and 4a
Being independent is
a totally different thing
that being mutually exclusive
Independent and Dependent Independent and Dependent EventsEvents
Two events are dependent dependent if the occurrence of one event impacts the
probability of the other event occurring.
See later Probability Rules 4 and 4a
Assessing ProbabilitiesAssessing Probabilities
• Theoretical: Number of Possibilities• Relative Frequency• Subjective
Theoretical Probability Theoretical Probability AssessmentAssessment
Theoretical Probability AssessmentTheoretical Probability Assessment refers to the method of determining probability based on the ratio of the
number of ways the event of interest can occur to the total number of ways any event can occur when the
individual elementary events are equally likely.
THEORETICAL PROBABILITY MEASUREMENTTHEORETICAL PROBABILITY MEASUREMENT
Theoretical Probability Theoretical Probability AssessmentAssessment
events elementary ofnumber Total
occurcan E waysofNumber )P(E i
i
• What is the probability that three components are acceptable?
B: three components are acceptable– B = {(AAA)}
– P(B) = 1/8
• What is the probability that at least one component is acceptable?
C: at least one component is acceptable– C = {(ADD),(DAD),(DDA),(AAD),(ADA),(DAA),(AAA)}
– P(C) = 7/8
Example 1 (cont.)Example 1 (cont.)Theoretical probability assessmentTheoretical probability assessment
Relative Frequency of Relative Frequency of OccurrenceOccurrence
Relative Frequency of OccurrenceRelative Frequency of Occurrence refers to a method that defines probability as the number of times an
event occurs, divided by the total number of times an experiment is performed in a large number of trials.
Relative Frequency of Relative Frequency of OccurrenceOccurrence
RELATIVE FREQUENCY OF OCCURRENCERELATIVE FREQUENCY OF OCCURRENCE
where:
Ei = the event of interest
RF(Ei) = the relative frequency of Ei occurring
n = number of trials
noccurs E timesof Number
RF(E ii )
Relative Frequency of Relative Frequency of OccurrenceOccurrence
Relative Frequency of OccurrenceRelative Frequency of OccurrenceExample 2Example 2
On a production line products are testedthroughout a day.Out of 10000 products, 52 are found to bedefective.
Probability of defective product = 52/10000= 0.52%
Law of Large Numbers (LLN)Law of Large Numbers (LLN)
• If the events are independent, then as the number of trials increases, the long-run relative frequency of an event gets closer and closer to a single value.
Subjective Probability Subjective Probability AssessmentAssessment
Subjective Probability AssessmentSubjective Probability Assessment refers to the method that defines probability of an event as
reflecting a decision maker’s state of mind regarding the chances that the particular event
will occur.
Partner in consulting company puts in a bid fora contract, and subjectively assesses the chanceof getting the contract as 70%.
The Rules of ProbabilityThe Rules of Probability
PROBABILITY RULE 1PROBABILITY RULE 1
For any event Ei
0.0 P(Ei) 1.0 for all i
The Complement of an EventThe Complement of an Event
The complement complement of an event E is the collection of all possible elementary
events not contained in event E. The complement of event E is represented by
E or EC or E’
The complement of E is the same as “not E”
The Rules of ProbabilityThe Rules of Probability
PROBABILITY RULE 2PROBABILITY RULE 2
)(1)( EPECP
E EC
Sample Space
• We pick a component at random from a production line
• The inspection reveals that a component can be good (G) or defective (D)
• Two disjoint outcomes are possible : S = {G,D}• G = DC and D = GC
• If P(G) = 0.90 determine P(D) = ?P(D) = P(S) - P(G) = 1 - P(G) = 1 – 0.90 = 0.10
Example 3:Example 3:The complement ruleThe complement rule
The Rules of ProbabilityThe Rules of Probability
PROBABILITY RULE 3PROBABILITY RULE 3
Addition or “OR” rule for any two events E1 and E2:
P(E1 or E2) = P(E1) + P(E2) - P(E1 and E2)
E1 E2
2E 1E and
The Rules of ProbabilityThe Rules of Probability
SPECIAL CASE OF THE “OR” RULE SPECIAL CASE OF THE “OR” RULE
“OR” rule for mutually exclusive (disjoint) events E1 and E2:
P(E1 or E2) = P(E1) + P(E2)
• Let A and B two disjoint events of S, with P(A) = 0.4 and P(B) = 0.45, we have:
Example 4Example 4Addition rule for two disjoint eventsAddition rule for two disjoint events
0.4=0-0.4=B) andP(A -P(A)=)B andP(A
0.15=0.85-1=
B)or P(A -1= B)or P(A =)B and P(A
0.85=0.45+0.4=P(B)+P(A)=B)or P(A
0=B) andP(A
0.6=0.4-1=P(A)-1=)P(A
C
CCC
C
A SB
• 400 students were recently interviewed concerning the newspapers they read.
• The study revealed that:– A: 165 read « The Citizen"
– B: 240 read « The National Post"
– A and B: 90 read both
Example 5Example 5Addition rule for any two eventsAddition rule for any two events
A B
A B
• 400 students were recently interviewed concerning the newspapers they read.
• The study revealed that:– A: 165 read « The Citizen"– B: 240 read « The National Post"– A and B: 90 read both
• What is the probability that a student reads “The Citizen”; “The National Post”
Example 5Example 5Addition rule for any two eventsAddition rule for any two events
A B
A B
• 400 students were recently interviewed concerning the newspapers they read.
• The study revealed that:– A: 165 read « The Citizen"
– B: 240 read « The National Post"
– A and B: 90 read both
• What is the probability that a student reads “The Citizen”; “The National Post”
Example 5Example 5Addition rule for any two eventsAddition rule for any two events
A B
A B
0.60400
240P(B) and 0.4125
400
165P(A)
Example 5Example 5Addition rule for any two eventsAddition rule for any two events
• How many students read only The Citizen (A); only the National Post (B); None of them?
A B
B: Nat. Post BC:(Nat. Post)C
A : The Citizen 90 165
AC: (The Citiz.)C
240 400
A B
A B
Example 5Example 5Addition rule for any two eventsAddition rule for any two events
• How many students read only The Citizen (A); only the National Post (B); None of them?
A B
B: Nat. Post BC:(Nat. Post)C
A : The Citizen 90 75 165
AC: (The Citiz.)C 150 85 235
240 160 400
A B
A B
• Illustrating using Venn diagram:
75
S85
90 150
Example 5Example 5Addition rule for any two eventsAddition rule for any two events
A B
A B
• What is the probability that a student reads either of these two newspapers? (Think OR)
Example 5Example 5Addition rule for any two eventsAddition rule for any two events
A B
A B
78.75%=400
315=
400
90-
400
240+
400
165=
B) andP(A -P(B)+P(A)=B)or P(A
Conditional ProbabilityConditional Probability
Conditional probabilityConditional probability refers to the probability that an event will occur given
that some other event has already happened.
The Rules of ProbabilityThe Rules of Probability
PROBABILITY RULE 4PROBABILITY RULE 4
Conditional probability for any two events E1 , E2:
0)(
)(
)()|(
2
2
2121
EP
EP
EandEPEEP
Definition of IndependenceDefinition of Independence
• Independence of two events means that the outcome of one event does not influence the probability of the other.
• With our new notation for conditional probabilities, we can now formalize this definition:– Events A and B are independent whenever
P(B|A) = P(B).
(Likewise, events A and B are independent whenever P(A|B) = P(A).)
• A: The consumer bought the product
• B: The consumer saw the commercial on the TV
A: bought Not_A: did'nt buy Total
B: saw the comm. 5 25 30Not_B: did'nt see the commercial 15 55 70Total 20 80 100
Example 6 Example 6 Conditional probability and Conditional probability and
independenceindependence
• A: The consumer bought the product
• B: The consumer saw the commercial on the TV
A: bought Not_A: did'nt buy Total
B: saw the comm. 5 25 30Not_B: did'nt see the commercial 15 55 70Total 20 80 100
30%=10030=P(B) and 20%
10020P(A)
Example 6 Example 6 Conditional probabilityConditional probability
• What is the probability that the consumer bought the product (A) and saw the commercial (B)?
%5100
5) and ( BAP
Example 6 Example 6 Conditional probabilityConditional probability
Example 6 Example 6 Conditional probabilityConditional probability
• What is the probability that the consumer bought the product (A) and saw the commercial (B)?
%5100
5) and ( BAP
• What is the probability that the consumer bought the product (A) given that he saw the commercial (B)?
%67.16100/30
100/5
)(
) and ()/(
BP
BAPBAP
The Rules of ProbabilityThe Rules of Probability
PROBABILITY RULE 4aPROBABILITY RULE 4a
Multiplication or “AND” rule for any two events
E1 and E2:
)|()()(
)|()()(
21212
12121
EEPEPEandEP
and
EEPEPEandEP
Rule 4 revisited
The Rules of ProbabilityThe Rules of Probability
SPECIAL CASE OF THE “AND” RULESPECIAL CASE OF THE “AND” RULE
Multiplication or “and” rule for two independent events
E1 , E2:
)()()( 2121 EPEPEandEP
Recall: Definition of independence: Two events are independent if:P(A|B)=P(A)
Independent IS NOT THE SAME Independent IS NOT THE SAME AS Mutually ExclusiveAS Mutually Exclusive
Independent Check whether P(B|A) = P(B)OrCheck whether P(A|B) = P(A)OrCheck whether P(A and B) = P(A) * P(B)
Disjoint (mutually exclusive)
Check whether P(A and B) = 0OrCheck whether the events A and B overlap in the sample space diagram.OrCheck whether both events can occur together
• Are the events A and B independent?– We have to prove that:
– Step 1:
– Step 2:
– Step 3:
Thus, A and B are not independent
Example 6 (cont.) Example 6 (cont.) IndependenceIndependence
P(A B) = P(A) P(B)
P(A B) = 5%
5% 6%%660.0)3.0()2.0()()( BPAP
Tree DiagramsTree Diagrams
• A tree diagram helps us think through conditional probabilities by showing sequences of events as paths that look like branches of a tree.
Example 7Example 7
• A construction Company has a 60% chance of obtaining an important contract. If this first contract is obtained, the company has a 2/3 chance of getting a second contract.
• If it doesn't obtain the first contract, its chances of obtaining the second one fall to 30%.
• What is the probability that the company obtains the second contract?
• Let:– A: The company obtains the first contract– B: The company obtains the second contract
• We know the following:– P(A) = 0.6 thus P(AC) = 0.4– P(B/A) = 2/3 thus P(BC/A) = 1/3– P(B/AC) = 0.3 thus P(BC/AC) = 0.7
• We have to find P(B) given:– P(A), and– The conditional probabilities
Example 7Example 7
Example 7Example 7
SA A’
B
P(B) = ?
P B P A B P A B( ) ( ) ( ' )
P(A)=0.6
P(A')=0.4
Example 7Example 7
P(A)=0.6
P(A')=0.4
P(B/A)=2/3
P(B'/A)=1/3
P(B/A')=0.3
P(B'/A')=0.7
Example 7Example 7
P(A)=0,6
P(A')=0,4
P(B/A)=2/3
P(B'/A)=1/3
P(B/A')=0.3
P(B'/A')=0.7
P A B P A P B A( ) ( ) ( / )
P A B' P A P B'/A( ) ( ) ( )
P A B P A P B A( ' ) ( ' ) ( / ' )
P A B' P A P B'/A( ' ) ( ' ) ( ' )
Example 7Example 7
P(A)=0,6
P(A')=0,4
P(B/A)=2/3
P(B'/A)=1/3
P(B/A')=0.3
P(B'/A')=0.7
P A B P A P B A( ) ( ) ( / )
P A B' P A P B'/A( ) ( ) ( )
P A B P A P B A( ' ) ( ' ) ( / ' )
P A B' P A P B'/A( ' ) ( ' ) ( ' )
=(.6)(2/3)=.4
=(.6)(1/3)=.2
=(.4)(.3)=.12
=(.4)(.7)=.28
Example 7Example 7
Example 7Example 7
SA A’
B
%5252.012.040.0
)'/()'()/()(
)'()()(
ABPAPABPAP
BAPBAPBP
40.0)( BAP 12.0)'( BAP
Let A be an event of S and E1, E2, ..., En, a set of mutually exclusive and exhaustive events of S
GeneralizationGeneralization
E1
A
E3
E5
E4
En
E2A A E A E
A E A En
( ) ( )
( ) ( )1 2
3
P Eii
n
( ) 1
1
Let A be an event of S and E1, E2, ..., En, a set of mutually exclusive and collectively exhaustive events of S
:
P A P A E P A E P A E n( ) ( ) ( ) ( ) 1 2
A A E A E
A E A En
( ) ( )
( ) ( )1 2
3
GeneralizationGeneralization
E1
A
E3
E5
E4
En
E2
P Eii
n
( ) 1
1
Let A be an event of S and E1, E2, ..., En, a set of mutually exclusive and collectively exhaustive events of S
P A P A E P A E P A E n( ) ( ) ( ) ( ) 1 2
)E/A(P)E(P
)E/A(P)E(P)E/A(P)E(P)A(P
nn
2211
)E/A(P)E(P)A(P i
n
1ii
Generalization: Probability rule 10Generalization: Probability rule 10
ExampleExample 8 8
• A manufacturing company produces mechanical components using 3 machines: M1, M2 and M3 in the following proportions: 50% from M1, 30% from M2 and 20% from M3.
• The probability of having a defective component given that it was produced by machine M1, M2 and M3 are 3%, 4% and 5% respectively.
• We pick a component at random from the components produced during the day.
• What is the probability that the component is defective?
• We want to know the probability of the event:– A: The component is defective
• Let:– E1: The picked component comes from M1
– E2: The picked component comes from M2
– E3: The picked component comes from M3
• E1, E2 and E3 are mutually exclusive
ExampleExample 8 8
• We don’t have the probability of A (directly), but we have the following probabilities:– P(E1) = 0.5 P(E2) = 0.3 P(E3) = 0.2
– P(A/E1) = 0.03 P(A/E2) = 0.04 P(A/ E3) = 0.05
ExampleExample 8 8
P Eii
n
( ) 1
1
S
A
E1 E2 E3
P(A) = ?
• We obtain the probability of A using:
ExampleExample 8 8
P A P E P A Eii
n
i( ) ( ) ( / )
1
%7.3037.0
)05.02.0()04.03.0()03.05.0()(
AP
• Thus, the probability to have a defective component produced during the day equals 3.7%
Reversing the ConditioningReversing the Conditioning
• Reversing the conditioning of two events is rarely intuitive.
• Suppose we want to know P(A|B), but we know only P(A), P(B), and P(B|A).
• We also know P(A and B), since
P(A and B) = P(A) x P(B|A)
• From this information, we can find P(A|B):
(A and B)(A|B)(B)
PPP
Reversing the Conditioning (cont.)Reversing the Conditioning (cont.)
• When we reverse the probability from the conditional probability that you’re originally given, you are actually using Bayes’ Rule (or Theorem) .
Let A be an event of S and E1, E2, ..., En, a set of mutually exclusive and exhaustive events of S, To obtain P(Ej / A):
P E AP E A
P Aj
j( / )
( )
( )
Bayes’ TheoremBayes’ Theorem
Let A be an event of S and E1, E2, ..., En,, a set of mutually exclusive and exhaustive events of S, To obtain P(Ej / A):
P E AP E A
P Aj
j( / )
( )
( )
P E A P E P A Ej j j( ) ( ) ( / )
Bayes’ TheoremBayes’ Theorem
Let A be an event of S and E1, E2, ..., En, a set of mutually exclusive and exhaustive events of S, To obtain P(Ej / A):
P E AP E A
P Aj
j( / )
( )
( )
P E A P E P A Ej j j( ) ( ) ( / )
P A P E P A Eii
n
i( ) ( ) ( / )
1
Bayes’ TheoremBayes’ Theorem
Let A be an event of S and E1, E2, ..., En, a set of mutually exclusive and exhaustive events of S, To obtain P(Ej / A):
P E AP E A
P Aj
j( / )
( )
( )
P E A P E P A Ej j j( ) ( ) ( / )
P A P E P A Eii
n
i( ) ( ) ( / )
1
P E AP E P A E
P E P A Ej
j j
ii
n
i
( / )( ) ( / )
( ) ( / )
1
Bayes’ TheoremBayes’ Theorem
• In the previous example (ex. 8), we knew P(A/Ei), i.e., the probability that a component is defective given that it is produced by machine Mi (event Ei)
• If we now pick a component at random from the production of the day, and we notice that it is defective (A), what is the probability that the component was produced by machine M3 (event E3)?
Example 9Example 9
Bayes’ RuleBayes’ Rule
• We are interested in determining the probability that the picked component comes from M3 (event E3) given that the component is defective, i.e., P(E3/A).
• We know the following:
– P(E1) = 0.5 P(E2) = 0.3 P(E3) = 0.2
– P(A/E1) = 0.03 P(A/E2) = 0.04 P(A/ E3) = 0.05
Example 9Example 9
Bayes’ RuleBayes’ Rule
• Using Bayes’ theorem, we have:
Example 9Example 9
Bayes’ RuleBayes’ Rule
P E AP E P A E
P E P A Ej
j j
ii
n
i
( / )( ) ( / )
( ) ( / )
1P(A)
%2727.0037.0
01.0
)05.02.0()04.03.0()03.05.0(
05.02.0)/( 3
AEP
What Can Go Wrong?What Can Go Wrong?
• In most situations where we want to find a probability, we’ll use the rules in combination.
• Beware of probabilities that don’t add up to 1.• Don’t add probabilities of events if they’re not
disjoint.• Don’t multiply probabilities of events if they’re
not independent.• Don’t confuse disjoint and independent—disjoint
events can’t be independent.
• Using Bayes’ theorem, we have:
Example 9Example 9
Bayes’ RuleBayes’ Rule
P E AP E P A E
P E P A Ej
j j
ii
n
i
( / )( ) ( / )
( ) ( / )
1P(A)
• Thus, once we know that the component is defective, the probability that it comes from the machine M3 is revised upward from 20% to 27%
%2727.0037.0
01.0
)05.02.0()04.03.0()03.05.0(
05.02.0)/( 3
AEP
Bayes TheoremBayes TheoremWhen to Use itWhen to Use it
• I know P(X|Y)
• I want to know P(Y|X)
• I have an initial estimate of P(Y) (Prior prob.)
• I do a test and find that X is true
• I update my initial estimate to get P(Y|X) (Posterior prob.)
Ethics in ActionEthics in ActionHair Salon: Facials or Massages?Hair Salon: Facials or Massages?
• Survey of salons offering facials and massages– 50% of customers wanting hair styling also want
facials
– 90% of customers want hair styling or massages
• Therefore we should offer massages.• Ethical Issue
– Item A: Probabilities not comparable (conditional/OR)
• Ethical Solution– Report all details of survey.