HUDM4122Probability and Statistical Inference

February 18, 2015

HW

• Getting harder…

HW

• Getting harder…

Difficulties

A lot of trouble with sample spacecalculation

A reminder

• The sample space is the total number ofcombination of things that can happen

• If you flip a fair coin twice, the sample space is4: HH TH HT TT

Problem 1

• What is the sample space if you flip a coin 6times?

• Correct answer: 2x2x2x2x2x2 = 64• Most common answer: 12• Also: 1/64

Problem 2

• You have made friends with a specially trainedmouse, who, on a given step, randomly goesleft 1/3 of the time, forwards 1/3 of the time,and right 1/3 of a time. If the mouse takes 7steps, what is the sample space?

• Correct answer: 3x3x3x3x3x3x3• Common wrong answer: 21

Problem 5

• You and your friends order a pizza with 8 slices.One of the slices, for some obscure reason, hasanchovies. You *HATE* anchovies. Before you getto the pizza, each of your 7 friends takes a singleslice, apparently at random. What is the samplespace of this meal?

• Correct answer: 8*7*6*5*4*3*2*1• Common wrong answers: 8, 8^7

– Why are these wrong?

Probability Calculation

Problem 6

• You and your friends order a pizza with 8 slices.One of the slices, for some obscure reason, hasanchovies. You *HATE* anchovies. Before you getto the pizza, each of your 7 friends takes a singleslice, apparently at random. What is theprobability that you end up with anchovies?

• Correct answer: 1/8• Common wrong answer: 1/40320

Problem 8• You and your friends order a pizza with 8 slices.

One of the slices, for some obscure reason, hasanchovies. You *HATE* anchovies. Before you getto the pizza, 2 of your 7 friends take a single slice,apparently at random. They both did not getanchovies. What is the probability that you endup with anchovies?

• Correct answer: 0%– Why?

• Common wrong answers: 1/6, 1/5

Problem 11

• 21% of Americans went to an art gallery ormuseum in the last year. 23% of Americanswent to a baseball game last year. 4% ofAmericans went to both (I totally made thatlast one up). What percent of americans wentto a art gallery OR a museum OR a baseballgame last year?

• What’s the answer?

Problem 11

• 21% of Americans went to an art gallery ormuseum in the last year. 23% of Americans wentto a baseball game last year. 4% of Americanswent to both (I totally made that last one up).What percent of americans went to a art galleryOR a museum OR a baseball game last year?

• Correct answer: 40%• Common wrong answers: 44%, 48%

Problem 14• The probability that a New Yorker takes the

subway is 37%. Let's say that the probability thata New Yorker goes to a museum or gallery eachyear is 34%. The probability that a NewYorker goes to a museum or gallery each year, ifthey take the subway, is 41%. What is theprobability that a New Yorker takes the subwayAND goes to a museum or gallery each year?

• Correct answer: 15%• Common wrong answer: 41%

– Why is this wrong?

Problem 14• The probability that a New Yorker takes the

subway is 37%. Let's say that the probability thata New Yorker goes to a museum or gallery eachyear is 34%. The probability that a NewYorker goes to a museum or gallery each year, ifthey take the subway, is 41%. What is theprobability that a New Yorker takes the subwayAND goes to a museum or gallery each year?

• Correct answer: 15%• Another common wrong answer: 13%

– Why is this wrong?

Problem 15

• The probability that a student passes "Intro toBasketweaving" is 72%. The probability that astudent passes "Intro to Psychoceramics" is 21%if they fail "Intro to Basketweaving", and is 94% ifthey pass "Intro to Basketweaving". What is theprobability that a student passes both classes?

• Why is the correct answer 68% rather than 20%?

Combinations and Permutations

Problem 10

• Professor Padeiro owns 7 computers. Shewants to take 3 of them with her on a trip.How many combinations of computers couldshe take?

• What is the correct answer?

Problem 10

• Professor Padeiro owns 7 computers. Shewants to take 3 of them with her on a trip.How many combinations of computers couldshe take?

• What is the correct answer?!! !=35

Problem 10

• Professor Padeiro owns 7 computers. She wantsto take 3 of them with her on a trip. How manycombinations of computers could she take?

• What is the correct answer?!! !=35

• Common wrong answer = 210

What we didn’t cover last time

General Multiplication Rule

• ∩ = ( | )• What if A and B are independent?

• Like two flips of a fair coin

General Multiplication Rule

• ∩ = ( | )• What if A and B are independent?

• Like two flips of a fair coin

• In that case, P(B|A)=P(B)

Multiplication Rule For IndependentEvents

• ∩ = ,• If A and B are independent

Multiplication Rule For IndependentEvents

• ∩ = ,• If A and B are independent

• This is the rule we were using, when wecomputed…– Multiple coin flips– Multiple rolls of a 6-sided die

Any last comments or questions forthe day?

Today

• Ch. 4.7 in Mendenhall, Beaver, & Beaver

Today

• Bayes’ Rule

Today

• Bayes’ Rule

• Also (more frequently) called– Bayes’ Theorem– Bayes’ Law

Very Important Rule in Statistics

• Underpins Bayesian Statistics– One of the two core branches of Statistics– Not a focus of this class, which is focused on the

other branch, Frequentist statistics

• Underpins major areas of Data Mining andMachine Learning– Including core methods of educational data

mining, such as Bayesian Knowledge Tracing

Classic Version

• = ( )( )

Let’s apply it

• = ( )( )• P(B|A) = 0.4• P(A) = 0.7• P(B) = 0.3• P(A|B)=?

Let’s apply it

• = ( )( )• P(B|A) = 0.4• P(A) = 0.7• P(B) = 0.3• P(A|B)= 0.93

Example• Maria is using Reasoning Mind software to learn

mathematics

• If she knows a skill, there’s a 60% chance she getsthe problem right

• There’s a 40% chance she knows the skill• There’s a 70% chance she gets the problem right

• What’s the probability that if she gets theproblem right, she knows the skill?

A = knows skill, B = gets problem right

• Maria is using Reasoning Mind software to learnmathematics

• If she knows a skill, there’s a 60% chance she getsthe problem right

• There’s a 40% chance she knows the skill• There’s a 70% chance she gets the problem right

• What’s the probability that if she gets theproblem right, she knows the skill?

A = knows skill, B = gets problem right

• Maria is using Reasoning Mind software to learnmathematics

• If she knows a skill, there’s a 60% chance she gets theproblem right. P(B|A)

• There’s a 40% chance she knows the skill. P(A)• There’s a 70% chance she gets the problem right. P(B)

• What’s the probability that if she gets the problemright, she knows the skill?

Example• Maria is using Reasoning Mind software to learn

mathematics

• If she knows a skill, there’s a 60% chance she gets theproblem right

• There’s a 40% chance she knows the skill• There’s a 70% chance she gets the problem right

• What’s the probability that if she gets the problemright, she knows the skill?– 34.2%

Be careful…

• About what your A is• And what your B is

Do this one in pairs• Dan is taking an online course using the Purdue Course

Signals platform, that detects when a student is at-risk offailing the course

• If he is at-risk, there’s a 80% chance he skips the firsthomework

• There’s a 50% chance he is at-risk• There’s a 60% chance he skips the first homework

• What’s the probability that if he skips the first homework,he is at-risk?

Do this one in pairs• Dan is taking an online course using the Purdue Course

Signals platform, that detects when a student is at-riskof failing the course

• If he is at-risk, there’s a 80% chance he skips the firsthomework

• There’s a 50% chance he is at-risk• There’s a 60% chance he skips the first homework

• What’s the probability that if he skips the firsthomework, he is at-risk?– 66.7%

Do this one in pairs• The Yonkers College of Holistic Phrenology just had an

unspeakably embarrassing scandal

• Historically, among colleges of this type that are deniedaccreditation, 20% have had a recent scandal

• There’s a 4% chance of a college of this type beingdenied accreditation

• There’s a 1% chance of a college of this type having ascandal

• Given that this college just had a scandal, what is theprobability it will be denied accreditation?

Do this one in pairs• The Yonkers College of Holistic Phrenology just had an

unspeakably embarrassing scandal

• Historically, among colleges of this type that are deniedaccreditation, 20% have had a recent scandal

• There’s a 4% chance of a college of this type being deniedaccreditation

• There’s a 1% chance of a college of this type having ascandal

• Given that this college just had a scandal, what is theprobability it will be denied accreditation?– 80%

Questions? Comments?

Where did Bayes’ Rule come from?

Where did Bayes’ Rule come from?

Where did Bayes’ Rule come from?

P(Actually Bayes) = 0.1

Where did Bayes’ Rule come from?

• Simple to derive

Recall General Multiplication Rule

• ∩ = ( | )

Note Also That

• ∩ = ( | )• ∩ = ( | )

Which Means That

• P(B)P(A|B) = ( | )

Divide Both Sides by P(B)

• = ( )( )

Questions? Comments?

This was the Classic Version of Bayes’ Rule

• When people talk about Bayes’ Rule, theygenerally mean this version

This was the Classic Version of Bayes’ Rule

• When people talk about Bayes’ Rule, theygenerally mean this version

• There is also a General Version seen in thebook

Before we get there…

• Law of Total Probability

Law of Total Probability

Example

• I’m wondering whether a student will quit theproblem set before completing

• The student might be (exhaustively, mutuallyexclusively)– Working in System Appropriately– Gaming the System– Getting Answers From a Friend

Example• I’m wondering whether a student will quit the problem set

before completing

• The student might be (exhaustively, mutually exclusively)– P(Working in System Appropriately) = 0.7– P(Quit | WSA) = 0.02

– P(Gaming the System) = 0.1– P(Quit | GS) = 0.01

– P(Getting Answers From a Friend) = 0.2– P(Quit | GAFF) = 0.3

Note that P(WSA)+P(GS)+P(GAFF) = 1

• I’m wondering whether a student will quit the problem setbefore completing

• The student might be (exhaustively, mutually exclusively)– P(Working in System Appropriately) = 0.7– P(Quit | WSA) = 0.02

– P(Gaming the System) = 0.1– P(Quit | GS) = 0.01

– P(Getting Answers From a Friend) = 0.2– P(Quit | GAFF) = 0.3

Note that P(WSA)+P(GS)+P(GAFF) = 1If this isn’t true,

it’s not exhaustiveor not mutually exclusive

• I’m wondering whether a student will quit the problem setbefore completing

• The student might be (exhaustively, mutually exclusively)– P(Working in System Appropriately) = 0.7– P(Quit | WSA) = 0.02

– P(Gaming the System) = 0.1– P(Quit | GS) = 0.01

– P(Getting Answers From a Friend) = 0.2– P(Quit | GAFF) = 0.3

Example

• P(Quit) =P(Working in System Appropriately) * P(Quit | WSA) +P(Gaming the System) * P(Quit | GS) +P(Getting Answers From a Friend) * P(Quit | GAFF)

Example

• P(Quit) =P(Working in System Appropriately) * P(Quit | WSA) +P(Gaming the System) * P(Quit | GS) +P(Getting Answers From a Friend) * P(Quit | GAFF)

• P(Quit) = 0.7*0.02+ 0.1*0.01 + 0.2*0.3

Example

• P(Quit) =P(Working in System Appropriately) * P(Quit | WSA) +P(Gaming the System) * P(Quit | GS) +P(Getting Answers From a Friend) * P(Quit | GAFF)

• P(Quit) = 0.7*0.02+ 0.1*0.01 + 0.2*0.3 = 0.014+0.001+0.06= 0.075

Do this one in pairs

• I’m wondering whether my kiddo has stomach flu

• The kiddo might be (exhaustively, mutuallyexclusively)– P(Puking) = 0.05– P(Stomach Flu | Puking) = 0.4

– P(Not Puking) = 0.95– P(Stomach Flu | Not Puking) = 0.01

• Your answer?

Questions? Comments?

If we take the law of total probability

• And compare it to the multiplied probabilitycoming out of one event…

• We get…

Extended Form of Bayes’ Rule

• What the book simply refers to as Bayes’ Rule(but most people don’t)

Extended Form of Bayes’ Rule

• ( | ) = )∑ ( | )• A is an event• S1 through Sk represent a group of mutually

exclusive and exhaustive sub-populations

Example

• In ASSISTments, on the first attempt at problem P,a student can request a hint, give a commonincorrect answer, give an uncommon incorrectanswer, or give a correct answer

• P(hint) = 0.3• P(common incorrect) = 0.2• P(uncommon incorrect) = 0.4• P(correct) = 0.1

Example• P(hint) = 0.3• P(common incorrect) = 0.2• P(uncommon incorrect) = 0.4• P(correct) = 0.1

• P(knows skill | hint) = 0.1• P(knows skill | common incorrect) = 0.2• P(knows skill | uncommon incorrect) = 0.1• P(knows skill | correct) = 0.7

Example• P(hint) = 0.3• P(common incorrect) = 0.2• P(uncommon incorrect) = 0.4• P(correct) = 0.1

• P(knows skill | hint) = 0.1• P(knows skill | common incorrect) = 0.2• P(knows skill | uncommon incorrect) = 0.1• P(knows skill | correct) = 0.7

• What is P(correct | knows skill)?

Example• P(hint) = 0.3 P(S1)• P(common incorrect) = 0.2 P(S2)• P(uncommon incorrect) = 0.4 P(S3)• P(correct) = 0.1 P(S4)

• P(knows skill | hint) = 0.1 P(A|S1)• P(knows skill | common incorrect) = 0.2 P(A|S2)• P(knows skill | uncommon incorrect) = 0.1 P(A|S3)• P(knows skill | correct) = 0.7 P(A|S4)

• What is P(correct | knows skill)? P(S4|A)

• P(hint) = 0.3 P(S1)• P(common incorrect) = 0.2 P(S2)• P(uncommon incorrect) = 0.4 P(S3)• P(correct) = 0.1 P(S4)

• P(knows skill | hint) = 0.1P(A|S1)

• P(knows skill | common incorrect) = 0.2P(A|S2)

• P(knows skill | uncommon incorrect) = 0.1 P(A|S3)• P(knows skill | correct) = 0.7 P(A|S4)

• What is P(correct | knows skill)? P(S4|A)

• ( | ) = )∑ ( | )

• P(hint) = 0.3 P(S1)• P(common incorrect) = 0.2 P(S2)• P(uncommon incorrect) = 0.4 P(S3)• P(correct) = 0.1 P(S4)

• P(knows skill | hint) = 0.1 P(A|S1)• P(knows skill | common incorrect) = 0.2 P(A|S2)• P(knows skill | uncommon incorrect) = 0.1 P(A|S3)• P(knows skill | correct) = 0.7 P(A|S4)

• What is P(correct | knows skill)? P(S4|A)

• ( | ) = ) ( | )

• P(hint) = 0.3 P(S1)• P(common incorrect) = 0.2 P(S2)• P(uncommon incorrect) = 0.4 P(S3)• P(correct) = 0.1 P(S4)

• P(knows skill | hint) = 0.1 P(A|S1)• P(knows skill | common incorrect) = 0.2 P(A|S2)• P(knows skill | uncommon incorrect) = 0.1 P(A|S3)• P(knows skill | correct) = 0.7 P(A|S4)

• What is P(correct | knows skill)? P(S4|A)

• ( | ) = . ( . ). . . . . . ( . )( . )

• P(hint) = 0.3 P(S1)• P(common incorrect) = 0.2 P(S2)• P(uncommon incorrect) = 0.4 P(S3)• P(correct) = 0.1 P(S4)

• P(knows skill | hint) = 0.1 P(A|S1)• P(knows skill | common incorrect) = 0.2 P(A|S2)• P(knows skill | uncommon incorrect) = 0.1 P(A|S3)• P(knows skill | correct) = 0.7 P(A|S4)

• What is P(correct | knows skill)? P(S4|A)

• ( | ) = . ( . ). . . . . . ( . )( . )=0.39

Comments? Questions?

Any last comments or questions forthe day?

Upcoming Classes

• 2/23 Discrete Random Variables and TheirProbability Distributions– Ch. 4-8

• 2/25 Binomial Probability Distribution– Ch. 5-2– HW 4 due

Homework 4

• Due in 7 days• In the ASSISTments system

Questions? Comments?