07: variance, bernoulli, binomial - stanford...
TRANSCRIPT
07: Variance, Bernoulli, BinomialJerry CainApril 12, 2021
1
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Quick slide reference
2
3 Variance 07a_variance_i
10 Properties of variance 07b_variance_ii
17 Bernoulli RV 07c_bernoulli
22 Binomial RV 07d_binomial
34 Exercises LIVE
Variance
3
07a_variance_i
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Stanford, CA๐ธ high = 68ยฐF๐ธ low = 52ยฐF
Washington, DC๐ธ high = 67ยฐF๐ธ low = 51ยฐF
4
Average annual weather
Is ๐ธ ๐ enough?
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Stanford, CA๐ธ high = 68ยฐF๐ธ low = 52ยฐF
Washington, DC๐ธ high = 67ยฐF๐ธ low = 51ยฐF
5
Average annual weather
0
0.1
0.2
0.3
0.4
0
0.1
0.2
0.3
0.4
๐๐=๐ฅ
๐๐=๐ฅ
Stanford high temps Washington high temps
35 50 65 80 90 35 50 65 80 90
68ยฐF 67ยฐF
Normalized histograms are approximations of PMFs.
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Variance = โspreadโConsider the following three distributions (PMFs):
โข Expectation: ๐ธ ๐ = 3 for all distributionsโข But the โspreadโ in the distributions is different!โข Variance, Var ๐ : a formal quantification of โspreadโ
6
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Variance
The variance of a random variable ๐ with mean ๐ธ ๐ = ๐ is
Var ๐ = ๐ธ ๐ โ ๐ (
โข Also written as: ๐ธ ๐ โ ๐ธ ๐ !
โข Note: Var(X) โฅ 0โข Other names: 2nd central moment, or square of the standard deviation
Var ๐
def standard deviation SD ๐ = Var ๐7
Units of ๐!
Units of ๐
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Variance of Stanford weather
8
Varianceof ๐
Var ๐ = ๐ธ ๐ โ ๐ธ ๐ !
Stanford, CA๐ธ high = 68ยฐF๐ธ low = 52ยฐF
0
0.1
0.2
0.3
0.4
๐๐=๐ฅ
Stanford high temps
35 50 65 80 90
๐ ๐ โ ๐ !
57ยฐF 121 (ยฐF)2
๐ธ ๐ = ๐ = 68ยฐF
71ยฐF 9 (ยฐF)2
75ยฐF 49 (ยฐF)2
69ยฐF 1 (ยฐF)2
โฆ โฆ
Variance ๐ธ ๐ โ ๐ ! = 39 (ยฐF)2
Standard deviation = 6.2ยฐF
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Stanford, CA๐ธ high = 68ยฐF
Washington, DC๐ธ high = 67ยฐF
9
Comparing variance
0
0.1
0.2
0.3
0.4
0
0.1
0.2
0.3
0.4
๐๐=๐ฅ
๐๐=๐ฅ
Stanford high temps Washington high temps
35 50 65 80 90 35 50 65 80 90
Var ๐ = 39 ยฐF ! Var ๐ = 248 ยฐF !
Varianceof ๐
Var ๐ = ๐ธ ๐ โ ๐ธ ๐ !
68ยฐF 67ยฐF
Properties of Variance
10
07b_variance_ii
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Properties of varianceDefinition Var ๐ = ๐ธ ๐ โ ๐ธ ๐ !
def standard deviation SD ๐ = Var ๐
Property 1 Var ๐ = ๐ธ ๐! โ ๐ธ ๐ !
Property 2 Var ๐๐ + ๐ = ๐!Var ๐
11
Units of ๐!
Units of ๐
โข Property 1 is often easier to compute than the definitionโข Unlike expectation, variance is not linear
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Properties of varianceDefinition Var ๐ = ๐ธ ๐ โ ๐ธ ๐ !
def standard deviation SD ๐ = Var ๐
Property 1 Var ๐ = ๐ธ ๐! โ ๐ธ ๐ !
Property 2 Var ๐๐ + ๐ = ๐!Var ๐
12
Units of ๐!
Units of ๐
โข Property 1 is often easier to compute than the definitionโข Unlike expectation, variance is not linear
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Computing variance, a proof
13
Var ๐ = ๐ธ ๐ โ ๐ธ ๐ !
= ๐ธ ๐! โ ๐ธ ๐ !
= ๐ธ ๐! โ ๐!= ๐ธ ๐! โ 2๐! + ๐!= ๐ธ ๐! โ 2๐๐ธ ๐ + ๐!
= ๐ธ ๐ โ ๐ ! Let ๐ธ ๐ = ๐
Everyone, please
welcome the second
moment!
Varianceof ๐
Var ๐ = ๐ธ ๐ โ ๐ธ ๐ !
= ๐ธ ๐! โ ๐ธ ๐ !
=,"
๐ฅ โ ๐ !๐ ๐ฅ
=,"
๐ฅ! โ 2๐๐ฅ + ๐! ๐ ๐ฅ
=,"
๐ฅ!๐ ๐ฅ โ 2๐,"
๐ฅ๐ ๐ฅ + ๐!,"
๐ ๐ฅ
โ 1
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Let Y = outcome of a single die roll. Recall ๐ธ ๐ = 7/2 .Calculate the variance of Y.
14
Variance of a 6-sided dieVarianceof ๐
Var ๐ = ๐ธ ๐ โ ๐ธ ๐ !
= ๐ธ ๐! โ ๐ธ ๐ !
1. Approach #1: Definition
Var ๐ =161 โ
72
!+162 โ
72
!
+163 โ
72
!+164 โ
72
!
+165 โ
72
!+166 โ
72
!
2. Approach #2: A property
๐ธ ๐! =161! + 2! + 3! + 4! + 5! + 6!
= 91/6
Var ๐ = 91/6 โ 7/2 !
= 35/12 = 35/12
2nd moment
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Properties of varianceDefinition Var ๐ = ๐ธ ๐ โ ๐ธ ๐ !
def standard deviation SD ๐ = Var ๐
Property 1 Var ๐ = ๐ธ ๐! โ ๐ธ ๐ !
Property 2 Var ๐๐ + ๐ = ๐!Var ๐
15
Units of ๐!
Units of ๐
โข Property 1 is often easier to compute than the definitionโข Unlike expectation, variance is not linear
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Property 2: A proofProperty 2 Var ๐๐ + ๐ = ๐!Var ๐
16
Var ๐๐ + ๐= ๐ธ ๐๐ + ๐ ! โ ๐ธ ๐๐ + ๐ ! Property 1
= ๐ธ ๐!๐! + 2๐๐๐ + ๐! โ ๐๐ธ ๐ + ๐ !
= ๐!๐ธ ๐! + 2๐๐๐ธ ๐ + ๐! โ ๐! ๐ธ[๐] ! + 2๐๐๐ธ[๐] + ๐!
= ๐!๐ธ ๐! โ ๐! ๐ธ[๐] !
= ๐! ๐ธ ๐! โ ๐ธ[๐] !
= ๐!Var ๐ Property 1
Proof:
Factoring/Linearity of Expectation
Bernoulli RV
17
07c_bernoulli
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Consider an experiment with two outcomes: โsuccessโ and โfailure.โdef A Bernoulli random variable ๐ maps โsuccessโ to 1 and โfailureโ to 0.
Other names: indicator random variable, boolean random variable
Examples:โข Coin flipโข Random binary digitโข Whether a disk drive crashed
Bernoulli Random Variable
18
๐ ๐ = 1 = ๐ 1 = ๐๐ ๐ = 0 = ๐ 0 = 1 โ ๐๐~Ber(๐)
Support: {0,1} VarianceExpectation
PMF
๐ธ ๐ = ๐Var ๐ = ๐(1 โ ๐)
Remember this nice property of expectation. It will come back!
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Jacob Bernoulli
19
Jacob Bernoulli (1654-1705), also known as โJamesโ, was a Swiss mathematician
One of many mathematicians in Bernoulli familyThe Bernoulli Random Variable is named for him
My academic great14 grandfather
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Defining Bernoulli RVs
20
Serve an ad.โข User clicks w.p. 0.2โข Ignores otherwise
Let ๐: 1 if clicked
๐~Ber(___)๐ ๐ = 1 = ___๐ ๐ = 0 = ___
Roll two dice.โข Success: roll two 6โsโข Failure: anything else
Let ๐ : 1 if success
๐~Ber(___)
๐ธ ๐ = ___
๐~Ber(๐) ๐" 1 = ๐๐" 0 = 1 โ ๐๐ธ ๐ = ๐
Run a programโข Crashes w.p. ๐โข Works w.p. 1 โ ๐
Let ๐: 1 if crash
๐~Ber(๐)๐ ๐ = 1 = ๐๐ ๐ = 0 = 1 โ ๐ ๐ค
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Defining Bernoulli RVs
21
Serve an ad.โข User clicks w.p. 0.2โข Ignores otherwise
Let ๐: 1 if clicked
๐~Ber(___)๐ ๐ = 1 = ___๐ ๐ = 0 = ___
Roll two dice.โข Success: roll two 6โsโข Failure: anything else
Let ๐ : 1 if success
๐~Ber(___)
๐ธ ๐ = ___
๐~Ber(๐) ๐" 1 = ๐๐" 0 = 1 โ ๐๐ธ ๐ = ๐
Run a programโข Crashes w.p. ๐โข Works w.p. 1 โ ๐
Let ๐: 1 if crash
๐~Ber(๐)๐ ๐ = 1 = ๐๐ ๐ = 0 = 1 โ ๐
Binomial RV
22
07d_binomial
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Consider an experiment: ๐ independent trials of Ber(๐) random variables.def A Binomial random variable ๐ is the number of successes in ๐ trials.
Examples:โข # heads in n coin flipsโข # of 1โs in randomly generated length n bit stringโข # of disk drives crashed in 1000 computer cluster
(assuming disks crash independently)
Binomial Random Variable
23
๐ = 0, 1, โฆ , ๐:
๐ ๐ = ๐ = ๐ ๐ = ๐๐ ๐! 1 โ ๐ "#!๐~Bin(๐, ๐)
Support: {0,1, โฆ , ๐}
PMF
๐ธ ๐ = ๐๐Var ๐ = ๐๐(1 โ ๐)Variance
Expectation
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021 24
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Reiterating notation
The parameters of a Binomial random variable:โข ๐: number of independent trialsโข ๐: probability of success on each trial
25
1. The random variable
2. is distributed as a
3. Binomial 4. with parameters
๐ ~ Bin(๐, ๐)
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Reiterating notation
If ๐ is a binomial with parameters ๐ and ๐, the PMF of ๐ is
26
๐ ~ Bin(๐, ๐)
๐ ๐ = ๐ = ๐๐ ๐( 1 โ ๐ )*(
Probability Mass Function for a BinomialProbability that ๐takes on the value ๐
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Three coin flipsThree fair (โheadsโ with ๐ = 0.5) coins are flipped.โข ๐ is number of headsโข ๐~Bin 3, 0.5
Compute the following event probabilities:
27
๐~Bin(๐, ๐) ๐ ๐ = ๐๐ ๐# 1 โ ๐ $%#
๐ ๐ = 0
๐ ๐ = 1
๐ ๐ = 2
๐ ๐ = 3
๐ ๐ = 7P(event)
๐ค
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Three coin flipsThree fair (โheadsโ with ๐ = 0.5) coins are flipped.โข ๐ is number of headsโข ๐~Bin 3, 0.5
Compute the following event probabilities:
28
๐~Bin(๐, ๐) ๐ ๐ = ๐๐ ๐# 1 โ ๐ $%#
๐ ๐ = 0 = ๐ 0 = 30 ๐$ 1 โ ๐ % = &
'
๐ ๐ = 1
๐ ๐ = 2
๐ ๐ = 3
๐ ๐ = 7
= ๐ 1 = 31 ๐& 1 โ ๐ ! = %
'
= ๐ 2 = 32 ๐! 1 โ ๐ & = %
'
= ๐ 3 = 33 ๐% 1 โ ๐ $ = &
'
= ๐ 7 = 0P(event) PMF
Extra math note:By Binomial Theorem,we can proveโ()$* ๐ ๐ = ๐ = 1
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Consider an experiment: ๐ independent trials of Ber(๐) random variables.def A Binomial random variable ๐ is the number of successes in ๐ trials.
Examples:โข # heads in n coin flipsโข # of 1โs in randomly generated length n bit stringโข # of disk drives crashed in 1000 computer cluster
(assuming disks crash independently)
Binomial Random Variable
29
๐~Bin(๐, ๐)Range: {0,1, โฆ , ๐} Variance
Expectation
PMF
๐ธ ๐ = ๐๐Var ๐ = ๐๐(1 โ ๐)
๐ = 0, 1, โฆ , ๐:
๐ ๐ = ๐ = ๐ ๐ = ๐๐ ๐! 1 โ ๐ "#!
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Ber ๐ = Bin(1, ๐)
Binomial RV is sum of Bernoulli RVs
Bernoulliโข ๐~Ber(๐)
Binomialโข ๐~Bin ๐, ๐โข The sum of ๐ independent
Bernoulli RVs
30
๐ =,+)&
*
๐+ , ๐+ ~Ber(๐)
+
+
+
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Consider an experiment: ๐ independent trials of Ber(๐) random variables.def A Binomial random variable ๐ is the number of successes in ๐ trials.
Examples:โข # heads in n coin flipsโข # of 1โs in randomly generated length n bit stringโข # of disk drives crashed in 1000 computer cluster
(assuming disks crash independently)
Binomial Random Variable
31
๐~Bin(๐, ๐)Range: {0,1, โฆ , ๐}
๐ธ ๐ = ๐๐Var ๐ = ๐๐(1 โ ๐)Variance
Expectation
PMF ๐ = 0, 1, โฆ , ๐:
๐ ๐ = ๐ = ๐ ๐ = ๐๐ ๐! 1 โ ๐ "#!
Proof:
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
Consider an experiment: ๐ independent trials of Ber(๐) random variables.def A Binomial random variable ๐ is the number of successes in ๐ trials.
Examples:โข # heads in n coin flipsโข # of 1โs in randomly generated length n bit stringโข # of disk drives crashed in 1000 computer cluster
(assuming disks crash independently)
Binomial Random Variable
32
๐~Bin(๐, ๐)Range: {0,1, โฆ , ๐}
PMF
๐ธ ๐ = ๐๐Var ๐ = ๐๐(1 โ ๐)
Weโll prove this later in the course
VarianceExpectation
๐ = 0, 1, โฆ , ๐:
๐ ๐ = ๐ = ๐ ๐ = ๐๐ ๐! 1 โ ๐ "#!
Lisa Yan, Chris Piech, Mehran Sahami, and Jerry Cain, CS109, Spring 2021
No, give me the variance proof right now
33
proofwiki.org