week 3 classical probability, part ipersonal.psu.edu/acq/401/course.info/week3.pdfwith probability...

OutlineSample Spaces, Events and Set Operations

Probability and Equally Likely OutcomesAxioms and Properties of Probability

Week 3Classical Probability, Part I

Week 3 Classical Probability, Part I



Week 3 Objectives

• Proper understanding of common statistical practices such asconfidence intervals and hypothesis testing requires some familiaritywith probability theory.

•We start with classical probability, which arose from games ofchance such as rolling dice or dealing cards. In such experiments,where all outcomes are equally likely, the probability of an event isdetermined by enumerating the outcomes making up the event.

After reviewing basic set operations, the necessary countingtechniques are presented.

• The transition to experiments where the outcomes are not equallylikely can be seamless. The axioms and properties of probability ingeneral experiments are presented, and the important notion of aprobability mass function is introduced.




1 Sample Spaces, Events and Set Operations

2 Probability and Equally Likely Outcomes

The Probability Mass Function and Probability Sampling

Counting Techniques

3 Axioms and Properties of Probability




Overview

The central role probability plays in statistics is the reasonwhy statistics classes typically cover this subject.Classical probability, which arose from games of chance,includes combinatorics and the concepts of conditionalprobability and independence.Probability evolved to deal with modeling the randomnessof phenomena such as the number of earthquakes, theamount of rainfall, the life time of a given electricalcomponent, or the relation between education level andincome, etc. Such probability models will be discussed inChapters 3 and 4.




Sample Spaces

• The set of all possible outcomes of a random experiment iscalled the sample space of the experiment, and will bedenoted by S. some examples follow.

1) The sample space of the experiment which selects twofuses and classifies each as non-defective or defective isS1 = {(0,0), (0,1), (1,0), (1,1)}, where 0 and 1 stand fornon-defective and defective, respectively.

2) The sample space of the experiment which selects twofuses and records how many are defective is S2 = {0,1,2}.

3) The sample space of the experiment which records thenumber of fuses inspected until the second defective isfound is S3 = {2,3,4, . . .}.




4) Three undergraduate students are selected and theiropinions about expanding the use of solar energy arerecorded on a scale from 1 to 10. Give the sample spaceof this experiment. What is the size of this sample space?Solution: The set of all possible outcomes consist of thetriplets (x1, x2, x3), where x1, x2 and x3 denote the responseof the 1st, 2nd and 3rd student, respectively. Thus,

S4 = {(x1, x2, x3) : x1 = 1,2, . . . ,10, x2 = 1,2, . . . ,10,

x3 = 1,2, . . . ,10}.

There are 10 · 10 · 10 = 1000 possible outcomes.




5) Only the average rating, provided by the threeundergraduate students of the previous example, isrecorded. Give the sample space, S5, of this experiment.What is the size of this sample space?Solution: S5 is the collection of all distinct averages(x1 + x2 + x3)/3 formed from the 1000 triplets of S4. Theword ”distinct” is emphasized because, for example,(5,6,7) and (4,6,8) both yield an average of 6. The size ofS5 is determined, most easily, in R:

S4=expand.grid(x1=1:10,x2=1:10,x3=1:10) # all triplets inthe sample space S4length(table(rowMeans(S4))) # returns 28 for the numberof different averages




Events

In experiments with many possible outcomes, investigatorsoften classify individual outcomes into distinct categories.

Opinion ratings may be classified into low (L = {0,1,2,3}),medium (M = {4,5,6}) and high (H = {7,8,9,10}).Opinion ratings of three students can be classifiedaccording to the 28 distinct average values.

DefinitionCollections of individual outcomes are called events. An eventconsisting of only one outcome is called a simple event.

Events are denoted by letters such as A, B, C, etc.




We say that a particular event A has occurred if theoutcome of the experiment is a member of (i.e., containedin) A.

If the result of 5 coin tosses is two heads and three tails, theevent A = {at most 3 heads in five tosses of a coin} hasoccurred.

The sample space of an experiment is an event whichalways occurs when the experiment is performed.




Set Operations

The union, A ∪ B, of events A and B, is the eventconsisting of all outcomes that are in A or in B or in both.The intersection, A ∩ B, of A and B, is the eventconsisting of all outcomes that are in both A and B.The complement, A′ or Ac , of A is the event consisting ofall outcomes that are not in A.The events A and B are said to be mutually exclusive ordisjoint if they have no outcomes in common. That is, ifA ∩ B = ∅, where ∅ denotes the empty set.The difference A− B is defined as A ∩ Bc .A is a subset of B, A ⊂ B, if e ∈ A implies e ∈ B.Two sets are equal, A = B, if A ⊂ B and B ⊂ A.




Union of A and BA∪B

A B

Intersection of A and BA∩B

A B

Figure: Venn diagrams for union and intersection




The complement of AAc

A

The difference operationA −B

A B

Figure: Venn diagrams for complement and difference




A B

B

A

Figure: Venn diagram illustrations of A, B disjoint, and A ⊂ B




Commutative Laws: a) A ∪ B = B ∪ A,b) A ∩ B = B ∩ A

Associative Laws: a) (A ∩ B) ∩ C = A ∩ (B ∩ C)b) (A ∪ B) ∪ C = A ∪ (B ∪ C)

Distributive Laws: a) (A ∪ B) ∩ C = (A ∩ C) ∪ (B ∩ C),b) (A ∩ B) ∪ C = (A ∪ C) ∩ (B ∪ C)

De Morgan’s Laws: a) (A ∪ B)c = Ac ∩ Bc

b) (A ∩ B)c = Ac ∪ Bc




ExampleThe following table classifies 100 cell phone calls according totheir duration and number of handovers a it undergoes.

Number of HandoversDuration 0 1 > 1≥ 3 10 20 10< 3 40 15 5

Let A = {call undergoes 1 handover}, B = {call lasts < 3 min}.(a) How many of the 100 calls are in each of A ∪ B and A ∩ B?(b) Give word descriptions of (A ∪ B)c and Ac ∩ Bc . Are thesesets equal? Which property is confirmed?

achanges of the source cell, as when the phone is moving into the areacovered by another source




Example (Continued)Solution:(a) There are 80 calls that undergo 1 handover or last < 3 min(or both). There are 15 calls that undergo 1 handover and last< 3 min.(b) (A ∪ B)c consists of the 20 calls that are not in A ∪ B, i.e.,the calls that last ≥ 3 min and undergo either 0 or > 1handovers. Ac ∩ Bc consists of the calls that do not undergo 1handover and do not last < 3 min. Thus, (A ∪ B)c = Ac ∩ Bc

confirming the fist of De Morgan’s laws.

Read also the examples in p. 57




The Probability Mass Function and Probability SamplingCounting Techniques

Definition of Probability

The probability of an event E , denoted by P(E), is used toquantify the likelihood of occurrence of E by assigning anumber from the interval [0,1].Higher numbers indicate that the event is more likely tooccur.A probability of 1 indicates that the event will occur withcertainty, while a probability of 0 indicates that the eventwill not occur.The likelihood of occurrence of an event is also quantifiedas a percent or in terms of the odds; read Section 2.3.1., p.60.





Assignment of Probabilities

It is simplest to introduce probability in experiments with afinite number of equally likely outcomes, such as thoseused in games of chance, or simple random sampling.

Probability for equally likely outcomes

If the sample space consists of N outcomes whichare equally likely to occur, then the probability ofeach outcome is 1/N.





Population Proportions as Probabilities

Let a unit be selected by s.r. sampling from a finite statisticalpopulation of a categorical variable. If category i has Ni units,then the probability the selected unit came from category i is

pi = Ni/N.

where N is the total number of units (so N =∑

Ni ). Thus,

(a) In rolling a die, the probability of a three is p = 1/6.

(b) If 160 out of 500 tin plates have one scratch, and one tinplate is selected at random, the probability that the selectedplate has one scratch is p = 160/500.





Efron’s Dice

Die A: four 4s and two 0sDie B: six 3sDie C: four 2s and two 6sDie D: three 5’s and three 1’s

Specify the events A > B, B > C, C > D, D > A.

Find the probabilities that A > B, B > C, C > D, D > A.

Hint: When two dice are rolled, the 36 possible outcomes areequally likely.





Outline




Counting Techniques






• Even when the population units are equally likely, the valuesof the random variable recorded may not be equally likely.

When die is rolled twice, each of the 36 possible outcomesare equally likely. But if we record the sum of the two rolls,these outcomes are not equally likely.

DefinitionThe probability mass function, or pmf, of a discrete randomvariable X , is a list of the probabilities p(x) for each value x ofthe sample space SX of X .





ExampleRoll a die twice. Find the pmf of X = the sum of the two dierolls.

Solution: The sample space of X is SX = {2,3, . . . ,12}. Thepmf can be found with the R commands:options(digits = 2); S=expand.grid(X1=1:6,X2=1:6);table(S$X1+S$X2)/length(S$X1)The probabilities these commands return are show in thefollowing table:

x 2 3 4 5 6 7 8 9 10 11 12p(x) 0.028 0.056 0.083 0.111 0.139 0.167 0.139 0.111 0.083 0.056 0.028

Try also which(S$X1+S$X2==7);S[which(S$X1+S$X2==7),].





Sample Space Population

• It is useful to think of a random experiment as random (but notsimple random) sampling from the sample space population.

1 Sampling a tin plate and recording the number of scratches(recall there can be either 0, 1 or 2 scratches) can bethought of as random (but not simple random) samplingfrom S = {0,1,2}.

2 Sampling a US voter and recording his/her opinion, on ascale from 1 to 10, regarding expanding the use of solarenergy can be thought of as random (but not simplerandom!) sampling from S = {0,1, . . . ,10}.





DefinitionWhen a sample space is thought of as the set from which wesample, we refer to it as sample space population. Therandom sampling from the sample space population (which isneed not be simple random sampling) is called probabilitysampling, or sampling from a pmf.

This idea makes it possible to think of different experimentsas sampling from the same population. For example

Inspecting 50 products and recording the number ofdefectives, andInterviewing 50 people and recording if they read New YorkTimes

can both be thought as probability sampling from theircommon sample space S = {0,1,2, . . . ,50}.





Simulations

• The concepts of pmf and sample space population andprobability sampling are useful for conducting simulations, i.e.,for generating a large number of repetitions of an experiment.Such simulations are used to better understand aspects ofcomplex populations.

Example (Simulating an Experiment with R)

Use the pmf of X = sum of two die rolls to simulate 1000repetitions of the experiment which records the sum of two dierolls. Take their mean and use it to guess the population mean.The R commands areS=expand.grid(X1=1:6,X2=1:6) ; pmf=table(S$X1+S$X2)/36mean(sample(2:12, size=1000, replace=T, prob=pmf))





Outline




Counting Techniques






Why Count?

In classical probability counting is used for calculatingprobabilities. if the sample space consists of a finitenumber of equally likely outcomes, then for the probabilityof an event A we need to know

the number of outcomes in A, N(A), andthe total number of outcomes, N(S), because

P(A) =N(A)N(S)

Some counting questions are difficult (e.g., how manydifferent five-card hands are possible from a deck of 52cards?). Thus we need specialized counting techniques.





Combinations

• The number of samples of size n that can be formed from Nunits is called the number of combinations of n objectsselected from N, is denoted by

(Nn

), and equals(

Nn

)=

N!

n!(N − n)!, where k ! = 1× 2× · · · × k .

For example, there are(

525

)=

52!5!47!

= 2,598,960

different five-card hands from a deck of N = 52 cards.Knowing the above, we can calculate the probability of thehand with 4 aces and the king of hearts, as well as ofA = {the hand has 4 aces}. What are these probabilities?





• Other counting questions can also be thought of as thenumber of different samples, of a certain size, that can be takenfrom a population.

How many different n-long sequences consisting of k 1sand n − k 0s can be formed?The answer is the number of combinations

(nk

). (Why?)

•When inspecting n items as they come off the assembly line,the probability of the event

E = {k of the n inspected items are defective}

is calculated using the concept of independence and fromknowing the number of n-long sequences consisting of k 1s (fordefective) and n − k 0s (for non defective).





• In what follows we will justify the formula for(n

k

).

• In the process we will introduce the notion of permutations.





The Product Rules

• The Simple Product Rule: Suppose a task can becompleted in two stages. If stage 1 has n1 outcomes, and ifstage 2 has n2 outcomes regardless of the outcome in stage 1,then the task has n1 × n2 outcomes.

How many five-card hands with four aces are there?Solution: Think of the task of forming a hand with four acesfrom a deck of 52 cards. This task can be completed in twostages: First select the 4 aces, and then select oneadditional card. Here n1 = 1 and n2 = 48 (why?). Thus,there are 1× 48 = 48 such hands.





Example (1)1 In how many ways can we select the 1st and 2nd place

winners from the four finalists Niki, George, Sophia andMartha?Answer: 4× 3 = 12. (Why?)

2 In how many ways can we select two from Niki, George,Sophia and Martha?

Answer:122

= 6. (Why?)

Note: 6 = # of combinations =

(42

).





• The General Product Rule: If a task can be completed in kstages and stage i has ni outcomes, regardless of theoutcomes the previous stages, then the task hasn1 × n2 × · · · × nk outcomes

How many binary sequences of length 10 (i.e., a 10-longsequence of 0s and 1s) are there?Answer: Think of the task of forming a binary sequence oflength 10. This task consists of 10 stages and each stagehas two outcomes (i.e., either 0 or 1) regardless of theoutcomes the previous stages. Thus, there are 210 = 1024different sequences.

Read also Example 2.3-4, p. 63.





Example (2)1 In how many ways can we select a 1st, 2nd and 3rd place

winners from Niki, George, Sophia and Martha?Answer: 4× 3× 2 = 24. (Why?)

2 In how many ways can we select three from Niki, George,Sophia and Martha?

Answer:246

= 4. (Why?)

Note: 4 = # of combinations =

(43

).





Permutations

The answer to Example (1), part 1, i.e. 12, is the number ofpermutations of 2 items selected from 4.

The answer to Example (2), part 1, i.e. 24, is the number ofpermutations of 3 items selected from 4.

DefinitionThe number of ordered selections (i.e. when we keep track ofthe order of selection) of k items from n is called the number ofpermutations of k items selected from n, it is denoted by Pk ,n,and equals

Pk ,n = n × (n − 1)× . . .× (n − k + 1) =n!

(n − k)!





Combinations

In the answer to Example (1), part 2, i.e. 122 , the 2 in the

denominator is the number of permutations of 2 itemsselected from 2 (P2,2 = 2× 1).

In the answer to Example (2), part 2, i.e. 246 , the 6 in the

denominator is the number of permutations of 3 itemsselected from 3 (P3,3 = 3× 2× 1).

Extending the reasoning used to obtain these answers, wehave

The number of combinations of k items selected from agroup of n is (

nk

)=

Pk ,n

k !=

n!k !(n − k)!





Binomial Coefficients

The numbers(n

k

)are called binomial coefficients because

of the Binomial Theorem:

(a + b)n =n∑

k=0

(nk

)akbn−k .





Example(a) How many paths going from the lower left corner of a 4× 3grid to its upper right corner are there? Assume one is allowedto move either to the right or upwards.(b) How many of the paths in (a) pass through the (2,2) point ofthe grid?(c) What is the probability that a randomly selected path willpass through the (2,2) point?Hints: (a) A path can be represented by a 7-long binarysequence with four 1s (1 denotes a step to the right) and three0s (steps upwards).(b) The # of paths from the lower left corner to the (2,2) pointtimes the # of paths from (2,2) to the upper right corner.





ExampleAn order comes in for 5 palettes of low grade shingles. In thewarehouse there are 10 palettes of high grade, 15 of mediumgrade, and 20 of low grade shingles. An inexperienced shippingclerk is unaware of the distinction in grades of asphalt shinglesand he ships 5 randomly selected palettes.

1 How many different groups of 5 palettes are there?(455

)= 1,221,759.

2 What is the probability that all of the shipped palettes arelow grade?

(205

)/(45

5

)= 15,504/1,221,759 = 0.0127.

3 What is the probability that 2 of the shipped palettes are ofmedium grade and 3 are from low grade?[(15

2

)(203

)]/(45

5

)= (105× 1140)/1,221,759 = 0.098.





ExampleA communication system consists of 15 indistinguishableantennas arranged in a line. The system functions as long asno two non-functioning antennas are next to each other.Suppose six antennas stop functioning.

a) How many different arrangements of the sixnon-functioning antennas result in the system beingfunctional? (Hint: The 9 functioning antennas, lined upamong themselves, define 10 possible locations for the 6non-functioning antennas so the system functions.)

b) If the arrangement of the 15 antennas is random, what isthe probability the system is functioning?





ExampleWhat is the probability that 5 randomly dealt cards form a fullhouse?Solution: First, the number of all 5-card hands is(52

5

)=

52!5!47!

= 2,598,960. Next, think of the task of forming afull house as consisting of two stages. In Stage 1 choose twocards of the same kind, and in stage 2 choose three cards ofthe same kind. Since there are 13 kinds of cards, stage 1 canbe completed in

(131

)(42

)= (13)(6) = 78 ways (why?). For each

outcome of stage 1, the task of stage 2 becomes that ofselecting three of a kind from one of the remaining 12 kinds.This can be completed in

(121

)(43

)= 48 ways. Thus there are

(78)(48) = 3,744 possible full houses, and the desiredprobability is 0.0014.





Multinomial Coefficients

• The number of ways n units can be divide in r groups ofspecified sizes n1, . . . ,nr is given by(

nn1,n2, . . . ,nr

)=

n!n1!n2! · · · nr !

These numbers are called multinomial coefficients because ofthe Multinomial Theorem.

In how many ways can 8 engineers be assigned to work onprojects A, B, and C, so that 3 work on project A, 2 work onB, and 3 work on C?Answer: 560





Read Examples 2.3-8, 2.3-11, pp. 66, 67




The axioms governing any assignment of probabilities are:

Axiom 1: P(E) ≥ 0, for all events EAxiom 2: P(S) = 1Axiom 3: If E1,E2, . . . are disjoint

P(E1 ∪ E2 ∪ . . .) =∞∑

i=1

P(Ei)




Properties of Probability

• The following properties follow from the three axioms:

1) P(∅) = 0

2) If E1, . . . ,Em are disjoint, then

P(E1 ∪ · · · ∪ Em) = P(E1) + · · ·+ P(Em)

3) If A ⊂ B then P(A) ≤ P(B)

4) P(A) = 1− P(Ac), for any event A




• Two additional properties deal with the probability of the unionof events that are not disjoint:

5) P(A ∪ B) = P(A) + P(B)− P(A ∩ B)

6) P(A ∪ B ∪ C) = P(A) + P(B) + P(C)

− P(A ∩ B)− P(A ∩ C)− P(B ∩ C) + P(A ∩ B ∩ C)

• The formula for property 6 follows the so-called inclusion -exclusion principle and extends to the union of more than threeevents.




ExampleThe probability that a firm will open a branch office in Toronto is0.7, that it will open one in Mexico City is 0.4, and that it willopen an office in at least one of the cities is 0.8. Find theprobabilities that the firm will open an office in:

1 neither of the cities (Answer: 1− 0.8 = 0.2)2 both cities (Answer: 0.7 + 0.4− 0.8 = 0.3)3 exactly one of the cities (Answer:

(0.7− 0.3) + (0.4− 0.3) = 0.5, or 0.8− 0.3 = 0.5)




ExampleThe R commands attach(expand.grid(X1=0:1,X2=0:1,X3=0:1,X4=0:1)); table(X1+X2+X3+X4)/length(X1) yields thefollowing pmf for the random variable X = number of heads infour flips of a coin:

x 0 1 2 3 4p(x) 0.0625 0.25 0.375 0.25 0.0625

.

(a) What do Axiom 2 and property 2 say about the sum of allprobabilities? (Answer: They sum to 1)

(b) What is P(X ≥ 2)? (Answer:0.375 + 0.25 + 0.0625 = 0.6875)

Read also Examples 2.4-2, 2.4-3, pp. 75, 76


week 3 classical probability, part ipersonal.psu.edu/acq/401/course.info/week3.pdfwith probability...

Documents