chapter 2 probabilitybaek.math.umbc.edu/stat355/ch2.pdfseungchul baek stat 355 introduction to...

Chapter 2 Probability

Seungchul Baek

STAT 355 Introduction to Probability and Statistics for Scientists andEngineers

Seungchul Baek STAT 355 Introduction to Probability and Statistics for Scientists and Engineers 1

Overview of Probability

Probability is a measure of one’s belief in the occurrence of a future event.Here are some events to which we may wish to assign a probability:

tomorrow’s temperature exceeding 80 degreesmanufacturing a defective partconcluding one fertilizer is superior to another when it isn’tthe NASDAQ losing 5 percent of its value.you being diagnosed with lung cancer in the next 20 years.


Interesting Problem (We Will Revisit This Problem)

(Birthday Problem) What is the probability that at least two students inthe class (n = 48) have the same birthday–month and day? Suppose thatyear has 365 days (forget leap year) and equal likelihood for each day.


Elements of Probability: Random Experiment

Tossing a coin once.

Tossing a coin three times.

Observing the lifetime of an electronic component.

Drawing three cards (with replacement) from a standard deck of cards.

Prior to performing the experiment or activity, you do not know for certainwhich particular outcome will occur.


Elements of Probability: Sample Space

The set of all possible outcomes for a given random experiment is called thesample space, denoted by S. (discrete vs. continuous)

Tossing a coin once. S =

Tossing a coin three times. S =

Drawing three cards (with replacement) from a standard deck of cards.S =

Observing the lifetime of an electronic component. S =

The element of the sample space is called outcome, denoted by ω. We usenS to denote the number of elements in the sample space.


Elements of Probability: Event

An event is any collection of outcomes contained in the sample spaceS.

An event is a set of possible outcomes that is of interest. Usingmathematical notation, suppose that S is a sample space for a randomexperiment. We say that A is an event in S if the outcome ω satisfies{ω ∈ S : ω ∈ A}.

We would like to develop a mathematical framework so that we canassign probability to an event A. This will quantify how likely the eventis. The probability that the event A occurs is denoted by P(A).


Example 2.3 and 2.6

Two gas stations are located at a certain intersection. Each one has six gaspumps. Consider experiment in which the number of pumps in use at aparticular time of day is determined for each of stations.

What is the sample space?

What is the event that the total number of pumps in use five?

What is the event that at least one pump is in use at each station?


Set Theory

The union of two events A and B is the set of all outcomes ω in eitherevent or both. By notation,

A ∪ B = {ω : ω ∈ A or ω ∈ B}

Using Venn diagram, the union can be expressed as


Set Theory

The intersection of two events A and B is the set of all outcomes ωin the both events. By notation,

A ∩ B = {ω : ω ∈ A and ω ∈ B}

Using Venn diagram, the intersection can be expressed as


Set Theory

Mutually exclusive events can not occur at the same time. If eventsA and B are mutually exclusive, then A ∩ B = ∅, where ∅ is the nullevent.

Using Venn diagram, the mutually exclusive events can be expressed as


Example

Suppose there are three events, A, B, and C in the sample space S, andeach of them is not disjoint to each other.

Draw Venn diagrams for the following:

A ∪ B ∪ C

A ∩ B ∩ C

A ∩ B ∪ C

A ∩ (B ∪ C)


Complementary Events

The complement of an event is the set of the outcomes not included inthe event, but still part of the sample space. The complement of A isdenoted by A or AC or A′:

AC = {ω : ω /∈ A and ω ∈ S}

The Venn diagram is


Complementary Events

DeMorgan’s Law: it is useful when the calculation involves two events andtheir complements. Let A and B be two events

(A ∪ B)C = AC ∩ BC

(A ∩ B)C = AC ∪ BC


Kolmogorov Axioms

For any sample space S, a probability P must satisfy

0 ≤ P(A) ≤ 1, for any event AP(S) = 1If A1, A2, . . . , An are pairwise mutually exclusive events, then

P( n⋃

i=1Ai

)=

n∑i=1

P(Ai).

Note that “pairwise mutually exclusive” means that Ai ∩Aj = ∅ for all i 6= j ,i , j = 1, 2, ..., n.


Properties

These three properties are usually referred to as the Axioms ofProbability. Any function P satisfies the Axioms of Probability is called aprobability function. Other nice properties can be derived simply by usingthem. For example,

P(∅) = 0

If A ⊆ B then P(A) ≤ P(B)

P(A) = 1− P(AC )


Properties

If events A and B intersect, you have to subtract out the “doublecount”. From the Venn diagram, it is easy to see

P(A ∪ B) = P(A) + P(B)− P(A ∩ B)

which is called additive law.

If A and B are mutually exclusive, then A ∩ B = ∅ and P(∅) = 0, itfollows that

P(A ∪ B) = P(A) + P(B)


Example: Roommate Profile

Suppose your old roommate moves out, and you want to pick up a newroommate.The following table summarizes the roommate profile

Table 1: Roommate Profile (Frequency-Counts)

Snore Not.snore sumLike.party 150 100 250Dislike.party 200 550 750sum 350 650 1000


Example: Roommate Profile

Table 2: Roommate Profile (Relative Frequency-Probability)

Snore Not.snore sumLike.party 0.15 0.10 0.25Dislike.party 0.20 0.55 0.75sum 0.35 0.65 1.00

What is the probability that a randomly chosen roommate will snore?

What is the probability that a randomly chosen roommate will like toparty?

What is the probability that a randomly chosen roommate will snore orlike to party?


Equiprobability Model

Suppose that a sample space S contains nS <∞ outcomes, each ofwhich is equally likely. If the event A contains nA outcomes, then

P(A) = nAnS

Tossing a fair coin three times. Let eventA = {exactly two heads} = { }. Then, P(A) =

If I randomly pull a card out of a pack of 52 cards, what is the chanceit’s a spade card? Not a diamond?

Warning: If the outcomes in S are not equally likely, then this result isnot applicable.


Interpretation of Probability

One particular interpretation of probability is to view probability as alimiting proportion.

If the experiment is repeatable, then P(A) can be interpreted as thepercentage of times that A will occur “over the long run.” This iscalled the relative frequency interpretation.


Simulation: Toss A Fair Coin 5000 Times

0 1000 2000 3000 4000 5000

0.4

0.5

0.6

0.7

0.8

0.9

1.0

Toss fair coin 5000 timesand keep track of proportion of heads

Times

Pro

port

ion

of h

eads


Example 2.14

In a certain residential suburb, 60% of all households get Internet servicelocal cable company, 80% get television service from that company, and50% get both services from that company. Suppose that a household israndomly selected.

What is the probability that it gets at least one of these services fromthe company?

What is the probability that it gets exactly one of these services fromthe company?


Counting Methods

For equal likelihood probability model, the probability of event Adepends on the size of A, i.e., the number of elements in A, which isdenoted by nA.

Recall that if an experiment can result in any one of N different, butequally likely, outcomes and if exactly n of these outcomes correspondsto event A, then the probability of event A is

P(A) = nAnS

= nN .


Counting Methods

Consider selecting an element from a set having m elements and selectinganother from a set having n elements. Then there are m × n possible pairsthat could be selected.

Example 1. Tossing 3 coins, the number of possible outcomes is

2× 2× 2 = 8.

Example 2. We roll a 6-sided die, a 4-sided die, and a 20-sided die.The number of possible outcomes?


Counting Methods

There are 5 processes needed to manufacture the side panel for a car: clean,press, cut, paint, polish. Our plant has 6 cleaning stations, 3 pressingstations, 8 cutting stations, 5 painting stations, and 8 polishing stations.

How many possible different “pathways” through the manufacturingexist?

What is the number of “pathways” that include a particular pressingstation?

Assuming stations are chosen randomly, what is the probability that apanel follows any particular path?

Assuming stations are chosen randomly, what is the probability that apanel goes through pressing station 1?


Permutation: Ordered Samples

Definition: A permutation is an ordered arrangement of a specific numberof distinct objects (Order matters!). We denote the number of possiblepermutations of n distinct objects, taken r at a time, as:

nPr = Pnr = Pr ,n =≡ n× (n− 1)× (n− 2)× · · · × (n− r + 1) = n!

(n − r)! ,

where n ≥ r and n! ≡ n(n − 1)(n − 1) · · · (2)(1). n! read as “n factorial”and we define 0! = 1.

Example: Let there be n = 10 people and suppose that we are to sit rof them into r = 6 chairs, assume the chairs are numbered. Then thenumber of ways of doing this is . . . . . . . . . . . .


Combinations: Unordered Samples

Definition: Different selections of objects in which the order of objectsdoes not matter are called combinations.

Theorem: The number of unordered subsets of size r chosen (withoutreplacement) from n available objects is:

nCr = Cnr =

(nr

)≡ n!

r !(n − r)!

- Example: Let there be 10 people and suppose we want to obtain a sampleof size 5 from this group, with the order in which they are included notimportant. Then the number of possible samples is . . . . . . . . . . . .


Relationship

nPr = n!(n − r)!

nCr = n!r !(n − r)!

=⇒ nCr × r ! =n Pr .

Why? Once we have chosen r objects, there are r ! ways to permutethose r objects.


Example: Counting Methods

9 out of 100 computer chips are defective. We choose a random sample ofn = 3.

How many different samples of 3 are possible?

How many of the samples of 3 contain exactly 1 defective chip?

What is the probability of choosing exactly 1 defective chip in a randomsample of 3? (This is called hypergeometric distribution later).

What is the probability of choosing at least 1 defective chip in arandom sample of 3? (Hint: it is easier to calculate the probability ofchoosing no defective chip.)


Example: Coin Tossing

What is the probability of getting 4 or 5 heads when tossing a fair coin 10times?


Example: Coin Tossing (cont’d.)

It follows that P(A) =(10

4)0.510. By a similar argument, one can show that

P(B) =(10

5)0.510. Therefore,

P(A ∪ B) = P(A) + P(B) =(104

)0.510 +

(105

)0.510 = 0.4512.

Later, we will solve this problem using binomial distribution.


Example 2.23

A university warehouse has received a shipment of 25 printers, of which 10are laser printers and 15 are inkjet models. If 6 of these 25 are selectedrandomly to be checked by a technician, what is the probability that exactly3 of those selected are laser printers?


Conditional Probability

A conditional probability is the probability that an event will occur, whenanother event is known to occur or to have occurred

Definition: Let A and B be events in a sample space S with P(B) > 0.The conditional probability of A, given that B has occurred, is

P(A|B) = P(A ∩ B)P(B) .

Solving the conditional probability formula for the probability of theintersection of A and B yields

P(A ∩ B) = P(A|B)× P(B),

which is called the multiplication rule.


Conditional Probability

If B = S, then the conditional probability of A, given B has occurred, is

P(A|B) = P(A|S) = P(A ∩ S)P(S) = P(A)

P(S) = P(A)1 = P(A)

If B 6= S, then we can understand the conditional probability in this informalway: we are only interested in the probability of event A when event B hasoccurred. So, we manually “shrink” the sample space S to be B so that wecan calculate the probability of A in the restricted space with the scale of B.


Example

We roll a fair die. Define A = ‘Roll a 5’ and B = ‘Roll an odd number’.Clearly P(A) = 1/6 and P(B) = 1/2. Given that B is true,what is theprobability of A?


Example

In a company, 36 percent of the employees have a degree from a SECuniversity, 22 percent of the employees that have a degree from the SEC arealso engineers, and 30 percent of the employees are engineers. An employeeis selected at random.

Compute the probability that an employee is an engineer and is fromthe SEC.

Compute the conditional probability that an employee is from the SEC,given that s/he is an engineer.

Let A = {The employee is an engineer} andB = {The employee is from SEC}.


Example: Roommate Profile Revisit

Given that a randomly chosen roommate snores, what is the probability thathe/she likes to party?

Table 3: Roommate Profile (Frequency-Counts)

Snore Not.snore sumLike.party 150 100 250Dislike.party 200 550 750sum 350 650 1000


Example: Conditional Probability (cont’d.)

Note that, in both examples, the conditional probability P(B|A) andthe unconditional probability P(B) are NOT equal. In other words,knowledge that A “has occurred” has changed the likelihood that Boccurs.

In other situations, it might be that the occurrence (or non-occurrence)of a companion event has no effect on the probability of the event ofinterest. This leads us to the definition of independence.


Independence

Definition: Two events are independent if any one of the followingequivalent statement is true:

(1) P(A|B) = P(A)

(2) P(B|A) = P(B)

(3) P(A ∩ B) = P(A)P(B)


Example 2.29

An electronics store sells three different brands of DVD players. Of its DVDplayer sales, 50% are brand 1 (the least expensive), 30% are brand 2, and20% are brand 3. Each manufacturer offers a 1-year warranty on parts andlabor. It is known that 25% of brand 1’s DVD players require warrantyrepair work, whereas the corresponding percentages for brands 2 and 3 are20% and 10%, respectively.

What is the probability that a randomly selected purchaser has boughta brand 1 DVD player that will need repair while under warranty?

What is the probability that a randomly selected purchaser has a DVDplayer that will need repair while under warranty?

If a customer returns to the store with a DVD player that needswarranty repair work, what is the probability that it is a brand 1 DVDplayer? A brand 2 DVD player? A brand 3 DVD player?


Example 2.34

It is known that 30% of a certain company’s washing machines requireservice while under warranty, whereas only 10% of its dryers need suchservice. If someone purchases both a washer and a dryer made by thiscompany, what is the probability that both machines will need warrantyservice?


Example: Complementary Events

The probability that Tom will be alive in 20 years is 0.75 (A). Theprobability that Nancy will be alive in 20 years is 0.85 (B).

Assuming independence, what is the probability that neither will bealive 20 years from now?

Still assuming independence, what is the probability that only one ofthe two, Tom or Nancy, will be alive in twenty years?


Question

If events A and B are mutually exclusive, then A and B are independent.True or false?


Partition

Definition: For some positive integer k, the collection of sets{B1, B2, · · · , Bk} forms “a” partition of S if:

(i) S = B1 ∪ B2 ∪ · · · ∪ Bk

(ii) Bi ∩ Bj = ∅, for i 6= j

Note: For any subset A of S and any partition {B1, B2, · · · , Bk} of S,we can decompose A as:

A = A ∩ S = A ∩ (B1 ∪ B2 ∪ · · ·Bk)

= (A ∩ B1) ∪ (A ∩ B2) ∪ · · · ∪ (A ∩ Bk).

Question: Partition is unique?


The Law of Total Probability

A = A ∩ S = A ∩ (B1 ∪ B2 ∪ . . . Bk)= (A ∩ B1) ∪ (A ∩ B2) ∪ · · · ∪ (A ∩ Bk)= P(A ∩ B1) + P(A ∩ B2) + · · ·+ P(A ∩ Bk)= P(A | B1)P(B1) + P(A | B2)P(B2) + · · ·+ P(A | Bk)P(Bk)

=k∑

i=1P(A | Bi)P(Bi),

which is called the Law of Total Probability.


Bayes Theorem

P(B | A) = P(A ∩ B)P(A) = P(A | B)P(B)

P(A)

Further?


Example 2.31

Only 1 in 1000 adults is afflicted with a rare disease for which a diagnostictest has been developed. The test is such that when an individual actuallyhas the disease, a positive result will occur 99% of the time, whereas anindividual without disease will show a positive test result only 2% of thetime. If a randomly selected individual is tested and the result is positive,what is the probability that the individual has the disease?


Example

We purchase 30% of our parts from Vendor A. Vendor A’s defectiverate is 5%. What is the probability that a randomly chosen part is adefective part from Vendor A?

We are manufacturing widgets. 50% are red and 30% are white. Whatis the probability that a randomly chosen widget will not be white?


Example

When a computer goes down, there is a 75% chance that it is due tooverload and a 15% chance that it is due to a software problem. There isan 85% chance that it is due to an overload or a software problem. What isthe probability that both of these problems are at fault?


Example

It has been found that 80% of all accidents at foundries involve human errorand 40% involve equipment malfunction. 35% involve both problems. If anaccident involves an equipment malfunction, what is the probability thatthere was also human error?


Example

Consider the following electrical circuit: The probability on the componentsis their reliability (probability that they will operate). Components areindependent of each other. What is the probability that the circuit will notoperate when the switch is thrown?


Example

Consider the electrical circuit below. Probabilities on the components arereliabilities and all components are independent. What is the probabilitythat the circuit will work when the switch is thrown?


Example

An insurance company classifies people as ‘accident-prone’ and‘nonaccident-prone’. For a fixed year, the probability that an accident-proneperson has an accident is 0.4, and the probability that a non-accident-proneperson has an accident is 0.2. The population is estimated to be 30 percentaccident-prone. Define the events

A = {policy holder has an accident}

B = {policy holder is accident-prone}

We are given that P(B) = 0.3, P(A | B) = 0.4, and P(A | BC ) = 0.2.

What is the probability that a new policy-holder will have an accident?

Suppose that the policy-holder does have an accident. What is theprobability that s/he is ‘accident-prone’?


Birthday Problem

What is the probability that at least two students in the class (n=48) havethe same birthday - month and day?

Year has 365 days, i.e., forget leap year.Equal likelihood for each day.


Birthday Problem: R Code

It seems that there are two different ways to calculate with R. One codemay result in an error.

choose(365, 48) * factorial(48) / 365^48factorial(365) / (365^48 * factorial(365-48))


chapter 2 probabilitybaek.math.umbc.edu/stat355/ch2.pdfseungchul baek stat 355 introduction to...

Documents