STATISTIC & INFORMATION THEORY

(CSNB134)

MODULE 4PRINCIPLES OF PROBABILITY

Points to Ponder

Activities pertaining computer information systems are basically surrounded by uncertainty.

For example, the chances that our hard disk will crash within 2 years, the chances that data will get corrupted over a transmission channel, the accuracy of an error detection mechanism etc. – we can never be certain of these events.

However, it is important that we are able to induce these measurements as such we could take safeguards in protecting our data and our IT infrastructure.

Hence :- the applications of probability

Probability

Probability allows us to use sample data to conclude about the population.

Probability distribution models some random variables of the population.

Probability can be calculated given data obtained from observations.

An experiment is the process by which an observation (or measurement) is obtained.

An event is an outcome of an experiment. Experiment Events

Record an age A: person is 30 years oldB: person is older than 65

Toss a die A: observe an odd numberB: observe a number greater than 2

Overview of Probability

Experiment Events

Alien encounter A: An alien came up to my front door

P(A) = 0????

Sample Space

An event that cannot be decomposed is called a simple event. .

Denoted by E with a subscript. Each simple event will be assigned a probability,

measuring “how often” it occurs. The set of all simple events of an experiment is

called the sample space, , S. E.g. Toss a die S={E1, E2, E3, E4, E5, E6} Events::

A: observe an odd number {E1,E3,E5}

B: observe a number greater than 2 {E3,E4,E5,E6}

C: observe a 6 {E6}

D: observe a 3 {E3}

Mutual Exclusive

Two events are mutually exclusive if, when one event occurs, the other cannot, and vice versa.

E.g. Experiment: Toss a die Events:: A: observe an odd number {E1,E3,E5} B: observe a number greater than 2 {E3,E4,E5,E6} C: observe a 6 {E6} D: observe a 3 {E3}Mutual Exclusive? (1) A & B – NO (2) C & D – YES (3) A & C – YES (4) B & D - NO

The Probability of an Event

The probability of an event A measures “how often” we think A will occur. We write P(A).

Suppose that an experiment is performed n times. The relative frequency for an event A is

n

f

n

occurs A times ofNumber

n

fAP

nlim)(

n

fAP

nlim)(

If we let n get infinitely large,

The Probability of an Event (cont.)

P(A) must be between 0 and 1. If event A can never occur, P(A) = 0. If event A always occurs when the experiment is

performed, P(A) =1. The sum of the probabilities for all simple

events in S equals 1. The probability of an event A is found by

adding the probabilities of all the simple events in A.

E.g. Experiment: Toss a dieEvents Probability

A: observe an odd number P(A) = 3/6 = 1/2

B: observe a number greater than 2

P(B) = 4/6 = 2/3

C: observe a 6 P(C) = 1/6

Exercise 1

Toss a fair coin twice. What is the probability of observing at least one head?

HH

1st Coin 2nd Coin Ei P(Ei)

HH

TT

TT

HH

TT

HHHH

HTHT

THTH

TTTT

1/4

1/4

1/4

1/4

P(at least 1 head)

= P(E1) + P(E2) + P(E3)

= 1/4 + 1/4 + 1/4 = 3/4

P(at least 1 head)

= P(E1) + P(E2) + P(E3)

= 1/4 + 1/4 + 1/4 = 3/4

Exercise 2

A bowl contains three M&Ms®, one red, one blue and one green. A child selects two M&Ms at random. What is the probability that at least one is red?

1st M&M 2nd M&M Ei P(Ei)

RBRB

RGRG

BRBR

BGBG

1/6

1/6

1/6

1/6

1/6

1/6

P(at least 1 red)

= P(RB) + P(RG) +P(BR) + P(GR)

= 1/6 + 1/6 + 1/6 + 1/6 = 4/6 = 2/3

P(at least 1 red)

= P(RB) + P(RG) +P(BR) + P(GR)

= 1/6 + 1/6 + 1/6 + 1/6 = 4/6 = 2/3

m

m

m

m

m

m

m

m

mGBGB

GRGR

Counting Rules

Fortunately, there are several counting rules that can be applied in order to calculate the probability of events of an

experiment.- the mn rules

- permutations

- combinations

The mn Rule

Example: Toss two coins. The total number of simple events is:

2 X 2 = 4

If an experiment is performed in two stages, with m ways to accomplish the first stage and n ways to accomplish the second stage, then there are mn ways to accomplish the experiment.

This rule is easily extended to k stages, with the number of ways equal to n1 n2 n3 … nk

Exercise 3

Example: Toss three coins. The total number of simple events is:

Example: Two M&Ms are drawn from a dish containing two red and two blue candies. The total number of simple events is:

Example: Toss two dice. The total number of simple events is:

6 X 6 = 36

4 X 3 = 12

2 X 2 X 2 = 8

Exercise 4

Example: In data networking, a packet can take 2 routes from Router A to Router B, 3 routes from Router B to router C and 3 routes from Router C to Router D respectively. How many possible A to D routes are available?

If one link between Router B and C is down, how many possible A to D routes are available?:

2 X 3 X 3 = 18

2 X 2 X 3 = 12

Permutations

The number of ways you can arrange n distinct objects, taking them r at a time is

Example: How many 3-digit lock combinations can we make from the numbers 1, 2, 3, and 4?

.1!0 and )1)(2)...(2)(1(! where

)!(

!

nnnn

rn

nPnr

24)2)(3(4!1

!443 P 24)2)(3(4

!1

!443 PThe order of the

choice is important!

Exercise 5

Example: A lock consists of five parts and can be assembled in any order. A quality control engineer wants to test each order for efficiency of assembly. How many orders are there?

120)1)(2)(3)(4(5!0

!555 P 120)1)(2)(3)(4(5

!0

!555 P

The order of the choice is important!

Combinations

The number of distinct combinations of n distinct objects that can be formed, taking them r at a time is

Example: Three members of a 5-person committee must be chosen to form a subcommittee. How many different subcommittees could be formed?

)!(!

!

rnr

nC nr

101)2(

)4(5

1)2)(1)(2(3

1)2)(3)(4(5

)!35(!3

!553

C 10

1)2(

)4(5

1)2)(1)(2(3

1)2)(3)(4(5

)!35(!3

!553

CThe order

of the choice is not important!

Exercise 6

A box contains six M&Ms®, four red and two green. A child selects two M&Ms at random. What is the probability that exactly one is red?

The order of the choice is not important!

Ms.&M 2 choose toways

15)1(2

)5(6

!4!2

!662 C

M.&Mgreen 1

choose toways

2!1!1

!221 C

M.&M red 1

choose toways

4!3!1

!441 C 4 2 =8 ways to

choose 1 red and 1 green M&M.

P( exactly one red) = 8/15

In a Nutshell………

How to differentiate between Permutation & Combination? The ‘MADU 3’ ScenarioE.g. Abu is given a chance to choose 3 women as his wives amongst five pretty women Aminah, Salmah, Eton, Esah and Joyah. Case 1: What is the probability that Eton, Esah and Joyah are chosen as the first, second and third wife respectively?Order is important!! First wife must be Eton!!Use Permutations!!Case 2: What is the probability that Eton, Esah and Joyah are chosen as the wives?Order is not important!! Anyone can be the first wife. Use Combinations!!

Something to ponder….

Practice makes PURRRfect!!

U may do exercises from any text books on Probability & Statistics; amongst those that I am using:-

Mendenhall, Beaver & Beaver, ‘Introduction to Probability & Statistics’, 12th Edition, Thomson Brooks

STATISTIC & INFORMATION THEORY

(CSNB134)

PRINCIPLES OF PROBABILITY--END--