some basic probability concepts

Some Basic Probability Concepts

Some Basic Probability Concepts 2


Experiments, Outcomes and Random Variables• An experiment is the process by which an observation is made.• Sample Space: ‘set of all possible well distinguished outcomes of an

experiment’ and is usually denoted by the letter ‘S’.• For example, Tossing a coin: S= {H, T}, Tossing a die: S = {1,2,3,4,56}• Sample Point: ‘each outcome in a sample space’• Event: ‘Subset of the sample space’• A random variable is ‘a real valued function defined on the sample

space’.• A random variable is a variable whose value is unknown until it is

observed. The value of a random variable results from an experiment; it is not perfectly predictable.



• A discrete random variable can take only a finite number of values that can be counted by using the positive integers.

• A continuous random variable can take any real value (not just whole numbers) in an interval on the real number line

• A continuous random variable can take any real value (not just whole numbers) in an interval on the real number line.


The Probability Distribution of a Random Variable

• The term Probability is used to give a quantitative measure to the uncertainty associated with outcomes of a random experiment.

• Probability: The Classical Definition• In a random experiment, if there are ‘n’ equally likely and

mutually exclusive outcomes, of which ‘f’ are favorable to an event ‘A’, then the probability of occurrence of event A, denoted by P(A), is given by the ratio, f/n.

• The frequency approach: ‘the limit of relative frequency as the number of observations approached infinity’.


Some Basic Postulates

• Postulate 1: The probability of an event is a nonnegative real number; that is, 0 P (Ai) 1 for each subset Si of S;

• Postulate 2: P(S) = 1• Postulate 3: If S1, S2, S3,…Sn are mutually exclusive events defined on

the sample space S, then P(S1U S2U S3… U Sn) = P(S1) + P(S2) P(S3)+…+P(Sn)

• An Illustration:• Suppose we have information about the population in Comilla . We are

interested in two characteristics only, Sex (M or F) and economic status (Poor or Non poor). The two characteristics are not mutually exclusive.

• S = { (M & P), (F & P), (M & NP), (F &NP)}


If the population is finite, then the distribution is

Economic Status Totals

Sex

Poor Non poor

Male β α + β

Female γ δ γ + δ

Total α + γ + δα +β +γ +δ =

N


In terms of probabilities, the distribution would look like

Economic Status

Sex

Poor Non poor

Male P(M∩P) P(M∩NP) P(M)

Female P(F∩P) P(F∩NP) P(F)

P(poor) P(Non poor) 1


• The probabilities pertaining to intersection of sets are called joint probabilities. For instance, P (Male ∩ Poor) is the probability that a person selected at random in Comilla will be both male and poor, i.e., has two joint characteristics.

• The probabilities that appear in the last row and in the last column of the table are called marginal probabilities. P (M) gives the probability of drawing a male regardless of his economic status.

• It may be noted that marginal probabilities are equal to the corresponding joint probabilities.


• What is the probability that a person of given sex is poor, or that a person of given economic status is a male (female)? Such probabilities are called conditional probabilities. For instance, P(Poor/Male) means that we have a male and we want to find out the probability that he is poor, which is given by

)()()/(

MPPMPMPP


• When the values of a discrete random variable are listed with their chances of occurring, the resulting table of outcomes is called a probability function.

• For a discrete random variable X the value of the probability function f(x) is the probability that the random variable X takes the value x, f(x) =P(X=x).

• Therefore, 0 f(xi) 1 and, if X takes n values x1, .., xn, then.• For the continuous random variable Y the probability density

function f(y) can be represented by an equation, which can be described graphically by a curve. For continuous random variables the area under the probability density function corresponds to probability.


Probability function & Its Advantages

• Consider the experiment of tossing two six-sided dice. Define the random variable as the sum total of dots observed. Its values range from 2, 3.. to 12. The sample space will consist of all possible permutations of the two sets of numbers from 1 to 6. In sum, there will be 36 permutations.


The resulting probability distribution will be as follows:

X Elements of sample space

F(x)

2 11 1/363 12, 21 2/364 13, 31, 22 3/365 14, 41, 23, 32 4/366 15, 51, 24, 42, 33 5/367 16, 61, 25, 52, 34, 43 6/368 26, 62, 35, 53, 44 5/369 36, 63, 45, 54 4/3610 46, 64, 55 3/3611 56, 65 2/3612 66 1/36

1

3676

)(

x

xf


Expected Values Involving a Single Random Variable

• The Rules of Summation• If X takes n values x1, ..., xn then their sum is

• If a is a constant, then

• If a is a constant then

1 21

n

i ni

x x x x

1

n

i

a na

1 1

n n

i ii i

ax a x


• If X and Y are two variables, then

• If X and Y are two variables, then

• The arithmetic mean (average) of n values of X is

• Also,

1 1 1

( )n n n

i i i ii i i

x y x y

1 1 1

( )n n n

i i i ii i i

ax by a x b y

1 21

n

ini

xx x xx

n n

1

( ) 0n

ii

x x


• We often use an abbreviated form of the summation notation. For example, if f(x) is a function of the values of X,

• Several summation signs can be used in one expression. Suppose the variable Y takes n values and X takes m values, and let f(x, y) =x+y. Then the double summation of this function is

1 21

( ) ( ) ( ) ( )

= ( ) ("Sum over all values of the index ")

( ) ("Sum over all possible values of ")

n

i ni

ii

x

f x f x f x f x

f x i

f x X


• To evaluate such expressions work from the innermost sum outward. First set i=1 and sum over all values of j, and so on.

• To illustrate, let m = 2 and n = 3. Then

1 1 1 1

( , ) ( )m n m n

i j i ji j i j

f x y x y

2 3 2

1 2 31 1 1

1 1 1 2 1 3

2 1 2 2 2 3

, , , ,

, , ,

, , ,

i j i i ii j i

f x y f x y f x y f x y

f x y f x y f x y

f x y f x y f x y


• The order of summation does not matter, so

1 1 1 1

( , ) ( , )m n n m

i j i ji j j i

f x y f x y


The Mean of a Random Variable

• The expected value of a random variable X is the average value of the random variable in an infinite number of repetitions of the experiment (repeated samples); it is denoted E[X].

• If X is a discrete random variable which can take the values x1, x2,…,xn with probability density values f(x1), f(x2),…, f(xn), the expected value of X is

1 1 2 2

1

[ ] ( ) ( ) ( )

( )

( )

n n

n

i ii

x

E X x f x x f x x f x

x f x

xf x


Expectation of a Function of a Random Variable

• If X is a discrete random variable and g(X) is a function of it, then

• However, in general, if X is a discrete random variable and g(X) = g1(X) + g2(X), where g1(X) and g2(X) are functions of X, then

[ ( )] ( ) ( )x

E g X g x f x

1 2

1 2

1 2

[ ( )] [ ( ) ( )] ( )

( ) ( ) ( ) ( )

[ ( )] [ ( )]

x

x x

E g X g x g x f x

g x f x g x f x

E g x E g x


• The expected value of a sum of functions of random variables, or the expected value of a sum of random variables, is always the sum of the expected values.

• If c is a constant,

• If c is a constant and X is a random variable, then

• If a and c are constants then

[ ]E c c

[ ] [ ]E cX cE X

[ ] [ ]E a cX a cE X


The Variance of a Random Variable

• Let a and c be constants, and let Z = a + cX. Then Z is a random variable and its variance is

22 2 2var( ) [ ( )] ( ) [ ] [ ( )]X E g X E X E X E X E X

2 2var( ) [( ) ( )] var( )a cX E a cX E a cX c X


A Recap

• Probability: Basic Concepts• Classical & Frequency approaches• Some Basic Postulates• Some Examples• Probability function & its advantages• Mathematical expectation


Using Joint Probability Functions

• Marginal Probability Functions• If X and Y are two discrete random variables then

• Conditional Probability Functions

( ) ( , ) for each value can take

( ) ( , ) for each value can takey

x

f x f x y X

f y f x y Y

( , )( | ) [ | ]( )

f x yf x y P X x Y yf y


Independent Random Variables

• If X and Y are independent random variables, then

for each and every pair of values of x and y. The converse is also true.

• If X1, …, Xn are statistically independent the joint probability function can be factored and written as

( , ) ( ) ( )f x y f x f y

1 2 1 1 2 2( , , , ) ( ) ( ) ( )n n nf x x x f x f x f x


• If X and Y are independent random variables, then the conditional probability function of X given that Y=y is

for each and every pair of values x and y. The converse is also true.

( , ) ( ) ( )( | ) ( )( ) ( )

f x y f x f yf x y f xf y f y


The Expected Value of a Function of Several Random Variables: Covariance and Correlation

• If X and Y are random variables, then their covariance is

• If X and Y are discrete random variables, f(x,y) is their joint probability function, and g(X,Y) is a function of them, then

cov( , ) [( [ ])( [ ])]X Y E X E X Y E Y

[ ( , )] ( , ) ( , )x y

E g X Y g x y f x y


• If X and Y are discrete random variables and f(x,y) is their joint probability function, then

• If X and Y are random variables then their correlation is

cov( , ) [( [ ])( [ ])]

[ ( )][ ( )] ( , )x y

X Y E X E X Y E Y

x E X y E Y f x y

cov( , )=var( ) var( )

X YX Y


• The Mean of a Weighted Sum of Random Variables

• If X and Y are random variables, then

[ ] ( ) ( )E aX bY aE X bE Y

E X Y E X E Y


The Variance of a Weighted Sum of Random Variables

• If X, Y, and Z are random variables and a, b, and c are constants, then

• If X, Y, and Z are independent, or uncorrelated, random variables, then the covariance terms are zero and:

2 2 2var var var var

2 cov , 2 cov , 2 cov ,

aX bY cZ a X b Y c Z

ab X Y ac X Z bc Y Z

2 2 2var var var varaX bY cZ a X b Y c Z


• If X, Y, and Z are independent, or uncorrelated, random variables, and if a = b = c = 1, then

var var var varX Y Z X Y Z


Theoretical Derivation of Sampling Distribution of Estimators & Test Statistics:

• Binomial Distribution:• Comilla Story: Picking a BPL Person• Let p be the proportion of BPL population in Comilla and q

be the proportion of APL population. • Let n denote the sample size. • Let be the proportion of BPL in the sample. • Let X denote the number of poor in the sample. • If the person picked up happens to be poor, the

experiment is a success and its probability is p. Otherwise, it is a failure with a probability given by q, that is, (1-p).


• Let us define the sampling distributions of X and for samples of various sizes. Since = (X/n) or X = n , by the different results that we have learnt so far, we can determine the distribution of , if we know that of X and vice versa.

p̂


Sampling Distribution for n = 1

Number of Poor Probability: f(x) x f(x) x2 f(x)

0 P(F) = q 0 0

1 P(S) = p p p

Sum p + q = 1 p p


• Mean and variance of X:

• Mean and variance of :

i

ii pxfxXE )()(

pqpppp

xfxxfxXEXEXVari

iiii

i

)1(

)]([)()]([)()(

2

2222

p̂

E( ) – E(X/n) = E(X) = pp̂

Var( )=Var(X/n)=Var(X)=pq^p



Number of Poor Probability: f(x) xf(x) x2 f(x)

0 P(F)P(F) = q2 0 0

1P(F)P(S)+P(S)P(F)

= 2pq2pq 2pq

2 P(S)P(S) = p2 2p2 4p2

Sum (p+q)2 = 1 2p(p+q) 2p(2p+q)



• Mean and variance of :

i

ii pqppxfxXE 2)(2)()(

pqpppqppqp

xfxxfxXEXEXVari

iiii

i

2442)2()2(2

)]([)()]([)()(

222

2222

p̂

p E(X/2) E(X/n) )pE(^

(pq/2) )(1/4)Var(X Var(X/2) )p(Var ^



Number of

Poor

Probability:

f(x)xf(x) x2 f(x)

0 P(F) = q3 0 0

1pqq+ qpq+

qqp= 3pq23p2q 3q2p

2ppq+ pqp+

qpp=3p2q6pq2 12qp2

3 ppp = p3 3p3 9p2

Sum (p+q)3 = 1



• Mean and variance of

pqppppqqppqppqxfxXEi

ii 3)(3)2(3363)()( 222322


• In general, we have:


• That is, the probability of getting x poor people in a sample size of ‘n’ is

• Properties:


• E( ) = p, that is, unbiased estimator.• ,that is, the distribution gets concentrated

as sample size increases. This property together with (i) implies is a consistent estimator. The dispersion of the sampling distribution decreases in inverse proportion to the square root of sample size. That is, if sample size increases k times, then the std. deviation of the sampling distribution decreases times.

^p

^p

k


• The sampling distribution of is most dispersed when the population parameter p is equal to ½ and is least dispersed when p is 0 or 1.

^p


• The asymmetry (skewness) of the sampling distribution of decreases in inverse proportion to the square root of sample size (since ))))))))))))))))).

• It is least skewed when p = ½ and is most skewed when p is 0 or 1.

^p


The Normal Distribution

Properties: •The distribution is continuous and symmetric around its mean μ. This implies: (i) mean = median = mode; and (ii) the mean divides the area under the normal curve into exact halves. •The range of the distribution extends from -∞ to + ∞. In other words, the distribution is unbounded.


• The curve attains maximum height at x = μ; the points of inflection occur at x = μ σ(which means the standard deviation measures the distance from the center of the distribution to a point of inflection).

• Normal distribution is fully specified by two parameters, mean (μ) and variance (σ2). If we know these two parameters, we know all there is to know about it.

• If X, Y,…, Z are normally and independently distributed random variables and a,b,…,c are constants, then the linear combination aX+bY+…+cZ is also normally distributed.


How to calculate probabilities for a normal random variable?

• From tabulated results • Different normal distributions lead to different probabilities due to

differences in mean and variance. For the same reason, if we know the area under one specific normal curve, the area under any other normal curve can be computed by accounting for the differences in mean and variance.

• One specific distribution for which areas have been tabulated is a normal distribution with mean μ = 0 and variance σ2 = 1, called the standard normal distribution (also called unit normal distribution).

• Given that (i) X is normally distributed with mean μ and variance σ2; and (ii) the areas under the standard normal curve, how to determine the probability that x lies in some interval, say, (x1 and x2) ?


• Let Z denote a normally distributed variable with mean zero and variance equal to unity. That is,

• P(x1 < x < x2) = probability that X will lie between x1 and x2(x1 < x2); and P(z1< z <z2) = probability that Z will lie between z1 and z2 (z1 < z2) .

• Since X is normally distributed, a linear function of X will also be normal.


• Let it be denoted by aX + b, where a and b are constants.

• Choose a and b such that (aX+b) is a standard normal variable. That is,


• Solving for a and b , we get

• Thus, we have aX+b = = Z

• In other words, any variable with mean μ and variance σ2 can be transformed into a standard normal variable by expressing it as a deviation from its mean and dividing by σ.


• Consider P(x1 < x < x2) where x1 < x2. • From = Z, we get X = Z+. Hence, we

can write x1 = z1 + and x2 = z2 + • Now, P(x1 < x < x2) = P(z1 + < Z + <z2 +

) = P(z1 < Z < z2) • where z1 = and z2 =

)( X

) ( X

)( X

Thank You

some basic probability concepts

Documents

probability distribution

basic probability conceptswhat

probability th

term probability

probability of occurrence

random experiment

random variable results

sample spacea random