Chapter 12

Review of Calculus and Probability

to accompany

Operations Research: Applications and Algorithms

4th edition

by Wayne L. Winston

Copyright (c) 2004 Brooks/Cole, a division of Thomson Learning, Inc.

2

Description

A review of some basic topics in calculus and probability, which will be useful in later chapters.

3

12.1 Review of Differential Calculus

The limit is one of the most basic ideas in calculus.

Definition: The equation

means that as x gets closer to a (but not equal to a), the value of f(x) gets arbitrarily close to c.

It is also possible that may not exist.

cxfax

)(lim

)(lim xfax

4

Example 1

1. Show that

2. Show that does not exist.

Solution

1. To verify this result, evaluate x2-2x for values of x close to, but not equal to, 2.

2. To verify this result, observe that as x gets near 0, becomes either a very large positive number or a very large negative number.. Thus, as x approaches 0, will not approach any single number.

x

1

x

1

0)2(222lim 22

2

xx

x

xx

1lim

0

5

Definition: A function f(x) is continuous at a point a if

If f(x) is not continuous at x=a, we say that f(x) is discontinuous (or has a discontinuity) at a.

)()(lim afxfax

6

Example 2

Bakeco orders sugar from Sugarco. The per-pound purchase price of the sugar depends on the size of the order (see Table 1). Letx = number of pounds of sugar purchased by Bakeco

f(x) = cost of ordering x pounds of sugar

Then f(x) =25x for 0 ≤ x < 100f(x) =20x for 100 ≤ x ≤ 200f(x) =15x for x > 200

For all values of x, determine if x is continuous or discontinuous.

7

Example 2 – cont’d

SolutionIt is clear that and

do not exist. Thus, f(x) is discontinuous at x=100 and x=200 and is continuous for all other values of x satisfying x ≥ 0.

)(lim100

xfx

)(lim200

xfx

8

Higher Derivatives We define f(2)(a)=f ′′(a) to be the derivative of the

function f ′(x) at x=a.

Similarly, we can define (if it exists) f(n)(a) to be the derivative of f(n-1)(x) at x=a.

Thus, for Example 3,f′′ (p) = 3000e-p(-1) – 3000e-p(1-p)

Taylor Series Expansion In our study of queuing theory (Chapter 8), we will

need the Taylor series expansion of a function f(x).

9

Given that fn-1(x) exists for every point on the interval [a,b], Taylor’s theorem allows us to write for any h satisfying 0 ≤ h ≤ b – a,

(1)

where (1) will hold for some number p between a and a + h.

Equation (1) is the nth-order Taylor series expansion of the f(x) about a.

1)1(

1

)(

)!1(

)(

!

)()()(

n

ni

ni

i

i

hn

pfh

i

afafhaf

10

Example 4 Find the first-order Taylor series expansion of

e-x about x = 0.

SolutionSince f′(x) = -e-x and f′′(x) = -e-x, we know that (1) will hold on any interval [0,b]. Also, f(0) = 1, f′(0) = -1, and f′′(x) = e-x. Then (1) yields the following first-order Taylor series expansion for e-x about x=0:

This equation holds for some p between 0 and h.

21)(

2 ph eh

hhfe

11

Partial Derivatives We now consider a function f of n>1 variables (x1, x2,

…,xn), using the notation f (x1, x2, …,xn) to denote such a function.

Definition: The partial derivative of (x1, x2, …,xn) with respect to the variable xi is written , where

ix

f

i

ninii

xi x

xxxfxxxxf

x

f

i

),...,,...,(),...,,...,(lim 11

0

12

Suppose that for each i, we increase xi by a small amount Δxi. Then the value of f will increase by approximately

We will also use second-order partial derivatives extensively. We use the notation

to denote a second-order partial derivative.

i

ni

i i

xx

f

1

ji xx 2

13

Review of Integral Calculus

A knowledge of the basics of integral calculus is valuable when studying random variables.

Consider two functions: f(x) and F(x). If F′(x), we say that F(x) is the indefinite integral of f(x).

The fact that F(x) is the indefinite integral of f(x) is written

The definite integral of f(x) from x=a to x=b is written

dxxfxF )()(

b

adxxf )(

14

The Fundamental Theorem of Calculus states that if f(x) is continuous fro all x satisfying a ≤ x ≤ b, then

where F(x) is any indefinite integral of f(x).

)()()( aFbFdxxfb

a

15

12.2 Differentiation of Integrals

In order to differentiate a function whose value depends on an integral you should let f(x, y) be a function of variables x and y, and let g(y) and h(y) be functions of y. Then

is a function only of y.

Leibniz’s rule for differentiating an integral states that

)(

)(),()(

yh

ygdxyxfyF

dxy

yxfyygfygyyhfyhyFdxyxfyF

yh

yg

yh

yg

)(

)(

)(

)(

),()),(()()),(()()(then,),()(

16

12.3 Basic Rules of Probability Definition: Any situation where the outcome is

uncertain is called an experiment.

Definition: For any experiment, the sample space S of the experiment consists of all possible outcomes for the experiment.

Definition: An event E consists of any collection of points (set of outcomes) in the sample space.

Definition: A collection of events E1, E2,…,En is said to be a mutually exclusive collection of events if for i ≠ j (i=1,2,…,n and j=1,2,…n), Ei and Ej have no points in common.

17

With each event E, we associate an event Ē. Ē consists of the points in the sample space that are not in E.

With each event E, we also associate a number P(E), which is the probability that event E will occur when we perform the experiment.

The probabilities of events must satisfy the following rules of probability:

18

Rule 1For any event E, P(E) ≥ 0.

Rule 2 If E=S (that is, if E contains all points in the sample space), then P(E) = 1.

Rule 3If E1, E2,…,En is mutually exclusive collection of events, then

Rule 4P(Ē) = 1 – P(E)

nk

kkn EPEEEP

121 )()...(

19

Definition: Suppose events E1 and E2 both occur with positive probability. Events E1 and E2 are independent if and only if P(E2|E1)=P(E2)(or equivalently, P(E1|E2) = P(E1)).

20

12.4 Bayes’ Rule

Generally speaking, n mutually exclusive states of the world (S1, S2,…, Sn) may occur.

The states of the world are collectively exhaustive: S1, S2,…, Sn include all possibilities.

Suppose a decision maker assigns a probability P(Si) to Si. P(Si) is the prior probability of Si.

Suppose that for each possible outcome Oj and each possible state of the world Si, the decision maker knows P(Oj|Si), the likelihood of the outcome Oj given the state of the world Si.

21

Bayes’ rule combines prior probabilities and likelihoods with the experimental outcomes to determine a post-experimental probability, or posterior probability, for each state of the world.

Bayes’ rule:

nk

kkkj

iijj

SPSOP

SPSOPOSP

1

1

)()|(

)()|()|(

22

12.5 Random Variables, Mean, Variance, and Covariance

Definition: A random variable is a function that associates a number with each point in an experiment’s sample space. We denote random variables by boldface capital letters (usually X, Y, or Z).

Definition: A random variable is discrete if it can assume only discrete values x1, x2,…. A discrete random variable X is characterized by the fact that we know the probability that X = xi (written P(X=xi)).

23

P(X=xi) is the probability mass function (pmf) for the random variable X.

Definition: The cumulative distribution function F(x) = P(X≤x). For a discrete random variable X,

)()( kxxhavingxall

xPxFk

X

24

Continuous Random Variables

If, for some interval, the random variable X can assume all values on the interval, then X is a continuous random variable.

Probability statements about a continuous random variable X require knowing X’s probability density function (pdf).

The probability density function f(x) for a random variable X may be interpreted as follows: For Δ small,

)()( xfxxP X

25

Mean and Variance of a Random Variable

The mean (or expected value) and variance are two important measures that are often used to summarize information contained in a random variable’s probability distribution.

The mean of a random variable X (written E(X)) is a measure of central location for the random variable.

Mean of a discrete random variable X.

all k

kk xPxE )()( XX

26

Mean of a continuous random variable,

For a function h(X) of a random variable X (such as X2 and eX), E[h(X)] may be computed as follows: If X is a discrete random variable,

If X is a continuous random variable,

dxxxfE )()(X

kall

kk xPxhhE )()()]([ XX

dxxfxhhE )()()]([ X

27

Variance of the discrete random variable X,

Variance of a continuous random variable, X,

Also, var X may be found from the relation

For any random variable X, (var X)½ is the standard deviation of X (written σx).

kall

kk xPEx )()]([var 2 XXX

dxxfEx )()]([)var( 2XX

22 )()(var XXX EE

28

Independent Random Variables Definition: Two random variables X and Y

are independent if and only if for any two sets A and B,

The definition of independence generalizes to situations where more than two random variables are of interest.

Loosely speaking, a group of n random variables is independent if knowledge of the values of any subset of the random variables does not change our view of the distribution of any of the other random variables.

)()()( BPAPBandAP YXYX

29

For two random variables X and Y, the covariance of X and Y (written (X,Y) is defined by

)]}()][({[)cov( YYXXYX, EEE

30

Mean, Variance, and Covariance for Sums of Random Variables

From given random variables X1 and X2, we often create new random variables (c is a constant): cX1, X1+c, X1+X2.

The following rules can be used to express the mean, variance, and covariance of these random variables in terms of E(X1), E(X2), varX1, varX2, and cov(X1,X2).

E(cX1)=cE(X1)

E(X1+c)=E(X1)+c

E(X1+X2)=E(X1)+E(X2)

31

var cX1 = c2varX1

var(X1+c) = var X1

If X1 and X2 are independent variables,

var(X1+X2) = varX1 + varX2

In general,

var(X1+X2) = varX1 + varX2 + 2cov(X1+X2)

For random variables X1, X2 ,… Xn,

Finally, for constants a and b,cov(aX1, bX2)=ab cov(X1,X2)

ji

jinn )( ),cov(varvarvarvar 2121 XXXXXXXX

32

12.6 The Normal Distribution

Definition: A continuous random variable X has a normal distribution if for some µ and σ > 0, the random variable has the following density function:

2

2

21 2

)(exp

)2(

1)(

xxf

33

Useful Properties of Normal Distributions

Property 1: If X is N(µ,σ2), then cX is N(cµ,c2 σ2).

Property 2: If X is N(µ,σ2), then X + c (for any constant c) is N(µ+c, σ2).

Property 3: If X1 is N(µ1,σ12), X2 is N(µ2,σ2

2), and X1 and X2 are independent, then X1+X2 is N(µ1+µ2,σ1

2+ σ22).

34

Finding Normal Probabilities via Standardization

If Z is a random variable that is N(0,1), the Z is said to be a standardized normal random variable.

If X is N(µ,σ2), then (X- µ)/σ is N(0,1).

Suppose X is (µ,σ2) and we want to find P(a ≤ X ≤ b). We use the following relations (this procedure is called standardization):

aF

bF

baP

baPbaP

Z

XX )(

35

If X1, X2,…Xn are independent random

variables, then for n sufficiently large (usually n>30), the random variable X = X1 + X2 +…+Xn may be closely approximated by a normal random variable X′ that has E(X′) = E(X1) + E(X2) +…+E(Xn) and varX′ = varX1 + varX2+…+varXn.

This result is known as the Central Limit Theorem.

When we say that X′ closely approximates X, we mean that P(a ≤ X ≤ b) is close to P(a ≤ X′ ≤ b).

36

1.7 Finding Normal Probabilities with Excel

Probabilities involving a standard normal random variable can be determines with the EXCEL=NORMSDIST function.

The S in NORMSDIST stands for standardized normal.

For example, P(Z≤-1) can be found by entering the formula =NORMSDIST(-1).

The EXCEL=NORMSDIST function can be used to determine a normal probability for any normal (not just a standard normal) random variable.

37

If X is N(µ,σ2) the entering formula =NORMSDIST(a,µ,σ,1) will return P(X≤a).

The 1 ensures that EXCEL returns the cumulative normal probability. Changing the last argument to “0” causes EXCEL to return the height of the normal density function for X=a.

38

12.7 Z-Transforms

Consider a discrete random variable X whose only possible values are nonnegative integers.

For n=0,1,2,… let P(X=n)=an. We define (for |z|≤1) the z-transform of X (call it px

T(z)) to be

n

n

nn zaz

0)( X