mathcad for in class examples in a random processes course
TRANSCRIPT
2006-812: MATHCAD FOR IN-CLASS EXAMPLES IN A RANDOM PROCESSESCOURSE
James Reising, University of EvansvilleJAMES A. REISING is an Associate Professor of Electrical Engineering at the University ofEvansville, Evansville, Indiana, where he has taught since 1980. Prior to that time he wasemployed by Eagle-Picher Industries at the Miami Research Laboratories and the Electro-OpticMaterials Department. He is a senior member of IEEE.
© American Society for Engineering Education, 2006
Page 11.913.1
Mathcad™ for In-class Examples in a Random Processes Course
Abstract
Some textbooks1,2 used for courses in Probability and Random Processes use MATLAB™ for
computer demonstrations and exercises, while others3,4,5
do not emphasize specific computer
exercises or examples, and at least one6 uses C programs. I find it convenient to use Mathcad for
lecture examples. The appearance of calculations onscreen is practically identical to equations
appearing in the text, and graphs of illustrative results appear on the same page as the equations.
Calculations are all "live", causing graphs to be updated when the values of one or more
parameters are changed. This rapid response provides faster visual representation of important
concepts.
This paper gives examples of Mathcad worksheets for use during lecture in a course discussing
probability and random processes. The ability to easily and quickly change parameters used in
the equations with automatic updating of graphical display of results helps illustrate the concepts
involved.
Background
EE 413 Random Signals and Noise has been offered three times at the University of Evansville.
By the second offering it had become obvious that students found many of the concepts in
probability and random processes difficult to grasp, and the need for additional in-class examples
was apparent. I used Mathcad examples in class for the first time in the fall semester 2005.
The textbooks1,2 used (in different semesters) both use MATLAB in sample problems and
homework. MATLAB is used as well in a number of other courses in the EE curriculum and is
available on the department network, so students need not purchase the program. I have found it
useful to use Mathcad during lecture for illustrating the role of various parameters in
probabilistic examples for the reasons cited in the abstract section above. I use Mathcad only for
lecture examples, so students need not use or purchase the software.
Probability Density and Distribution Functions
The probability density function of a random variable X, )(xf X , and the cumulative probability
distribution function of X, )(xFX , are key concepts in the study of random variables. Mathcad
includes built-in functions for common distributions of both discrete and continuous random
variables, including the discrete binomial and Poisson distributions, and the continuous
exponential, Cauchy, and Gaussian (normal) distributions. All these functions include one or
more parameters characterizing a particular instance of the distribution. Simple plots of the
density and distributions functions can be used in class to examine the effects of varying the
values of the parameters.
Page 11.913.2
Figure 1 Figure 2
Figures 1 and 2 above illustrate a Mathcad worksheet using two different values of the parameter
p for the binomial distribution. Since the worksheet calculations are “live”, all the plots are
updated as soon as the parameter value is changed at the top of the page. By observing the plots
for several different values of p, one not only sees how the distribution functions change, but
may also notice that the peak of the density function occurs at or near a value that is the product
of n and p. The theoretical result that the mean value of a random variable having a binomial
distribution is np then seems quite reasonable. The Mathcad functions dbinom and pbinom
return values of the probability density and cumulative distribution, respectively. The third plot
shows that summing the probability density function gives the same result for the cumulative
distribution function as the built-in function.
Figures 3 and 4 below illustrate similar results for two values of the parameter α in the Poisson
distribution.
Figure 3 Figure 4
Binomial Distribution
n 20:= p .7:= k 0 n..:=
1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 210
0.1
0.2
dbinom k n, p,( )
k
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 200
0.5
1
pbinom k n, p,( )
k
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 200
0.5
1
0
k
i
dbinom i n, p,( )∑=
k
Binomial Distribution
n 20:= p .2:= k 0 n..:=
1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 210
0.1
0.2
dbinom k n, p,( )
k
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 200
0.5
1
pbinom k n, p,( )
k
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 200
0.5
1
0
k
i
dbinom i n, p,( )∑=
k
Poission Random Variables
j 1− 25..:=
λ .1:= T 20:= α λ T⋅:= α 2=
1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 250
0.2
dpois j α,( )
j
1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 250
0.5
1
ppois j α,( )
j
Poission Random Variables
j 1− 25..:=
λ .8:= T 20:= α λ T⋅:= α 16=
1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 250
0.2
dpois j α,( )
j
1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 250
0.5
1
ppois j α,( )
j Page 11.913.3
A discrete probability distribution for which Mathcad has no built-in functions is the Pascal
distribution given in the text by Yates and Goodman2 , where kxk
X ppk
xxf −−
−
−= )1(
1
1)( for
10 << p , k is an integer such that 1≥k , and x is an integer. This may be expressed in a form
that relates it to the binomial distribution as kxk
X ppk
x
x
kxf −−
= )1()( , which is just
x
ktimes the
binomial density function. The Pascal density function should be zero for values of x less than k.
Figure 5 Figure 6
Figures 5 and 6 above show a Mathcad worksheet for two different combinations of values of p
and k.
Probability density and cumulative distribution functions for continuous random variables may
be displayed and examined in much the same way. For continuous random variables, however,
the probability density function gives the probability that the random variable has a value in an
infinitesimal interval about a particular value. To display the probability that a random variable
assumes a value within a finite range of values, a histogram is useful. While the probability
density function for a continuous random variable may assume a value greater than one at some
point, the probability that the variable has a value within a certain range must always be less than
or equal to one. The Mathcad worksheet in Figures 7 and 8 below illustrates this feature of
continuous distributions as well as the generation of a sequence of random numbers with a
particular probability distribution.
Pascal Random Variable as in Yates and Goodman
p .4:= k 7:= x 0 40..:=
0 10 20 30 400
0.05
0.1
k
xdbinom k x, p,( )⋅
x
Pascal Random Variable as in Yates and Goodman
p .5:= k 4:= x 0 40..:=
0 10 20 30 400
0.1
0.2
k
xdbinom k x, p,( )⋅
x
Page 11.913.4
Figure 7 Figure 8
This worksheet uses the built-in function rnorm to generate a vector of random numbers having a
normal distribution. It also uses the histogram function provided in Mathcad. The histogram
function produces a two-column matrix where the first column (h<0>
in Mathcad notation)
contains the bin centers for the histogram, and the second column (h<1>
) contains the frequencies
with which the data falls into the bins. The plots may be used to examine several key points.
Observing the change in the shape of the distribution due to variations in the mean and standard
deviation, µ and σ, respectively, provides some feel for what these parameters mean.
Contrasting the plot of the density function and the solid line representing theoretical
probabilities on the histogram gives some insight into the difference between the probability
density and the probability that a continuous random variable assumes a value within a certain
range. Changing the number of data points and comparing the actual mean and histogram of
random numbers to theoretical values illustrates the relationship between theory and sample size.
Figures 9 and 10 below show the same worksheet with changes in the distribution parameters.
The gaussian, or normal, probability distribution
n 50:= the number of random numbers to generate
L 3:= the absolute value of the maximum argument
µ 1:= the mean value σ .25:= the standard deviation
note the pdf may be greater than 1 dnorm µ µ, σ,( ) 1.596=
i 0 20 L⋅ 1+..:= intvlsi
i
10L−:=
v rnorm n µ, σ,( ):= the vector of random numbers mean v( ) 1.044=
h histogram intvls v,( ):= q pnorm h0⟨ ⟩
.05+ µ, σ,( ) pnorm h0⟨ ⟩
.05− µ, σ,( )−:=
First look at the density function
3 2 1 0 1 2 30
1
2
dnorm h0⟨ ⟩ µ, σ,( )
h0⟨ ⟩
Then look at histogram and randomly generated numbers
3 2 1 0 1 2 30
0.06
0.12
0.18
0.24
0.3
h1⟨ ⟩
n
q
h0⟨ ⟩
h0⟨ ⟩,
The gaussian, or normal, probability distribution
n 50000:= the number of random numbers to generate
L 3:= the absolute value of the maximum argument
µ 1:= the mean value σ .25:= the standard deviation
note the pdf may be greater than 1 dnorm µ µ, σ,( ) 1.596=
i 0 20 L⋅ 1+..:= intvlsi
i
10L−:=
v rnorm n µ, σ,( ):= the vector of random numbers mean v( ) 1.00111=
h histogram intvls v,( ):= q pnorm h0⟨ ⟩
.05+ µ, σ,( ) pnorm h0⟨ ⟩
.05− µ, σ,( )−:=
First look at the density function
3 2 1 0 1 2 30
1
2
dnorm h0⟨ ⟩ µ, σ,( )
h0⟨ ⟩
Then look at histogram and randomly generated numbers
3 2 1 0 1 2 30
0.06
0.12
0.18
0.24
0.3
h1⟨ ⟩
n
q
h0⟨ ⟩
h0⟨ ⟩,
Page 11.913.5
Figure 9 Figure 10
Not all distribution functions of interest are Mathcad built-in functions. The Rayleigh density
function1 is one example:
b
x
xeb
xf X
22)(
−= bx
exFX
2
1)(−−= for x>0, b>0
Making use of the fact that1 solving the equation xyFY =)( for y as a function of x, where
)(yFY is the cumulative distribution function for the random variable Y, allows one to generate
random numbers y from uniformly distributed numbers x, 0 < x < 1. The Mathcad worksheet
shown as Figure 11 illustrates this example and compares the results to theory.
The gaussian, or normal, probability distribution
n 50000:= the number of random numbers to generate
L 3:= the absolute value of the maximum argument
µ .5:= the mean value σ 1:= the standard deviation
note the pdf may be greater than 1 dnorm µ µ, σ,( ) 0.399=
i 0 20 L⋅ 1+..:= intvlsi
i
10L−:=
v rnorm n µ, σ,( ):= the vector of random numbers mean v( ) 0.49563=
h histogram intvls v,( ):= q pnorm h0⟨ ⟩
.05+ µ, σ,( ) pnorm h0⟨ ⟩
.05− µ, σ,( )−:=
First look at the density function
3 2 1 0 1 2 30
0.2
0.4
dnorm h0⟨ ⟩ µ, σ,( )
h0⟨ ⟩
Then look at histogram and randomly generated numbers
3 2 1 0 1 2 31 .10 4
0.02
0.04
0.06
0.08
0.1
h1⟨ ⟩
n
q
h0⟨ ⟩
h0⟨ ⟩,
The gaussian, or normal, probability distribution
n 50:= the number of random numbers to generate
L 3:= the absolute value of the maximum argument
µ .5:= the mean value σ 1:= the standard deviation
note the pdf may be greater than 1 dnorm µ µ, σ,( ) 0.399=
i 0 20 L⋅ 1+..:= intvlsi
i
10L−:=
v rnorm n µ, σ,( ):= the vector of random numbers mean v( ) 0.58955=
h histogram intvls v,( ):= q pnorm h0⟨ ⟩
.05+ µ, σ,( ) pnorm h0⟨ ⟩
.05− µ, σ,( )−:=
First look at the density function
3 2 1 0 1 2 30
0.2
0.4
dnorm h0⟨ ⟩ µ, σ,( )
h0⟨ ⟩
Then look at histogram and randomly generated numbers
3 2 1 0 1 2 30
0.02
0.04
0.06
0.08
0.1
h1⟨ ⟩
n
q
h0⟨ ⟩
h0⟨ ⟩,
Page 11.913.6
Generating random numbers with Rayleigh distribution
b 2= the Rayleigh parameter n 100000= number of numbers to generate
FX x( ) 1 e
x2−
b−= is the CDF
x runif n 0, 1,( )= array of n uniformly distributed numbers between 0 and 1
y b− ln 1 x−( )⋅→
= array of n Rayleigh distributed numbers
L 5= maximum argument for plot i 0 10 L⋅..= intvlsi
i
10.05+=
h histogram intvls y,( )= mean y( ) 1.254=
Theoretical results are as below:
µπ b⋅
4= µ 1.253=
q e
h0⟨ ⟩
.05−( )2−
be
h0⟨ ⟩
.05+( )2−
b−=
0 1 2 3 4 50
0.05
h1⟨ ⟩
n
q
h0⟨ ⟩
Figure 11
As in the earlier examples, changes made in the Rayleigh parameter b and/or the length n of the
data vector are reflected in the corresponding plot.
Pairs of Random Variables
The joint probability function is used in computing probabilities and expected values of
functions. Calculations involve the evaluation of double integrals. Figure 12 below shows a
very simple example of a joint probability function and multiple integral evaluation.
Page 11.913.7
Figure 12
A commonly discussed1,2 joint probability density function is that of bivariate, or jointly
Gaussian, random variables. Examining probability density function plots with different
correlation coefficient values provides some indication its role. Figure 13 shows a Mathcad
worksheet with nearly uncorrelated variables, while Figure 14 shows the same worksheet with
strongly correlated variables. In each of the two cases, the means and standard deviations are the
same.
Joint PDF example of multiple integral evaluation
C
0
1
y
0
1
x1 x y⋅−( )⌠⌡
d⌠⌡
d:= C3
4= required normalization
fXY x y,( )1
C1 x y⋅−( )⋅:= for 0 x≤ 1≤ and 0 y≤ 1≤ , 0 otherwise
check:
0
1
y
0
1
xfXY x y,( )⌠⌡
d⌠⌡
d 1=
0
1
x
0
1 x−
yfXY x y,( )⌠⌡
d⌠⌡
d11
18= is P(x+y < 1)
0
1
x
1 x−
1
yfXY x y,( )⌠⌡
d⌠⌡
d7
18= is P(x+y > 1)
0
1
x
x
1
yfXY x y,( )⌠⌡
d⌠⌡
d1
2= is P(y > x)
0
1
x
0
x
yfXY x y,( )⌠⌡
d⌠⌡
d1
2= is P(y < x)
0
1
y
0
1
xx 1 x y⋅−( )⋅⌠⌡
d⌠⌡
d1
3= is the mean value of x
0
1
y
0
1
xx y2
⋅ 1 x y⋅−( )⋅⌠⌡
d⌠⌡
d1
12= is the mean value of xy2
Page 11.913.8
Figure 13 Figure 14
White Noise Simulation and Autocorrelation Sequence
A final example uses a sequence of random numbers with Gaussian distribution as simulated
white noise and computes the autocorrelation sequence. Theoretically an infinite sequence of
such numbers would have an autocorrelation sequence with a single non-zero value at zero offset
equal to the variance of the Gaussian distribution. This value indicates the power per unit
frequency in the white noise. A finite sequence used to simulate a white noise sequence will
have an autocorrelation sequence exhibiting the effects of the finite (rectangular-windowed)
sequence used. Figures 15 and 16 on the following pages show the Mathcad worksheet. Figure
15 uses a sequence of 10,000 random numbers as simulated noise. The effect of the finite
sequence length is barely noticeable at the vertical scale of the autocorrelation plot. Figure 16,
on the other hand, uses a sequence of only 100 random numbers, and the effects of the finite
sequence length are clearly visible on the corresponding autocorrelation plot. In-class
experimentation with different sequence lengths provides an indication of the relationship
between sequence length and its effect on the autocorrelation sequence.
Jointly Gaussian Random Variable PDF
mx 0:= my 1:= mean values
σx 1:= σy 1.5:= standard deviations
ρ .1:= correlation coefficient
i 50− 49−, 50..:= j 50− 49−, 50..:=
fXYi j,
1
2 π⋅ σx⋅ σy⋅ 1 ρ2
−⋅
e
1−
2 1 ρ2−( )⋅
i
10mx−
2
σx2
2 ρ⋅i
10mx−
⋅j
10my−
⋅
σx σy⋅−
j
10my−
2
σy2
+
⋅
⋅:=
fXY
Jointly Gaussian Random Variable PDF
mx 0:= my 1:= mean values
σx 1:= σy 1.5:= standard deviations
ρ .9:= correlation coefficient
i 50− 49−, 50..:= j 50− 49−, 50..:=
fXYi j,
1
2 π⋅ σx⋅ σy⋅ 1 ρ2
−⋅
e
1−
2 1 ρ2−( )⋅
i
10mx−
2
σx2
2 ρ⋅i
10mx−
⋅j
10my−
⋅
σx σy⋅−
j
10my−
2
σy2
+
⋅
⋅:=
fXY
Page 11.913.9
White Noise Simulation and its Autocorrelation Sequence
First generate a sequence of zero-mean Gaussian random variables
n 10000:= is the length of the sequence x rnorm n 0, 5,( ):= length x( ) 1 104
×=
Plot the sequence as bars m 0 length x( )..:=
0 2000 4000 6000 8000 1 .10420
0
20
xm
m
Compute and plot the autocorrelation sequence. Note the zero-offset value is the first element
in the array, next are the positive offsets, followed by the negative offsets. The sequence must
be rearranged fo a plot to show zero-offset value in the center, as is customary.
rcorrel x x,( )
length x( ):= i 0 length r( ) 1−..:= r
024.83= length r( ) 2 10
4×=
0 5000 1 .104 1.5 .10420
0
20
40
ri
iFinally plot the autocorrelation sequence in customary form after rearranging it.
foldlength r( ) 1−
2:= j 0 fold 1−..:= k fold length r( ) 1−..:=
dk
rk fold−:= d
jrj fold+ 1+:=
0 5000 1 .104 1.5 .10420
0
20
40
di
i
Figure 15
Page 11.913.10
White Noise Simulation and its Autocorrelation Sequence
First generate a sequence of zero-mean Gaussian random variables
n 100:= is the length of the sequence x rnorm n 0, 5,( ):= length x( ) 100=
Plot the sequence as bars m 0 length x( )..:=
0 20 40 60 80 10020
0
20
xm
m
Compute and plot the autocorrelation sequence. Note the zero-offset value is the first element
in the array, next are the positive offsets, followed by the negative offsets. The sequence must
be rearranged fo a plot to show zero-offset value in the center, as is customary.
rcorrel x x,( )
length x( ):= i 0 length r( ) 1−..:= r
030.769= length r( ) 199=
0 50 100 15050
0
50
ri
iFinally plot the autocorrelation sequence in customary form after rearranging it.
foldlength r( ) 1−
2:= j 0 fold 1−..:= k fold length r( ) 1−..:=
dk
rk fold−:= d
jrj fold+ 1+:=
0 50 100 15020
0
20
40
di
i
Figure 16
Page 11.913.11
Conclusion
The examples presented here illustrate several ways in which important concepts in probability
and random processes can be demonstrated in the classroom. The effects of changes in model
parameters are easy to investigate due to the “live” calculations of Mathcad worksheets. The
graphical user interface with its standard mathematical symbols provides an additional
advantage.
Students unanimously stated in informal evaluations that they found the additional Mathcad
examples helpful and thought their use should be continued.
Mathcad 13 examples used in this paper are available at http://csserver.evansville.edu/~reising .
Bibliography
1. Peebles, Peyton Z., Jr. Probability, Random Variables and Random Signal Principles 4th Edition.
McGraw-Hill, Inc., 2001.
2. Yates, Roy, and David Goodman. Probability and Stochastic Processes A Friendly Introduction for
Electrical and Computer Engineers 2nd Edition. John Wiley & Sons, Inc., 2005.
3. Papoulis, Athanasios, and S. Unnikrishna Pillai. Probability, Random Variables and Stochastic Processes
Fourth Edition. Boston, MA: McGraw-Hill, 2002.
4. Stark, Henry, and John W. Woods. Probability and Random Processes with Applications to Signal
Processing 3rd Edition. Upper Saddle River, NJ: Prentice-Hall, 2002.
5. Viniotis, Yannis. Probability and Random Processes for Electrical Engineers. Boston, MA:
WCB/McGraw-Hill, 1998.
6. Leon-Garcia, Alberto. Probability and Random Processes for Electrical Engineering 2nd Edition. Reading,
MA: Addison-Wesley Longman, 1994.
Page 11.913.12