Random Signals and Processes Primerwith MATLAB

Gordana Jovanovic Dolecek

Random Signalsand Processes Primerwith MATLAB

Gordana Jovanovic DolecekDepartment of ElectronicsInstituto Nacional de AstrofisicaOptica y Electronica (INAOE)

Tonantzintla, Puebla, Mexico

Additional material to this book can be downloaded from http://extra.springer.com

ISBN 978-1-4614-2385-0 ISBN 978-1-4614-2386-7 (eBook)DOI 10.1007/978-1-4614-2386-7Springer New York Heidelberg Dordrecht London

Library of Congress Control Number: 2012935100

# Springer Science+Business Media New York 2013This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part ofthe material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission orinformation storage and retrieval, electronic adaptation, computer software, or by similar or dissimilarmethodology now known or hereafter developed. Exempted from this legal reservation are brief excerptsin connection with reviews or scholarly analysis or material supplied specifically for the purpose of beingentered and executed on a computer system, for exclusive use by the purchaser of the work. Duplicationof this publication or parts thereof is permitted only under the provisions of the Copyright Law of thePublisher’s location, in its current version, and permission for use must always be obtained fromSpringer. Permissions for use may be obtained through RightsLink at the Copyright Clearance Center.Violations are liable to prosecution under the respective Copyright Law.The use of general descriptive names, registered names, trademarks, service marks, etc. in thispublication does not imply, even in the absence of a specific statement, that such names are exemptfrom the relevant protective laws and regulations and therefore free for general use.While the advice and information in this book are believed to be true and accurate at the date ofpublication, neither the authors nor the editors nor the publisher can accept any legal responsibility forany errors or omissions that may be made. The publisher makes no warranty, express or implied, withrespect to the material contained herein.

Printed on acid-free paper

Springer is part of Springer Science+Business Media (www.springer.com)

In memory of my parents, Vojo and Smilja.

Preface

The concept of randomness is important in almost all aspects of modern engineering

systems and sciences. This includes computer science, biology, medicine, the social

sciences, and management sciences, among others.

Whenever we are unable to know the exact future behavior of some phenomena,

we say that it is random. As such, random phenomena can only be approximated.

Because of this, work with random phenomena calls for an understanding of

probability concepts as well as the basic concepts of the theory of random signals

and processes.

Despite the number of books written on this topic, there is no book which

introduces the concept of random variables and processes in a simplified way.

The majority of books written on this topic are built on a strong mathematical

background which sometimes overwhelms the reader with mathematical definitions

and formulas, rather than focusing on a full understanding of the concepts of

random variables and related terms. In this way, sometimes the reader learns the

mathematical definitions without a full understanding of the meaning of the under-

lying concepts.

Additionally, many books which claim to be introductory are very large. This is

not encouraging for readers who really desire to understand the concept first.

Taking into account that the concept of random signals and processes is also

important in many fields without the strong mathematical background, one of our

main purposes in writing this book is to explain the basic random concepts in a

more reader-friendly way; thus making them accessible to a broader group of

readers.

As the title indicates, this text is intended for theses who are studying random

signals for the first time.

It can be used for self-study and also in courses of varying lengths, levels, and

emphases. The only prerequisite is knowledge of elementary calculus.

vii

Motivating students is a major challenge in introductory texts on random signals

and processes. In order to achieve student motivation, this book has features that

distinguish it from others in this area which we hope will attract prospective

readers:

l Besides the mathematical definitions, simple explanations of different terms like

density, distribution, mean value, variance, random processes, autocorrelation

function, ergodicity, etc., are provided.l In an effort to facilitate understanding, a lot of examples which illustrate

concepts are followed by the numerical exercises which are already solved.l It is widely accepted that the use of visuals reinforces better understanding.

As such, MATLAB exercises are used extensively in the text, starting in Chap. 2,

to further explain and to give visual and intuitive representations of different

concepts.l We chose MATLAB because MATLAB, along with its accompanying tool-

boxes, is the tool of choice for most educational and research applications. In

order for the reader to be able to focus on understanding the subject matter and

not on programming, all exercises and MATLAB codes are provided.l Pedagogy recognizes the need for more active learning in improving the quality

of learning. Making a reader active rather than passive can enhance under-

standing. In pursuit of this goal, each chapter provides several demo programs

as a complement to the textbook. These programs allow the reader to become an

active participant in the learning process. The readers can run the programs by

themselves as many times as they want, choosing different parameters.l Also, in order to encourage students to think about the concepts presented, and to

motivate them to undertake further reading, we present different questions at the

end of each chapter. The answers to these are also provided so that the reader can

check their understanding immediately.

The text is divided into seven chapters.

The first chapter is a brief introduction to the basic concepts of random experi-

ments, sample space, and probability, illustrated with different examples.

The next chapter explains what a random variable is and how it is described by

probability distribution and density function while considering discrete, continuous,

and mixed variables. However, there are many applications—when a partial

description of random variable is either needed or possible—in which we need

some parameters for the characterization of the random variable. The most impor-

tant parameters, mean value and variance, are explained in detail and are illustrated

with numerous examples.

We also show how to find the characteristics of the random variable after its

transformation, if the characteristics of the random variable before transformation

are known. Some other important characteristics are also described, such as charac-

teristic function and moment generating function.

viii Preface

The concept of a multidimensional random variable is examined in Chap. 3

considering a two-dimensional random variable, which is described by the joint

distribution and density. Expected values and moments are also introduced and

discussed. The relations between random variables such as dependence and corre-

lation are also explained. This chapter also discusses some other important topics,

such as transformation and the characteristic function.

The most important random variable the Gaussian, or normal random variable is

described in detail in Chap. 4 and the useful properties which make it so important

in different applications are also explained.

Chapter 5 presents some important random variables and describes their useful

properties. Some of these include lognormal, Rayleigh, Rician, and exponential

random variables. We also discuss variables related to exponential variables such as

Laplacian, Gamma, Erlang’s, and Weibull random variables.

Special attention is also paid to some important discrete random variables such

as Bernoulli, binomial, Poisson, and geometric random variables.

The concept of a random process is introduced and explained in Chap. 6. A

description of random process is given and its important characteristics, such as

stationarity and ergodicity, are explained. Special attention is also given to an

autocorrelation function, and its importance and properties are explained.

Chapter 7 examines the spectral description of a random process based on the

Fourier transform. Difficulties in the application of the Fourier transform to random

processes and differences in applying the Fourier transform of deterministic signals

are elaborated. The power spectral density is explained along with its important

properties. Some important operations of processes like sum, multiplication with a

sinusoidal signal, and LTI filtering are described through the lens of the spectral

description of the process.

Puebla, Mexico Gordana Jovanovic Dolecek

Preface ix

Acknowledgements

The author would like to offer thanks for the professional assistance, technical help,

and encouragement that she received from the Institute for Astrophysics, Optics,

and Electronics (INAOE), Puebla, Mexico. Without this support it would not have

been possible to complete this project.

She would also thank her colleague Professor Massimiliano Laddomada for his

useful comments for improving the text.

xi

Contents

1 Introduction to Sample Space and Probability . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.1 Sample Space and Events. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.1.1 Operations with Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.2 Probability of Events. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.2.1 Axioms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.2.2 Properties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.3 Equally Likely Outcomes in the Sample Space. . . . . . . . . . . . . . . . . . . . . 8

1.4 Relative Frequency Definition of Probability . . . . . . . . . . . . . . . . . . . . . . . 9

1.5 Conditional Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

1.6 Total Probability and Bayes’ Rule. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

1.6.1 Total Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

1.6.2 Bayes’ Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

1.7 Independent Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

1.8 Numerical Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

1.9 Questions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

1.10 Answers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

2 Random Variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

2.1 What Is a Random Variable? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

2.1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

2.1.2 Definition of a Random Variable . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

2.2 Distribution Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

2.2.1 Definition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

2.2.2 Properties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

2.3 Probability Density Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

2.3.1 Definition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

2.3.2 Delta Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

xiii

2.3.3 Densities of Discrete and Mixed Random Variables . . . . . . 46

2.3.4 Properties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

2.3.5 Examples of Density Functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

2.4 How to Estimate Density and Distribution in MATLAB . . . . . . . . . 52

2.4.1 Density Function (MATLAB File: PDF.m) . . . . . . . . . . . . . . . 52

2.4.2 Distribution Function

(MATLAB File: Distribution.m) . . . . . . . . . . . . . . . . . . . . . . . . . . 57

2.5 Conditional Distribution and PDF. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

2.5.1 Definition of Conditional Distribution and PDF . . . . . . . . . . 57

2.5.2 Definition of a Conditioned Event . . . . . . . . . . . . . . . . . . . . . . . . . 58

2.6 Transformation of a Random Variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

2.6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

2.6.2 Monotone Transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

2.6.3 Nonmonotone Transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

2.6.4 Transformation of Discrete Random Variables . . . . . . . . . . . 72

2.7 Mean Value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

2.7.1 What Is a Mean Value? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

2.7.2 Concept of a Mean Value of a Discrete

Random Variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

2.7.3 Mean Value of a Continuous Random Variable. . . . . . . . . . . 84

2.7.4 General Expression of Mean Value for Discrete,

Continuous, and Mixed Random Variables . . . . . . . . . . . . . . . 91

2.7.5 Conditional Mean Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

2.8 Moments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

2.8.1 Moments Around the Origin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

2.8.2 Central Moments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

2.8.3 Moments and PDF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104

2.8.4 Functions Which Give Moments . . . . . . . . . . . . . . . . . . . . . . . . . . 110

2.8.5 Chebyshev Inequality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118

2.9 Numerical Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

2.10 MATLAB Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142

2.11 Questions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150

2.12 Answers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151

3 Multidimensional Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155

3.1 What Is a Multidimensional Random Variable? . . . . . . . . . . . . . . . . . . 155

3.1.1 Two-Dimensional Random Variable . . . . . . . . . . . . . . . . . . . . . . 155

3.2 Joint Distribution and Density. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

3.2.1 Joint Distribution. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

3.2.2 Joint Density Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162

3.2.3 Conditional Distribution and Density. . . . . . . . . . . . . . . . . . . . . . 165

3.2.4 Independent Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . 168

xiv Contents

3.3 Expected Values and Moments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169

3.3.1 Expected Value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169

3.3.2 Joint Moments Around the Origin . . . . . . . . . . . . . . . . . . . . . . . . . . 174

3.3.3 Joint Central Moments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175

3.3.4 Independence and Correlation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178

3.4 Transformation of Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

3.4.1 One-to-One Transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

3.4.2 Nonunique Transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187

3.4.3 Generalization for N Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188

3.5 Characteristic Functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

3.5.1 Definition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

3.5.2 Characteristic Function of the Sum

of the Independent Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190

3.5.3 Moment Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192

3.5.4 PDF of the Sum of Independent

Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192

3.6 Numerical Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194

3.7 MATLAB Exercises. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212

3.8 Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218

3.9 Answers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219

4 Normal Random Variable. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227

4.1 Normal PDF. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227

4.1.1 Definition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227

4.1.2 Properties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230

4.2 Normal Distribution. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233

4.2.1 Definition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233

4.2.2 Practical Calculation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236

4.2.3 The “3 s Rule”. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242

4.3 Transformation of Normal Random Variable . . . . . . . . . . . . . . . . . . . . . . 243

4.3.1 Monotone Transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243

4.3.2 Nonmonotone Transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245

4.4 How to Generate a Normal Variable in MATLAB?. . . . . . . . . . . . . . . . 249

4.5 Sum of Independent Normal Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252

4.5.1 Characteristic Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252

4.5.2 Characteristic Function of the

Sum of Independent Variables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254

4.5.3 Sum of Linear Transformations of Independent

Normal Random Variables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256

4.5.4 Central Limit Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258

4.6 Jointly Normal Variables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259

4.6.1 Two Jointly Normal Variables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259

4.6.2 N-Jointly Normal Random Variables . . . . . . . . . . . . . . . . . . . . . . . 261

Contents xv

4.7 Summary of Properties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263

4.8 Numerical Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264

4.9 MATLAB Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277

4.10 Questions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289

4.11 Answers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291

5 Other Important Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297

5.1 Lognormal Random Variable. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297

5.1.1 Density Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297

5.1.2 Distribution Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298

5.1.3 Moments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300

5.1.4 What Does a Lognormal Variable Tell Us? . . . . . . . . . . . . . . . 301

5.2 Rayleigh Random Variable. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302

5.2.1 Density Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302

5.2.2 Distribution Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303

5.2.3 Moments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303

5.2.4 Relation of Rayleigh and Normal Variables. . . . . . . . . . . . . . . 305

5.3 Rician Random Variable. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307

5.3.1 Relation of Rician, Rayleigh, and Normal Variables . . . . . 307

5.4 Exponential Random Variable. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309

5.4.1 Density and Distribution Function. . . . . . . . . . . . . . . . . . . . . . . . . 309

5.4.2 Characteristic Function and Moments . . . . . . . . . . . . . . . . . . . . . 311

5.4.3 Memory-Less Property. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312

5.5 Variables Related with Exponential Variable . . . . . . . . . . . . . . . . . . . . . 312

5.5.1 Laplacian Random Variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312

5.5.2 Gamma and Erlang’s Random Variables . . . . . . . . . . . . . . . . . . 315

5.5.3 Weibull Random Variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318

5.6 Bernoulli and Binomial Random Variables . . . . . . . . . . . . . . . . . . . . . . . 319

5.6.1 Bernoulli Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319

5.6.2 What Is a Binomial Random Variable? . . . . . . . . . . . . . . . . . . . 320

5.6.3 Binomial Distribution and Density . . . . . . . . . . . . . . . . . . . . . . . . 321

5.6.4 Characteristic Functions and Moments . . . . . . . . . . . . . . . . . . . . 321

5.6.5 Approximation of Binomial Variable. . . . . . . . . . . . . . . . . . . . . . 325

5.7 Poisson Random Variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327

5.7.1 Approximation of Binomial Variable. . . . . . . . . . . . . . . . . . . . . . 327

5.7.2 Poisson Variable as a Counting Random Variable . . . . . . . . 328

5.7.3 Distribution and Density Functions. . . . . . . . . . . . . . . . . . . . . . . . 329

5.7.4 Characteristic Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 330

5.7.5 Sum of Independent Poisson Variables. . . . . . . . . . . . . . . . . . . . 331

5.7.6 Poisson Flow of Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332

5.8 Geometric Random Variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335

5.8.1 What Is a Geometric Random Variable

and Where Does This Name Come From? . . . . . . . . . . . . . . . . 335

5.8.2 Probability Distribution and Density Functions . . . . . . . . . . . 335

xvi Contents

5.8.3 Characteristic Functions and Moments . . . . . . . . . . . . . . . . . . . 336

5.8.4 The Memory-Less Property of Geometric

Random Variable. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 338

5.9 Numerical Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339

5.10 MATLAB Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350

5.11 Questions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367

5.12 Answers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368

6 Random Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 371

6.1 What Is a Random Process? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 371

6.1.1 Deterministic and Nondeterministic

Random Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372

6.1.2 Continuous and Discrete Random Processes . . . . . . . . . . . . . 372

6.2 Statistics of Random Processes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373

6.2.1 Description of a Process in One Point . . . . . . . . . . . . . . . . . . . . 373

6.2.2 Description of a Process in Two Points . . . . . . . . . . . . . . . . . . 375

6.2.3 Description of Process in n Points . . . . . . . . . . . . . . . . . . . . . . . . 376

6.3 Stationary Random Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377

6.4 Mean Value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379

6.5 Autocorrelation Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381

6.5.1 Definition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381

6.5.2 WS Stationary Processes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382

6.5.3 What Does Autocorrelation Function Tell Us

and Why Do We Need It? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383

6.5.4 Properties of Autocorrelation Function

for WS Stationary Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385

6.5.5 Autocovariance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392

6.6 Cross-Correlation Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393

6.6.1 Definition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393

6.6.2 Jointly WS Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394

6.6.3 Properties of Cross-Correlation Function

for Jointly WS Stationary Processes . . . . . . . . . . . . . . . . . . . . . . 396

6.6.4 Cross-Covariance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399

6.7 Ergodic Processes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 400

6.7.1 Time Averaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 400

6.7.2 What Is Ergodicity? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 400

6.7.3 Explanation of Ergodicity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401

6.7.4 Mean Ergodic Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404

6.7.5 Autocorrelation and Cross-Correlation

Ergodic Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405

6.8 Numerical Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407

6.9 MATLAB Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 418

6.10 Questions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444

6.11 Answers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444

Contents xvii

7 Spectral Characteristics of Random Processes . . . . . . . . . . . . . . . . . . . . . . . . 447

7.1 Power Spectral Density . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 447

7.1.1 Fourier Transformation of Deterministic Signals . . . . . . . . . . 447

7.1.2 How to Apply the Fourier Transformation

to Random Signals? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 448

7.1.3 PSD and Autocorrelation Function . . . . . . . . . . . . . . . . . . . . . . . . . 451

7.1.4 Properties of PSD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454

7.1.5 Interpretation of PSD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 457

7.2 Classification of Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 459

7.2.1 Low-Pass, Band-Pass, Band-Limited,

and Narrow-Band Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 459

7.2.2 White and Colored Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 459

7.2.3 Nonzero Mean Process. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 462

7.2.4 Random Sinusoidal Process. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 463

7.2.5 Bandwidth of Random Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 464

7.3 Cross-Power Spectrum. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 468

7.3.1 Definition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 468

7.3.2 Properties of Cross-Spectral Density Function . . . . . . . . . . . . . 469

7.4 Operation of Processes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 472

7.4.1 Sum of Random Processes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 472

7.4.2 Multiplication of Random Process

with a Sinusoidal Signal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 474

7.4.3 Filtering of Random Process. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 476

7.5 Numerical Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 479

7.6 MATLAB Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 494

7.7 Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 505

7.8 Answers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 506

Appendix A: Definite Integrals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 509

Appendix B: Indefinite Integrals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 511

Appendix C: Gamma Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 513

Appendix D: Useful Mathematical Quantities . . . . . . . . . . . . . . . . . . . . . . . . . . . . 515

Appendix E: Fourier Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 517

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 519

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 521

xviii Contents