# b probability and random processes control-ii-2017/probability... · probability and random...

Post on 30-Aug-2018

215 views

Embed Size (px)

TRANSCRIPT

B

PROBABILITY ANDRANDOM PROCESSES

WILEY SURVIVAL GUIDES IN ENGINEERINGAND SCIENCE

Emmanuel Desurvire, Editor

Wiley Survival Guide in Global Telecommunications: Signaling Principles,

Network Protocols, and Wireless Systems Emmanuel Desurvire

Wiley Survival Guide in Global Telecommunications: Broadband Access,

Optical Components and Networks, and CryptographyEmmanuel Desurvire

Fiber to the Home: The New Empowerment Paul E. Green, Jr.

Probability and Random Processes Venkatarama Krishnan

BPROBABILITY ANDRANDOM PROCESSES

Venkatarama KrishnanProfessor Emeritus of Electrical Engineering

University of Massachusetts Lowell

A John Wiley & Sons, Inc., Publication

Copyright # 2006 by John Wiley & Sons, Inc. All rights reserved

Published by John Wiley & Sons, Inc., Hoboken, New Jersey

Published simultaneously in Canada

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form

or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as

permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior

written permission of the Publisher, or authorization through payment of the appropriate per-copy

fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400,

fax (978) 750-4470, or on the web at www.copyright.com. Requests to the Publisher for permission

should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken,

NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley.com/go/permission.

Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts

in preparing this book, they make no representations or warranties with respect to the accuracy or

completeness of the contents of this book and specifically disclaim any implied warranties of

merchantability or fitness for a particular purpose. No warranty may be created or extended by sales

representatives or written sales materials. The advice and strategies contained herein may not be

suitable for your situation. You should consult with a professional where appropriate. Neither the

publisher nor author shall be liable for any loss of profit or any other commercial damages, including

but not limited to special, incidental, consequential, or other damages.

For general information on our other products and services or for technical support, please contact our Customer

Care Department within the United States at (800) 762-2974, outside the United States at (317) 572-3993

or fax (317) 572-4002.

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print

may not be available in electronic formats. For more information about Wiley products, visit our web site at

www.wiley.com.

Library of Congress Cataloging-in-Publication Data:

Krishnan, Venkatarama,

Probability and random processes / by Venkatarama Krishnan.p. cm. (Wiley survival guides in engineering and science)

Includes bibliographical references and index.

ISBN-13: 978-0-471-70354-9 (acid-free paper)

ISBN-10: 0-471-70354-0 (acid-free paper)

1. Probabilities. 2. Stochastic processes. 3. EngineeringStatistical methods.

4. ScienceStatistical methods. I. Title. II. Series

QA273.K74 2006

519.2dc22 2005057726

Printed in the United States of America

10 9 8 7 6 5 4 3 2 1

Contents

Preface, xi

CHAPTER 1Sets, Fields, and Events, 1

1.1 Set Definitions, 1

1.2 Set Operations, 3

1.3 Set Algebras, Fields, and Events, 8

CHAPTER 2Probability Space and Axioms, 10

2.1 Probability Space, 10

2.2 Conditional Probability, 14

2.3 Independence, 18

2.4 Total Probability and Bayes

Theorem, 20

CHAPTER 3Basic Combinatorics, 25

3.1 Basic Counting Principles, 25

3.2 Permutations, 26

3.3 Combinations, 28

CHAPTER 4Discrete Distributions, 37

4.1 Bernoulli Trials, 37

4.2 Binomial Distribution, 38

4.3 Multinomial Distribution, 41

4.4 Geometric Distribution, 42

4.5 Negative Binomial Distribution, 44

4.6 Hypergeometric Distribution, 46

4.7 Poisson Distribution, 48

4.8 Logarithmic Distribution, 55

4.9 Summary of Discrete Distributions, 62

CHAPTER 5Random Variables, 64

5.1 Definition of Random Variables, 64

5.2 Determination of Distribution and

Density Functions, 66

5.3 Properties of Distribution and Density

Functions, 73

5.4 Distribution Functions from Density

Functions, 75

CHAPTER 6Continuous Random Variables and BasicDistributions, 79

6.1 Introduction, 79

6.2 Uniform Distribution, 79

6.3 Exponential Distribution, 80

6.4 Normal or Gaussian Distribution, 84

CHAPTER 7Other Continuous Distributions, 95

7.1 Introduction, 95

7.2 Triangular Distribution, 95

7.3 Laplace Distribution, 96

7.4 Erlang Distribution, 97

vii

7.5 Gamma Distribution, 99

7.6 Weibull Distribution, 101

7.7 Chi-Square Distribution, 102

7.8 Chi and Other Allied Distributions,

104

7.9 Student-t Density, 110

7.10 Snedecor F Distribution, 111

7.11 Lognormal Distribution, 112

7.12 Beta Distribution, 114

7.13 Cauchy Distribution, 115

7.14 Pareto Distribution, 117

7.15 Gibbs Distribution, 118

7.16 Mixed Distributions, 118

7.17 Summary of Distributions of

Continuous Random Variables, 119

CHAPTER 8Conditional Densities and Distributions, 122

8.1 Conditional Distribution and Density

for P(A)= 0, 122

8.2 Conditional Distribution and Density

for P(A) 0, 1268.3 Total Probability and Bayes Theorem

for Densities, 131

CHAPTER 9Joint Densities and Distributions, 135

9.1 Joint Discrete Distribution

Functions, 135

9.2 Joint Continuous Distribution

Functions, 136

9.3 Bivariate Gaussian Distributions, 144

CHAPTER 10Moments and Conditional Moments, 146

10.1 Expectations, 146

10.2 Variance, 149

10.3 Means and Variances of Some

Distributions, 150

10.4 Higher-Order Moments, 153

10.5 Bivariate Gaussian, 154

CHAPTER 11Characteristic Functions and GeneratingFunctions, 155

11.1 Characteristic Functions, 155

11.2 Examples of Characteristic

Functions, 157

11.3 Generating Functions, 161

11.4 Examples of Generating

Functions, 162

11.5 Moment Generating Functions, 164

11.6 Cumulant Generating Functions, 167

11.7 Table of Means and Variances, 170

CHAPTER 12Functions of a Single Random Variable, 173

12.1 Random Variable g(X ), 173

12.2 Distribution of Y g(X ), 17412.3 Direct Determination of Density

fY ( y) from fX(x), 194

12.4 Inverse Problem: Finding g(x) Given

fX(x) and fY ( y), 200

12.5 Moments of a Function of

a Random Variable, 202

CHAPTER 13Functions of Multiple RandomVariables, 206

13.1 Function of Two Random Variables,

Z g(X,Y ), 20613.2 Two Functions of Two Random

Variables, Z g(X,Y ),W h(X,Y ), 222

13.3 Direct Determination of Joint Density

fZW(z,w ) from fXY(x,y ), 227

13.4 Solving Z g(X,Y ) Using an AuxiliaryRandom Variable, 233

13.5 Multiple Functions of

Random Variables, 238

viii Contents

CHAPTER 14Inequalities, Convergences, and LimitTheorems, 241

14.1 Degenerate Random Variables, 241

14.2 Chebyshev and Allied

Inequalities, 242

14.3 Markov Inequality, 246

14.4 Chernoff Bound, 248

14.5 CauchySchwartz Inequality, 251

14.6 Jensens Inequality, 254

14.7 Convergence Concepts, 256

14.8 Limit Theorems, 259

CHAPTER 15Computer Methods for GeneratingRandom Variates, 264

15.1 Uniform-Distribution Random

Variates, 264

15.2 Histograms, 266

15.3 Inverse Transformation

Techniques, 269

15.4 Convolution Techniques, 279

15.5 AcceptanceRejection

Techniques, 280

CHAPTER 16Elements of Matrix Algebra, 284

16.1 Basic Theory of Matrices, 284

16.2 Eigenvalues and

Eigenvectors of Matrices, 293

16.3 Vectors and Matrix

Differentiations, 301

16.4 Block Matrices, 308

CHAPTER 17Random Vectors andMean-Square Estimation, 311

17.1 Distributions and Densities, 311

17.2 Moments of Random Vectors, 319

17.3 Vector Gaussian Random

Variables, 323

17.4 Diagonalization of

Covariance Matrices, 330

17.5 Simultaneous Diagonalization of

Covariance Matrices, 334

17.6 Linear Estimation of Vector

Variables, 337

CHAPTER 18Estimation Theory, 340

18.1 Criteria of Estimators, 340

18.2 Estimation of Random

Variables, 342

18.3 Estimation of Parameters (Point

Estimation), 350

18.4 Interval Estimation (Confidence

Intervals), 364

18.5 Hypothesis Testing (Binary), 373

18.6 Bayesian Estimation, 384

CHAPTER 19Random Processes, 406

19.1 Basic Definitions, 406

19.2 Stationary Random Processes, 420

19.3 Ergodic Processes, 439

19.4 Estimation of Parameters of

Random Processes, 445

19.5 Power Spectral Density, 472

CHAPTER 20Classification of Random Processes, 490

20.1 Specifications of Random

Processes, 490

20.2 Poisson Process, 492

20.3 Binomial Process, 505

20.4 Independent Increment Process, 507

20.5 Random-Walk Proce

Recommended