# probability, statistics, and random processes for electrical

Post on 02-Jan-2017

336 views

Embed Size (px)

TRANSCRIPT

Probability, Statistics, and Random Processes

for Electrical EngineeringThird Edition

Alberto Leon-GarciaUniversity of Toronto

Upper Saddle River, NJ 07458

Library of Congress Cataloging-in-Publication Data

Leon-Garcia, Alberto.Probability, statistics, and random processes for electrical engineering / Alberto Leon-Garcia. -- 3rd ed.

p. cm.Includes bibliographical references and index.ISBN-13: 978-0-13-147122-1 (alk. paper)1. Electric engineering--Mathematics. 2. Probabilities. 3. Stochastic processes. I. Leon-Garcia, Alberto. Probabilityand random processes for electrical engineering. II. Title.TK153.L425 2007519.202'46213--dc22

2007046492

Vice President and Editorial Director, ECS: Marcia J. HortonAssociate Editor: Alice DworkinEditorial Assistant: William OpaluchSenior Managing Editor: Scott DisannoProduction Editor: Craig LittleArt Director: Jayen ConteCover Designer: Bruce KenselaarArt Editor: Greg DullesManufacturing Manager: Alan Fischer Manufacturing Buyer: Lisa McDowellMarketing Manager: Tim Galligan

2008 Pearson Education, Inc.Pearson Prentice HallPearson Education, Inc.Upper Saddle River, NJ 07458

All rights reserved. No part of this book may be reproduced, in any form or by any means, without permission inwriting from the publisher.

Pearson Prentice HallTM is a trademark of Pearson Education, Inc. MATLAB is a registered trademark of The MathWorks, Inc. All other product or brand names are trademarks or registered trademarks of their respective holders.

The author and publisher of this book have used their best efforts in preparing this book. These efforts include thedevelopment, research, and testing of the theories and programs to determine their effectiveness. The author andpublisher make no warranty of any kind, expressed or implied, with regard to the material contained in this book. Theauthor and publisher shall not be liable in any event for incidental or consequential damages in connection with, orarising out of, the furnishing, performance, or use of this material.

Printed in the United States of America

10 9 8 7 6 5 4 3 2 1

ISBN 0-13-147122-8978-0-13-147122-1

Pearson Education Ltd., LondonPearson Education Australia Pty. Ltd., SydneyPearson Education Singapore, Pte. Ltd.Pearson Education North Asia Ltd., Hong KongPearson Education Canada, Inc., TorontoPearson Educacin de Mexico, S.A. de C.V.Pearson EducationJapan, TokyoPearson Education Malaysia, Pte. Ltd.Pearson Education, Upper Saddle River, New Jersey

TO KAREN, CARLOS, MARISA, AND MICHAEL.

This page intentionally left blank

v

Contents

Preface ix

CHAPTER 1 Probability Models in Electrical and Computer Engineering 1

1.1 Mathematical Models as Tools in Analysis and Design 21.2 Deterministic Models 41.3 Probability Models 41.4 A Detailed Example: A Packet Voice Transmission System 91.5 Other Examples 111.6 Overview of Book 16

Summary 17Problems 18

CHAPTER 2 Basic Concepts of Probability Theory 21

2.1 Specifying Random Experiments 212.2 The Axioms of Probability 302.3 Computing Probabilities Using Counting Methods 412.4 Conditional Probability 472.5 Independence of Events 532.6 Sequential Experiments 592.7 Synthesizing Randomness: Random Number Generators 672.8 Fine Points: Event Classes 702.9 Fine Points: Probabilities of Sequences of Events 75

Summary 79Problems 80

CHAPTER 3 Discrete Random Variables 96

3.1 The Notion of a Random Variable 963.2 Discrete Random Variables and Probability Mass Function 993.3 Expected Value and Moments of Discrete Random Variable 1043.4 Conditional Probability Mass Function 1113.5 Important Discrete Random Variables 1153.6 Generation of Discrete Random Variables 127

Summary 129Problems 130

***

*

vi Contents

CHAPTER 4 One Random Variable 141

4.1 The Cumulative Distribution Function 1414.2 The Probability Density Function 1484.3 The Expected Value of X 1554.4 Important Continuous Random Variables 1634.5 Functions of a Random Variable 1744.6 The Markov and Chebyshev Inequalities 1814.7 Transform Methods 1844.8 Basic Reliability Calculations 1894.9 Computer Methods for Generating Random Variables 194

4.10 Entropy 202Summary 213Problems 215

CHAPTER 5 Pairs of Random Variables 233

5.1 Two Random Variables 2335.2 Pairs of Discrete Random Variables 2365.3 The Joint cdf of X and Y 2425.4 The Joint pdf of Two Continuous Random Variables 2485.5 Independence of Two Random Variables 2545.6 Joint Moments and Expected Values of a Function of Two Random

Variables 2575.7 Conditional Probability and Conditional Expectation 2615.8 Functions of Two Random Variables 2715.9 Pairs of Jointly Gaussian Random Variables 278

5.10 Generating Independent Gaussian Random Variables 284Summary 286Problems 288

CHAPTER 6 Vector Random Variables 303

6.1 Vector Random Variables 3036.2 Functions of Several Random Variables 3096.3 Expected Values of Vector Random Variables 3186.4 Jointly Gaussian Random Vectors 3256.5 Estimation of Random Variables 3326.6 Generating Correlated Vector Random Variables 342

Summary 346Problems 348

*

Contents vii

CHAPTER 7 Sums of Random Variables and Long-Term Averages 359

7.1 Sums of Random Variables 3607.2 The Sample Mean and the Laws of Large Numbers 365

Weak Law of Large Numbers 367Strong Law of Large Numbers 368

7.3 The Central Limit Theorem 369Central Limit Theorem 370

7.4 Convergence of Sequences of Random Variables 3787.5 Long-Term Arrival Rates and Associated Averages 3877.6 Calculating Distributions Using the Discrete Fourier

Transform 392Summary 400Problems 402

CHAPTER 8 Statistics 411

8.1 Samples and Sampling Distributions 4118.2 Parameter Estimation 4158.3 Maximum Likelihood Estimation 4198.4 Confidence Intervals 4308.5 Hypothesis Testing 4418.6 Bayesian Decision Methods 4558.7 Testing the Fit of a Distribution to Data 462

Summary 469Problems 471

CHAPTER 9 Random Processes 487

9.1 Definition of a Random Process 4889.2 Specifying a Random Process 4919.3 Discrete-Time Processes: Sum Process, Binomial Counting Process,

and Random Walk 4989.4 Poisson and Associated Random Processes 5079.5 Gaussian Random Processes, Wiener Process

and Brownian Motion 5149.6 Stationary Random Processes 5189.7 Continuity, Derivatives, and Integrals of Random Processes 5299.8 Time Averages of Random Processes and Ergodic Theorems 5409.9 Fourier Series and Karhunen-Loeve Expansion 544

9.10 Generating Random Processes 550Summary 554Problems 557

*

**

viii Contents

CHAPTER 10 Analysis and Processing of Random Signals 577

10.1 Power Spectral Density 57710.2 Response of Linear Systems to Random Signals 58710.3 Bandlimited Random Processes 59710.4 Optimum Linear Systems 60510.5 The Kalman Filter 61710.6 Estimating the Power Spectral Density 62210.7 Numerical Techniques for Processing Random Signals 628

Summary 633Problems 635

CHAPTER 11 Markov Chains 647

11.1 Markov Processes 64711.2 Discrete-Time Markov Chains 65011.3 Classes of States, Recurrence Properties, and Limiting

Probabilities 66011.4 Continuous-Time Markov Chains 67311.5 Time-Reversed Markov Chains 68611.6 Numerical Techniques for Markov Chains 692

Summary 700Problems 702

CHAPTER 12 Introduction to Queueing Theory 713

12.1 The Elements of a Queueing System 71412.2 Littles Formula 71512.3 The M/M/1 Queue 71812.4 Multi-Server Systems: M/M/c, M/M/c/c, And 72712.5 Finite-Source Queueing Systems 73412.6 M/G/1 Queueing Systems 73812.7 M/G/1 Analysis Using Embedded Markov Chains 74512.8 Burkes Theorem: Departures From M/M/c Systems 75412.9 Networks of Queues: Jacksons Theorem 758

12.10 Simulation and Data Analysis of Queueing Systems 771Summary 782Problems 784

Appendices

A. Mathematical Tables 797B. Tables of Fourier Transforms 800C. Matrices and Linear Algebra 802

Index 805

M>M>

*

**

ix

Preface

This book provides a carefully motivated, accessible, and interesting introduction toprobability, statistics, and random processes for electrical and computer engineers. Thecomplexity of the systems encountered in engineering practice calls for an understand-ing of probability concepts and a facility in the use of probability tools. The goal of theintroductory course should therefore be to teach both the basic theoretical conceptsand techniques for solving problems that arise in practice. The third edition of thisbook achieves this goal by retaining the proven features of previous editions:

Relevance to engineering practice Clear and accessible introduction to probability Computer exercises to develop intuition for randomness Large number and variety of problems Curriculum flexibility through rich choice of topics Careful development of random process concepts.

This edition also introduces two major new features:

Introduction to statistics Extensive use of MATLAB/Octave.

RELEVANCE TO ENGINEERING PRACTICE

Motivating students is a major challenge in introductory probability courses. Instructorsneed to respond by showing students the relevance of probability theory to engineeringpractice. Chapter 1 addresses this challenge by discussing the role of probability modelsin engineering design. Practical current applications from various areas of electrical andcomputer engineering are used to show how averages and relative frequencies providethe proper tools for handling the design of systems that involve randomness. These ap-plication areas include wireless and digital communications, digital media and signalprocessing, system reliability, computer networks, and Web systems. These areas areused i

Recommended