PROBABILITYTHEORYSTOCHASTICPROCESSES

FOURTH EDITION

Y Mallikarjuna ReddyDepartment of Electronics and Communication EngineeringVasireddy Venkatadri Institute of Technology, Guntur, A.R

<®Universities Press

Contents

Preface

I Introduction to Probability1.1 Introduction 1

1.2 Set Theory 2

1.2.1 Terminology 2

1.2.2 Set Operations 3

1.2.3 Laws of Sets 6

1.2.4 Sample Spaces 7

1.2.5 Events 8

1.3 The Relative Frequency and Axioms ofProbability 9

1.3.1 Probability Introduced through Relative Frequency 9

1.3.2 Probability Introduced through Axioms 10

1.3.3 Classical Definition ofProbability 11

1.4 Mathematical Model ofExperiments 12

1.4.1 Examples ofExperiments 13

1.5 Joint and Conditional Probability 14

1.5.1 Joint Probability 14

1.5.2 Conditional Probability 15

1.5.3 Properties ofConditional Probability 16

1.6 Total Probability Theorem 18

1.7 Bayes' Theorem 19

1.8 Independent Events 22

1.8.1 Multiplication Theorem ofProbability 22

1.8.2 Properties of Independent Events 23

1.9 Combined Sample Space 23

1.9.1 Independent Experiments 24

1.9.2 Permutations and Combinations 25

1.10 Bernoulli Trials 27

Additional Problems 29

More SolvedExamples 52

Questions 70

Problems 71

Answers 74

Multiple-Choice Questions 75

Answers 78

Contents

2 The Random Variable 79

2.1 Introduction 79

2.1.1 Random Variable 79

2.1.2 Classifications ofRandom Variables 80

2.2 Probability Distribution Function 81

2.2.1 Expression for Distribution Function 83

2.3 Probability Density Function 84

2.3.1 Expression for Density Function 85

2.3.2 Properties ofProbability Distribution Functions 86

2.3.3 Properties ofProbability Density Functions 87

2.3.4 Probability Mass Function 88

2.4 Examples of Distribution and Density Functions 90

2.4.1 Gaussian Density Function 91

2.4.2 Uniform Density Function 94

2.4.3 Exponential Probability Density Function 95

2.4.4 Rayleigh Probability Density Function 96

2.4.5 Binomial Probability Density Function 98

2.4.6 Poisson's Probability Density Function 99

2.5 Conditional Distribution Function 99

2.5.1 Properties ofConditional Distribution Function 100

2.6 Conditional Density Function 100

2.6.1 Properties ofConditional Density Functions 100

2.7 Distribution Function for a Conditional Event 103

Additional Problems 105

More Solved Examples 114

Questions 136

Problems 137

Answers 140

Multiple-Choice Questions 140

Answers 142

3 Operations on a Single Random Variable 143

3.1 Introduction 143

3.2 Mathematical Expectation 143

3.2.1 Expected Value ofa Random Variable 143

3.2.2 Expected Value ofa Function ofa Random Variable 144

3.2.3 Conditional Expectation ofa Random Variable 144

3.3 Properties ofExpectation 145

3.4 Moments

3.4.1 Moments about the Origin

3.4.2 Moments about the Mean

3.5 Variance

3.5.1 Physical Significance ofVariance and Standard Deviation

3.5.2 Skew and Coefficient ofSkewness

3.5.3 Properties ofVariance

3.5.4 Relationship between Central Moments and Moments about Origin

3.6 Functions for Moments

3.6.1 Characteristic Function

3.6.2 Properties ofthe Characteristic Function

3.6.3 Moment Generating Function

3.6.4 Properties ofthe Moment Generating Function

3.7 Inequalities

3.7.1 Chebychev's Inequality

3.7.2 Markov Inequality

3.7.3 Chernoffs Inequality and Bound

3.8 Transformations ofa Random Variable

3.8.1 Monotonic Transformation ofa Continuous Random Variable

3.8.2 Non-Monotonic Transformation ofa Continuous Random Variable

3.8.3 Transformation ofa Discrete Random Variable

Additional Problems

More Solved Examples

QuestionsProblems

Answers

Multiple-Choice Questions

Answers

Multiple Random Variables

4.1 Introduction

4.2 Joint probability Distribution Functions

4.2.1 Properties of Joint Distribution Functions

4.3 Joint Probability Density Function

4.3.1 Properties of Joint Density Function

4.4 Conditional Distribution and Density Functions

4.4.1 Point Conditioning

4.4.2 Internal Conditioning

Contents

4.5 Statistical Independence ofRandom Variables 241

4.6 Sum ofRandom Variables 243

4.6.1 Two Random Variables 243

4.6.2 Multiple Random Variables 244

4.7 Central Limit Theorem 245

4.8 Probability Mass Function 248

Additional Problems 251

More Solved Examples 273

Questions287

Problems 287

Answers289

Multiple-Choice Questions290

Answers 293

5 Operations on Multiple Random Variables 294

5.1 Introduction 294

52 Function ofJoint Random Variables 294

5.3 Joint Moments 294

5.3.1 Joint Moments about the Origin 294

5.3.2 Correlation 295

5.3.3 Properties ofCorrelation 295

5.3.4 Joint Central Moments 297

5.3.5 Covariance 298

5.3.6 Correlation Coefficient 298

5.3.7 Properties ofCovariance 298

5.4 Joint Characteristic Function 303

5.4.1 Properties ofJoint Characteristic Functions 303

5.5 Joint Moment Generating Function 307

5.5.1 Properties ofJoint Moment Generating Functions 307

5.6 Gaussian Random Variables 311

5.6.1 Two Random Variables 311

5.6.2 ^Random Variables 312

5.6.3 Properties ofGaussian Random Variables 316

5.7 Transformation ofRandom Variables 316

5.8 Linear Transformation ofGaussian Random Variables 318

5.9 Conditional Gaussian Density Functions 322

Additional Problems 323

More Solved Examples 344

Contents

Questions 360

Problems 360

Answers 363

Multiple-Choice Questions 363

Answers 370

6 Random Processes 371

6.1 Introduction 371

6.2 Definition 371

6.3 Classification of Random Processes 373

6.3.1 Continuous Random Processes 373

6.3.2 Discrete Random Processes 373

6.3.3 Continuous Random Sequencees 373

6.3.4 Discrete Random Sequencees 374

6.4 Distribution and Density Functions of Random Processes 374

6.4.1 Joint Distribution Functions ofa Random Process 375

6.4.2 Joint Density Functions of a Random Process 375

6.5 Independent Random Processes 376

6.6 Statistical Properties ofRandom Processes 376

6.6.1 Mean 376

6.6.2 Autocorrelation 376

6.6.3 Cross Correlation 377

6.7 Stationary Processes 377

6.7.1 First-order Stationary Processes 377

6.7.2 Second-Order Stationary Processes 378

6.7.3 Wide-Sense Stationary Processes (WSS) 378

6.7.4 Jointly Wide-Sense Stationary Processes 379

6.7.5 Strict-Sense Stationary Processes (SSS) 379

6.8 Time Averages ofa Random Process 381

6.8.1 Time Average Function 381

6.8.2 Time Autocorrelation Function 381

6.8.3 Time Cross Correlation Function 381

6.9 Ergodic Theorem and Ergodic Processes 382

6.9.1 Ergodic Processes 382

6.9.2 Jointly Ergodic Processes 382

6.9.3 Mean Ergodic Processes 383

6.9.4 Autocorrelation Ergodic Processes 383

6.9.5 Cross Correlation Ergodic Processes 383

Contents

6.10 Properties of Autocorrelation Functions 383

6.11 Properties of Cross Correlation Functions 387

6.12 Covariance Functions for Random Processes 389

6.12.1 Autocovariance Function 389

6.12.2 Cross Covariance Function 390

6.13 Gaussian Random Processes 395

6.14 Poisson Random Processes 396

Additional Problems 398

More SolvedExamples 402

Questions 423

Problems 424

Answers 426

Multiple-Choice Questions 427

Answers 432

7 Random Processes: Spectral Characteristics 433

7.1 Introduction 433

12 Power Density Spectrum 433

7.2.1 Average Power ofthe Random Process 434

7.3 Properties ofthe Power Density Spectrum 436

7.4 Bandwidth ofthe Power Density Spectrum 441

7.5 Cross Power Density Spectrum 444

7.5.1 Average Cross Power 444

7.6 Properties of Cross Power Density Spectrum.

446

Additional Problems 451

More Solved Examples 458

Questions 411

Problems 478Answers 480

Multiple-Choice Questions 481Answers 486

8 Linear Systems with Random Processes 487

8.1 Introduction 487

82 Linear and Time Invariant Systems 487

8.2.1 Linear System 487

8.2.2 Response of a Linear System 488

8.2.3 Linear Time Invariant Systems (LTI) 489

Contents

8.2.4 Transfer Function ofan LTI System 490

8.2.5 Causal Systems 492

8.2.6 Stable Systems 493

8.2.7 Ideal Systems 494

8.3 Response ofLinear Systems to Random Signals 495

8.3.1 System Response 495

8.3.2 Mean Value of Output Response 495

8.3.3 Mean Square Value of Output Response 4%

8.3.4 Autocorrelation Function of Output Response 497

8.3.5 Cross Correlation Function of Response 498

8.4 Spectral Characteristics of System Response 500

8.4.1 Power Density Spectrum of Response 500

8.4.2 Spectrum Bandwidth 502

8.5 Types of Random Processes 502

8.5.1 Lowpass Random Processes 503

8.5.2 Bandpass Random Processes 503

8.5.3 Band Limited Random Processes 504

8.5.4 Narrow Band Random Processes 504

8.5.5 Properties ofBand Limited Random Processes 505

8.6 Noise 506

8.6.1 Introduction 506

8.7 Classification ofNoise 507

8.7.1 External Noise 507

8.7.2 Internal Noise 508

8.8 White Noise or White Gaussian Noise 511

8.8.1 Power Spectrum of White Noise 512

8.8.2 Band Limited White Noise 512

8.9 ResistorNoise Voltage 515

8.10 Equivalent Noise Resistor 515

8.11 Resistor Noise Spectral Density 516

8.12 Available Noise Power 517

8.13 Equivalent Noise Temperature 518

8.14 Noise through Two Port Networks 521

8.15 Signal-to-Noise Ratio 521

8.16 Available Power Gain 522

8.17 Equivalent Noise Bandwidth 522

8.18 Equivalent (Effective) Input Noise Temperature 526

8.19 Noise Figure 526

Contents

8.19.1 Noise Figure in Terms ofAvailable Power Gain 527

8.19.2 Noise Figure in Terms ofInput Noise Temperature 527

8.19.3 Noise Figure in Terms ofSignal-to-Noise Ratio 527

8.19.4 Noise Figure in Terms ofNetwork Transfer Function 528

8.19.5 Spot Noise Figure 529

8.19.6 Average Operating Noise Figure 529

820 Output Noise Power and System Noise Power 530

821 Noise in CascadeAmplifiers 531

822 AntennaNoise Temperature 534

823 Narrow Band Noise 534

8.23.1 In Phase and Quadrature Components ofa Narrow Band Noise 535

824 Properties ofa Narrow Band Noise 535

825 Ideal Narrow Band White Noise 536

Additional Problems 538

More Solved Examples 555

Questions 572

Problems 573

Answers 577

Multiple-Choice Questions 577

Answers 584

More Solved Questions on All Chapters 585

Appendix A: Indefinite Integrals, Definite Integrals and Finite Series 653

Appendix B: Fourier Transform Pairs 655

Bibliography 656

Index 657