Stochastic Modeling in BroadbandCommunicationsSystems

MM08Kaj_fm1.qxd 9/27/02 2:22 PM Page 1

Dow

nloa

ded

10/2

4/12

to 1

28.1

48.2

52.3

5. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p

About the SeriesIn 1997, SIAM began a new series on mathematical modeling andcomputation. Books in the series develop a focused topic from itsgenesis to the current state of the art; these books

• present modern mathematical developments with directapplications in science and engineering;

• describe mathematical issues arising in modern applications;

• develop mathematical models of topical physical, chemical, orbiological systems;

• present new and efficient computational tools and techniquesthat have direct applications in science and engineering; and

• illustrate the continuing, integrated roles of mathematical,scientific, and computational investigation.

Although sophisticated ideas are presented, the writing style ispopular rather than formal. Texts are intended to be read by audiences with little more than a bachelor’s degree in mathematics orengineering. Thus, they are suitable for use in graduate mathematics, science, and engineering courses.

By design, the material is multidisciplinary. As such, we hope tofoster cooperation and collaboration between mathematicians,computer scientists, engineers, and scientists. This is a difficult taskbecause different terminology is used for the same concept in different disciplines. Nevertheless, we believe we have been successful and hope that you enjoy the texts in the series.

Joseph E. Flaherty

Ingemar Kaj, Stochastic Modeling in Broadband Communications Systems

Peter Salamon, Paolo Sibani, and Richard Frost, Facts, Conjectures, andImprovements for Simulated Annealing

Lyn C. Thomas, David B. Edelman, and Jonathan N. Crook, Credit Scoring and ItsApplications

Frank Natterer and Frank Wübbeling, Mathematical Methods in ImageReconstruction

Per Christian Hansen, Rank-Deficient and Discrete Ill-Posed Problems: NumericalAspects of Linear Inversion

Michael Griebel, Thomas Dornseifer, and Tilman Neunhoeffer, NumericalSimulation in Fluid Dynamics: A Practical Introduction

Khosrow Chadan, David Colton, Lassi Päivärinta, and William Rundell, AnIntroduction to Inverse Scattering and Inverse Spectral Problems

Charles K. Chui, Wavelets: A Mathematical Tool for Signal Analysis

Editorial Board

Ivo BabuskaUniversity of Texas at Austin

H. Thomas BanksNorth Carolina StateUniversity

Margaret CheneyRensselaer PolytechnicInstitute

Paul DavisWorcester PolytechnicInstitute

Stephen H. DavisNorthwestern University

Jack J. DongarraUniversity of Tennesseeat Knoxville and OakRidge NationalLaboratory

Christoph HoffmannPurdue University

George M. HomsyStanford University

Joseph B. KellerStanford University

J. Tinsley OdenUniversity of Texas at Austin

James SethianUniversity of Californiaat Berkeley

Barna A. SzaboWashington University

SIAM Monographs on Mathematical Modelingand Computation

Editor-in-ChiefJoseph E. FlahertyRensselaer PolytechnicInstitute

MM08Kaj_fm1.qxd 9/27/02 2:22 PM Page 2

Dow

nloa

ded

10/2

4/12

to 1

28.1

48.2

52.3

5. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p

Stochastic Modeling in BroadbandCommunicationsSystems

Ingemar KajUppsala UniversityUppsala, Sweden

Society for Industrial and Applied MathematicsPhiladelphia

MM08Kaj_fm1.qxd 9/27/02 2:22 PM Page 3

Dow

nloa

ded

10/2

4/12

to 1

28.1

48.2

52.3

5. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p

Copyright © 2002 by the Society for Industrial and Applied Mathematics.

10 9 8 7 6 5 4 3 2 1

All rights reserved. Printed in the United States of America. No part of this book may bereproduced, stored, or transmitted in any manner without the written permission of the publisher. For information, write to the Society for Industrial and Applied Mathematics, 3600 University City Science Center, Philadelphia, PA 19104-2688.

The PING Utility Library is a public domain software package developed by MarkLindner and is distributed under the terms of the GNU Lesser General PublicLicense. The package can be freely downloaded from http://www.dystance.net/ping

Academy Award is a registered trademark of the Academy of Motion Picture Arts andSciences.

Library of Congress Cataloging-in-Publication Data

Kaj, Ingemar.Stochastic modeling in broadband communications systems / Ingemar Kaj.

p. cm. — (SIAM monographs on mathematical modeling and computation)Includes bibliographical references and index.ISBN 0-89871-519-91. Broadband communication systems—Mathematical models. 2. Stochastic analysis. I.

Title. II. Series.

TK5103.4 .K35 2002621.382—dc21 2002029186

is a registered trademark.

MM08Kaj_fm1.qxd 9/27/02 2:22 PM Page 4

Dow

nloa

ded

10/2

4/12

to 1

28.1

48.2

52.3

5. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p

“kaj”2002/10/8page v

✐

✐

✐

✐

✐

✐

✐

✐

Contents

Preface ix

Notation and Notions from Probability Theory xiii

1 Introduction 11.1 A brief introduction to networking concepts. . . . . . . . . . . . . . 11.2 Modeling aspects of general networks. . . . . . . . . . . . . . . . . 21.3 Broadband traffic characteristics. . . . . . . . . . . . . . . . . . . . 41.4 Three introductory examples. . . . . . . . . . . . . . . . . . . . . . 9

1.4.1 Simple collision model . . . . . . . . . . . . . . . . . . 91.4.2 Basic arrivals process. . . . . . . . . . . . . . . . . . . 131.4.3 Periodic streams. . . . . . . . . . . . . . . . . . . . . . 17

1.5 Exercises. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2 Markov Service Systems 212.1 Discrete-time service systems. . . . . . . . . . . . . . . . . . . . . . 212.2 Arrival and service rates, continuous time. . . . . . . . . . . . . . . . 242.3 Ideas of stationarity and equilibrium states. . . . . . . . . . . . . . . 282.4 Balance equations, slotted time. . . . . . . . . . . . . . . . . . . . . 302.5 Balance equations, continuous time. . . . . . . . . . . . . . . . . . . 322.6 Jackson networks. . . . . . . . . . . . . . . . . . . . . . . . . . . . 352.7 Markov loss systems. . . . . . . . . . . . . . . . . . . . . . . . . . . 382.8 Delay analysis in Markov systems. . . . . . . . . . . . . . . . . . . . 39

2.8.1 Delay in M/M/1 . . . . . . . . . . . . . . . . . . . . . . 392.8.2 A client-server Jackson network. . . . . . . . . . . . . . 41

2.9 Exercises. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

3 Non-Markov Systems 453.1 Performance measures. . . . . . . . . . . . . . . . . . . . . . . . . . 453.2 Integrated processes and time averages. . . . . . . . . . . . . . . . . 473.3 Some ideas from renewal theory. . . . . . . . . . . . . . . . . . . . 50

3.3.1 Renewal reward processes. . . . . . . . . . . . . . . . . 513.3.2 Renewal rate and on–off processes. . . . . . . . . . . . 513.3.3 Hand-off termination probability. . . . . . . . . . . . . 53

v

Dow

nloa

ded

10/2

4/12

to 1

28.1

48.2

52.3

5. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p

“kaj”2002/10/8page vi

✐

✐

✐

✐

✐

✐

✐

✐

vi Contents

3.3.4 Reliable data transfer. . . . . . . . . . . . . . . . . . . 543.4 The loss and delay time balance. . . . . . . . . . . . . . . . . . . . . 57

3.4.1 Little’s formula . . . . . . . . . . . . . . . . . . . . . . 583.5 The M/G/1 system. . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

3.5.1 Simple examples leading to non-Markovity. . . . . . . . 603.5.2 Pollaczek–Khinchin formulas. . . . . . . . . . . . . . . 623.5.3 Lindley recursion for M/G/1. . . . . . . . . . . . . . . . 653.5.4 The M/G/1 virtual waiting time distribution. . . . . . . . 663.5.5 Heavy traffic limit in M/G/1. . . . . . . . . . . . . . . . 683.5.6 Deterministic service times, M/D/1. . . . . . . . . . . . 70

3.6 The M/G/∞model . . . . . . . . . . . . . . . . . . . . . . . . . . . 723.7 Exercises. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

4 Cell-Switching Models 774.1 m × m crossbar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

4.1.1 Output loss crossbar. . . . . . . . . . . . . . . . . . . . 784.1.2 Output queuing with a shared buffer. . . . . . . . . . . . 804.1.3 Input buffer blocking . . . . . . . . . . . . . . . . . . . 824.1.4 Input blocking, loss system. . . . . . . . . . . . . . . . 86

4.2 Exercises. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

5 Cell and Burst Scale Traffic Models 935.1 Cell-level traffic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

5.1.1 Isochron multiplexing. . . . . . . . . . . . . . . . . . . 945.1.2 Voice packet streams in Internet telephony. . . . . . . . 965.1.3 Round-trip time distribution, PING data. . . . . . . . . 1005.1.4 Packet fragmentation in video communications. . . . . . 103

5.2 Burst-level rate models. . . . . . . . . . . . . . . . . . . . . . . . . 1065.2.1 Anick–Mitra–Sondhi model. . . . . . . . . . . . . . . . 1065.2.2 Markov modulated Poisson process. . . . . . . . . . . . 108

5.3 Long-range dependence traffic models. . . . . . . . . . . . . . . . . 1095.3.1 Self-similarity . . . . . . . . . . . . . . . . . . . . . . . 1115.3.2 Heavy-tailed rate models. . . . . . . . . . . . . . . . . 1135.3.3 Fractional Brownian motion. . . . . . . . . . . . . . . . 1145.3.4 Statistical methods. . . . . . . . . . . . . . . . . . . . . 118

5.4 Exercises. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

6 Traffic Control 1236.1 Admission control. . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

6.1.1 Effective bandwidth. . . . . . . . . . . . . . . . . . . . 1246.1.2 Statistical multiplexing gain. . . . . . . . . . . . . . . . 125

6.2 Access control. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1276.2.1 Leaky bucket systems. . . . . . . . . . . . . . . . . . . 1276.2.2 The M/M/1 leaky bucket. . . . . . . . . . . . . . . . . . 1286.2.3 The generic cell rate algorithm. . . . . . . . . . . . . . 1316.2.4 A slotted version of the leaky bucket filter. . . . . . . . . 131D

ownl

oade

d 10

/24/

12 to

128

.148

.252

.35.

Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

“kaj”2002/10/8page vii

✐

✐

✐

✐

✐

✐

✐

✐

Contents vii

6.3 Multiaccess modeling. . . . . . . . . . . . . . . . . . . . . . . . . . 1346.3.1 The slotted Aloha Markov chain. . . . . . . . . . . . . . 1366.3.2 Diffusion approximation approach. . . . . . . . . . . . 1376.3.3 Remark on stochastic differential equation approximation 1406.3.4 CSMA and CSMA/CD . . . . . . . . . . . . . . . . . . 1416.3.5 A collision resolution algorithm. . . . . . . . . . . . . . 147

6.4 Congestion control. . . . . . . . . . . . . . . . . . . . . . . . . . . . 1496.4.1 A controlled Aloha network. . . . . . . . . . . . . . . . 1496.4.2 Window control . . . . . . . . . . . . . . . . . . . . . . 1506.4.3 Modeling TCP window size. . . . . . . . . . . . . . . . 1526.4.4 TCP window dynamics. . . . . . . . . . . . . . . . . . 1536.4.5 Meanfield approximation of interacting TCP sources . . . 159

6.5 Exercises. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162

Bibliography 167

Index 173

Dow

nloa

ded

10/2

4/12

to 1

28.1

48.2

52.3

5. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p

“kaj”2002/10/8page ix

✐

✐

✐

✐

✐

✐

✐

✐

Preface

This text is intended for students in mathematics, applied mathematics, and stochasticswho have an interest in network modeling and for students in computer science and relatedareas with an open view toward mathematical models. The material also will be useful formany practitioners in the computer communications or telecommunications industry whouse probabilistic models and methods.

Mathematical methods based on the theory of stochastic processes have long beenused effectively in telephone traffic modeling. The original telephone traffic models de-veloped and published (1909–1927) by the Danish mathematician A. K. Erlang formed thetheoretical framework for planning and dimensioning the growing telephone networks fordecades to come. The work of Erlang at a telephone company in Copenhagen must in factbe considered among the single most successful theories in the history of applied mathemat-ics. Not until the development of the emerging techniques in high capacity communicationsystems has it become clear that Erlang’s legacy has reached its limits. Even basic math-ematical traffic modeling requires a wider ranging selection of ideas and techniques. Theinherent structure of modern network traffic, which is distinctly different from traditionalvoice traffic, generates challenging mathematical and statistical problems. Industry ac-knowledges the need for mathematical competence in this area, judging from the growthof conferences, academia–industry cooperative projects, and recruitment to industry-basedresearch departments.

This book covers material suitable for final-year undergraduate students to the Ph.D.level in mathematics, probability and statistics, computer science, and computer engineer-ing. The selection of topics depends on the reader’s background and interest. The reader isexpected to have basic knowledge of calculus and probability, including random variables,probability distributions, and expected values. A brief introduction to networking conceptsis included. The presentation of the main material covers a variety of models and situationsranging over different time scales of calls, bursts, and cells and over different protocol layersfor transport, control, and applications. The mechanisms of queuing, collisions, delay, andloss appear in various forms, and the effects of buffering, retransmission, multiplexing, andtraffic control are studied. Typically, the end result is some form of load-throughput analy-sis. The common theme is that all models are formulated in terms of appropriate stochasticquantities and the main mathematical tools are those of equilibrium Markov chain theory,renewal theory, and asymptotic limit results.

The classical Markov queuing systems and more general single-server systems arecovered as starting points and reference systems. The reader will find relatively simplestochastic models for more realistic networking problems such as reliable data transfer

ix

Dow

nloa

ded

10/2

4/12

to 1

28.1

48.2

52.3

5. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p

“kaj”2002/10/8page x

✐

✐

✐

✐

✐

✐

✐

✐

x Preface

protocols, the forced termination problem in cellular networks, space division switching,Internet telephony traffic, leaky bucket filters, the ethernet local area network protocolscarrier-sensing multiple access (CSMA) and CSMA with collision detection (CSMA/CD),collision resolution protocols, and the window dynamics in transmission control protocol.Moreover, the reader will find material related to arrival process modeling, which is designedfor network traffic exhibiting long-range dependence and self-similarity. This includesstatistical methods and approximation by means of fractional Brownian motion.

The selection of topics should serve as a background from which those who have aninterest in the area will be able to continue in one or another direction. As several topicstouch on or intersect with current research, the text could serve as a basis for independentinvestigations. It is my hope that the chosen level of mathematical rigor is acceptable formost readers.

The purpose of this text is to give an overview of stochastic models and mathematicaltechniques based on stochastic processes for application in the fields of telecommunica-tions and computer communication networks. The presentation introduces readers withvarious backgrounds and training in mathematics to a number of useful techniques in trafficmodeling. The intended audience consists, on one hand, of students and professionals intelecommunications and computer engineering with an interest in using applied mathemat-ics, in particular stochastics, to improve their understanding of communications systems.On the other hand, it is written for the purpose of introducing students and professionalsin probability and applied mathematics to a huge area of interesting problems and modelsarising from today’s accelerating developments in broadband channel transmission systems.

Given this twofold purpose, the text should be concise and based on sound mathemat-ical reasoning yet be accessible for an audience unwilling to spend more than a fair shareof time on mathematical detail and generality. The approach suggested here to serve thisobjective is to rely on the language and concepts of random variables and stochastics andthe strength in intuitive reasoning they provide. Main ideas and notions are introduced anddiscussed along with specific network applications, and most calculations are motivatedand carried out in detail. Probabilistic arguments are preferred to analytical ones—for ex-ample, we have chosen not to use moment-generating functions. We state and use generalresults from the theory of Markov chains and renewal processes, but for detailed proofs andsystematic treatment, readers should consult existing textbooks on mathematical queuingtheory, such as Kleinrock [28], Asmussen [3], Wolff [69], and Brémaud [7].

Basic calculus and probability as prerequisites should be enough as a starting point.Some of the models we discuss require mathematically more advanced material, such asdiffusion approximations and nonlinear differential equations. However, we provide intro-ductions and emphasize intuitive probabilistic arguments. A number of illustrations withgraphs of real data or model simulations are given for clarity. Exercises are provided at theend of each chapter partly to promote the idea of using the book within a course. Someexercises support training in probabilistic calculus, some study variations of models dis-cussed in the main text, and additional comprehensive exercises could be used as courseassignments.

Chapter 1 contains a brief introduction to basic concepts in networking and com-munication systems and also to the nature of real traffic data. For readers with limitedbackground in probability theory we provide a summary of notions and distributions inelementary probability and discuss introductory examples, including a short presentation ofD

ownl

oade

d 10

/24/

12 to

128

.148

.252

.35.

Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

“kaj”2002/10/8page xi

✐

✐

✐

✐

✐

✐

✐

✐

Preface xi

the Poisson process.Chapter 2 gives a summary and introduction to Markov chain theory in discrete and

continuous time, focusing on equilibrium properties. Elements of queuing, loss, and delayare covered as well as the Jackson network of Markov service systems.

In Chapter 3 we begin with performance measures and study load-versus-throughputrelationships. The main objective is to discuss non-Markov dynamics and study variousmodeling techniques, including renewal processes, renewal rate processes, and on–off pro-cesses, and to cover standard material in queuing theory, such as Little’s formula and theM/G/1 model. Specific applications include forced termination of a mobile phone andreliable data transfer protocols.

Chapter 4 is devoted to the study of the simplest loss and contention protocols inpacket switching. The basic example is anm × n crossbar switch, where packets arrivingonm input lines are randomly switched onton outputs. This results in either loss or bufferdelay due to contention for output lines. A particular artifact in some of these models is theso-called head-of-line blocking phenomenon.

Chapter 5 addresses traffic modeling relevant for cell and burst time scales. We beginwith specific models for isochronous cell streams, Internet telephony, and a fragmentationprocedure for video communications. Then we turn to the multiplexing of independentsources over a joint transmission channel—for example, the Anick–Mitra–Sondhi modelof superpositioned on–off sources. An important finding from recent research in this areashows that the addition of distributions with heavy tails to this class of models will result inlong-range dependence. We discuss the related idea of self-similarity and give an accounton the topic of approximating, in a certain sense, network traffic using the continuous self-similar process known as fractional Brownian motion. A section on data analysis includesstatistical methods for identifying heavy tails.

In Chapter 6 we apply a number of techniques and models to the study of variousaspects of traffic control. As part of admission control we study the topics of effectivebandwidth and statistical multiplexing gain. Access control includes several versions of theleaky bucket mechanism. Multiaccess control from a mathematical perspective involvesmodeling the retransmission mechanisms in contention protocols. We introduce the ideasusing principles of the simplest Aloha-net and generalize to the ethernet protocols CSMAand CSMA/CD. Contention protocols based on collision-resolution algorithms are modeledusing rather different methods. Finally, we treat congestion control of Internet traffic in adetailed study of the transmission control protocol (TCP) and Internet protocol (IP) windowdynamics scheme.

Obviously many topics of interest have been omitted from our presentation or aretouched on only briefly. Some areas would have required a background of rather sophis-ticated mathematics, such as the theory of large deviations, which has found key appli-cations in, for example, large-scale asymptotics of buffer overflow probabilities [57], andmatrix-analytic methods leading, for example, to numerical schemes for calculating lossprobabilities.

Acknowledgments This book was developed partly on the basis of lecture notes forcourses given over several years. A main impulse was the opportunity to spend one termat Carleton University, Ottawa, and give a joint graduate course for students at the De-D

ownl

oade

d 10

/24/

12 to

128

.148

.252

.35.

Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

“kaj”2002/10/8page xii

✐

✐

✐

✐

✐

✐

✐

✐

xii Preface

partment of Systems and Computer Engineering and the Department of Mathematics andStatistics. I am most grateful to Amit Bose at the Laboratory for Research in Statisticsand Probability for taking the initiative for this project and for his continued support andcooperation. Thanks to further support from Ioannis Lambadaris and Michael Devetsikiotisat the Broadband Networks Laboratory, Carleton University, I was provided with excellentworking conditions in an inspiring research environment. It is my pleasure to thank GunnarKarlsson, Department of Microelectronics and Information Technology, Royal Institute ofTechnology, Kista, and Mats Rudemo, Chalmers University of Technology, Gothenburg,for similar opportunities to lecture graduate courses for other groups of advanced students.

Many students in these and other courses influenced the content and style of the bookand gave valuable input and inspiration for selecting topics and problems. Among theseare Tarkan Taralp, Matthias Falkner, Mattias Östergren, and Anders Andersson. Some havebecome coworkers and are directly involved in research reported in the book: RaimundasGaigalas, Jörgen Olsén, and Ian Marsh. I am grateful to Evsey Morozov, PetrozavodskUniversity, for carefully reading the manuscript and providing many useful comments.Special thanks are due to Ian Marsh, Swedish Institute of Computer Science, for readingthe manuscript in great detail and for numerous comments that improved both language andcontent.

During the writing of this book I met my wife, Olga. Her support and love have beeninvaluable.

I.K.

Dow

nloa

ded

10/2

4/12

to 1

28.1

48.2

52.3

5. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p

“kaj”2002/10/8page xiii

✐

✐

✐

✐

✐

✐

✐

✐

Notation and Notions from ProbabilityTheory

It is assumed that the reader is familiar with the basic ideas of elementary probability theory,in particular the concepts of stochastic variables, probability distributions, and expectedvalues. For the reader’s convenience and to introduce notation used throughout the text,some of these notions are recapitulated here, including a listing of standard distributions.In addition, this section contains some terminology and lists a few properties related toconditional expectations, Markov chains, and convergence of random variables. For detailedaccounts the reader should consult such textbooks as Gut [16].

Distribution function: F(x) = P(X ≤ x).

Quantile: The numberxα that satisfiesF(xα) = 1 − α.

Discrete random variable: X is discrete if it assumes a finite or countable number ofvaluesx1, x2, . . . with probabilitiesp(x1), p(x2), . . . , wherep(x) is theprobabilityfunctionfor X.

Continuous random variable: X is continuous if it assumes all values in an intervalaccording to adensity functionf (x),

1. F(x) is continuous for allx,

2. F ′(x) = f (x) for all x where the derivative exists, and

3. P(a < X < b) = ∫ b

af (x)dx.

Joint distributions:

F(x1, . . . , xr) = P(X1 ≤ x1, . . . , Xr ≤ xr)

p(x1, . . . , xr) = P(X1 = x1, . . . , Xr = xr)

f (x1, . . . , xr) = ∂∂x1

. . . ∂∂xr

F (x1, . . . , xr).

Expected value:

E(X) = µ =

∑i

xi p(xi) (X discrete)∫ ∞

−∞x f (x) dx (X continuous).

xiii

Dow

nloa

ded

10/2

4/12

to 1

28.1

48.2

52.3

5. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p

“kaj”2002/10/8page xiv

✐

✐

✐

✐

✐

✐

✐

✐

xiv Notation

Variance: V (X) = σ 2 = E((X − µ)2).

Standard deviation: D(X) = σ = √V (X).

Covariance: Cov(X, Y )= E((X − µx)(Y − µy)).

Correlation coefficient: ρ(X, Y ) = Cov(X,Y )

D(X)D(Y ).

Independence: Two random variablesX andY are said to beindependentif

P(X ≤ x, Y ≤ y) = P(X ≤ x) P (Y ≤ y) for all x andy.

This impliesE(XY) = E(X)E(Y ), hence Cov(X, Y )= 0, in which caseX andY

are said to beuncorrelated.

The following are the most common discrete distributions.

Binomial distribution: X ∈ Bin(n, p) if p(k) = (n

k

)pk (1 − p)n−k, 0 ≤ k ≤ n,

µ = np, σ 2 = np(1 − p).

Geometric distribution: X ∈ Ge(p) if p(k) = p(1 − p)k, k = 0, 1, . . . ,

µ = 1−p

p, σ 2 = 1−p

p2 .

Positive geometric distribution: X ∈ Ge+(p) if p(k) = p(1 − p)k−1, k = 1,2, . . . ,

µ = 1p, σ 2 = 1−p

p2 .

Poisson distribution: X ∈ Po(m) if p(k) = mk

k! e−m, k = 0, 1, . . . ,

µ = m, σ 2 = m.

Hypergeometric distribution: X ∈ Hyp(N, n, p) if p(k) = (Np

k

)(N(1−p)

n−k

)/(N

n

), k =

0, 1, . . . , n (for k that are possible, e.g.,k ≤ Np),

µ = np, σ 2 = np(1 − p) N−nN−1 .

Multinomial distribution: X = (X1, · · · , Xr) ∈ Multnom(n;p1, . . . , pr)

if p(k1, . . . , kr ) = n!k1!...kr ! p

k11 . . . pkr

r ,∑r

j=1 kj = n,∑r

j=1 pj = 1,

E(Xj ) = npj , V (Xj ) = npj (1 − pj ),

Cov(Xi,Xj ) = −npipj , i = j .

Similarly, the most important continuous probability distributions are

Uniform (rectangular) distribution: X ∈ Re(a, b) if f (x) = (b − a)−1, a ≤ x ≤ b,

µ = a+b2 , σ 2 = (b−a)2

12 .

�-distribution: X ∈ !(p, a) if f (x) = !(p)−1apxp−1e−ax , x ≥ 0,

µ = p/a, σ 2 = p/a2.Dow

nloa

ded

10/2

4/12

to 1

28.1

48.2

52.3

5. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p

“kaj”2002/10/8page xv

✐

✐

✐

✐

✐

✐

✐

✐

Notation xv

Exponential distribution: X ∈ Exp(a) if X ∈ !(1, a), f (x) = ae−ax , x ≥ 0,

µ = 1/a, σ2 = 1/a2.

Normal distribution: X ∈ N(m, σ) if f (x) = 1√2π σ

e− (x−m)2

2σ2 , whereµ = m is the

expected value andσ 2 the variance.

For N(0, 1) the distribution function is written$(x), the densityϕ(x), and the quan-tiles zα.

Slightly more advanced material includes conditional probabilities and conditional expec-tations; we list some relations particularly useful for calculations:

Conditional mean: E(X) = E(E(X|Y )).

Conditional variance: V (X) = E(V (X|Y )) + V (E(X|Y )).

Conditional covariance: Cov(X, Y )= E(Cov(X, Y|Z)) + Cov(E(X|Z),E(Y |Z)).

Stochastic sums: Let {Xi} be independent identically distributed random variables, letN

be an integer valued random variable independent of (or a stopping time for){Xi},and putY = ∑N

i=1 Xi . Then

E(Y ) = E(N)E(X),

V (Y ) = E(N)V (X) + E(X)2V (N).

Markov chains are sequences of random variablesX1, X2, . . ., in which the future outcomeXn+1 depends on the present variableXn but is independent of the way in which the presentstate arose from its predecessorsX1, . . . , Xn−1.

Markov chain property: A sequence(Xn)n≥1 of random variables is a Markov chain iffor all n andx1, . . . , xn+1,

P(Xn+1 = xn+1|X1 = x1, . . . , Xn = xn) = P(Xn+1 = xn+1|Xn = xn).

Markov process: A process{Xt, t ≥ 0} in continuous time with discrete states is a Markovprocess if for any given trajectory{xr, 0 ≤ r ≤ t} of states and arbitrarys < t,

P (Xt = xt |Xr = xr, 0 ≤ r ≤ s) = P(Xt = xt |Xs = xs).

Several convergence concepts are used in probability theory. We state four of them, ofwhich the most important for the applications in this book are convergence almost surelyand convergence in distribution.

Let (Xn)n≥1 be a sequence of random variables.

Convergence almost surely: (Xn)n≥1 converges almost surely (a.s.) to a random variableX, Xn

a.s.→ X ifP(Xn → X asn → ∞) = 1.D

ownl

oade

d 10

/24/

12 to

128

.148

.252

.35.

Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

“kaj”2002/10/8page xvi

✐

✐

✐

✐

✐

✐

✐

✐

xvi Notation

Convergence in probability: (Xn)n≥1 converges in probability toX, XnP→ X if

for all ε > 0 P(|Xn − X| > ε) → 0, n → ∞.

Convergence in L2: (Xn)n≥1 converges inL2 to X, XnL2→ X if

E(Xn − X)2 → 0, n → ∞.

Convergence in distribution: (Xn)n≥1 converges in distribution toX, Xnd→ X if

P(Xn ≤ x) → P(X ≤ x), n → ∞, at every pointx ∈ C(F),

whereC(F) is the set of continuity points of the distribution functionF of X.

Dow

nloa

ded

10/2

4/12

to 1

28.1

48.2

52.3

5. R

edis

trib

utio

n su

bjec

t to

SIA

M li

cens

e or

cop

yrig

ht; s

ee h

ttp://

ww

w.s

iam

.org

/jour

nals

/ojs

a.ph

p