Paul Gerhard Hoel - Introduction to Stochastic Processes (the Houghton Mifflin Series in Statistics) (Houghton Mifflin,1972,0395120764)

Download Paul Gerhard Hoel - Introduction to Stochastic Processes (the Houghton Mifflin Series in Statistics) (Houghton Mifflin,1972,0395120764)

Post on 29-Jul-2015




3 download

Embed Size (px)


<p>Hoel Por Stone Iintroduction toStochastic Processes , Th.HoughtonMifinSeriesinStatistics undlertheEitorshipof HerlmanChernof LEOBREIMAN ProbabilityandStochasticJProcesses:WithaViewTowardApplications Statistics:WithaViewTowardApplications PAULG.HOEL,SIDNEYC.PORT,ANDCHARLESJ. STONE IntroductiontoProbabilityTheory IntroductiontoStatisticalT'eory IntroductiontoStochastic}rocesses PAULF.LAZARSFELDANDN1EIL. HENRY LatentStructureAnalysis G01IFRIEDE.NOETHER IntroductiontoStatistics-.AFreshApproach 3. b. CHO,HERBERTROBBns,W DAVSmGMUI) GreatExpectations:TheTheoryof OptimalStopping I.RICHARDSA NAGE Statistics:Uncertaintyand.Behavior nlDuCOD DbDCH5ClDC055C5Paul G.Hoel SidneyC.Port CharlesJ.Stone UniversityofCalifornia,LosAngeles HOUGHTONMIFFLIINCOMPANYBOSTON NewYorkAtlantaGeneva,IllinoisDaillasPaloAlto LOEYKLH1 l 972 UY HOLLH1ON MllLN LOMEANY.Al rightsresered.No Jpartofthisworkmaybt!reproducedortransmittedin any formorby anymeans,electronicormechanical,includingphotocopying andrecording,orbyany informationstorageorretrievalsystem,wlthout Jpermissioninwritingfromthepublisher. PRINTEDINTHEU.S.A. LmRARYOFCONGRESSCATALOCARDNUMBER:79-165035 ISBN:0-395-1 2076-4 General Preface Thisthree-volumeseriesgre'Noutofathree-quartercourseinproba.bility, statistics,andstochastic process(!staughtfor anumberof yearsatUCLA. VVefelt aneedforaseriesof booksthatwouldtreatthesesubjectsinaway thatiswell coordinate: d, but which would also give adequate emphasis to each subject as being interestingandusefulon itsownmerits. Thefrstvolume,oc|o o iooco|/|y I/eoy, presentsthefundarnental ideasofprobabilitytheoryandalsopreparesthestudentbothforcoursesin statistics and for further study in probability theory, including stochastic pro(;esses.Thesecondvolume,oc|o o se|s|ce/ Deoy, evelopsthebasic theoryofmathematicalstatisticsinasystematic,unifedmanner.Togethe:r,the frst two volumes contain the material that is often covered in a two-semester course inmathemlaticalstatistics. Thethirdvolume,Ioc|o o soc/es|c iocesses, treatsMarkovchains, Poissonprocesses,birthanddeathprocesses,Gaussianprocesses,Bro'wnian motion,andprocessesdefnedintermsofBrownianmotionbymeansofelementary stochastic diferentialequations.VPreface Inrecentyearstherehas been aneverincreasinginterestinthestudyofsystemswhichvaqtintimeinarandomnanner. Mathematica!mode|sofsuchsyste|sareknownas stochasticrocesses.Inthis bookwe resentane|ementaryaccountofsome ofthe important toics in thetheory ofsuch rocesses. We have tried tose|ect toics that are concetua||y interesting and that have found f|uitfu|a|icationin variousbranches ofscience and techno|ogy.A soc/:u|cocesscanbedehnedquitegenera||y as anyco||ectionofrandomvanab|es.:(/), EI, dehnedonacommonrobabi|itysace,where Iisasubsetof( 0 co)andisthoughtofasthetimearameter set. The rocessisca||edaco/|oseeeeocessif1'isaninterva|havingositive|engthanda/sce/eeeeeocessifIis asubset ofthe integers. Ifthe randomvariab|esx()a||take onva| uesfromthehxed set f,thenf isca||edthes/eeseceoftherocess.Many si.ochastic rocesses oftheoretica| and a|ied interest ossess th: ro-enythat, given the resent state ofthe rocess, the ast history does not adectconditiona| robabi|ities ofevents dehned in terms ofthe future. Suchrocessesareca||edMeko:ocesses. InChaters | andzwe studymeko:c/e|s,whichare discretearameter Markov processes whose state sace is hnite orcountab|yinhnite. InChater+westudythecorresondingcontinuousarameterrocesses,withthe"l'oissonrocess"as asecia|case.InChaters+wediscusscontinuousarameterrocesseswhosestatesace istyica||y the rea| |ine. In Chater 4we introduce 6ess|eocesses, which arecharacterized by the roerty that every |inear con.bination i nvo|ving a hnitenumber of the random variab|es x(/),/EI, is norma||y distributed. t\s animortant secia| case, we discuss the 0|eeocess, which arises as a n:athe-matica|mode| forthehysica|henomenon known as "Brownianmotion. "In Chater we discuss integ:ationand dierentiation ofstochastic rocesses. There we a|so usethe Wiener rocess to give a mathematica| mode| for 'whitenoise. "In Chater we discuss so|utions to nonhomogeneous ordinary dierentia|equations havingconstantcoemcients whose right-hand side iseither a stochasticrocess orwhite noise. We a|so discussestimationrob|ems invo|ving stochasticrocesses,andbrieyconsiderthe"sectra|distnbution"ofarocess.vv 70f900This text has been designed for a one-semester courseinstochasticrocesses.Written in c|ose conjunction v-ith a/oc|o /o ooeo4|/y I/eoy, the hrstvo|ume ofourthree-vo|ume series, itassumesthatth e studentisacquaintedwiththe materia| covered in aone-s:mestercourseinrobabi|ityforwhiche|ementaryca|cu|us is arerequisite.Some ofthe roofs in Chat:rs | and zare somevhat more dimcu|t than therestofthetext, andtheyaearinaendicestothesechaters. Theseroofsandthestarred materia| in Sectionz. robab|yshou|d b| omitted or discussed on|ybriey in an e|ementary courseAninstructorusingthis textinaone-quartercoursewi||robab|ynothavetimeto cover the entire text. He may wishto cover the hrst three chaters thorough|yandtheremainderastimeermits,erhasdiscussingthosetoicsinthe|astthreechaters that invo|ve the Wienerrocess. Anotherotion, however, isoe|ha-size continuous arameter roc:sses by omitting or skimming Chaters | and zand concentrating on Chaters 1-. (For exam|e , the instructor cou|d skiSections | . . | , | . . z, | . , z. z. z, z. . | , z. . | , and z. s)With some aid from theinstructor, the student shou|d be ab|e to read Chat|r 1 without having studiedthe hrst tvo chaters thorough|y. Chaters +are indeendent ofthe hrst twochatersanddeendonCater1on|y inminorways, main|yinthatthePoissonprocess introduced in Chater+is used in exam|es in the |ater chaters. Theroerties ofthe Poissoniocessthatareneeded|aterare summarizedinChater4andcanberegardedas axiomsforthePoissonproc:ss.The authors wish to thank the UCLA students who to|erated re|in:inaryversions of this text and whose comments resu|ted in numerous imrovernents.Mr. Luis0orostiza obtained the answers to the exercises and a|so made manysuggestions that resu|ted in signihcant imrovements. Fina||y, we wish to thankMrs.RuthGo|dsteinforherexce||enttying.T able of Contents 1MlarkovChains l1.1Markovchains havingtwo states 21.2Transition functionandinitia|distribution 51.3Exam|es 61.4Comutations with transitionfunctions l 21.4.1Hittingtimes l 41.4.2Transitionmatrix l 6 1.5;Transientandrecurrentstates l 7 1.6;Decomositionofthe state sace 2l 1.6.1Absortion robabi|ities 251.6.2Martinga|es 271.7'Birth and death chains 291.8:Branchingand queuingchains 331.8.1Branchingchain 341.8.2Queuingchain 36Aendix1.91Proofofresu|ts for the branchingandqueuingchains 361.9.1Branchingchain 3&amp;1.9.2Queuingchain 392St.ltionaryDistribut:ions of a MarkovChain 472.1E|ementaryroerties ofstationary distributions 472.2,Exam|es 492.2.1Birth and deathchain 502.2.2Partic|es in abox 532.3Average number ofvisits to arecurrent state 562.4Nu|| recurrent and ositive recurrent states 602.5;Existence and uniqueness ofstationaq/ distributions 632.5.1Reducib|echains 672.6;Queuingchain 692.6.1Proof 70XX 8D0 0f0.0t0Dt82.'7Convergenceto the stationary distribution 72Aendix2.: 8Proofofconvergence 752.S.1Periodiccase 772.8.2A resu|tfromnumbertheory 793NlarkovPureJum.Processes &amp;43.1Construction ofjumrocesses &amp;43.2Birth and deathrocesses &amp;93.2.1Two-statebirthanddeathrocess 923.2.2Poissonrocess 943.2.3Pure birthrocess 9&amp;3.2.4Inhnite serer queue 993.:3Proerties ofa Markovurejumrocess l 023.3.1A|icationsto birth and death rocesses l 044S.condOrderProcesses l l l 4.1Meanandcovariancefunctions l l l 4.:2Gaussian rocesses l l 94..3The Wiener rocess l 225C,ontinuity,Integrltion, andDiferelntiationofSecon,d OrderProcessesl 2&amp;5.1Continuity assumtionsl 2&amp;5.1.1Continuity ofthe mean andcovariance functions l 2&amp;5.1.2Continuity ofthe sam|e functions l 305.:2Integrationl 325.:3Dierentiation l 355.4White noise l4l6St:ochasticDiferelltialEquations,I:stimationTheor", alldSpectralDistriibutionsl 526.1First orderditrentia|equations l 546.:ZDierentia|equations ofordernl 596.2.1The case n=2 l 666.3 6.4 Estimationtheory6.3.1Genera|rinci|es ofestimation6.3.2Some exam|es ofotima|redictionSectra| distributionAnswers to ExercisesG|ossary ofNotationIndexl 70l 73l 74l 77l 90l 9920l1 MarkovChains (onsider asystemthat can beinany one of a fniteor countably infnite number of states. Letfdenote this set of states.We can assumethatfis a subset of the integers.Theset!iscalledthestatespaceofthesystem,.Letthesystembe observed at the discrete moments of time n=0,1, 2,. . . , and let Xn denote the state of thesystemattinlen. Sinceweareinterestedinnon-deterministicsystems,wethinkof Xmn&gt;0,as random variables defned on a common probability space.Little can besaid about such random variablesunlessSOlneadditionalstructureis imposeduponthem.1rhesimplestpossiblestructureisthatof independentrandomvariables. This wouldbeagoodmodelforsuchsystemsasrepeatedexperimLentsin whichfuture statesof thesystemareindependentof pastandpresentstates. Inmostsystems that arise in practice,however,pastand presentstatesof thesysteminfuence the future states even if they do not uniquely determine them. vanysystemshave the propertythatgiventhe present state,thepaststateshave noinfuenceonthefuture.ThispropertyiscalledtheMarkovproperty,and systemshavingthispropertyar(calledMarkovchains.TheMarkovpropertyis definedpreciselyby therequirenlentthat for every choiceof thenonnegativeinteger1andthenumbers Xo,. . . ,xn+ 1,each in C.The conditional probabilities P(Xn+ 1=YIXn=x) arc called the transition probabilitiesofthechain.InthisbookwewillstudyMarkovchainshaving stationarytransitionprobabilities,i . e. , thosesuchthatP(Xn + 1=YIXn=x)is independentof n.Fromnow011,whenwesaythatXmn&gt;0,formsaMarkov chain, we mean that these rando]t variablessatisfy theMarkov property andhave sta1tionary transition probabilities. lrhestudyofsuchMarkovchainsisworthwhilefromtwoviewpoints.First, theyhavearichtheory,muchof whichcanbepresentedatanelementarylevel . Secondly,therearealargenurnberofsystemsarisinginpracticethatcanbe modeledbyMarkovchains,sothesubjecthasmanyusefulapplications. 1 Z 8rK0v lR8D8In orderto he|pmotivatethe genera|resu|tsthatwi||bediscussed|ater,webeginbyconsideringMarkovchainshavingon|ytwo states.1,, 1.Markov chai n!;havi ng twostate!; Foranexamp|eofa1arkovchainhavingtvostates,consideramachinethatatthestartofanyarticu|ardayiseitherbrokendownorinoperatingcondition.Assumethat ifthemachineis brokendown atthestartofthenth day, the probabi|ity is that it wi|| be successfu||y repaired and inoperatingconditionat thestart ofthe(n + l )thday. Assumea|sothatifthe machine is in oerating condition at the start of the nth day, theprobabi|ity is q that it wi|| have a fai|ure causing it to be broken downatthestart ofthe( + l)thday. Fina||y,|eta(0) denotetheprobabi|itythatthemachine is brokendowninitia||y,i . e, at thestartofthe0thday.Let the state 0 correspond to the machine being broken down and |etthe state l correspond to the machine being in operatingcondition. LetX"be the random variab|e denoting the state ofthe machine at time M.Accordingto the abovedescriptionand!'(X,,1 =l IX, =0) =,!'(X,,1 =0 IX, =l ) =q,P(X =0) =aa(0).Sincethere are on|y two states, 0 and l , itfohowsimmediate|ythatP(,,1 =0 IX, =0) "' l - ,P(,,1=l IX, =l) "' l - q,andthattheprobabi|itya( l ) ofbeinginitia|ly instate l isgvenbya( l ) =P(X =l ) =l - a(0).Fromthisinformation,wecaneasi|ycomputeP(X, =0)andP(X, =l). W 'e observethat'(X,,1 =0) =P(X, `" 0 andX,,1 =0) -- P(X, " l andX,,1 =0)=P(X, '" 0)P(X,,</p> <p>, =0 IX, =0)+ P(\, =l)P(X,,1 =0IX, =l)=(| -)(X, =0) + }P(X,=l)=( l- )P(X, =0) + q(l - P(X, =0)=(l - lJ- q)P(X, =0) + q. .. 8rk`0v 08D8 8vD tW0 8t8t08Now!( =0) =a(0),so!(,:=0) =(I - - q)::(0) + qand!(,=0) =(I - - q)!(, =0) + q=(I - - q),a(0) + q[I + (I - p- q)|. Itiseasi|yseenbyrepeatingthisprocedurentimesthat,,(2) l(, =c)=- - ),a(0) + - - )|.i= OInthetrivia|case =q =0,itisc|earthatfora||n !(, =0) =a(0)and!(', =I) =a(J).dSuppose now that p+ q &gt;0. Then by the formu|a for the sum of ahnitegeometricprogression,f </p> <p>)|= </p> <p> - - )i= O + W:concludefrom (2) that(1) l(X, =0) =_L+ - - ),a(0) -, ,+ + andconsequent|ythat(4) !X, = </p> <p>.=_L+ - - ),a(!) -, . + + Supposethatandqareneitherboth equa|tozeronorbothequa|toI. Then0 0, is ca||ed a eaoe/k. It is a Markov chain whose state space is the integers and uhosetransitionfunctionisgivenby!(,))}() - x)Toverifythis,|etrdenotethedistribution ofX. Then!(x x,, .. ., x, x,) !(. ,, , , - , . ., , , - x,.,J !( ,)I(, , - x) !(, =x,- x,,) r(0,isaMarkovchainonV (c, l , 2,. 4 .} .88rK0v 8D8The transition function of this Markov chain is easi|y computed.Suppose thatthere are J ba||s in box l at time Thenwithprobabi|ityxjdthe ba|| drawn onthe (a + l )th tria| wi|| be from box l and wi|| betransferred to box 2. In this case there wi|| be x - A ba||s in box l attime a + l .Simi|ar|y, with probabi|ity (d - x)jdthe ba|| drawn on the( +l )thtria|wi||befrombox2andwi||betransferredtobox l ,resu|tinginx + l ba||sinboxl attime + l . ThusthetransitionfunctioncfthisMarkovchainis givenbyx a')" x- I,I(x,))"Ix -_)* .+ I, 0, e|sewhere.Note that the Ehrenfest chain canin onetransition on|ygo fromstatexto x - l orx + l with positiveprobabi|ity.A stateeofa Markovchain isca||edaneosoo|pseeifI(e,e) =lor, equiva|ent|y, if I(e,y) " 0 for y ie The next examp|e uses thisdehnition.Exampl e 3.Gambl er'srui nchai n. Suppose a gamb|er starts outwith a certain initia| capita| indo||ars and makes a series ofone do||arbetsagainstthehouse. Assumethat he hasrespectiverobabi|itiespand l - pofwinning and |osing each bet, and that ifhis capita| everreaches zero, he is ruin:d and his capita| reinains zero thereafter.LetX,,a &gt;0, denotethegamb|er'scapita| attime Thisis a Markovchaininwhich 0isan absorbingstate, andforx &gt;l(l 5) ,I(x,y) =p, 0,y =x - A, y =x + l ,e|sewhere.Such a chain is ca||ed apeo/e`s|c/e| on V " (c, l , 2,. .} .Wecan modify this mode| by supposing that ifthe capita| ofthe garab|erincreases to d do||ars h: quits p|aying. In this case 0 and d are bothabsorbingstates, and( l 5) ho|dsforx =l , . . ., d - l . Forana|ternativeinteqretationofthe|atterchain,wecanassumethattwogamb|ersaremakingaseriesofonedo||arbetsagainsteachotherandthatbetween themtheyhaveatota|capita| ofd do||ars. Supposethehrstgatnb|erhasprobabi|itypofwinninganygivenbet,andthesecondgamb|erhasprobabi|ity =l - pofwinning. Thetwo gamb|ers |ay unti| one-d- LXB'Ql88 8ofthemgoes broke. Let,denotethe capita| ofthe hrst gamb|er attimen. Then_ n &gt;0, isa gamb|er'sruin chain on {0, l ,. . .,} .Exampl e 4. Bi rthanddeathchai n. ConsideraMarkovchaineitheron V ={0, l , 2,...}or on V ={0, l ,... ,ti}suchthatstartingfromx th: chain wi|| be at xl , x, or x + l after one step. The transitionfunction ofsuchachainisgivenby|(x,y) = ;,, y =x -I,y x, y x + I,e|sewhere,wherer,,,,and,arenonnegativenumberssuchthat), + , + ,=l . The Ehrenfest chain and the two versions of the gamb|er's ruin chainan exam|es ofo|// oaeo/ c/e|as The phrase "birth and death"stemsfromapp|ic...</p>