Random Processes - Stochastic Processes... · Random Processes Dr. Ali Muqaibel Dr. Ali Hussein Muqaibel…

Download Random Processes - Stochastic Processes... · Random Processes Dr. Ali Muqaibel Dr. Ali Hussein Muqaibel…

Post on 11-Jul-2018

212 views

Category:

Documents

0 download

Embed Size (px)

TRANSCRIPT

<ul><li><p>3/31/2013</p><p>1</p><p>RandomProcesses</p><p>Dr.AliMuqaibel</p><p>Dr.AliHusseinMuqaibel 1</p><p>Introduction Reallife:time(space)waveform(desired+undesired) Ourprogressanddevelopmentrelaysonourabilitytodealwithsuchwave</p><p>forms. Thesetofallthefunctionsthatareavailable(orthemenu)iscallthe</p><p>ensembleoftherandomprocess.</p><p>Dr.AliHusseinMuqaibel</p><p>Thegraphofthefunction , ,versus tfor fixed,iscalledarealization,Samplepath,orsamplefunction oftherandomprocess.</p><p>Foreachfixedfromtheindexedset, , isarandomvariable</p><p>2</p></li><li><p>3/31/2013</p><p>2</p><p>FormalDefinition Considerarandomexperimentspecifiedbytheoutcomes fromsome</p><p>samplespace ,andbytheprobabilitiesontheseevents. Supposethattoeveryoutcome ,weassignafunctionoftime</p><p>accordingto somerule: , , . Wehavecreatedanindexedfamilyofrandomvariables, , , . Thisfamilyiscalledarandomprocess (stochasticprocesses). Weusuallysuppressthe anduse todenote arandomprocess. Astochasticprocessissaidtobediscretetime iftheindexset isa</p><p>countable set(i.e.,thesetofintegersorthesetofnonnegativeintegers). </p><p> Acontinuoustime stochasticprocessisonewhich is continuous(thermalnoise)</p><p>Dr.AliHusseinMuqaibel 3</p><p>DeterministicandnondeterministicProcesses</p><p> Nondeterministic:futurevaluescannot bepredictedfromcurrentones. mostoftherandomprocessesarenondeterministic.</p><p> Deterministic: like:</p><p>( , ) cos(2 ) X t s s t t </p><p>( , ) cos(2 )Y t s t s </p><p>Sinusoidwithrandomamplitude[1,1]</p><p>Sinusoidwithrandomphase ,Dr.AliHusseinMuqaibel 4</p></li><li><p>3/31/2013</p><p>3</p><p>DistributionandDensityFunctions Ar.v.isfullycharacterizedbyapdf orCDF.Howdowecharacterize</p><p>randomprocesses? Tofullydefinearandomprocesses,weneed dimensionaljoint</p><p>densityfunction. DistributionandDensityFunctions Firstorder:</p><p> ; Secondorder jointdistributionfunction</p><p> , ; , , Nth orderjointdistributionfunction , , ; , , , . , , , ; , , , , ; , ,</p><p>Dr.AliHusseinMuqaibel 5</p><p>StationaryandIndependence StatisticalIndependence , , , , , ; , , , , , =, , , ; , , , , ; , , Stationary</p><p> Ifallstatisticalpropertiesdonotchangewithtime FirstorderStationaryProcess ; ; ,stationarytoorderone =&gt; Proof</p><p> , ; ; Let </p><p>Dr.AliHusseinMuqaibel 6</p></li><li><p>3/31/2013</p><p>4</p><p>Cyclostationary</p><p>Dr.AliHusseinMuqaibel</p><p>).,...,,( </p><p>),...,,(</p><p>21)(),...,(),(</p><p>21)(),..,(),(</p><p>21</p><p>21</p><p>kmTtXmTtXmTtX</p><p>ktXtXtX</p><p>xxxF</p><p>xxxF</p><p>k</p><p>k</p><p>Wesaythat iswidesensecyclostationary ifthemeanandautocovariancefunctionsareinvariantwithrespecttoshiftsinthetimeoriginbyintegermultiplesof ,thatis,foreveryinteger .</p><p>).,(),()()(</p><p>2121 ttCmTtmTtCtmmTtm</p><p>XX</p><p>XX</p><p>Notethatif iscyclostationary,theniffollowsthat isalsowidesensecyclostationary.</p><p>Adiscretetimeorcontinuoustimerandomprocess issaidtobecyclostationary ifthejointcumulativedistributionfunctionofanysetofsamplesisinvariantwithrespecttoshiftsoftheoriginbyintegermultiplesofsomeperiod</p><p>Forall , andallchoicesofsamplingtimes , , </p><p>7</p></li><li><p>3/31/2013</p><p>1</p><p>CrossCorrelationFunctionanditsproperties</p><p> , IfXandYarejointlyw.s.s.wemaywrite</p><p>. Orthogonalprocesses , 0 IfXandYarestatisticallyindependent =</p><p> Ifinadditiontobeingindependenttheyareatleastw.s.s.</p><p>Dr.AliHusseinMuqaibel 8</p><p>Somepropertiesfor</p><p> 0 0 0 0 Thegeometricmeanistighterthanthearithmeticmean</p><p> 0 0 0 0</p><p>Dr.AliHusseinMuqaibel 9</p></li><li><p>3/31/2013</p><p>2</p><p>MeasurementofCorrelationFunction</p><p> Inreallife,wecannevermeasurethetruecorrelation. Weassumeergodicity anduseportionoftheavailabletime. Assumeergodicity,noneedtoprovemathematicallyphysicalsense Assumejointlyergodic =&gt;stationary Let 0, 2 Similarly,wemayfind &amp;</p><p>Delay</p><p>Delay</p><p>Product12 . 2</p><p>2 12</p><p>Dr.AliHusseinMuqaibel 10</p><p>Example Usetheabovesystemtomeasurethe for . 2 cos cos cos cos 2 2 where cos cos 2 Ifwerequirethe tobeatleast20timeslessthanthelargestvalueof</p><p>thetrueautocorrelation 0.05 0 0.05 Waitenoughtime!Dependingonthefrequency</p><p>Dr.AliHusseinMuqaibel 11</p></li><li><p>3/31/2013</p><p>3</p><p>Matlab:Measuringthecorrelation</p><p> %Dr.AliMuqaibel %MeasurementofCorrelationfunction clearall closeall clc T=100; A=1; omeg=0.2; t=T:T; thet=2*pi*rand(1,1); X=A*cos(omeg*t+thet); [R,tau]=xcorr(X,'unbiased'); %R=R/(2*T); True_R=A^2/2*cos(omeg*tau); Err=A^2/2*cos(omeg*tau+2*thet)*sin(2*omeg*T)/(2*omeg*T); subplot(3,1,1) plot(tau,True_R,tau,R+Err,':') title('TrueVariance') subplot(3,1,2) plot(tau,R,':') title('Measured') %error subplot(3,1,3) plot(tau,Err) title('Error') %error</p><p>-100 -80 -60 -40 -20 0 20 40 60 80 100-0.5</p><p>0</p><p>0.5True Variance</p><p>-100 -80 -60 -40 -20 0 20 40 60 80 100-0.5</p><p>0</p><p>0.5Measured</p><p>-100 -80 -60 -40 -20 0 20 40 60 80 100-0.05</p><p>0</p><p>0.05Error</p><p>-40 -30 -20 -10 0 10 20 30 40-1</p><p>0</p><p>1True Variance</p><p>-40 -30 -20 -10 0 10 20 30 40-1</p><p>0</p><p>1Measured</p><p>-40 -30 -20 -10 0 10 20 30 40-0.1</p><p>0</p><p>0.1Error</p><p>T=50,OMEGA=0.2</p><p>T=20,OMEGA=0.2</p><p>Notetheerrorislessthan5%</p><p>Dr.AliHusseinMuqaibel 12</p><p>Arandomprocess isaGaussianrandomprocess ifthesamples, , </p><p>arejointlyGaussian randomvariablesforall ,andallchoicesof , , .Thisdefinitionappliesfordiscretetimeandcontinuoustimeprocesses.Thejointpdf ofjointlyGaussianrandomvariablesisdeterminedbythevectorofmeansandbythe covariancematrix:</p><p>212/</p><p>)()(21</p><p>1,.....,,)2(</p><p>),....,(1</p><p>21C</p><p>exxfk</p><p>mXCmX</p><p>kXXX</p><p>T</p><p>k</p><p>where</p><p>)(...</p><p>)(</p><p>m</p><p>1</p><p>kX</p><p>X</p><p>tm</p><p>tm</p><p>)(...)(</p><p>....</p><p>...),(...),(),()(...),(),(</p><p>,1,</p><p>22212</p><p>,12111</p><p>kkXkX</p><p>kXXX</p><p>kXXX</p><p>ttCttC</p><p>ttCttCttCttCttCttC</p><p>C</p><p>Gaussian Random Processes</p><p>Dr.AliHusseinMuqaibel 13</p></li><li><p>3/31/2013</p><p>4</p><p>Letthediscretetimerandomprocess beasequenceofindependentGaussianrandomvariableswithmean andvarianceThecovariancematrixforthetimes , , is</p><p>,}{)},({ 221 IttC ijjX </p><p>where 1 when and0otherwise,and istheidentitymatrix.Thusthecorrespondingjointpdf is</p><p>)()...()( </p><p>2/)(exp)2(</p><p>1),.....,,(</p><p>21</p><p>1</p><p>22</p><p>2221,.....1</p><p>kXXX</p><p>k</p><p>iikkXX</p><p>xfxfxf</p><p>mxxxxfk</p><p>Example iid GaussianSequence</p><p>Dr.AliHusseinMuqaibel 14</p><p>ExampleofaGaussianRandomProcess</p><p> AGaussianRandomProcesswhichisW.S.S. 4 and25 | | 16</p><p> Specifythejointdensityfunctionforthreer.v. , 1,2,3 ,, </p><p> , 1,2,3, 25 16 25 16 4 </p><p> 251</p><p>11</p><p>Dr.AliHusseinMuqaibel 15</p></li><li><p>3/31/2013</p><p>5</p><p>ComplexRandomProcesses</p><p> Acomplexrandomprocess isgivenby , ,</p><p> Notetheconjugate</p><p> Therecouldbeafactorof insomebooks SeeexampleinPeebles</p><p>Dr.AliHusseinMuqaibel 16</p><p>Supposeweobserveaprocess ,whichconsistsofadesiredsignalplusnoise .</p><p>Findthecrosscorrelationbetweentheobservedsignalandthedesiredsignal assumingthat and areindependentrandomprocesses.</p><p>, 1 2 1</p><p>1</p><p>1 1</p><p>1 2 1 2</p><p>1 2</p><p>2</p><p>2 2</p><p>2 2</p><p>1 2</p><p>( , ) [ ( ) ] [ ( ) ] [ ( ) ] [ ( ) ] </p><p>( ){</p><p> ( , ) [ ( )] [ ( )] ( , ) ( ) ( )</p><p>( ) ( )}( ) ( )</p><p>X Y</p><p>XX</p><p>XX X N</p><p>R t t E X t Y tX t N t</p><p>X tE X tE X t E X tR t t E X t E N tR t t m</p><p>N t</p><p>t m t</p><p>wherethethirdequalityfollowedfromthefactthat andareindependent.</p><p>Example SignalPlusNoise</p><p>Dr.AliHusseinMuqaibel 17</p></li><li><p>3/31/2013</p><p>6</p><p>Dr.AliHusseinMuqaibel 18</p><p>See LeonGarciaProbability,Statistics,andRandomProcessesforElectricalEngineers,3rd Edition</p><p>9.5GAUSSIANRANDOMPROCESSES,WIENERPROCESS,ANDBROWNIANMOTION</p><p>EXAMPLES OF DISCRETE_TIME &amp; Continuous-Time RANDOM PROCESSES</p><p>iid Random Processes</p><p>Let beadiscretetimerandomprocessconsistingofasequenceofindependent,identicallydistributed(iid)randomvariableswithcommoncdfmean andvariance . Thesequence iscalledtheiid randomprocess.Thejointcdf foranytimeinstants , . , isgivenby</p><p>),()...()( </p><p>),...,,[),....,,(</p><p>21</p><p>221121,...1</p><p>kXXX</p><p>kkkXX</p><p>xFxFxFxXxXxXPxxxF</p><p>K</p><p>whereforsimplicity denotes .Theequationaboveimpliesthatif isdiscretevalues,thejointpmf factorsintotheproductofindividualpmfs,andif iscontinuousvalued,thejointpdf factorsintotheproductoftheindividualpdfs.</p><p>Themeanofaniid process isobtained</p><p>Thus,themeanisconstant.Theautocovariance functionisobtainedfromasfollows.If ,then</p><p>nmXEnm nX allfor ][)( </p><p>EXAMPLES OF DISCRETE_TIME RANDOM PROCESSES</p><p>Dr.AliHusseinMuqaibel 19</p></li><li><p>3/31/2013</p><p>7</p><p>1 2</p><p>1 2</p><p>1 2( , ) [( )( )]</p><p> [( )] [( )] 0X n n</p><p>n n</p><p>C n n E X m X m</p><p>E X m E X m</p><p>since and areindependentrandomvariables.If then</p><p> ])[(),( 2221 mXEnnC nX</p><p>Wecanexpresstheautocovariance oftheiid processincompactformasfollows:</p><p>,),(21</p><p>221 nnX nnC </p><p>whereif and0otherwiseTheautocorrelationfunctionoftheiid processis:</p><p>121nn</p><p>22121 ),(),( mnnCnnR XX </p><p>Dr.AliHusseinMuqaibel 20</p><p>Let beasequenceofindependentBernoullirandomvariables. isthenaniid randomprocesstakingonvaluesfromtheset{0,1}.ArealizationofsuchaprocessisshowninFigure.Forexample, couldbeanindicatorfunctionfortheeventalightbulbfailsandisreplacedondayn.Since isaBernoullirandomvariable,ithasmeanandvariance</p><p>1Theindependenceofthe makesprobabilitieseasytocompute.Forexample,theprobabilitythatthefirst4bitsinthesequenceare1001is</p><p>1, 0, 0, 1 1 0 0 11</p><p>Similarly,theprobabilitythatthesecondbitis0andtheseventhis1is0, 1 0 1 1</p><p>Realization of a Bernoulli process. 1 indicatesthat a light bulb fails and is replaced in day . </p><p>Example : BernoulliRandomProcess</p><p>Dr.AliHusseinMuqaibel 21</p></li><li><p>3/31/2013</p><p>8</p><p>(b)Realizationofabinomialprocess. denotesthenumberoflightbulbsthathavefaileduptotimen.</p><p>Manyinterestingrandomprocessesareobtainedasthesumofasequenceofiid randomvariables, , , .</p><p> . , 1,2, </p><p>Thesumprocess 0 ,canbegeneratedinthisway.</p><p>SumProcesses:TheBinomialCountingandRandomWalkProcesses</p><p>where 0.Wecall thesumprocess.Thepdf orpmf of isfoundusingtheconvolution.Notethat dependsonthepast, , , onlythrough ,thatis,isindependentofthepastwhen isknown.ThiscanbeseenclearlyfromthepreviousFigure,whichshowsarecursiveprocedureforcomputing .Thus isaMarkovprocess.</p><p>Dr.AliHusseinMuqaibel 22</p><p>Letthe bethesequenceofindependentBernoullirandomvariablesina previousExample,andlet bethecorrespondingsumprocess. isthenthecountingprocess thatgivesthenumberofsuccessesinthefirst Bernoullitrials.Thesamplefunctionfor correspondingtoaparticularsequenceof</p><p>isshownintheFigureup.If indicatesthatalightbulbfailsandisreplacedondayn,then denotesthenumberoflightbulbsthathavefaileduptoday .</p><p>Since isthesumof independentBernoullirandomvariables, isabinomialrandomvariablewithparametersn and 1</p><p>,0for )1(][ njppjn</p><p>jSP jnjn </p><p>andzerootherwise.Thus hasmean andvariance 1 .Notethatthemeanandvarianceofthisprocessgrowlinearlywithtime( ).</p><p>Example BinomialCounting Process</p><p>Dr.AliHusseinMuqaibel 23</p></li><li><p>3/31/2013</p><p>9</p><p>Let betheiid processof 1 randomvariableasinthepreviousexample,andlet bethecorrespondingsumprocess. isthenthepositionoftheparticleattimen.Therandomprocess isanexampleofaonedimensionalrandomwalk.Asamplefunctionof isshownintheFigureThepmf of isfoundasfollows.Ifthereare " 1" inthefirst trials,thenthereare " 1" and 2 .Conversely, ifthenumberof 1" is .If isnotaninteger,then cannotequal .Thus },...,1,0{for )1(]2[ nkppk</p><p>nnkSP knkn </p><p>Example OneDimensionalRandomWalk</p><p>Dr.AliHusseinMuqaibel 24</p><p>n</p><p>Jnj(nj)/2(nj)/2</p><p>Dr.AliHusseinMuqaibel</p><p>Example Sumofiid GaussianSequenceLet beasequenceofiid Gaussianrandomvariableswithzeromeanandvariance .Findthejointpdf ofthecorrespondingsumprocessattimes and .Thesumprocess isalsoaGaussianrandomprocesswithmeanzeroandvariance .Thejointpdf of attimes and isgivenby</p><p>21</p><p>21</p><p>]2)12(2/[212</p><p>11221</p><p>2/</p><p>21</p><p>)(</p><p>212</p><p>11221,</p><p>21</p><p>)(21 </p><p>)()(),(</p><p> nyyy</p><p>SSSS</p><p>en</p><p>enn</p><p>yfyyfyyfnn</p><p>nnnnn</p><p>25</p></li><li><p>3/31/2013</p><p>10</p><p>Poisson Process</p><p>Considerasituationinwhicheventsoccuratrandominstantsoftimeatanaveragerateofacustomertoaservicestationorthebreakdownofacomponentinsomesystem.Let bethenumberofeventoccurrencesinthetimeinterval[0,t]. isthenanondecreasing,integervalued,continuoustimerandomprocessasshowninFigure.</p><p>AsamplepathofthePoissoncountingprocess.Theeventoccurrencetimesaredenotedby, , . .Thej th interevent time</p><p>isdenotedby</p><p>EXAMPLES OF CONTINUOUS-TIME RANDOM PROCESSES</p><p>Dr.AliHusseinMuqaibel 26</p><p>Iftheprobabilityofaneventoccurrenceineachsubintervalisp,thenthe expectednumberofeventoccurrencesintheinterval[0,t]isnp.Sinceeventsoccuratarateofeventspersecond,theaveragenumberofeventsinthe interval[0,t]isalso.Thuswemusthavethat</p><p>Ifwenowlet . . , 0 and 0 while remainsfixed,thenthebinomialdistributionapproachesaPoissondistributionwith parameter .Wethereforeconcludethatthenumberofeventoccurrences intheinterval0, hasaPoissondistributionwithmean :</p><p>! , 0,1, Forthisreason iscalledthePoissonprocess.</p><p>PoissonProcess..FromBinomial</p><p>Dr.AliHusseinMuqaibel 27</p><p>,0for )1(][ njppjn</p><p>jSP jnjn </p><p>Fordetailedderivation,pleaseseehttp://www.vosesoftware.com/ModelRiskHelp/index.htm#Probability_theory_and_statistics/Stochastic_processes/Deriving_the_Poisson_distribution_from_the_Binomial.htm</p><p>Replacepwith /</p></li><li><p>3/31/2013</p><p>11</p><p>PoissonRandomProcess AlsoknownasPoissonCountingProcess Arrivalofcustomers,failureofparts,lightning,.internett&gt;0 Twoconditions:</p><p> Eventsdonotcoincide. #ofoccurrenceinanygiventimeintervalisindependentofthenumberinanynonoverlapping</p><p>timeinterval.(independentincrements) Averagerateofoccurrence= . ! , 0,1,2, 0, ! 1 Theprobabilitydistributionofthewaitingtimeuntilthenextoccurrenceisan</p><p>exponentialdistribution. Theoccurrencesaredistributeduniformly onanyintervaloftime.</p><p>Dr.AliHusseinMuqaibelhttp://en.wikipedia.org/wiki/Poisson_process</p><p>28</p><p>JointprobabilitydensityfunctionforPoissonRandomProcess</p><p> Thejointprobabilitydensityfunctionforthepoisonprocessattimes0 ! , 0,1,2, Theprobabilityof occurrenceover 0, giventhat eventsoccurredover0, ,isjusttheprobabilitythat eventsoccurredover , , whichis ! For ,thejointprobabilityisgivenby , ] ! ! Thejointdensitybecomes , , Example:demonstratethehigherdimensionalpdf</p><p>Dr.AliHusseinMuqaibel 29</p></li><li><p>3/31/2013</p><p>12</p><p>Thearrivalrateinsecondsis inquiriespersecond.Writingtimeinseconds,theprobabilityofinterestis</p><p>10 3 60 45 2Byapplyingfirsttheindependentincrementsproperty,andthenthestationaryincrementsproperty,weobtain</p><p>!2)415(</p><p>!3)410( </p><p>]2)4560([]3)10([ ]2)45()60([]3)10([ </p><p>]2)45()60( and 3)10([</p><p>41524103 </p><p>eeNPNP</p><p>NNPNPNNNP</p><p>InquiriesarriveatarecordedmessagedeviceaccordingtoaPoissonprocessofrate15inquiriesperminute.Findtheprobabilitythatina1minuteperiod,3inquiriesarriveduringthefirst10secondsand2inquiriesarriveduringthe last15seconds.</p><p>ExampleI</p><p>Dr.AliHusseinMuqaibel 30</p><p>Dr.AliHusseinMuqaibel</p><p>FindthemeanandvarianceofthetimeuntilthearrivalofthetenthinquiryinthepreviousExample. Thearrivalrateis 1/4 inquiriespersecond,sotheinterarrivaltimesareexponentialrandomvariableswithparameter .</p><p>FromTables,themeanandvarianceofaninterarrivaltimearethen1/ and1/ ,respectively.Thetimeofthetentharrivalisthesumoftensuchiid randomvariables,thus</p><p>sec4010][10][ 10 TESE</p><p>2210 sec160</p><p>10VAR[T]10][VAR </p><p>S</p><p>31</p><p>ExampleII</p></li><li><p>3/31/2013</p><p>13</p><p>Considerarandomprocess thatassumesthevalues 1 .Supposethat0 1 withprobability andsupposethat thenchangespolarity</p><p>witheachoccurrenceofaneventinaPoissonprocessofrate .Thenextfigureshowsasamplefunctionof .</p><p>Thepmf of isgivenby</p><p> ].1)0([]1)0(1)( </p><p>]1)0([]1)0(1 )([]1)([</p><p>XPXtP[X </p><p>XPXtXPtXP</p><p>Example RandomTelegraphSignal</p><p>Samplepathofarandomtelegraphsignal.Thetimesbetweentransitions areiidexponentialrandomvariables.</p><p>Dr.AliHusseinMuqaibel 32</p><p>Theconditionalpmfs arefoundbynotingthat willhavethesamepolarityas 0 onlywhenanevennumberofeventsoccurintheinterval 0, .Thus</p><p>}1{21 </p><p>}{21 </p><p>)!2()( </p><p>integer]even )([]1)0(1)([</p><p>2</p><p>0</p><p>2</p><p>t</p><p>ttt</p><p>j</p><p>tj</p><p>e</p><p>eee</p><p>ej</p><p>t</p><p>tNPXtXP</p><p>Dr.AliHusseinMuqaibel 33</p><p>andX(0)willdifferinsignifthenumberofeventsintisodd:</p><p>}.1{21 </p><p>}{21...</p></li></ul>

Recommended

View more >