markovprocessesmeng

8/4/2019 MarkovProcessesMeng

http://slidepdf.com/reader/full/markovprocessesmeng 1/29



OutlinesMarkov models

/process/chain/property/HMM

Matlab simulations

Applications

Advantages and LimitationsConclusions.



Markov ProcessMarkov process is a simple stochasticprocess in which the distribution of future states depends only on thepresent state and not on how it arrivedin the present state.



Markov Models

– A finite state representation



Markov PropertyMany systems in real world have theproperty that given present state, thepast states have no influence on thefuture. This property is called Markov

property.



State Space and Time SpaceState Space

Time Space Discrete Continuous

Discrete

(Markov

chain) X Continuous

X X



Markov ChainLet {X t : t is in T} be a stochastic process withdiscrete-state space S and discrete-time space

T satisfying Markov property

P( X n+1 = j|X n = i, X n-1 = in-1 , · · ·,X 0 = i0) = P( X

n+1

= j|X n

= i) for any set of state i0 , i1 , · · · , in-1 , i, j in S andn ≥ 0 is called a Markov Chain.



Hidden Markov Models (HMM)In an hidden Markov Model(HMM), we don’tknow the state sequence. However we know

some probabilistic function of it.In plain English, Markov model can be viewedas a probabilistic finite state engine. Thestate is changing over time. But we have no

way to determine the exact changes of thestate. We are able to observe some fuzzyreflections of the change. Our objective is toestimate the states of the machine via the

(possibly fuzzy) observations.



Markov Model exampleThe weather in Dallas of past 26 daysSTATES = { pretty, pretty, pretty, pretty, pretty, pretty,

pretty, pretty, pretty, pretty, rainy, rainy, rainy, rainy, rainy,pretty, pretty, pretty, pretty, pretty, pretty, pretty, pretty,pretty, pretty, pretty }



Markov Model exampleThe weather in Dallas of past 26 daysSTATES = { pretty, pretty, pretty, pretty, pretty, pretty, pretty,

pretty, pretty, pretty, rainy, rainy, rainy, rainy, rainy, pretty,pretty, pretty, pretty, pretty, pretty, pretty, pretty, pretty,pretty, pretty }

Tomorrow’s weather Today’s

WeatherPretty Rainy

Pretty 0.95 0.05

Rainy 0.2 0.8



Markov Model example (2)

Pretty

Rainy

0.95

0.8

0.05

0.2



Markov Model example(3)Hidden model.

Many time we don’t have a direct observation of the

change of the states. Therefore we say the model ishidden. However we are still able to observe an emissionof the state changes, and the emission could be fuzzy.

For example, you are isolated in a closed room during theexperiment. In the room, you have no direct observation

of how weather changes. Everyday, an assistant deliversmeals for you once a day. The only way you have clue of the weather is to observe the how the guy’s dress

changes.



Markov Model example(4)In the room, you might observe that yourassistant dresses with regular coat (CT), rain

coat(RN), or brings an umbrella (UM). Yourobservation sequence may be

SEQ = {CT, CT , CT , UM, CT , CT , CT , CT , CT , CT

, CT , RN , CT , RN , UM , CT , CT, CT, RN, CT, CT,CT , CT, CT, CT , CT}



Markov Model example(5)Regular Coat Rain Coat Umbrella

Pretty 5/6 1/12 1/12

Rainy 1/3 1/3 1/3



Mathematical ElementsA set of states over time, denoted by STATES

A set of emissions , or observations over time,

denoted by SEQ

An M -by-M transition matrix TRANS whoseentry(i,j) is the probability of a transition from

state i to state j.An M -by-N emission matrix EMIS whose i,kentry gives the probability of emitting symbols k given that the model is in state i. EMIS





Matlab simulations

Applications

Advantages and Limitations

Conclusions.



Why Matlab?Matlab is a tool for doing numericalcomputations with matrices andvectors. It can also display informationgraphically.

Combined with numerous mathematical

libraries, Matlab has become one of thefew tools that can catch up with myideas.



Why Matlab?

MATLAB Compiler translates MATLAB code to ANSI standard C code. With the MATLABCompiler, we will be able to automaticallygenerate optimized C and C++ code. Bytranslating MATLAB code to C and C++, the

compiler can significantly speed up MATLABapplications and development.>> mcc -t -L C myfun1 % yields myfun1.c

mcc -t -L C myfun2 % yields myfun2.c

mcc -W main -L C myfun1 myfun2 libmmfile.mlib

% yields myfun1_main.c



Matlab simulationsMatlab Statistics Toolbox 4.1

(Released in May 2003)hmmdecode

hmmgenerate

hmmestimatehmmtrain

hmmviterbi



Matlab simulations

Matlab scripts demo.m in Matlab6.5 environment





Matlab simulations

Applications


Conclusions.



Applications of Markov ModelsSpeech recognition,

Modeling of coding/noncoding regions in

DNA, Protein binding sites in DNA,

Protein folding,

Protein superfamilies,

Multiple sequence alignment,

Flood predictions,

Ion channel recordings,

Optical character recognition.





Matlab simulations

Applications


Conclusions.



Advantages of Markov ModelsMMs and HMMs have proved effective in a number of domains.

The basic theory of HMMs is very elegant and easy tounderstand. This makes it easier to analyse and

implement, with the help of Matlab. Because MM uses only positive data, their scalabilityis very good. Dr. Dunham’s research group is

investigating an incremental extension algorithm of Markov chain, which fits for dynamic data processing.

It is complementary of other non-linear models suchas neural networks and time-series analysis.



Limitations of Markov ModelsIt is a data hog.

Markov property.



Conclusions

http://www-2.cs.cmu.edu/~awm/tutorials/








Any Questions?

markovprocessesmeng

Documents