inference with constrained hidden markov models in prism

Inference withConstrained Hidden Markov Models

in PRISM

Henning Christiansen Christian Theil HaveOle Torp Lassen Matthieu Petit

Research group PLIS: Programming, Logic and Intelligent SystemsDepartment of Communication, Business and Information Technologies

Roskilde University, P.O.Box 260, DK-4000 Roskilde, Denmark

ICLP 2010 in Edinburgh, July 16, 2010

Henning Christiansen, Christian Theil Have, Ole Torp Lassen, Matthieu Petit Inference with CHMMs in PRISM

Outline

1 Motivation and backgroundHidden Markov ModelsBiological sequence alignment with pair HMMs

2 Constrained HMMsCHMMs as a constraint programDecoding algorithm for CHMMsImplementation in PRISM

3 Summary and future directions


Motivation and background

Motivation

In the LoSt project we explore the use of Probabilistic Logicprogramming (with PRISM) for biological sequence analysis

Hidden Markov Models are one of the most widely used models forsequence analysis

Constrained Hidden Markov Models (CHMMs) allow us to expressrelevant prior knowledge and prune the search space


Motivation and background

Contributions and similar work

Contributions

Expressing CHMMs as a constraint problem

Adaptation of Viterbi decoding algorithm for CHMMs

Efficient implementation of CHMMs in PRISM

Similar work

In [SK08] introduces the concept of CHMMs

[CPC08] express relationships in Bayesian Network as a CLP

Constrained Conditional Models [CRR08] augments (discriminative)models with declarative constraints


Motivation and background Hidden Markov Models

Hidden Markov Models

Definition

A Hidden Markov Model (HMM) is a 4-tuple 〈S ,A,T ,E 〉, where

S = {s0, s1, . . . , sm} is a set of states which includes an initial statereferred to as s0;

A = {e1, e2, . . . , ek} is a finite set of emission symbols;

T = {(p(s0; s1), . . . , p(s0; sm)), . . . , (p(sm; s1), . . . , p(sm; sm))} is a setof transition probability distributions representing probabilities totransit from one state to another;

E = {(p(s1; e1), . . . , p(s1; ek)), . . . , (p(sm; e1), . . . , p(sm; ek))} is a setof emission probability distributions representing probabilities to emiteach symbol from one state.



Hidden Markov Model, graphical example

Postcard

Greetings from wherever, where I am having agreat time. Here is what I have been doing: Thefirst two days, I stayed at the hotel reading agood book. Then, on the third day I decided togo shopping. The next three days I did nothingbut lie on the beach. On my last day, I wentshopping for some gifts to bring home and wroteyou this postcard.

Sincerely, Some friend of yours

Observation sequence



Hidden Markov Model run

Definition

A run of an HMM as a pair consisting of a sequence of statess(0)s(1) . . . s(n), called a path and a corresponding sequence of emissionse(1) . . . e(n), called an observation, such that

s(0) = s0;

∀i , 0 ≤ i ≤ n − 1, p(s(i); s(i+1)) > 0(probability to transit from s(i) to s(i+1));

∀i , 0 < i ≤ n, p(s(i); e(i)) > 0(probability to emit e(i) from s(i)).

Definition

The probability of such a run is defined as∏i=1..n p(s(i−1); s(i)) · p(s(i); e(i))



Decoding with Hidden Markov Models

Decoding: Inferring the hidden path given the observation sequence.

argmaxpathP(path|observation)

Computed in linear time using dynamic programming:The Viterbi algorithm can be seen as keeping track of, for each prefix ofan observed emission sequence, the most probable (partial) path leading toeach possible state, and extending those step by step into longer paths,eventually covering the entire emission sequence.

source: wikipedia


Motivation and background Biological sequence alignment with pair HMMs

A tiny bit of biological motivation

Biologists often want to align sequences (e.g. DNA, proteins, etc.)which are thought to have descended from a common ancestor

Sequence alignment is fundamental to bioinformatics

We (computer scientists) can just consider biological sequences as alist of symbols

The number of possible of alignments is exponential

An optimal alignment can be found using a Pair HMM inO(|sequence|2 ∗ |states|2)



Pair HMMs for sequence alignment

A pair HMM is a special kind of HMM that emits two sequences

The match state emit a pair (xiyj) of symbols

The insert state emits one symbol xi , from sequence x

The delete state emits one symbol yj , from sequence y

A run of this model produces an alignment of x and y



Alignment with a pair Hidden Markov Model

Observation sequence x

H G K K G A A Q V

Observation sequence y

K G P K K A Q A

sequence xsequence yalignment b





H G K K G A A Q V


K G P K K A Q A

sequence x Hsequence yalignment b i





H G K K G A A Q V


K G P K K A Q A

sequence x H Gsequence yalignment b i i





H G K K G A A Q V


K G P K K A Q A

sequence x H G Ksequence yalignment b i i i





H G K K G A A Q V


K G P K K A Q A

sequence x H G K Ksequence y Kalignment b i i i m





H G K K G A A Q V


K G P K K A Q A

sequence x H G K K Gsequence y K Galignment b i i i m m





H G K K G A A Q V


K G P K K A Q A

sequence x H G K K G Asequence y K G Palignment b i i i m m m





H G K K G A A Q V


K G P K K A Q A

sequence x H G K K G Asequence y K G P Kalignment b i i i m m m d





H G K K G A A Q V


K G P K K A Q A

sequence x H G K K G Asequence y K G P K Kalignment b i i i m m m d d





H G K K G A A Q V


K G P K K A Q A

sequence x H G K K G A Asequence y K G P K K Aalignment b i i i m m m d d m





H G K K G A A Q V


K G P K K A Q A

sequence x H G K K G A A Qsequence y K G P K K A Qalignment b i i i m m m d d m m





H G K K G A A Q V


K G P K K A Q A

sequence x H G K K G A A Q Vsequence y K G P K K A Q Aalignment b i i i m m m d d m m m


Constrained HMMs





Constrained HMMs

Constrained HMMs

Definition

A constrained HMM (CHMM) is a 5-tuple 〈S ,A,T ,E ,C 〉 where,

〈S ,A,T ,E 〉 is an HMM

C is a set of constraints, each of which is a mapping from HMM runsinto {true, false}.

A run of a CHMM, 〈path, observation〉 is a run of the corresponding HMMfor which C (path, observation) is true.


Constrained HMMs

Constrained HMMs

Why extend an HMM with side-constraints?

To create better, more specific models with fewer states

Convenient to express prior knowledge in terms of constraints

No need to change underlying HMM

Sometimes it is not possible or feasible to express such constraints asHMM structure (e.g. all different)→ infeasibly huge state and parameter space

fewer paths to consider for any given sequence→ decreased running time


Constrained HMMs CHMMs as a constraint program

Runs of a CHMM as a constraint program

We consider runs of CHMM as a constraint program over finite domains.

A run of CHMM is a solution of the constraint program

run([s(0), S1, . . . ,Sn], [E1, . . . ,En]),dom(Si ) = S \ {s0} ∧ dom(Ei ) = E

The variable Si represents the HMM state at step i

The variable Ei represents the emission at the step i



Definition of the HMM run program

The run predicate is specified as follows,

run([s(0), S1, . . . ,Sn], [E1, . . . ,En]) is true iff

∃s(1) ∈ dom(S1), . . . ,∃s(n) ∈ dom(Sn)∧∃e(1) ∈ dom(E1), . . . ,∃e(n) ∈ dom(En)∧C (s(0)s(1) . . . s(n), e(1) . . . e(n)) is true, s(0) = s0∧p(s(0); s(1)) · p(s(1); e(1)) . . . p(s(n−1); s(n)) · p(s(n); e(n)) > 0.

This states that

s(0)s(1) . . . s(n) and e(1) . . . e(n) is a HMM run which satisfies C .



The HMM structure

The last line in the definition,

p(s(0); s(1)) · p(s(1); e(1)) . . . p(s(n−1); s(n)) · p(s(n); e(n)) > 0

indicates the (local) relationships between Si ,Si+1 and Si ,Ei whichcorrespond to the structure of the HMM. We define these relationships interms of constraints,

trans(Si−1, Si ) ∧ emit(Si ,Ei ), for all i , 1 ≤ i ≤ n

trans(Si ,Si+1) is true iff

∃s(i) ∈ dom(Si ) ∧ s(i+1) ∈ dom(Si+1)

such that p(s(i); s(i+1)) > 0

emit(Si ,Ei ) is true iff

∃s(i) ∈ dom(Si ) ∧ e(i) ∈ dom(Ei )

such that p(s(i); e(i)) > 0



Revisiting the pair HMM

Consider adding constraints to the pair HMM introduced earlier.

For instance..

In a biological context, we may want to onlyconsider alignments with a limited number ofinsertions and deletions given the assumptionthat the two sequences are closely related.

C = {cardinality atmost(Nd , [S1, . . . ,Sn], delete),cardinality atmost(Ni , [S1, . . . ,Sn], insert)} .

The constraint cardinality atmost(N, L,X ) is satisfied whenever L is a listof elements, out of which at most N are equal to X .



Computation of best alignment with constraints


Constrained HMMs Decoding algorithm for CHMMs

Decoding with a CHMM

Decoding is the process of finding the most probable state sequence givena known sequence of observations.

Consider a given observation e(1) . . . e(n), a CHMM 〈S ,A,T ,E ,C 〉, and itsconstraint program

run([s(0),S1, . . . ,Sn], [e(1), . . . , e(n)]).

The most probable path is computed by finding the valuation s(1), . . . , s(n)

that maximizes the objective function: the run probability.

In standard HMMs this is usually accomplished in polynomial time usingthe Viterbi dynamic programming algorithm.But we need to consider additional constraints, C .



An algorithm for decoding with CHMMs

The computation starts from an initial set of 5-tuples

{〈s(0), 0, 1, ε,C ∧ trans(s(0), S1) ∧∧

1≤i≤n−1 trans(Si , Si+1) ∧∧

1≤i≤n emit(Si , ei )〉}.

Then, the following two rules are applied as long as possible

trans ctr

Σ := Σ ∪ {〈s ′, i +1, p · p(s; s ′) · p(s ′; e(i+1)), π s ′, σ ∧ Si+1 = s ′〉}whenever 〈s, i , p, π, σ〉 ∈ Σ, p(s; s ′), p(s ′; e(i+1)) > 0 andcheck constraints(σ ∧ Si+1 = s ′) and prune ctr does not apply.

prune ctr

Σ := Σ \ {〈s, i +1, p′, π′, σ′〉}whenever there is another 〈s, i +1, p, π, σ〉 ∈ Σ withp ≥ p′ and sol(σ′) ⊆ sol(σ)



Illustration of the trans ctr transition rule

Constraints:

cardinality atmost(1, [S1, . . . ,Sn], delete)cardinality atmost(1, [S1, . . . ,Sn], insert)

check constraintsHenning Christiansen, Christian Theil Have, Ole Torp Lassen, Matthieu Petit Inference with CHMMs in PRISM



Constraints:


check constraints → OKHenning Christiansen, Christian Theil Have, Ole Torp Lassen, Matthieu Petit Inference with CHMMs in PRISM



Constraints:


Path extendedHenning Christiansen, Christian Theil Have, Ole Torp Lassen, Matthieu Petit Inference with CHMMs in PRISM



Constraints:





Constraints:


check constraints → failHenning Christiansen, Christian Theil Have, Ole Torp Lassen, Matthieu Petit Inference with CHMMs in PRISM



Constraints:


Path not extendedHenning Christiansen, Christian Theil Have, Ole Torp Lassen, Matthieu Petit Inference with CHMMs in PRISM



Constraints:





Constraints:


check constraints → failHenning Christiansen, Christian Theil Have, Ole Torp Lassen, Matthieu Petit Inference with CHMMs in PRISM



Constraints:


Path not extendedHenning Christiansen, Christian Theil Have, Ole Torp Lassen, Matthieu Petit Inference with CHMMs in PRISM


Illustration of the prune ctr rule

Two (partial) paths, p and p′, reach thesame state at time i

sol(σ′) ⊆ sol(σ): Are all statesreachable from p′ at any time j > i alsoreachable from p?

no →yes →

yesno →






no →

yes →

yesno →






no →rule not does apply (keep both paths)

yes →

yesno →







yes → Is P(p) ≥ P(p′)?

yesno →








yes

no →








yes →discard lowest probability path p′

no →








yes →discard lowest probability path p′

no →rule does not apply (keep both paths)


Constrained HMMs Implementation in PRISM

PRISM

PRogramming In Statistical Modelling developed by Sato, Kameyaand Zhou

An extension of Prolog with random variables, called MSWs

Provides efficient generalized inference algorithms (Viterbi, EM, etc)using tabling



Example HMM in PRISM

values/2

declares the outcomesof random variables

msw/2

simulates a randomvariable, stochasticallyselecting one of theoutcomes

Model in Prolog

Specifies relationbetween variables

Example HMM in PRISM

values(trans(_), [sunny,rainy]).

values(emit(_), [shop,beach,read]).

hmm(L):- run_length(T),hmm(T,start,L).

hmm(0,_,[]).

hmm(T,State,[Emit|EmitRest]) :-

T > 0,

msw(trans(State),NextState),

msw(emit(NextState),Emit),

T1 is T-1,

hmm(T1,NextState,EmitRest).

run_length(7).



Adding constraint checking to the HMM

HMM with constraint checking

hmm(T,State,[Emit|EmitRest],StoreIn) :-

T > 0,

msw(trans(State),NxtState),

msw(emit(NxtState),Emit),

check_constraints([NxtState,Emit],StoreIn,StoreOut),

T1 is T-1,

hmm(T1,NxtState,EmitRest,StoreOut).

Call to check constraints/3 after each distinct sequence of mswapplications

Side-constaints: The constraints are assumed to be declaredelsewhere and not interleaved with model specification

Extra Store argument in the probabilistic predicate



Checking the constraints

The goal check constraints/3 calls constraint checkers for allconstraints declared on the model.For instance, with our example pair HMM constraint,

C = {cardinality atmost(Nd , [S1, . . . ,Sn], delete),cardinality atmost(Ni , [S1, . . . ,Sn], insert)} .

We have the following incremental constraint checker implementation

A cardinality atmost constraint checker

init_constraint_store(cardinality_atmost(_,_), 0).

check_sat(cardinality_atmost(U,Max), U, In, Out) :-

Out is In + 1,Out =< Max.

check_sat(cardinality_atmost(X,_),U,S,S) :- X \= U.



A library of global constraints for Hidden Markov Models

Our implementation contains a few well-known global constraints adaptedto Hidden Markov Models.

Global constraints

cardinality lock to sequence

all different lock to set

In addition, the implementation provides operators which may be used toapply constraints to a limited set of variables.

Constraint operators

state specific

emission specific

forall subseq (sliding window operator)for range (time step range operator)



Improving tabling with PRISM

Problem: The extra Store argument makes PRISM table multiple goals(for different constraint stores) when it should only store one.

hmm(T,State,[Emit|EmitRest],Store)

To get rid of the extra argument, check constraints dynamicallymaintains it as a stack using assert/retract:

check_constraints(Update) :-

get_store(StoreBefore),

check_constraints(Update,StoreBefore,StoreAfter),

forward_store(StoreAfter).

get_store(S) :- store(S), !.

forward_store(S) :-

asserta(store(S)) ;

retract(store(S)),fail.



Impact of using a separate constraint store stack


Summary and future directions






Summary

We have demonstrated

That CHMMs can be a useful tool to

Incorporate prior knowledge in probabilistic modelPrune the search space and allow faster decoding

That CHMMs may be useful in biological sequence analysis

How CHMMs can be formulated as a constraint problem

An adaptation of the Viterbi algorithm for CHMMs

How CHMMs can be efficiently implemented in PRISM



Future directions

Improve method to employ existing state-of-the-art CSP techniques

Soft constraints

Parameter learning for CHMMs

Grammar constraints

More biological applications



Questions

Questions?



Bibliography I

V.S. Costa, D. Page, and J. Cussens.CLP(BN): Constraint logic programming for probabilistic knowledge.Probabilistic Inductive Logic Programming, LNAI 4911:156–188, 2008.

M-W Chang, L-A Ratinov, and D. Rizzolo, N. Roth.Learning and inference with constraints.In Proc. of AAAI Conference on Artificial Intelligence, pages1513–1518, Chicago, USA, July 2008.

T. Sato and Y. Kameya.New advances in logic-based probabilistic by PRISM.In Probabilistic Inductive Logic Programming, LNCS, pages 118–155.Springer, 2008.


inference with constrained hidden markov models in prism

Technology

e i henning christiansen

hidden markov models

matthieu petit inference

chmms implementation

n ps i1 s i ps

sequences henning christiansen

wikipedia henning christiansen

hidden path