dobbiaco lectures 2010 (3) solved and unsolved problems in ...outline probabilistic temporal logic...

OutlineProbabilistic Temporal Logic and Model Checking

Causation

Dobbiaco Lectures 2010 (3)Solved and Unsolved Problems in Biology

Bud Mishra

Room 1002, 715 Broadway, Courant Institute, NYU, New York, USA

Dobbiaco

B Mishra Dobbiaco Lectures 2010 (3)Solved and Unsolved Problems in


Causation

Outline

1 Probabilistic Temporal Logic and Model Checking

2 Causation



Causation

PART VII: Uncertainty, Logic and Time



Causation

Main theses

“...1 “...2 “What is the case (a fact) is the existence of states of

affairs.3 “A logical picture of facts is a thought.4 “A thought is a proposition with sense.5 “A proposition is a truth-function of elementary

propositions.6 “The general form of a proposition is the general form of a

truth function, which is: 〈p, ξ,¬ξ〉

7 “Where (or of what) one cannot speak, one must passover in silence. ”

–Ludwig Wittgenstein, Tractatus Logico-Philosophicus, 1921.



Causation

Outline


2 Causation



Causation

Logic

Propositional Logic (Boolean Connectives)

First Order and Higher Order Logic (Quantifiers)

Modal Logic (Modes)

Temporal Logic (Temporal Modes: Always, Eventually, etc.)

Linear Time (e.g., LTL) and Branching Time (e.g., CTL ...Computational Tree Logic)



Causation

Proof and Model Theories

Axioms and Rules of Inference [Proofs and Theorems]

Models Associated with a Theory [Model Checking]

Satisfiability

Verification and Complexity



Causation

Probabilistic Temporal Logics

PCTL was introduced by Hansson and Jonsson.

It concentrates on probabilistic model checking withrespect to probabilistic deterministic systems (PDS)

A PDS is a state labelled Markov chain

PCTL allows one to check temporal properties of the kind“starting from a state s, a certain PCTL path formula holdswith a probability of at least p” in terms of a probabilisticoperator.

This probabilistic operator may be seen as a counterpart tothe A and E operators in CTL (i.e. it can be applied topath-formulae only).

Similar to CTL, PCTL can as well be extended by relaxingthe PCTL-syntax to PCTL*



Causation

Probabilistic Deterministic Systems (PDS)

A Probabilistic Deterministic Systems (PDS) is essentiallya state labelled Finite Markov chain (we restrict ourselvesto finite systems).

Probability Distribution

Given a finite set S, a probability distribution on S is a functionµ : S → [0, 1] such that

∑

s∈S µ(s) = 1.

Given a probability distribution on S, supp(µ) denotes thesupport, i.e. the states s of S with µ(s) > 0.

For s ∈ S, µ1s denotes the unique distribution on S that

satisfies µ1s(s) = 1.

With Distr(S) we denote the set of all probabilitydistributions on S.



Causation

Finite Markov Chain

Definition

Finite Markov Chain

A finite Markov chain is a tuple M = (S, T , p), where

S is a finite set,

T ⊆ S × S is a set of transitions and

p : S × S → [0, 1] is a function such that for all s ∈ S

Furtherp(s, .) is a probability distribution on S and(s, t) ∈ T iff t ∈ supp(p(s, .)).Notation puv instead of p(u, v) for u, v ∈ S.(S, T ) is the underlying graph of M. Note that T is total,thus (S, T ) has no terminal nodes.



Causation

Starting Probability Distribution

A Markov chain is normally equipped with a startingprobability distribution.

This induces a stochastic process on the set S of its statesin a natural way. The probability that the process starts in acertain state with step 0 is determined by the startingdistribution.

Being in state u in the (t − 1)th step, the probability that theprocess is in state v in the t th step is equal to puv .

These probabilities do not depend on the earlier steps(history-independent or memoryless) — This is called theMarkov property .



Causation

PDS

Definition: Ergodic Set

Let G = (V , E) be a finite directed graph. We call C ⊆ V anergodic set of G, iff C is a terminal strongly connectedcomponent of G, i.e. (i) ∀(u, v) ∈ E : u ∈ C ⇒ v ∈ C and (ii)∀u, v ∈ C : ∃ a path from u to v in G

An ergodic set is a strongly connected component that cannotbe left once the execution sequence reached one of its states.

Definition: Ergodic Set of a Markov Chain

Given a Markov chain M = (S, T , p), we call C ⊆ S an ergodicset of M, if C is an ergodic set of the underlying graph of M.



Causation

PDS

Definition: Probabilistic Deterministic System (PDS)

A probabilistic deterministic system (PDS) is a tupleTPDS = (M, AP, L, s), where

M = (S, T , p) is a finite Markov chain,

AP is a finite set of atomic propositions,

L is a labelling function L : S → 2AP that labels any states ∈ S with those atomic propositions that are supposed tohold in s and

s ∈ S is the starting state.



Causation

Path, Trace and Cylinder

Definition: Path

A path of T is a finite or infinite sequence π = s0, s1, . . . ofstates, such that (si , si+1) ∈ T for all i under consideration.Given a finite path ρ = s0, s1, . . . , sn, denote s0 by first(ρ) and sn

by last(ρ). The length |ρ| of ρ is equal to n. For an infinite pathπ, the length is equal to ∞. Given a path π = s0, s1, . . . (finite orinfinite) and i ≤ |π|, we denote the i th state of π by πi (i.e.πi = si ) and the i-th prefix by π ↑i= s0, s1, . . . , si . Given aninfinite path π = s0, s1, . . ., denote the suffix starting at πi byπ ↑i , i.e. π ↑i= si , si+1, . . .. Furthermore we denote by Pathsfin(resp. Pathsinf ) the set of finite (resp. infinite) paths of a givenPDS and by Pathsfin(s) (resp. Pathsinf (s)) the set of finite (resp.infinite) paths of a given PDS starting at the state s.



Causation

Path, Trace and Cylinder

Definition: Trace

We define the trace of a (finite) path π = s0, s1, . . . to be the(finite) word over the alphabet 2AP which we get from thefollowing:

trace(π) = L(π0), L(π1), . . . = L(s0), L(s1), . . .

Definition: Basic Cylinder

For ρ ∈ Pathsfin the basic cylinder of ρ is defined as

∆(ρ) = {π ∈ Pathsinf : π ↑|ρ|= ρ}.

Let s ∈ S. Following Markov chain theory and measure theory,we can define a probability space on the set of paths starting ins.



Causation

Probability Space of a Markov Chain

Definition: Probability Space of a Markov Chain

Given a Finite Markov chain M = (S, T , p) and s ∈ S, we definea probability space

Ψs = (∆(s),∆s, probs),

such that

∆s is the σ-algebra generated by the empty set and thebasic cylinders over S that are contained in ∆(s).

probs is the uniquely induced probability measure whichsatisfies the following: probs(∆(s)) = 1 and for all basiccylinders ∆(s, s1, . . . , sn) over S:

probs(∆(s, s1, . . . , sn)) = pss1 · · · psn−1sn



Causation

Probability Space of a Markov Chain

Lemma: Given a Markov chain M = (S, T , p), the followingholds for all s ∈ S:

ES = {π ∈ Pathsinf (s) : ∃C ergodic set of M s.t.

∀c ∈ C : πi = c for infinitely many i ’s }

probs(ES) = 1.

This means that given a Markov chain M = (S, T , p) and anarbitrary starting state s it holds that the stochastic processdescribed by M and starting in s will reach an ergodic set C ofM and visit each state of C infinitely often with probability 1.Next we will define the syntax and semantics of PCTL.



Causation

PCTL syntax

Define the PCTL syntax + some intuitive explanationsabout the corresponding semantics when meaningful.The formal semantics follows:

PCTL-Syntax

The syntax of PCTL is defined by the following grammar:PCTL-state formulae

Φ ::= true|a|Φ1 ∧ Φ2|¬Φ|[Φ]1p

PCTL-path formulae:

Φ ::= Φ1U≤tΦ2|Φ1UΦ2|XΦ

where t ∈ N, p ∈ [0, 1] ⊂ R and a ∈ AP, 1∈ {>,≥,≤, <}.



Causation

PCTL-formulae

The set of all PCTL-formulae is formPCTL = {f : f is a PCTLformula}, where we assume a fixed set of atomicpropositions AP. The symbol 1 is used for “>,” “≥,” “<,” or“≤” respectively.

Length of a PCTL-Formula

We define the length of a PCTL-formula Φ as the number ofatomic propositions, temporal, Boolean and probabilisticoperators that are contained inside the formula and write |Φ|.

Other Boolean operators (i.e. ∨, →, ↔) are not definedexplicitly but derived from ∧ and ¬.



Causation

PCTL-formulae

State formulae represent properties which can be anyatomic proposition (i.e. a ∈ AP) or Boolean combinationsof them as well as [.]1p properties that require a certain“amount” (wrt a measure, as definition earlier) of paths toexist starting in the current state and satisfying theenclosed path formula.

Path formulae involve the strong until (be it the boundedU≤t or the unbounded U ) or the nextstep operator X .These are three independent operators. Intuitively the pathformula Φ1U

≤tΦ2 states that Φ1 holds continuously fromnow on until within at most t time units when Φ2 becomestrue. The unbounded operator Φ1UΦ2 does not require anybound but nevertheless requires Φ2 to become trueeventually.



Causation

PCTL-Modal Operators

Similar to “always 2” and “eventually 3” operators in CTLthere are operators in PCTL.

[3≤tΦ]1p := [true U≤tΦ]1p.

[2≤tΦ]1p := ¬[true U≤t¬Φ]1(1−p)

= ¬[3≤t¬Φ]1(1−p).

Again, the definitions are the same for the unboundedversions of these operators.The main difference between CTL and PCTL lies in theextended ability to quantify over paths.PCTL versus pCTL: In pCTL there is no boundeduntil-operator. Otherwise, they are essentially same.



Causation

The Semantics of PCTL

Unlike CTL (or RTCTL) syntax which is interpreted overKripke structures, PCTL formulae (or PCTL? formulae) areconsidered in the context of probabilistic systems — E.g.,deterministic (PDS) or nondeterministic (PNS).

We will only discuss the deterministic case. For a PDS T ,define the satisfaction relation|=PDS, that is

|=PDS⊆ (ST ∪ Pathsinf ) × formPCTL,

where ST denotes the state space of T .



Causation


We define the equivalence relation≡⊂ formPCTL × formPCTL of two PCTL-formulae Φ1 and Φ2

Φ1 ≡ Φ2 iff s |= Φ1 ⇔ s |= Φ2,

for all PDS T and for all s ∈ ST .

Certain relations between CTL and PCTL can be showneasily...



Causation

CTL vs. PCTL

Following equivalences between CTL formulae (left) andPCTL formulae (right) hold (where a, b are atomicpropositions):

∀(Xa) ≡ [Xa]≥1 (1)

∃(Xa) ≡ [Xa]>0 (2)

∃(a Ub) ≡ [a Ub]>0 – but (3)

∀(a Ub) 6≡ [a Ub]≥1 (4)

∀(2a) ≡ [2a]≥1](also,∀(a Ub) ≡ [a Ub]≥1 (5)

∃(2a) 6≡ [2a]>0](also,∃(a Ub) 6≡ [a Ub]>0 (6)



Causation

Counter Example for 4 and 6:

T = 〈{s1, s2}, s1, P〉

P =

[

1/2 1/20 1

]



Causation

Model Checking of PCTL Against a PDS

The basic idea of model checking a PCTL (state) formulaΦ against a probabilistic deterministic system (PDS) Tessentially follows the idea of CTL Model Checking (asdescribed earlier (Clarke, Emerson and Sistla) andinvolves the calculation of satisfaction sets

Sat(Φ) ≡ {s ∈ ST : s |= Φ}.



Causation

In order to calculate these sets the syntax tree of Φ isconstructed and the subformulae are processed in abottom-up manner.

The algorithm traverses the syntax tree in this (postfix)order and calculates the satisfaction sets recursively (i.e.the syntax tree is not constructed explicitly...).

We will consider schemes for calculating solutions for thebounded and unbounded until operator

Then, we can give an algorithm that sketches a completemodel checking process for PCTL.



Causation

The Bounded Until

The case [Φ1 U≤tΦ2]1p, p arbitrary

We need to calculate P t(s) using the following recursiveequation (recursion 1)

P i(s) =

1, if s |= Φ2;0, if s 6|= Φ1 and s 6|= Φ2;0, if i = 0 and s |= Φ2,∑

s′∈STpss′ · P i−1(s′), otherwise.

This recursion follows the semantics given earlier.



Causation

The Unounded Until

The case [Φ1 UΦ2]>0

We need to simply calculate [Φ1 U≤|ST |Φ2]>0

Because of the finiteness of ST the paths can beshortened by removing cycles such that the bounded untilformula holds.



Causation

The Unbounded Until

The case [Φ1 UΦ2]1p, p arbitrary

We need to do something more as without a bound on t ,presumably, we may not have a terminating condition...

We use a recursion, based on a partitioning of ST intothree subsets U0, U1 and U?.

Ss = Sat(Φ2)

Sf = {s ∈ ST : s 6∈ Sat(Φ1) ∧ s 6∈ Sat(Φ2)},

Si = {s ∈ ST : s ∈ Sat(Φ1) ∧ s 6∈ Sat(Φ2)},

U0 = Sf ∪ {s ∈ Si no path in Si from s to s′ ∈ Ss}

, U1 = Ss ∪ {s ∈ Si a.a. paths reach Ss through Si starting in s

U? = S \ (U1 ∪ U0).



Causation

The Unbounded Until

With these sets the following recursion describes themeasure for the unbounded until operator. (recursion 2)

P∞(s) =

1, if s ∈ U1;0, if s ∈ U0;∑

s′∈STpss′ · P∞(s′), otherwise.

This recursion defines a linear system of equation, withunique solution

xs =∑

s′∈U?

pss′ · xs′ +∑

s′′∈U1

pss′ , s ∈ U?

This linear equation system can be solved in polynomialtime using Gaussian-elimination.



Causation

Algorithm 1 : PCTL-MC(1) - pseudo code

Algorithm 1 PCTL MODELCHECK (Φ, T , s);1

Input : PDS T , PCTL formula Φ, s ∈ ST

Output : Truth value of s |=PDS φcalculate Sat(Φ) with algorithm 2;2

if s ∈ Sat(Φ) then3

return true4

else5

return false6

end7



Causation

Algorithm 2 : PCTL-MC(2) - pseudo code

Algorithm 2 Sat (Φ);1

Input : PCTL state formulaOutput : set of all s ∈ S that satisfy Φswitch Φ do2

case true : 7→ return ST ;3

case a : 7→ return {s ∈ ST : a ∈ L(s)};4

case Φ1 ∧ Φ2 : 7→ return Sat(Φ1) ∩ Sat(Φ2);5

case ¬Φ′ : 7→ return ST \ Sat(Φ′);6

case [XΦ′]1p : 7→ calculate S′ = Sat(Φ′) & return7

{s ∈ ST : (∑

s′∈S′ pss′) 1 p} ;case [Φ1U

≤tΦ2]1p : 7→8

calculate Sat(Φ1) and Sat(Φ2);9

for s ∈ ST do P0(s) = Is∈Sat(Φ2);10

for i = 1 to t do11

for s ∈ ST do compute P i(s) using the recurrence12

ruleend13

return {s ∈ ST : P t(s) 1 p};14

end15

case [Φ1UΦ2]>0 : 7→ ...see discussions;16

case [Φ1UΦ2]1p : 7→ ...see discussions;17

end18



Causation

Complexity of PCTL Model Checking

The complexity of the model checking algorithm forchecking a PCTL formula Φ against a PDS. Given a PDST = (M, AP, L, s), M = (S, T , p)

Note the number of subformulae to be checked is≤ length(Φ).

If Φ is an atomic proposition, its negation or a Booleancombination, then Sat(Φ) can be computed in time O(|S|)

If Φ is a nextstep expression [XΦ′]1p then Sat(Φ) can becomputed in time O(|S| + |T |))

If Φ is a bounded until expression [Φ1U≤tΦ2]1p then

Sat(Φ) can be computed in time O(|S| · |T | · t))



Causation

Complexity of PCTL Model Checking

If Φ is a unbounded until expression [Φ1UΦ2]1p thenSat(Φ) can be computed in time O((|S| + |T |) · poly(|S|)))

Theorem 1

Let T be a PDS and Phi a PCTL state formula over the set ofatomic propositions of T . Then Sat(Φ) can be computed intime O(poly(|S|) · tΦ · length(Φ)).



Causation

PART VIII: Data



Causation

Outline


2 Causation



Causation

Algorithmization

How can one represent the complex definition of causalrelationship in a probabilistic temporal logic:

C ; E



Causation

Complex Structures

“Gene a & Gene b being up-regulated until Gene c isdown-regulated causes Gene d to be activated between 6to 10 time units.”

(a↑ ∧ b↑)Uc↓ ;[6,10] d↑.

Check (in a suitable model or using the data directly) if

P[FC ; FE ] > P[¬FC ; FE ].

Causes can be complex temporal formula (which mayfurther encode other causal relationships).“Three strikes, you are out.”

Repeated conviction for crimes when exceedsthree occurrences causes (in California) eventualincarnation without recourse to parole.



Causation

Prima Facie Causes

We shall use the operator

a ;[s,t]≥p b

to means “with probability p or higher, a state satisfying bis temporally preceded by a state satisfying a in antime-interval [s, t] (0 < s ≤ t ≤ ∞).”

“c (pf)-causes e” (c is a prima-facie cause of e.)

∃0<p≤1∃1≤s≤t≤∞(F≤∞>0 c) ∧ (F≤∞

<p e) ∧ (c ;[s,t]≥p e).



Causation

Statistical Significance

Define

εx (c, e) = P(c ∧ x ; e) − P(¬c ∧ x ; e)

εavg(c, e) =

∑

x∈X\c εx (c, e)

|X \ c|



Causation

ε-Significance

c is an ε-insignificant cause of e, if (i) c is a prima-faciecause of e and (ii) |εavg | < ε.

c is an ε-significant cause of e, if it is a non-ε-insignificantprima-facie cause of e.



Causation

Spurious and Genuine Causes

Separating causal relations into spurious and genuinegroups.Through FDR (false discovery rate) control.

1 Finding a null-model of spurious prima-facie causes.2 Computing p-values3 q-values and thresholding

Empirical Bayes Methods A la Efron...



Causation

Empirical Bayes FDR control

Set of statistical tests: (1) Obtain z-scores from ’s, (2)

z-score(x) =x − µ(x)

σ(x).

Prior probabilities of test: (1) Null, p0; (2) Non-null, p1

Assume: p0 ≫ p1.

Probability density functions of two classes:

f0(z) ≡ null density

f1(z) ≡ non-null density



Causation

Empirical Bayes FDR control

Mixture density

f (z) = p0f0(z) + p1f1(z).

Probability of being null (spurious cause), given a z value:

fdr(z) = P(null|z) ≈p0f0(z)

f (z).



Causation

Neural Circuits



Causation

Communication Among Neurons

Neurons communicate with one another via synapses,where the axon terminal or en passant boutons (terminalslocated along the length of the axon) of one cell impingesupon another neuron’s dendrite, soma or, less commonly,axon.

1 Neurons such as Purkinje cells in the cerebellum can haveover 1000 dendritic branches, making connections withtens of thousands of other cells; other neurons, such as themagnocellular neurons of the supraoptic nucleus, have onlyone or two dendrites, each of which receives thousands ofsynapses.

2 Synapses can be excitatory or inhibitory and will eitherincrease or decrease activity in the target neuron. Someneurons also communicate via electrical synapses, whichare direct, electrically-conductive junctions between cells.



Causation


In a chemical synapse, the process of synaptictransmission is as follows:

1 When an action potential reaches the axon terminal, itopens voltage-gated calcium channels, allowing calciumions to enter the terminal.

2 Calcium causes synaptic vesicles filled withneurotransmitter molecules to fuse with the membrane,releasing their contents into the synaptic cleft.

3 The neurotransmitters diffuse across the synaptic cleft andactivate receptors on the postsynaptic neuron.



Causation



Causation


How can we tell which neurons are (functionally)connected to which others?

Physical connection may not have a functionalconsequence. Use of the (functional) connectivity mayvary significantly over time.Assume:

1 A neuron can fire randomly (dependent on noise level).2 A firing of a neuron generates a spike train.3 A spike train from one neuron can cause another neuron’s

firing



Causation



Causation

Restitution Time

There is a window [20, 40] time units during which oneneuron’s firing can cause another to fire.

Neuron A causes B to fire, if

A ;[20,40]≥p B.



Causation



Causation

Results (1)

AITIA: a causes b in 20-40 time units

Granger: a causes b in 20 time units (no window oftime)... With linear regression in MSBVAR R package

BN: a causes b (no way to specify time)... With TETRADIV from SGS

DBN: a causes b at t in [20, 40] (specific t , not window)...With BANJO package



Causation



Causation

Results (2)

Note

FDR =# false positives# all positives

FNR =# false negatives# all negatives

There are few true positives compared to the number ofrelationships tested, so this measure looks artificially low

Intersection is for a particular pattern and particular noiselevel, what fraction of relationships are found in both



Causation



Causation

[End of Lecture #3]


dobbiaco lectures 2010 (3) solved and unsolved problems in ...outline probabilistic temporal logic...

Documents