the what, why, and how of probabilistic veriﬁcation...probabilistic programming joost-pieter...

The What, Why, and How of Probabilistic VerificationPart 1: Motivation and Models

Joost-Pieter Katoen

CAV Invited Tutorial 2015, San Francisco

Joost-Pieter Katoen What, Why, and How of Probabilistic Verification 1/38

Roadmap of This Tutorial

Part 1. Motivation and Models▸ More Than 5 Reasons for Probabilistic Analysis▸ Elementary Models and Properties

Part 2. Algorithmic Foundations▸ Reachability and Beyond in Discrete Markov Models▸ Timed Reachability in Continuous Markov Models

Part 3. Treating Gigantic Markov Models▸ Abstraction: Precise, Aggressive, and Compositional

Part 4. Recent Research Developments▸ Parameter Synthesis and Model Repair▸ Counterexample Generation▸ Probabilistic Programming


Overview

The Relevance of Probabilities

Markov Models and Properties



Overview





More Than Five Reasons for Probabilities

1. Randomised Algorithms2. Reducing Complexity3. Probabilistic Programming4. Reliability5. Performance6. Optimization7. Systems Biology



Randomised Algorithms: Simulating a Die [Knuth & Yao, 1976]

Heads = “go left”; tails = “go right”.



Randomised Algorithms: Simulating a Die [Knuth & Yao, 1976]

Heads = “go left”; tails = “go right”. Does this model a six-sided die?



Distributed Computing

FLP impossibility result [Fischer et al., 1985]

In an asynchronous setting, where only one processor might crash, there is nodistributed algorithm that solves the consensus problem—getting a distributednetwork of processors to agree on a common value.



Distributed Computing

FLP impossibility result [Fischer et al., 1985]

In an asynchronous setting, where only one processor might crash, there is nodistributed algorithm that solves the consensus problem—getting a distributednetwork of processors to agree on a common value.

Ben-Or’s possibility result [Ben-Or, 1983]

If a process can make a decision based on its internal state, the messagestate, and some probabilistic state, consensus in an asynchronous setting isalmost surely possible.



Reducing Complexity: Matrix Multiplication [Freivald, 1977]

Input: three O(N2) square matrices A, B , and C

Output: yes, if A ×B = C ; no, otherwise






Deterministic: compute A × B and compare with C

Complexity: in O(N3), best known complexity O(N2.37)








Randomised: 1. take a random bit-vector x⃗ of size N2. compute A × (B x⃗) − C x⃗3. output yes if this yields the null vector; no otherwise4. repeat these steps k times








Randomised: 1. take a random bit-vector x⃗ of size N2. compute A × (B x⃗) − C x⃗3. output yes if this yields the null vector; no otherwise4. repeat these steps k times

Complexity: in O(k ⋅N2), with false positive with probability ⩽ 2−k



Probabilistic Programming

2013, DARPA launched a 48M (US dollar) program on

“Probabilistic Programming (PP) for Advanced Machine Learning (ML)”

“PP is a new programming paradigm for managing uncertain information.By incorporating it into ML, we seek to greatly increase the number of people

who can successfully build ML applications,and make ML experts radically more effective”.



Probabilistic Programming: Once Upon a Time . . . . . .



Duelling Cowboys [McIver and Morgan, 2005]



Duelling Cowboys [McIver and Morgan, 2005]

Claim: cowboy A wins the duel with probability at least (1−b)⋅aa+b−a⋅b



Survivor Probability




Survivor Probability


Usage: security, machine learning, approximate computing . . . . . .



Reliability Engineering



Reliability: (Dynamic) Fault Trees [Dugan et al., 1990]



A Fault Tree Example



A Fault Tree Example

(D)FTs: one of —if not the— most prominent models for risk analysisAims: quantify system reliability and availability, MTTF, . . . . . .



Reliability: Architectural Languages [Feiler et al., 2010]




[System] Power.Imp

Nominal

pr imary

b a c k u p

b a t t 1 . e m p t yb a t t 2 . e m p t y

pr imary b a c k u p

[Device] Battery.imp: batt1

NominalError Data

c h a r g e d

ene rgy ‘ = -0 .02 e n e r g y ‘ > = 2 0

e n e r g y > = 1 5 vo l t age := f ( ene rgy)

d e p l e t e d

ene rgy ‘ = -0 .03

e m p t y = > e n e r g y < 2 0

vo l t age := f ( ene rgy )

ok

d e a d

e m p t y

e n e r g y ini t 100

pr imary

vo l t age

vo l t age

[Device] Battery.imp: batt2

NominalError Data

c h a r g e d

ene rgy ‘ = -0 .02 e n e r g y ‘ > = 2 0

e n e r g y > = 1 5 vo l t age := f ( ene rgy)

d e p l e t e d

ene rgy ‘ = -0 .03

e m p t y = > e n e r g y < 2 0

vo l t age := f ( ene rgy )

ok

d e a d

e m p t y

e n e r g y ini t 100

b a c k u p

vo l t age

vo l t age



Performance: GSPNs [Ajmone Marsan et al., 1984]

The early days:




The early days:

More modern times: Petri nets with

▸ Timed transitions▸ Immediate transitions▸ Natural weights

t0

k0t1

k1

t2 k2

λ

µ




The early days:

More modern times: Petri nets with

▸ Timed transitions▸ Immediate transitions▸ Natural weights

t0

k0t1

k1

t2 k2

λ

µ

Aims: quantify arrivals, waiting times, QoS, soft deadlines, . . . . . .GSPNs: very—if not the most—popular in performance modeling



Stochastic Scheduling



Encyclopedia of Optimization 2008



Stochastic Scheduling

▸ Job processing times are subject to random variability▸ machine breakdowns and repairs, job parameters, . . .▸ N independent jobs with mean duration 1

µi

▸ M identical machines▸ job processing with (or without) pre-emption

▸ Objective = minimal expected makespan—finishing time of last job

▸ SEPT policy yields minimal expected makespan (Bruno et al., JACM 1981)

“it is hard to calculate these expected values”

Which policy maximises the probability to finish all jobs on time?



Stochastic Scheduling (N = 4;M = 2)



Stochastic Model



Systems Biology



Systems Biology

Enzyme-catalysed substrate conversion



Stochastic Chemical Kinetics

▸ Types of reaction described by stochiometric equations:

E + Sk1

⇌k2

Ck3

→ E + P

▸ N different types of molecules that randomly collidewhere state X(t) = (x1, . . . , xN) with xi =# molecules of sort i





E + Sk1

⇌k2

Ck3

→ E + P


▸ Reaction probability within infinitesimal interval [t, t+∆):αm(x⃗) ⋅∆ = Pr{reaction m in [t, t+∆) ∣ X(t) = x⃗}

where αm(x⃗) = km ⋅ # possible combinations of reactant molecules in x⃗





E + Sk1

⇌k2

Ck3

→ E + P


▸ Reaction probability within infinitesimal interval [t, t+∆):αm(x⃗) ⋅∆ = Pr{reaction m in [t, t+∆) ∣ X(t) = x⃗}

where αm(x⃗) = km ⋅ # possible combinations of reactant molecules in x⃗

▸ Process has the Markov property and is time-homogeneous



Substrate Conversion in the Small

2400

1310

0220

2301

1211

0121

2202

1112

0022

2103

1013 2004

8

32

11

1000

21000

6

22

11

1000

21000

4

12

11

1000

21000

211

1000

States: init goalenzymes 2 2substrates 4 0complex 0 0products 0 4

Transitions: E + S1⇌1C

0.001→ E +P

e.g., (xE , xS , xC , xP)0.001⋅xC→ (xE + 1, xS , xC − 1, xP + 1) for xC > 0



Overview





Common Feature

All these applications consider Markov models1

1Non-exponential distributions are approximated by phase-type distributions.Joost-Pieter Katoen What, Why, and How of Probabilistic Verification 32/38


Discrete-Time Markov Models

A Markov chainfor Knuth-Yao’s algorithm A Markov decision process

for the cowboy program



Continuous-Time Markov Models

2400

1310

0220

2301

1211

0121

2202

1112

0022

2103

1013 2004

8

32

11

1000

21000

6

22

11

1000

21000

4

12

11

1000

21000

211

1000

A Markov chainfor substrate conversion

t0

t1

t2

λ

µ

p1,3

p2,3

p1,5

p4 p6

p2,5 p2,7

τ

τ

τ

λ

µ1

A Markov decision processfor the GSPN



Fault Trees are Continuous-Time MDPs

Markov models of a cold, warm and hot basic event(dormancy factor µ = α⋅λ)



Continuous-Time Markov Models

Markov decision process for stochasticscheduling Markov decision processa for a SPARE gate

aIn fact, an interactive Markov chain.



Markov Models

Nondeterminism Nondeterminismno yes

Discrete time discrete-time Markov decisionMarkov chain (DTMC) process (MDP)

Continuous time CTMC CTMDP

Other models: e.g., probabilistic variants of (priced) timed automata



Properties

Logic Monitors

Discrete time probabilistic deterministic automataCTL (safety and LTL)

Continuous time probabilistic deterministictimed CTL timed automata



Properties

Logic Monitors

Discrete time probabilistic deterministic automataCTL (safety and LTL)

Continuous time probabilistic deterministictimed CTL timed automata

Core problem: computing (timed) reachability probabilities


the what, why, and how of probabilistic veriﬁcation...probabilistic programming joost-pieter...

Documents