metrics for real time probabilistic processes
DESCRIPTION
Metrics for real time probabilistic processes. Radha Jagadeesan, DePaul University Vineet Gupta, Google Inc Prakash Panangaden, McGill University Josee Desharnais, Univ Laval. Outline of talk. Models for real-time probabilistic processes - PowerPoint PPT PresentationTRANSCRIPT
Metrics for real time probabilistic processes
Radha Jagadeesan, DePaul University
Vineet Gupta, Google Inc
Prakash Panangaden, McGill University
Josee Desharnais, Univ Laval
Outline of talk
Models for real-time probabilistic processes
Approximate reasoning for real-time probabilistic processes
Discrete Time Probabilistic processes Labelled Markov Processes
For each state sFor each label a K(s, a, U)
Each state labelledwith propositional information
0.50.3
0.2
Discrete Time Probabilistic processes Markov Decision Processes
For each state sFor each label a K(s, a, U)
Each state labelledwith numerical rewards
0.50.3
0.2
Discrete time probabilistic proceses
+ nondeterminism : label does not determine probability distribution uniquely.
Real-time probabilistic processes
Add clocks to Markov processes
Each clock runs down at fixed rate r c(t) = c(0) – r t
Different clocks can have different rates
Generalized SemiMarkov Processes Probabilistic multi-rate timed automata
Generalized semi-Markov processes.
Each state labelledwith propositional Information
Each state has a setof clocks associated with it.
{c,d}
{d,e} {c}
s
tu
Generalized semi-Markov processes.
Evolution determined bygeneralized states <state, clock-valuation>
<s,c=2, d=1>
Transition enabled when a clockbecomes zero
{c,d}
{d,e} {c}
s
tu
Generalized semi-Markov processes.
<s,c=2, d=1> Transition enabled in 1 time unit
<s,c=0.5,d=1> Transition enabled in 0.5 time unit
{c,d}
{d,e} {c}
s
tu
Clock c
Clock d
Generalized semi-Markov processes.
Transition determines:
a. Probability distribution on next states
b. Probability distribution on clock values for new clocks
{c,d}
{d,e} {c}
s
tu
Clock c
Clock d
0.2 0.8
Generalized semi Markov proceses
If distributions are continuous and states are finite:
Zeno traces have measure 0
Continuity results. If stochastic processes from <s, > converge to the stochastic process at <s, >
Equational reasoning
Establishing equality: Coinduction Distinguishing states: Modal logics Equational and logical views coincide Compositional reasoning: ``bisimulation is
a congruence’’
Labelled Markov Processes
PCTL Bisimulation [Larsen-Skou,
Desharnais-Panangaden-Edalat]
Markov Decision Processes
Bisimulation [Givan-Dean-Grieg]
Labelled Concurrent Markov Chains
PCTL [Hansson-Johnsson]
Labelled Concurrent Markov chains (with tau)
PCTLCompleteness: [Desharnais-
Gupta-Jagadeesan-Panangaden]
Weak bisimulation [Philippou-Lee-Sokolsky,
Lynch-Segala]
With continuous time
Continuous time Markov chains
CSL [Aziz-Balarin-Brayton-
Sanwal-Singhal-S.Vincentelli]
Bisimulation,Lumpability
[Hillston, Baier-Katoen-Hermanns]
Generalized Semi-Markov processes
Stochastic hybrid systems
CSL
Bisimulation:?????
Composition:?????
Alas!
Instability of exact equivalence
Vs
Vs
Problem!
Numbers viewed as coming with an error estimate.
(eg) Stochastic noise as abstraction Statistical methods for estimating
numbers
Problem!
Numbers viewed as coming with an error estimate.
Reasoning in continuous time and continuous space is often via discrete approximations.
eg. Monte-Carlo methods to approximate probability distributions by a sample.
Idea: Equivalence metrics
Jou-Smolka, Lincoln-Scedrov-Mitchell-Mitchell
Replace equality of processes by (pseudo)metric distances between processes
Quantitative measurement of the distinction between processes.
Criteria on approximate reasoning
Soundness Usability Robustness
Criteria on metrics for approximate reasoning Soundness
Stability of distance under temporal evolution: ``Nearby states stay close '‘ through temporal evolution.
``Usability’’ criteria on metrics
Establishing closeness of states: Coinduction.
Distinguishing states: Real-valued modal logics.
Equational and logical views coincide: Metrics yield same distances as real-valued modal logics.
``Robustness’’ criterion on approximate reasoning The actual numerical values of the
metrics should not matter --- ``upto uniformities’’.
Uniformities (same)
m(x,y) = |x-y| m(x,y) = |2x + sinx -2y – siny|
Uniformities (different)
m(x,y) = |x-y|
Our results
Our results
For Discrete time models: Labelled Markov processes Labelled Concurrent Markov chains Markov decision processes
For continuous time: Generalized semi-Markov processes
Results for discrete time models
Bisimulation Metrics
Logic (P)CTL(*) Real-valued modal logic
Compositionality Congruence Non-expansivity
Proofs Coinduction Coinduction
Results for continuous time models
Bisimulation Metrics
Logic CSL Real-valued modal logic
Compositionality ??? ???
Proofs Coinduction Coinduction
Metrics for discrete time probablistic processes
Bisimulation
Fix a Markov chain. Define monotone F on equivalence relations:
Defining metric: An attempt
Define functional F on metrics.
Metrics on probability measures
Wasserstein-Kantorovich
A way to lift distances from states to a distances on distributions of states.
Metrics on probability measures
Metrics on probability measures
Example 1: Metrics on probability measures
Unit measure concentrated at x
Unit measure concentrated at y
x y
m(x,y)
Example 1: Metrics on probability measures
Unit measure concentrated at x
Unit measure concentrated at y
x y
m(x,y)
Example 2: Metrics on probability measures
Example 2: Metrics on probability measures
THEN:
Lattice of (pseudo)metrics
Defining metric coinductively
Define functional F on metrics
Desired metric is maximum fixed point of F
Real-valued modal logic
Real-valued modal logic
Tests:
Real-valued modal logic (Boolean)
q
q
Real-valued modal logic
Results
Modal-logic yields the same distance
as the coinductive definition However, not upto uniformities since glbs
in lattice of uniformities is not determined by glbs in lattice of pseudometrics.
Variant definition that works upto uniformities
Fix c<1. Define functional F on metrics
Desired metric is maximum fixed point of F
Reasoning upto uniformities
For all c<1, get same uniformity
[see Breugel/Mislove/Ouaknine/Worrell]
Variant of earlier real-valued modal logic incorporating discount factor c characterizes the metrics
Metrics for real-time probabilistic processes
Generalized semi-Markov processes.
{c,d}
{d,e} {c}
s
tu
Clock c
Clock d
Evolution determined bygeneralized states <state, clock-valuation>
: Set of generalized states
Generalized semi-Markov processes.
{c,d}
{d,e} {c}
s
tu
Clock c
Clock d
Path:
Traces((s,c)): Probability distribution on a set of paths.
Accomodating discontinuities: cadlag functions
(M,m) a pseudometric space. cadlag if:
Countably many jumps, in general
Defining metric: An attempt
Define functional F on metrics. (c <1)
traces((s,c)), traces((t,d)) are distributions on sets of cadlag functions.
What is a metric on cadlag functions???
Metrics on cadlag functions
Not separable!
are at distance 1 for unequal x,y
x y
Skorohod metrics (J2)
(M,m) a pseudometric space. f,g cadlag with range M.
Graph(f) = { (t,f(t)) | t \in R+}
t
fg
(t,f(t))
Skorohod J2 metric: Hausdorff distance between graphs of f,g
f(t)g(t)
Skorohod J2 metric
(M,m) a pseudometric space. f,g cadlag
Examples of convergence to
Example of convergence
1/2
Example of convergence
1/2
Examples of convergence
1/2
Examples of convergence
1/2
Examples of non-convergence
Jumps are detected!
Non-convergence
Non-convergence
Non-convergence
Non-convergence
Summary of Skorohod J2
A separable metric space on cadlag functions
Defining metric coinductively
Define functional on 1-bounded pseudometrics (c <1)
Desired metric: maximum fixpoint of F
a. s, t agree on all propositions
b.
Real-valued modal logic
Real-valued modal logic
Real-valued modal logic
h: Lipschitz operator on unit interval
Real-valued modal logic
Real-valued modal logic
Base case for path formulas??
Base case for path formulas
First attempt:
Evaluate state formula F on stateat time t
Problem: Not smooth enough wrt time sincepaths have discontinuities
Base case for path formulas
Next attempt:
``Time-smooth’’ evaluation of state formula F at time t on path
Upper Lipschitz approximation to evaluatedat t
Real-valued modal logic
Non-convergence
Illustrating Non-convergence
1/2
1/2
Results
For each c<1, modal-logic yields the same uniformity as the coinductive definition
All c<1 yield the same uniformity. Thus, construction can be carried out in lattice of uniformities.
Proof steps
Continuity theorems (Whitt) of GSMPs yield separable basis
Finite separability arguments yield closure ordinal of functional F is omega.
Duality theory of LP for calculating metric distances
Results
Approximating quantitative observables:
Expectations of continuous functions are continuous
Continuous mapping theorems for establishing continuity of quantitative observables
Summary
Approximate reasoning for real-time probabilistic processes
Results for discrete time models
Bisimulation Metrics
Logic (P)CTL(*) Real-valued modal logic
Compositionality Congruence Non-expansivity
Proofs Coinduction Coinduction
Results for continuous time models
Bisimulation Metrics
Logic CSL Real-valued modal logic
Compositionality ??? ???
Proofs Coinduction Coinduction
Questions?