refined probabilistic abstraction
DESCRIPTION
Computer networks and embedded systems are ubiquitous and critical parts of our daily life. Therefore performance and reliability guarantees for these systems are crucial. To this end, versatile probabilistic modelling and analysis techniques have been developed. However existing probabilistic analysis methods are inherently limited to small systems. This dissertation introduces a new probabilistic analysis method that scales to large and even infinite systems which are far out of reach of previous methods. The key idea is to approximate a given system by a smaller abstraction which is refined automatically until sufficient precision has been achieved.TRANSCRIPT
![Page 1: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/1.jpg)
Refined Probabilistic Abstraction
Bjorn Wachter
December 8, 2010
![Page 2: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/2.jpg)
• Bug in control software of power network
⇒ 50 million people without electricity
2 / 40
![Page 3: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/3.jpg)
Model checking:
“does a computing system behave as intended?”
• mathematical model M of system
• specification ϕ
• automatic proof or refutation of:
M � ϕ• Example: ϕ = no arithmetic overflow
s0
error
e.g., arithmetic overflow
3 / 40
![Page 4: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/4.jpg)
Probabilities are Important
• computer networks• performance: P(message loss) = 2%• reliability: P(node failure) = 3%
• randomized algorithms• network protocols• sorting algorithms• ...
4 / 40
![Page 5: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/5.jpg)
Probabilistic Model checking
s0
same IP
P(same IP) < 0.01
• models: Markov chains
• properties: PCTL
• Zeroconf protocol
• IP for new member picked probabilistically
• bad: two members have the same IP!
5 / 40
![Page 6: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/6.jpg)
Probabilistic Model checking
s0
same IP
P(same IP) < 0.01
• models: Markov chains
• properties: PCTL
• Zeroconf protocol
• IP for new member picked probabilistically
• bad: two members have the same IP!
5 / 40
![Page 7: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/7.jpg)
Why Abstraction?
• Limitations of probabilistic model checking• based on state-space exploration• state-space explosion problem
• Abstraction very successful• Refinement fits the abstraction to the property
M
M ]abstraction
M � ϕ
M ] � ϕguarantees
refine
6 / 40
![Page 8: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/8.jpg)
Why Abstraction?
• Limitations of probabilistic model checking• based on state-space exploration• state-space explosion problem
• Abstraction very successful
• Refinement fits the abstraction to the property
M M ]abstraction
M � ϕ
M ] � ϕguarantees
refine
6 / 40
![Page 9: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/9.jpg)
Why Abstraction?
• Limitations of probabilistic model checking• based on state-space exploration• state-space explosion problem
• Abstraction very successful
• Refinement fits the abstraction to the property
M M ]abstraction
M � ϕ M ] � ϕ
guarantees
refine
6 / 40
![Page 10: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/10.jpg)
Why Abstraction?
• Limitations of probabilistic model checking• based on state-space exploration• state-space explosion problem
• Abstraction very successful
• Refinement fits the abstraction to the property
M M ]abstraction
M � ϕ M ] � ϕguarantees
refine
6 / 40
![Page 11: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/11.jpg)
Why Abstraction?
• Limitations of probabilistic model checking• based on state-space exploration• state-space explosion problem
• Abstraction very successful
• Refinement fits the abstraction to the property
M M ]abstraction
M � ϕ M ] � ϕguarantees
refine
6 / 40
![Page 12: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/12.jpg)
Why Abstraction?
• Limitations of probabilistic model checking• based on state-space exploration• state-space explosion problem
• Abstraction very successful• Refinement fits the abstraction to the property
M M ]abstraction
M � ϕ M ] � ϕguarantees
refine
6 / 40
![Page 13: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/13.jpg)
Contribution
Abstraction refinement for very large probabilistic models
• ... even infinite ones!
• implementation in PASS tool
• successful on various network protocols• Wireless LAN• IPV4• BRP• ...
7 / 40
![Page 14: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/14.jpg)
Background
8 / 40
![Page 15: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/15.jpg)
Probabilistic Programs
// parallel composition of modules
module sender
i : i n t ; // variable definition
...
endmodule
module channelK
[aF] (k=0) -> 0.98 : (k’=1) // probabilistic
+ 0.02 : (k ’=2); // guarded command
endmodule
i n i t T=false & ... e n d i n i t // initial states
9 / 40
![Page 16: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/16.jpg)
Semantics: Markov Decision Process (MDP)states S ∼= assignments to program variables
. . .. . .
s0
s1 s2 s3
s4
b
12
12
b
12
12
c
13
13
13
c
13
13
13
c23
13
• Markov chain ∼= deterministic MDP
10 / 40
![Page 17: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/17.jpg)
Semantics: Markov Decision Process (MDP)states S ∼= assignments to program variablesprobabilistic transitions . . . probabilistic choice
. . .
s0
s1 s2 s3
s4
b
12
12
b12
12
c
13
13
13
c
13
13
13
c23
13
• Markov chain ∼= deterministic MDP
10 / 40
![Page 18: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/18.jpg)
Semantics: Markov Decision Process (MDP)states S ∼= assignments to program variablesprobabilistic transitions . . . probabilistic choicenon-deterministic choice . . . concurreny
s0
s1 s2 s3
s4b
12
12
b
12
12
c
13
13
13
c
13
13
13
c23
13
• Markov chain ∼= deterministic MDP
10 / 40
![Page 19: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/19.jpg)
Semantics: Markov Decision Process (MDP)states S ∼= assignments to program variablesprobabilistic transitions . . . probabilistic choicenon-deterministic choice . . . concurreny
s0
s1 s2 s3
s4b
12
12
b
12
12
c
13
13
13
c
13
13
13
c23
13
• Markov chain ∼= deterministic MDP
10 / 40
![Page 20: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/20.jpg)
Properties: Probabilistic Reachability
• probabilities to reach states F ⊆ S• valuations [0, 1]S ∼= S → [0, 1]
s0
s1 s2 s3
s4
b
12
12
b
12
12
c
13
13
13
c
13
13
13
c23
13
• reachability probability pηF• depends on adversary η
• minimal/maximal
pminF = inf
ηpηF
pmaxF = sup
ηpηF
least fixpoint of Pre : [0, 1]S → [0, 1]S
w 7→ λs.
1 ; s ∈ F0 ; s ∈ F0
a∈A(s)
∑(u,t)∈U×S
R(s, u, t) · w(t) ; ow.
11 / 40
![Page 21: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/21.jpg)
Properties: Probabilistic Reachability
• probabilities to reach states F ⊆ S• valuations [0, 1]S ∼= S → [0, 1]
s0
s1 s2 s3
s4
b
12
12
b
12
12
c
13
13
13
c
13
13
13
c23
13
• reachability probability pηF• depends on adversary η
• minimal/maximal
pminF = inf
ηpηF
pmaxF = sup
ηpηF
reachability probability: 1
least fixpoint of Pre : [0, 1]S → [0, 1]S
w 7→ λs.
1 ; s ∈ F0 ; s ∈ F0
a∈A(s)
∑(u,t)∈U×S
R(s, u, t) · w(t) ; ow.
11 / 40
![Page 22: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/22.jpg)
Properties: Probabilistic Reachability
• probabilities to reach states F ⊆ S• valuations [0, 1]S ∼= S → [0, 1]
s0
s1 s2 s3
s4
b
12
12
b
12
12
c
13
13
13
c
13
13
13
c23
13
• reachability probability pηF• depends on adversary η
• minimal/maximal
pminF = inf
ηpηF
pmaxF = sup
ηpηF
reachability probability: 12
least fixpoint of Pre : [0, 1]S → [0, 1]S
w 7→ λs.
1 ; s ∈ F0 ; s ∈ F0
a∈A(s)
∑(u,t)∈U×S
R(s, u, t) · w(t) ; ow.
11 / 40
![Page 23: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/23.jpg)
Properties: Probabilistic Reachability
• probabilities to reach states F ⊆ S• valuations [0, 1]S ∼= S → [0, 1]
s0
s1 s2 s3
s4
b
12
12
b
12
12
c
13
13
13
c
13
13
13
c23
13 • reachability probability pηF
• depends on adversary η
• minimal/maximal
pminF = inf
ηpηF
pmaxF = sup
ηpηF
adversary η : Paths → A
• resolves non-determinism
⇒ induces a Markov chain
least fixpoint of Pre : [0, 1]S → [0, 1]S
w 7→ λs.
1 ; s ∈ F0 ; s ∈ F0
a∈A(s)
∑(u,t)∈U×S
R(s, u, t) · w(t) ; ow.
11 / 40
![Page 24: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/24.jpg)
Properties: Probabilistic Reachability
• probabilities to reach states F ⊆ S• valuations [0, 1]S ∼= S → [0, 1]
s0
s1 s2 s3
s4
b
12
12
b
12
12
c
13
13
13
c
13
13
13
c23
13 • reachability probability pηF
• depends on adversary η
• minimal/maximal
pminF = inf
ηpηF
pmaxF = sup
ηpηF
adversary η : Paths → A
• resolves non-determinism
⇒ induces a Markov chain
least fixpoint of Pre : [0, 1]S → [0, 1]S
w 7→ λs.
1 ; s ∈ F0 ; s ∈ F0
a∈A(s)
∑(u,t)∈U×S
R(s, u, t) · w(t) ; ow.
11 / 40
![Page 25: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/25.jpg)
Properties: Probabilistic Reachability
• probabilities to reach states F ⊆ S• valuations [0, 1]S ∼= S → [0, 1]
s0
s1 s2 s3
s4
b
12
12
b
12
12
c
13
13
13
c
13
13
13
c23
13 • reachability probability pηF
• depends on adversary η
• minimal/maximal
pminF = inf
ηpηF
pmaxF = sup
ηpηF
adversary η : Paths → A
• resolves non-determinism
⇒ induces a Markov chain
least fixpoint of Premin : [0, 1]S → [0, 1]S
w 7→ λs.
1 ; s ∈ F0 ; s ∈ F0
mina∈A(s)
∑(u,t)∈U×S
R(s, u, t) · w(t) ; ow.
11 / 40
![Page 26: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/26.jpg)
Properties: Probabilistic Reachability
• probabilities to reach states F ⊆ S• valuations [0, 1]S ∼= S → [0, 1]
s0
s1 s2 s3
s4
b
12
12
b
12
12
c
13
13
13
c
13
13
13
c23
13 • reachability probability pηF
• depends on adversary η
• minimal/maximal
pminF = inf
ηpηF
pmaxF = sup
ηpηF
adversary η : Paths → A
• resolves non-determinism
⇒ induces a Markov chain
least fixpoint of Premin : [0, 1]S → [0, 1]S
w 7→ λs.
1 ; s ∈ F0 ; s ∈ F0
mina∈A(s)
∑(u,t)∈U×S
R(s, u, t) · w(t) ; ow.
11 / 40
![Page 27: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/27.jpg)
Properties: Probabilistic Reachability
• probabilities to reach states F ⊆ S• valuations [0, 1]S ∼= S → [0, 1]
s0
s1 s2 s3
s4
b
12
12
b
12
12
c
13
13
13
c
13
13
13
c23
13 • reachability probability pηF
• depends on adversary η
• minimal/maximal
pminF = inf
ηpηF
pmaxF = sup
ηpηF
adversary η : Paths → A
• resolves non-determinism
⇒ induces a Markov chain
least fixpoint of Premax : [0, 1]S → [0, 1]S
w 7→ λs.
1 ; s ∈ F0 ; s ∈ F0
maxa∈A(s)
∑(u,t)∈U×S
R(s, u, t) · w(t) ; ow.
11 / 40
![Page 28: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/28.jpg)
Abstraction
12 / 40
![Page 29: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/29.jpg)
Abstraction for Probabilistic Reachability
• Problem: many states S
1 merge states to blocks Q
• in example, 16 states but only 3 blocks Q = {B1, B2, B3}.
2 compute abstract valuations [0, 1]Q
1.0
0.2
0.3
0.0
in general, hard to compute
B3
B2B1
≤≤B3
B2B1 1.0
0.8
1.0
B3
B2B1 0.1
0.0
1.0
0.5
0.1
0.7
0.8
in general, hard to compute
B3
B2B1
≤≤B3
B2B1 1.0
0.8
1.0
B3
B2B1 0.1
0.0
1.0
1.0
1.0
0.4
0.3
in general, hard to compute
B3
B2B1
≤≤B3
B2B1 1.0
0.8
1.0
B3
B2B1 0.1
0.0
1.0
1.0
1.0
0.1
0.2
in general, hard to compute
B3
B2B1
≤≤B3
B2B1 1.0
0.8
1.0
B3
B2B1 0.1
0.0
1.0
reachability probability
lower-bound analysis upper-bound analysis
13 / 40
![Page 30: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/30.jpg)
Abstraction for Probabilistic Reachability
• Problem: many states S
1 merge states to blocks Q
• in example, 16 states but only 3 blocks Q = {B1, B2, B3}.
2 compute abstract valuations [0, 1]Q
1.0
0.2
0.3
0.0
in general, hard to compute
B3
B2B1
≤≤B3
B2B1 1.0
0.8
1.0
B3
B2B1 0.1
0.0
1.0
0.5
0.1
0.7
0.8
in general, hard to compute
B3
B2B1
≤≤B3
B2B1 1.0
0.8
1.0
B3
B2B1 0.1
0.0
1.0
1.0
1.0
0.4
0.3
in general, hard to compute
B3
B2B1
≤≤B3
B2B1 1.0
0.8
1.0
B3
B2B1 0.1
0.0
1.0
1.0
1.0
0.1
0.2
in general, hard to compute
B3
B2B1
≤≤B3
B2B1 1.0
0.8
1.0
B3
B2B1 0.1
0.0
1.0
reachability probability
lower-bound analysis upper-bound analysis
13 / 40
![Page 31: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/31.jpg)
Abstraction for Probabilistic Reachability
• Problem: many states S
1 merge states to blocks Q
• in example, 16 states but only 3 blocks Q = {B1, B2, B3}.
2 compute abstract valuations [0, 1]Q
1.0
0.2
0.3
0.0
in general, hard to compute
B3
B2B1
≤≤
B3
B2B1
1.0
0.8
1.0
B3
B2B1 0.1
0.0
1.0
0.5
0.1
0.7
0.8
in general, hard to compute
B3
B2B1
≤≤
B3
B2B1
1.0
0.8
1.0
B3
B2B1 0.1
0.0
1.0
1.0
1.0
0.4
0.3
in general, hard to compute
B3
B2B1
≤≤
B3
B2B1
1.0
0.8
1.0
B3
B2B1 0.1
0.0
1.0
1.0
1.0
0.1
0.2
in general, hard to compute
B3
B2B1
≤≤
B3
B2B1
1.0
0.8
1.0
B3
B2B1 0.1
0.0
1.0
reachability probability
lower-bound analysis upper-bound analysis
13 / 40
![Page 32: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/32.jpg)
Abstraction for Probabilistic Reachability
• Problem: many states S
1 merge states to blocks Q
• in example, 16 states but only 3 blocks Q = {B1, B2, B3}.
2 compute abstract valuations [0, 1]Q
1.0
0.2
0.3
0.0
in general, hard to compute
B3
B2B1
≤
≤
B3
B2B1 1.0
0.8
1.0
B3
B2B1 0.1
0.0
1.0
0.5
0.1
0.7
0.8
in general, hard to compute
B3
B2B1
≤
≤
B3
B2B1 1.0
0.8
1.0
B3
B2B1 0.1
0.0
1.0
1.0
1.0
0.4
0.3
in general, hard to compute
B3
B2B1
≤
≤
B3
B2B1 1.0
0.8
1.0
B3
B2B1 0.1
0.0
1.0
1.0
1.0
0.1
0.2
in general, hard to compute
B3
B2B1
≤
≤
B3
B2B1 1.0
0.8
1.0
B3
B2B1 0.1
0.0
1.0
reachability probability
lower-bound analysis upper-bound analysis
13 / 40
![Page 33: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/33.jpg)
Abstraction for Probabilistic Reachability
• Problem: many states S
1 merge states to blocks Q
• in example, 16 states but only 3 blocks Q = {B1, B2, B3}.
2 compute abstract valuations [0, 1]Q
1.0
0.2
0.3
0.0
in general, hard to compute
B3
B2B1
≤≤B3
B2B1 1.0
0.8
1.0
B3
B2B1 0.1
0.0
1.00.5
0.1
0.7
0.8
in general, hard to compute
B3
B2B1
≤≤B3
B2B1 1.0
0.8
1.0
B3
B2B1 0.1
0.0
1.01.0
1.0
0.4
0.3
in general, hard to compute
B3
B2B1
≤≤B3
B2B1 1.0
0.8
1.0
B3
B2B1 0.1
0.0
1.01.0
1.0
0.1
0.2
in general, hard to compute
B3
B2B1
≤≤B3
B2B1 1.0
0.8
1.0
B3
B2B1 0.1
0.0
1.0
reachability probability
lower-bound analysis upper-bound analysis
13 / 40
![Page 34: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/34.jpg)
Challenge of Analysis Design
complex interplay
probabilism
uncertainty from abstraction
concurrency
• Open Question:
what does an optimal analysis look like?
• Our solution:
• Recipe: Abstract Interpretation [Cousot77]
• Ingredients:• abstraction functions
14 / 40
![Page 35: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/35.jpg)
Challenge of Analysis Design
complex interplay
probabilism
uncertainty from abstraction
concurrency
• Open Question:
what does an optimal analysis look like?
• Our solution:
• Recipe: Abstract Interpretation [Cousot77]
• Ingredients:• abstraction functions
14 / 40
![Page 36: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/36.jpg)
Challenge of Analysis Design
complex interplay
probabilism
uncertainty from abstraction
concurrency
• Open Question:
what does an optimal analysis look like?
• Our solution:
• Recipe: Abstract Interpretation [Cousot77]
• Ingredients:• abstraction functions
14 / 40
![Page 37: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/37.jpg)
Abstraction & concretization
1 abstraction functions:• mappings
[0, 1]S 7→ [0, 1]
Q
• lower bound:αl(w) = λB. inf
s∈Bw(s)
• upper bound:αu(w) = λB. sup
s∈Bw(s)
2 concretiztion function(or meaning function)
• [0, 1]Q 7→ [0, 1]
S
• γ(w]) = λs. w]([s]).
[0, 1]S [0, 1]Q
B3
B2B1
w1.0
0.2
0.3
0.0
0.5
0.1
0.7
0.8
1.0
1.0
0.4
0.3
1.0
1.0
0.1
0.2
0.1
1.0
0.0
B3
B2B1
0.1
0.1
0.0
0.0
0.1
0.1
0.0
0.0
1.0
1.0
0.0
0.0
1.0
1.0
0.0
0.0
B3
B2B1
1.0
1.0
0.8
0.8
1.0
1.0
0.8
0.8
1.0
1.0
0.8
0.8
1.0
1.0
0.8
0.8
≤≤
B3
B2B1 0.1
0.0
1.0
α l
≤
B3
B2B1 1.0
0.8
1.0
αu
γ
γ
15 / 40
![Page 38: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/38.jpg)
Abstraction & concretization
1 abstraction functions:• mappings
[0, 1]S 7→ [0, 1]
Q
• lower bound:αl(w) = λB. inf
s∈Bw(s)
• upper bound:αu(w) = λB. sup
s∈Bw(s)
2 concretiztion function(or meaning function)
• [0, 1]Q 7→ [0, 1]
S
• γ(w]) = λs. w]([s]).
[0, 1]S [0, 1]Q
B3
B2B1
w1.0
0.2
0.3
0.0
0.5
0.1
0.7
0.8
1.0
1.0
0.4
0.3
1.0
1.0
0.1
0.2
0.1
1.0
0.0
B3
B2B1
0.1
0.1
0.0
0.0
0.1
0.1
0.0
0.0
1.0
1.0
0.0
0.0
1.0
1.0
0.0
0.0
B3
B2B1
1.0
1.0
0.8
0.8
1.0
1.0
0.8
0.8
1.0
1.0
0.8
0.8
1.0
1.0
0.8
0.8
≤≤
B3
B2B1 0.1
0.0
1.0
α l
≤
B3
B2B1 1.0
0.8
1.0
αu
γ
γ
15 / 40
![Page 39: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/39.jpg)
Abstraction & concretization
1 abstraction functions:• mappings
[0, 1]S 7→ [0, 1]
Q
• lower bound:αl(w) = λB. inf
s∈Bw(s)
• upper bound:αu(w) = λB. sup
s∈Bw(s)
2 concretiztion function(or meaning function)
• [0, 1]Q 7→ [0, 1]
S
• γ(w]) = λs. w]([s]).
[0, 1]S [0, 1]Q
B3
B2B1
w1.0
0.2
0.3
0.0
0.5
0.1
0.7
0.8
1.0
1.0
0.4
0.3
1.0
1.0
0.1
0.2
0.1
1.0
0.0
B3
B2B1
0.1
0.1
0.0
0.0
0.1
0.1
0.0
0.0
1.0
1.0
0.0
0.0
1.0
1.0
0.0
0.0
B3
B2B1
1.0
1.0
0.8
0.8
1.0
1.0
0.8
0.8
1.0
1.0
0.8
0.8
1.0
1.0
0.8
0.8
≤≤
B3
B2B1 0.1
0.0
1.0
α l
≤
B3
B2B1 1.0
0.8
1.0
αu
γ
γ
15 / 40
![Page 40: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/40.jpg)
Abstraction & concretization
1 abstraction functions:• mappings
[0, 1]S 7→ [0, 1]
Q
• lower bound:αl(w) = λB. inf
s∈Bw(s)
• upper bound:αu(w) = λB. sup
s∈Bw(s)
2 concretiztion function(or meaning function)
• [0, 1]Q 7→ [0, 1]
S
• γ(w]) = λs. w]([s]).
[0, 1]S [0, 1]Q
B3
B2B1
w1.0
0.2
0.3
0.0
0.5
0.1
0.7
0.8
1.0
1.0
0.4
0.3
1.0
1.0
0.1
0.2
0.1
1.0
0.0
B3
B2B1
0.1
0.1
0.0
0.0
0.1
0.1
0.0
0.0
1.0
1.0
0.0
0.0
1.0
1.0
0.0
0.0
B3
B2B1
1.0
1.0
0.8
0.8
1.0
1.0
0.8
0.8
1.0
1.0
0.8
0.8
1.0
1.0
0.8
0.8
≤≤
B3
B2B1 0.1
0.0
1.0
α l
≤
B3
B2B1 1.0
0.8
1.0
αu
γ
γ
15 / 40
![Page 41: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/41.jpg)
How do we get an Abstract Analysis?
• abstract analysis• function f ] : [0, 1]Q → [0, 1]
Q
⇒ lower/upper bound = fixpoint of f ]
• Best-transformer paradigm [Cousot 2002]
[0, 1]Q [0, 1]Q
f ]
MDP
γ α
|pmin
F
|pmax
F
[ ]
αu ◦ PreminF ◦ γ
[ ]
abstract abstract
concrete transformer
Premin F
PremaxF
αl αu αl αu
16 / 40
![Page 42: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/42.jpg)
How do we get an Abstract Analysis?
• abstract analysis• function f ] : [0, 1]Q → [0, 1]
Q
⇒ lower/upper bound = fixpoint of f ]
• Best-transformer paradigm [Cousot 2002]
[0, 1]Q [0, 1]Qf ]
MDP
γ α
|pmin
F
|pmax
F
[ ]
αu ◦ PreminF ◦ γ
[ ]
abstract abstract
concrete transformer
Premin F
PremaxF
αl αu αl αu
16 / 40
![Page 43: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/43.jpg)
How do we get an Abstract Analysis?
• abstract analysis• function f ] : [0, 1]Q → [0, 1]
Q
⇒ lower/upper bound = fixpoint of f ]
• Best-transformer paradigm [Cousot 2002]
[0, 1]Q [0, 1]Qf ]
MDP
γ α
|pmin
F
|pmax
F
[ ]
αu ◦ PreminF ◦ γ
[ ]
abstract abstract
concrete transformer
Premin F
PremaxF
αl αu αl αu
16 / 40
![Page 44: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/44.jpg)
How do we get an Abstract Analysis?
• abstract analysis• function f ] : [0, 1]Q → [0, 1]
Q
⇒ lower/upper bound = fixpoint of f ]
• Best-transformer paradigm [Cousot 2002]
[0, 1]Q [0, 1]Qf ]
MDP
γ α
|pmin
F
|pmax
F
[ ]
αu ◦ PreminF ◦ γ
[ ]
abstract abstract
concrete transformer
Premin F
PremaxF
αl αu αl αu
16 / 40
![Page 45: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/45.jpg)
Abstract Transformers = Stochastic Games
(αu ◦ PreminF ◦ γ(w]))(B) = sup
s∈BPremin
F (γ(w]))(s)
= sups∈B mina∈A(s)∑s′∈S
R(s, a)(s′) · (γ(w]))(s′)
• Markov models• Markov chain = 1
2 player• MDP = 1 1
2 player• stochastic game = 2 1
2 player
• minimum / maximum over all strategies for both players
0 1
[
pmin,minF
]
pmax ,minF
[
pmin,maxF
]
pmax ,maxF
concurrenyplayer
abstractionplayer
abstractionplayer
17 / 40
![Page 46: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/46.jpg)
Abstract Transformers = Stochastic Games
(αu ◦ PreminF ◦ γ(w]))(B) = sup
s∈BPremin
F (γ(w]))(s)
= sups∈B mina∈A(s)∑s′∈S
R(s, a)(s′) · (γ(w]))(s′)
• Markov models• Markov chain = 1
2 player• MDP = 1 1
2 player• stochastic game = 2 1
2 player
• minimum / maximum over all strategies for both players
0 1
[
pmin,minF
]
pmax ,minF
[
pmin,maxF
]
pmax ,maxF
concurrenyplayer
abstractionplayer
abstractionplayer
17 / 40
![Page 47: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/47.jpg)
Wrap-up
• diagram defines an abstract semantics (mathematics)
MDP abstract MDP
reachability probability
[lb, ub]∈
probabilistic program
semanticsNext: how to compute this
18 / 40
![Page 48: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/48.jpg)
Wrap-up
• diagram defines an abstract semantics (mathematics)
MDP abstract MDP4
reachability probability
[lb, ub]∈
probabilistic program
semanticsNext: how to compute this
18 / 40
![Page 49: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/49.jpg)
Wrap-up
• diagram defines an abstract semantics (mathematics)
MDP abstract MDP4
reachability probability [lb, ub]
∈
probabilistic program
semanticsNext: how to compute this
18 / 40
![Page 50: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/50.jpg)
Wrap-up
• diagram defines an abstract semantics (mathematics)
MDP abstract MDP4
reachability probability [lb, ub]∈
probabilistic program
semanticsNext: how to compute this
18 / 40
![Page 51: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/51.jpg)
Wrap-up
• diagram defines an abstract semantics (mathematics)
MDP abstract MDP4
reachability probability [lb, ub]∈
probabilistic program
semantics
Next: how to compute this
18 / 40
![Page 52: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/52.jpg)
Wrap-up
• diagram defines an abstract semantics (mathematics)
MDP abstract MDP4
reachability probability [lb, ub]∈
probabilistic program
semantics
Next: how to compute this
18 / 40
![Page 53: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/53.jpg)
Wrap-up
• diagram defines an abstract semantics (mathematics)
MDP abstract MDP4
reachability probability [lb, ub]∈
probabilistic program
semanticsNext: how to compute this
18 / 40
![Page 54: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/54.jpg)
State of the Art before Thesis• de Alfaro, Roy. CAV 2007
Magnifying-Lens Abstraction for Markov Decision Processes..• Chatterjee, Henzinger, Jhala, Majumdar. UAI 2005Counterexample Guided Planning.• D’Argenio, Jeannet, Jensen, Larsen. PAPM-PROBMIV 2002Reduction and Refinement Strategies for Probabilistic Analysis.
• ... build full semantics
⇒ expensive or even impossible
Premiere: 1st symbolic abstraction for probabilistic programs• abstraction at the language level
Probabilistic program
12
12
X
Prob.
symbolic abstraction
19 / 40
![Page 55: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/55.jpg)
State of the Art before Thesis• de Alfaro, Roy. CAV 2007
Magnifying-Lens Abstraction for Markov Decision Processes..• Chatterjee, Henzinger, Jhala, Majumdar. UAI 2005Counterexample Guided Planning.• D’Argenio, Jeannet, Jensen, Larsen. PAPM-PROBMIV 2002Reduction and Refinement Strategies for Probabilistic Analysis.
• ... build full semantics⇒ expensive or even impossible
Premiere: 1st symbolic abstraction for probabilistic programs• abstraction at the language level
Probabilistic program
12
12
X
Prob.
symbolic abstraction
19 / 40
![Page 56: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/56.jpg)
State of the Art before Thesis• de Alfaro, Roy. CAV 2007
Magnifying-Lens Abstraction for Markov Decision Processes..• Chatterjee, Henzinger, Jhala, Majumdar. UAI 2005Counterexample Guided Planning.• D’Argenio, Jeannet, Jensen, Larsen. PAPM-PROBMIV 2002Reduction and Refinement Strategies for Probabilistic Analysis.
• ... build full semantics⇒ expensive or even impossible
Premiere: 1st symbolic abstraction for probabilistic programs• abstraction at the language level
Probabilistic program
12
12
X
Prob.
symbolic abstraction
19 / 40
![Page 57: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/57.jpg)
Predicate Abstraction
• Predicates ∼= expressions over program variables• e.g., x > 0, x < y
define⇒ partition
• Blocksdefine⇒ abstract model
• stochastic game
x ≥ 0 4 y ≥ 0 4
x = 0y = 0 x = 5
y = 3
x ≥ 0 4 y ≥ 0 7
x = 0y = −1 x = 5
y = −7
x ≥ 0 7 y ≥ 0 4
x = −2y = 0 x = −5
y = 3
x ≥ 0 7 y ≥ 0 7
x = −2y = −1 x = −8
y = −7
20 / 40
![Page 58: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/58.jpg)
Predicate Abstraction
• Predicates ∼= expressions over program variables• e.g., x > 0, x < y
define⇒ partition
• Blocksdefine⇒ abstract model
• stochastic game
x ≥ 0 4 y ≥ 0 4
x = 0y = 0 x = 5
y = 3
x ≥ 0 4 y ≥ 0 7
x = 0y = −1 x = 5
y = −7
x ≥ 0 7 y ≥ 0 4
x = −2y = 0 x = −5
y = 3
x ≥ 0 7 y ≥ 0 7
x = −2y = −1 x = −8
y = −7
20 / 40
![Page 59: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/59.jpg)
Predicate Abstraction
• Predicates ∼= expressions over program variables• e.g., x > 0, x < y
define⇒ partition• Blocks
define⇒ abstract model• stochastic game
x ≥ 0 4 y ≥ 0 4
x = 0y = 0 x = 5
y = 3
x ≥ 0 4 y ≥ 0 7
x = 0y = −1 x = 5
y = −7
x ≥ 0 7 y ≥ 0 4
x = −2y = 0 x = −5
y = 3
x ≥ 0 7 y ≥ 0 7
x = −2y = −1 x = −8
y = −7
20 / 40
![Page 60: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/60.jpg)
Predicate Abstraction
• Predicates ∼= expressions over program variables• e.g., x > 0, x < y
define⇒ partition• Blocks
define⇒ abstract model• stochastic game
x ≥ 0 4 y ≥ 0 4
x = 0y = 0 x = 5
y = 3
x ≥ 0 4 y ≥ 0 7
x = 0y = −1 x = 5
y = −7
x ≥ 0 7 y ≥ 0 4
x = −2y = 0 x = −5
y = 3
x ≥ 0 7 y ≥ 0 7
x = −2y = −1 x = −8
y = −7
20 / 40
![Page 61: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/61.jpg)
Predicate Abstraction
• Predicates ∼= expressions over program variables• e.g., x > 0, x < y
define⇒ partition• Blocks
define⇒ abstract model• stochastic game
• reduce abstraction to satisfiability of logical formulas⇒ implemented by SMT solver• SMT = Satisfiability Modulo Theories
20 / 40
![Page 62: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/62.jpg)
Example• Consider program
module main
s : [0..2]; // control flow
x,y : i n t ; // integer variables
[a] s=0 -> 1.0:(s’=1) & (x’=y);
[b] s=0 & x>10 -> 0.5:(s’=0)+ 0.5:(s’=2);
endmodule
• predicates s = 0,s = 1,s = 2,x = 0,x > 0,x < 0
s = 0, x > 0
s = 1, x < 0 s = 1, x = 0 s = 1, x > 0
s = 2, x > 0
ab
0.5
0.5
?
21 / 40
![Page 63: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/63.jpg)
Example• Consider program
module main
s : [0..2]; // control flow
x,y : i n t ; // integer variables
[a] s=0 -> 1.0:(s’=1) & (x’=y);
[b] s=0 & x>10 -> 0.5:(s’=0)+ 0.5:(s’=2);
endmodule
• predicates s = 0,s = 1,s = 2,x = 0,x > 0,x < 0
s = 0, x > 0
s = 1, x < 0 s = 1, x = 0 s = 1, x > 0
s = 2, x > 0
ab
0.5
0.5
?
21 / 40
![Page 64: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/64.jpg)
Example• Consider program
module main
s : [0..2]; // control flow
x,y : i n t ; // integer variables
[a] s=0 -> 1.0:(s’=1) & (x’=y);
[b] s=0 & x>10 -> 0.5:(s’=0)+ 0.5:(s’=2);
endmodule
• predicates s = 0,s = 1,s = 2,x = 0,x > 0,x < 0
s = 0, x > 0
s = 1, x < 0 s = 1, x = 0 s = 1, x > 0
s = 2, x > 0
ab
0.5
0.5
?
21 / 40
![Page 65: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/65.jpg)
Example• Consider program
module main
s : [0..2]; // control flow
x,y : i n t ; // integer variables
[a] s=0 -> 1.0:(s’=1) & (x’=y);
[b] s=0 & x>10 -> 0.5:(s’=0)+ 0.5:(s’=2);
endmodule
• predicates s = 0,s = 1,s = 2,x = 0,x > 0,x < 0
s = 0, x > 0
s = 1, x < 0 s = 1, x = 0 s = 1, x > 0
s = 2, x > 0
ab
0.5
0.5
?
21 / 40
![Page 66: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/66.jpg)
Example• Consider program
module main
s : [0..2]; // control flow
x,y : i n t ; // integer variables
[a] s=0 -> 1.0:(s’=1) & (x’=y);
[b] s=0 & x>10 -> 0.5:(s’=0)+ 0.5:(s ’=2);
endmodule
• predicates s = 0,s = 1,s = 2,x = 0,x > 0,x < 0
s = 0, x > 0
s = 1, x < 0 s = 1, x = 0 s = 1, x > 0
s = 2, x > 0
ab
0.5
0.5
?
21 / 40
![Page 67: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/67.jpg)
Example• Consider program
module main
s : [0..2]; // control flow
x,y : i n t ; // integer variables
[a] s=0 -> 1.0:(s’=1) & (x’=y);
[b] s=0 & x>10 -> 0.5:(s’=0)+ 0.5:(s ’=2);
endmodule
• predicates s = 0,s = 1,s = 2,x = 0,x > 0,x < 0
s = 0, x > 0
s = 1, x < 0 s = 1, x = 0 s = 1, x > 0
s = 2, x > 0
ab
0.5
0.5
?
21 / 40
![Page 68: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/68.jpg)
Example• Consider program
module main
s : [0..2]; // control flow
x,y : i n t ; // integer variables
[a] s=0 -> 1.0:(s’=1) & (x’=y);
[b] s=0 & x>10 -> 0.5:(s’=0)+ 0.5:(s ’=2);
endmodule
• predicates s = 0,s = 1,s = 2,x = 0,x > 0,x < 0
s = 0, x > 0
s = 1, x < 0 s = 1, x = 0 s = 1, x > 0
s = 2, x > 0
ab
0.5
0.5
?
21 / 40
![Page 69: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/69.jpg)
Example• Consider program
module main
s : [0..2]; // control flow
x,y : i n t ; // integer variables
s=0 -> 1.0:(s’=1) & (x’=y);
s=0 & x>10 -> 0.5:(s’=0)+ 0.5:(s ’=2);
endmodule
• predicates s = 0,s = 1,s = 2,x = 0,x > 0,x < 0
stochastic two-player games = 0, x > 0
s = 1, x < 0 s = 1, x = 0 s = 1, x > 0
s = 2, x > 0
ab
0.5
0.5
?
s = 0, x > 0
s = 1, x < 0 s = 1, x = 0 s = 1, x > 0
s = 2, x > 0
ab
0.5
0.5
?
21 / 40
![Page 70: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/70.jpg)
Wrap-up
• fully automatic and symbolic abstraction• for given predicate set
• ... but where do predicates come from?
MDP abstract MDP
probabilistic program
semantics
22 / 40
![Page 71: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/71.jpg)
Wrap-up
• fully automatic and symbolic abstraction• for given predicate set
• ... but where do predicates come from?
MDP abstract MDP
probabilistic program
semanticspredicate abstraction
22 / 40
![Page 72: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/72.jpg)
Wrap-up
• fully automatic and symbolic abstraction• for given predicate set
• ... but where do predicates come from?
MDP abstract MDP
probabilistic program
semanticspredicate abstraction
22 / 40
![Page 73: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/73.jpg)
Reachability Properties
s0
e
Pη]
B ( e) < 0.03
23 / 40
![Page 74: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/74.jpg)
Abstraction Refinement
refinement steps
reac
hab
ility
prob
abili
ty
Pη]
B ( e) < p
24 / 40
![Page 75: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/75.jpg)
Abstraction Refinement
refinement steps
reac
hab
ility
prob
abili
ty
due to abstraction
Pη]
B ( e) < p
24 / 40
![Page 76: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/76.jpg)
Abstraction Refinement
refinement steps
reac
hab
ility
prob
abili
tydue to abstraction
Pη]
B ( e) < p
24 / 40
![Page 77: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/77.jpg)
Abstraction Refinement
refinement steps
reac
hab
ility
prob
abili
tydue to abstraction
Pη]
B ( e) < p
24 / 40
![Page 78: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/78.jpg)
Abstraction Refinement
refinement steps
reac
hab
ility
prob
abili
ty
Pη]
B ( e) < p
24 / 40
![Page 79: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/79.jpg)
Abstraction Refinement
refinement steps
reac
hab
ility
prob
abili
ty
Pη]
B ( e) < p
24 / 40
![Page 80: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/80.jpg)
Abstraction Refinement
refinement steps
reac
hab
ility
prob
abili
ty
Pη]
B ( e) < p
24 / 40
![Page 81: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/81.jpg)
Abstraction Refinement
refinement steps
reac
hab
ility
prob
abili
ty
Pη]
B ( e) < p
24 / 40
![Page 82: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/82.jpg)
Property shown
refinement steps
reac
hab
ility
prob
abili
ty
Pη]
B ( e) < p
4
24 / 40
![Page 83: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/83.jpg)
Property shown
refinement steps
reac
hab
ility
prob
abili
ty
Pη]
B ( e) < p
4
24 / 40
![Page 84: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/84.jpg)
Property refuted
refinement steps
reac
hab
ility
prob
abili
ty
Pη]
B ( e) < p
7
24 / 40
![Page 85: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/85.jpg)
Property refuted
refinement steps
reac
hab
ility
prob
abili
ty
Pη]
B ( e) < p
7
24 / 40
![Page 86: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/86.jpg)
Inconclusive: refinement due
refinement steps
reac
hab
ility
prob
abili
ty
Pη]
B ( e) < p
?
24 / 40
![Page 87: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/87.jpg)
Inconclusive: refinement due
refinement steps
reac
hab
ility
prob
abili
ty
Pη]
B ( e) < p
?
24 / 40
![Page 88: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/88.jpg)
Inconclusive: refinement due
refinement steps
reac
hab
ility
prob
abili
ty
Pη]
B ( e) < p
?
24 / 40
![Page 89: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/89.jpg)
CEGAR: Counterexample-Guided Abstraction Refinement
• refinement technique in software model checking• SLAM project at Microsoft [Ball/Rajamani 2002,...]• Blast• ...
programproperty
build abstraction
analyse abstractionanalyse CE
4CE 7
abstract CE
new predicates
What is an abstract CE?
• pioneering work inprobabilistic verification
25 / 40
![Page 90: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/90.jpg)
CEGAR: Counterexample-Guided Abstraction Refinement
• refinement technique in software model checking• SLAM project at Microsoft [Ball/Rajamani 2002,...]• Blast• ...
programproperty
build abstraction
analyse abstractionanalyse CE
4CE 7
abstract CE
new predicates What is an abstract CE?
• pioneering work inprobabilistic verification
25 / 40
![Page 91: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/91.jpg)
Counterexamples
Safety: Error State Unreachable Probabilistic Reachability
Transition System
resolvenondet.choice
23
13 1
3
13
13
23
13
13
13
13
. . .
23
13
13
23
13
13
13
23
13
26 / 40
![Page 92: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/92.jpg)
Counterexamples
Safety: Error State Unreachable Probabilistic Reachability
Transition System
resolvenondet.choice
23
13 1
3
13
13
CE is a Path
23
13
13
13
13
“error state is reachable”
. . .
23
13
13
23
13
13
13
23
13
26 / 40
![Page 93: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/93.jpg)
Counterexamples
Safety: Error State Unreachable Probabilistic Reachability
Transition Systemresolvenondet.choice
23
13 1
3
13
13
CE is a Path
23
13
13
13
13
“error state is reachable”
. . .
23
13
13
23
13
13
13
23
13
26 / 40
![Page 94: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/94.jpg)
Counterexamples
Safety: Error State Unreachable Probabilistic Reachability
Transition Systemresolvenondet.choice
Stochastic Game
23
13 1
3
13
13
CE is a Path
23
13
13
13
13
“error state is reachable”
. . .
23
13
13
23
13
13
13
23
13
26 / 40
![Page 95: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/95.jpg)
Counterexamples
Safety: Error State Unreachable Probabilistic Reachability
Transition Systemresolvenondet.choice
Stochastic Game
23
13 1
3
13
13
CE is a Path Markov Chain
23
13
13
13
13
“error state is reachable” “probability to reach error state = 13”
. . .
23
13
13
23
13
13
13
23
13
26 / 40
![Page 96: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/96.jpg)
Counterexamples
Safety: Error State Unreachable Probabilistic Reachability
Transition Systemresolvenondet.choice
Stochastic Game
23
13 1
3
13
13
CE is a Path Markov Chain
23
13
13
13
13
“error state is reachable” “probability to reach error state = 13”
. . .
23
13
13
23
13
13
13
23
13
26 / 40
![Page 97: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/97.jpg)
Conventional counterexample analysis
• check if the abstract counterexample is realisable ...
• if so, we’ve found a bug
B0 B1B2 B3
error
B4
27 / 40
![Page 98: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/98.jpg)
Conventional counterexample analysis
• check if the abstract counterexample is realisable ...• if so, we’ve found a bugB0 B1
B2 B3 B4
27 / 40
![Page 99: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/99.jpg)
Conventional counterexample analysis
• check if the abstract counterexample is realisable ...• if so, we’ve found a bugB0 B1
B2 B3 B4
• ... or spurious• abstraction too coarse, i.e., needs refinementB0 B1
B2B0 B1
reachable via prefixcan do postfix
B3 B4
27 / 40
![Page 100: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/100.jpg)
Conventional counterexample analysis
• check if the abstract counterexample is realisable ...• if so, we’ve found a bugB0 B1
B2 B3 B4
• ... or spurious• abstraction too coarse, i.e., needs refinementB0 B1
B2B0 B1
reachable via prefixcan do postfix
B3 B4
• Implementation with SMT solver:• convert path to formula• path realisable ⇐⇒ formula satisfiable• generate splitting predicate, e.g., by interpolation
27 / 40
![Page 101: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/101.jpg)
Analysis of Probabilistic CE
• Is there a matching real counterexample?• replay decisions of abstract counterexample
abstractCE
B0 B1 B2
c0
u1,13
u2,23
c2 u3
c1u4
• Challenge: CE correspond to many paths• Markov chain: cyclic & has probabilistic branching
• Goal 1: Leverage conventional counterexample analysis• path analysis based on SMT and interpolation
• Goal 2: avoid exploring too many paths
28 / 40
![Page 102: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/102.jpg)
Analysis of Probabilistic CE
• Is there a matching real counterexample?• replay decisions of abstract counterexample
abstractCE
B0 B1 B2
c0
u1,13
u2,23
c2 u3
c1u4
• Challenge: CE correspond to many paths• Markov chain: cyclic & has probabilistic branching
• Goal 1: Leverage conventional counterexample analysis• path analysis based on SMT and interpolation
• Goal 2: avoid exploring too many paths
28 / 40
![Page 103: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/103.jpg)
Probabilistic CEGAR
• enumerate paths of CE Markov chain
• visit paths with highest probability first [Han&Katoen 2007]
• path σ]1• if spurious generate predicate (interpolation)
• path σ]2• ...
• realisable probability mass > p?
σ]1
Pη]
B (realisable paths) > p?
abstract probability mass Pη]
B ( e)
29 / 40
![Page 104: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/104.jpg)
Probabilistic CEGAR
• enumerate paths of CE Markov chain
• visit paths with highest probability first [Han&Katoen 2007]
• path σ]1• if spurious generate predicate (interpolation)
• path σ]2
• ...
• realisable probability mass > p?
σ]1 σ]2
Pη]
B (realisable paths) > p?
abstract probability mass Pη]
B ( e)
29 / 40
![Page 105: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/105.jpg)
Probabilistic CEGAR
• enumerate paths of CE Markov chain
• visit paths with highest probability first [Han&Katoen 2007]
• path σ]1• if spurious generate predicate (interpolation)
• path σ]2• ...
• realisable probability mass > p?
σ]1 σ]2 σ]3
Pη]
B (realisable paths) > p?
abstract probability mass Pη]
B ( e)
29 / 40
![Page 106: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/106.jpg)
Probabilistic CEGAR
• enumerate paths of CE Markov chain
• visit paths with highest probability first [Han&Katoen 2007]
• path σ]1• if spurious generate predicate (interpolation)
• path σ]2• ...
• realisable probability mass > p?
σ]1 σ]2 σ]3 σ]4
Pη]
B (realisable paths) > p?
abstract probability mass Pη]
B ( e)
29 / 40
![Page 107: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/107.jpg)
Probabilistic CEGAR
• enumerate paths of CE Markov chain
• visit paths with highest probability first [Han&Katoen 2007]
• path σ]1• if spurious generate predicate (interpolation)
• path σ]2• ...
• realisable probability mass > p?
σ]1 σ]2 σ]3 σ]4 σ]5
Pη]
B (realisable paths) > p?
abstract probability mass Pη]
B ( e)
29 / 40
![Page 108: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/108.jpg)
Probabilistic CEGAR
• enumerate paths of CE Markov chain
• visit paths with highest probability first [Han&Katoen 2007]
• path σ]1• if spurious generate predicate (interpolation)
• path σ]2• ...
• realisable probability mass > p?
σ]1 σ]2 σ]3 σ]4 σ]5
Pη]
B (realisable paths) > p?
abstract probability mass Pη]
B ( e)
29 / 40
![Page 109: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/109.jpg)
Computing realisable probability is harder than it seems
23 =
realisable probability 6=∑{P(σ]) | realizable path} = 1
B0
B1
B2
B3
c0
u1,13
u2,23
c1 u3
c1u3
30 / 40
![Page 110: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/110.jpg)
Computing realisable probability is harder than it seems23 = realisable probability 6=
∑{P(σ]) | realizable path} = 1
B0
B1
B2
goal states
B3Ps0( e) = 23
Ps1( e) = 13
c0
u1,13
u2,23
c0
u1,13
u2,23
c1
c1
30 / 40
![Page 111: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/111.jpg)
Computing realisable probability is harder than it seems23 = realisable probability 6=
∑{P(σ]) | realizable path} = 1
B0
B1
B2
goal states
B3Ps0( e) = 23
Ps1( e) = 13
c0
u1,13
u2,23
c0
u1,13
u2,23
c1
c1
Theorem (Realisable probability)
Realisable probability as optimisation problem:MaxSmt(exp1, . . . , expn) = max {
∑ni=1[[expi]]s · pi | s ∈ [[I ∧ F (B)]]}
where expi characterizes path σi
30 / 40
![Page 112: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/112.jpg)
Probabilistic CEGAR
• analyse paths in Markov chain with decreasing probability• spurious paths give predicates• realizable paths can improve lower bound
• MaxSMT
σ]1 σ]2 σ]3 σ]4 σ]5
Pη]
B (C)
MaxSMT (Creal)
• Semi-decision procedure for probabilistic CE analysis• always terminates returning either
7 CE realizable4 CE spurious and predicate? don’t know and predicate
• incomplete to ensure termination• limit on number of spurious paths
31 / 40
![Page 113: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/113.jpg)
Probabilistic CEGAR
• analyse paths in Markov chain with decreasing probability• spurious paths give predicates• realizable paths can improve lower bound
• MaxSMT
σ]1 σ]2 σ]3 σ]4 σ]5
Pη]
B (C)
MaxSMT (Creal)
• Semi-decision procedure for probabilistic CE analysis• always terminates returning either
7 CE realizable4 CE spurious and predicate? don’t know and predicate
• incomplete to ensure termination• limit on number of spurious paths
31 / 40
![Page 114: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/114.jpg)
Justification for Incompleteness: Undecidability
• A CE analysis problem consists of• probabilistic program M• threshold p• abstraction G of M• CE (B, η]) in G, i.e., Pη
]
B ( e)> p
• decide if the CE is real• there is a corresponding concrete CE (s, η) with Pηs ( e)> p
32 / 40
![Page 115: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/115.jpg)
Justification for Incompleteness: Undecidability
• A CE analysis problem consists of• probabilistic program M• threshold p• abstraction G of M• CE (B, η]) in G, i.e., Pη
]
B ( e)> p
• decide if the CE is real• there is a corresponding concrete CE (s, η) with Pηs ( e)> p
• assume: expressions language = linear arithmetic over the integers
⇒ conventional counterexample analysis decidable
32 / 40
![Page 116: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/116.jpg)
Justification for Incompleteness: Undecidability
• A CE analysis problem consists of• probabilistic program M• threshold p• abstraction G of M• CE (B, η]) in G, i.e., Pη
]
B ( e)> p
• decide if the CE is real• there is a corresponding concrete CE (s, η) with Pηs ( e)> p
• assume: expressions language = linear arithmetic over the integers
⇒ conventional counterexample analysis decidable
Theorem (Undecidability of Counterexample Analysis)
Counterexample analysis for probabilistic programs is undecidable
Proof.
Halting problem for counter machines can be reduced to CE analysis
32 / 40
![Page 117: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/117.jpg)
Backward Refinement (VMCAI 2010)
• Refinement if no threshold is available
• Target uncertainty from abstraction (leverage game strategies)
33 / 40
![Page 118: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/118.jpg)
The PASS tool
parserprogram property
predicateabstraction
game
prob.reachability
refinement
34 / 40
![Page 119: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/119.jpg)
Case Study: BRP
• Probability to reach:
“receiver does not receive any chunk and sender tried to send a chunk”
• Can be analyzed for an infinite parameter range with PASS• for any file size ≥ 16, probability is 1.6E − 7
• PASS provides proof for arbitrary file size
• BRP is just one case study:
Case study(parameters)
Conventional AbstractionProperty states trans time states trans refs preds paths time
5 315 k=3 5,195K 11,377K 93 34K 36K 9 120 604 72WLAN 6 315 k=3 12,616K 28,137K 302 34K 42K 9 116 604 88
(BOFF T) 6 315 k=6 12,616K 28,137K 2024 771K 113K 9 182 582 3066 9500 k=6 – – TO 771K 113K 9 182 582 3113 p1 41K 52K 10 1K 2K 8 58 28 9
CSMA/CD 4 p1 124K 161K 56 6K 9K 14 100 56 38(BOFF) 3 p2 41K 52K 10 0.5K 0.9K 12 41 28 10
4 p2 124K 161K 21 0.5K 1.5K 12 41 44 1116 3 p1 2K 3K 5.4 2K 3K 9 46 41 9
BRP 32 5 p1 5K 7K 12 5K 7K 9 64 111 21(N MAX) 64 5 p1 10K 14K 26 10K 14K 8 95 585 91
>16 3 p4 ∞ – – 0.5K 0.9K 7 26 17 3>16 4 p4 ∞ – – 0.6K 1K 7 27 17 4>16 5 p4 ∞ – – 0.7K 1K 8 28 18 5
SW goodput ∞ – – 5K 11K 3 40 7 87timeout ∞ – – 27K 44K 3 49 6 89
35 / 40
![Page 120: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/120.jpg)
Why abstraction works.
• absolute values do matter but only, e.g., differences between variables
• cannot contribute to optimum
[0.2, 0.3] [0.4, 0.8]
36 / 40
![Page 121: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/121.jpg)
Contributions
a novel analysis method for probabilistic programs:
• symbolic abstraction to tackle large state spaces• Premiere: predicate abstraction in probabilistic verification
• prior work in qualitative software verification
• Challenge
complex interplay
probabilism
uncertainty from abstraction
concurrency
• refinement to achieve full automation• Premiere: Probabilistic CEGAR
• prior work: CEGAR in qualitative software verification
• Challenge• counterexamples are Markov chains
• implemented in PASS tool
37 / 40
![Page 122: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/122.jpg)
Limitations
• Abstraction is not a panacea / silver bullet• can be less efficient for certain finite-state models
• Probabilistic CEGAR:• lower thresholds for minimal reachability?
• No support for state-dependent probabilities:
[] m=1 & x>0 -> 1x : (x’=x-1) + x−1
x : (m ’=3);
38 / 40
![Page 123: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/123.jpg)
Avenues for Future Work
• Beyond probabilistic reachability for MDPs• rewards and expectations• Exponential distributions• Support for full PCTL
• Richer input language• Modest
39 / 40
![Page 124: Refined Probabilistic Abstraction](https://reader030.vdocuments.mx/reader030/viewer/2022020309/568c331a1a28ab02358b98c8/html5/thumbnails/124.jpg)
Thesis StatementAbstraction enables automatic verification of probabilistic programs
with large and, for the first time, infinite state spaces.
40 / 40