and/or search spaces for graphical models
DESCRIPTION
AND/OR search spaces for graphical models. Vibhav Gogate (Slides courtesy of Prof. Rina Dechter , UC Irvine and her past and present students). Probabilistic Inference Tasks. Probability of evidence and Partition function Belief updating: Finding most probable explanation (MPE) - PowerPoint PPT PresentationTRANSCRIPT
AND/OR search spaces for graphical models
Vibhav Gogate(Slides courtesy of Prof. Rina Dechter, UC Irvine
and her past and present students)
Probabilistic Inference Tasks Probability of evidence and Partition function
Belief updating:
Finding most probable explanation (MPE)
Finding maximum a-posteriory hypothesisX/Aa
*k
*1 e),xP(maxarg)a,...,(a
evidence)|xP(X)BEL(X iii
e),xP(maxarg*xx
variableshypothesis: XA
𝑷 (𝒆𝒗𝒊𝒅𝒆𝒏𝒄𝒆 )
Belief Updating and Probability of Evidence
lung Cancer
Smoking
X-ray
Bronchitis
Dyspnoea
P(lung cancer=yes | smoking=no, dyspnoea=yes ) = ?P(smoking=no, dyspnoea=yes ) = ?
Recap
• Bucket elimination (BE)• Junction tree propagation• Treewidth (w*) and Tree Decompositions
– BE is time and space exponential in (w*+1).• Why is it cool?
– The complexity of exact inference can be determined by manipulating the (undirected) interaction graph
• also called primal graph, moral graph and markov network
Exact Graphical Models Inference
Eliminationand propagation
Bucket eliminationJunction tree propagationDynamic Programming
Time: exp(w*)Space: exp(w*)
Hybrids:AND/OR search
spaces
Search: ConditioningTime: exp(n)Space: linear
Depth-first search
Conditioning generates the probability tree
0
),|(),|()|()|()()0,(ebcb
cbePbadPacPabPaPeaP
Complexity of conditioning: exponential time, linear space
Conditioning+Elimination
0
),|(),|()|()|()()0,(edcb
cbePbadPacPabPaPeaP
Idea: conditioning until of a (sub)problem gets small*w
OR search spaceA
D
B C
E
F
0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1
E
C
F
D
B
A 0 1
Ordering: A B E C D F
AND/OR search space
AOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
AND 0 10 1 0 10 1 0 10 1 0 10 1
A
D
B C
E
F
A
D
B
CE
F
Primal graph DFS tree
A
D
B C
E
F
A
D
B C
E
F
OR vs AND/OR AOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
AND 0 10 1 0 10 1 0 10 1 0 10 1
E 0 1 0 1 0 1 0 1
0C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0 1
A 0 1
E 0 1 0 1 0 1 0 1
0C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0 1
A 0 1
AND/OR
OR
A
D
B C
E
FA
D
B
CE
F
1
1
1
0
1
0
Zero probability(A=1,B=1)(B=0,C=0)
AND/OR vs. OR
E 0 1 0 1 0 1 0 1
0C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0 1
A 0 1
AOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
AND 0 10 1 0 10 1 0 10 1 0 10 1
E 0 1 0 1 0 1 0 1
0C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0 1
A 0 1
E 0 1 0 1 0 1 0 1
0C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0 1
A 0 1
AND/OR
OR
A
D
B C
E
FA
D
B
CE
F
AND/OR size: exp(4), OR size exp(6)
Zero probability(A=1,B=1)(B=0,C=0)
AND/OR vs. OR
E 0 1 0 1 0 1 0 1
0C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0 1
A 0 1
AOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
AND 0 10 1 0 10 1 0 10 1 0 10 1
E 0 1 0 1 0 1 0 1
0C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0 1
A 0 1
E 0 1 0 1 0 1 0 1
0C 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0 1
A 0 1
AND/OR
OR
A
D
B C
E
FA
D
B
CE
F
Zero probability(A=1,B=1)(B=0,C=0)
AND/OR vs. OR
E 0 1 0 1 0 1
C 1 1 0 1 0 1 1 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0
A 0 1
AOR
0AND 1
BOR B
0AND 1 0
EOR C E C E C
OR D F D F D F D F
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
AND 10 1 0 10 1 10 1
E 0 1 0 1 0 1
C 1 1 0 1 0 1 1 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0
A 0 1
E 0 1 0 1 0 1
C 1 1 0 1 0 1 1 1
F 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
D 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0B 1 0
A 0 1
AND/OR
OR
A
D
B C
E
FA
D
B
CE
F
(A=1,B=1)(B=0,C=0)
OR space vs. AND/OR space
width heightOR space AND/OR space
time(sec.) nodes backtracks time(sec.) AND nodes OR nodes
5 10 3.154 2,097,150 1,048,575 0.03 10,494 5,247
4 9 3.135 2,097,150 1,048,575 0.01 5,102 2,551
5 10 3.124 2,097,150 1,048,575 0.03 8,926 4,463
4 10 3.125 2,097,150 1,048,575 0.02 7,806 3,903
5 13 3.104 2,097,150 1,048,575 0.1 36,510 18,255
5 10 3.125 2,097,150 1,048,575 0.02 8,254 4,127
6 9 3.124 2,097,150 1,048,575 0.02 6,318 3,159
5 10 3.125 2,097,150 1,048,575 0.02 7,134 3,567
5 13 3.114 2,097,150 1,048,575 0.121 37,374 18,687
5 10 3.114 2,097,150 1,048,575 0.02 7,326 3,663
AND/OR search tree for graphical models
• The AND/OR search tree of R relative to a spanning-tree, T, has:– Alternating levels of: OR nodes (variables) and AND nodes (values)
• Successor function:– The successors of OR nodes X are all its consistent values along its path– The successors of AND <X,v> are all X child variables in T
• A solution is a consistent subtree• Task: compute the value of the root node
0
A
B
0
E
OR
AND
OR
AND
OR
AND
C
0
OR
AND
D
0 1
F
0 1
1
D
0 1
F
0 1
0 1
1
E C
0
D
0 1
F
0 1
1
D
0 1
F
0 1
0 1
1
B
0
E C
0
D
0 1
F
0 1
1
D
0 1
F
0 1
0 1
1
E C
0
D
0 1
F
0 1
1
D
0 1
F
0 1
0 1
A
D
B C
E
F
A
D
B
CE
F
From DFS trees to pseudo-trees (Freuder 85, Bayardo 95)
(a) Graph
4 61
3 2 7 5
(b) DFS treedepth=3
(c) pseudo- treedepth=2
(d) Chaindepth=6
4 6
1
3
2 7
5 2 7
1
4
3 5
6
4
6
1
3
2
7
5
From DFS trees to Pseudo-trees
a
1
2
a
3
a
4
OR
AND
OR
AND
OR
AND
AND
OR
ba
b c
c
b
4
ba c
c
4
ba c
b
3
a
4
ba c
b
4
ba c
c
4
ba c
c
3
a
4
ba c
b
4
ba c
c
4
ba c
7
a
5
a
6
ba c
b
6
ba c
c
6
ba c
b
5
a
6
ba c
b
6
ba c
c
6
ba c
c
5
a
6
ba c
b
6
ba c
c
6
ba c
a
1
3
a
4
b c
OR
AND
OR
AND
OR
AND
b c
a
2
b ca
b
4
b ca
2
b ca
c
4
b ca
2
b ca
5
a
7
b ca
6
b ca
b
7
b ca
6
b ca
c
7
b ca
6
b ca
DFS treedepth = 3
pseudo- treedepth = 2
Finding min-depth backbone trees
• Finding min depth DFS, or pseudo tree is NP-complete, but:
• Given a tree-decomposition whose tree-width is w*, there exists a pseudo tree T of G whose depth, satisfies (Bayardo and Mirankar, 1996, bodlaender and Gilbert, 91):
m <= w* log n,
Generating pseudo-trees from Bucket trees
FAC
B DE
E
A
C
B
D
F (AF) (EF)
(A)
(AB)
(AC) (BC)
(AE)
(BD) (DE)
Bucket-tree based on dreverse elim order: A B C E D F
E
A
C
B
D
F
Induced graph
E
A
C
B
D F
Bucket-tree used as pseudo-tree
AND
AND
AND
AND
0 1
BOR B
0 1 0 1
COR E C E C E C E
OR D F D F D F D F D F D F D F D F
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 10 1 0 10 1 0 10 1 0 10 1
AOR
AND/OR search tree
Bucket-tree
ABE
A
ABC
AB
BDE AEF
bucket-A
bucket-E
bucket-B
bucket-C
bucket-D bucket-F
(AE) (BE)
AND/OR Search-tree properties (k = domain size, m = pseudo-tree depth. n = number of variables)
• Theorem: Any AND/OR search tree based on a pseudo-tree is sound and complete (expresses all and only solutions)
• Theorem: Size of AND/OR search tree is O(n km)Size of OR search tree is O(kn)
• Theorem: Size of AND/OR search tree can be bounded by O(exp(w* log n))
• Related to: (Freuder 85; Dechter 90, Bayardo et. al. 96, Darwiche 1999, Bacchus 2003)
• Generalizes OR search: When the pseudo-tree is a chain we get an OR space
AND/OR Tree DFS Algorithm (Probability of evidence)
AND node: Combination operator (product)
OR node: Marginalization operator (weighted summation)
Value of node = updated belief for sub-problem below
0
A
B
0
E
OR
AND
OR
AND
OR
AND
C
0
OR
AND
D
0 1
1
D
0 1
0 1
1
E C
0
D
0 1
1
D
0 1
0 1
1
B
0
E C
0
D
0 1
1
D
0 1
0 1
1
E C
0
D
0 1
1
D
0 1
0 1
A
D
B C
E
A
D
B
CE
B C D=0
D=1
0 0 .2 .80 1 .1 .91 0 .3 .71 1 .5 .5
),|( CBDP
.7.8 .9 .5 .7.8 .9 .5
Evidence: D=1
A B E=0
E=1
0 0 .4 .60 1 .5 .51 0 .7 .31 1 .2 .8
),|( BAEP
Evidence: E=0
.4 .5 .7 .2
A B=0
B=1
0 .4 .61 .1 .9
)|( ABPA C=
0C=1
0 .2 .81 .7 .3
)|( ACPA P(A
)0 .61 .4
)(AP
.2 .8 .2 .8 .1 .9 .1 .9
.4 .6 .1 .9
.6 .4
.8 .9
.8 .9
.7 .5
.7 .5
.8 .9
.8 .9
.7 .5
.7 .5
.4 .5 .7 .2.88 .54 .89 .52
.352 .27 .623 .104
.3028 .1559
.24408
.3028 .1559
Result: P(D=1,E=0)
Complexity of AND/OR Tree Search
AND/OR tree OR treeSpace O(n) O(n)
TimeO(n km)
O(n kw* log n)[Freuder & Quinn85], [Collin, Dechter & Katz91], [Bayardo & Miranker95], [Darwiche01]
O(kn)
k = domain sizem = depth of pseudo-treen = number of variablesw*= treewidth
From Search Trees to Search Graphs
• Any two nodes that root identical subtrees (subgraphs) can be merged
From Search Trees to Search Graphs
• Any two nodes that root identical subtrees (subgraphs) can be merged
AND/OR TreeA
D
B C
E
F
A
D
B
CE
F
G H
J
K
G
H
J
KAOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND
AND 0 10 1 0 10 1 0 10 1 0 10 1
OR
OR
AND
AND
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
An AND/ORgraph
A
D
B C
E
F
A
D
B
CE
F
G H
J
K
G
H
J
KAOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND
AND 0 10 1 0 10 1 0 10 1 0 10 1
OR
OR
AND
AND
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
Merging Based on Context
One way of recognizing nodes that can be merged:
context (X) = ancestors of X in pseudo tree that are connected to X, or to descendants of X
[ ]
[A]
[AB]
[AE][BC]
[AB]
A
D
B
EC
F
pseudo tree
A
E
C
B
F
D
A
E
C
B
F
D
AND/OR Tree DFS Algorithm
.8 .9 .7.7.8
AND node: Combination operator (product)OR node: Marginalization operator (summation)
Value of node = updated belief for sub-problem below
0
A
B
0
E C
0
D
0 1
1
D
0 1
0 1
1
E C
0
D
0 1
1
D
0 1
0 1
1
B
0
E C
0
D
0 1
1
D
0 1
0 1
1
E C
0
D
0 1
1
D
0 1
0 1
B C D=0
D=1
0 0 .2 .80 1 .1 .91 0 .3 .71 1 .5 .5
),|( CBDP
.7.8 .9 .5 .7.8 .9 .5
Evidence: D=1
A B E=0
E=1
0 0 .4 .60 1 .5 .51 0 .7 .31 1 .2 .8
),|( BAEP
Evidence: E=0
.4 .5 .7 .2
A B=0
B=1
0 .4 .61 .1 .9
)|( ABPA C=
0C=1
0 .2 .81 .7 .3
)|( ACPA P(A
)0 .61 .4
)(AP
.2 .8 .2 .8 .1 .9 .1 .9
.4 .6 .1 .9
.6 .4
.9
.8 .9
.5
.7 .5 .8 .9
.5
.7 .5
.4 .5 .7 .2.88 .54 .89 .52
.352 .27 .623 .104
.3028 .1559
.24408
.3028 .1559
Result: P(D=1,E=0)
A
D
B
CE
[ ]
[A]
[AB]
[BC]
[AB]
ContextA
D
B C
E
AND/OR Graph DFS Algorithm
.7.8
0
A
B
0
E C
0
D
0 1
1
D
0 1
0 1
1
E C
0
D
0 1
1
D
0 1
0 1
1
B
0
E C
0 10 1
1
E C
0 10 1
A
D
B C
E
B C D=0
D=1
0 0 .2 .80 1 .1 .91 0 .3 .71 1 .5 .5
),|( CBDP.7.8 .9 .5
Evidence: D=1
A B E=0
E=1
0 0 .4 .60 1 .5 .51 0 .7 .31 1 .2 .8
),|( BAEP
Evidence: E=0
.4 .5 .7 .2
A B=0
B=1
0 .4 .61 .1 .9
)|( ABPA C=
0C=1
0 .2 .81 .7 .3
)|( ACPA P(A
)0 .61 .4
)(AP
.2 .8 .2 .8 .1 .9 .1 .9
.4 .6 .1 .9
.6 .4
.9
.8 .9
.5
.7 .5 .8 .9 .7 .5
.4 .5 .7 .2.88 .54 .89 .52
.352 .27 .623 .104
.3028 .1559
.24408
.3028 .1559
A
D
B
CE
[ ]
[A]
[AB]
[BC]
[AB]
Context
B C Value0 0 .80 1 .91 0 .71 1 .1
Cache table for D
Result: P(D=1,E=0)
AND/OR context minimal graph C
0
K0
H0
L0 1
N N
0 1 0 1
F F F
1 1
0 1 0 1
F
G
0 1
1
A0 1
B B
0 1 0 1
E EE E
0 1 0 1
J JJ J
0 1 0 1
A0 1
B B
0 1 0 1
E EE E
0 1 0 1
J JJ J
0 1 0 1
G
0 1
G
0 1
G
0 1
M
0 1
M
0 1
M
0 1
M
0 1
P
0 1
P
0 1
O
0 1
O
0 1
O
0 1
O
0 1
L0 1
N N
0 1 0 1
P
0 1
P
0 1
O
0 1
O
0 1
O
0 1
O
0 1
D
0 1
D
0 1
D
0 1
D
0 1
K0
H0
L0 1
N N
0 1 0 1
1 1
A0 1
B B
0 1 0 1
E EE E
0 1 0 1
J JJ J
0 1 0 1
A0 1
B B
0 1 0 1
E EE E
0 1 0 1
J JJ J
0 1 0 1
P
0 1
P
0 1
O
0 1
O
0 1
O
0 1
O
0 1
L0 1
N N
0 1 0 1
P
0 1
P
0 1
O
0 1
O
0 1
O
0 1
O
0 1
D
0 1
D
0 1
D
0 1
D
0 1
B A
C
E
F G
HJ
D
K ML
N
OP
C
HK
D
M
F
G
A
B
E
J
O
L
N
P[AB]
[AF][CHAE]
[CEJ]
[CD]
[CHAB]
[CHA]
[CH]
[C]
[ ]
[CKO]
[CKLN]
[CKL]
[CK]
[C]
(C K H A B E J L N O D P M F G)
C
HK
D
M
F
G
A
B
E
J
O
L
N
P[AB]
[AF][CHAE]
[CEJ]
[CD]
[CHAB]
[CHA]
[CH]
[C]
[ ]
[CKO]
[CKLN]
[CKL]
[CK]
[C]
(C K H A B E J L N O D P M F G)
How Big Is the Context?
Theorem: The maximum context size for a pseudo tree is equal to the treewidth of the graph along the pseudo tree.
C
HK
D
M
F
G
A
B
E
J
O
L
N
P[AB]
[AF][CHAE]
[CEJ]
[CD]
[CHAB]
[CHA]
[CH]
[C]
[ ]
[CKO]
[CKLN]
[CKL]
[CK]
[C]
(C K H A B E J L N O D P M F G)
B A
C
E
F G
HJ
D
K ML
N
OP
max context size = treewidth
AND/OR search algorithms
• AO(i) – i = the max size of a cache table (i.e. number of
variables in a context)
i = 0 i = w* treewidth
Space:Time:
O(n exp w* )
O(n exp w* )
O( n )
O( exp (w* log n) )
Searching AND/OR Graphs
• AO(i): searches depth-first, cache i-context – i = the max size of a cache table (i.e. number of
variables in a context)
i=0
i=w*
Space: O(n) Time: O(exp(w* log n))
Space: O(exp w*) Time: O(exp w*)
Space: O(exp(i) )Time: O(exp(m_i+i )
i
CachingA
D
B C
E
F
A
D
B
CE
F
G H
J
K
G
H
J
KAOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND
AND 0 10 1 0 10 1 0 10 1 0 10 1
OR
OR
AND
AND
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
context(D)={D}context(F)={F}
CachingA
D
B C
E
F
A
D
B
CE
F
G H
J
K
G
H
J
KAOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND
AND 0 10 1 0 10 1 0 10 1 0 10 1
OR
OR
AND
AND
0
G
H H
0 1 0 1
0 1
1
G
H H
0 1 0 1
0 1
0
J
K K
0 1 0 1
0 1
1
J
K K
0 1 0 1
0 1
context(D)={D}context(F)={F}
All four search spaces
EC
FD
BA
0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1
0 1
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
AND 0 10 1 0 10 1 0 10 1 0 10 1
0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1
0 1
0 1 0 1
EC
FD
BA 0 1
0 1
0 1 0 1 0 1
AOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND 0 1 0 1
AND 0 10 1 0 1 0 1 0 1
Full OR search tree
Full AND/OR search tree
Context minimal OR search graph
Context minimal AND/OR search graph
A
D
B C
E
F
A
D
B
CE
F
All four search spaces
EC
FD
BA
0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
0 1 0 1
0 1
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1
AND 0 10 1 0 10 1 0 10 1 0 10 1
0 1 0 1 0 1 0 1
0 1 0 1 0 1 0 1
0 1
0 1 0 1
EC
FD
BA 0 1
0 1
0 1 0 1 0 1
AOR
0AND 1
BOR B
0AND 1 0 1
EOR C E C E C E C
OR D F D F D F D F D F D F D F D F
AND 0 1 0 1
AND 0 10 1 0 1 0 1 0 1
Full OR search tree
Full AND/OR search tree
Context minimal OR search graph
Context minimal AND/OR search graph
A
D
B C
E
F
A
D
B
CE
F
Time-space
Available code
• http://graphmod.ics.uci.edu/group/Software
SummaryType AND/OR Tree AND/OR graphStructure Pseudo tree Pseudo tree with
contextComputations Assign labels,
compute valuesAssign labels, compute values
Time Complexity O(n exp(h)) [max h is w*logn]
O(n exp(w*))
Space Complexity O(h) O(n exp(w*))Caching No Yes
Research issues
• Orderings– Heuristically Constructing a best pseudo tree
• Caching strategies• Exploiting Symmetries
– Exploiting Determinism and context-specific independence
• Approximations• Dynamic vs Static Search