![Page 1: Inferring structure to make substantive conclusions: How does it work?](https://reader036.vdocuments.mx/reader036/viewer/2022070407/568142ec550346895daf490a/html5/thumbnails/1.jpg)
1
Inferring structure to make substantive conclusions:How does it work?
Hypothesis testing approaches:
Tests on deviances, possibly penalised (AIC/BIC, etc.), MDL, cross-validation...
Problem is how to search model space when dimension is large
![Page 2: Inferring structure to make substantive conclusions: How does it work?](https://reader036.vdocuments.mx/reader036/viewer/2022070407/568142ec550346895daf490a/html5/thumbnails/2.jpg)
2
Inferring structure to make substantive conclusions:How does it work? Bayesian approaches:
Typically place prior on all graphs, and conjugate prior on parameters (hyper-Markov laws, Dawid & Lauritzen), then use MCMC to update both graphs and parameters to simulate posterior distribution
![Page 3: Inferring structure to make substantive conclusions: How does it work?](https://reader036.vdocuments.mx/reader036/viewer/2022070407/568142ec550346895daf490a/html5/thumbnails/3.jpg)
3
Graph moves
Giudici & Green (Biometrika, 1999) develop a full Bayesian methodology for model selection in Gaussian models, assuming
decomposability
(= graph triangulated
= no chordless
-cycles)
7 6 5
2 3 414
![Page 4: Inferring structure to make substantive conclusions: How does it work?](https://reader036.vdocuments.mx/reader036/viewer/2022070407/568142ec550346895daf490a/html5/thumbnails/4.jpg)
4
• How many graphs are decomposable?
• Models using decomposable graphs are ‘dense’
Number ofvertices
Proportion of graphsthat aredecomposable
3 all 4 61/64 – all but: 6 ~80% 16 ~45%
Is decomposability a serious constraint?
22n
out of
![Page 5: Inferring structure to make substantive conclusions: How does it work?](https://reader036.vdocuments.mx/reader036/viewer/2022070407/568142ec550346895daf490a/html5/thumbnails/5.jpg)
5
Is decomposability any use?
• Maximum likelihood estimates can be computed exactly in decomposable models
• Decomposability is a key to the ‘message passing’ algorithms for probabilistic expert systems (and peeling genetic pedigrees)
lj
jkllijijkl n
nnNE
)(ˆ1 2
4 3
![Page 6: Inferring structure to make substantive conclusions: How does it work?](https://reader036.vdocuments.mx/reader036/viewer/2022070407/568142ec550346895daf490a/html5/thumbnails/6.jpg)
6
Graph moves
We can traverse graph space by adding and deleting single edges
Some are OK,but others makegraphnon-decomposable
7 6 5
2 3 41
![Page 7: Inferring structure to make substantive conclusions: How does it work?](https://reader036.vdocuments.mx/reader036/viewer/2022070407/568142ec550346895daf490a/html5/thumbnails/7.jpg)
7
Graph moves
Frydenberg & Lauritzen (1989) showed that all decomposable graphs are connected by single-edge moves
Can we test formaintaining decomposabilitybefore committing tomaking the change?
7 6 5
2 3 41
![Page 8: Inferring structure to make substantive conclusions: How does it work?](https://reader036.vdocuments.mx/reader036/viewer/2022070407/568142ec550346895daf490a/html5/thumbnails/8.jpg)
8
Cliques
A clique is a maximal complete subgraph: here the cliques are {1,2},{2,6,7}, {2,3,6}, and {3,4,5,6}
7 6 5
2 3 41
![Page 9: Inferring structure to make substantive conclusions: How does it work?](https://reader036.vdocuments.mx/reader036/viewer/2022070407/568142ec550346895daf490a/html5/thumbnails/9.jpg)
9
Deleting edges?
Deleting an edge maintains decomposability if and only if it is contained in exactly one clique of the current graph (Frydenberg & Lauritzen)
7 6 5
2 3 41
![Page 10: Inferring structure to make substantive conclusions: How does it work?](https://reader036.vdocuments.mx/reader036/viewer/2022070407/568142ec550346895daf490a/html5/thumbnails/10.jpg)
10
7 6 5
2 3 41
12
267 236 345626 36
2
a cliqueanother cliquea separator
The running intersection property:For any 2 cliques C and D, CD is a subset of every node between them in the junction tree
A graph is decomposableif and only if it can be represented by ajunction tree (which isnot unique)
![Page 11: Inferring structure to make substantive conclusions: How does it work?](https://reader036.vdocuments.mx/reader036/viewer/2022070407/568142ec550346895daf490a/html5/thumbnails/11.jpg)
11
7 6 5
2 3 41
12
267 236 345626 36
2
a cliqueanother cliquea separator
A graph is decomposableif and only if it can be represented by ajunction tree (which isnot unique)
The running intersection property:For any 2 cliques C and D, CD is a subset of every node between them in the junction tree
![Page 12: Inferring structure to make substantive conclusions: How does it work?](https://reader036.vdocuments.mx/reader036/viewer/2022070407/568142ec550346895daf490a/html5/thumbnails/12.jpg)
12
Non-uniquenessof junction tree
7 6 5
2 3 41
12
267 236 345626 36
2
![Page 13: Inferring structure to make substantive conclusions: How does it work?](https://reader036.vdocuments.mx/reader036/viewer/2022070407/568142ec550346895daf490a/html5/thumbnails/13.jpg)
13
7 6 5
2 3 41
12
267 236 345626 36
2
12
2
Non-uniquenessof junction tree
![Page 14: Inferring structure to make substantive conclusions: How does it work?](https://reader036.vdocuments.mx/reader036/viewer/2022070407/568142ec550346895daf490a/html5/thumbnails/14.jpg)
14
Adding edges? (Giudici & Green)
Adding an edge (a,b) maintains decomposability if and only if either:
7 6 5
2 3 41
• there exist sets R and T such that aR and bT are cliques and RT is a separator on the path in the junction tree between them
• a and b are in different connected components, or
![Page 15: Inferring structure to make substantive conclusions: How does it work?](https://reader036.vdocuments.mx/reader036/viewer/2022070407/568142ec550346895daf490a/html5/thumbnails/15.jpg)
15
You can add edge (1,7) since 1R and 7T are cliques (with R={2} and T={2,6}) and RT={2} is a separator on path between them
7 6 5
2 3 41
12
267 236 345626 36
2
![Page 16: Inferring structure to make substantive conclusions: How does it work?](https://reader036.vdocuments.mx/reader036/viewer/2022070407/568142ec550346895daf490a/html5/thumbnails/16.jpg)
16
You cannot add edge (1,4) since the only cliques containing 1 and 4 resp. are {1,2} and {3,4,5,6}, and {2}{3,5,6} is not a separator on path between them
7 6 5
2 3 41
12
267 236 345626 36
2
![Page 17: Inferring structure to make substantive conclusions: How does it work?](https://reader036.vdocuments.mx/reader036/viewer/2022070407/568142ec550346895daf490a/html5/thumbnails/17.jpg)
17
Adding edges? (Giudici & Green)
Adding an edge (a,b) maintains decomposability if and only if either:
7 6 5
2 3 41
• there exist sets R and T such that aR and bT are cliques and RT is a separator on the path in the junction tree between them
• a and b are in different connected components, or
![Page 18: Inferring structure to make substantive conclusions: How does it work?](https://reader036.vdocuments.mx/reader036/viewer/2022070407/568142ec550346895daf490a/html5/thumbnails/18.jpg)
18
Proof (in connected case)
First suppose that there are no such sets R and T. We have to show that adding edge (a,b) makes graph non-deomposable.Let aR and bT be the cliques containing a and b that have shortest connecting path in the junction tree: by assumption, RT is not a separator (it may be empty): so all separators on the path are proper supersets of RT. So there is a shortest path in the original graph: arv1...vktb with k0, rR\T, tT\R and all v’s RT. Joining (a,b) will make a chordless (k+4)-cycle, making the graph non-decomposable.
![Page 19: Inferring structure to make substantive conclusions: How does it work?](https://reader036.vdocuments.mx/reader036/viewer/2022070407/568142ec550346895daf490a/html5/thumbnails/19.jpg)
19
You cannot add edge (1,4) since the only cliques containing 1 and 4 resp. are {1,2} and {3,4,5,6}, and {2}{3,5,6} is not a separator on path between them
7 6 5
2 3 41
12
267 236 345626 36
2
![Page 20: Inferring structure to make substantive conclusions: How does it work?](https://reader036.vdocuments.mx/reader036/viewer/2022070407/568142ec550346895daf490a/html5/thumbnails/20.jpg)
20
Proof (in connected case)
S bSaS
abS
Conversely, suppose such sets R and T do exist.We can suppose aR and bT are adjacent in the junction tree (otherwise it is quite easy to show that the junction tree can be manipulated until this is true).Let S=RT, P=R\T and Q=T\R. There are 4 cases according to whether P and Q are empty or not.
Both P and Q empty:
(it is easy to see that you still have a tree & that running intersection property is maintained)
![Page 21: Inferring structure to make substantive conclusions: How does it work?](https://reader036.vdocuments.mx/reader036/viewer/2022070407/568142ec550346895daf490a/html5/thumbnails/21.jpg)
21
S bSaSP
abS bS bSQaSaSP
S bSQaS
S bSQaSP
aS abSaSP bS bSQabS
Neither P nor Q empty:
Only Q empty: Only P empty:
![Page 22: Inferring structure to make substantive conclusions: How does it work?](https://reader036.vdocuments.mx/reader036/viewer/2022070407/568142ec550346895daf490a/html5/thumbnails/22.jpg)
22
Once the test is complete, actually committing to adding or deleting the edge is little work
7 6 5
2 3 41
12
267 236 345626 36
2
![Page 23: Inferring structure to make substantive conclusions: How does it work?](https://reader036.vdocuments.mx/reader036/viewer/2022070407/568142ec550346895daf490a/html5/thumbnails/23.jpg)
23
7 6 5
2 3 41
127
267 236 345626 36
27
12
2
It makes onlya (relatively)local change to the junction tree
Once the test is complete, actually committing to adding or deleting the edge is little work
![Page 24: Inferring structure to make substantive conclusions: How does it work?](https://reader036.vdocuments.mx/reader036/viewer/2022070407/568142ec550346895daf490a/html5/thumbnails/24.jpg)
24
7 6 5
2 3 41
127
267 236 345626 36
27
It makes onlya (relatively)local change to the junction tree
6
Once the test is complete, actually committing to adding or deleting the edge is little work
![Page 25: Inferring structure to make substantive conclusions: How does it work?](https://reader036.vdocuments.mx/reader036/viewer/2022070407/568142ec550346895daf490a/html5/thumbnails/25.jpg)
26
Once the test is complete, actually committing to adding or deleting the edge is little work
7 6 5
2 3 4
It makes onlya (relatively)local change to the junction tree
127
267 236 35626 36
27
345
35
1
The End