1

Inferring structure to make substantive conclusions:How does it work?

Hypothesis testing approaches:

Tests on deviances, possibly penalised (AIC/BIC, etc.), MDL, cross-validation...

Problem is how to search model space when dimension is large

2

Inferring structure to make substantive conclusions:How does it work? Bayesian approaches:

Typically place prior on all graphs, and conjugate prior on parameters (hyper-Markov laws, Dawid & Lauritzen), then use MCMC to update both graphs and parameters to simulate posterior distribution

3

Graph moves

Giudici & Green (Biometrika, 1999) develop a full Bayesian methodology for model selection in Gaussian models, assuming

decomposability

(= graph triangulated

= no chordless

-cycles)

7 6 5

2 3 414

4

• How many graphs are decomposable?

• Models using decomposable graphs are ‘dense’

Number ofvertices

Proportion of graphsthat aredecomposable

3 all 4 61/64 – all but: 6 ~80% 16 ~45%

Is decomposability a serious constraint?

22n

out of

5

Is decomposability any use?

• Maximum likelihood estimates can be computed exactly in decomposable models

• Decomposability is a key to the ‘message passing’ algorithms for probabilistic expert systems (and peeling genetic pedigrees)

lj

jkllijijkl n

nnNE

)(ˆ1 2

4 3

6

Graph moves

We can traverse graph space by adding and deleting single edges

Some are OK,but others makegraphnon-decomposable

7 6 5

2 3 41

7

Graph moves

Frydenberg & Lauritzen (1989) showed that all decomposable graphs are connected by single-edge moves

Can we test formaintaining decomposabilitybefore committing tomaking the change?

7 6 5

2 3 41

8

Cliques

A clique is a maximal complete subgraph: here the cliques are {1,2},{2,6,7}, {2,3,6}, and {3,4,5,6}

7 6 5

2 3 41

9

Deleting edges?

Deleting an edge maintains decomposability if and only if it is contained in exactly one clique of the current graph (Frydenberg & Lauritzen)

7 6 5

2 3 41

10

7 6 5

2 3 41

12

267 236 345626 36

2

a cliqueanother cliquea separator

The running intersection property:For any 2 cliques C and D, CD is a subset of every node between them in the junction tree

A graph is decomposableif and only if it can be represented by ajunction tree (which isnot unique)

11

7 6 5

2 3 41

12

267 236 345626 36

2

a cliqueanother cliquea separator

A graph is decomposableif and only if it can be represented by ajunction tree (which isnot unique)

The running intersection property:For any 2 cliques C and D, CD is a subset of every node between them in the junction tree

12

Non-uniquenessof junction tree

7 6 5

2 3 41

12

267 236 345626 36

2

13

7 6 5

2 3 41

12

267 236 345626 36

2

12

2

Non-uniquenessof junction tree

14

Adding edges? (Giudici & Green)

Adding an edge (a,b) maintains decomposability if and only if either:

7 6 5

2 3 41

• there exist sets R and T such that aR and bT are cliques and RT is a separator on the path in the junction tree between them

• a and b are in different connected components, or

15

You can add edge (1,7) since 1R and 7T are cliques (with R={2} and T={2,6}) and RT={2} is a separator on path between them

7 6 5

2 3 41

12

267 236 345626 36

2

16

You cannot add edge (1,4) since the only cliques containing 1 and 4 resp. are {1,2} and {3,4,5,6}, and {2}{3,5,6} is not a separator on path between them

7 6 5

2 3 41

12

267 236 345626 36

2

17

Adding edges? (Giudici & Green)

Adding an edge (a,b) maintains decomposability if and only if either:

7 6 5

2 3 41

• there exist sets R and T such that aR and bT are cliques and RT is a separator on the path in the junction tree between them

• a and b are in different connected components, or

18

Proof (in connected case)

First suppose that there are no such sets R and T. We have to show that adding edge (a,b) makes graph non-deomposable.Let aR and bT be the cliques containing a and b that have shortest connecting path in the junction tree: by assumption, RT is not a separator (it may be empty): so all separators on the path are proper supersets of RT. So there is a shortest path in the original graph: arv1...vktb with k0, rR\T, tT\R and all v’s RT. Joining (a,b) will make a chordless (k+4)-cycle, making the graph non-decomposable.

19

You cannot add edge (1,4) since the only cliques containing 1 and 4 resp. are {1,2} and {3,4,5,6}, and {2}{3,5,6} is not a separator on path between them

7 6 5

2 3 41

12

267 236 345626 36

2

20

Proof (in connected case)

S bSaS

abS

Conversely, suppose such sets R and T do exist.We can suppose aR and bT are adjacent in the junction tree (otherwise it is quite easy to show that the junction tree can be manipulated until this is true).Let S=RT, P=R\T and Q=T\R. There are 4 cases according to whether P and Q are empty or not.

Both P and Q empty:

(it is easy to see that you still have a tree & that running intersection property is maintained)

21

S bSaSP

abS bS bSQaSaSP

S bSQaS

S bSQaSP

aS abSaSP bS bSQabS

Neither P nor Q empty:

Only Q empty: Only P empty:

22

Once the test is complete, actually committing to adding or deleting the edge is little work

7 6 5

2 3 41

12

267 236 345626 36

2

23

7 6 5

2 3 41

127

267 236 345626 36

27

12

2

It makes onlya (relatively)local change to the junction tree

Once the test is complete, actually committing to adding or deleting the edge is little work

24

7 6 5

2 3 41

127

267 236 345626 36

27

It makes onlya (relatively)local change to the junction tree

6

Once the test is complete, actually committing to adding or deleting the edge is little work

26

Once the test is complete, actually committing to adding or deleting the edge is little work

7 6 5

2 3 4

It makes onlya (relatively)local change to the junction tree

127

267 236 35626 36

27

345

35

1

The End