![Page 1: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship](https://reader035.vdocuments.mx/reader035/viewer/2022081515/56649d545503460f94a319ab/html5/thumbnails/1.jpg)
Graphical Models
Lei Tang
![Page 2: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship](https://reader035.vdocuments.mx/reader035/viewer/2022081515/56649d545503460f94a319ab/html5/thumbnails/2.jpg)
Review of Graphical Models
Directed Graph (DAG, Bayesian Network, Belief Network)
Typically used to represent causal relationship
Undirected Graph (Markov Random Field, Markov Network)
Usually when the relationship between variables are not very clear.
![Page 3: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship](https://reader035.vdocuments.mx/reader035/viewer/2022081515/56649d545503460f94a319ab/html5/thumbnails/3.jpg)
Some rules(1)
A graph to represent a regression problem Plate is used to represent repetition.
![Page 4: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship](https://reader035.vdocuments.mx/reader035/viewer/2022081515/56649d545503460f94a319ab/html5/thumbnails/4.jpg)
Some rules(2)
Suppose we have some parameters
Observations are shaded.
![Page 5: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship](https://reader035.vdocuments.mx/reader035/viewer/2022081515/56649d545503460f94a319ab/html5/thumbnails/5.jpg)
Model Representation (DAG)
Usually, the higher-numbered variables corresponds to terminal nodes of the graph, representing the observations; Lower-numbered nodes are latent variables.
A graph representing the naïve Bayes model.
![Page 6: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship](https://reader035.vdocuments.mx/reader035/viewer/2022081515/56649d545503460f94a319ab/html5/thumbnails/6.jpg)
Factorization
For directed graph:
(Ancestral Sampling) For undirected graph: Potential
Function
Partition Function
Energy Function
![Page 7: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship](https://reader035.vdocuments.mx/reader035/viewer/2022081515/56649d545503460f94a319ab/html5/thumbnails/7.jpg)
Directed-> Undirected Graph
![Page 8: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship](https://reader035.vdocuments.mx/reader035/viewer/2022081515/56649d545503460f94a319ab/html5/thumbnails/8.jpg)
Moralization (marrying the parents)
Moralization adds the fewest extra links but remains the maximum number of independence properties.
![Page 9: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship](https://reader035.vdocuments.mx/reader035/viewer/2022081515/56649d545503460f94a319ab/html5/thumbnails/9.jpg)
Perfect Map
Every independence property of the distribution is reflected in the graph and vice versa, then the graph is a perfect map.
Not all directed graph can not be represented as undirected graph. (As in previous example)
Not all undirected graph can be represented as directed graph.
![Page 10: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship](https://reader035.vdocuments.mx/reader035/viewer/2022081515/56649d545503460f94a319ab/html5/thumbnails/10.jpg)
Inference on a Chain(1)
N variables, each one has K states, then O(K^(N-1))
![Page 11: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship](https://reader035.vdocuments.mx/reader035/viewer/2022081515/56649d545503460f94a319ab/html5/thumbnails/11.jpg)
Inference on a Chain(2)
Complexity: O( KN)
![Page 12: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship](https://reader035.vdocuments.mx/reader035/viewer/2022081515/56649d545503460f94a319ab/html5/thumbnails/12.jpg)
Inference on a Chain(3)Message Passed forwards along the
chain
Message Passed backwards along
the chain
![Page 13: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship](https://reader035.vdocuments.mx/reader035/viewer/2022081515/56649d545503460f94a319ab/html5/thumbnails/13.jpg)
Inference on a Chain(4)
This message passing is more efficient to find the marginal distributions of all variables.
If some of the nodes in the graph are observed, then there is no summation for the corresponding variable.
If some parameters are not observed, apply EM algorithm (discussed later)
![Page 14: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship](https://reader035.vdocuments.mx/reader035/viewer/2022081515/56649d545503460f94a319ab/html5/thumbnails/14.jpg)
Factor Graph
We can apply similar strategy (message passing) to undirected/directed trees and polytrees as well.
Polytree is a tree that one node has two or more parents. In a factor graph, a node (circle) represents a variable,
and additional nodes (squares) represents a factor.
![Page 15: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship](https://reader035.vdocuments.mx/reader035/viewer/2022081515/56649d545503460f94a319ab/html5/thumbnails/15.jpg)
Factor Graph is not unique
![Page 16: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship](https://reader035.vdocuments.mx/reader035/viewer/2022081515/56649d545503460f94a319ab/html5/thumbnails/16.jpg)
A poly tree example
It is still a tree without loops!!
![Page 17: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship](https://reader035.vdocuments.mx/reader035/viewer/2022081515/56649d545503460f94a319ab/html5/thumbnails/17.jpg)
The sum-product algorithm
This algorithm is the same as belief propagation which is proposed for directed graphs without loops.
![Page 18: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship](https://reader035.vdocuments.mx/reader035/viewer/2022081515/56649d545503460f94a319ab/html5/thumbnails/18.jpg)
An intuitive Example
![Page 19: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship](https://reader035.vdocuments.mx/reader035/viewer/2022081515/56649d545503460f94a319ab/html5/thumbnails/19.jpg)
![Page 20: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship](https://reader035.vdocuments.mx/reader035/viewer/2022081515/56649d545503460f94a319ab/html5/thumbnails/20.jpg)
![Page 21: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship](https://reader035.vdocuments.mx/reader035/viewer/2022081515/56649d545503460f94a319ab/html5/thumbnails/21.jpg)
in General Graphs
Exact Inference: Junction tree algorithm. Inexact inference: No closed form for the distribution. Dimensionality of latent space is too high.