Upload File

graphical models lei tang. review of graphical models directed graph (dag, bayesian network, belief...

  • Home
  • Documents
  • Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship
21
Graphical Models Lei Tang

Post on 21-Dec-2015

222 views

Category:

Documents


2 download

Report

Tags:

TRANSCRIPT

Page 1: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship

Graphical Models

Lei Tang

Page 2: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship

Review of Graphical Models

Directed Graph (DAG, Bayesian Network, Belief Network)

Typically used to represent causal relationship

Undirected Graph (Markov Random Field, Markov Network)

Usually when the relationship between variables are not very clear.

Page 3: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship

Some rules(1)

A graph to represent a regression problem Plate is used to represent repetition.

Page 4: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship

Some rules(2)

Suppose we have some parameters

Observations are shaded.

Page 5: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship

Model Representation (DAG)

Usually, the higher-numbered variables corresponds to terminal nodes of the graph, representing the observations; Lower-numbered nodes are latent variables.

A graph representing the naïve Bayes model.

Page 6: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship

Factorization

For directed graph:

(Ancestral Sampling) For undirected graph: Potential

Function

Partition Function

Energy Function

Page 7: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship

Directed-> Undirected Graph

Page 8: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship

Moralization (marrying the parents)

Moralization adds the fewest extra links but remains the maximum number of independence properties.

Page 9: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship

Perfect Map

Every independence property of the distribution is reflected in the graph and vice versa, then the graph is a perfect map.

Not all directed graph can not be represented as undirected graph. (As in previous example)

Not all undirected graph can be represented as directed graph.

Page 10: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship

Inference on a Chain(1)

N variables, each one has K states, then O(K^(N-1))

Page 11: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship

Inference on a Chain(2)

Complexity: O( KN)

Page 12: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship

Inference on a Chain(3)Message Passed forwards along the

chain

Message Passed backwards along

the chain

Page 13: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship

Inference on a Chain(4)

This message passing is more efficient to find the marginal distributions of all variables.

If some of the nodes in the graph are observed, then there is no summation for the corresponding variable.

If some parameters are not observed, apply EM algorithm (discussed later)

Page 14: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship

Factor Graph

We can apply similar strategy (message passing) to undirected/directed trees and polytrees as well.

Polytree is a tree that one node has two or more parents. In a factor graph, a node (circle) represents a variable,

and additional nodes (squares) represents a factor.

Page 15: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship

Factor Graph is not unique

Page 16: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship

A poly tree example

It is still a tree without loops!!

Page 17: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship

The sum-product algorithm

This algorithm is the same as belief propagation which is proposed for directed graphs without loops.

Page 18: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship

An intuitive Example

Page 19: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship
Page 20: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship
Page 21: Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship

in General Graphs

Exact Inference: Junction tree algorithm. Inexact inference: No closed form for the distribution. Dimensionality of latent space is too high.


Object Segmentation and Tracking Based on Graphical … · 2018-01-09 · graphical probabilistic models (or graphical models), especially Bayesian networks ... 5.1 Bayesian network

Graphical Models - stats.ox.ac.uk

Network Models - University of Washington8 Network Models Many important optimization problems can best be analyzed by means of a graphical or network representation. In this chapter,

Directed Graphical Models

E6885 Network Science Lecture 11: Graphical … Network Science Lecture 11: Graphical Models and Analysis E 6885 Topics in Signal Processing -- Network Science Ching-Yung Lin, Dept

Rajagiri School of Engineering & TechnologyUndirected graphical models and directed graphical models. Undirected graphical models, also called Markov Random Fields (MRFs) or Markov

Graphical Causal Models

4. Probabilistic Graphical Models Directed Models - TUM › ... › mlcv16 › graphicalmodels.pdf · 2016-05-06 · 4. Probabilistic Graphical Models Directed Models. ... • In

MML, HYBRID BAYESIAN NETWORK GRAPHICAL …users.monash.edu/~dld/Publications/2010/Dowe2010... · MML, HYBRID BAYESIAN NETWORK GRAPHICAL MODELS, STATISTICAL CONSISTENCY, INVARIANCE

High-dimensional Gaussian graphical models on network ... · Graphical models are commonly used to represent independence relationships between random variables, with each variable

Probabilistic Graphical Models

Causal Network Models for Correlated Quantitative Traitspages.stat.wisc.edu › ~yandell › talk › jax › 2010 › 2010JaxNet.pdf · causal graphical models in systems genetics

Quantum Probabilistic Graphical Models for Cognition and ...cmoreira.eu/pdfs/thesis/PhDThesis.pdf · probabilistic graphical models. We also propose a Quantum-Like Bayesian Network

CSCI 252: Neural Networks and Graphical Models Fall … · CSCI 252: Neural Networks and Graphical Models Fall Term 2016 Prof. Levy Architecture #2: The Hopfield Network (Hopfield

Temporal Graphical Models for Cross-Species Gene ... · Temporal Graphical Models for Cross-Species Gene Regulatory Network Discovery ... data. Other methods have been proposed to

Statistical Methods for Machine Learning II Murat A ... · Modeling idea: graphical models on latent variables, neural network models for observations Composing graphical models with

Probabilistic Graphical Models - EPFL€¦ · Probabilistic Graphical Models Lecture 2: Graphical Models. Belief Propagation Volkan Cevher, Matthias Seeger Ecole Polytechnique Fédérale

- Relational - Graphical Models

01 graphical models

Statistical Machine Learning€¦ · Principal Component Analysis Autoencoders Graphical Models 1 Graphical Models 2 Graphical Models 3 Sampling Sequential Data 1 Sequential Data

Probabilistic graphical models...Probabilistic graphical models • The basic technical concepts behind probabilistic graphical models and how to work with them. • Applications in

Graphical Models - Inference -

Probabilistic Graphical Modelsassets.disi.unitn.it/uploads/doctoral_school/... · Introduction Probabilistic Graphical Models Need: probabilistic models that can represent models

Bayesian Graphical Models

Incomplete Graphical Models

Biological Network Inference via Gaussian Graphical Models

Poisson Graphical Models

Graphical network models for international financial flows - Economia

Graphical Models - Modeling -

Graphical Models 4dummies

Probabilistic Graphical Models - wnzhangwnzhang.net/teaching/cs420/slides/8-graphical-model.pdf · •Inferences in Graphical Models. A Simple Bayesian Network •One of the powerful

Compiling Graphical Models

Conditional Graphical Models for Protein Structure Predictionyanliu/file/draft.pdf · Conditional Graphical Models for Protein Structure Prediction Yan Liu ... 3 Graphical models

Banerjee Graphical Models

Network Models - WordPress.com€¦ · 08.02.2013 · Network Models Many important optimization problems can best be analyzed by means of a graphical or network representation. In