up2date january 30th, 2009parallel.vub.ac.be/~jan/papers/introduction to bayesian...up2date...

Bayesian networks

Slide 1

ww

w.m

aste

rsin

inn

ova

tio

n.c

om

Confidential 30.01.2009

Bayesian Networks

Jan Lemeire

Department of Electronics and Informatics

Vrije Universiteit Brussel

[email protected]

www.vub.ac.be

UP2DATE – January 30th, 2009

www.mastersininnovation.com

Commercially confidence – This presentation contains ideas and information which are proprietary of VERHAERT, Masters in Innovation*, it is given in confidence. You are authorized to

open and view the electronic copy of this document and to print a single copy. Otherwise, the material may not in whole or in part be copied, stored electronically or communicated to third

parties without prior agreement of VERHAERT, Masters in Innovation*.

* VERHAERT, Masters in Innovation is a registered trade name of Verhaert Consultancies N.V.

http://www.qinetiq.com/index.htm

mailto:[email protected]

mailto:[email protected]

Bayesian networks

Slide 2

ww

w.m

aste

rsin

inn

ova

tio

n.c

om


Introduction: From Bayes’ rule to

Bayesian networks

The use of Bayesian networks

Learning from data

Comparison with other techniques

Conclusions

Overview


Bayesian networks

Slide 3

ww

w.m

aste

rsin

inn

ova

tio

n.c

om


Belief Update during Medical Diagnosis

Information P(cancer)

Prior belief 0.001

Short of breath (symptom) 0.002

Smoker 0.008

X-rays (clinical test) 0.04

It is not bronchitis 0.5


Bayesian networks

Slide 4

ww

w.m

aste

rsin

inn

ova

tio

n.c

om


Bayes’ rule

Thomas Bayes

P(cancer | smoker) = P(cancer). P(smoker |cancer)

P(smoker)

Prior belief New knowledgePosterior belief


Bayesian networks

Slide 5

ww

w.m

aste

rsin

inn

ova

tio

n.c

om


Types of questions

Probabilistic queries

"I've got a temperature of 101, I'm a 37-year-old

Male and my tongue feels kind of funny but I have

no headache. What's the chance that I've got

bubonic plague?"

Take decisions about tests

Utility of tests: which tests give me maximal

information

Take decisions about interventions

Explain things in terms of causal relations

How to represent this knowledge?

expert


Bayesian networks

Slide 6

ww

w.m

aste

rsin

inn

ova

tio

n.c

om


Bayesian Networks

Intuitive graphical representation expressing the relations among the

variables. Can be causal relations.

Plus probabilities attached to each node.

Visited

AsiaPollution Smoker

Tuberculosis Cancer Bronchitis

positive X-ray

resultsincreased total

serum calciumshort of breath

(Dyspnoea)

P(visited Asia)P(pollution)P(Smoker)P(tuberculosis | visited Asia)P(cancer |smoker , pollution)P(Bronchitis|smoker)…


Bayesian networks

Slide 7

ww

w.m

aste

rsin

inn

ova

tio

n.c

om

Confidential

Overview

30.01.2009


Bayesian networks


Learning from data


Conclusions


Bayesian networks

Slide 8

ww

w.m

aste

rsin

inn

ova

tio

n.c

om


Typical use of Bayesian networks

to model and explain a domain.

to update beliefs about states of certain variables when some other variables were observed,

e.g.: P(car breaks down | age of car = 16, changed oil = no).

≈ prediction

to find most probable configurations of variables

to support decision making under uncertainty (a Bayesian Network is a probabilistic model)

to find good strategies for solving tasks in a domain with uncertainty


Bayesian networks

Slide 9

ww

w.m

aste

rsin

inn

ova

tio

n.c

om

Confidential

Typical use of Bayesian networks II

30.01.2009

Smoking Cancer

Smoking Cancer

Smoking &

Cancer Gene

Explain things in terms of causal relations

E.g.: smoking causes lung cancer?

To answer qualitative questions

E.g.: if the national bank would lower interest rates, but the confidence remains low, would it help the economy?

or


Bayesian networks

Slide 10

ww

w.m

aste

rsin

inn

ova

tio

n.c

om


Dynamic Bayesian Networks

Models a dynamic system: the state at time t is affected by the state

at time t-1

Used in reliability analysis

Graphical Duration Models

Rail defect


Bayesian networks

Slide 11

ww

w.m

aste

rsin

inn

ova

tio

n.c

om


Examples

Norsys (http://www.norsys.com/ )

http://www.norsys.com/netlibrary/index.htm

Coronary Risk: a Bayesian Network to predict risk of Coronary Heart

Disease

Agricultural Yield

Car Diagnosis

Chest Clinic Decision: A graphical method for solving a decision

analysis problem

Oil Wildcatter Extended: decision network

Win95pts: An expert system for printer troubleshooting in Windows 95.


http://www.norsys.com/


Bayesian networks

Slide 12

ww

w.m

aste

rsin

inn

ova

tio

n.c

om

Confidential

Overview

30.01.2009


Bayesian networks


Learning from data


Conclusions


Bayesian networks

Slide 13

ww

w.m

aste

rsin

inn

ova

tio

n.c

om


Learning Bayesian networks

From data and expert knowledge

I. Parameterization (‘the probabilities’)

Based on assumptions on the relations

e.g. linear with Gaussian errors

When the structure is known


Bayesian networks

Slide 14

ww

w.m

aste

rsin

inn

ova

tio

n.c

om

Confidential

Learning Bayesian networks II

II. The structure of the graph

1. Find minimal model that best explains the data

Trade-off between goodness-of-fit and model complexity

2. Find model that explains the conditional independencies

A causal structure implies conditional independencies

30.01.2009

overfitting in regression

Causal structure is learned from data!!

Smoking Cancer positive X-ray

results


Bayesian networks

Slide 15

ww

w.m

aste

rsin

inn

ova

tio

n.c

om

Confidential


Bayesian networks


Learning from data


Conclusions

Overview

30.01.2009


Bayesian networks

Slide 16

ww

w.m

aste

rsin

inn

ova

tio

n.c

om

Confidential

Statistical Learning

Goal: learning from data, try to understand the underlying system

that generated the data

Supervised learning

30.01.2009

Classification of galaxies by Hubble telescope

learn to predict E from A, B, C & D


Bayesian networks

Slide 17

ww

w.m

aste

rsin

inn

ova

tio

n.c

om


Techniques for supervised learning

RegressionSupport Vector Machines

Neural network


Bayesian networks

Slide 18

ww

w.m

aste

rsin

inn

ova

tio

n.c

om

Confidential

Statistical Learning II

Unsupervised learning

there is no outcome measure

the goal is to describe the associations and patterns of the data

30.01.2009

Example: find types of

consumers

Cluster Analysis


Bayesian networks

Slide 19

ww

w.m

aste

rsin

inn

ova

tio

n.c

om

Confidential

Provide ‘black box’ models for the relation between the variables

that we know and the ones we want to predict.

They are usually better in that task.

Bayesian networks model the relations among all variables.

Bayesian networks provide insight.

Bayesian networks are not good in pattern recognition (e.g.

recognizing characters from a printed text).

30.01.2009

Techniques for statistical learning compared


Bayesian networks

Slide 20

ww

w.m

aste

rsin

inn

ova

tio

n.c

om

Confidential

Knowledge representation

30.01.2009

Expert Systems

Rule-based systems

Decision trees

A => BB & C => D

Example: Evaluating car insurance risks.


Bayesian networks

Slide 21

ww

w.m

aste

rsin

inn

ova

tio

n.c

om

Confidential

Knowledge representation II

30.01.2009

Fuzzy Logic Fuzzy Rules


Bayesian networks

Slide 22

ww

w.m

aste

rsin

inn

ova

tio

n.c

om

Confidential

Knowledge representation techniques compared

Bayesian networks can be regarded as the underlying (causal)

structure, from which (fuzzy) rules can be extracted

While the graph describes the relations among the variables, other

techniques describe the type and structure of the relations better .

30.01.2009

Digital Circuit


Bayesian networks

Slide 23

ww

w.m

aste

rsin

inn

ova

tio

n.c

om

Confidential

Overview

30.01.2009


Bayesian networks


Learning from data


Conclusions


Bayesian networks

Slide 24

ww

w.m

aste

rsin

inn

ova

tio

n.c

om


Conclusions

A lot of data is available nowadays + a lot computing power

Data mining and statistical learning becomes interesting

Academic researchers are in need of real-world data and problems!

A Bayesian network provides an intuitive graphical representation

Knowledge representation

Learning algorithms exist that can learn the models from data

Extract knowledge from data

Useful when the relations among the variables matter

Not when the (causal) relations are trivial


Bayesian networks

Slide 25

ww

w.m

aste

rsin

inn

ova

tio

n.c

om


References

My research: http://parallel.vub.ac.be => research => causal

inference

Norsys: http://www.norsys.com/

http://www.norsys.com/netlibrary/index.htm (examples)

tutorials

Bayesia: http://www.bayesia.com

Examples:

http://www.bayesia.com/en/products/bayesialab/resources.php

Bayesian Network Repository

http://compbio.cs.huji.ac.il/Repository/

Statistical Data Mining Tutorials:

http://www.autonlab.org/tutorials/


http://parallel.vub.ac.be/

http://www.norsys.com/


http://www.bayesia.com/

http://www.bayesia.com/en/products/bayesialab/resources.php

http://compbio.cs.huji.ac.il/Repository/

http://www.autonlab.org/tutorials/

Bayesian networks

Slide 26

ww

w.m

aste

rsin

inn

ova

tio

n.c

om


Verhaert New Products & Services nvHogenakkerhoekstraat 21

9150 Kruibeke

Belgium

Tel +32 (0)3 250 19 00

Fax +32 (0)3 254 10 08

www.verhaert.com

[email protected]

www.mastersininnovation.com


up2date january 30th, 2009parallel.vub.ac.be/~jan/papers/introduction to bayesian...up2date...

Documents