Download - URKS Uncertainty Reasoning in Knowledge Systems D. Bollé H. Bruyninckx M. Nuttin D. De Schreye
URKSURKS
Uncertainty ReasoningUncertainty Reasoning
in Knowledge Systemsin Knowledge Systems
D. BolléD. Bollé
H. BruyninckxH. Bruyninckx
M. NuttinM. Nuttin
D. De SchreyeD. De Schreye
2
Course overview:Course overview:
Introduction on uncertainty in A.I. Introduction on uncertainty in A.I. Motivation - examples - some basic conceptsMotivation - examples - some basic concepts
D. De Schreye (1 session)D. De Schreye (1 session)
Probability concepts, techniques and systemsProbability concepts, techniques and systems Bayesian theory and approachBayesian theory and approach
D. Bollé and H. Bruyninckx (+/-5 sessions)D. Bollé and H. Bruyninckx (+/-5 sessions) Fuzzy concepts, techniques and systemsFuzzy concepts, techniques and systems
Zadeh theory and approach/possibility theoryZadeh theory and approach/possibility theory M. Nuttin (+/- 6 sessions)M. Nuttin (+/- 6 sessions)
Comparison and question sessionComparison and question session
3
Practical sessions:Practical sessions:
5 practical (hands-on) sessions of 2.5 hours:5 practical (hands-on) sessions of 2.5 hours:
firstfirst: introduction to matlab: introduction to matlab 2 sessions on probability examples2 sessions on probability examples 2 sessions on fuzzy examples 2 sessions on fuzzy examples (second is pen-(second is pen-
paper)paper) examination: exercises related to lab-sessions examination: exercises related to lab-sessions
(1/2 of points)(1/2 of points)open bookopen book
remainder: written theory exam.remainder: written theory exam. closed bookclosed book
Http://www.cs.kuleuven.ac.be/~dannyd/URKSHttp://www.cs.kuleuven.ac.be/~dannyd/URKS
5
fuzzyfuzzy
uncertainuncertain
possiblepossible
probableprobable
likelylikely
assumingassuming
vaguevague
impreciseimprecise
approximatingapproximating
Levels of certainty on the uncertainLevels of certainty on the uncertain
6
Overview:Overview:ContentsContents ContentsContents
Inference under uncertainty:Inference under uncertainty: Abductive reasoning Abductive reasoning Probabilistic reasoningProbabilistic reasoning GenInferGenInfer Opinion Nets Opinion Nets
Sources of Uncertainty.Sources of Uncertainty.
What is a probability?What is a probability?
Diagnosis and weak implications.Diagnosis and weak implications.
Quantification types for uncertainty.Quantification types for uncertainty.
Prior versus Conditional probability.Prior versus Conditional probability.
Introducing Fuzzy sets.Introducing Fuzzy sets.
Joint probabilistic distributions.Joint probabilistic distributions. Motivating Bayes ruleMotivating Bayes rule
Axioms of probability.Axioms of probability.
Probabilistic rules.Probabilistic rules.
Utility and decision theory.Utility and decision theory.
7
Sources of uncertainty:Sources of uncertainty:
1. Information obtained may take the form of1. Information obtained may take the form of weak implicationsweak implications
Ex.:Ex.: in diagnosis:in diagnosis:
disease(p, Cavity) disease(p, Cavity) (0.8)(0.8) symptom(p, Toothache) symptom(p, Toothache)
quantification of frequency quantification of frequency with which the rule applieswith which the rule applies
2. Imprecise language:2. Imprecise language:
““often”, “sometimes”, “frequently”, “hardly ever”, …often”, “sometimes”, “frequently”, “hardly ever”, …
- need to quantify these in terms of frequency,- need to quantify these in terms of frequency,
- need to deal with proposed frequency in rules.- need to deal with proposed frequency in rules.
8
Study of relation between imprecise Study of relation between imprecise language and frequency:language and frequency:
9
Sources of uncertainty (2):Sources of uncertainty (2):
We observe that We observe that grass_is_wetgrass_is_wet, but have no , but have no information on information on sprinklersprinkler nor nor rainrain.. How to reason?How to reason?
3. Unknown information.3. Unknown information.
grass_is_wet grass_is_wet sprinkler_was_on sprinkler_was_ongrass_is_wet grass_is_wet rain_last_night rain_last_night
quantification of possible conclusionsquantification of possible conclusions
4. Conflicting information.4. Conflicting information.
Ex.:Ex.: Several experts have provided conflicting Several experts have provided conflicting information:information:
quantification of measure of beliefquantification of measure of belief
Abductive reasoningAbductive reasoning Note: can be Note: can be “ranges“ranges”” of unknown, depending of unknown, depending
on on additional evidence.additional evidence.
10
Sources of uncertainty (3):Sources of uncertainty (3):5. Vague Concepts:5. Vague Concepts:
Herman is tall. Herman is tall.
What is Herman’s height?What is Herman’s height?
- at least - at least 1.80 m1.80 m??- could Herman be - could Herman be 1.78 m1.78 m and still tall? and still tall?- if Herman is in the population of Basketball players- if Herman is in the population of Basketball players
is Herman still tall?is Herman still tall?- if Herman would be a kid of 9 years and - if Herman would be a kid of 9 years and 1.45 m1.45 m
is Herman also tall?is Herman also tall?
We may want to quantify the We may want to quantify the degree in which Herman degree in which Herman belongs to the set of belongs to the set of ‘tall_people’‘tall_people’
11
Sources of uncertainty (4):Sources of uncertainty (4):6. Precise specifications may be too complex:6. Precise specifications may be too complex:
Plan_90: leave 90 minutes before departure Plan_90: leave 90 minutes before departure
ProblemProblem: will Plan_90 succeed?: will Plan_90 succeed?again depends on unknown again depends on unknown information (traffic jam?, information (traffic jam?, earthquake?, accident?)earthquake?, accident?)
BUTBUT: enumeration of all conditions may be impossible or impractical:: enumeration of all conditions may be impossible or impractical:
Succeed(Plan_90) Succeed(Plan_90) not(car_break_down) not(car_break_down) andand
not(out_of_gas) andnot(out_of_gas) and not(accident) andnot(accident) and not(traffic_jam) andnot(traffic_jam) and ... ...
Quantification of estimated degree of success Quantification of estimated degree of success instead of specification of all conditionsinstead of specification of all conditions
12
Sources of uncertainty (5):Sources of uncertainty (5):7. Propagation of uncertain information:7. Propagation of uncertain information:
Tomorrow(cold) Tomorrow(cold) (0.7)(0.7)Tomorrow(rain)Tomorrow(rain) (0.6)(0.6)
Can we easily determine the uncertainty for:Can we easily determine the uncertainty for:
Tomorrow(cold) Tomorrow(cold) Tomorrow(rain) Tomorrow(rain) (?)(?)
Or for:Or for:
Tomorrow(cold) Tomorrow(cold) Tomorrow(rain) Tomorrow(rain) (?)(?)
Not without sufficient information on the Not without sufficient information on the interdependenciesinterdependencies of the events! of the events!
In In absence of interdependenciesabsence of interdependencies: propagation : propagation of uncertain knowledge of uncertain knowledge increases uncertaintyincreases uncertainty of the conclusionsof the conclusions
13
Utility theory:Utility theory:Plan_90: Plan_90: leave 90 minutes before departure leave 90 minutes before departure Plan_120: Plan_120: leave 120 minutes …leave 120 minutes …Plan_1440: Plan_1440: leave 24 hours ...leave 24 hours ...
Assume that Assume that Plan_90Plan_90 is the right thing to do: is the right thing to do: whatwhat would this mean?would this mean?
Plan_120 is Plan_120 is more likelymore likely to succeed. to succeed.Plan_1440 is Plan_1440 is practically surepractically sure to succeed. to succeed.BUTBUT:: Plan_90 attempts to Plan_90 attempts to optimize all our goalsoptimize all our goals::
- - arrive on time for the flightarrive on time for the flight- avoid long waiting at the airport- avoid long waiting at the airport- avoid getting speeding tickets for the - avoid getting speeding tickets for the
drivedrive- ...- ...
Utility theoryUtility theory is used to is used to representrepresent and and reason reason about preferences.about preferences.
14
Decision theory:Decision theory:
If we have expressed preferences using If we have expressed preferences using Utility Utility TheoryTheory and we have expressed probabilities of and we have expressed probabilities of events and effects by events and effects by Probability TheoryProbability Theory::
Decision theory Decision theory == Probability theoryProbability theory ++ Utility theoryUtility theory
A system is A system is rationalrationalif it chooses the action that yields the if it chooses the action that yields the
highest highest expected utilityexpected utility, averaged over all , averaged over all probabilities of outcomesprobabilities of outcomes of the of the
action.action.
15
More on diagnosis andMore on diagnosis andweak implications:weak implications:
Is it possible to capture diagnosis in hard rules ?Is it possible to capture diagnosis in hard rules ?
disease(p, Cavity) disease(p, Cavity) symptom(p, Toothache) symptom(p, Toothache)
is simply wrongis simply wrong there may be other diseases !there may be other diseases !
disease(p, Cavity) disease(p, Cavity) disease(p, GumDisease) disease(p, GumDisease) disease(p, ImpactedWisdom) disease(p, ImpactedWisdom) … …
symptom(p, Toothache) symptom(p, Toothache)
Again enumeration problem: do we know them all ?Again enumeration problem: do we know them all ?
symptom(p, Toothache) symptom(p, Toothache) disease(p, Cavity) disease(p, Cavity)
wrong againwrong again not each cavity causes pain !not each cavity causes pain !
There isThere is no correct logical rule. no correct logical rule.
The best we can do is provideThe best we can do is provide “ “a quantification of beliefa quantification of belief””
16
Degree of membership:Degree of membership:
What kind of quantifications?What kind of quantifications?
Basic distinctionBasic distinction Degree of beliefDegree of belief
Degree of membershipDegree of membership
Degree of belief:Degree of belief:
Probability (respecting the axioms of Probability (respecting the axioms of probability theory)probability theory)
Given Toothache, there is 80% chance of CavityGiven Toothache, there is 80% chance of Cavity
Certainty factors (don’t respect the Certainty factors (don’t respect the axioms)axioms)Toothache gives Cavity with factor 0.7 (in [-1,1])Toothache gives Cavity with factor 0.7 (in [-1,1])
Fuzzy Logic (measures in vague Fuzzy Logic (measures in vague concepts)concepts)
Herman is tall (with 95% measure of Herman is tall (with 95% measure of belonging to the set of ‘tall people’)belonging to the set of ‘tall people’)
17
What is a probability?What is a probability?
An often used intuition: An often used intuition: countingcounting
P(A)P(A) = number in the range [0,1] expressing the = number in the range [0,1] expressing the degree of belief in A.degree of belief in A.
P( randomly chosen person is Chinese) P( randomly chosen person is Chinese) = ?= ?all_peopleall_people
ChineseChinese
#(Chinese)#(Chinese) #(all_people)#(all_people)==
Interesting intuition to verify basic axioms and rules of probabilityInteresting intuition to verify basic axioms and rules of probability BUTBUT: counting is not always possible, nor desirable. : counting is not always possible, nor desirable.
18
But even more often:But even more often:
What is a probability? (2)What is a probability? (2) Statistics may help:Statistics may help:
Crucial is:Crucial is: Belief and probability Belief and probability changes with new gathered changes with new gathered
information !information !
Prior probabilityPrior probability Conditional probabilityConditional probability
Count a randomlyCount a randomly selected subsetselected subset of the populationof the population determinedetermine the ratiothe ratio (e.g.: of Chinese) from this subset(e.g.: of Chinese) from this subset
taken as the probabilitytaken as the probability
A general measure of A general measure of beliefbelief on the basis ofon the basis of prior experienceprior experience intuitionintuition ... ...
19
Prior versus Conditional:Prior versus Conditional:
The Chinese student storyThe Chinese student story The Bombed plane storyThe Bombed plane story
Prior Probability:Prior Probability: A A = = a randomly chosen student in some classroom is a randomly chosen student in some classroom is ChineseChinese P(P(AA)) = = 1/61/6
Conditional Probability:Conditional Probability: add information:add information: B B = = the chosen classroom is in the chosen classroom is in K.U.LeuvenK.U.Leuven P(P(A | BA | B)) = = 1/601/60
add information:add information: C C = = the classroom is from MAIthe classroom is from MAI
P(P(A | B A | B C C)) = = 1/41/4
20
Prior versus Conditional (2):Prior versus Conditional (2):The Bombed plane storyThe Bombed plane story
Prior Probability:Prior Probability:
A A = = there is a bomb on boardthere is a bomb on board P(P(AA)) = = 1/101/1066
Conditional Probability:Conditional Probability: add information:add information: A’ A’ = = there is (independently) there is (independently) anotheranother bomb on board bomb on board P(P(A | A’A | A’)) = = 1/ 101/ 101212
change A’:change A’: B B = = I bring a second bomb myselfI bring a second bomb myself
P(P(A | BA | B)) = = 1/ 101/ 1066
Prior probability (Prior probability (AA):): probability of probability of AA in absence of any other informationin absence of any other information
Conditional Probability (Conditional Probability (AA||BB):): probability of probability of AA given that we already knowgiven that we already know BB
21
Toothache example:Toothache example:P(Cavity) = 0.1P(Cavity) = 0.1 10% of all individuals have a Cavity10% of all individuals have a Cavity
P(Toothache) = 0.05P(Toothache) = 0.05 5% have a Toothache5% have a Toothache
P(Cavity|Toothache) = 0.8P(Cavity|Toothache) = 0.8given that we know the individual has given that we know the individual has Toothache, there is 80% chance of him having Toothache, there is 80% chance of him having CavityCavityP(Toothache|Cavity) = 0.4P(Toothache|Cavity) = 0.4conditional probability is NOT symmetricconditional probability is NOT symmetric
P(Cavity|Toothache P(Cavity|Toothache not Gumdisease) = 0.9 not Gumdisease) = 0.9additionally given that another diagnosis is additionally given that another diagnosis is already excluded, conditional probability already excluded, conditional probability increasesincreasesP(Cavity|Toothache P(Cavity|Toothache FalseTeeth) = 0 FalseTeeth) = 0adding information does not necessarily increase adding information does not necessarily increase the probabilitythe probability
22
A sensible semantics for A sensible semantics for probabilistic rules:probabilistic rules:
A probabilistic ruleA probabilistic rule
A A (factor)(factor) B1 B1 B2 B2 … … Bn Bn
should best have the semantics:should best have the semantics:
P(A| B1 P(A| B1 B2 B2 … … Bn) = Bn) = factorfactor
So:So: we simply have alternative syntax:we simply have alternative syntax:
disease(p, Cavity) disease(p, Cavity) (0.8)(0.8) symptom(p, Toothache) symptom(p, Toothache)
P(Cavity|Toothache) = P(Cavity|Toothache) = 0.80.8
23
But this is NOT standard at all !But this is NOT standard at all !In probabilistic Logic Programming In probabilistic Logic Programming (Subramanian et al.)(Subramanian et al.)
A A [n1,n2][n1,n2] B B [m1,m2][m1,m2]
Means:Means: IfIf the probability that in some possible the probability that in some possible world world BB is true is between is true is between m1m1 and and m2m2
ThenThen the probability that in some the probability that in some world world AA is is true is between true is between n1n1 and and nn22
Example:Example: flip a coin, with eventsflip a coin, with events AA == headhead,, BB == tailtail
head head [0.5,0.5] [0.5,0.5] tail tail [0.5,0.5][0.5,0.5]
IfIf the probability of a world in which you get the probability of a world in which you get tailtail is is 0.50.5 ThenThen the probability of a world in which you get the probability of a world in which you get headhead is also is also 0.50.5
Notice: no world in the intersection !Notice: no world in the intersection !
In conditional probabilities semantics:In conditional probabilities semantics: head head 0.0 0.0 tail tail
24
The axioms of ProbabilityThe axioms of Probability
1.1. 0 0 P(A) P(A) 1 1
2.2. P(True) = 1 , P(False) = 0 P(True) = 1 , P(False) = 0
3.3. P(A P(A B) = P(A) + P(B) - P(A B) = P(A) + P(B) - P(A B) B)
AA BBA A B B
Derived: Derived: P(P(A) = 1 - P(A)A) = 1 - P(A)
Are the major difference with Are the major difference with “certainty factor”“certainty factor” systems: systems: do NOT respect these axioms do NOT respect these axioms (Mycin: factors in range -1, 1)(Mycin: factors in range -1, 1)
25
Joint probability distributionJoint probability distribution Given Given 2 properties/events2 properties/events: list the entire distribution of : list the entire distribution of all all
probability assignmentsprobability assignments to all possible combinations of to all possible combinations of truth-values for the properties/eventstruth-values for the properties/events
ToothacheToothache ToothacheToothache
CavityCavity
CavityCavity
0.040.04
0.010.01
0.060.06
0.890.89
All prior and conditional probabilities can be derived ! All prior and conditional probabilities can be derived !
P(Toothache|Cavity) = P(Toothache|Cavity) = 0.04 / 0.04 + 0.06 = 0.04 / 0.04 + 0.06 = 0.40.4
BUTBUT: gathering this distribution is often not possible or at least very tedious.: gathering this distribution is often not possible or at least very tedious.
Bayes Rule !Bayes Rule !
26
Inference under Uncertainty.Inference under Uncertainty.The The logic versionlogic version of the burglary-alarm example: of the burglary-alarm example:
burglaryburglary earthquakeearthquake
alarmalarm
MaryCallsMaryCallsJohnCallsJohnCalls
phoneRingsphoneRings
unlessLoudmusicunlessLoudmusic
In Logic:In Logic:
JohnCalls JohnCalls Alarm AlarmJohnCalls JohnCalls PhoneRings PhoneRingsMaryCalls MaryCalls Alarm Alarm LoudmusicLoudmusic
Alarm Alarm Burglary BurglaryAlarm Alarm EarthQuake EarthQuake
What can we deduce from an observation that What can we deduce from an observation that John callsJohn calls,, Mary doesn’t Mary doesn’t andand Mary’s CD-player was Mary’s CD-player was broken broken ??
27
Abductive ReasoningAbductive Reasoning Deductive reasoning:Deductive reasoning: (using Modus Ponens)(using Modus Ponens)
AAB B A A
BB
Abductive reasoning:Abductive reasoning: (assume (assume AA is unknown) is unknown)BBB B A A
AA
AbduceAbduce that that AA holds as an explanation for the observation holds as an explanation for the observation BB More generally:More generally: given given a set of observationsa set of observations and and a set of logical rulesa set of logical rules: find : find a set of a set of
hypotheseshypotheses (from the unknown properties) that allow to deduce the observations (from the unknown properties) that allow to deduce the observations
28
Abduction in burglar-alarm:Abduction in burglar-alarm:
burglaryburglary earthquakeearthquake
alarmalarm
MaryCallsMaryCallsJohnCallsJohnCalls
phoneRingsphoneRingsunlessLoudmusicunlessLoudmusic
Unknown information :Unknown information :
JohnCalls JohnCalls Alarm AlarmJohnCalls JohnCalls PhoneRingsPhoneRingsMaryCalls MaryCalls Alarm Alarm LoudmusicLoudmusic
Alarm Alarm BurglaryBurglaryAlarm Alarm EarthQuakeEarthQuake
Observation:Observation: JohnCalls JohnCalls
abduceabduce
PhoneRingsPhoneRingsdeducededuce
AlarmAlarm
abduceabduce
BurglaryBurglary
abduceabduce
EarthQuakeEarthQuake
3 possible solutions:3 possible solutions: PhoneRings PhoneRings Burglary Burglary EarthQuake EarthQuake
29
Abduction in burglar-alarm (2):Abduction in burglar-alarm (2):
Observation:Observation: JohnCalls JohnCalls MaryCallsMaryCalls LoudMusic LoudMusic
JohnCalls JohnCalls Alarm AlarmJohnCalls JohnCalls PhoneRingsPhoneRingsMaryCalls MaryCalls Alarm Alarm LoudmusicLoudmusic
Alarm Alarm BurglaryBurglaryAlarm Alarm EarthQuakeEarthQuake
abduceabduce
PhoneRingsPhoneRingsdeducededuce
AlarmAlarm deducededuce AlarmAlarm
hypothesis of hypothesis of alarmalarm is is inconsistent with observationsinconsistent with observations
1 possible explanation:1 possible explanation: PhoneRingsPhoneRings
Note: abductive procedures may be complicated !Note: abductive procedures may be complicated !
30
Belief Networks orBelief Networks orBayesian NetsBayesian Nets
The The probabilistic versionprobabilistic version of the burglary-alarm example: of the burglary-alarm example:
burglaryburglary earthquakeearthquake
alarmalarm
MaryCallsMaryCallsJohnCallsJohnCalls
P(E)P(E).002.002
P(B)P(B).001.001
B E P(A)B E P(A)T T .95T T .95T F .94T F .94F T .29F T .29F F .001F F .001
A P(J)A P(J)T .90T .90F .05F .05
A P(M)A P(M)T .70T .70F .01F .01
Acyclic directed network !Acyclic directed network !
Prior probability for roots, conditional (on parents) Prior probability for roots, conditional (on parents) for lower levelsfor lower levels
31
Inference in Belief NetworksInference in Belief Networksburglaryburglary earthquakeearthquake
alarmalarm
MaryCallsMaryCallsJohnCallsJohnCalls
P(E)P(E).002.002
P(B)P(B).001.001
A P(M)A P(M)T .70T .70F .01F .01
A P(J)A P(J)T .90T .90F .05F .05
B E P(A)B E P(A)T T .95T T .95T F .94T F .94F T .29F T .29F F .001F F .001
0.0160.016 (Bayes)(Bayes)
Many (all) types of questions can be answered, using Bayes Many (all) types of questions can be answered, using Bayes Rule.Rule.
What is the probability that there is What is the probability that there is no burglaryno burglary, , nor nor earthquakeearthquake, but that the , but that the alarm wentalarm went and both and both John and John and
Mary Mary calledcalled??
= P (J = P (J M M A A B B E) = 0.00062 E) = 0.00062
What is the probability that there is a What is the probability that there is a burglaryburglary, given that , given that JohnJohn callscalls??
= P(B | J) = ?= P(B | J) = ?
32
An application: GENINFERAn application: GENINFER A couple is expecting a child.A couple is expecting a child.
The (expecting) mother has a The (expecting) mother has a hemophiliac risk hemophiliac risk determinedetermine the probability of hemophiliacthe probability of hemophiliac for the childfor the child
Hemophiliac disease is genetically determined:Hemophiliac disease is genetically determined: Due to a defected Due to a defected XX chromosome chromosome
mothermother fatherfather
Chromosomes:Chromosomes: XX XX X X yy
childchild
XX XX XX y yXX XX XX y y
carriercarrier hemophiliachemophiliac
33
The Bayesian Network:The Bayesian Network:
mother_carriermother_carrier father_hemophfather_hemoph
child_recessivechild_recessive
P(M)P(M).00008.00008
P(F)P(F).00008.00008
M F P(C)M F P(C)T T .75T T .75T F .50T F .50F T .50F T .50F F 0F F 0
A family tree:A family tree:
okok
okok
okok
HH
??
fatherfathermothermother
grandfathergrandfather
great grandfathergreat grandfathergreat grandmothergreat grandmother
grandmothergrandmothergreat unclegreat uncle
CC
??
??
34
Expanding to full network:Expanding to full network:
GGMGGM GGFGGF
GMGM GFGFGUGU
MM FF
CC
P(GGM)P(GGM) 11
P(GGF)P(GGF) 00
P(GF)P(GF) 00
P(F)P(F) 00
P(GU)P(GU) 11
Tempting solution:Tempting solution:but these are not priorbut these are not priorprobabilitiesprobabilities
But in fact remains correctBut in fact remains correctif you interpret events differentlyif you interpret events differently
35
Expanding to full network (2)Expanding to full network (2)
GGMGGM GGFGGF
GMGM GFGFGUGU
MM FF
CC
P(GGM)P(GGM).00008.00008
P(GGF)P(GGF).00008.00008
P(GF)P(GF).00008.00008
P(F)P(F).00008.00008M F P(C)M F P(C)
T T .75T T .75T F .50T F .50F T .50F T .50F F 0F F 0
All dependenciesAll dependenciesare based on:are based on:
Compute: P(GGM| GU Compute: P(GGM| GU GGF) = 1 GGF) = 1 Compute: P(GM| GGM Compute: P(GM| GGM GGF) = 0.5 , etc. GGF) = 0.5 , etc.
0.50.5
0.250.25
0.1250.125
36
And if there are uncles?And if there are uncles?
GGMGGM GGFGGF
GMGM GFGFGUGU
MM FF
CC
U1U1 U2U2
Recompute: P(GM| GMM Recompute: P(GM| GMM GGF GGF U1 U1 U2) U2) Propagate the information to Mother and Child Propagate the information to Mother and Child
0.50.5
0.0280.028
37
And brothers?And brothers?
GGMGGM GGFGGF
GMGM GFGFGUGU
MM FF
CC
U1U1 U2U2
B1B1 B2B2 B3B3
Probability under additional condition of 3 healthy bothers: Probability under additional condition of 3 healthy bothers:
further decreasefurther decrease
0.0070.007
38
Belief Networks as Rule SystemsBelief Networks as Rule Systems
Burglary Burglary (0.01)(0.01) Earthquake Earthquake (0.02)(0.02)
Alarm Alarm (0.95)(0.95) Burglary Burglary Earthquake EarthquakeAlarm Alarm (0.94)(0.94) Burglary Burglary EarthquakeEarthquakeAlarm Alarm (0.29)(0.29) Burglary Burglary Earthquake EarthquakeAlarm Alarm (0.001)(0.001) Burglary Burglary EarthquakeEarthquake
JohnCalls JohnCalls (0.90)(0.90) Alarm AlarmJohnCalls JohnCalls (0.05)(0.05) AlarmAlarm
MaryCalls MaryCalls (0.70)(0.70) Alarm AlarmMaryCalls MaryCalls (0.01)(0.01) AlarmAlarm
Doesn’t add anything …but shows that you need Doesn’t add anything …but shows that you need many rulesmany rules to represent the full Bayesian Net to represent the full Bayesian Net
In many cases you may not have all this information!In many cases you may not have all this information!
burglaryburglary earthquakeearthquake
alarmalarm
MaryCallsMaryCallsJohnCallsJohnCalls
P(E)P(E).002.002
P(B)P(B).001.001
A P(M)A P(M)T .70T .70F .01F .01
A P(J)A P(J)T .90T .90F .05F .05
B E P(A)B E P(A)T T .95T T .95T F .94T F .94F T .29F T .29F F .001F F .001
39
Opinion Nets:Opinion Nets:when dependencies are unknownwhen dependencies are unknown
Will the stock (on stock market) split ?Will the stock (on stock market) split ?
Ask the opinion of 2 brokers and of 2 mystics.Ask the opinion of 2 brokers and of 2 mystics.
OROR
OROR
ANDAND
Broker 1Broker 1
Broker 2Broker 2
Mystic 1Mystic 1
Mystic 2Mystic 2
Brokers’ opinionBrokers’ opinion
Mystics’ opinionMystics’ opinion
Overall opinionOverall opinion
Stock_split Stock_split Brokers_say_split Brokers_say_split Mystics_say_split Mystics_say_splitBrokers_say_split Brokers_say_split Boker1 Boker1 Broker2 Broker2Mystics_say_split Mystics_say_split Mystic1 Mystic1 Mystic2 Mystic2
+ opinions (in probabilities) of Brokers and Mystics+ opinions (in probabilities) of Brokers and Mystics
40
Opinion Nets (2)Opinion Nets (2) Problem:Problem:
We don’t know the dependencies between the opinions We don’t know the dependencies between the opinions of the brokers and/or mystics !of the brokers and/or mystics ! How to propagate their (probabilistic) opinions?How to propagate their (probabilistic) opinions?
You need upper and lower bounds!You need upper and lower bounds!
P(E)P(E)
00
11Upperbound(E)Upperbound(E)
Lowerbound(E)Lowerbound(E)
UU(E) (E) P(E) P(E)LL(E) (E) P(E) P(E)
We try to make it We try to make it LeastLeast UpperboundsUpperbounds and and GreatestGreatest LowerboundsLowerbounds
41
Rules governing the propagation Rules governing the propagation of boundsof bounds
ORORAA
BBA A oror B B… … for the for the oror connective connective
UU(A) (A) UU(A (A oror B) B)
UU(B) (B) UU(A (A oror B) B)
LL(A) (A) LL(A (A oror B) - B) - UU(B) (B)
LL(B) (B) LL(A (A oror B) - B) - UU(A) (A)
UU(A (A oror B) B) UU(A) + (A) + UU(B) (B) LL(A (A oror B) B) max[ max[LL(A) , (A) , LL(B)] (B)]
… … similar for the similar for the andand connective connective
42
Some example inferences:Some example inferences:AA
BB BB
BBBBAA
AA
AA
max[P(A), P(B)] max[P(A), P(B)] P(A P(A oror B) B) P(A) + P(B) P(A) + P(B)
LL(A)(A)
LL(B)(B)max(max( ,, ))
max(max( LL(A) (A) , , LL(B)(B))) is is aa lowerbound lowerbound for P(A for P(A oror B) B)
max(max( LL(A) (A) , , LL(B)(B)) ) LL(A (A oror B) B)
43
Propagation of opinions:Propagation of opinions:Using forward propagation:Using forward propagation:
00
11
0.40.40.10.1
00
11
0.40.40.30.3
OROR
00
110.80.8
UU(A (A oror B) B) UU(A) + (A) + UU(B)(B)
0.30.3
LL(A (A oror B) B) max[ max[LL(A) , (A) , LL(B)] (B)]
00
11
00
11OROR
00
110.80.8
0.30.3
0.80.8
0.80.8
UU(A) (A) UU(A (A oror B) B)
UU(B) (B) UU(A (A oror B) B)
Or backward propagation:Or backward propagation:
44
The global propagation The global propagation (restricted to lowerbounds):(restricted to lowerbounds):
OROR
OROR
ANDAND
00
110.750.75
0.250.25
00
110.660.66
0.330.33
00
11
0.150.150.150.15
00
110.850.850.850.85
LL(A (A oror B) B) max[ max[LL(A) , (A) , LL(B)] (B)]
00
11
0.330.33
00
11
0.850.85
00
11
0.18 0.18 = 0.33 + = 0.33 + 0.85 -10.85 -1
LL(A (A andand B) B) LL(A) + (A) + LL(B) - 1(B) - 1
Not just OpinionsNot just Opinions: : anyany probability propagation inprobability propagation in absence absence of dependenciesof dependencies needs to apply approximations.needs to apply approximations.
45
Going Fuzzy …Going Fuzzy …for a few minutes.for a few minutes.
Examples of Fuzzy statements:Examples of Fuzzy statements: The motor is running The motor is running very hotvery hot.. Tom is a Tom is a very tallvery tall guy. guy. Electric cars are Electric cars are not very fastnot very fast.. HighHigh-performance drives require -performance drives require very rapidvery rapid dynamics dynamics and and preciseprecise regulation. regulation. Leuven is Leuven is quite a short distancequite a short distance from Brussels. from Brussels. Leuven is a Leuven is a beautifulbeautiful city. city. The maximum range of an electronic vehicle is The maximum range of an electronic vehicle is shortshort..
If If shortshort means means: 300 km or less, would 301 km be : 300 km or less, would 301 km be longlong ? ? Want to express Want to express to what degreeto what degree a property holds. a property holds.
46
Relations, sets and functionsRelations, sets and functions
Offer Offer alternative representationsalternative representations of logical statementsof logical statements
loves(John, Ann)loves(John, Ann)loves(Mary, Phil)loves(Mary, Phil)loves(Carl,Susan)loves(Carl,Susan)
Relations:Relations:
loves = loves = {(John,Mary), {(John,Mary), (Mary, Phil), (Mary, Phil), (Carl, Susan)}(Carl, Susan)}
Sets:Sets:
loves(John,Mary) = 1loves(John,Mary) = 1loves(Mary,Phil) = 1loves(Mary,Phil) = 1loves(Carl,Susan) = 1loves(Carl,Susan) = 1
Functions to {0,1}:Functions to {0,1}:
Equivalent !Equivalent !
This one allows refined statementsThis one allows refined statements
47
Fuzzy sets:Fuzzy sets: Are functions: Are functions: f:f: domaindomain [0,1][0,1]
11
00150150 160160 170170 180180 190190 200200 210210 cmcm
Crisp set Crisp set (tall men)(tall men)::
Fuzzy set Fuzzy set (tall men)(tall men)::11
00150150 160160 170170 180180 190190 200200 210210 cmcm
48
Representing a domain:Representing a domain:
Crisp sets Crisp sets (men’s height)(men’s height)::
11
00150150 160160 170170 180180 190190 200200 210210 cmcm
shortshort mediummedium talltall
Fuzzy set Fuzzy set (men’s height)(men’s height)::
11
00150150 160160 170170 180180 190190 200200 210210 cmcm
shortshort mediummedium talltallshortshort