evidence theory [episode 2]implementation issues and ...theo/evidenz/evidenz_slides.pdf · evidence...
TRANSCRIPT
Evidence Theory [episode 2]Implementation Issues and Applications
David Lesage <[email protected]>
LRDE seminar, May 28, 2003
Table of Contents
Table of Contents
Fed up with TV series? ....................................................................... 4
Episode 1............................................................................................ 5
Episode 2............................................................................................ 6
Episode 3............................................................................................ 8
Belief State and Decision after 3 episodes ................................................. 10
Implementation .................................................................................... 14
Mass Functions representation ................................................................ 15
Focal Elements representation ................................................................ 19
Potentials representation ........................................................................ 24
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 1
Table of Contents
Combination & fusion issues ................................................................... 26
Evidenz: an evidence theory engine ................................................. 29
Evidenz characteristics .......................................................................... 30
Performances ...................................................................................... 31
Evidenz engine future improvements ........................................................ 35
Applications ......................................................................................... 38
Document classification ......................................................................... 39
Cytology (Adhoc2)................................................................................. 42
Satellite image classification ................................................................... 46
Conclusion ........................................................................................... 50
Questions? ........................................................................................... 51
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 2
Table of Contents
References ........................................................................................... 52
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 3
Fed up with TV series?
Fed up with TV series?
Someone is a traitor...
Jamey Farrell Tony Almeida Nina Myers
• 1 variable: traitor = Jamey, Tony, Nina
• belief mass on focal element X = belief that X contains the traitor
• potentials = episodes
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 4
Fed up with TV series? Episode 1
Episode 1
Jamey is reliable
Richard Walsh
. m(Tony, Nina) = 0.7
! Richard is not omnipotent.
. m(Jamey, Tony,Nina) = 0.3
• Reminder: ∑A⊆Ω
m(A) = 1
ep1 = Tony, Nina, Jamey, Tony,Nina
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 5
Fed up with TV series? Episode 2
Episode 2
Statistically, peoplewith a family are more sensitive to
pressure or moneytemptation
JackBauer
. m(Nina) = 0.4
. m(Jamey, Tony) = 0.6
ep2 = Nina, Jamey, Tony
: Statistical support
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 6
Fed up with TV series? Episode 2
• Combination of ep1 and ep2
ep1 = T,N[0.7], J, T,N[0.3]ep2 = N[0.4], J, T[0.6]
. Reminder : Combination- focal elements intersections- mD=A∩B=A∩C = mA ×mB + mA ×mC
. Generated focals elements:
ep1⊕2 = N, T, J, T. Belief Masses:
m(Nina) = 0.7× 0.4 + 0.3× 0.4 = 0.4
m(Tony) = 0.7× 0.6 = 0.42
m(Jamey, Tony) = 0.3× 0.6 = 0.18
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 7
Fed up with TV series? Episode 3
Episode 3
This clue accuses Nina !
GeorgesMasson
. m(Nina) = 0.8
. m(Jamey, Tony) = 0.2
ep3 = Nina, Jamey, Tony
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 8
Fed up with TV series? Episode 3
• Combination of ep3 and ep1⊕2
ep3 = N[0.8], J, T[0.2]ep1⊕2 = N[0.4], T[0.42], J, T[0.18]
. Generated focals elements:
ep1⊕2⊕3 = N, T, J, T. Conflict C:
N ∩ T = ∅ and N ∩ J, T = ∅
C = 0.8× 0.42 + 0.8× 0.18 + 0.2× 0.4 = 0.56. Belief Masses:
m(Nina) = 0.8×0.41−C ' 0.73
m(Tony) = 0.2×0.421−C ' 0.19
m(Jamey, Tony) = 0.2×0.181−C ' 0.08
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 9
Fed up with TV series? Belief State and Decision after 3 episodes
Belief State and Decision after 3 episodes
• Reminder:belA =
∑∅6=X⊆A
m(X)
plA = 1− belA =∑
X∩A 6=∅
m(X)
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 10
Fed up with TV series? Belief State and Decision after 3 episodes
ep1⊕2⊕3 = N[0.73], T[0.19], J, T[0.08]
. Jamey
belJamey = 0 plsJamey = 0.08
. Tony
belTony = 0.19 plsTony = 0.19 + 0.08 = 0.27
. Nina
belNina = 0.73 plsNina = 0.73
: Take care about interpretations !
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 11
Fed up with TV series? Belief State and Decision after 3 episodes
• Decision : probability construction (Smets et al., 1992)
BetP (x,m) =∑
x∈A⊆Ω
m(A)|A|
ep1⊕2⊕3 = N[0.73], T[0.19], J, T[0.08]
. Jamey
BetP (J) =0.082
= 0.04
. Tony
BetP (T ) = 0.19 +0.082
= 0.24
. NinaBetP (N) = 0.73
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 12
Fed up with TV series? Belief State and Decision after 3 episodes
Is this theory right?To be continued...
illustrations from “24” TV series: http://www.fox.com/24/
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 13
Implementation
Implementation
• Mass Functions representation
• Focal Elements representation
• Potentials representation
• Combination & fusion issues
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 14
Implementation Mass Functions representation
Mass Functions representation
• Mass function = distribution of belief on Ω
B In theory :mass functions assign mass values to sets.
B In reality :masses depend on context (events).
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 15
Implementation Mass Functions representation
Example, in the field of Medical Diagnosis :
Combining “general rules” (potentials), and applying the system obtainedon multiple cases, according to patient’s characteristics.
: Need for functors
m : 2Ω × E → [0, 1]<(A, e) → v
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 16
Implementation Mass Functions representation
: Model principles :
Credal System construction:
combinations of mass functors:m(A, ?)
Credal System interrogation:
application of mass functors (context):v = m(A, e)
belief, plausability interrogations
e
Pignistic OperationsDecision
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 17
Implementation Mass Functions representation
• Consequences of this model
. Different levels of evaluation
. Approximations impossible
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 18
Implementation Focal Elements representation
Focal Elements representation
• Focal Element = set of variable configurations : hypothesis
B Operations: set intersections, projections, extensions
B Size issues
B Efficiency issues
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 19
Implementation Focal Elements representation
• Lists?
« Nina » « Tony » « Jamey »
Nina, Tony, Jamey
. speed, size, ordering problems
• Ordered sets?
. solve only ordering problems
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 20
Implementation Focal Elements representation
• Bitsets?
People
Reason
Jamey Tony Nina
Money
Private
(Jamey, Money), (Jamey, Private), (Tony, Money), (Nina, Private)
< 11 10 01 >
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 21
Implementation Focal Elements representation
Bitsets properties
B efficient operations on sets
B set bit number = configurations number
B constant size:
S =nbvars∏
i=1
#Xibits
ex: for 23 binary variables, size(FS) = 223 bits = 1 Mo
B Index ⇔ Configuration correspondancy formulasB Independence towards original representation !
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 22
Implementation Focal Elements representation
• Other representations
. Normal forms
Disjunctive forms
r(A) =h1⋃i=1
n⋂j=1
Sij
Sij ∈ Θxj
Conjunctive forms
r(A) =h1⋂i=1
n⋃j=1
Sij
Sij ∈ Θxj
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 23
Implementation Potentials representation
Potentials representation
• Potential = collection of pairs (Fi, mi)with Fi ∈ FS and mi mass function : 1 source of information
. Operations on potentials: combinations, fusions
. Need for efficient research and combination : regroupments
. Lists? research: O(n) regroupment: O(n2)
. Balanced Trees? research: O(log(n)) regroupment: O(n×log(n))
. Hash Tables? research: O(1) regroupment: O(n2/s)
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 24
Implementation Potentials representation
Example of a combination of 2 potentials of 1000 focal elements with 1000configurations:
balanced trees (AVL) 6 secondslists 14 hours 22 minutes
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 25
Implementation Combination & fusion issues
Combination & fusion issues
. New potentials creation : focal elements creation (by intersection)
. New mass functions creation : functors creation
. Mass functions combinations and regroupment
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 26
Implementation Combination & fusion issues
m(A, e) m(B, e) m(A, e) m(C, e)
* *
+
Mass Function of D = A ∩ B = A ∩ C
Regroupment [optional]
A ∩ B = A ∩ C = D
Combination
Comb(A, B)
Combination
Comb(A, C)
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 27
Implementation Combination & fusion issues
B indirections due to functors !
B example, with 1 potential P , 10 focal elements of 10 configurations, on1 variable with 10 realisations, after 5 combinations:
construction time 1 secondmemory load 1 Mo
original focal elements 10final focal elements 163
interrogation time (1 Focal element) 2 minutesmass regroupments (plus indirections) 1.6059× 109
mass combinations (times indirections) 3.2642× 109
• optimization?
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 28
Evidenz: an evidence theory engine
Evidenz: an evidence theory engine
• Existing engines: research prototypes, written in Lisp...(Haenni and Lehmann, 2001) (Saffiotti and Umkehrer, 1991)
. Evidenz characteristics
. Performances
. Evidenz engine future improvements
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 29
Evidenz: an evidence theory engine Evidenz characteristics
Evidenz characteristics
. TBM exact modeling (Smets et al., 1992)
. written in C++ (dynamic version)
. focal elements represented by bitsets (Boost)
. generic towards configurations representation
. potentials represented by balanced trees (STL maps)
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 30
Evidenz: an evidence theory engine Performances
Performances
• Study of 6= factors influence on execution time and memory load :
. variables
. variable realisations
. focal elements per potential
. configurations per focal element
. one combination of 2 potentials randomly generated
• example, on focal elements per potential influence:
. 2 variables with 500 realisations
. 100 configurations per focal element
. 100 to 700 focal elements per potential
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 31
Evidenz: an evidence theory engine Performances
100 200 250 300 350 400 450 500 550 600 650 7000
20406080
100120
140160
180200
time(s)
config sets number
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 32
Evidenz: an evidence theory engine Performances
100
200
250
300
350
400
450
500
550
600
650
7000
100200300400500600
700800
9001000
memoryload (Mo)
config sets number
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 33
Evidenz: an evidence theory engine Performances
• Performance study conclusions
. Time and memory load evolutions are exponential towards all thefactors
. Problems become quickly infeasible !
. In practice, applications are much more reasonable...
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 34
Evidenz: an evidence theory engine Evidenz engine future improvements
Evidenz engine future improvements
• Heuristics for optimizing elimination sequences (using Join trees )
• Decision algorithms, like decision trees
• Memoizing (set intersections and mass functions)
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 35
Evidenz: an evidence theory engine Evidenz engine future improvements
• 2 use cases:
. Need of quick, unique application :
Credal System ConstructionSystem definition
+Credal System Interrogation
Events, application
Pignistic Leveldecision
. Approximations
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 36
Evidenz: an evidence theory engine Evidenz engine future improvements
. Need of multiple applications on different events :
Credal SystemConstructionSystem definition
Credal SystemInterrogation
Events, application
PignisticLeveldecision
CodeGenerationOptimization,
inlining
. code generation : generates an optimal system at the end of credaldefinition phase
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 37
Applications
Applications
• Document classification
• Cytology (Adhoc2)
• Satellite image classification
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 38
Applications Document classification
Document classification
• Automatic classification and data extraction
. to associate a model to a document
• A document contains anchors : logos, text boxes...
• Each anchor corresponds to one or several models
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 39
Applications Document classification
RF logo
Cerfa logo
Document title
Document reference
Form box 1
Form box 2
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 40
Applications Document classification
• Modelisation:
. 1 variable: document model = model1,model2,model3, ...,modelk
. potentials = anchors
• Anchor recognition : belief masses for model sets
. combining anchor masses : global belief for each document model
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 41
Applications Cytology (Adhoc 2)
Cytology (Adhoc 2)
• Cancerous cells detection
. diagnosis help
. 6= factors:∗ cell/nucleus size∗ cell/nucleus regularity∗ color...
• Fuzzy , subjective decision rules
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 42
Applications Cytology (Adhoc 2)
0
0,10,2
0,3
0,4
0,50,6
0,7
0,8
5 6 7 8 9 10 11 12 13 14 15
belief mass
cellsize(nm)
Size factor
normal cellnot a cell or abnormal cellcancerous cell
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 43
Applications Cytology (Adhoc 2)
Regularity factor
normal cellAbnormal or cancerous cell
0
0,2
0,4
0,6
0,8
1
1,2
0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1
belief mass
regularity
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 44
Applications Cytology (Adhoc 2)
• Modelisation:
. 1 variable: cell type = normal, abnormal, not a cell, cancerous
. potentials = criters: size, regularity, color...
• Cell characteristics : belief masses for cell type sets
. combining criters : global belief for each cell type
. justified decision, quantified ignorance...
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 45
Applications Satellite image classification
Satellite image classification
• Satellite images composed of several layers:
. 7 layers according to wave length
. Different visions of the same image : different informations
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 46
Applications Satellite image classification
Layer x Layer y
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 47
Applications Satellite image classification
Layer x Layer y
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 48
Applications Satellite image classification
• Modelisation:
. 1 variable: pixel type = forest, water, road, building, ...
. potentials = layers
• Pixel gray level (context ) : belief masses for pixel type sets
. combining layers : global belief for each pixel type
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 49
Conclusion
Conclusion
• Theory designed for dealing with uncertainty
. Modelisation power
. Computational complexity
• Evidenz engine
. Code generation and algorithmic extension/improvement
• Future research applications:
. Adhoc2 cytology project
. Satellite image classification
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 50
Questions?
Questions?
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 51
References
References
Burrus, N. (2003). Evidence theory [part 1]. lrde seminar oral presentation.
Burrus, N. and Lesage, D. (2003). Evidence theory. Technical report,EPITA Research And Development Laboratory, Paris, France.
Haenni, R. and Lehmann, N. (2001). Implementing belief functioncomputations. Technical Report 01-28, Department of Informatics,University of Fribourg.
Saffiotti, A. and Umkehrer, E. (1991). Pulcinella: a general tool forpropagating uncertainty in valuation networks. pages 323–331.
Smets, P., Hsia, Y., Saffiotti, A., Kennes, R., Xu, H., and Umkehrer, E.(1992). The transferable belief model. pages 91–98.
Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 52