Evidence Theory [episode 2]Implementation Issues and Applications

David Lesage <david@lrde.epita.fr>

LRDE seminar, May 28, 2003

Table of Contents

Table of Contents

Fed up with TV series? ....................................................................... 4

Episode 1............................................................................................ 5

Episode 2............................................................................................ 6

Episode 3............................................................................................ 8

Belief State and Decision after 3 episodes ................................................. 10

Implementation .................................................................................... 14

Mass Functions representation ................................................................ 15

Focal Elements representation ................................................................ 19

Potentials representation ........................................................................ 24

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 1

Table of Contents

Combination & fusion issues ................................................................... 26

Evidenz: an evidence theory engine ................................................. 29

Evidenz characteristics .......................................................................... 30

Performances ...................................................................................... 31

Evidenz engine future improvements ........................................................ 35

Applications ......................................................................................... 38

Document classification ......................................................................... 39

Cytology (Adhoc2)................................................................................. 42

Satellite image classification ................................................................... 46

Conclusion ........................................................................................... 50

Questions? ........................................................................................... 51

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 2

Table of Contents

References ........................................................................................... 52

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 3

Fed up with TV series?

Fed up with TV series?

Someone is a traitor...

Jamey Farrell Tony Almeida Nina Myers

• 1 variable: traitor = Jamey, Tony, Nina

• belief mass on focal element X = belief that X contains the traitor

• potentials = episodes

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 4

Fed up with TV series? Episode 1

Episode 1

Jamey is reliable

Richard Walsh

. m(Tony, Nina) = 0.7

! Richard is not omnipotent.

. m(Jamey, Tony,Nina) = 0.3

• Reminder: ∑A⊆Ω

m(A) = 1

ep1 = Tony, Nina, Jamey, Tony,Nina

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 5

Fed up with TV series? Episode 2

Episode 2

Statistically, peoplewith a family are more sensitive to

pressure or moneytemptation

JackBauer

. m(Nina) = 0.4

. m(Jamey, Tony) = 0.6

ep2 = Nina, Jamey, Tony

: Statistical support

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 6

Fed up with TV series? Episode 2

• Combination of ep1 and ep2

ep1 = T,N[0.7], J, T,N[0.3]ep2 = N[0.4], J, T[0.6]

. Reminder : Combination- focal elements intersections- mD=A∩B=A∩C = mA ×mB + mA ×mC

. Generated focals elements:

ep1⊕2 = N, T, J, T. Belief Masses:

m(Nina) = 0.7× 0.4 + 0.3× 0.4 = 0.4

m(Tony) = 0.7× 0.6 = 0.42

m(Jamey, Tony) = 0.3× 0.6 = 0.18

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 7

Fed up with TV series? Episode 3

Episode 3

This clue accuses Nina !

GeorgesMasson

. m(Nina) = 0.8

. m(Jamey, Tony) = 0.2

ep3 = Nina, Jamey, Tony

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 8

Fed up with TV series? Episode 3

• Combination of ep3 and ep1⊕2

ep3 = N[0.8], J, T[0.2]ep1⊕2 = N[0.4], T[0.42], J, T[0.18]

. Generated focals elements:

ep1⊕2⊕3 = N, T, J, T. Conflict C:

N ∩ T = ∅ and N ∩ J, T = ∅

C = 0.8× 0.42 + 0.8× 0.18 + 0.2× 0.4 = 0.56. Belief Masses:

m(Nina) = 0.8×0.41−C ' 0.73

m(Tony) = 0.2×0.421−C ' 0.19

m(Jamey, Tony) = 0.2×0.181−C ' 0.08

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 9

Fed up with TV series? Belief State and Decision after 3 episodes

Belief State and Decision after 3 episodes

• Reminder:belA =

∑∅6=X⊆A

m(X)

plA = 1− belA =∑

X∩A 6=∅

m(X)

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 10

Fed up with TV series? Belief State and Decision after 3 episodes

ep1⊕2⊕3 = N[0.73], T[0.19], J, T[0.08]

. Jamey

belJamey = 0 plsJamey = 0.08

. Tony

belTony = 0.19 plsTony = 0.19 + 0.08 = 0.27

. Nina

belNina = 0.73 plsNina = 0.73

: Take care about interpretations !

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 11

Fed up with TV series? Belief State and Decision after 3 episodes

• Decision : probability construction (Smets et al., 1992)

BetP (x,m) =∑

x∈A⊆Ω

m(A)|A|

ep1⊕2⊕3 = N[0.73], T[0.19], J, T[0.08]

. Jamey

BetP (J) =0.082

= 0.04

. Tony

BetP (T ) = 0.19 +0.082

= 0.24

. NinaBetP (N) = 0.73

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 12

Fed up with TV series? Belief State and Decision after 3 episodes

Is this theory right?To be continued...

illustrations from “24” TV series: http://www.fox.com/24/

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 13

Implementation

Implementation

• Mass Functions representation

• Focal Elements representation

• Potentials representation

• Combination & fusion issues

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 14

Implementation Mass Functions representation

Mass Functions representation

• Mass function = distribution of belief on Ω

B In theory :mass functions assign mass values to sets.

B In reality :masses depend on context (events).

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 15

Implementation Mass Functions representation

Example, in the field of Medical Diagnosis :

Combining “general rules” (potentials), and applying the system obtainedon multiple cases, according to patient’s characteristics.

: Need for functors

m : 2Ω × E → [0, 1]<(A, e) → v

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 16

Implementation Mass Functions representation

: Model principles :

Credal System construction:

combinations of mass functors:m(A, ?)

Credal System interrogation:

application of mass functors (context):v = m(A, e)

belief, plausability interrogations

e

Pignistic OperationsDecision

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 17

Implementation Mass Functions representation

• Consequences of this model

. Different levels of evaluation

. Approximations impossible

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 18

Implementation Focal Elements representation

Focal Elements representation

• Focal Element = set of variable configurations : hypothesis

B Operations: set intersections, projections, extensions

B Size issues

B Efficiency issues

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 19

Implementation Focal Elements representation

• Lists?

« Nina » « Tony » « Jamey »

Nina, Tony, Jamey

. speed, size, ordering problems

• Ordered sets?

. solve only ordering problems

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 20

Implementation Focal Elements representation

• Bitsets?

People

Reason

Jamey Tony Nina

Money

Private

(Jamey, Money), (Jamey, Private), (Tony, Money), (Nina, Private)

< 11 10 01 >

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 21

Implementation Focal Elements representation

Bitsets properties

B efficient operations on sets

B set bit number = configurations number

B constant size:

S =nbvars∏

i=1

#Xibits

ex: for 23 binary variables, size(FS) = 223 bits = 1 Mo

B Index ⇔ Configuration correspondancy formulasB Independence towards original representation !

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 22

Implementation Focal Elements representation

• Other representations

. Normal forms

Disjunctive forms

r(A) =h1⋃i=1

n⋂j=1

Sij

Sij ∈ Θxj

Conjunctive forms

r(A) =h1⋂i=1

n⋃j=1

Sij

Sij ∈ Θxj

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 23

Implementation Potentials representation

Potentials representation

• Potential = collection of pairs (Fi, mi)with Fi ∈ FS and mi mass function : 1 source of information

. Operations on potentials: combinations, fusions

. Need for efficient research and combination : regroupments

. Lists? research: O(n) regroupment: O(n2)

. Balanced Trees? research: O(log(n)) regroupment: O(n×log(n))

. Hash Tables? research: O(1) regroupment: O(n2/s)

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 24

Implementation Potentials representation

Example of a combination of 2 potentials of 1000 focal elements with 1000configurations:

balanced trees (AVL) 6 secondslists 14 hours 22 minutes

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 25

Implementation Combination & fusion issues

Combination & fusion issues

. New potentials creation : focal elements creation (by intersection)

. New mass functions creation : functors creation

. Mass functions combinations and regroupment

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 26

Implementation Combination & fusion issues

m(A, e) m(B, e) m(A, e) m(C, e)

* *

+

Mass Function of D = A ∩ B = A ∩ C

Regroupment [optional]

A ∩ B = A ∩ C = D

Combination

Comb(A, B)

Combination

Comb(A, C)

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 27

Implementation Combination & fusion issues

B indirections due to functors !

B example, with 1 potential P , 10 focal elements of 10 configurations, on1 variable with 10 realisations, after 5 combinations:

construction time 1 secondmemory load 1 Mo

original focal elements 10final focal elements 163

interrogation time (1 Focal element) 2 minutesmass regroupments (plus indirections) 1.6059× 109

mass combinations (times indirections) 3.2642× 109

• optimization?

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 28

Evidenz: an evidence theory engine

Evidenz: an evidence theory engine

• Existing engines: research prototypes, written in Lisp...(Haenni and Lehmann, 2001) (Saffiotti and Umkehrer, 1991)

. Evidenz characteristics

. Performances

. Evidenz engine future improvements

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 29

Evidenz: an evidence theory engine Evidenz characteristics

Evidenz characteristics

. TBM exact modeling (Smets et al., 1992)

. written in C++ (dynamic version)

. focal elements represented by bitsets (Boost)

. generic towards configurations representation

. potentials represented by balanced trees (STL maps)

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 30

Evidenz: an evidence theory engine Performances

Performances

• Study of 6= factors influence on execution time and memory load :

. variables

. variable realisations

. focal elements per potential

. configurations per focal element

. one combination of 2 potentials randomly generated

• example, on focal elements per potential influence:

. 2 variables with 500 realisations

. 100 configurations per focal element

. 100 to 700 focal elements per potential

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 31

Evidenz: an evidence theory engine Performances

100 200 250 300 350 400 450 500 550 600 650 7000

20406080

100120

140160

180200

time(s)

config sets number

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 32

Evidenz: an evidence theory engine Performances

100

200

250

300

350

400

450

500

550

600

650

7000

100200300400500600

700800

9001000

memoryload (Mo)

config sets number

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 33

Evidenz: an evidence theory engine Performances

• Performance study conclusions

. Time and memory load evolutions are exponential towards all thefactors

. Problems become quickly infeasible !

. In practice, applications are much more reasonable...

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 34

Evidenz: an evidence theory engine Evidenz engine future improvements

Evidenz engine future improvements

• Heuristics for optimizing elimination sequences (using Join trees )

• Decision algorithms, like decision trees

• Memoizing (set intersections and mass functions)

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 35

Evidenz: an evidence theory engine Evidenz engine future improvements

• 2 use cases:

. Need of quick, unique application :

Credal System ConstructionSystem definition

+Credal System Interrogation

Events, application

Pignistic Leveldecision

. Approximations

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 36

Evidenz: an evidence theory engine Evidenz engine future improvements

. Need of multiple applications on different events :

Credal SystemConstructionSystem definition

Credal SystemInterrogation

Events, application

PignisticLeveldecision

CodeGenerationOptimization,

inlining

. code generation : generates an optimal system at the end of credaldefinition phase

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 37

Applications

Applications

• Document classification

• Cytology (Adhoc2)

• Satellite image classification

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 38

Applications Document classification

Document classification

• Automatic classification and data extraction

. to associate a model to a document

• A document contains anchors : logos, text boxes...

• Each anchor corresponds to one or several models

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 39

Applications Document classification

RF logo

Cerfa logo

Document title

Document reference

Form box 1

Form box 2

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 40

Applications Document classification

• Modelisation:

. 1 variable: document model = model1,model2,model3, ...,modelk

. potentials = anchors

• Anchor recognition : belief masses for model sets

. combining anchor masses : global belief for each document model

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 41

Applications Cytology (Adhoc 2)

Cytology (Adhoc 2)

• Cancerous cells detection

. diagnosis help

. 6= factors:∗ cell/nucleus size∗ cell/nucleus regularity∗ color...

• Fuzzy , subjective decision rules

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 42

Applications Cytology (Adhoc 2)

0

0,10,2

0,3

0,4

0,50,6

0,7

0,8

5 6 7 8 9 10 11 12 13 14 15

belief mass

cellsize(nm)

Size factor

normal cellnot a cell or abnormal cellcancerous cell

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 43

Applications Cytology (Adhoc 2)

Regularity factor

normal cellAbnormal or cancerous cell

0

0,2

0,4

0,6

0,8

1

1,2

0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1

belief mass

regularity

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 44

Applications Cytology (Adhoc 2)

• Modelisation:

. 1 variable: cell type = normal, abnormal, not a cell, cancerous

. potentials = criters: size, regularity, color...

• Cell characteristics : belief masses for cell type sets

. combining criters : global belief for each cell type

. justified decision, quantified ignorance...

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 45

Applications Satellite image classification

Satellite image classification

• Satellite images composed of several layers:

. 7 layers according to wave length

. Different visions of the same image : different informations

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 46

Applications Satellite image classification

Layer x Layer y

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 47

Applications Satellite image classification

Layer x Layer y

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 48

Applications Satellite image classification

• Modelisation:

. 1 variable: pixel type = forest, water, road, building, ...

. potentials = layers

• Pixel gray level (context ) : belief masses for pixel type sets

. combining layers : global belief for each pixel type

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 49

Conclusion

Conclusion

• Theory designed for dealing with uncertainty

. Modelisation power

. Computational complexity

• Evidenz engine

. Code generation and algorithmic extension/improvement

• Future research applications:

. Adhoc2 cytology project

. Satellite image classification

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 50

Questions?

Questions?

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 51

References

References

Burrus, N. (2003). Evidence theory [part 1]. lrde seminar oral presentation.

Burrus, N. and Lesage, D. (2003). Evidence theory. Technical report,EPITA Research And Development Laboratory, Paris, France.

Haenni, R. and Lehmann, N. (2001). Implementing belief functioncomputations. Technical Report 01-28, Department of Informatics,University of Fribourg.

Saffiotti, A. and Umkehrer, E. (1991). Pulcinella: a general tool forpropagating uncertainty in valuation networks. pages 323–331.

Smets, P., Hsia, Y., Saffiotti, A., Kennes, R., Xu, H., and Umkehrer, E.(1992). The transferable belief model. pages 91–98.

Evidence Theory [episode 2] Implementation Issues and Applications, David Lesage - LRDE seminar, May 28, 2003 52