algorithmic facets of human centricity in computing with fuzzy sets

Algorithmic Facets of Human Centricity in Computing with Fuzzy

Sets

ISDA-2009, Pisa, Italy, November 30-December 2, 2009

pedrycz@ee.ualberta.ca

Agenda

Human centricity and information granules

Design of information granules – approaches of knowledge-basedclustering

Granular representation of computing with fuzzy sets

Human Centricity and information granules

Information granules as conceptual entities inherently associated with human pursuits (decision-making, perceptioncontrol, prediction)

Interaction with and processing in intelligent systems realized at the level of information granules (fuzzy sets, rough sets, intervals…)

Emergence of Human-Centric computing (HC2)

Knowledge sharing and collaboration in distributed systems

Human Centricity and fuzzy sets

Two fundamental quests:

Construction of information granules (fuzzy sets);use of existing experimental evidence and its interpretationCast in the framework of users/designer

Qualitative, user-centric interpretation of results of computing with fuzzy sets

Clustering as aconceptual and algorithmic framework of information

granulationData information granules (clusters) abstraction of data

Formalism of: set theory (K-Means) fuzzy sets (FCM) rough sets

shadowed sets

Main categories of clustering

Graph-oriented and hierarchical (single linkage, complete linkage, average linkage..)

Objective function-based clustering

Diversity of formalisms and optimization tools(e.g., methods of Evolutionary Computing)

Key challenges of clustering

Data-driven methods

Selection of distance function (geometry of clusters)

Number of clusters

Quality of clustering results

The dichotomy and the shift of paradigm

Human-centricityGuidance mechanisms

Fuzzy Clustering: Fuzzy C-Means (FCM)

Given data x1, x2, …, xN, determine its structure byforming a collection of information granules – fuzzy sets

Objective function

2ik

N

1k

mik

c

1i||||uQ vx

Minimize Q; structure in data (partition matrix and prototypes)

Fuzzy Clustering: Fuzzy C-Means (FCM)

Vi – prototypes

U- partition matrix

FCM – optimization

2ik

N

1k

mik

c

1i||||uQ vx

Minimize

subject to

(a) prototypes

(b) partition matrix

Domain Knowledge:Category of knowledge-

oriented guidance

Context-based guidance: clustering realized in a certain contextspecified with regard to some attribute

Viewpoints: some structural information is provided

Partially labeled data: some data are provided with labels (classes)

Proximity knowledge: some pairs of data are quantified interms of their proximity (resemblance, closeness)

Clustering with domain knowledge

(Knowledge-based clustering)

Data

Information granules (structure)

CLUSTERING

Domain knowledge

Data-driven Data- and knowledge-driven

Data

Information granules (structure)

CLUSTERING

Context-based clustering

Clustering : construct clusters in input space X

Context-based Clustering : construct clusters in input space X given some context expressed in output space Y

Active role of the designer [customization of processing]

Context-based clustering:Conmputational considerations

•computationally more efficient,•well-focused, •designer-guided clustering process

Data

structure

Data

structure

context

Context-based clustering:focus mechanism

Determine structure in input space given the output is high

Determine structure in input space given the output is medium

Determine structure in input space given the output is low

Input space (data)

Context-based clustering:examples

Find a structure of customer data [clustering]

Find a structure of customer data considering customers making weekly purchases in the range [$1,000 $3,000]

Find a structure of customer data considering customers making weekly purchases at the level of

around $ 2,500

Find a structure of customer data considering customers making significant weekly purchases who

are young

no context

context

context

context(compound)

Context-oriented FCM

Data (xk, targetk), k=1,2,…,N

Contexts: fuzzy sets W1, W2, …, Wp

wjk = Wi(targetk) membership of j-th context for k-th data

c

1i

N

1kikjkikikj iNu0andk wu|0,1u)(WU

Context-driven partition matrix

Context-oriented FCM:Optimization flow

Objective function

Iterative adjustment of partition matrix and prototypes

2ik

c

1i

N

1k

mik ||||uQ vx

c

1j

1m

2

jk

ik

jkik

wu

vx

vx

N

1k

mik

N

1kk

mik

i

u

u xv

Subject to constraint U in U(Wj)

Viewpoints: definition

Description of entity (concept) which is deemed essential in describing phenomenon (system) and helpful in castingan overall analysis in a required setting

“external” , “reinforced” clusters

Viewpoints: definition

-150

-100

-50

0

50

100

150

200

0 100 200 300 400 500

x1

x2

a

b

x1

x2

a

viewpoint (a,b) viewpoint (a,?)

Viewpoints: definition

Description of entity (concept) which is deemed essential in describing phenomenon (system) and helpful in castingan overall analysis in a required setting

“external” , “reinforced” clusters

Viewpoints in fuzzy clustering

x1

x2

a

b

otherwise 0,

viewpointby the determined is B of rowth -i theof featureth -j theif 1,b ij

0

0

1

0

0

1

B

0

0

b

0

0

a

F

B- Boolean matrix characterizing structure: viewpoints prototypes (induced by data)

Viewpoints in localization of “extreme” information granules

specification of viewpoints through evolutionary/population-basedoptimization

Viewpoints in fuzzy clustering

Q = 2ijkj

n

1:bji,1j

mik

c

1i

N

1k

2ijkj

n

0:bji,1j

mik

c

1i

N

1k

)f(xu)v(xu

ijij

1b if f

0bif vg

ijij

ijijij

2ijkj

n

1j

mik

c

1i

N

1k

)g(xuQ

Labelled data and their description

Characterization in terms of membership degrees:

F = [fik] i=12,…,c , k=1,2, …., N

supervision indicator b = [bk], k=1,2,…, N

Augmented objective function

€

Q =i=1

c

∑ uik2

k=1

N

∑ || xk − vi ||2 +β∑ (uik − fik )2bk || xk − vi ||2∑

> 0

Proximity hints

Characterization in terms of proximity degrees:

Prox(k, l), k, l=1,2, …., N

and supervision indicator matrix B = [bkl], k, l=1,2,…, N

Prox(k,l)

Prox(s,t)

Proximity measure

Properties of proximity:

(a)Prox(k, k) =1

(b)Prox(k,l) = Prox(l,k)

Proximity induced by partition matrix U:

€

Prox(k,l) = min(uik

i=1

c

∑ ,uil )

Linkages with kernel functions K(xk, xl)

Augmented objective function

€

Q =i=1

c

∑ uik2

k=1

N

∑ || xk − vi ||2 +βi=1

c

∑k1=1

N

∑ [Prox(k1,k2) − Prox(U)(k1,k2)]2b(k1, k2) || xk1 − xk2 ||2

k2=1

N

∑

> 0

Two general development strategies

SELECTION OF A “MEANINGFUL” SUBSET OF INFORMATION GRANULES

Two general development strategies

(1) HIERARCHICAL DEVELOPMENT OF INFORMATION GRANULES (INFORMMATION GRANULES OF HIGHER TYPE)

Information granulesType -1

Information granulesType -2

Two general development strategies

(2) HIERARCHICAL DEVELOPMENT OF INFORMATION GRANULES AND THE USE OF VIEWPOINTS

Information granulesType -1

Information granulesType -2

viewpoints

Two general development strategies

(3) HIERARCHICAL DEVELOPMENT OF INFORMATION GRANULES – A MODE OF SUCCESSIVE CONSTRUCTION

Fuzzy Computing:Interpretability

Interpretation of fuzzy sets - departure from pure numeric quantification of membership grades

A= [0.11 0.19 0.34 0.45 1.00 0.98 0.821 0.447…]

Granulation of fuzzy sets

Granulation of membership grades

low, high, medium membership of alternative x

Granulation of membership grades and universe of discourse

low membership for a collection of alternatives….

Granulation of membershipgrades

A= [0.11 0.19 0.34 0.45 1.00 0.98 0.821 0.447…]

A= [L L L M M L L M…]

Granulation of membershipgrades: optimization

A= [L L L M M L L M…]

Entropy minimization

G= {G1, G2, …, Gc}

€

x

∑ H(G i

i=1

c

∑ (x))⇒ MinG

Granulation of fuzzy sets

A= [L M L M…]

Granulation of fuzzy sets:optimization

G1

Gi

Gc

1

i c

€

Vol = Vol(G i

i=1

c

∑ ,Wi) ⇒ MinG

Interpretability of fuzzy set computing

Fuzzy set computing

Interpretability layer

Granulation of fuzzy sets

Interpretability of fuzzy set computing

Fuzzy set computing

Interpretability layer

Granulation of fuzzy sets

Interpretability of fuzzy set computing

Equivalence sought with respect with assumed levelinterpretability:•stability•Equivalence of models

distinguishability

Non-distinguishability

Fuzzy set computing: a retrospective

interpretability

accuracy

~1970

after ~1990

neurofuzzy

evolutionary

Rule-based

Conclusions

Leitmotiv of human-centricity and its underlying reliance on information granules

Design of information granules – shift from data to knowledge-enhanced clustering

Revisiting the practice of fuzzy computing and its interpretabilitycapabilities