infinite block models for belief networks, social networks, and cultural knowledge josh tenenbaum,...
Post on 21-Dec-2015
215 views
TRANSCRIPT
Infinite block models for belief networks, social networks, and
cultural knowledge
Josh Tenenbaum, MIT
2007 MURI Review Meeting
Work of Charles Kemp, Chris Baker, Tom Griffiths, Pat Shafto, Vikash
Mansinghka, Dan Roy
Goal
• Algorithmic tools for uncovering structure in belief networks, social networks, and joint structure (social-belief networks).
• Why?– Joint social-belief structure ~ culture – Algorithms let us map cultural knowledge quickly
and semi-automatically, detect changes and track dynamics.
Approach
• Data– People’s beliefs about properties of objects – Relations between people– People’s beliefs about relations between objects (or people).
• Representation: cluster-based models– Clusters of things: categories– Clusters of people: social groups– Clusters of people who share similar beliefs about clusters of
things (or people): cultural groups
Approach• Learning: Bayesian inference from data
– Relational models: analyze arbitrary relational databases of beliefs, not just a single matrix
– Nonparametric models: automatically determine complexity of representations
– Hierarchical models: multiple levels of structure– Nested models: structures with structure
Result: a flexible toolkit that goes qualitatively beyond standard algorithms. – e.g., ability to discover cultural groups characterized by a shared understanding of the
environment’s relational structure.
Talk outline
• Classic cluster-based methods
• New methods– Clustering with arbitrary relational systems– Hierarchical relational clustering– Cross-cutting clustering with nested models– Cross-cutting relational clustering
• Application to Guatemala data from Atran & Medin
• Conclusions and future directions
Classic cluster-based methods
• Belief networks: mixture models
Classic cluster-based methods
• Belief networks: mixture models
Classic cluster-based methods
• Social networks: block models
DefersTo(Pi, Pj)
Classic cluster-based methods
• Cultural knowledge (joint social/belief structure): cultural consensus model
Not cluster-based.
SVD on matrix of people x questions
Problems with classic methods
• No principled tools for discovering different cultural groups characterized by different belief networks. – CCM not intended to find cultural groups, but rather to
uncover (and test for) shared knowledge and authoritativeness in a single cultural group. “Test theory without an answer key”
• Can only represent simple forms of knowledge that fit into a single two-mode matrix.– Cultural knowledge is often much richer….
Talk outline
• Classic cluster-based methods
• New methods– Clustering with arbitrary relational systems– Hierarchical relational clustering– Cross-cutting clustering with nested models– Cross-cutting relational clustering
• Application to Guatemala data from Atran & Medin
• Conclusions and future directions
peop
lepeople
social relation
• Alyawarra tribe, central Australia (Denham)– 104 individuals– 27 binary social relations– 3 attributes: kinship class, age, sex
(used only for cluster validation, not learning)
peop
le
attributes
Clustering arbitrary relational systems
Infinite relational model (IRM) discovers 15 clusters
Clustering arbitrary relational systems
Clustering arbitrary relational systems
International relations circa 1965 (Rummel)– 14 countries: UK, USA, USSR, China, ….– 54 binary relations representing interactions between countries:
exports to( USA, UK ), protests( USA, USSR ), …. – 90 (dynamic) country features: purges, protests, unemployment,
communists, # languages, assassinations, ….
Hierarchical relational clustering
• Models so far all learn a single system of clusters.
• We would like to be able to discover multiple cross-cutting systems of clusters.– Within an individual’s mind: multiple mental
models of a single complex domain. – Across individuals in a population: multiple
cultural groups with different characteristic mental models.
Cross-cutting clustering with nested models
Conventional mixture model
Cross-cutting clustering with nested models
CrossCat model
Cross-cutting clustering with nested models
Analysis of US Senate votes 1989-90
101 senators x 638 issues 10 systems of classes.
Core democratic platform “Hot-button” socialissues
Law and order Military Environment& agriculture
Nested relational model:
Cross-cutting clustering with nested modelspe
ople
people
relation
Infinite relational model:
peop
lepeople
relation
Discovering cultural groups based on shared relational knowledge
• Guatemala studies of Atran & Medin– Subjects
• 12 native Itza’ maya
• 12 immigrant Ladino
• 12 immigrant Q’eqchi’ maya
– Questions• Does plant i help animal j?
anim
al
plant
people
Nested relational model:
Discovering cultural groups based on shared relational knowledge
I1I2I3I5I7I8I9I10I12
L1L2L3L4L5L6L7L8L9L10L11L12I6I11
Clusters of people found:• Guatemala studies of Atran & Medin– Subjects
• 12 native Itza’ maya
• 12 immigrant Ladino
• 12 immigrant Q’eqchi’ maya
– Questions• Does plant i help animal j?
Q3Q6Q8Q9Q10Q11Q12
Q1Q2Q4Q5Q7
I4
Talk outline
• Classic cluster-based methods
• New methods– Clustering with arbitrary relational systems– Hierarchical relational clustering– Cross-cutting clustering with nested models– Cross-cutting relational clustering
• Application to Guatemala data from Atran & Medin
• Conclusions and future directions
Conclusions
• A flexible toolkit for statistical learning about cultural knowledge and cultural groups. – Relational models: analyze arbitrary relational databases of beliefs,
not just a single matrix– Nonparametric models: automatically determine complexity of
representations– Hierarchical models: multiple levels of structure– Nested models: structures with structure
• Can automatically discover important qualitative structure in real-world data (Atran & Medin, DARPA CPoF).
Ongoing and future work
• Algorithms that can scale to very large networks.• More dynamic data and models.
– Second-generation Guatemala data
– Political data sets: voting records, international relations
• Better statistical models for sparse networks.• More structured representations necessary to capture
“cultural stories”: grammars, logical schemas. • Multi-level statistical models for learning about network
structure from raw event data.
Learning networkstructure from rawevent data
edge (N)
class (Z)
edge (N)
1 2 3 4 5 6
7 8 9 10 11 12 13 14 15 16
# of samples: 20 80 1000
Data D
Network N
Data D
Network N
AbstractClasses
1 2 3 4 5 6…
7 8 9 10 11 12 1314 15 16…
…
0.40.0
0.0 0.0…
…
(Mansinghka, Kemp, Tenenbaum, Griffiths UAI 06)
c1 c2
c1
c2
c1
c2
Classes Z
edge (N)
class (Z)
edge (N)
12
3
4567
8
9
1011 12
# of samples: 40 100 1000
Data D
Network N
Data D
Network N
AbstractClasses
1 2 3 4 5 6 7 8
9 10 11 12…
0.1
c1
c1
c1
Classes Z
…
…
…
Learning networkstructure from rawevent data
(Mansinghka, Kemp, Tenenbaum, Griffiths UAI 06)
Learning abstract structure in networks
Primate troop Bush administration Prison inmates New Guinea islands “beats” “told” “likes” “trades with”
Dominance hierarchy Tree Cliques Ring
The end
Discovering structure in relational data
391
135
117
142
106
1248
15
3 9 1 13 5 11 7 14 2 10 6 12 4 8 15
3 9 113 511
7 14 2
10 6
12 4
8 15
123456789
101112131415
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Input Output
pers
on
TalksTo(person,person)
person
O
z
Infinite Relational Model (IRM)
3 9 113 511
7 14 2
10 6
12 4
8 15
0.90.1 0.1
0.1 0.1 0.9
0.9 0.1 0.1
391
135
117
142
106
1248
15
3 9 1 13 5 11 7 14 2 10 6 12 4 8 15
Model fitting
O
z
Infinite relational model (IRM)
3 9 113 511
7 14 2
10 6
12 4
8 15
0.90.1 0.1
0.1 0.1 0.9
0.9 0.1 0.1
391
135
117
142
106
1248
15
3 9 1 13 5 11 7 14 2 10 6 12 4 8 15
O
z
Infinite relational model (IRM)
3 9 113 511
7 14 2
10 6
12 4
8 15
0.90.1 0.1
0.1 0.1 0.9
0.9 0.1 0.1
123456789
101112131415
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
• Independent symmetric beta priors on :
• Chinese Restaurant Process over z:
• Goal: – Infer
– Infer (integrating out to reduce space of unknowns)
Generating and z
)(Beta~ ββ,ηij
)|,( Oηzp)|( Ozp
class new a is
0
),,|( 11C
αn
α
nαn
n
zzCzPC
C
nn
Global-local search process
Joint modeling of belief systems and social systems
anim
al
plant
person
helps(plant,animal,person judging)
Data from Atran and Medin
Itza Ladinos
