june 11, 2005interface / csna meeting st. louis1 estimating the cluster tree of a density werner...

June 11, 2005 Interface / CSNA Meeting St. Louis 1

Estimating the Cluster Treeof a Density

Werner StuetzleRebecca NugentDepartment of StatisticsUniversity of Washington


Groups K-means with k = 2


• Detect that there are 5 or 6 distinct groups.

• Assign group labels to observations.

40 45 50 55

74

76

78

80

82

84


-2 0 2 4 6 8 100

10

02

00

30

0

Feature histogram


ClusteringasastatisticalproblemAssumethatfeaturevectorsx1;:::;xniid»p(x).DenelevelsetL(;p)byL(;p)=fxjp(x)gLetL1(;p);L2(;p);:::betheconnectedcomponentsofL(;p).If2>1thenforanyi;jeither²Li(2;p)½Lj(1;p),or²theyaredisjoint.ThereforethetotalcollectionofsetscanbearrangedintoamodetreeLeavesofmodetreecorrespondtonodesofp(x).

Considerfeaturevectorsx1;:::;xnasarandomsamplefromsome(innite)population.Letp(x)bethepopulationdensity.(Think"histogramwithinnitesimallysmallbins")DenelevelsetL(;p)byL(;p)=fxjp(x)gLetL1(;p);L2(;p);:::betheconnectedcomponentsofL(;p).If2>1thenforanyi;jeither²Li(2;p)½Lj(1;p),or²theyaredisjoint.Thereforethetotalcollectionofsetscanbearrangedintoatree,theclustertreeofthedensity.Leavesofclustertreecorrespondtomodesofp(x).

(a) (b) (c)

(d) (e) (f)




(a) (b) (c)

(d) (e) (f)


0 1 2 3 4

-10

12

34

rs = 1

rs = 5

rs = 2


Note large tree fragments in bottom left panel

-2 0 2 4 6

-3-2

-10

12

3

Weakly bimodal data

-2 0 2 4 6

-3-2

-10

12

3

MST for weakly bimodal data

-2 0 2 4 6

-3-2

-10

12

3

MST after removal of longest edges

0 20 40 60 80

01

02

03

04

0

runts .bimodal2[runts.bimodal2 > 1]

Histogram of runt sizes


-2 -1 0 1 2 3

-3-2

-10

12

3

Unimodal data

-2 -1 0 1 2 3

-3-2

-10

12

3

MST for unimodal data

-2 -1 0 1 2 3

-3-2

-10

12

3


0 20 40 60 80

01

02

03

04

0

runts .unimodal[runts.unimodal > 1]



leve

l

10

12

14

16

A 8

A 7 A 9

A 8

A 2

A 3A 3

A 5A 6


cc1

cc2

-4 -2 0 2 4 6

-4-2

02

4

11

1

111

1

1

11

11

1

1

1

1

1

1

1

1

1

1

1

1

1

22

22

22

2

2

2

2

2

2

2

2

2222

2 2

2 22

2

2

2

2

222

2

2

2

2222

2

2

22

2

2

2

2 22 22

22

2

2

2

2

2

33

3

3

33

3

3

3

3

33

3

3

3

3

3

33

33

3

3

3

3

3

3

3

33

33 3

3

33

3

3

3

3

33

333

33

3

3

3

3

3

3 33

3

33

3

33

3 3

3

3

3

3333

3 33

3

3

3

3

33

3

33

3

33

33

3

3

3

33

3 33

3

3

333

33

33

3

33

3

3

3

3

3

3

3 33

3

33

3

3

3

3

33

3

3

333

3

3

3

3 33

3

3

3

33

33

33 3

33

3

33

3

3

3

3

33

33

33

33

3

3

3

3

33

3

33

3

3

33

33

3

33

3

3

3

33

33

3 3 3

3

333

3

3

3

3

3

3 3

3

3

3

3

4

44

4

4

44

4

4

4

4

44

44

4

4

4

4

4

4

4

44

4

4

44

4

44

4

44

4 4

Areas 1-4, projected on crimcords


0 1 2 3 4

-10

12

34

rs = 1

rs = 5

rs = 2


Runt Pruning algorithm:

generate_cluster_tree_node (mst, runt_size_threshold) { node = new_cluster_tree_node node.leftson = node.rightson = NULL node.obs = leaves (mst) cut_edge = longest_edge_with_large_runt_size (mst, runt_size_threshold) if (cut_edge) { node.leftson = generate_cluster_tree_node (left_subtree (cut_edge), runt_size_threshold) node.rightson = generate_cluster_tree_node (right_subtree (cut_edge), runt_size_threshold) } return (node)}

0 1 2 3 4

-10

12

34

rs = 1

rs = 5

rs = 2


Note large tree fragments in bottom left panel

-2 0 2 4 6

-3-2

-10

12

3

Weakly bimodal data

-2 0 2 4 6

-3-2

-10

12

3

MST for weakly bimodal data

-2 0 2 4 6

-3-2

-10

12

3


0 20 40 60 80

01

02

03

04

0

runts .bimodal2[runts.bimodal2 > 1]



-2 -1 0 1 2 3

-3-2

-10

12

3

Unimodal data

-2 -1 0 1 2 3

-3-2

-10

12

3

MST for unimodal data

-2 -1 0 1 2 3

-3-2

-10

12

3


0 20 40 60 80

01

02

03

04

0

runts .unimodal[runts.unimodal > 1]



leve

l

10

12

14

16

A 8

A 7 A 9

A 8

A 2

A 3A 3

A 5A 6


cc1

cc2

-4 -2 0 2 4 6

-4-2

02

4

11

1

111

1

1

11

11

1

1

1

1

1

1

1

1

1

1

1

1

1

22

22

22

2

2

2

2

2

2

2

2

2222

2 2

2 22

2

2

2

2

222

2

2

2

2222

2

2

22

2

2

2

2 22 22

22

2

2

2

2

2

33

3

3

33

3

3

3

3

33

3

3

3

3

3

33

33

3

3

3

3

3

3

3

33

33 3

3

33

3

3

3

3

33

333

33

3

3

3

3

3

3 33

3

33

3

33

3 3

3

3

3

3333

3 33

3

3

3

3

33

3

33

3

33

33

3

3

3

33

3 33

3

3

333

33

33

3

33

3

3

3

3

3

3

3 33

3

33

3

3

3

3

33

3

3

333

3

3

3

3 33

3

3

3

33

33

33 3

33

3

33

3

3

3

3

33

33

33

33

3

3

3

3

33

3

33

3

3

33

33

3

33

3

3

3

33

33

3 3 3

3

333

3

3

3

3

3

3 3

3

3

3

3

4

44

4

4

44

4

4

4

4

44

44

4

4

4

4

4

4

4

44

4

4

44

4

44

4

44

4 4

Areas 1-4, projected on crimcords


-10 -5 0 5 10

-50

51

0

PC1

PC

2


-10 -5 0 5 10

-50

51

0

PC1

PC

2

-10 -5 0 5 10

-50

51

0

PC1

PC

2


Runt Pruning algorithm:

generate_cluster_tree_node (mst, runt_size_threshold) { node = new_cluster_tree_node node.leftson = node.rightson = NULL node.obs = leaves (mst) cut_edge = longest_edge_with_large_runt_size (mst, runt_size_threshold) if (cut_edge) { node.leftson = generate_cluster_tree_node (left_subtree (cut_edge), runt_size_threshold) node.rightson = generate_cluster_tree_node (right_subtree (cut_edge), runt_size_threshold) } return (node)}

0 1 2 3 4

-10

12

34

rs = 1

rs = 5

rs = 2

june 11, 2005interface / csna meeting st. louis1 estimating the cluster tree of a density werner...

Documents

interface csna meeting

louis2 slide

louis3 slide

louis5 slide

louis7 slide

louis8 slide

louis9 slide

louis10 slide