Comparison of Five Clustering Algorithms to ClassifyPhytoplankton From Flow Cytometry DataMalcolm F. Wilkins,1 Sam A. Hardy,2 Lynne Boddy,1* and Colin W. Morris2

1Cardiff School of Biosciences, Cardiff, United Kingdom2School of Computing, University of Glamorgan, Trefforest, Pontypridd, United Kingdom

Received 29 September 2000; Revision Received 8 March 2001; Accepted 12 April 2001

Background: Artificial neural networks (ANNs) havebeen shown to be valuable in the analysis of analyticalflow cytometric (AFC) data in aquatic ecology. Automatedextraction of clusters is an important first stage in derivingANN training data from field samples, but AFC data pose anumber of challenges for many types of clustering algo-rithm. The fuzzy k-means algorithm recently has beenextended to address nonspherical clusters with the use ofscatter matrices. Four variants were proposed, each opti-mizing a different measure of clustering “goodness.”Methods: With AFC data obtained from marine phyto-plankton species in culture, the four fuzzy k-means algo-rithm variants were compared with each other and withanother multivariate clustering algorithm based on criticaldistances currently used in flow cytometry.

Results: One of the algorithm variants (adaptive dis-tances, also known as the Gustafson–Kessel algorithm)was found to be robust and reliable, whereas the othersshowed various problems.Conclusions: The adaptive distances algorithm was supe-rior in use to the clustering algorithms against which it wastested, but the problem of automatic determination of thenumber of clusters remains to be addressed. Cytometry 44:210–217, 2001. © 2001 Wiley-Liss, Inc.

Key terms: automatic cluster extraction; clustering algo-rithms; phytoplankton; neural networks

Analytical flow cytometry (AFC) is a proven tool inaquatic ecology for the rapid analysis of water samples fordetecting and quantifying microorganisms including phy-toplankton and bacteria (1–3). The various light scatter,diffraction, and fluorescence parameters measured by AFCcan provide characteristic “signatures” for each microbialcell, which allow taxa to be discriminated with the use ofpattern-recognition techniques such as artificial neuralnetworks (ANNs) (4–9). More than 70 species have beenidentified successfully by ANNs trained on AFC data ob-tained from pure cultures of marine microalgae grownunder controlled conditions in the laboratory (6). How-ever, species growing in the field are likely to showgreater variability in size, shape, and pigmentation due toa multitude of environmental factors (10), thus producinga corresponding increase in the variability of the AFCsignatures. ANNs require training on a representative sam-ple of each species that is to be recognized, and, unlessthe training data reflect such biological variation, ANNanalysis of field samples will not be reliable. There istherefore a need for a procedure for extracting ANNtraining data from field samples. The first step in such amethod is to separate the AFC data obtained from fieldsamples into their constituent clusters, e.g., for cell sort-ing with subsequent microscopic identification. Aside

from obtaining ANN training data, discerning clusters isthe first step in the interpretation of AFC data.

The large data sets generated by AFC pose a number ofchallenges for analysis. The data sets are often multidimen-sional, making visual analysis by two-dimensional or three-dimensional scatter plots difficult, even with the use ofdimensionality reduction techniques such as principalcomponent analysis (11). Recourse is generally made tosome form of clustering algorithm capable of exploitingthe full multivariate nature of the data. However, thetypical size of AFC data sets (.104 patterns) precludesapplication of many standard clustering algorithms such aspair-group methods that rely on calculation of distancesbetween pairs of points. AFC clusters are frequentlyhighly elongated, and the variance–covariance structurecan differ considerably between clusters. Some clustersare large and sparse, whereas others are compact anddense and perhaps even degenerate (possessing zero vari-ance in one or more dimensions and therefore zero vol-

Grant sponsor: AIMS, Commission of the European Community; Grantnumber: (CEC) MAS3-CT97-0080.

*Correspondence to: Lynne Boddy, Cardiff School of Biosciences, P.O.Box 915, Park Place, Cardiff CF10 3TL, UK.

E-mail: boddyl@cardiff.ac.uk

© 2001 Wiley-Liss, Inc. Cytometry 44:210–217 (2001)

ume). Demers et al. (12) proposed an iterative clusteranalysis algorithm for AFC data capable of modeling theheterogeneous nature of typical AFC clusters, but it wasnot always successful (see below). The current study com-pares the performance of the algorithm of Demers et al.(12) with four others (apparently not previously appliedto AFC data) and with manual cluster extraction usingtwo-dimensional and three-dimensional scatter plots withthe use of artificially constructed marine phytoplanktonAFC data sets.

THE ALGORITHMSThe algorithm of Demers et al. (12) (Appendix 1),

henceforth denoted as DKLL, was based on an extensionof the classic k-means algorithm (13) to allow for non-spherical clusters through the use of variance–covariancematrices (scatter matrices) to model the data distributionaround each cluster center. A common problem with suchiterative algorithms is that, over the course of severaliterations, one cluster tends to expand to include all thedata at the expense of the other clusters. The DKLLalgorithm incorporated the use of threshold “critical dis-tances” to prevent this. At each step of the algorithm, onlythose points within the critical distance of the clustercenter were used in calculating the corresponding scattermatrix. The critical distances were increased graduallyduring successive iterations under the assumption that theclustering progressively would become more reliable. Inpractice, however, that approach was not ideal: there areno guidelines as to how rapidly the critical distancesshould be increased, and the number of data points withinthe critical distance of the cluster center may well fall tozero during the course of the algorithm, leaving the clus-ter scatter matrix undefined. Furthermore, the algorithmfrequently fails to converge to an optimal clustering solu-tion, requiring user intervention to reject inadequate re-sults (12).

Fuzzy k-means clustering (14,15) might overcome theseproblems. It is a variant of classic k-means clustering thatallows data points to become associated to some degreewith all clusters and not just the closest cluster, therebyreflecting the inherent uncertainty in allocating a datapoint to a single cluster where there are several potentialcandidates. The algorithm is in practice stable and robustand less likely than the classic k-means algorithm to pro-duce inadequate clustering solutions. The algorithm hasbeen extended by different investigators to include theuse of scatter matrices, thereby allowing modeling of datawith nonspherical clusters. Rousseeuw et al. (16) unifiedthat work by proposing four variants of a generic fuzzyk-means algorithm: adaptive distances (AD), maximumlikelihood (ML), minimum total volume (MTV), and sumof all normalized determinants (SAND; Appendix 2). Eachvariant was designed to minimize a different objectivecriterion (measure of clustering “goodness”). The AD al-gorithm, also known as the Gustafson–Kessel algorithm(16), seeks to minimize the fuzzy sum of squared gener-alized distances of the data patterns to the cluster centers,subject to the constraint that the determinant of the scat-

ter matrices (a measure of cluster volume) can be fixed inadvance. The ML algorithm assumes that each clusterrepresents a multivariate normal probability distributionand attempts to find a clustering solution that maximizesthe overall likelihood of the data set over all clusters anddata items; this method tends to seek cluster solutionswhere all clusters have similar volumes (16). The MTValgorithm minimizes the total cluster volume and is biasedtoward finding clustering solutions where the clustershave similar densities rather than similar volumes. TheSAND algorithm attempts to reduce this bias by normaliz-ing the cluster volumes by dimensionality.

MATERIALS AND METHODSData Sets

A large set of 7-parameter FACScan flow cytometric datafor 63 marine phytoplankton species, each grown in pureculture (6), was used. The data sets were examined visu-ally using three-dimensional principal component analysisscatter plots to determine which species had data distri-butions that overlapped and which did not. Two artificialdata sets, A and B, respectively, each of 10 species, wereconstructed from the plots to produce a data set compris-ing species for which the corresponding AFC clusters didnot overlap appreciably (Fig. 1a) and a data set comprisingspecies for which the corresponding AFC clusters over-lapped considerably (Fig. 1b). Data sets were constructedby randomly selecting 103 data patterns for each of thesespecies. Both data sets thus contained a total of 104 datapatterns.

Comparing the Performances of theFive Algorithms

Each algorithm was run five times on the nonoverlap-ping (A) and overlapping (B) data sets for k 5 6, k 5 10,and k 5 14 clusters. Two criteria were used for evaluatingperformance: (i) consistency of results between multiplereplicate runs of the algorithm for the same value of k and(ii) performance of the algorithm in comparison with a“gold standard” clustering scheme (corresponding to theactual identity of the data patterns, which for this problemwas known in advance). Consistency between multiplealgorithm runs was assessed by averaging a similarity mea-sure (see next section) between clustering schemes frompairs of replicates. Consistency with the gold standardwas assessed by averaging the similarity measure betweenthe clustering schemes from five replicate results obtainedwith k 5 10 and the gold standard clustering scheme. Forthe nonoverlapping data set, each algorithm was evalu-ated with both criteria; for the overlapping data set, onlycriterion (i) was used, because the gold standard is takenas the known identity of a particular cluster: for thenonoverlapping data, each species represents a distinctcluster, but this is not the case for the overlapping dataset. By using traditional taxonomic characters, the 10species in data set B would form distinct clusters, but thiswas not the case with the seven flow cytometric param-eters measured in the present study, which resulted in

211COMPARISON OF FIVE CLUSTERING ALGORITHMS

overlapping distributions. Hence, in the case of data set B,we do not know what the “gold standard” should be.

A Similarity Measure for Comparing the Resultsof Two Clusterings

A good clustering algorithm should produce results thatare similar to the clusters that can be noted visually in thedata (where these are clearly defined) and are consistent,i.e., similar over several replicate runs of the algorithm onthe same data. The result of running a clustering algorithmon a set of data is a partitioning of the data space intoregions; applying this partition to the data set (by assign-ing each data point an identity based on the region inwhich it resides) results in a “clustering scheme” for thedata. A clustering scheme also can be defined manually byvisual inspection of the data with the use of scatter plots.An objective method is thus required for quantifying thesimilarity between two clustering schemes. Given twoclustering schemes, C and D, the proposed measure ofsimilarity between them, m(C,D), is defined as the a prioriprobability that two randomly selected points drawn fromthe data set will be clustered in the same way under bothclustering schemes; i.e., that both clustering schemes willassign both points to the same cluster or both clusteringschemes will assign them to separate clusters. By letting cbe the event “both points are assigned to the same clusterunder C” and d be the event “both points are assigned tothe same cluster under D,” we can write m(C,D) 5p(c)p(duc) 1 p(c9)p(d9uc9). The characteristics of this sim-ilarity measure are: (i) if C and D are identical (perfectcorrelation), p(duc) 5 p(d9uc9) 5 1.0, thus m(C,D) 5 1.0;(ii) if C and D are independent (no correlation), p(duc) 5

p(d) and p(d9uc9) 5 p(d9), thus m(C,D) 5 p(c)p(d) 1p(c9)p(d9); and (iii) it is commutative; i.e., m(C,D) 5m(D,C).

Because of the large size of typical AFC data sets, deter-mining m(C,D) by exhaustive enumeration of all possiblepairs of data points is computationally unfeasible; how-ever, the proportion can be estimated to a desired level ofaccuracy, and confidence bounds placed on the value areobtained by repeatedly, randomly drawing a large numbern of pairs of data points and finding the proportion p ofsuch trials for which the two schemes agree. The sam-pling standard deviation of p, a measure of the size of theuncertainty in the value of p as an estimator of m(C,D), isgiven by q 5 (np[1 2 p])1/2/n 5 (p[1 2 p]/n)1/2. Allsimilarity measures reported in this paper were evaluatedfrom n 5 107 pairs of data points.

Computer Software

All experiments used AimsNet software running on a PCunder Microsoft Windows NT (http://www.cf.ac.uk/biosi/staff/wilkins/aimsnet), a computer package devel-oped as part of AIMS, a CEC-funded project (grant MAS3-CT97-0080).

RESULTS AND DISCUSSIONData Set A (Nonoverlapping Clusters)

With k 5 10 clusters, AD produced clustering schemesin which each cluster was dominated by data patternsfrom a single species (Fig. 2); it had the highest consis-tency (i.e., lowest variability) between replicates and per-formed best in comparison with the gold standard (Table

FIG. 1. Plots illustrating the distributions of the 10 species in data sets A (a; nonoverlapping clusters) and B (b; overlapping clusters) projected in eachcase onto the plane of the first two principal components. For data set A, all 10 clusters are clearly resolved in three-dimensional plots of principalcomponents analysis (not presented).

212 WILKINS ET AL.

1). DKLL performed marginally less well, being less con-sistent between replicates and occasionally finding “com-posite” clusters containing two species, resulting inpoorer performance in comparison with the gold stan-dard. SAND performed slightly less well in terms of con-sistency between replicates, producing a higher propor-

tion of composite clusters. MTV frequently failed toconverge to a solution, particularly when the number ofclusters specified differed from the optimal, or “natural,”number of clusters (further investigation showed that ittended to become trapped cycling between two cluster-ing solutions). ML frequently produced a large aggregatecluster comprising patterns from eight or more species,resulting from the unusual way in which it partitioned thedata space: it tended to generate small regions (represent-ing the densely populated clusters) enclosed by a singlelarge region (representing the remainder of the space). Allthe other algorithms produced partitions in which theregions were adjoining but continuous (i.e., in which noregion was contained within another).

The effect of decreasing and increasing the specifiednumber of clusters on the clustering solutions was inves-tigated. Decreasing the number of clusters specified hadthe effect of combining some species within a cluster andsplitting some species between clusters (Fig. 3). Under-specifying the number of clusters in comparison with the“natural” number resulted in the results of the AD algo-rithm becoming less consistent (i.e., more variable) be-tween replicates than DKLL (Table 1). Overspecifying thenumber of clusters usually resulted in splitting speciesbetween clusters, with most clusters representing only asingle species; under those circumstances, the results of

FIG. 2. Typical partitioning of data set A(nonoverlapping clusters) between 10 clus-ters by algorithms for adaptive distances(AD), Demers (DKLL), sum of all normalizeddeterminants (SAND), and maximum likeli-hood (ML). The bars show the percentage ofthe data set associated with each cluster andthe composition of the data associated witheach cluster in terms of the 10 constituentspecies.

Table 1Comparison of Algorithms for Data Set A

(Nonoverlapping Clusters)

Number ofclusters

Consistency betweenreplicatesa

Consistency of resultscompared with “gold

standard”

6 10 14 10

ADb 0.906 0.962 0.991 0.963DKLL 0.921 0.947 0.965 0.948ML 0.857 0.890 0.929 0.843MTV c c c c

SAND 0.866 0.939 0.955 0.939aProportion of trials for which two schemes agree. Values

range between 0 and 1.0 for no agreement to complete agree-ment. Standard deviation of the proportion can be derived fromthe equation given in the text.

bAD, adaptive distances; DKLL, from Demers et al. (12); ML,maximum likelihood; MTV, minimum total volume; SAND, sumof all normalized determinants.

cDid not converge to a solution.

213COMPARISON OF FIVE CLUSTERING ALGORITHMS

the AD algorithm were much more consistent betweenreplicates than DKLL.

Data Set B (Overlapping Clusters)

All algorithms converged when 10 clusters were spec-ified, although MTV failed to do so when other numbers ofclusters were specified. AD performed significantly moreconsistently than the DKLL or SAND for k 5 10 and k 514, although not for k 5 6 (Table 2). The results for SAND(Fig. 4) appeared to produce closer agreement with thevisually defined clusters (Fig. 1b).

Defining the Number of Clusters

In the present study, the number of clusters k wasspecified beforehand, based, e.g., on preliminary visualinspection of the data or prior knowledge of the problemdomain. AD performed the most consistently with bothdata sets, provided that the number of clusters specified

was the same as or greater than the actual number ofclusters present in the data. DKLL was almost as good, andboth are worth further investigation. SAND was morevariable, ML was consistently poor, and MTV was prone tononconvergence, making these algorithms unsuitable forthe type of clustering application described in this study.

In general, no prior information is available on thenumber of clusters in a data set, and trying to decide onthe number of clusters is the classic problem in clusteranalysis (e.g., 17). The view has been taken that numberof clusters has to be decided subjectively and that thisnecessitates the involvement of a biologist expert in thefield. A trial-and-error approach often is adopted to depictthe data structure appropriately (e.g., 18). Because com-puters have become fast and efficient, this is not a seriousdisadvantage because a wide range of potential numbersof clusters can be tried. It can be cogently argued thatnonhierarchical clustering approaches do not stand wellon their own and, in many cases, should be used withother complementary methods of exploratory data analy-sis, which will give further insight into numbers of clus-ters (18).

Many techniques for automatically determining thenumber of clusters from the data have been proposed,with most of the early ones incorporating sums of squaresand having a geometric or statistical interpretation (19).Interesting, recent developments have used minimizationof a cost function and/or persistence of clusters at differ-ent length scales (20–22). However, that method may notyield the number of clusters expected or noted visually.

There are some clustering approaches that do not re-quire the number of clusters to be specified in advance,e.g., density clustering (23) and adaptive clustering (24).Density clustering is a nonparametric, noniterative statis-tical method that essentially examines the local event

FIG. 3. Typical partitioning of data set A(nonoverlapping clusters) between 6 and 14clusters by algorithms for adaptive distances(AD) and Demers (DKLL); symbols and dataas in Figure 2.

Table 2Comparison of Algorithms for Data Set B

(Overlapping Clusters)

Number ofclusters

Consistency betweenreplicatesa

6 10 14

ADb 0.878 0.971 0.973DKLL 0.938 0.923 0.930ML 0.764 0.880 0.876MTV c 0.954 c

SAND 0.877 0.923 0.959aAs in Table 1.bAD, adaptive distances; DKLL, from Demers et al. (12); ML,

maximum likelihood; MTV, minimum total volume; SAND, sumof all normalized determinants.

cDid not converge to a solution.

214 WILKINS ET AL.

density (based on histogram counts) to determine modes,and then regions of monotonically decreasing densityaround each of these are grouped into a cluster (23).However, a user-specified threshold determines the size ofclusters, which is subjective. Real-time adaptive clusteringis an ANN approach in which the data pattern for each cellis presented to the network and then allocated to a clusterof similar patterns (24). When a pattern is not similar toany existing cluster, it is allocated to a new cluster. TheANN is trained to classify events according to a predefinedsimilarity measure, but this similarity measure is subjec-tively determined. Thus, although these approaches havethe advantage of not requiring the number of clusters tobe specified in advance, they are not entirely objective.

CONCLUSIONS AND FUTURE RESEARCHThe AD algorithm was superior in use to the algorithms

against which it was tested. Moreover, it is widely appli-cable and likely to be of use with other very large datasets, such as might be produced by other high-technologyequipment with rapid processing capabilities. This isclearly a big step forward but its potential cannot be fullyrealized until a suitable objective method for determiningthe number of clusters in a data set has been developed, toallow completely automatic clustering. Nonetheless, evenif subjective decisions (with or without the help of ex-ploratory data analysis methods) are made about the num-

ber of clusters, using the AD algorithm to extract data setsthat can subsequently be used for training ANNs will beextremely valuable. Ultimately, clustering software couldbe used to drive cytometer cell-sorting facilities, whichwill enable the operator to examine cells clustered to-gether on the basis of their flow cytometric signatureswith the use of other biological or physicochemical anal-yses. Cell sorting also would allow the user to checkwhether clusters contain single or several cell types.Therefore, it would be invaluable if the next generation offlow cytometers has the ability to interface custom-writtensoftware with, e.g., the cell-sorting facility.

In future research, it would be valuable to compare ADwith other clustering approaches that have been usedwith flow cytometric data including density clustering(23), real-time adaptive clustering (24), and Kohonen self-organizing maps (SOMs) (25–27). The first two ap-proaches are important because they do not require spec-ification of the number of clusters. SOMs are worthcomparing because they result in a low-dimensional, to-pology-preserving representation of the data set. They arean ANN approach in which high-dimensional data aremapped onto a two-dimensional (usually) lattice of nodes,each of which competes to represent the data presentedto the network. Patterns represented by nodes that areclose to each other in the lattice typically are similar toeach other. This approach is alluring although methods of

FIG. 4. Typical partitioning of data set B(overlapping clusters) between 10 clustersby algorithms for adaptive distances (AD),demers (DKLL), sum of all normalized deter-minants (SAND), and maximum likelihood(ML).

215COMPARISON OF FIVE CLUSTERING ALGORITHMS

separating clusters have not been fully developed, andthere is still the problem of deciding on the number ofclusters.

ACKNOWLEDGMENTSAimsNet was developed during AIMS, a project funded

by the Commission of the European Community, CECgrant MAS3-CT97-0080. Thanks are due to Dr. G. A. Tarranfor provision of the AFC data on which this work wasbased, collected as part of a Natural Environment Re-search Council PRiME Special Topic Award (GST/02/1062).

LITERATURE CITED1. Burkill PH. Analytical flow cytometry and its application to marine

microbial ecology. In: Sleigh MA, editor. Microbes in the sea. Chich-ester: Horwood; 1987. p 139–166.

2. Burkill PH, Mantoura RFC. The rapid analysis of single marine cells byflow cytometry. Phil Trans R Soc 1990;A333:99–112.

3. Hofstraat JW, Van Zeijl WJM, De Vreeze MEJ, Peeters JCH, PeperzakL, Colijn F, Rademaker TWM. Phytoplankton monitoring by flow-cytometry. J Plankton Res 1994;16:1197–1224.

4. Balfoort HW, Snoek J, Smits JRM., Breedveld LW, Hofstraat JW, Rin-gelberg J. Automatic identification of algae: neural network analysis offlow cytometric data. J Plankton Res 1992;14:575–589.

5. Boddy L, Morris CW, Wilkins MF, Tarran GA, Burkill PH. Neuralnetwork analysis of flow cytometric data for 40 marine phytoplank-ton species. Cytometry 1994;15:283–293.

6. Boddy L, Morris CW, Wilkins MF, Al-Haddad L, Tarran GA, Jonker RR,Burkill PH. Identification of 72 phytoplankton species by radial basisfunction neural network analysis of flow cytometric data. Mar EcolProg Ser 2000;195:47–59.

7. Frankel DS, Olson RJ, Frankel SL, Chisholm SW. Use of a neural netcomputer system for analysis of flow cytometric data of phytoplank-ton populations. Cytometry 1989;10:540–550.

8. Frankel DS, Frankel SL, Binder BJ, Vogt RF. Application of neuralnetworks to flow cytometry data analysis and real-time cell classifica-tion. Cytometry 1996;23:290–302.

9. Wilkins MF, Boddy L, Morris CW, Jonker R. Identification of phyto-plankton from flow cytometry data using radial basis function neuralnetworks. Appl Environ Microbiol 1999;65:4404–4410.

10. Boddy L, Wilkins MF, Morris CW, Tarran GA, Burkill PH, Jonker RR.Techniques for neural network identification of phytoplankton forthe EurOPA flow cytometer. In: Conference proceedings of Oceans94 OSATES, Brest, 1994.

11. Jolliffe IT. Principal components analysis. Springer series in statistics.New York: Springer-Verlag; 1986.

12. Demers S, Kim J, Legendre P, Legendre L. Analysing multivariate flowcytometric data in aquatic sciences. Cytometry 1992;13:291–298.

13. Hartigan J. Clustering algorithms. New York: Wiley; 1975.14. Bezdek JC. Numerical taxonomy with fuzzy sets. J Math Biol 1974;1:

57–71.15. Bezdek JC. Pattern recognition with fuzzy objective function algo-

rithms. New York: Plenum Press; 1981.16. Rousseeuw PJ, Kaufman L, Trauwaert E. Fuzzy clustering using scat-

ter matrices. Comput Stat Data Anal 1996;23:135–151.17. Dale MB. Knowing when to stop: cluster concept–concept cluster.

Coenoses 1988;3:11–32.18. Podani J. Introduction to the exploration of multivariate biological

data. Leiden, The Netherlands: Backhuys Publishers; 2000.19. Milligan GW, Cooper MC. An examination of procedures for deter-

mining the number of clusters in a data set. Psychrometrika 1985;50:159–179.

20. Chakravarthy SV, Ghosh J. Scale-based clustering using the radialbasis function network. IEEE Trans Neural Netw 1996;7:1250–1261.

21. Frattale Mascioli FM, Rizzi A, Panella M, Martinelli G. Scale-basedapproach to hierarchical fuzzy clustering. Sig Proc 2000;80:1001–1016.

22. Kothari R, Pitts D. On finding the number of clusters. Pattern RecogLett 1999;20:405–416.

23. Conrad MP. A rapid nonparametric clustering scheme for flow cy-tometry data. Pattern Recog 1987;20:229–235.

24. Fu L, Yang M, Braylan R, Benson N. Real-time adaptive clustering offlow cytometric data. Pattern Recog 1993;26:365–373.

25. Kohonen T. The self-organising map. Proc IEEE 1990;78:1464–1480.

26. Boddy L, Morris CW. Artificial neural networks for pattern recogni-tion. In: Fielding AH, editor. Machine learning methods for ecologicalapplications. Boston: Kluwer; 1999. p 37–87.

27. Boddy L, Wilkins MF, Morris CW. Pattern recognition in flow cytom-etry. Cytometry 2001. Forthcoming.

APPENDIX 1: DKLL ALGORITHMThe DKLL algorithm described by Demers et al. (12) can

be summarized as follows:

1. For each cluster t, initialize mt to a randomly selecteddata pattern and initialize St to the identity matrix.

2. Assign each data pattern xi to closest cluster t, i.e.,for which dit 5 mink dik.

3. For each cluster t, update mt to the mean of the setof patterns assigned to cluster t.

4. For each cluster t, calculate the critical distance, i.e.,the Euclidean distance to the closest neighboring clustercenter: ct 5 mink Þ t imk 2 mt i.

5. For each cluster t, update St to the covariance matrixof the set of patterns assigned to cluster t and fallingwithin a distance ctz of the cluster center.

6. Calculate the objective criterion F 5 Oi

di2, where

di is the generalized distance between pattern i and thecentroid of the cluster to which it has been assigned.

7. Repeat steps 2–6 until F attains a minimum value.

The quantity z is initially small (around 0.25); this isdesigned to exclude patterns for which the cluster iden-tity is uncertain from influencing the calculation of thescatter matrices. As the algorithm progresses, the clusterassignments are supposed to become more reliable and zis increased accordingly. In the implementation usedhere, z is increased by 0.005 at each iteration, up to amaximum value of 0.5. Any cluster with less than 3p datapatterns assigned to it after step 2 (where p is the datadimensionality) is deemed to be irrelevant and is deleted.

APPENDIX 2: AD, ML, MTV, AND SANDALGORITHMS

The four algorithms proposed by Rousseeuw et al. (16)are variant forms of one generic fuzzy clustering algo-rithm, summarized below. In this algorithm, each of the kclusters is characterized completely by the position of itscenter, mt, and a scatter matrix, St. The distance dit be-tween the i-th data pattern xi and the cluster center isdefined by the Mahalanobis generalized distance metricdit

2 5 (xi 2 mt)T (St)

21 (xi 2 mt). Each data pattern isassociated to some extent with all the clusters, not just theclosest; the extent of this association is given by uit, thefuzzy cluster membership of the i-th data pattern to clustert. The cluster memberships range from 0 (no association)to 1 (complete association) and are subject to the

constraint that Oi

uit 5 1; in other words, for any data

pattern, the sum of its cluster memberships over all clus-ters is 1. Constraining uit strictly to the values 0 and 1produces a “crisp” clustering, for which each data patternis associated exclusively with only one cluster. Allowing

216 WILKINS ET AL.

uit to take intermediate values between 0 and 1 results ina fuzzy clustering.

The generic algorithm described by Rousseeuw et al.(16) can be summarized as follows:

1. For each cluster t, initialize mt to a randomly selecteddata pattern and initialize St to the identity matrix.

2. For each cluster t, calculate coefficients At and Bt (asbelow).

3. Calculate memberships for each data pattern xi:i. Initialize Ti to an empty set.ii. For each cluster t, calculate Bit 4 Btdit

2.iii. For each cluster t, calculate

uit 41/BitO

r¸Ti

1/Bir

21

Bit 3 Or¸Ti

Ar/Bir

Or¸Ti

1/Bir

2 At4, t¸Ti

uit 4 0, t [ Ti

iv. If any uit , 0, add t to set Ti and repeat step (iii)4. For each cluster t, update mt and St as follows:

mt 4

Oi

uit2 xi

Oi

uit2 St 4

Oi

uit2 ~xi 2 mt!~xi 2 mt!

T

Oi

uit

5. If no membership uit changed by more than a smallvalue e during the previous iteration, stop; otherwisereturn to step 2.

The different variants are obtained by changing themanner of calculating the quantities At and Bt. For thealgorithms AD, MTV, and SAND:

At 51

2

t

bnt

pb2t21~utuStu!b, Bt 5 ntpb2t21~utuStu!b

where p is the data dimensionality, nt 5 Oi

uit, and the

parameters b and t are given by b 5 1/p, t 5 0 (AD), b 51/2, t 5 p/2 (MTV), and b 5 1/p, t 5 1 (SAND). For thesethree variants, a reasonable a priori assumption is that allclusters should start with the same volume, i.e., ut 5 1 forall t.

For the ML algorithm, which minimizes a different typeof objective function,

At 5 21

2loguStu, Bt 5 1.

For all four algorithms, the value of e, indicating conver-gence, was taken to be 0.01.

217COMPARISON OF FIVE CLUSTERING ALGORITHMS