an ordered clustering algorithm based on fuzzy c-means and...

Vol.:(0123456789)1 3

International Journal of Machine Learning and Cybernetics (2019) 10:1423–1436 https://doi.org/10.1007/s13042-018-0824-7

ORIGINAL ARTICLE

An ordered clustering algorithm based on fuzzy c-means and PROMETHEE

Chengzu Bai1,2 · Ren Zhang1,2 · Longxia Qian1,2 · Lijun Liu3 · Yaning Wu4

Received: 4 June 2017 / Accepted: 8 May 2018 / Published online: 16 May 2018 © Springer-Verlag GmbH Germany, part of Springer Nature 2018

AbstractThe ordered clustering problem in the context of multicriteria decision aid has been increasingly examined in management science and operational research during the past few years. However, the existing clustering algorithms may not provide an exact suggestion for a partition number for decision makers by using the diagram method. In addition, these methods may be not appropriate for real-life problems under big data environments due to their high computational complexities. Therefore, we propose a new clustering algorithm called the ordered fuzzy c-means clustering algorithm (OFCM) to overcome the abovementioned deficiencies. Different from the classical fuzzy c-means clustering algorithm, we use the net outranking flow of PROMETHEE and validity measures for clustering to establish a new objective function, whose properties are math-ematically justified as well. Finally, we employ OFCM to solve a practical ordered clustering problem concerning the human development indexes. A comparison analysis with existing approaches is also conducted to validate the efficiency of OFCM.

Keywords Multicriteria decision aid · Ordered cluster · Fuzzy c-means clustering · PROMETHEE method

1 Introduction

Supervised classification, i.e., assigning alternatives to pre-defined classes, is a classic problem encountered in multic-riteria decision aid (MCDA). To be more specific, this topic has received tremendous attention from fields such as clini-cal problem [31], marketing [22], medicine [15], produc-tion systems [21], economic and financial management [34],

etc. [20, 26, 29, 33]. Moreover, in the context of MCDA, a number of scholars have developed novel and excellent approaches such as ELECTRE-SORT [13], ELECTRE TRI [27], Flowsort [18], PAIRCLASS [10], PROAFTN [1], UTADIS [32], etc. Generally, these methods assume the classes are defined a priori by a set of alternatives and their central or limit profiles.

The challenge with the supervised classification is that the groups sometimes are unknown a priori, i.e., no informa-tion is given on the data structure. Recently, several methods have addressed this problem of classifying similar alterna-tives into undefined groups from a multicriteria perspec-tive [9, 12, 14]. This topic—known as MCDA combined with unsupervised classification, or clustering—mainly distinguishes three types of issues: no-relational clustering, relational clustering and ordered clustering (total or partial order of the clusters, Boujelben [4]; Meyer and Olteanu [14]). In particular, total ordered clustering can serve as a complementary tool for the ranking since it helps obtain ordered clusters and provide the priority relations among a subset of alternatives for each cluster. In addition, multicri-teria ordered clustering allows giving additional perspectives to illustrate developed rankings in terms of other informa-tion such as the academic ranking of world universities, the economy, the human development index (HDI), etc.

Electronic supplementary material The online version of this article (https ://doi.org/10.1007/s1304 2-018-0824-7) contains supplementary material, which is available to authorized users.

* Ren Zhang [email protected]

1 Research Center of Ocean Environment Numerical Simulation, College of Meteorology and Oceanography, National University of Defense and Technology, Shuanglong Road, Nanjing 211101, China

2 Collaborative Innovation Center on Forecast Meteorological Disaster Warning and Assessment, Nanjing University of Information Science and Technology, Nanjing, China

3 Meteorologic Bureau of Air Force Staff, Beijing, China4 Research Center of Software Engineering, Institute

of Information System, PLA University of Science and Technology, Nanjing, China

http://orcid.org/0000-0002-9315-330X

http://crossmark.crossref.org/dialog/?doi=10.1007/s13042-018-0824-7&domain=pdf

https://doi.org/10.1007/s13042-018-0824-7

1424 International Journal of Machine Learning and Cybernetics (2019) 10:1423–1436

1 3

For the problem of determining ordered clusters in a multicriteria context, although several statistical methods such as regression trees [19] and cluster analysis [17] may also obtain rankings, these approaches have some deficien-cies such as the definition of dependent variables and the exclusion of important factors [9]. De Smet and Gilbart [8] first established an optimized PROMETHEE method to rank groups of objects according to country risk. Then, De Smet et al. [9] evolved their initial work and proposed an exact algorithm to identify a total ordered partition based on the inconsistency matrix and the pairwise preference relations. Nevertheless, De Smet et al.’s method did not fully exploit the data structure since it only considered the ordinal prop-erties of the pairwise preference degrees [7]. As a result, Chen et al. [7] developed an ordered algorithm combining K-means and the PROMETHEE method. Compared with De Smet et al.’s method, the ordered K-means clustering algo-rithm (OKM) presented a more robust and consistent result in HDI ranks. However, OKM concluded that the larger the cluster number, the more acceptable the results obtained, which may be not appropriate in real-life problems with a large number of objects. Moreover, the existing ordered clus-tering approaches paid little attention to the fuzziness of an individual alternative’s belonging to each cluster.

The fuzzy c-means (FCM) algorithm [2] is one of the most well-known fuzzy clustering methods since it is sim-ple for modelling and can address uncertainties in practical issues. A considerable number of works have focused on this novel algorithm. For instance, Wang et al. [23] stated that an appropriate assignment of feature-weight can improve the performance of FCM clustering. Zheng et al. [30] pro-posed a generalized FCM and a hierarchical FCM to tackle image noise. Xu and Wu [25] extended the fuzzy c-means algorithm to the intuitionistic fuzzy environment. Zarinbal et al. [28] added relative entropy to the objective function to maximize the dissimilarity between clusters. Militello et al. [16] successfully introduced FCM to Gamma Knife treatment planning for improving brain tumour segmenta-tion and volume measurement. However, the existing FCM algorithms are mainly appropriate for problems of clustering the alternatives into predefined groups that have no relations among them. In the field of MCDA, the decision makers (DMs) may prefer to get “ordered clusters” in which there is ranking information among the clusters.

This paper aims to overcome the drawbacks of the earlier works and provide a new ordered fuzzy c-means clustering algorithm (OFCM). Motivated by the PROMETHEE method [5] and the validity measure for clustering [11, 24], we develop a new objective function to measure compactness and sepa-ration of ordered clusters. This function is based on the net outranking flow of PROMETHEE, which helps capture the relative priority of degrees of alternatives. Then, we construct a procedure to minimize the objective function for searching

the optimal partition number. Meanwhile, the new method is devoted to obtain acceptable ordered clustering results.

The remainder of this paper is organized as follows. Sec-tion 2 reviews the classical fuzzy c-means clustering algo-rithm, the conventional validity measure for clustering and PROMETHEE method. In Sect. 3, we propose a new objec-tive function to develop OFCM. Some important properties are mathematically justified as well. Then, Sect. 4 presents a real-life case that illustrates the advantages of our meth-odology. Moreover, we provide a comparative analysis with the similar approaches for ordered clustering problem in multicriteria context. Finally, Sect. 5 closes this paper with some conclusions.

2 Preliminaries

This section mainly presents a review of the classical fuzzy c-means clustering algorithm, a well-established validity criteria for clustering and the PROMETHEE method as follows.

2.1 Classical fuzzy c‑means clustering algorithm

The classical fuzzy c-means clustering algorithm (FCM) was proposed by Bezdek [2] to handle the clustering problem. FCM produces the clustering result by minimizing the fol-lowing objective function with fuzzy membership �ij and cluster centroid Vi

where c is the cluster number, m is the weighting expo-nent ( m ∈ [1.5, 3.0] is found to be experimentally fine, Bezdek et al. [3]), xj is an element of the sample set X =

{xj|j = 1, 2,… , n

}⊂ ℝ

p , p and n is the dimension and number of, xj respectively. ‖ ⋅ ‖ denotes the distance norm (usually, the Euclidean norm) applied by FCM. The FCM is implemented in Algorithm 1 [3]:

Algorithm 1Step 1: Randomly initialize memberships �ij of xj belong-

ing to cluster i , and

Step 2: Calculate the fuzzy centroid Vi

(1)J =

c∑i=1

n∑j=1

(�ij

)m‖‖‖xj − Vi‖‖‖

(2)c∑

i=1

�ij = 1

(3)Vi =

∑n

j=1

��ij

�mxj∑n

j=1

��ij

�m

1425International Journal of Machine Learning and Cybernetics (2019) 10:1423–1436

1 3

Step 3: Update �ij based on

Step 4: Repeat steps 2 and 3 until the value of J has only negligible changes. Then we derive the cluster results that the alternatives xj(j = 1, 2,… , n) is classified into c clusters Ck(k = 1, 2,… , c).

2.2 Traditional validity criteria

A well-known cluster validity measure is the separation index D1 [11]:

where

From Eqs. (5)–(7), it is easily seen that D1 identifies “compactness and separation” (CS) of clusters. The CS clus-tering of X can be solved by max2≤c≤n

{maxΩc

D1

} , where

Ωc represents the optimal candidates at fixed c . It is proved that if D1 > 1 , the CS ccluster can produce an acceptable result that X is classified into partitions. The larger D1 , the more separate the clusters. However, calculating this cri-terion becomes computationally very complex as c and n increase [24].

2.3 The implementation procedure of PROMETHEE method

PROMETHEE [5] is an efficient framework for imple-menting pairwise comparisons of a given set of alterna-tives A =

{ai|i = 1, 2,… , n

} evaluated on a set of criteria

G ={gk|k = 1, 2,… , s

} . The main purpose of this algo-

rithm is to obtain a total valued preference degree �(ai, aj

)

reflecting the preference intensity of ai to aj according to all the criteria. For simplicity, let gk

(ai) be the evaluation

of the alternative ai on each criterion gk . Then we present a step by step procedure of computing �

(ai, aj

) and the

associate important functions in Algorithm 2.

(4)�ij =

�1

‖xj−Vi‖� 1

m−1

∑c

i=1

�1

‖xj−Vi‖� 1

m−1

(5)D1 = min1≤i≤c

{min

i+1≤j≤c−1

{dis

(Ci,Cj

)

max1≤k≤c

{dia

{Ck

}}}}

(6)dis(Ci,Cj

)= min

xi∈Ci,xj∈Cj

‖‖‖xi − xj‖‖‖

(7)dia(Ck

)= max

xi,xj∈Ck

‖‖‖xi − xj‖‖‖

Algorithm 2Step 1: Determine the difference, denoted as dk

(ai, aj

) ,

between the evaluations of ai and aj with respect to the criterion gk:

Step 2: Transform the difference dk(ai, aj

) into unic-

riterion preference degrees using a preference function Pk

(ai, aj

) for each criteriongk.

where fk(⋅) is a monotonically non-decreasing function vary-ing between 0 and 1, i.e., the greater its value, the higher the preference of ai over aj on gk . Brans and Mareschal [5] defined six types of fk(⋅) : usual, U-shape, V-shape, level, linear and Gaussian.

Step 3: Compute the preference degree �(ai, aj

) by

aggregating all the unicriterion preference degrees Pk

(ai, aj

) in the form of weighted sum:

Step 4: Calculate the positive outranking flow �+(ai)

and the negative outranking flow �−(ai) using Eqs. (11)

and (12), respectively:

The positive outranking flow �+(ai) denotes how much

the alternative ai prefers to the other alternatives. The larger �+

(ai) , the better the alternative ai ; the negative

outranking flow �−(ai) measures how much the other

alternatives prefer to ai . The larger �−(ai) , the worse the

alternative ai.Step 5: Deduce the net outranking flow �

(ai):

�(ai) represents the total priority of ai over all the other

alternatives. The larger the net outranking flow �(ai) ,

the better the alternative ai . If �(ai)= 1 , then it means

that ai is absolutely better than the other alternatives; If �(ai)= �

(aj) , then the alternative ai is equivalent to aj.

Thus, we can obtain a complete ranking result based on the net outranking flow of each alternative.

(8)dk(ai, aj

)= gk

(ai)− gk

(aj)

(9)Pk

(ai, aj

)= fk

(dk(ai, aj

))

(10)�(ai, aj

)=

s∑k=1

wk ⋅ Pk

(ai, aj

)

(11)�+(ai)=

1

s − 1

∑x∈A�{ai}

�(ai, x

)

(12)�−(ai)=

1

s − 1

∑x∈A�{ai}

�(x, ai

)

(13)�(ai)= �+

(ai)− �−

(ai)


1 3

3 A new ordered clustering method combining with FCM and PROMETHEE

As is well known, FCM has been widely used and usu-ally applies the Euclidean norm to measure the similarity between objects. However, the Euclidean norm cannot con-sider the relative importance of the criteria under evaluation. Fortunately, the PROMETHEE method not only considers the differences between criteria but can also obtain the pri-ority degree for each pair of objects. Therefore, our interest here is to propose a new unsupervised clustering algorithm, the ordered fuzzy c-means clustering algorithm (OFCM). This algorithm will search for the best c-ordered partition and obtain acceptable clustering results based on the FCM in coordination with PROMETHEE. Some associated math-ematical properties of OFCM are also discussed.

3.1 The ordered fuzzy c‑means clustering algorithm

Let A ={aj|j = 1, 2,… , n

} be a set of alternatives of inter-

est and G ={gk|k = 1, 2,… , s

} be a set of criteria of cri-

teria. An ordered partition of A into c clusters satisfies the following three conditions:

• A =⋃

i=1,2,…,c Ci,

• ∀i ≠ j ∶ Ci ∩ Cj = �,

• C1 ≻ C2 ≻ ⋯ ≻ Cc,

where Ci denotes the i th order cluster and the symbol ≻ in Ci ≻ Cj represents the cluster Cj has a lower rank than Ci , i.e., C1 is considered to be the best cluster [9].

Similarly to the classical FCM, we define a new objective function:

where �i represents the fuzzy centroid of i th ordered cluster, and �

(aj) is the net outranking flow.

In comparison with the conventional FCM, the proposed objective function has three main advantages:

• The net outranking flow allows considering the weight of each criterion for each alternative;

• The net outranking flow can present the preferences of all the alternatives;

• J2 takes the separation of each pair of fuzzy c-partitions into consideration.

In addition, J1 in Eq. (14) describes the fuzzy deviation of �Ci

(aj) from the i-th ordered cluster. As we know, the

(14)min Jm =

∑c

i=1

∑n

j=1

��ij

�m��aj�− �i

��2

cmin1≤i,j≤c,i≠j��i − �j

��2

=J1

J2

smaller the deviation of each object in the same cluster, and the more separate the clusters, the more acceptable the clus-tering results we obtain. Therefore, the problem of divid-ing all the objects into c ordered clusters can be solved by minimizing Jm . Moreover, the objective function Jm is not a convex function, so finding an analytical solution of Eq. (14) is NP-hard, which may lead us to search for a local optimiza-tion solution instead of a global optimization. Motivated by Algorithm 1 [3], we established a procedure in Algorithm 3 to address the optimization problem of the objective func-tion [See Eq. (14)].

Algorithm 3Step 1: Compute the net outranking flow �

(aj) of each

alternative with respect to all the criteria based on Algo-rithm 2, and let c = 2.

Step 2: Randomly initialize memberships �ij of �(aj)

belonging to cluster i.Step 3: Calculate the fuzzy centroid �i

Step 4: Rank the clusters according to the fuzzy centroid �i of each cluster. For example, if 𝜗i > 𝜗j , then Ci ≻ Cj.

Step 5: Update �ij based on

Step 6: Repeat steps 3 and 4 until the value of J1 in Eq. (14) has only negligible changes.

Step 7: Compute J2 and Jm . Then, let c = c + 1 . If c = � ( � is a stop-value), stop; otherwise, return to step 2.

Let us note that for most applications, we do not need very large c , e.g., let � = n∕3 , which would very likely reach the optimal (or local optimal) solution [24]. In addi-tion, we can apply a punishing function to stop the itera-tions in Algorithm 3, which can be found in Dunn [11].

Thus, we have proposed the new objective function and the corresponding implementation procedure to solve the function. Figure 1 presents a flowchart of OFCM. In the following subsection, we will discuss several mathematical characteristics of OFCM.

3.2 Mathematical justifications

Theorem 1 The OFCM (see Algorithm 3) converges to a local minimum of Jm in the finite iterations.

(15)�i =

∑n

j=1

��ij

�m��aj�

∑n

j=1

��ij

�m

(16)�ij =

m−1

�1

‖�(aj)−�i‖∑c

i=1

�m−1

�1

‖�(aj)−�i‖�


1 3

Proof Since the six types of preference functions given by Brans and Mareschal [5] in the PROMETHEE method are monotonic functions, then the net outranking flow �

(aj) for

the associated alternative aj is one-to-one. This means that Steps 2–6 for minimizing J1 in Algorithm 3 are approximate to Steps 1–3 in Algorithm 1 for the classical FCM. Bezdek et al. [3] has already mathematically justified the theoretical convergence of FCM, i.e., FCM always converges to a local optimal solution of J1.

In addition, from Step 7 in Algorithm 3, we note that the computation of J2 is independent of minimizing J1 (i.e., Steps 2–6 in Algorithm 3). Thus, calculating J2 cannot exert influence on the convergence to a local minimum of J1 . Meanwhile, the feasible solution set of objective function Jm is finite because of the stop-value � with the upper bound. Therefore, the proof is completed. □

Property 1 Let A ={aj|j = 1, 2,… , n

} be a set of alter-

natives with the fuzzy centroid �i(i = 1, 2,… , c) of each ordered cluster Ci , �ij be the membership of the alternative aj belonging to Ci , and Jm the objective function of OFCM. If Jm < 1 , there is a distinct c partitions of A. The lower value of Jm, the more separate the clusters.

Proof Based on Eqs. (2) and (16), we can obtain

(17)

⎧⎪⎪⎨⎪⎪⎩

�i =

∑n

j=1

��ij

�m��aj�

∑n

j=1

��ij

�m ≥

∑n

j=1

��ij

�m�min(A)∑n

j=1

��ij

�m = �min(A)

�i =

∑n

j=1

��ij

�m��aj�

∑n

j=1

��ij

�m ≤

∑n

j=1

��ij

�m�max(A)∑n

j=1

��ij

�m = �max(A)

where �min(A) and �max(A) represents the maximum and minimum of all the net outranking flows �

(aj) , respectively.

Eq. (18) denotes the fuzzy centroid �i is inside the bound-ary of ordered cluster Ci for i = 1, 2,… , c . Then motivated by Xie and Beni [24], we try to find out the relationship between Jm and the separation index D1 proposed from Dunn [11]. Recalling Eqs. (7), (14) and (18), thus

In addition, based on Eqs. (6) and (14),

Thus it can be obtained from Eqs. (5), (18) and (19) that

As mentioned in Sect. 2.1.2, it has been proved that if D1 > 1 , it is appropriate to classify objects into c partitions. The larger D1 , the more separate the clusters. Therefore, the proof is completed according to the Eq. (20) and several properties of the separation index D1 . □

Finally, let us note that, as well known, the classical FCM is very simple and fast for implementation. The main framework of the OFCM is established from FCM. Thus the implementation of OFCM may be only slightly computa-tionally heavier than FCM, but generally, the computational complexities of determining the separation index D1 become much higher when c and n increase.

4 Case study and comparative analysis

In this section, we use the human development index (HDI) problem (adapted from Chen et al. [7] and De Smet et al. [9]) to validate the efficiency of OFCM. The HDI empha-sizes that people and their capabilities should receive more consideration than economic growth for assessing the development of a country. The United Nations Devel-opment Program (UNDP) has proposed the HDI rank-ing, in which 179 United Nations countries are evaluated with respect to three criteria: life expectancy, education and income index. The HDI value can be obtained by

(18)

J1 =

c∑i=1

n∑j=1

(�ij

)m‖‖‖�Ci

(aj)− �i

‖‖‖2

≤

c∑i=1

n∑j=1

‖‖‖�Ci

(aj)− �i

‖‖‖2

≤

c∑i=1

max(dia2

(Ci

))

≤ cmax(dia2

(Ci

))

(19)J2 = c min1≤i,j≤c,i≠j

‖‖‖�i − �j‖‖‖2

≥ cmin(dis2

(Ci,Cj

))

(20)Jm =J1

J2≤

cmax(dia2

(Ci

))

cmin(dis2

(Ci,Cj

)) =1(

D1

)2

Compute the net outranking flow ( )jaφ

Randomly initialize memberships ijµ

c=2

Calculate the fuzzy centroid iϑ

Rank the clusters iC

Update ijµ

Compute mJ

c=c+1

c γ<

11 1t tJ J −− = ∆

ε∆ <

YES

NO

YES

Output with

NO

minmJ{ } 1:i i c

C=

Fig. 1 The OFCM algorithm


1 3

aggregating the evaluations of the three criteria with equal weights. This aggregation leads to the final HDI rank-ing. Our interest here is just to regroup the countries into ordered clusters on the basis of the HDI and conduct a com-parative analysis with other ordered clustering algorithms.

4.1 OFCM for regrouping countries on the basis of HDI

This subsection illustrates how to use OFCM for regroup-ing the countries according to HDI for the year 2008. The complete list of the countries, as well as their associated evaluations on the three criteria, can be seen in Appendix of De Smet et al.’s work [9]. Let A =

{aj|j = 1, 2,… , n

} be

a set of the countries. The country aj corresponds to the j th place in the HDI ranking. Then, a step by step procedure for clustering based on Algorithm 3 is given as follows:

Step 1: We first compute preference degrees �(ai, aj

)

between each pair of countries and then obtain the net out-ranking flow �

(aj) of each country based on Algorithm 2.

For each criterion we choose the same linear preference function as De Smet et al.’s:

where the values of threshold pl and the weights of three criteria have been determined by De Smet et al. [9, See Table 1].

Then, we can obtain the preferences between each two countries using Eq. (10) and �

(aj) for j = 1, 2,… , 179

using Eq. (13). The results are shown in Figs. 2 and 3, respectively. Then, let c = 2.

Step 2: Randomly initialize memberships �ij of �(aj)

belonging to cluster i.Step 3: Calculate the fuzzy centroid �i for each cluster

using Eq. (15). Let m = 2.Step 4: Rank the clusters according to the fuzzy cen-

troid �i . Thus, we obtain C1 and C2.Step 5: Update �ij based on Eq. (16).

(21)fk(v) =

⎧⎪⎨⎪⎩

0, v ≤ 0

v∕pl, 0 ≤ v ≤ pl, l = 1, 2, 3

1, v ≥ pl

Step 6: Repeat steps 3 and 4 until |||Jt1 − Jt−11

||| ≤ � , where

t denotes the iteration and � = 0.001.Step 7 : Compute Jm . Then, let c = c + 1 , i f

c = 179∕3 ≈ 60 , stop; otherwise, return to Step 2. The values of Jm for different c are listed in Table 2.

It can be seen from Table 2 that the minimum of Jm is c = 2 , the second smallest value is c = 4 . According to Prop-erty 1, it is most appropriate to set the number of clusters as 2 or 4. For the same HDI problem, the cluster number predefined by De Smet et al. [9] is 4, which is acceptable and mathematically justified by our new algorithm. Therefore, we also choose 4 as the final number of the ordered clusters, which correspond to very high human developed countries, high ones, medium ones and low ones. Then, we present the ordered clustering result obtained by OFCM in Fig. 4, where

Table 1 The preference thresholds and the weights of three criteria. Adapted from De Smet et al. [9]

Parameters Life expectancy Adult literacy index

GDP

Strict preference threshold pl 0.704 0.719 0.828Weight of criterion wl 0.333 0.333 0.333

Fig. 2 The pairwise preference degrees �(ai, aj

) for each pair among

the 179 countries. The red color indicates a high preference degree whereas the blue one represents a lower preference degree

Fig. 3 The net outranking flows �(aj) of the 179 countries


1 3

the x-axis denotes the HDI ranking for the 179 countries and the y-axis represents the i th cluster for i = 1, 2, 3, 4 . For instance, the 80th country belongs to the cluster C2 , whereas the 166th country belongs to the cluster C4 . From Fig. 3, it can be easily found that the ordered grouping is highly

consistent with the HDI ranks given by the UNDP. Only one inconsistency is presented for the 158th country. In addition, the clusters are not evenly distributed. Cluster C2 contains most of the countries, and C4 contains the least. This find-ing may also follow common sense in that the number of developing countries is always larger than the numbers of developed and undeveloped countries.

Moreover, to measure the consistency and efficiency of OFCM, we repeat the aforementioned step-by-step imple-mentation for 50 times using random fuzzy memberships. The ordered clustering results are presented in Fig. S1 in the supplementary materials. Fig. S1 validates the robustness of OFCM, since the clustering results for each time are all the same. We also present the number of iterations and compu-tational times of every repeated experiment in Table S1 in supplementary (the experiments were conducted on an AMD A6 PC with 8G RAM and MATLAB 2014b). We can see that OFCM can handle the problem within an acceptable time, considering that the maximum number of iterations is 38 and the largest time cost is 1.17 s.

Finally, to study the quality of the whole preference structure induced by OFCM, we compute the Kendall’s rank correlation coefficient (CC) among the UNDP ratings, the PROMETHEE II ranking [6] and OFCM’ ranking (See Table 3). According to Table 3, OFCM seems to be more

Table 2 Values of objective function Jm

No. of clusters 2 3 4 5 6 7 8 9 10

Jm 0.052 0.103 0.063 0.088 0.070 0.081 0.096 0.071 0.124No. of clusters 11 12 13 14 15 16 17 18 19Jm 0.110 0.125 0.124 0.164 0.143 0.251 0.318 0.864 0.773No. of clusters 20 21 22 23 24 25 26 27 28Jm 0.650 0.817 0.700 0.678 1.928 0.963 0.825 1.811 1.891No. of clusters 29 30 31 32 33 34 35 36 37Jm 1.713 3.124 2.328 3.527 1.802 1.032 6.425 21.593 1.870No. of clusters 38 39 40 41 42 43 44 45 46Jm 2.047 2.045 1.701 2.891 2.272 15.509 1.606 4.004 8.433No. of clusters 47 48 49 50 51 52 53 54 55Jm 4.711 5.479 4.156 17.891 4.619 7.680 121.684 4.694 3.389No. of clusters 56 57 58 59 60Jm 13.001 83.117 7.017 3.380 3.461

Fig. 4 The ordered clustering result using OFCM. The x-axis denotes the HDI ranking for the 179 countries and the y-axis represents the number of clusters

Table 3 Kendall’s rank correlation coefficient among the UNDP ratings and different ranking methods

The correlation values are significant at the 0.01 level

Kendall’s rank correlation PROMETHEE II OFCM FCM OKM De Smet et al.’s UNDP

PROMETHEE II 1.0000 0.9995 0.5060 1.0000 0.8990 0.9893OFCM 1.0000 0.5056 0.9995 0.8988 0.9898FCM 1.0000 0.5060 0.4357 0.5116OKM 1.0000 0.8990 0.9893De Smet et al.’s 1.0000 0.8788UNDP 1.0000


1 3

correlated with UNDP ratings, which verifies that OFCM can fully exploit the data structure. Although PROMETHEE II ranking is slightly less correlated with UNDP ratings than OFCM is, it can neither solely solve the problem of regroup-ing objects into ordered categories based on their preference degrees nor automatically identifying the cluster centres.

4.2 Comparison with other clustering approaches

To validate the advantages of OFCM further, the classical FCM, De Smet et al.’s method and the OKM are included for solving the same HDI problem mentioned above.

4.2.1 Regrouping the countries by using FCM

Based on an “FCM” function in MATLAB, we apply the classical FCM clustering algorithm to regroup the countries with respect to three different criteria of HDI. The number of clusters is predefined as 4. Figure 5 shows the results derived by FCM that there are apparent inconsistencies between the partition result and HDI-ranking. The main reason behind this may be that the classical FCM measures the degree of similarity between any two countries according to the Euclidean distance. In other words, the conventional FCM cannot present preference relationships between the objects and the clusters because of the symmetry of the Euclidean distance.

4.2.2 Regrouping the countries by using De Smet et al.’s method

We use the implementation procedure of the De Smet et al.’s method proposed in Algorithm 1 from De Smet et al. [9] to

solve the HDI ranking problem. Figure 6, given by De Smet et al.’s method, shows the ordered clustering results where the partition number is 4. A shown in Fig. 6, there is a higher consistency with the ordered cluster and the HDI ranks than the result obtained by FCM. Very few inconsistencies are shown, such as for the 85th, 132nd and 159th countries. However, there are still more inconsistencies than in OFCM. In addition, the number of countries in the first cluster is 80 in Fig. 5, whereas only the 50 alternatives belong to the first cluster in Fig. 3. Meanwhile, the number of objects belong-ing to very high human developed countries is much larger than that of high human developed countries, which may not meet the common sense mentioned above. Just as De Smet et al. [9] summarized, the order of the alternatives will be affected by several elements in the preference matrix (see Algorithm 1 from De Smet et al. [9]) with the same val-ues. Therefore, De Smet et al.’s method may not capture the “subtle” changes of the elements in the preference matrix.

Moreover, we compute the Kendall’s rank correlation coefficient between the UNDP ratings and De Smet et al.’s ranking (see Table 3) to study the quality of the whole pref-erence structure induced by this exact algorithm. Compared with the score obtained by OFCM, De Smet et al.’s method impairs the performance of OFCM by approximately 11.21% in CC, which is consistent with the conclusion that De Smet et al.’s method may not fully mine the data structure since it only considers the ordinal properties of the pairwise prefer-ence degrees [7].

4.2.3 Regrouping the countries by using OKM

We apply the procedure of OKM proposed in Sect. 3.4 from Chen et al. [7] to solve the HDI ranking problem.

Fig. 5 The clustering result using FCM. The x-axis denotes the HDI ranking for the 179 countries and the y-axis represents the number of clusters

Fig. 6 The clustering result using De Smet et al.’s method. The x-axis denotes the HDI ranking for the 179 countries and the y-axis repre-sents the number of clusters


1 3

Figure 7 given by OKM presents the ordered clustering results where the partition number is 4. We notice that the results in Fig. 7 are nearly the same as those in Fig. 4, except that the 157th country is inconsistent with the HDI-ranking. However, as Fig. 4 from Chen et al. [7] illustrated, the larger the partition number, the more acceptable the clustering results. This conclusion is contrary to Table 2. In addition, because OKM is based on the K-means algo-rithm to regroup the data set into km subsets, it requires o(nkmt

) calculations, where t represents the number of

iterations. Thus, the larger cluster number can bring much higher time complexities in big data environments than those of OFCM, which only needs o(nct) ( c ≪ km ) cal-culations since OFCM can partition alternatives into the appropriate c clusters. In fact, more classes may not help decision makers to identify prototypes or profiles of alter-natives in multicriteria contexts, but instead it may just increase computational complexities.

In summary, we present the total rankings of countries and their partitions using the different clustering methods in Table 4 in Appendix. As we can see in Table 4, the partitions are consistent with the rankings provided by OFCM and OKM, except for the 157th and 158th countries. The boundaries between the different clusters remain satisfactory since they divide the developing countries, the developed ones and unde-veloped ones with appropriate proportions, i.e., the number of developed and undeveloped countries is less than that of the developing countries respectively. Although De Smet at al.’s method achieves much better performances than FCM does, the ranking results induced by the classic algorithm cannot meet the common sense mentioned above and comparatively fails to exploit the whole data structure of alternatives (See

Table 3). Therefore, what is clear from the above is that the proposed OFCM can be validated by other similar techniques.

5 Conclusion

In this paper, we have proposed a new ordered clustering algo-rithm to address the multi-criteria ordered clustering prob-lems based on the fuzzy c-means clustering algorithm (FCM), which is defined as the ordered fuzzy c-means clustering algorithm (OFCM). Different from the classical FCM using Euclidean norms, the net outranking flow in PROMETHEE method and the traditional validity measure for clustering are employed to establish a new objective function, which has intuitive meaning and is very easy to implement calculations. Several important properties of OFCM are also mathemati-cally justified.

The efficiency of OFCM has been illustrated by the human development index (HDI) problem. Meanwhile, the classi-cal FCM, De Smet et al.’s method and the ordered K-means clustering algorithm are included for comparison. The ordered clustering results demonstrate that OFCM not only helps deci-sion makers to determine the partition number but also helps decision makers to obtain the highest consistency with the ordered groups and the HDI ranks. According to the results obtained, the strengths and weaknesses of OFCM can be sum-marized as follows:

• The OFCM can identify the optimal number of clusters in a quantitative way without repeated tests.

• The OFCM generally spends much less time obtaining acceptable clustering results.

• The OFCM can fully exploit the data structure based on its own robust theoretical properties.

• However, the stopping condition of the iterations may bring high computational complexity to OFCM in the case of large-scale datasets.

Therefore, in the future, we shall employ OFCM to per-form big data clustering in a much faster iterative way. Further research is also considered to discuss how to use the non-linear preference function in PROMETHEE for improving perfor-mances. Additionally, we attempt to extend the use of other approaches (e.g. ELECTRE, MAUT) in OFCM for more prac-tical applications.

Acknowledgements This paper was supported by the National Natural Science Foundation of China (No. 51609254) and the Specific Fund (CQZ-2014001) for the Industrial Site in the City of Tangshan.

Appendix

See Table 4.

Fig. 7 The clustering result using OKM. The x-axis denotes the HDI ranking for the 179 countries and the y-axis represents the number of clusters


1 3

Table 4 Comparison between rankings of countries and their partition in 4 classes

Country OFCM FCM OKM De Smet et al.’s

Iceland 1 1 1 1Norway 1 1 1 1Canada 1 1 1 1Switzerland 1 1 1 1Netherlands 1 1 1 1Germany 1 1 1 1Ireland 1 1 1 1United States 1 1 1 1Australia 1 1 1 1New Zealand 1 1 1 1Singapore 1 1 1 1Hong Kong, China (SAR) 1 1 1 1Liechtenstein 1 1 1 1Sweden 1 1 1 1United Kingdom 1 1 1 1Denmark 1 1 1 1Korea (Republic of) 1 1 1 1Israel 1 1 1 1Luxembourg 1 1 1 1Japan 1 1 1 1Belgium 1 1 1 1France 1 1 1 1Austria 1 1 1 1Finland 1 1 1 1Slovenia 1 1 1 1Spain 1 1 1 1Italy 1 1 1 1Czech Republic 1 1 1 1Greece 1 1 1 1Estonia 1 1 1 1Brunei Darussalam 1 1 1 1Cyprus 1 1 1 1Qatar 1 1 1 1Andorra 1 1 1 1Slovakia 1 1 1 1Poland 1 1 1 1Lithuania 1 1 1 1Malta 1 1 1 1Saudi Arabia 1 1 1 1Argentina 1 1 1 1United Arab Emirates 1 1 1 1Chile 1 1 1 1Portugal 1 1 1 1Hungary 1 1 1 1Bahrain 1 1 1 1Latvia 1 1 1 1Croatia 1 2 1 1Kuwait 1 2 1 1Montenegro 1 1 1 1Belarus 1 2 1 1


1 3

Table 4 (continued) Country OFCM FCM OKM De Smet et al.’s

Russian Federation 2 2 2 1Oman 2 2 2 1Romania 2 1 2 1Uruguay 2 1 2 1Bahamas 2 1 2 1Kazakhstan 2 2 2 1Barbados 2 1 2 1Antigua and Barbuda 2 2 2 1Bulgaria 2 1 2 1Palau 2 2 2 1Panama 2 2 2 1Malaysia 2 2 2 1Mauritius 2 2 2 1Seychelles 2 2 2 1Trinidad and Tobago 2 2 2 1Serbia 2 2 2 1Cuba 2 2 2 1Lebanon 2 2 2 1Costa Rica 2 2 2 1Iran (Islamic Republic of) 2 2 2 1Venezuela 2 2 2 1Turkey 2 2 2 1Sri Lanka 2 2 2 1Mexico 2 2 2 1Brazil 2 2 2 1Georgia 2 2 2 1Saint Kitts and Nevis 2 2 2 1Azerbaijan 2 2 2 1Grenada 2 2 2 1Jordan 2 2 2 1The former Yugoslav Republic of Macedonia 2 2 2 2Ukraine 2 2 2 2Algeria 2 2 2 2Peru 2 2 2 2Albania 2 2 2 1Armenia 2 2 2 2Bosnia and Herzegovina 2 2 2 2Ecuador 2 2 2 2Saint Lucia 2 2 2 2China 2 2 2 2Fiji 2 2 2 2Mongolia 2 2 2 2Thailand 2 2 2 2Dominica 2 2 2 2Libya 2 2 2 2Tunisia 2 2 2 2Colombia 2 2 2 2Saint Vincent and the Grenadines 2 2 2 2Jamaica 2 2 2 2Tonga 2 2 2 2


1 3


Belize 2 2 2 2Dominican Republic 2 2 2 2Suriname 2 2 2 2Maldives 2 2 2 2Samoa 2 2 2 2Botswana 2 2 2 2Moldova (Republic of) 2 2 2 2Egypt 2 2 2 2Turkmenistan 2 2 2 2Gabon 2 2 2 2Indonesia 2 2 2 2Paraguay 2 2 2 2Palestine, State of 2 2 2 2Uzbekistan 2 2 2 2Philippines 2 2 2 2El Salvador 2 2 2 2South Africa 2 2 2 2Viet Nam 2 2 2 2Bolivia (Plurinational State of) 2 2 2 2Kyrgyzstan 2 2 2 2Iraq 2 2 2 2Cabo Verde 2 2 2 2Micronesia (Federated States of) 2 2 2 2Guyana 2 2 2 2Nicaragua 2 2 2 2Morocco 3 2 3 2Namibia 3 3 3 2Guatemala 3 3 3 2Tajikistan 3 3 3 2India 3 3 3 2Honduras 3 3 3 3Bhutan 3 3 3 2Timor-Leste 3 3 3 3Syrian Arab Republic 3 3 3 3Vanuatu 3 3 3 3Congo 3 3 3 3Kiribati 3 3 3 3Equatorial Guinea 3 3 3 3Zambia 3 3 3 3Ghana 3 3 3 3Lao People’s Democratic Republic 3 3 3 3Bangladesh 3 3 3 3Cambodia 3 3 3 3Sao Tome and Principe 3 3 3 3Kenya 3 3 3 3Nepal 3 3 3 3Pakistan 3 3 3 3Myanmar 3 3 3 3Angola 3 3 3 3Swaziland 3 3 3 3


1 3

References

1. Belacel N (2000) Multicriteria assignment method PROAFTN: methodology and medical application. Eur J Oper Res 125:175–183

2. Bezdek JC (1981) Pattern recognition with fuzzy objective func-tion algorithms. Plenum Press, New York pp 44–47

3. Bezdek JC, Ehrlich R, Full W (1984) FCM: the fuzzy c-means clustering algorithm. Comput Geosci 10:191–203

4. Boujelben MA (2016) A unicriterion analysis based on the PROMETHEE principles for multicriteria ordered clustering. Omega 69:126–140

5. Brans JP, Mareschal B (2004) PROMETHEE Methods. In Mul-tiple criteria decision analysis state of the art surveys. Springer, New York pp 163–186

6. Brans JP, Vincke P, Mareschal B (1986) How to select and how to rank projects: the Promethee method. Eur J Oper Res 24:228–238

7. Chen L, Xu Z, Wang H, Liu S (2016) An ordered clustering algo-rithm based on K-means and the PROMETHEE method. Int J Mach Learn Cybern 1:1–10

8. De Smet Y, Gilbart F (2001) A cluster definition method for coun-try risk problem. Tech Rep SMG IS-MG 13: 11–20

9. De Smet Y, Nemery P, Selvaraj R (2012) An exact algorithm for the multicriteria ordered clustering problem. Omega 40:861–869

10. Doumpos M, Zopounidis C (2004) A multicriteria classifica-tion approach based on pairwise comparisons. Eur J Oper Res 158:378–389

11. Dunn JC (2008) Well-Separated Clusters and Optimal Fuzzy Partitions. Journal of Cybernetics 4:95–104

12. Eppe S, Roland J, De Smet Y (2014) On the use of valued action profiles for relational multi-criteria clustering. Int J Multicri Decis Mak 4:201–205

13. Ishizaka A, Nemery P (2014) Assigning machines to incom-parable maintenance strategies with ELECTRE-SORT. Omega 47:45–59

14. Meyer P, Olteanu AL (2013) Formalizing and solving the prob-lem of clustering in MCDA. Eur J Oper Res 227:494–502

15. Michalowski W, Rubin S, Slowinski R, Wilk S (2001) Triage of the child with abdominal pain: a clinical algorithm for emer-gency patient management. Paediatrics Child Health 6:23–28

16. Militello C et al (2015) Gamma Knife treatment planning: MR brain tumor segmentation and volume measurement based on unsupervised Fuzzy c-Means clustering. Int J Imaging Syst Technol 25:213–225


Tanzania (United Republic of) 3 3 3 3Nigeria 3 3 3 3Cameroon 3 3 3 3Madagascar 3 4 3 3Zimbabwe 3 4 3 3Mauritania 3 3 3 3Solomon Islands 3 4 4 4Papua New Guinea 4 3 3 4Comoros 3 3 3 3Yemen 4 4 4 4Lesotho 4 4 4 4Togo 4 4 4 4Haiti 4 4 4 4Rwanda 4 4 4 4Uganda 4 4 4 4Benin 4 4 4 4Sudan 4 4 4 4Djibouti 4 4 4 4South Sudan 4 4 4 4Senegal 4 4 4 4Afghanistan 4 4 4 4Mali 4 4 4 4Malawi 4 4 4 4Ethiopia 4 4 4 4Gambia 4 4 4 4Congo (Democratic Republic of the) 4 4 4 4Liberia 4 4 4 4Central African Republic 4 4 4 4Sierra Leone 4 4 4 4


1 3

17. Mumpower JL, Livingston S, Lee TJ (1987) Expert judgments of political riskiness. J Forecast 6:51–65

18. Nemery P, Lamboray C (2008) FLOWSORT: a flow-based sorting method with limiting or central profiles. TOP 16:90–113

19. Org Z (1991) The determinants of country risk ratings. J Int Bus Stud 22:135–142

20. Pacheco J, Casado S, Angel-Bello F, ÁLvarez A (2013) Bi-objec-tive feature selection for discriminant analysis in two-class clas-sification. Knowl Based Syst 44:57–64

21. Shen L, Tay FEH, Qu L, Shen Y (2000) Fault diagnosis using rough sets. Theory Comput Ind 43:61–72

22. Siskos Y, Grigoroudis E, Zopounidis C, Saurais O (1998) Measur-ing customer satisfaction using a collective preference disaggrega-tion model. J Global Optim 12:175–195

23. Wang XZ, Wang YD, Wang LJ (2004) Improving fuzzy c -means clustering based on feature-weight learning. Pattern Recogn Lett 25:1123–1132

24. Xie XL, Beni G (1991) A validity measure for fuzzy clustering. IEEE Trans Pattern Anal Mach Intell 13:841–847

25. Xu Z, Wu J (2010) Intuitionistic fuzzy C-means clustering algo-rithm. J Syst Eng Electron 21:580–590

26. Yeung D, Wang XZ (2002) Improving performance of similarity-based clustering by feature weight learning. IEEE Trans Pattern Anal Mach Intell 24:556–561

27. Yu W (1992) ELECTRE TRI: aspects méthodologiques et manuel d’utilisation. PhD thesis, Université Paris-Dauphine :135–140

28. Zarinbal M, Fazel Zarandi MH, Turksen IB (2014) Interval Type-2 Relative Entropy Fuzzy C-Means clustering. Inf Sci 260:49–72

29. Zhang Z, Gao G, Tian Y (2015) Multi-kernel multi-criteria opti-mization classifier with fuzzification and penalty factors for pre-dicting biological activity. Knowl-Based Syst 89:301–313

30. Zheng Y, Byeungwoo J, Xu D, Wu QMJ, Hui Z (2015) Image seg-mentation by generalized hierarchical fuzzy C-means algorithm. J Intell Fuzzy Syst 28:4024–4028

31. Zhu H, Tsang E, Wang XZ et al (2017) Monotonic classification extreme learning machine. Neurocomputing 225:205–213

32. Zopounidis C, Doumpos M (1999) Business failure prediction using the UTADIS multicriteria analysis method. J Oper Res Soc 50:1138–1148

33. Zopounidis C, Doumpos M (2002) Multicriteria classification and sorting methods: a literature review. Eur J Oper Res 138:229–246

34. Zopounidis C, Doumpos M, Zanakis S (1999) Stock evalua-tion using a preference disaggregation methodology. Decis Sci 30:313–336

Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

an ordered clustering algorithm based on fuzzy c-means and...

Documents