multi-objective evolutionary clustering : a survey

MultiobjectiveEvolutionaryClustering

Aiswarya Issac

Clustering

Multi-objectiveclustering

OptimizationProblems

MultiobjectiveClustering

EvolutionaryAlgorithms


Evolutionary Method

SolutionRepresentation

Initializing Population

Selection of Objectivefunctions

Operations

Final SolutionSelection

Applications

Conclusion

Multiobjective Evolutionary Clustering

Aiswarya Issac

27 January, 2016

Aiswarya Issac Multiobjective Evolutionary Clustering 27 January, 2016 1 / 37


Aiswarya Issac

Clustering






Evolutionary Method




Operations


Applications

Conclusion

Overview

Clustering

Multi-objective clusteringOptimization ProblemsMultiobjective Clustering

Evolutionary Algorithms

Multiobjective Evolutionary ClusteringEvolutionary MethodSolution RepresentationInitializing PopulationSelection of Objective functionsOperationsFinal Solution SelectionApplications



Aiswarya Issac

Clustering






Evolutionary Method




Operations


Applications

Conclusion

Clustering

I Partitioning into homogeneous groups based on somesimilarity metric.

I Click Here


random.gif


Aiswarya Issac

Clustering






Evolutionary Method




Operations


Applications

Conclusion

Outline

Clustering






Aiswarya Issac

Clustering






Evolutionary Method




Operations


Applications

Conclusion

Optimization Problem

I Single Objective OptimizationI Only one objective function to be minimized.I Eg. Knapsack problem

I Multiple Objective OptimizationI Two or more conflicting objectives need to be

optimized.I There will be a set of possible solutions rather than a

single optimal solution - Pareto optimal solutions.I Eg. Minimizing cost while maximizing comfort while

buying a car.



Aiswarya Issac

Clustering






Evolutionary Method




Operations


Applications

Conclusion

Optimization Problem

Figure: 1 Illustration of knapsack problem[Source:wikipedia]



Aiswarya Issac

Clustering






Evolutionary Method




Operations


Applications

Conclusion

Outline

Clustering






Aiswarya Issac

Clustering






Evolutionary Method




Operations


Applications

Conclusion

Multiobjective clustering

I Single Objective Clustering

Figure: 2 Comparison of different clustering[1]

I The final clusters do not represent a global optimizationresult.

I Different final clustering can happen based on the initialselection of cluster center.



Aiswarya Issac

Clustering






Evolutionary Method




Operations


Applications

Conclusion

Multiobjective Clustering

I Multi Objective Clustering

I Decompose a data-set into similar groups maximizingmultiple objectives in parallel.

Figure: 3 Output for multiobjective clustering[1]



Aiswarya Issac

Clustering






Evolutionary Method




Operations


Applications

Conclusion

Evolutionary AlgorithmsProcess of Evolution

Figure: 4 Schematic representation of evolutionary algorithm[2]



Aiswarya Issac

Clustering






Evolutionary Method




Operations


Applications

Conclusion

Evolutionary AlgorithmsProcess of Evolution

Figure: 5 Schematic representation datastructures[2]



Aiswarya Issac

Clustering






Evolutionary Method




Operations


Applications

Conclusion

Outline

Clustering






Aiswarya Issac

Clustering






Evolutionary Method




Operations


Applications

Conclusion

Evolutionary AlgorithmsWhy Evolutionary Method? [3]

I Antennas developed by NASA’s Evolvable SystemsGroup for use on satellites.

I Free of any human preconceptions or biases



Aiswarya Issac

Clustering






Evolutionary Method




Operations


Applications

Conclusion

Evolutionary AlgorithmsMultiobjective Clustering Steps[3]

I Choose a possible encoding of chromosome to representa clustering solution.

I Generate the initial population of chromosomes.

I Choose a suitable set of objective functions that are tobe optimized simultaneously.

I Design suitable evolutionary operators such as selection,crossover, and mutation.

I Define a fitness function to evaluate the clusteringsolutions.

I Develop a technique to obtain a single clusteringsolution from Pareto optimal set.



Aiswarya Issac

Clustering






Evolutionary Method




Operations


Applications

Conclusion

Outline

Clustering






Aiswarya Issac

Clustering






Evolutionary Method




Operations


Applications

Conclusion

Solution Representation

Figure: 6 Classification of solution representation[4]



Aiswarya Issac

Clustering






Evolutionary Method




Operations


Applications

Conclusion

Solution RepresentationPrototype based

I Centroid-based:I The coordinates of the cluster centers will be encoded

in the chromosome.I Eg: [2.4 5.9, 0.36 2.7, 5.3 10.2]

I Medoid-based:I Used when coordinates are not known.I An actual point that is most centrally located in a

cluster is used to represent that cluster.I Eg. datapoints = [1,2,3,4,5,6,7,8]

encoding = [3,7]

I Mode-based:I Similar to medoid basedI Computation is less expensive when compared with

medoid based.



Aiswarya Issac

Clustering






Evolutionary Method




Operations


Applications

Conclusion

Solution RepresentationPoint based

I Cluster Label-based:

Figure: 7 Cluster Label based encoding scheme[4]

I Locus-based Adjacency Graph:

Figure: 8 Locus based encoding scheme[4]



Aiswarya Issac

Clustering






Evolutionary Method




Operations


Applications

Conclusion

Outline

Clustering






Aiswarya Issac

Clustering






Evolutionary Method




Operations


Applications

Conclusion


I Prototype-based encodingI The prototypes in the initial population are usually

some randomly selected data points.

I Point based encodingI The cluster labels will be initialized with random strings

so that each point gets a random cluster label.



Aiswarya Issac

Clustering






Evolutionary Method




Operations


Applications

Conclusion

Outline

Clustering






Aiswarya Issac

Clustering






Evolutionary Method




Operations


Applications

Conclusion

Objective functions

I Overall Cluster Deviation

I Cluster Connectedness



Aiswarya Issac

Clustering






Evolutionary Method




Operations


Applications

Conclusion

Outline

Clustering






Aiswarya Issac

Clustering






Evolutionary Method




Operations


Applications

Conclusion

OperationsSelection[5]

Selection is based on fitness function.

I Tournament selection.

I Ranking

I Proportionate Selection

Figure: 9 Roulette wheel selection[2]



Aiswarya Issac

Clustering






Evolutionary Method




Operations


Applications

Conclusion

OperationsCrossover

Figure: 10 Classification of crossover schemes[4]



Aiswarya Issac

Clustering






Evolutionary Method




Operations


Applications

Conclusion

OperationsCrossover

I Single or multiple point crossover

I For prototype based:Parent1: [2.4 5.9, 0.36 2.7, 5.3 10.2]Parent2: [2.5 5.5, 1.2 2.3, 6.0 10.2]Offspring1: [2.4 5.9, 1.2 2.3, 6.0 10.2]Offspring2: [2.5 5.5, 0.36 2.7, 5.3 10.2]

I For point based:Uniform crossover approachParent1: [1 1 1 2 3 3 2 3 ]Parent2: [1 1 2 2 2 3 3 2 ]Mask : [0 0 1 1 1 1 0 1 ]Offspring1: [1 1 2 2 2 3 2 2 ]Offspring2: [1 1 1 2 3 3 3 3 ]



Aiswarya Issac

Clustering






Evolutionary Method




Operations


Applications

Conclusion

OperationsMutation

Figure: 11 Classification of mutation schemes[4]



Aiswarya Issac

Clustering






Evolutionary Method




Operations


Applications

Conclusion

OperationsMutation

I For prototype based encoding:The cluster center is modified as follows:

z ′k l = (1 ± 2ε)zk l

I For point based encoding:A point is chosen with probability 1/n.Its cluster label is randomly mutated, along withpredefined number of neighbours.



Aiswarya Issac

Clustering






Evolutionary Method




Operations


Applications

Conclusion

OperationsMutation

I For prototype based encoding:The cluster center is modified as follows:

z ′k l = (1 ± 2ε)zk lI For point based encoding:

A point is chosen with probability 1/n.Its cluster label is randomly mutated, along withpredefined number of neighbours.



Aiswarya Issac

Clustering






Evolutionary Method




Operations


Applications

Conclusion

Outline

Clustering






Aiswarya Issac

Clustering






Evolutionary Method




Operations


Applications

Conclusion

Final Solution Selection

Figure: 12 Classification of approaches for final solutions[4]



Aiswarya Issac

Clustering






Evolutionary Method




Operations


Applications

Conclusion

Final Solution Selection

I Independent Objective-based:Objective functions are used to evaluate the best

solution.

I Knee-based:Knee solution is one for which change in one

objective value induces maximum change in others.

I Cluster Ensemble-based:If some points are always clustered together by a

majority of the solutions, then these points may beassumed to be clustered appropriately.

So, this can be used to train a supervised classifier.



Aiswarya Issac

Clustering






Evolutionary Method




Operations


Applications

Conclusion

Outline

Clustering






Aiswarya Issac

Clustering






Evolutionary Method




Operations


Applications

Conclusion

Applications

Applications Tasks

BioinformaticsGrouping co-expressed genes

Clustering samplesProtein complex identification

Social Network Analytics Social network clustering



Aiswarya Issac

Clustering






Evolutionary Method




Operations


Applications

Conclusion

Different Algorithms

Figure: Caption



Aiswarya Issac

Clustering






Evolutionary Method




Operations


Applications

Conclusion

Conclusion

I Evolutionary Algorithms can be used to obtain solutionsfor unconventional problems like multiobjectiveclustering.

I Suggestions:I Chromosome Encoding: Fast decoding and small length.I Initialization: Pre-processing of input.I Final solution selection: Use multiple objectives

together.



Aiswarya Issac

Appendix

For Further Reading

Reference I

[1] Martin H. C. Law, Alexander P. Topchy, Anil K.Jain, ’Multiobjective Data Clustering’, IEEE ComputerSociety Conference on Computer Vision and PatternRecognition, 2004.

[2] Carlos A , Gary B and David A, Chapter 1 and 2, in’Evolutionary Algorithms for solving Multiobjectiveproblem’., 2nd ed, Springer, 2007.

[3] Daniel W. Dyer, ’The power of evolution’, in’Evolutionary Computation in Java’,’http://watchmaker.uncommons.org/manual/index.html’,2010



Aiswarya Issac

Appendix

For Further Reading

Reference II

[4] Anirban Mukhopadhyay, Ujjwal Maulik, andSanghamitra Bandyopadhyay. 2015. ’A survey ofmultiobjective evolutionary clustering’. ACM Comput.Surv. 47, 4, Article 61 (May 2015).

[5] Abdullah Konak, David W. Coit, Alice E. Smith,Multi-objective optimization using genetic algorithms: Atutorial’. Reliability Engineering and SystemSafety,Elsevier,2006


multi-objective evolutionary clustering : a survey

Technology