high-performance asset cluster analysis implemented with parallel genetic algorithms on the nvidia...

MotivationBackground

ApproachResults

References

High-performance asset cluster analysisimplemented with Parallel Genetic Algorithms on

the NVIDIA CUDA platform

by Dariusz Cieslakiewicz1, Diane Wilcox1, Tim Gebbie2

1Department of Computational and Applied MathematicsUniversity of Witwatersrand

2Investec Bank

CHPC Conference, Durban, December 2012

The financial assistance of the National Research Foundation (NRF) towards this research is hereby acknowledged.Opinions expressed and conclusions arrived at, are those of the author and are not necessarily attributed to the NRF.

by Dariusz Cieslakiewicz, Diane Wilcox, Tim Gebbie High-performance asset cluster analysis implemented with Parallel Genetic Algorithms on the NVIDIA CUDA platform


ApproachResults

References

Outline

1 MotivationNon-parametric cluster analysis

2 BackgroundGenetic AlgorithmsGenetic operatorsParallel Genetic AlgorithmsComputing Platform

3 ApproachTechnical architecture

4 ResultsResults

5 References



ApproachResults

References

Non-parametric cluster analysis

Outline




4 ResultsResults

5 References



ApproachResults

References


Cluster analysis



ApproachResults

References


Cluster analysis I

Cluster analysis groups objects and describes theirassociations.Configuration of clusters is represented by a setS = {C1, . . . ,Ci ,Ci+1, . . . ,Cn} where Ci is the cluster thatobject i belongs to and adheres to the following conditions:

Ci 6= ∅ for i = 1, . . . ,K ,Ci ∩ Cj = ∅ for i = 1, . . . ,K , j = 1, . . . ,K and i 6= j

and⋃K

i=1 Ci = S[Giada and Marsili, 2002] addressed the problem of dataclustering by introducing an unsupervised, parameter freeapproach based on the maximum likelihood principle (ML)principle.



ApproachResults

References


Log-likelihood function I

Log-likelihood function

[Giada and Marsili, 2002] computed the probability density P({~xi}|G,S) and used this to derive themaximum likelihood of a feature Lc thus resulting in a structure S as P(G∗,S|~xi ) ∝ eDL(S), wherethe resulting likelihood function is denoted by

Lc(S) =12

∑s:ns>1

[logns

cs+ (ns − 1) log

n2s − ns

n2s − cs

] (1)

The likelihood only depends on the Pearson’s coefficient of data:

ci,j =~xi~xj√∥∥∥~xi2∥∥∥ ∥∥∥~xj

2∥∥∥ (2)



ApproachResults

References

Genetic AlgorithmsGenetic operatorsParallel Genetic AlgorithmsComputing Platform

Outline




4 ResultsResults

5 References



ApproachResults

References


What are Genetic Algorithms? I

In 1975, Holland, in his book titled Adaptation in naturaland artificial systems, described how to apply theprinciples of natural evolution to optimization problems andbuilt the first Genetic Algorithms.

Genetic algorithms are computationally intensive globalsearch heuristics.

Search space contains all feasiable solutions among whichthe desired solution resides.



ApproachResults

References


Outline




4 ResultsResults

5 References



ApproachResults

References


Genetic operators I

SelectionCrossoverMutationElitismReplacement



ApproachResults

References


Genetic operators



ApproachResults

References


Cluster determination with GA

Representation: Integer-based individualCh =

{Ci1 ,Cj2 , . . . ,ClK

}for i , j , l = {1, . . . ,K } where Ci is

the cluster that the object at the ordinal position belongs to.Objective function:

Lc(S) =12

∑s:ns>1

[logns

cs+ (ns − 1) log

n2s − ns

n2s − cs

] (3)



ApproachResults

References


Outline




4 ResultsResults

5 References



ApproachResults

References


Parallelisation Schemes

GAs lend themselves to parallelisation.Parallelisation Schemes: Master-slave and Multiple-deme

Master-slave models

Multiple-deme PGAs



ApproachResults

References


Migration Topologies

MigrationDeme SizeMigration Topologies: Stepping Stone, One-Way Ring,Ring, Ring+1+2 and Ring+1+2+3

Migration rate



ApproachResults

References


Outline




4 ResultsResults

5 References



ApproachResults

References


NVIDIA CUDA platform

1 Compute Unified Device Architecture(CUDA) is NVIDIA’splatform for massively parallel high-performancecomputing on the NVIDIA GPUs.

2 Utilising Nsight Eclipse Edition 5 with CUDA 5 on LinuxMint 13 using two cards: GTX 560 Ti and GTX 660 Ti



ApproachResults

References

Technical architecture

Outline




4 ResultsResults

5 References



ApproachResults

References


Physical View: Master-slave GA on the the NVIDIACUDA platform



ApproachResults

References


Physical View: Multiple-deme GA on the the NVIDIACUDA platform [Pospichal et al., 2010]



ApproachResults

References


Physical View: Migration of individuals betweenislands/demes [Pospichal et al., 2010]



ApproachResults

References


Genetic Operators

Crossover Operator

Single-Point, Two-Point and Custom Crossover implemented and executed at Pc of 0.6 and0.8-0.9.

Custom Crossover utilises a Single-Point Crossover approach with a knowledge directedoperator applying the following measure

f = arg max‖∏→1‖

[12∑

s:ns>1[log nscs

+ (ns − 1) log n2s−ns

n2s−cs

]

]

Mutation Operator

Mutation implemented with Pm at 1genomelength .

Replacement with randomly generated value.



ApproachResults

References


Migration Topologies

MigrationDeme Size : VariableMigration Topologies: Stepping Stone, One-Way Ring andRingMigration rate:

Variable Default: 2



ApproachResults

References


Implementation: Multiple-deme PGA I

1 Generic Framework for PGAs comprising 50 code artefactsto research different aspects of PGAs on GPUs.

2 Includes templatised C++ artefacts: Processing Context,Chromsome, Individual, SubPopulation, Population,Archipalego, etc.



ApproachResults

References


Implementation: Multiple-deme PGA II



ApproachResults

References

Results

Outline




4 ResultsResults

5 References



ApproachResults

References

Results

Master-Slave model



ApproachResults

References

Results

Conclusion

Successfully implemented PGAs on the GPU.Speedups of more than 20x are attainable.Marsilli-Giada Log-likelihood function is a viable approachfor isolating residual clusters.



ApproachResults

References

References I

Chen, D., Chen, W., and Zheng, W. (2012).Cuda-zero: a framework for porting shared memory gpuapplications to multi-gpus.SCIENCE CHINA Information Sciences, 55(3):663–676.

Coley, D. (1999).An Introduction to Genetic Algorithms for Scientists andEngineers.World Scientific.

Giada, L. and Marsili, M. (2001).Data clustering and noise undressing of correlationmatrices.Phys. Rev. E, 63:061101.



ApproachResults

References

References II

Giada, L. and Marsili, M. (2002).Algorithms of maximum likelihood data clustering withapplications.Physica A: Statistical Mechanics and its Applications,315(3â4):650 – 664.

Hoberock, J. and Bell, N. (2010).Thrust: A parallel template library.Version 1.3.0.



ApproachResults

References

References III

Langdon, W. B. (2011).Debugging cuda.In Proceedings of the 13th annual conference companionon Genetic and evolutionary computation, GECCO ’11,pages 415–422, New York, NY, USA. ACM.

Noh, J. (2000).Phys. Rev. E, 61.

NVIDIA (2012a).CUDA Toolkit 4.1 CURAND Guide.NVIDIA Corporation.



ApproachResults

References

References IV

NVIDIA (2012b).NVIDIA CUDA C Programming Guide.NVIDIA Corporation.

Pospichal, P., Jaros, J., and Schwarz, J. (2010).Parallel genetic algorithm on the cuda architecture.In Proceedings of the 2010 international conference onApplications of Evolutionary Computation - Volume Part I,EvoApplicatons’10, pages 442–451, Berlin, Heidelberg.Springer-Verlag.



ApproachResults

References

References V

Robilliard, D., Marion-Poty, V., and Fonlupt, C. (2009).Genetic programming on graphics processing units.Genetic Programming and Evolvable Machines,10(4):447–471.

Sivanandam, S. and Deepa, S. (2010).Introduction to Genetic Algorithms.Springer.

Xiao, S. and chun Feng, W.Inter-block gpu communication via fast barriersynchronization.



ApproachResults

References

References VI

Zhang, S. and He, Z. (2009).Implementation of parallel genetic algorithm based oncuda.In Proceedings of the 4th International Symposium onAdvances in Computation and Intelligence, ISICA ’09,pages 24–30, Berlin, Heidelberg. Springer-Verlag.


high-performance asset cluster analysis implemented with parallel genetic algorithms on the nvidia...

Documents