the f-mpj challenge: solving complex problems on ...€¦ · inﬁniband network 32 gbps (mellanox...

IntroductionMessage Passing in Java with F-MPJ

Complex Application in Bioinformatic: ProtTestPerformance Evaluation

Conclusions

The F-MPJ Challenge: Solving ComplexProblems on Hierachical Architectures with

Java

Sabela Ramos GareaRoberto Rey Expósito

Group of Computer ArchitectureUniversity of A Coruña

[email protected], [email protected]

ComplexHPC Challenge 2011Sabela Ramos Garea, Roberto Rey Expósito The F-MPJ Challenge



Conclusions

Outline

1 Introduction

2 Message Passing in Java with F-MPJ

3 Complex Application in Bioinformatic: ProtTest

4 Performance Evaluation

5 Conclusions

Sabela Ramos Garea, Roberto Rey Expósito The F-MPJ Challenge



Conclusions

Java for HPC

1 IntroductionJava for HPC

2 Message Passing in Java with F-MPJMessage Passing in JavaF-MPJ: Fast MPJ

3 Complex Application in Bioinformatic: ProtTestProtTestParallel Strategies

4 Performance EvaluationExperimental ConfigurationF-MPJ PerformanceProtTest 3 Performance

5 Conclusions




Conclusions

Java for HPC

Java for High Performance Computing (HPC)

Features:

Network communication support.

Multithreading support.

Portable, platform independent.

Object Oriented.

Safe, robust, simple and witheasy maintenance.

Similar performance as nativelanguages (C, Fortran).

Parallel/distributed programming inJava:

Concurrency Framework.

Java Sockets.

Java RMI.

Message-Passing in Java (MPJ).




Conclusions

Message Passing in JavaF-MPJ: Fast MPJ





5 Conclusions




Conclusions


Message Passing in Java

Message-passing is the main HPC programming model.

Implementation approaches

RMI.

Wrapping a native library via JNI.(e.g., MPI libraries: OpenMPI, MPICH).

Sockets.

APIs implemented:

PVM-like.

mpiJava.

MPJ.

Others.




Conclusions


Pur

eJa

vaIm

pl. Socket

impl.High-speednetworksupport

API

Java

IO

Java

NIO

Myr

inet

Infin

iBan

d

SC

I

mpi

Java

1.2

JGF

MP

J

Oth

erA

PIs

MPJava X X X

Jcluster X X X

Parallel Java X X X

mpiJava X X X X

P2P-MPI X X X X

MPJ Express X X X X

MPJ/Ibis X X X X

JMPI X X X

F-MPJ X X X X X X X




Conclusions






5 Conclusions




Conclusions


F-MPJ Communication Devices

JVM

native comms

device layer smpdev

Java Threads

Shared Memory

MPJ Applications

ibvdev

TCP/IP

JNI

IBV

omxdev

Open−MX

InfiniBand EthernetMyrinet/Ethernet

Java Sockets

niodev/iodev

F−MPJ Library




Conclusions


F-MPJ Communication Devices for Heterogeneity

Different sorts of devices:

Distributed memory.

Native communication layers: ibvdev, omxdev.Java sockets: iodev, niodev.

Shared memory.

Java threads: smpdev.

Hybrid shared-distributed memory.

In development. It combines a distributed memory devicewith smpdev.




Conclusions


Optimizing performance:

No buffering layer for primitive types.

Multi-core aware collective operations library.

Configurable algorithms depending on the message sizeand the number of processors.




Conclusions


Multi-core aware algorithms for collective operations:

Operation Algorithms

Barrier BT, Gather+Bcast, BTe, Gather+Bcast Optimized

Bcast MST, NBFT, BFT

Scatter/v MST, NBFT

Gather/v MST, NBFT, NB1FT, BFT

Allgather/v NBFT, NBBDE, BBKT, NBBKT, BTe, Gather + Bcast

Alltoall/v NBFT, NB1FT, NB2FT, BFT

Reduce MST, NBFT, BFT

Allreduce NBFT, BBDE, NBBDE, BTe, Reduce + Bcast

Reduce-scatter BBDE, NBBDE, BBKT, NBBKT, Reduce + Scatter

Scan NBFT, OneToOne




Conclusions

ProtTestParallel Strategies





5 Conclusions




Conclusions


ProtTest

One of the most popular tools for selectingmodels of protein evolution.

Almost 4,000 registered users.Over 700 citations.

Written in Java.

Intensive in computational needs.

ProtTest 3 designed to take advantage ofparallel processing.




Conclusions






5 Conclusions




Conclusions


Shared Memory Implementation

Java concurrence API

Implementation of a thread pool.

Dynamic task distribution over the pool.




Conclusions


Distributed Memory Implementation

Message Passing in Java

Allow both distributions (static and dynamic).

Includes a distributor process with a negligible workload.




Conclusions


Hybrid Shared/Distributed Memory Implementation

MPJ + OpenMP

Scalability is limited by the task-based high level parallelization.

Solution:

Two-level parallelism.Combination of message passing with multithread computation oflikelihood.Implementation of a parallel version of PhyML using OpenMP.




Conclusions

Experimental ConfigurationF-MPJ PerformanceProtTest 3 Performance





5 Conclusions




Conclusions


Experimental Configuration:

Pluton: Departmental cluster (16 nodes)

2xIntel Xeon 5520 Quad-core CPU (8 cores with hyper-threading per node)

8 GB RAM

InfiniBand Network 16 Gbps (QLogic QLE7240 DDR)

Linux, Sun JDK 1.6, F-MPJ, MPJ Express, OpenMPI, MVAPICH

DAS-4 VU cluster (74 nodes)

2xIntel Xeon 5620 Quad-core CPU (8 cores with hyper-threading per node)

24 GB RAM

InfiniBand Network 32 Gbps (Mellanox MT26428 QDR)

Linux, OpenJDK 1.6, F-MPJ, MPJ Express, IntelMPI

Special shared memory node (node075):4xAMD Opteron 6172 12-core (48 cores) and 128 GB RAM




Conclusions






5 Conclusions




Conclusions


Point-to-Point Performance

Message size (bytes)

Point-to-point Performance on InfiniBand (Pluton)

0

10

20

30

40

50

60

70

80

4 16 64 256 1K

Late

ncy

(µs

)

1K 4K 16K 64K 256K 1M 2M 4M 0

1

2

3

4

5

6

7

8

9

10

11

Ban

dwid

th (

Gbp

s)

F-MPJ (ibvdev) - IBV MPJE (niodev) - IPoIB MVAPICH v1.2.0- IBV OpenMPI v1.3.3 - IBV




Conclusions




Point-to-point Performance on InfiniBand (DAS-4)

0

5

10

15

20

25

30

35

40

45

50

55

60

65

4 16 64 256 1K

Late

ncy

(µs

)

1K 4K 16K 64K 256K 1M 2M 4M 0

2

4

6

8

10

12

14

16

18

20

22

24

26

28

Ban

dwid

th (

Gbp

s)

F-MPJ (ibvdev) - IBV MPJE (niodev) - IPoIB IntelMPI 4 - IBV




Conclusions




Point-to-point Performance on Shared Memory (Pluton)

0

2

4

6

8

10

12

14

16

18

20

22

24

26

4 16 64 256 1K

Late

ncy

(µs

)

1K 4K 16K 64K 256K 1M 2M 4M 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80

Ban

dwid

th (

Gbp

s)

F-MPJ (ibvdev) F-MPJ (smpdev) MPJE (smpdev) MVAPICH v1.2.0 OpenMPI v1.3.3




Conclusions




Point-to-point Performance on Shared Memory (DAS-4)

0

5

10

15

20

25

30

35

40

45

4 16 64 256 1K

Late

ncy

(µs

)

1K 4K 16K 64K 256K 1M 2M 4M 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30

Ban

dwid

th (

Gbp

s)

F-MPJ (ibvdev) F-MPJ (smpdev) MPJE (smpdev) IntelMPI 4




Conclusions


Collective Operations Performance

0

20

40

60

80

100

120

140

160

180

200

220

240

260

1K 2K 4K 8K 16K 32K 64K 128K 256K 512K 1M 2M 4M

Agg

rega

ted

Ban

dwid

th (

Gbp

s)


Broadcast Performance (128 Processes)

F−MPJ (ibvdev) − IBVMPJE (niodev) − IPoIBMVAPICH − IBVOpenMPI − IBV




Conclusions



0

200

400

600

800

1000

1200

1400

1600

1800

1K 2K 4K 8K 16K 32K 64K 128K 256K 512K 1M 2M 4M

Agg

rega

ted

Ban

dwid

th (

Gbp

s)


Broadcast Performance on DAS−4 (512 Processes)

F−MPJ (ibvdev) − IBV Intel MPI 4 − IBV




Conclusions



0

20

40

60

80

100

120

140

160

180

200

1K 2K 4K 8K 16K 32K 64K 128K 256K 512K 1M 2M 4M

Agg

rega

ted

Ban

dwid

th (

Gbp

s)


Broadcast Performance (8 Threads)

F−MPJ (smpdev) MPJE (smpdev) MVAPICHOpenMPI




Conclusions



0

10

20

30

40

50

60

70

80

1K 2K 4K 8K 16K 32K 64K 128K 256K 512K 1M 2M 4M

Agg

rega

ted

Ban

dwid

th (

Gbp

s)


Broadcast Performance on DAS−4 (48 Threads)

F−MPJ (smpdev) MPJE (smpdev) IntelMPI 4




Conclusions






5 Conclusions




Conclusions


ProtTest 3 Performance

0

5

10

15

20

25

30

35

40

45

50

1 2 4 8 16 32 64 128

Spe

edup

Number of Processes

ProtTest 3 Distributed Memory Scalability on Pluton

F−MPJ (ibvdev)F−MPJ (ibvdev) + OpenMP




Conclusions



0

4

8

12

16

20

24

28

32

36

40

1 2 4 8 16 32 64 128

Spe

edup

Number of Processes

ProtTest 3 Distributed Memory Scalability on DAS−4

F−MPJ (ibvdev) − IBV F−MPJ (ibvdev) + OpenMP − IBV




Conclusions



0

1

2

3

4

5

6

7

8

9

10

11

1 2 4 8 16

Spe

edup

Number of Threads

ProtTest 3 Shared Memory Scalability on Pluton

F−MPJ (smpdev)ProtTest 3 (threads)




Conclusions



0

2

4

6

8

10

12

14

16

18

20

1 2 4 8 16 32 48

Spe

edup

Number of Threads

ProtTest 3 Shared Memory Scalability on DAS−4

F−MPJ (smpdev)ProtTest 3 (threads)F−MPJ (smpdev) + OpenMP




Conclusions

SummaryQuestions





5 Conclusions




Conclusions

SummaryQuestions

Summary

This work presents our current research efforts for HPC inJava with F-MPJ.

F-MPJ has been applied to a real complex problem withlarge scale needs for computational resources.

Taking advantage of hierarchical architectures: sharedmemory, distributed memory, hybrid shared/distributedmemory.

Other applications that benefit from the use of F-MPJ: ESAGaia project, jGadget, financial applications, petro-seismicJavaSeis, ...




Conclusions

SummaryQuestions

THE F-MPJ CHALLENGE: SOLVING COMPLEX PROBLEMS ON HIERACHICAL

ARCHITECTURES WITH JAVA

Sabela Ramos GareaRoberto Rey Expósito

University of A Coruña


the f-mpj challenge: solving complex problems on ...€¦ · inﬁniband network 32 gbps (mellanox...

Documents