Asynchronous Communication inJava over Infiniband and DECK

Rodrigo Righi, Philippe Navaux, Marcia Cera and Marcelo Pasin{rrrighi,navaux}@inf.ufrgs.br, {cera,pasin}@inf.ufsm.br

Universidade Federal de Santa Maria - BrazilUniversidade Federal do Rio Grande do Sul - Brazil

Contents

Part I: Introduction

Part II: Aldeia Communication System

Part III: Aldeia Experimental Evaluation

Part IV: Related Work

Part V: Final Considerations

2/20

Introduction

! Synchronous communication is a limiting factor for highperformance computing

! Asynchronous communication is an alternative to offerefficiency and flexibility

! Growing the usage of Java for parallel and distributedcomputing

! RMI and Sockets: synchronous communication overstandard networks (using TCP/IP)

3/20

Introduction

! Aldeia System! Asynchronous (not blocking) communications! Reimplements Java sockets for message passing! Clusters! Deployed to System Area Networks (SAN)! Nowadays Aldeia is implemented to work over:

! VAPI: Infiniband! DECK: Myrinet, SCI, VIA e TCP/IP

Aldeia’s main idea is to gather

! Java + asynchronous communication + SAN

4/20

Contents

Part I: Introduction

Part II: Aldeia Communication System

Part III: Aldeia Experimental Evaluation

Part IV: Related Work

Part V: Final Considerations

5/20

General Concepts of Aldeia

! Aldeia uses the same sockets syntaxes! Aldeia exchanges sockets semantics aiming to offer

asynchronous communication! Aldeia takes profit from asynchronous communication only

in sending operations! Portability: Already Java programs can be compiled in

order to use Aldeia

6/20

Aldeia’s Modular Structure

Aldeia Sockets

VAPI Adapter

uVAPI

DECKVAPI

LanguageJava

C Language

Application

Implemented

Modules

by Aldeia

! Aldeia Sockets acts as Aldeia’s Java interface! VAPI Adapter is a dynamic library which implements native

methods using VAPI interface for message passing! µVAPI is a library that reimplements some VAPI functions

needed by VAPI Adapter using DECK library7/20

Aldeia Classes’ Organization

Server

Client

getOutputStream()

getInputStream()

AldeiaOutputStream

AldeiaInputStream

AldeiaServerSocket

AldeiaSocket

AldeiaSocket

read()

write()

read()

getOutputStream()

getInputStream()

AldeiaOutputStream

AldeiaInputStream

write()

! Main methods: write(), flush(), read()! Message passing with variable length

! Sender carries out 100 bytes and receiver calls two timesthe read() method, each one to get 50 bytes

8/20

Aldeia Implementation Prototypes! Prototype 1

! Data segmentation occurs in Java! It is needed several calls to native method when data

exceeds a MTU! Prototype 2

! Data segmentation also occurs in Java! It is used a ByteBuffer object (nio Java packet)! It is needed several calls to native method when data

exceeds a MTU! Prototype 3

! Data segmentation is performed in C language! Single data transmission passing whole amount of data in

native method! Using C arithmetic pointer capability

9/20

Aldeia Implementation Prototypes! Prototype 1

! Data segmentation occurs in Java! It is needed several calls to native method when data

exceeds a MTU! Prototype 2

! Data segmentation also occurs in Java! It is used a ByteBuffer object (nio Java packet)! It is needed several calls to native method when data

exceeds a MTU! Prototype 3

! Data segmentation is performed in C language! Single data transmission passing whole amount of data in

native method! Using C arithmetic pointer capability

9/20

Aldeia Implementation Prototypes! Prototype 1

! Data segmentation occurs in Java! It is needed several calls to native method when data

exceeds a MTU! Prototype 2

! Data segmentation also occurs in Java! It is used a ByteBuffer object (nio Java packet)! It is needed several calls to native method when data

exceeds a MTU! Prototype 3

! Data segmentation is performed in C language! Single data transmission passing whole amount of data in

native method! Using C arithmetic pointer capability

9/20

Contents

Part I: Introduction

Part II: Aldeia Communication System

Part III: Aldeia Experimental Evaluation

Part IV: Related Work

Part V: Final Considerations

10/20

Aldeia’s Experimental Evaluation! It was built a simple application to measure

latency,communication time and asynchronous capability! Tests were made over DECK/TCP and VAPI

! Infiniband: Mellanox board 2 Gbps and Switch of 10 Gbps! Ethernet: 3Com 1 Gpbs and switch of 100 Mbps

! Application scheme

client: server:Get initial timerepeat i times: repeat i times:

out.write(d) in.read(d)Get asynchronous timeout.flush()Get synchronous time

11/20

Aldeia’s Evaluation Using DECK/TCP

0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000Message size in bytes

050

100150200250300350400450500550600650700750800850900950

1000

Tim

e in

mic

rose

cond

s

Synchronous communication

Asynchronous communication

Asynchronous communication using Aldeia over DECK/TCP

12/20

Aldeia’s Evaluation Using DECK/TCP

0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000Message size in bytes

050

100150200250300350400450500550600650700750800850900950

1000

Tim

e in

mic

rose

cond

s

Synchronous communication

Asynchronous communication

Asynchronous communication using Aldeia over DECK/TCP

Latency = 60 us

Bandwidth = 70 Mbps

13/20

Aldeia’s Evaluation Using Infiniband/VAPI

0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000Message size in bytes

0

25

50

75

100

125

150

175

200

Tim

e in

mic

rose

cond

s

Synchronous communication

Asynchronous communication

Asynchronous communication using Aldeia and VAPI

14/20

Aldeia’s Evaluation Using Infiniband/VAPI

0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000Message size in bytes

0

25

50

75

100

125

150

175

200

Tim

e in

mic

rose

cond

s

Synchronous communication

Asynchronous communication

Asynchronous communication using Aldeia and VAPI

Latency = 29 us

Bandwidth = 380 Mbps

15/20

Contents

Part I: Introduction

Part II: Aldeia Communication System

Part III: Aldeia Experimental Evaluation

Part IV: Related Work

Part V: Final Considerations

16/20

Related Work

! Java sockets reimplementation to work with SCI networks! There are not asynchronous Java sockets reimplementation! There are some asynchronous RMI systems

! Ibis! ProActive! JavaSymphony

! Future work: It is possible to integrate Aldeia in Ibis aimingto offer RMI over Infiniband

17/20

Contents

Part I: Introduction

Part II: Aldeia Communication System

Part III: Aldeia Experimental Evaluation

Part IV: Related Work

Part V: Final Considerations

18/20

Final Considerations! Aldeia achieved the asynchronous communication

advantages! Aldeia offers a sockets interface which can be integrated in

current Java sockets-based systems

DECK/TCP performance issues

! Aldeia obtains 60 µs for latency and 70 Mbps for bandwidth

Infiniband performance issues

! Aldeia obtained 30 µs for latency! The VAPI Notification functions overhead reduces network

bandwidth

19/20

Final Considerations! Aldeia achieved the asynchronous communication

advantages! Aldeia offers a sockets interface which can be integrated in

current Java sockets-based systems

DECK/TCP performance issues

! Aldeia obtains 60 µs for latency and 70 Mbps for bandwidth

Infiniband performance issues

! Aldeia obtained 30 µs for latency! The VAPI Notification functions overhead reduces network

bandwidth

19/20

Asynchronous Communication inJava over Infiniband and DECK

Rodrigo Righi, Philippe Navaux, Marcia Cera and Marcelo Pasin{rrrighi,navaux}@inf.ufrgs.br, {cera,pasin}@inf.ufsm.br

Universidade Federal de Santa Maria - BrazilUniversidade Federal do Rio Grande do Sul - Brazil