An Implementation ofVirtual Cluster on a Cloud

Pongsakorn U-chupalaKasetsart University

#PRAGMA24

Agenda

• Introduction• Design & Implementation• Benchmarking Results

Cluster

User

USE

VMVM

Storage Farm

VMVM

GPU Accelerated Nodes

VMVMVMVM

Compute Nodes

VMFrontend

*Example setup

High-speed Network

Virtual

Managing virtual clusterManuallyis a

Headache

http://openphoto.net/gallery/image.html?image_id=24436

Virtual Cluster Cloud

User

Virtual Cluster

USE

Virtual Cluster Template

Virtual Cluster Cloud

*Not limited to Rocks Cluster

Multi-site Infrastructure

Thailand

USA

Japan

*Countries listed are just for examples

Overlay Network

Design Goals

• Automatic resources management• Designed with scalability in mind• Acceptable performance• Virtual Cluster management as a single entity

– Create– Deploy– Suspend

– Stop– Delete

Agenda

• Introduction• Design & Implementation• Benchmarking Results

Architecture

Multi-tier VPN

Agenda

• Introduction• Design & Implementation• Benchmarking Results

Cloud Testbed

Network Benchmarking

• Average Bandwidth: measured with Iperf– Physical Network: 776.62 Mbits/s– Overlay Network: 314.26 Mbits/s

• Round-Trip-Time: measured with ping– Physical Network: 0.135 ms– Overlay Network: 0.455 ms

Virtual Cluster Testset

HPCC Benchmarking Results

DGEMM (CPU Intensive)

Single-host Single-site Single-site with Physical Network

Multi-site0123456789

10

4.197254

9.045199 9.100802 9.303938

DGEMM (CPU Intensive)

Ave

rage

Gflo

ps

STREAM (Memory Bandwidth)

Single-host Single-site Single-site with Physical Network

Multi-site0

5

10

15

20

2520

.318

253

18.6

0503

8

18.6

6931

1

21.1

8326

9

20.1

5301

6

19.2

6025

5

18.4

1577

6

21.5

0916

5

19.7

5952

18.5

6022

7

17.8

1989

1

21.1

1683

9

18.4

7008

1

17.8

158

17.4

7244

1

19.7

3825

9

Copy Scale Add Triad

Ave

rage

GB/

s

HPL (Peak Performance)

Single-host Single-site Single-site with Physical Network

Multi-site0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

0.3126

0.5229

1.873

0.4117

Gflo

ps

Conclusion

• The virtual cluster cloud works as expected• This cloud is suitable for non-network-

intensive application– Bio-Chemical-Related Research– Video Rendering– Brute-force Searches– Bitcoin Mining :D

Future Works

• Better Overlay Network implementation– Software-defined Networking (SDN)

• Intelligence VM placement mechanism (scheduling)– Swarm algorithm– Firefly algorithm

Q&AThank you for you attention