docker for hpc: how e4 is using it

Docker for HPC

Docker enables to build, ship and run any app, anywhere:

• HPC apps

• Web apps

• DBs

• Others

Very useful in HPC environment to align and simplify deployment of HPC apps on clusters.

Problem solved:- Deploy several version of one app on

the same node (e.g. Ansys 15 and Ansys 14)

- Deploy an old version of an HPC appbased on legacy environment

- Keep cluster nodes aligned with the same version of libriaries and binaries

3

• When images are updated, only ship relevant layers

• Easy of move HPC apps thanks to Image distribution protocol

• Images are broken down into layers

• Dependency between images (eg: web app depends on DB)

• On VMs or physical machine (or on your laptop!)

• Available on OS X and Windows (boot2docker)

• Benchmarks show no difference between containers and bare metal

Development

VMQA server Public Cloud Contributor’s

laptop

Mu

ltip

licit

y o

f h

ard

war

e en

viro

nm

en

ts

Production

ClusterCustomer Data

Center

Can

I migrate

smo

oth

ly and

qu

ickly

Configure Once, Run Anything

Static website

MPI libraries

CFD code

User DB

Ansys

HPC solver

V1 V1.5 V2 V2.5 V3 V3.5 V4

• Containers are created from an image

• Image creation is easy

• Image upgrade is fast

• Immutable infrastructure is easy to implement

• Legacy App image library

AppA

Hypervisor

Host OS

Server

GuestOS

Bins/Libs

AppA’

GuestOS

Bins/Libs

AppB

GuestOS

Bins/Libs

Ap

p A

’

Do

cker

Host OS

Server

Bins/Libs

Ap

p A

Bins/Libs

Ap

p B

Ap

p B

’

Ap

p B

’

Ap

p B

’

VM

Container

Containers are isolated,but share OS and, whereappropriate, bins/libraries

GuestOS

GuestOS

• Simplify HPC app deployment in cluster environment

• Standardize HPC configuration among nodes in a cluster

• Isolate HPC apps to eliminate inconsistency

• Run legacy HPC app and keep a library of images

• No virtualization overhead