ANR CIGC LEGO (ANR-CICG-05-11)Bordeaux, 2006, December 11th

Automatic Application Deployment on Grids

Landry Breuil, Boris Daix,Sébastien Lacour, Christian Pérez

PARIS Research TeamINRIA/IRISA

Rennes

How to Deploy my Applicationon Grid Resources?

Visualization

Homogeneous cluster

SAN

SAN

Homogeneous cluster

LAN

WAN

Supercomputer

Flowcode

Transport code

Controller

velocity

scalarsscalars

concentration

Manual Deployment Discover available grid resources Select grid resources for execution

OS, architecture compatibility Map the application onto the selected resources

MPI processes Components

Select compatible compiled executables Upload and install executables, stage input files

in Launch processes on remote computers Set configuration parameters of the application

Components' attributes Network topology information

Too complex!

Automatic Deployment Automatic

Resource discovery Execution node selection File installation Process launch Application configuration

Hide application complexity

Hide grid complexity

stop reading your e-mails!

Grid

Generic Application Description

Translator From specific to generic application description Straightforward to write

CCM Appl. Descriptio

n

MPI Appl. Descriptio

n

GridCCM Appl.

Description

CCM Appl. Descriptio

n

MPI Appl. Descriptio

n

GridCCM Appl.

Description

CCM Planner

MPI Planner

GridCCM Planner

Plan Execution

Plan Execution

Plan Execution

Generic Application Description

Deployment Planner

Deployment Plan Execution

Identification of the Stepsof Automatic Deployment

MPI Application Description

CCM Application Description

Resource Description

Generic Application Description

Control Parameters

Deployment Planning

Deployment Plan Execution

Application Configuration

Static

Applica

tio

ns

Deployment Tool

ADAGE Feature List MPI, JXTA, CCM application descriptions Network topology description Generic application description Simple control parameters, simple planner Deployment plan execution (RSH/SSH) Basic file transfer support

Aware of file visibility (~NFS) Does not check for already available files

Redeployment support Either re-play or add/remove elements

How Easy is it, in Practice? As simple as A-B-C

adage-deploy –resource http://host/file.xml–application my_appl.zip–ctrl_param

my_control_parameters Grid resource description

Written once for all by grid admins Optional control_parameters constraints

Keep control on the deployment process

ADAGE & Grid5000 ADAGE is not able to directly talk with OAR PERL script to automatically generate

resource description from OARgrid id oarstat2resources.pl –g 6332 –outputfile

r.xml adage-deploy –inres r.xml …

Some ADAGE results JXTA

1 peer == 1 process M. Jan deployed

~ 29.000 peers on ~ 145 machines in ~ 95 seconds

Most of the time take by ssh commands

CCM 1 component == 1 process H. Bouziane deployed

4000 components on 974 processors machines 432 machines on several G5K sites

Enable to CCM-plugin optimization

Towards ADAGE v2 Complete rewrite of ADAGE

By L. Breuil Objectives of ADAGE v2

Provide a clean code architecture Ease the additions of

Programming model (GridCCM), ie plugins Planner

Better back-end support GAT, Taktuk, …

Support of “dynamic” applications In cooperation with B. Daix, PhD

Discussion ADAGE: a model to support automatic application

deployment Ongoing/Future work

Finish ADAGE v2 (~ begining 2007 ) Stabilize plugins API Stabilize planner API wrt to resources Dynamic application management PadicoTM support? Adapt application description in function of resources

DIET support Not directly targeted

Fault tolerance Application monitoring