Managing HPC Clusters with Dasein

Udit AgarwalDept. of Computer ScienceClemson University

Amy Apon, Ph.D.Chair, Dept. of Computer ScienceClemson University

J. Craig Lowery, Ph.D.Sr. Distinguished EngineerDell Software Group

Linh B. Ngo, Ph.D.Research Associate,Dept. of Computer ScienceClemson University

2 Dell Unrestricted Software

Research Engagement• Deliverables

– Mapping of the Dasein model to Palmetto interfaces– Assessment of potential completeness and utility– Sample code implementing key functions– Final report

3 Dell Unrestricted Software

Functional Mapping Analysis

ComputeServices

getAutoScalingSupport()

getImageSupport()

getSnapshotSupport()

……………

listModules()

runJob()

jobStatus()

deleteJob()

PalmettoDasein

module avail

qsub filename.pbs

qstat –anu username

qdel jobid

Provides the list of software packages

Submitting jobs to run on the cluster

To check the status of your job

Killing a job submitted

Impl

emen

ts

Impl

emen

ts

Implements

Class NameMethod Name

New methods added

4 Dell Unrestricted Software

Solution ArchitectureDell Cloud Manager

Dasein

Amazon Web Service

JoyentPBS

Scheduler

Dasein API

AWS API

Joyent API

Common data in Dasein format

Cloud status in AWS data format

Cloud status in Joyent data format

Palmetto super-

computerCluster status in Palmetto data format

Linux Commands, Shell Scripts and Submission Scripts

Result of executed Commands

Code modified for this project

New management domain enabled.

PBS Scheduler API

5 Dell Unrestricted Software

Summary• Dasein (and, therefore, Dell Cloud Manager and similar

software) can meaningfully manage HPC clusters, but requires object model extensions.

• A mature solution could be used by faculty, staff, students and researchers to manage HPC jobs and “regular” cloud resources through a single portal/tool.

• Could simplify the use of HPC clusters for non-technical researchers who lack knowledge/training of the scheduler, Linux shell, etc.