nbcr science gateway: transparent access to remote resources through rich desktop clients nbcr...

NBCR Science Gateway: Transparent Access to Remote NBCR Science Gateway: Transparent Access to Remote Resources Through Rich Desktop ClientsResources Through Rich Desktop Clients

Luca Clementiclem@sdsc.edu

OutlineOutline

• NBCR/PRAGMA

• NBCR SOA infrastructure– Opal Overview – Opal Toolkit GUI– Opal Integration into Existing Tools– Access to Multiple Resources (TG, PRAGMA)

National Biomedical Computational ResourceNational Biomedical Computational Resource

• Funded by NIH (NCRR)

• Its mission is to conduct, catalyze, and enable biomedical research by harnessing, developing, and deploying forefront computational, information and grid technologies – Develop and deploy advanced

computational, data and visualization tools.– Integrate them in an advanced grid

framework.– Deliver and support this grid/Cyber-

infrastructure– Training (Summer Institute)

• Funded by NIH (NCRR)

• Its mission is to conduct, catalyze, and enable biomedical research by harnessing, developing, and deploying forefront computational, information and grid technologies – Develop and deploy advanced

computational, data and visualization tools.– Integrate them in an advanced grid

framework.– Deliver and support this grid/Cyber-

infrastructure– Training (Summer Institute)

National Biomedical Computational ResourceNational Biomedical Computational Resource

In the Pacific Rim and Globally 29 institutions

Strengthen Existing and Establish New Sustainable Collaborations

Work with Science Teams to Advance Grid Technologies and

Improve the Underlying Infrastructure

Pacific Rim Application and Grid Middleware Assembly (PRAGMA)

http://www.pragma-grid.net

A Practical Collaborative Framework

•Collaborations•Integration of existing applications•Deployment of a Grid infrastructure

MotivationMotivation

• Integration of different tools (client side application, workflow tools, web portals, etc.)– Several user interfaces

• Seamless access to computational resources– Easy from the scientific user perspective

Condor pool SGE Cluster PBS Cluster

Globus Globus Globus

Application Services Security Services (GAMA)

StateMgmt

Web server PMV/Vision Kepler

Big PictureBig Picture

Opal ToolkitOpal Toolkit

• Opal allows application developers to publish command-line applications using Web services– Minimal deployment effort: no coding, only a simple

configuration file – plain web service, SOAP– Common interface: every application uses the same

WSDL (same stubs on the client side)– Simple APIs (Launch job, Query job status, Get job

outputs, Get output as Base64, Destroy job, etc.)

Opal ToolkitOpal Toolkit

– Every application exposed with its own URL http://ws.nbcr.net/opal/services/Pdb2pqrService

– It takes care of data staging– It supports submission via:

• Fork• Globus GRAM• DRMAA• CSF4 (experimental)

– Run all the applications under a single user account– Used in NBCR, CAMERA, GLEON, among other

Opal Toolkit / Client SideOpal Toolkit / Client Side

• Several clients APIs available: Java, Python, PERL.

• Command line generic client:

# java edu.sdsc.nbcr.opal.GenericServiceClient -l

http://localhost:8080/axis/services/PDB2PQRServicePort -r launchJob -a "-ipdb sample.pdb -h -opdb output.pdb“ -f etc/sample.pdb

• Too complex for beginner users • Graphical User Interfaces

getAppMetadata

launchJob

getOutput…

PDB2PQR

getAppMetadata

launchJob

getOutput…

MEME

…Opal server

User

Vision

Kepler

User clients

Browser

Opal GUIOpal GUI

• Main characteristics:– Working out of the box (no configuration)– Multiplatform -> Web interface– Shipped with Opal

• Key features:– Access Statistics – List of services– Simple submission form– Advanced submission form

Access StatisticsAccess Statistics

Go to http://ws.nbcr.net

NBCR production server

List of ServicesList of Services

User has to input command line!

Too error prone.

Submission form should be customized on command line arguments

Simple Submission FormSimple Submission Form

Advanced Submission FormAdvanced Submission Form

• When deploying a scientific application there is an optional part to describe input and output of the application

• It is a command line syntax description language

Group 1

Group 2

Group 3

Group 4

Exclusive group

Input file

String

Exclusive enumeration

Flag

PDB2PQR Advanced Submission FormPDB2PQR Advanced Submission Form

Vision Opal NodeVision Opal Node

Kepler Opal Web Services ActorKepler Opal Web Services Actor

Autodock ToolsAutodock Tools

PBD2PQR Web SitePBD2PQR Web Site

Access to Multiple ResourcesAccess to Multiple Resources

CSF4CSF4

• Community Scheduler Framework:

– Open Source project and can be accessed at http://sourceforge.net/projects/gcsf

– Developed by Lab. of Distributed Computing and System Architecture, Jilin University, China

– It is a metascheduler framework hosted as an Execution Component in GT4 container

– It uses WSRF compliant services– It can submit jobs to Globus

CSF4 Meta-Scheduler

Grid Site GT2

LSF

Grid Site GT2

PBS

Grid Site GT4

SGE

Grid Site GT2

Condor……

•GT2 (gatekeeper)

•GT4 (WS-GRAM)

CSF4 Typical DeploymentCSF4 Typical Deployment

CSF4 New FeatureCSF4 New Feature

Users want to run applications

• Application based scheduling:

– CSF4 keeps a table of available applications

Application name Resources

namd tg1:/usr/local/namd

namd tg2:/usr/share/bin/namd

autodock pragma1:/usr/local/autodock

autodock pragma2:/usr/local/autodock

CSF4 New FeatureCSF4 New Feature

Users want to run applications

• Application based scheduling:

– CSF4 keeps a table of available applications

Application name Resources

namd tg1:/usr/local/namd

namd tg2:/usr/share/bin/namd

autodock pragma1:/usr/local/autodock

autodock pragma2:/usr/local/autodock

Avian Flu Grid

Opal CSF4 IntegrationOpal CSF4 Integration

• Opal plug-in for submission to CSF4

• Deployment architecture:

Opal Enable Clients

App A App B App C

Opal Server

App A App B App C

CSF4

ClusterAappAappB

Globus GRAM

…

Globus GRAM

ClusterBappAappC

WS/SOAP

WSRF

(Pre/WS)-GRAM

Future DirectionsFuture Directions

• Opal 2.0 will be released soon – Improved architecture (plug-in model)– URL support for input and output files– Embedded database (Hibernate)

• Support for other metascheduler (Gridway)• Opal GUI for Gridsphere (Portlet)

AcknowledgmentsAcknowledgments

• Wilfred Li, Peter Arzberg (NBCR vision)• Sriram Krishnan (Opal)• Xiaohui Wei (CFS4)• Many other NBCR developers...

• More information, downloads, documentation:– http://nbcr.net – http://nbcr.net/software/opal/