cpscg: constructive platform for specialized computing grid institute of high performance computing...

CPSCG:Constructive Platform for Specialized

Computing Grid

Institute of High Performance ComputingDepartment of Computer Science

Tsinghua UniversityMing CHEN, Guangwen YANG

[email protected], [email protected]

Advanced Grid Research Workshops through European and Asian Co-operation

Outline

1. Introduction

2. Architecture

3. Applications

4. Summary


Enabling fast, straightforward construction of grid system

• CPSCG is not only another grid middleware, but also a lightweight and ready-to-use platform for directly deploying grid service

• CPSCG provides flexible system-level services to make legacy applications grid-service ready


Key Features

1. Streamline deployment2. Support both plain web service and WSRF 3. Automate application service visualization4. Fully distributed while maintaining single image


Design Consideration

1. Enable fast deployment of, and easy access to application services

2. Endorse domain autonomy3. Adapt to dynamic, unreliable, unpredictable Internet

loose-coupled domains, fully distributed system, best effort services


Outline

1. Introduction

2. Architecture

3. Applications

4. Summary


Overview

One Platform– Information Center– Front End– Back End

Two Support Tool– Encapsulation– FiTMPI


Platform: Deployment

DomainFront end

Back EndWindows

Back EndLinux

Information Center

Linux ClusterDomain

Front End

Back EndIA32

Back EndIA64

LAN

WAN/LAN

WAN/LAN

DomainFront End

Back EndPC

Back EndCluster

Information Center

Domain A

Domain B

Domain C

Support Tool


Se

curi

ty MonitorMonitor ScheduleSchedule

LocalUser`

LocalUser`

LocalServices

LocalServices

LocalJobs

LocalJobs

Local Service Engine Remote Service ProxyDynamic Service Render

Remote DataAgent

Remote DataAgent

Local DataManagement

Local DataManagement

Back EndAgent

Back EndAgent

Back EndAgent

Back EndAgent

Service Layer

Function Layer

SynchronizerService List

Domains List Maintainer

Domains List Maintainer

Information Center

Platform Components

Back EndLocal SchedulerMonitor Local SchedulerMonitor


Information Centers

• Maintain information of all registered services for service query

• Support UDDI to be compatible with Web Service• Fully duplicated to be resilient to single failure• Synchronize with each other using Bloom Filter to reduce

network traffics and timestamp to keep consistent


Synchronization

InformationCenter A

ServiceList

Bloom Filter

ServiceList Digest

InformationCenter B

Exchange Digest

Synchronize


Service QueryJob Management

Status Query

Account ManagementNode Management

Service ManagementAccess Policy Management

Domain Front End

DomainFront End

Back EndPC

Back Endcluster

Domain Administrator

End User

Application

Web service/WSRF

Web Pages

Every front end represents a single-sign point


One-hop Routing: Improve Reliability

Remote Service Proxy Local Service Engine

Domain A

Remote Service Proxy

Intermediate Domain C

Domain B


Monitor Architecture

sensor-II data stream interface over LAN (Level-B Communication - TCP)

ComputerCluster

Super ComputerOr other PC

ClusterNodes

sensor-I

FrontEndNode

sensor-II

UDPInterconnection

Level-ACommunication

sensor-I

sensor-IILevel-A

Communication

Aggregator


Snapshot


Back End

• Deployed in computing PCs/Clusters, hidden from other domains

• Communicate with its CPSCG front end and execute job from front end

• Support MPI/PBS• Can be another CPSCG front end


Overview

One Platform– Information Center– Front End– Back End

Two Support Tool– Encapsulation– FiTMPI


Support Tool: Legacy Applications Encapsulation

• Make legacy applications, especially large volume of command-line based applications, grid service ready

• Fast, straightforward, but flexible encapsulation


Process of Encapsulation

Command-line Application

Command-line Application

Interpreter&

ServicePackage

Generator

Interpreter&

ServicePackage

Generator

GT4

Tomcat/JBoss

Web Service Package

WSRFPackage

Web Service Package

WSRFPackage

Web Service Package

Web Service Package

InteractiveUser Input


FitMPI

• Heterogeneous environment• Resource Monitoring• Auto deployment• Communication between intranets• Input/Output Management• Process Management


A “Hello World”-Like Demo

Install a system

Publish a back end

Deploy a service (Encapsulation): PClustalW

Invoke the service and get the result


Install


Publish A Back End


Deploy Service Step 1: Select An Installed Legacy Application


Deploy Service Step 2: Edit Service Interface


Deploy Service Step 2: Edit Service Interface (Cont.)


Deploy Service Step 3: Publish!


Invoke ServiceStep 1: Select An Intended Service

Service can also be accessed via Web Service or WSRF


Invoke ServiceStep 2: Input Parameters


Get the Result


Analyze Result with Other Tool


Application: BioInformatic Grid

Phrap

Cap3

Blaster

PClustalw (MPI)/Clustalw

…


Summary

1. CPSCG provides a platform helping end users fast and efficiently deploy grid system and grid services

2. CPSCG Supports domain autonomy via loose-coupled architecture, while offering global single image

Thank you!

cpscg: constructive platform for specialized computing grid institute of high performance computing...

Documents

end slide

end domain

grid service cpscg

end pc

end ia32

end windows

asian cooperation platform

grid service ready fast