crown project: a service-oriented grid middleware and its

APSCC, Guangzhou, China Dec 13

CROWN project: a Service-oriented Grid middleware and its challenges

Prof. Jinpeng HuaiBeihang [email protected]:// www.crown.org.cn

mailto:[email protected]

http://www.crown.org.cn/

ACT, Beihang University APSCC06, Guangzhou

Grid/P2P computing is much considered as the next wave of distributed computing, which

promise to enable the federation of heterogeneous and geographically dispersed computing resources to provide more powerful, reliable and uniform services. strive to integrate various resources, including computational power, data storage, communications, software and even human experts.


ContentOverview about CROWNNext for Distributed Computing?


Our MissionHuge Demand for network/Distributed Computing In China

Computing Capacity: First 50 Computers in Top100:30TFlops, More R&D Instruments Hundreds of National Key Labs …

Huge Demand from IT applications, e-Gov, e-Com, e-Oly Game…But, China invests a lot on new resources, At the same time a lot are seating idly, so we have to utilize idle or underused computing power and resources


Our MissionNeeds for greater computing capability

Science & Engineering computationHigh performance Information servicesSocial and economical problems to solve

Needs for cooperative worksConcurrent engineering and Distributed computing

Needs for resource sharing capabilityComputing power, Expensive instruments and services

ACT, Beihang University APSCC06, GuangzhouGrid and Advanced Computing Activities in

ChinaMinistry of Science and Technology

The biggest R&D program, National Hi-tech Program(863) started by Mr. Deng Xiaoping and supported by the Ministry of Sci. and Tech (MOST)

ChinaGrid, by Ministry of EducationNSFGrid, an e-Science Key Project

Network-based Scientific Research Environment


OverviewCROWN denotes:

China Research and Development Environment Over Wide-area Network2004 – 2007, NSF of China + 863 MOST

Goal: Providing Grid-enabled research environment for scientists

Resource SharingCollaboratingTrustworthy Distributed ComputingGrid Infrastructure Research & Construction


CROWNTop five universities and research institutes, in China, with other pilot applications providersMany valuable research resources


Resource Organization & Management

Node Domain RLDS

Domain RLDS

Region Switch

Domain RLDS

Node NodeServices

Device

Device Driver Service

Other Regions

Gateway to EU DataGrid

Gateway to Other Testbed


CROWN Partners


resource

service

Overview of CROWNapp

collective

cooperationmanageability

trustworthy

open, dynamic autonomic

heterogeneous

CROWN


PC Cluster Front End

Cluster Nodes

Device Host

DevicesResources

Node ServerInstall / Config



RLDS RLDS

Register to …

RLDSResource Management

RLDSRLDS

RLDS

Register to …

PortalsRich Client Platform

Scheduler

S S S Workflow EngineWfS

Query Info

CROWN Designer

Eclipse

JDT PDE

S

GenerateServices

AppApplication Layer App App App App

Middleware Layer

Resource Layer

Sec Sec Sec

Monitor

Rich Internet Application


A highly dynamic grid testbed2-Tier Resource Organization

Backbone (Stable Resources)Active Nodes (Volunteer Resources)

Connecting to Other Grid Testbeds

RLDS RLDS

RLDSCROWN Backbone

RLDS RLDS

RLDS

CROWN

Active Nodes

NSNS…

NSNS …NS

NS …

NS

NS…NS NS…

NSNS…


CROWN Node ServerNode Server: A GT 4.0.0-core Extension as Service Container

GAR: a File format for portable serviceHot/Remote Deploy with Security ConsiderationResource Dynamic Information Collecting & ReportingWeb Management UI


CROWN RLDSRLDS: Resource Locating & Description Service

A distributed Grid Information Service ArchitectureSoft-state, topology management of RLDSCollecting of Information of Resources (Hosts) & Web Service / Grid ServicesAdvanced GIQL Query to find available servicesUnified Grid meta info management (GIMS)S-Club: Overlay based high efficient Grid information organization and query


CROWN Designeran Eclipse Plug-in for Grid Service Development

Support the WSRF/WS-I Service Development and DeploymentAuto Generation of Code Skeleton for Java Web /Grid Services


CROWN Scheduler

…

GS

LS LS

Portal

LS …

LS LS

RLDS

RLDS

RLDSGS = Global SchedulerLS = Local Scheduler

Scheduler: A Hierarchal Job Submission and Execution Service

Job Submission : JSDLJob Persistency: HibernateHierarchal Job SchedulingJob Execution: POSIX / Web Service / PBS Job supportedInteroperability with other Job Execution Service


CROWN MonitorMonitor: An event based grid monitoring system

PUSH / PULL information collectingEvent Collectors / ConsumersEclipse RCP based UIQuasi real-time monitoring


Security Architecture in CROWN

Region CA IndentifyMapping&CredentialConverting

Service

Region 1

Domain CA

Node 1

Node 2

Domain AuthenticationService

Domain 1.1

Domain AuthorizationService

Node n

…

Region KDC

Domain KDCNode 1

Node 2 Domain Authentication

Service

Domain 2.1

Domain AuthorizationService

Node n

…

Region 2

？


Security ServicesNode Security Chain

SAM

LWS-Security

WS-Trust

WS-SecConv

WS-Policy

SecConvATNAuth

ServiceIdentityMapping

XA

CM

L

AuthzService

Authentication Handler

Security Chain Framework

Authorization Handler

SecConv & ATN Handler

CROWN Security

ACT, Beihang University APSCC06, GuangzhouRelease History

Jan Feb Mar Apr May

v1.0RC1

First Preview version v0.92 v1.0RC2

May 18, 2005 CROWN v1.0May 23, 2005 CROWN v1.0 English Version

Jun

Jul Aug Sep Oct Nov Dec

Dec 23, 2005CROWN v2.0 Release

v2.0RC2v2.0RC1

2005Internal update v1.5

Jan Feb Mar Apr May Jun2006

Internal update v2.5

Jul Aug Sep Oct Nov Dec

v2.5.1


Performance evaluation CROWN Node Server

MetricsThroughputAverage response timeSuccess rate

Benchmark serviceechoString: stateless servicecounter: stateful service

Performance Comparison with GT4.0


Performance evaluation Security

0 5 10 150.5

1

1.5

2

Concurrent Requests

Res

pons

e Ti

me

(s) Kerberos to PKI

PKI to Kerberos

0 10 20 30 40 500

10

20

30

40

50

Number of Concurrent Requests

Ove

rall

Exe

cutio

n Ti

me(

s) normalwith-TrustTicket

Identity mappingTrust negotiation


International TestingUK e-Science Engineering Task Force (ETF)

CROWN System Evaluation, May 2005First China Grid Middleware EvaluatedGT4 evaluated by ETF in June 2005

4 aspects evaluatedGeneralSystem ManagementUser ExperienceDeveloper Experience

3 UK Universities participatedUniversity of Southampton (SeSC)Imperial College (LeSC)University of Newcastle (NEReSC)


Applications - gVizRunning on UK White Rose Grid +

CROWN Grid (demo at AHM’05)


UK OMIIInteroperability between CROWN and OMIIOMII-China project

Beihang: coordinator of OMII-China

OMII-Europe projectEU FP6 projectBeihang: one of the five Chinese partners


Globus AllianceGlobus / ANL

Prof. Ian FosterExchange ides with CROWN team

MOU：grid middleware and trusted computing

INRIAWeb Service/ECHOGRID project


ApplicationsAREM: Advanced Regional Eta-coordinate Numerical Prediction Model. (This is a Numeric Simulation of Weather Forecasting for a certain region)MDP: Massive Multimedia Data Processing PlatformBlast: A well-known Gene Sequence Comparison Program

DSSR: Digital Sky Survey Retrieval, an of Virtual ObservatoryUDMGrid: University Digital Museum Powered by CROWN, a application of OGSA-DAI. Heterogamous Databases Integration……


Beijing

Hong Kong

ChangshaChongqing

Beihang UniversityCNIC, CASPeking UniversityTsinghua UniversityLASG, CAS

Chongqing University

National University of Defense Technology

HKUST: Hong Kong University of Science & Technology

CROWN Testbed (2006)

Xi’an

Hefei

Shanghai

China University of Science & Technology

Shanghai Super Computing CentreShanghai Jiaotong University

Xi’an Jiaotong University

Leeds

Melbourne


Interoperability among GridsDon’t interrupt deployed applications, Make least or no modification to grid middleware

Make full use of extension points of grid middlewareApproaches

Gateway Bridge different grid middleware togetherMake it easy for management, monitor and control

Workflow EnactorAssemble distributed services into oneWork in client side mannerExtensible or easily develop

AdaptorDifferent adaptor for different underlying middleware Service invocationSoap message translation and so on


Interoperation Demo, UK e-Science AHM 06

Cross Grid SchedulingJob Submission Interoperation

Between OMII- GridSAM / CROWN Scheduler

News Press: http://www.omii.ac.uk/projects/SS-CROWNGrid.jsp


Interoperation Demo, SuperComputing06

News Press: http://www.gridtoday.com/grid/1109586.html

SC 2006：Multi-Grids Interoperation DemoOpen Grid Forum HPC Profile WorkgroupCROWN has interoperated with Globus, OMII, UNICORE,etc

ACT, Beihang University APSCC06, GuangzhouInteroperation Demo, SuperComputing06


Interoperation Demo, SuperComputing06

•Globus•GridSAM•CROWN•EGEE

•gLite + CREAM• Microsoft•Platform•U. of Virginia

•.Net + Genesis II•Fujitsu - Unicore


ContentOverview about CROWNNext for Distributed Computing?


Vision

Host PC

Scope

NetworkApplication

WordProcessing

MathematicComputing

InternetApplication

Intranet

MainframeComputing

Client/Server

Client/Network

Internet

Client/ VirtualNetwork or Environment?


The trend of Grid Computing

computing service virtualization


New Challenges for distributed computing?

Network Computing

DistributedComputing

Distribute obj./component

CORBA DCOM Java/EJB

Web service

Service-Oriented Computing

Parallel computingand

Distributed system

clusterMeta-computing

Computing grid

What are big challenges for future Distributed ComputingService is the atomic unitOpen vs controlled environment

What is Distributed Computing if Service-oriented Computing?

our next “Internet”How to design dis. soft? Service-oriented SoftwareSecurity and trustEvaluation


What challenges?“Soft” issues is not soft, compared to hard issues/functional requirement in Internet!

Open Internet/wireless/mobile Net, dynamically changing computing on its evolution

Internet: Nodes – routers Links – optical fibersWWW: Nodes – document files Links – hyperlinks Social Networks: Nodes – individuals Links – relations

Soft/nonfunctional requirement become a new kernel, such as on intra-/inter-/extra-virtual organizations, dependability, trusting, availability…what is Internet programming model? or new internet model-virtual computing environment for the “Soft”


What challenges?What is our opportunities?

Virtual Service OverlaySynthetical is good, such as service composition with dependability + trusty +tolerancePractical app or real runtime monitor, and interoperations among VOs

The solution of every problem creates a new big problem/Challenges!


Nodes:ServersDevicesNetwork Elements

Router, Switch, Gateway

Links:Physical linkVirtual (VPN)Channel (PVC..)

Data:BitsPackets

Managed by: Information

Technology

Nodes:Services

informationtransactionstream

UsersVirtualized Devices

Actuator, sensor

Links:HyperlinksRelationships

Data:CommandsNotificationsTyped informationManaged by a

new group:Service

Technology

A new network: virtual Service Network

•Large-Scale•Dynamic + Distributed•Autonomous+Uncontrollable•Heterogeneous+Cross-domain•Needs: dependable, secure,Available…•……….

ACT, Beihang University APSCC06, GuangzhouWhere shall we go :

dependable Service-oriented Computing?1) is possible for Overlay-based “ Virtual Service Network” with Dependability

application-level distributed Service-oriented systemProvide a overlay to build the disordered internet into being Managed, dependable and AdaptiveSecurity, Privacy and information Integrity/AssuranceProvide a quantitive and provable approach for evaluating the Dependability, Availability and Reliability of the system


What is the link?Availability

SafetyReliability

IntegrityMaintainability

Dependabilityand

Security

Attributes

Means

Threats

FaultsFailuresErrors

Fault ToleranceFault Prevention

Fault ForecastingFault Remove


What is the link?

Availability

DependabilitySafety

Reliability

Integrity

Confidentiality

Maintainability

Security


dependable Service-oriented Computing?

2)High Dependable Soft and SystemsHow to combine the dependability into Service-Composition for Service on Demand, Protocol computingHigh Dependable Service-oriented Computing toolkits/ testing-bed on both of WR and CROWN ? High Configurable security services with dependability and online evolution

ACT, Beihang University APSCC06, GuangzhouService-based dependable software design &

ProductionPast New

decompose

Fixed goal+ control Service-based paradigm

S

S

S S

S

SS

entity

compose

s

S

service

Multi-copose


Protocol-based computing for cross-organizational SDP sys

Domain A

Domain B

Domain C

s1

s2

s3

protocols

protocolsprotocols

Service-based dependable software Design & Production


dependable Service-oriented Computing?3)Testing, Maintainability and Evaluation?

Maybe we should have some practical experiments /applications for evaluating our system, especially for dependability, availability, Security ( privacy, integrity), and their Combination to the Service-Overlay, softwareHow to combine DBMS into Service-oriented Computing?

4)Testing and Evaluation for distributed Software and Security is more complex

ACT, Beihang University APSCC06, GuangzhouCROWN3.0 Virtual Computing

Environment5-Tier Architecture

Service Coordination LayerService Management LayerOpen Service Layer*Virtual Resource Layer*Physical Resource Layer Virtual Resource Layer

Open Service Layer

Service Coordination Layer

Service Management Layer


ACT focus on “Future”

Quality and Security M

anagement

Basic platform

Middleware supporting platform

Component resource

Web service CORBA EJB Other components

Operatingsystem DB

Basicservice

Service & management

toolkits Service schedule

Software production platform

Toolkits for registry

and deploy

Require.Specification

toolkits for Software

Developing


Thank You!Q&A

More:[email protected]

mailto:[email protected]

crown project: a service-oriented grid middleware and its

Documents