crown grid tutorial qin li [email protected] beihang university

CROWN Grid TutorialQin Li

[email protected]

Beihang University

EU project: RIO31844-OMII-EUROPE

Agenda

• CROWN Overview

• CROWN User Environments

• CROWN System Administration


Agenda

• CROWN Overview– Architecture– Modules Introduction

• CROWN User Environments



What’s CROWN

• CROWN China Research and Development environment Over

Wide-area Network– Jointly Funded by

• National Natural Science Foundation of China– NSFC e-Science Program

• Ministry of Science and Technology of China– 863 Hi-Tech Program, OMII-China Project– 973 National Basic Research Program, VCE Project

• CROWN is– A Grid Middleware Suite– A Prototype of China E-Science Portal– A Research Platform of Grid Technology


CROWN Partners


23/4/19 6

CROWN Release History

Jan Feb Mar Apr May

First Preview version v0.92 v1.0RC1v1.0RC2

May 18, 2005 CROWN v1.0May 23, 2005 CROWN v1.0 English Version

Jun

Jul Aug Sep Oct Nov Dec

Dec 23, 2005CROWN v2.0 Release

v2.0RC2v2.0RC1

2005

Internal update v1.5

Jan Feb Mar Apr May Jun2006

Internal update v2.5

Jul Aug Sep Oct Nov Dec

CROWN v2.5.1 CROWN Pre 3.0


PC Cluster Front End

Cluster Nodes

Device Host

DevicesResources

Node Server

Install / Config

Node Server

Install / Config

Node Server

Install / Config

RLDS RLDS

Register to …

RLDSResource Management

RLDSRLDS

RLDSRegister to …

PortalsRich Client Framework

Scheduler

S S S Workflow Engine

WfS

Query Info

CROWN Designer

Eclipse

JDT PDE

S

GenerateServices

AppApplication Layer App App App App

MiddlewareLayer

Resource Layer

Sec Sec Sec

Mo

nito

r

Rich Internet Application


CROWN Node Server

• Node Server: An Extension to GT 4.0.0 WSRF Container– Remote Management (Remote/Hot Deploy)– Dynamics Resource Information Monitoring

WSRF-compliant Service Container(Globu Toolkit 4.0 core)

ServiceDeployment

Container Management

Service Management

Container Statistics

Computing...

Storage Network Devices

Physical Infrastructure

Resource Monitoring Service

Client tools and API

Excep

tion

Han

dlin

g


CROWN RLDS

• RLDS: Resource Locating & Description Service– A distributed Grid information Service Architecture– Soft-state maintenance, topology management of

RLDS– Collecting of Information of Resources (Hosts) &

Web / Grid Services– Advanced GIQL Query to find available services or

servers


Resource Organization & Management

Node Domain RLDS

Domain RLDS

Region Switch

Domain RLDS

Node NodeServices

Device

Device Driver Service

Other Regions

Gateway to Other Grid1

Gateway to Other Grid2


CROWN Designer

• Designer: an Eclipse Plug-in for Grid Service Development– Support the WSRF/WS-I Service Development,

Deployment and Debugging– Automatic Generation of Code Skeleton & WSDL for

Java Web Services


CROWN Portal

• Portal: A JSP-based Web Interface– User Registration / Certification Management– Application Integration & User Job Submission– Rich Internet Architecture (RIA) based User

Experience


• Scheduler: A Hierarchal Job Submission and Execution Service– Specification adopt:

• OGSA-BES,OGSA-JSDL,OGSA-HPCP

– Hierarchal Job Scheduling– Job Execution:

• POSIX / Web Service / PBS Job supported

CROWN Scheduler

GS

LS LS

Portal

LS …

LS LS…

RLDS

RLDS

RLDS

GS = Global SchedulerLS = Local Scheduler


CROWN Monitor

• Monitor: An event based grid monitoring system– PUSH / PULL info collecting– Event Collectors / Consumers– Eclipse RCP based UI– Quasi real-time monitoring


CROWN Data

• Data: A data mgmt. ,transmission and access system– Fast & Reliable data transmission– Transparent data access interface– Unified data perspective– Replication management– Data Locating


MetadataServerMetadataServer

MetadataServer

Local Data Service Sec Local Data Service Sec Local Data Service Sec

Regist/Report Regist/Report

Regist

HPSS, UniTree, DMF, ... Unix, NT, ...DB2, Oracle, Sybase, SQLServer, ...

Logical Data Resource Inspector

Rich Client Platform Portal

Query

ManagementClient

Physical Data Layer

Logical Data Layer

User Presentation

Layer

RegionGateway

RegionGateway

RegionGateway

MetadataServerTree

Region Federation

CROWN Data Architecture


CROWN Home• CROWN Home: provided GUI for Node Server

– Eclipse RCP based GUI– Desktop users

• Java Code Security in CROWN Node Server• CPU/Memory/Disk quota• Visualized Security Configuration & Certificate Mgmt.


Security Architecture for CROWN

Region CA

Identity Mapping&CredentialConverting

Service

Region 1

Domain CA

Node 1

Node 2

Domain AuthenticationService

Domain 1.1

Domain AuthorizationService

Node n

…

Region KDC

Domain KDCNode 1

Node 2 Domain Authentication

Service

Domain 2.1

Domain AuthorizationService

Node n

…

Region 2

？


Security ServicesNode Security ChainS

AM

L

WS-Security

WS-Trust

WS-SecConv

WS-P

olicy

SecConvATNAuth

ServiceIdentityMapping

XA

CM

L

AuthzService

Authentication Handler

Security Chain Framework

Authorization Handler

SecConv & ATN Handler

Security Architecture for CROWN


Beijing

Hong Kong

ChangshaChongqing

Beihang UniversityCNIC, CASPeking UniversityTsinghua UniversityLASG, CAS

Chongqing University

National University of Defense Technology

HKUST: Hong Kong University of Science & Technology

CROWN Testbed -China(2006)

Shanghai

Leeds

Melbourne

Chicago


CROWN Applications• AREM: Advanced Regional Eta-coordinate Numerical

Prediction Model. (This is a Numeric Simulation of Weather Forecasting for a certain region)

• MDP: Massive Multimedia Data Processing Platform• Blast: A well-known Gene Sequence Comparison Program

• DSSR: Digital Sky Survey Retrieval, an of Virtual Observatory

• UDMGrid: University Digital Museum Powered by CROWN, a application of OGSA-DAI. Heterogamous Databases Integration

• ……


CROWN 3.0 - Virtual Computing Environment-2007

• 5-Tier Architecture– Service Coordination Layer– Service Management Layer– Open Service Layer*– Virtual Resource Layer*– Physical Resource Layer

Virtual Resource Layer

Open Service Layer

Service Coordination Layer

Service Management Layer


Agenda

• CROWN Overview• CROWN User Environments

– How to setup a basic CROWN grid environment– How to develop and deploy a CROWN service


How to setup a basic CROWN grid environmentUser Case Scenario


How to setup a basic CROWN grid environment

• Preparation work for the best practice• Install a Single CROWN Node Server• Install Information Services• Install Schedule Service• Verify the Installation of CROWN


How to setup a basic CROWN grid environment- Preparation work for the best practice

• Set the environment variable for JDK• Install the MySQL database server• Get the software packages

– crown_nodeserver_2.5.zip– crown_schedule_service_2.5.zip– crown_rlds_service_2.5.zip– crown_gims_service_2.5.zip– crown_regionswitch_service_2.5.zip– crown_regionregistry_service_2.5.zip


How to setup a basic CROWN grid environment- Install a Single CROWN Node Server

• Configure & Admin the Node Sever• Remote & Hot Deployment/Undeployment• Remote Container Configuration• Remote Monitoring


How to setup a basic CROWN grid environment - Install a Single CROWN Node Server

• Configure & Admin the Node Sever– Download & Extract Node Server– Setup %GLOBUS_LOCATION% environment variable– Edit the configuration files

• config\container-config.xml• config\staticsysinfo.xml

– Start NodeServer• start-crown-server.sh

– Stop NodeServer• stop-crown-server.sh


How to setup a basic CROWN grid environment - Remote & Hot Deployment

• Get a gar(grid archive) file– crown_hello_service.gar

• First method (local)– copy the gar file to %GLOBUS_LOCATION%\auto-deploy

• Second method (under linux) (remote) – scp the gar file hostname:/pathto_ns/auto-deploy

• Third method (remote)– cd %GLOBUS_LOCATION%– bin\crown-remote-deploy -o Attachment –a crown_hello_service.gar -

s http://remotehost:8080/wsrf/services/RemoteDeployService


How to setup a basic CROWN grid environment Screen Output

[CROWN NodeServer Deployer] begin deploy preprocess

[CROWN NodeServer Deployer] cn.org.crown.server.deploy.preprocessor.GARIntegrity

CheckPreProcessor:the gar is ok

[CROWN NodeServer Deployer] cn.org.crown.server.deploy.preprocessor.WSDDValidate

PreProcessor:the WSDD file is valid!

[CROWN NodeServer Deployer] deploy preprocess complete

[CROWN NodeServer Deployer] begin to decompress gar package...

[CROWN NodeServer Deployer] begin to copy jar files...

[CROWN NodeServer Deployer] begin to copy WSDD file...

[CROWN NodeServer Deployer] begin to copy WSDL files...

[CROWN NodeServer Deployer] begin to copy bin files...

[CROWN NodeServer Deployer] begin to handle client-config.wsdd...

[CROWN NodeServer Deployer] begin to generate undeploy.xml...

[CROWN NodeServer Deployer] begin to handle post-deploy.xml...

[CROWN NodeServer] begin to update Axis Server...

[CROWN NodeServer] update Axis Server successfully

[Auto Deployer] GAR file crown_hello_service.gar redeployed successfuly

[ServerConfigUpdater] Server Config updated ...


How to setup a basic CROWN grid environment Remote Undeployment

• First method (local)– del %GLOBUS_LOCATION%\auto-deploy\

crown_hello_service.gar• Second method (remote)

– cd %GLOBUS_LOCATION%– bin\crown-remote-undeploy.bat –n

crown_hello_service –s http://remotehost:8080/wsrf/services/RemoteDeployService


How to setup a basic CROWN grid environment Screen Output

[Auto Deployer] Undeploying crown_hello_service.gar

[CROWN NodeServer Deployer] undeploy crown_hello_service.gar...

[CROWN NodeServer Deployer] begin to delete folder etc\HelloWorld...

[CROWN NodeServer Deployer] begin to delete file bin\cn\org\crown\HelloWorld.class...

[CROWN NodeServer Deployer] begin to delete file share\schema\HelloWorld.wsdl...

[CROWN NodeServer Deployer] begin to delete folder lib\HelloWorld...

[CROWN NodeServer Deployer] begin to delete undeploy.xml...

[CROWN NodeServer] begin to update Axis Server...

[CROWN NodeServer] update Axis Server successfully

[Auto Deployer] GAR file crown_hello_service.gar undeploy successfuly

[ServerConfigUpdater] Server Config updated ...


How to setup a basic CROWN grid environment Get Deployed Service List

> bin\crown-get-deployed-gars.bat

Deployed Gars are:

crown_container_config_service

crown_container_stat_service

crown_log_service

crown_remote_deploy

crown_resource_monitor

…

crown_hello_service


How to setup a basic CROWN grid environment Node Server Remote Configuration

• Enable/Disable Service– bin\crown-disable-service.bat

• Thread Pool Configuration– bin\crown-get-thread-pool-info.bat– bin\crown-set-pool-size.bat– bin\crown-set-high-water-mark.bat


How to setup a basic CROWN grid environment Remote Monitoring

• NodeServer will report CPU/Mem/Disk usage to RLDS• CROWN Monitor will collection those information

from RLDS


How to setup a basic CROWN grid environment- Install Information Services

• Install Information Services– Install RegionRegistry Service– Install RegionSwitch Service– Install GIMS Service– Install RLDS Service



• Install RegionRegistry Service– Unzip the file crown_regionregistry_service_2.5.zip– Copy the gar file to the %GLOBUS_LOCATION%\

auto-deploy directory• Install RegionSwitch Service

– Unzip the file crown_regionswitch_service_2.5.zip– Copy the gar file to the %GLOBUS_LOCATION%\

auto-deploy directory– Edit the configuration file etc\

crown_regionswitch_service\RegionSwitchService.conf



• Install GIMS Service– Unzip the file crown_gims_service_2.5.zip– Copy the gar file to %GLOBUS_LOCATION%\auto-

deploy directory– The Node Server will auto-deploy the service– Stop the Node Server– Configure the GIMS Service

• Import the DB table using etc\crown_gims_service\mysql\gims.sql

• Edit file etc\crown_gims_service\GIMSService.conf



• Install RLDS Service– Unzip the file crown_rlds_service_2.5.zip– Copy the gar to the %GLOBUS_LOCATION%/auto-

deploy directory– The Node Server will auto-deploy the service– Stop the Node Server– Configure the RLDS Service

• Import the DB table using etc\crown_rlds_service\mysql\rlds.sql

• Edit etc\crown_rlds_service\RLDSService.conf


How to setup a basic CROWN grid environment Install Schedule Service

• Install Schedule Service– Unzip the file crown_schedule_service_2.5.zip– Copy the gar file to %GLOBUS_LOCATION%\auto-

deploy directory– The Node Server will auto-deploy the service– Stop the Node Server– Configure the Schedule Service

• Import the DB table using etc/crown_schedule_service/CROWN_Scheduler.sql

• Edit the configure file etc/crown_schedule_service/schedule-config.xml


How to setup a basic CROWN grid environment

• Verify the Installation of CROWN– Fetch the region list of the test environment

• bin\crown-regionregistry-get-allregions.bat

– Query the topology of the RLDS• bin\crown-rlds-get-topology.bat

– Get the information model of certain region• bin\crown-gims-show-ims.bat

– Submit job using schedule client• bin\crown-schedule-client.bat


Agenda

• CROWN Overview

• CROWN User Environments– How to setup a basic CROWN grid environment– How to develop and deploy a CROWN service



How to develop and deploy a CROWN serviceUser Case Scenario

node Dnode B

node E

node A

RLDS

NodeServer

RegionSwitch

node A

node B node C

node Cs Deploy

Query

Undeploy


How to develop and deploy a CROWN service

• Install CROWN Designer• Use CROWN Designer to develop a Service• Query service information from RLDS• Deploy the service to the environment• Undeploy the service from the environment


How to develop and deploy a CROWN service Install CROWN Designer

• Install CROWN Designer– Install the Eclipse SDK 3.1 for win32– Unzip the crown_designer_2.5.zip– Copy the directory cn.org.crown.designer2.5 to the

plugins directory in where Eclipse installed– Launch the Eclipse Software

• Select a proper workspace directory

– Verify the installation


How to develop and deploy a CROWN service Install CROWN Designer

• How to verify the installation– From the menu Help -> About Eclipse SDK -> Plug-in

Details


How to develop and deploy a CROWN service Use CROWN Designer to develop a Service

• Use CROWN Designer to develop a Service– Create a CROWN Project from menu File -> New ->

Project



• The Project directory structure



• Create the Java implementation class for the service– Right click context menu New -> Class



• Create the WSDL/WSDD files for the service– Right click context menu CROWNDesigner -> New

Service



• Packaging the Service– Right click context menu CROWNDesigner->Make Gar– A Gar file will be generated


How to develop and deploy a CROWN service Query service information from RLDS

• Query service information from RLDS– Open a CROWN Explorer View

• Configure the Region Registry IP Address and Port

– Get the topology of the environment– Query the services information of the Node Server


How to develop and deploy a CROWN service Deploy/Undeploy the service to the environment

• Deploy and Undeploy the Service– Deploy – Drag and Drop to the Node Server– Undeploy – Right Click on the Service


Agenda

• CROWN Overview

• CROWN User Environments– How to setup a basic CROWN grid environment– How to develop and deploy a CROWN service

• CROWN System Administration– How to secure a CROWN service


How to secure a CROWN service - Service Security

• Service Security Type– X.509 Signature & Encryption– Authorize by user’s identity– Authorize by user’s IP address– …

• GUI Interface for Security Client– CROWN Launcher

• Sample Service used in this scenario– GetDeployedGars operation of CROWN remote

deploy service


How to secure a CROWN service Directory of configuration

• x.509– X.509 Signature & Encryption

• authz.id– Authorize by user’s identity

• authz.ip– Authorize by user’s IP address

• Client– Client side configuration file


How to secure a CROWN service Configuration file

• service-cert.pem– X.509 certificate of service

• service-key.pem– RSA private key of service

• security-config.xml– Security configuration of service

• auth.properties– Authentication parameter

• trustedCa.store– Trusted ca certificate keystore

• xacml.xml– XACML Access control policy

• attribute.xml– XAML attribute file


How to secure a CROWN serviceImport the Client cert and key


How to secure a CROWN serviceImport the Client cert and key

• Add user’s credential– Choice credential type– Specify the location of certificate Path– Specify the path of private key


How to secure a CROWN service X509

• Service side– Copy files in x.509 dir to

%GLOBUS_LOCATION%/etc/crown_remote_deploy• Client side

– Copy files in client to $HOME/.globus/– or add client user & key to Launcher

• Run– Without X.509 signature, the access will be denied– Using X.509 Signature, the access will be allowed


How to secure a CROWN service X509


How to secure a CROWN service Authorize by user’s Identity


%GLOBUS_LOCATION%/etc/crown_remote_deploy• Client side

– Copy files in client to $HOME/.globus/– or add client user & key to Launcher

• Run– According to XACML security policy, the access of user

deploy_user will be denied– After the modification, the access of deploy_user is allowed


How to secure a CROWN service Authorize by user’s Identity


How to secure a CROWN service Authorize by user’s IP address


%GLOBUS_LOCATION%/etc/crown_remote_deploy

• Client side– Copy files in client to $HOME/.globus/– or add client user & key to Launcher

• Run– According to XACML security policy, the access of certain IP

address will be denied– After the modification, the access of certain IP address is allowed


How to secure a CROWN service Authorize by user’s IP address


Conclusion

• CROWN is a middleware suite and a testbed for China e-Science users

• This tutorial shows– CROWN provides the function of resource

organization and management– CROWN provides GUI IDE for developer– CROWN provides easy to use security

configuration & security interoperation• CROWN Portal English Version

– http://www.crown.org.cn/en


Any Questions?

crown grid tutorial qin li [email protected] beihang university

Documents

eu project

crown v1

europe crown rlds rlds

europe crown workflow

vce project crown

europe crown partners

crown pre

omiichina project