vision of unicore – unigrids and beyond
DESCRIPTION
Vision of UNICORE – UniGrids and beyond. Daniel Mallmann [email protected] 2 nd CoreGRID Summer School Bonn, Germany 24 th – 28 th July 2006. Objectives. Develop Unicore/GS Based on Unicore and the Open Grid Services Architecture Compliant with the Web Services Resource Framework - PowerPoint PPT PresentationTRANSCRIPT
Vision of UNICORE – UniGrids and beyond
Daniel Mallmann [email protected]
2nd CoreGRID Summer SchoolBonn, Germany
24th – 28th July 2006
2
Objectives
Develop Unicore/GS
Based on Unicore and the Open Grid Services Architecture
Compliant with the Web Services Resource Framework
Develop generic software components
Visualisation and steering of simulations
Device monitoring and control
Access to distributed data and databases
Validate UniGrids concepts and demonstrate the application of Unicore/GS in Industrial and scientific domains
3
Duration, Funding
6th Framework Programme of the European Commission
2nd Call of the IST 2003-2004 Workprogramme♦ Objective: GRID-based systems and solving complex problems
Instrument: STReP
Duration: 24 months
Start: July 2004
Effort: 297 PM (12 developer + manager)
Costs: 3.0 M€
Grant: 1.9 M€
83
4
European Context
inteliGRIDSemantic Grid based virtual organisations
ProvenanceTrust and provenance
for Grids
DataminingGridDatamining
tools & services
K-WF GridKnowledge based
workflow & collaboration
GRIDCOORDBuilding the ERA in Grid research
OntoGridKnowledge Services for the semantic Grid
HPC4UFault tolerance,dependability
for Grid
SIMDATGrid-based generic enabling application technologies to
facilitate solution of industrial problems
NextGRID EU-driven Grid services architecture for businesS
and industry
AkogrimoMobile Grid architecture
and services for dynamic virtual organisations
CoreGRIDEuropean-wide virtual laboratory for longer term Grid research-creating the
foundation for next generation Grids
Specific support action Integrated project Network of excellence Specific targeted research project
Courtesy: Max Lemke, European Commission
UniGridsExtended OGSA
Implementation based on UNICORE
UniGridsExtended OGSA
Implementation based on UNICORE
5
European Context
SSAsSSAs supporting NESSI and the Grid communitysupporting NESSI and the Grid community
DataminingGrid
data, knowledge, data, knowledge, semanticssemantics
OntoGrid
InteliGridK-WF Grid
KnowArc Chemomentum
A-Ware Sorma
platforms, user platforms, user environmentsenvironments
Specific support action Integrated project Network of excellence Specific targeted research project
CoreGRIDvirtual laboratories
HPC4U
g-Eclipse
Gredia
GridComp
QosCosGrid
Grid4all
Provenance
AssessGridGridTrust
trust, securitytrust, security
Grid services, Grid services, business modelsbusiness models
ArguGrid Edutain @ Grid
GridEconGridCoord Grid@Asia
Nessi -GridChallengers Degree
NextGRIDservice
architecture
Akogrimomobile
services
BREINagents &
semantics
BeinGridbusiness
experiments
SIMDATindustrial
simulations
XtreemOS
ProvenanceProvenance
AssessGridAssessGridGridTrustGridTrust
trust, securitytrust, security
Grid services, Grid services, business modelsbusiness models
ArguGridArguGrid Edutain @ Grid
Edutain@ Grid
GridEconGridEconGridCoordGridCoord Grid@AsiaGrid@Asia
Nessi -GridNessi -GridChallengersChallengers DegreeDegree
NextGRIDservice
architecture
NextGRIDservice
architecture
Akogrimomobile
services
Akogrimomobile
services
BREINagents &
semantics
BREINagents &
semantics
BeinGridbusiness
experiments
SIMDATindustrial
simulations
SIMDATindustrial
simulations
XtreemOSLinux based
Gridoperating
system
BeinGridbusiness
experiments
BeinGridbusiness
experiments
Courtesy: Max Lemke, European Commission
UniGridsUniGrids
6
Partner
Research Centre Jülich(Project manager)
Consorzio Interuniversitarioper il Calcolo Automatico dell’Italia Nord Orientale
Fujitsu Laboratories of Europe
University of Warsaw
Intel GmbH
University of Manchester
T-Systems SfR
7
Technology Foundations
Develop Unicore Services
Job and file management
User authorisation
Dynamic resource discovery and management
Extend security to include explicit trusted delegation
NetworkJob
Supervisor
UnicoreGateway
Unicore Client
Target System A
Target System B
Target System C
ResourceDatabase
ResourceBroker
UserDatabase
8
Hosting Environment
Build on Jetty Server and Axis SOAP engine
WS Resource Framework ♦ ResourceProperties♦ Lifetime♦ BasicProfile♦ ServiceGroup
WS Notification♦ NotificationProducer♦ NotificationConsumer♦ SubscriptionManager
WS Addressing♦ Address URL♦ Resource Disambiguator
Technology Foundations
9
Technology Foundations
Web Service Unicore Gateway
HTTP and HTTPS connector
Support for other protocols
Dispatches incoming SOAP messages based on
♦ WS-Addressing headers or♦ URL information from the HTTP transport layer
Web Service Unicore User Database
Supports different storage formats – default XML file
Interface for secure, remote administration
Single UUDB can be used by multiple services
10
Usite B
Vsite B2Vsite B1
Usite A
Vsite A1
Technology Foundations
Multisite Job
Gateway
InternetInternet
TargetSystem
Interface
NetworkJob
Supervisor
TargetSystem
Interface
TargetSystem
Interface
NetworkJob
Supervisor
NetworkJob
Supervisor
Client
Gateway
11
Secondary Network Job Supervisor
Primary Network Job Supervisor
SSLSSL
Client
= User certificate = NJS certificate
Job
SubJob
Multisite Job
Consigner: entity that consigns a job or sub-job♦ Expressed by use in SSL connection
Endorser: entity that authorises the tasks to be performed♦ Expressed by signing of serialized AJO direct acyclic graph
Technology Foundations
12
Usite A
Vsite A1
Explicit Trust Delegation
Gateway
InternetInternet
TargetSystem
Interface
NetworkJob
Supervisor
Client
Portal
Technology Foundations
13
SSL
Network Job Supervisor
SSL
PortalWS- Client(Browser)
Job
User: name
= User certificate = Portal certificate
Explicit Trust Delegation
User: new role besides consignor and endorser♦ Entity (user) on whose behalf tasks will be performed
Trusted Agents (Portal)♦ Added to the UUDB explicitly♦ Allowed to endorse AJO on behalf of users
Technology Foundations
14
Interoperability
UniGrids Atomic Services
Job Management
Storage Management
File Transfer
Target System
Unicore Globus CGSP
GridClient
GridPortal
TargetSystemService
JobManagement
Service
StorageManagement
Service
FileTransferService
Target System Factory
UniGrids Atomic Services
Standardisation
OGSA-BESBasic Execution Services
OGSA-DMIData Movement Interface
15
Globus Toolkit 4 Hosting Environment
Globus Security
Target System Factory
TSF
Target SystemService
TSS
JobMgmt
ServiceJMS
StorageMgmt
ServiceSMS
FileTransferService
FTS
GridFTP/GASSGRAM/TSI
UniGrids Atomic Service Interfaces
UniGrids Atomic Services for GTK4
Available as Open Source (BSD license) at SourceForge (gpe4gtk.sourceforge.net)
Interoperability
16
Security
Client to server♦ Security provider concept
allows to plug in security model on client side
GTK4 server to Unicore/GS server♦ Extract original user certificate
from proxy certificate chain,pass to UGS (reduced security model)
Unicore/GS server to GTK4 server♦ UGS could use proxy certificate from
ETD-enabled MyProxy server
Generic, standards-based VOMS preferable
GSIETD
GTK4Unicore/GS
Interoperability
17
Expert ClientApplication
Client Portal Client
GB
GB GB
GBGB
GBGB
Clients and GridBeans
GPE Clients work on top of Unicore/GS, GTK4 or ChinaGrid
♦ Only requirement: UniGrids Atomic Services Interfaces
GridBeans are portable without modifications♦ Client API hides underlying infrastructure details
Interoperability
18
Grid Ontology
Cooperation with OntoGrid
Encoded in OWL, now published on the UniGrids web site (http://www.unigrids.org/ontology.html)
Permits reasoning and thus can fill in gaps in manually entered descriptions
Interoperability
19
Grid Economics
Economic Service Infrastructure
Full WSRF-Based Resource Broker♦ Use of extensible Economic terms for resource selection♦ Resource consumption feedback from Job Execution Engines
Resource Accounting System♦ Publication through
WS Resource Properties
Payment System♦ Based on MCS Payment
protocol♦ Uses broker as agent for
user and/or their VO
20
Higher-level Services
Visualization and application steering
VISualization Interface Toolkit (VISIT)♦ Light-weight library that supports
bi-directional data exchange between visualizations and parallel applications
VISIT GridBean♦ Manage participants of visualization
sessions via VISIT / GS family
Framework works with all VISIT - enabled applications
Unique Unicore/GS feature for interactive scenarios
21
Higher-level Services
Unicore/GS Streaming Framework (UGSF)
Components♦ Web service♦ Streaming server♦ Client library
Architecture♦ Streaming Service is a
stand-alone application that performs streaming
♦ Client uses dedicated protocol to connect to the streaming service and obtains real data stream
♦ Web service uses RMI to communicate with streaming service and setup streams
22
Higher-level Services
File transfer service (including data filter)
Exploiting GridFTP
Client – Server transfers
Server – Server transfers♦ Including SRB
Client GridBean available
23
Higher-level Services
Access to distributed data, access to databases
Interface to OGSA-DAI resources♦ WSRF Data services deployed in
Unicore/GS Container♦ OGSADAI GridBean♦ OGSADAI Client
24
Applications
Deploy test beds
Prove UniGrids concepts inindustrial and scientific domains
Oil reservoir modelling
Bio-molecular, genomic and proteomic
Nuclear risk management
Re-engineerindustrial applications using Unicore/GS WSRF approach
25
Standardisation
Influence standardisation process for selected Grid standards
JSDL
WS-RF
OGSA-BES
Promote European research results
Leverage partner participation in standards organisations
SAGA-RG
WS-N
OGSA-RSS
26
Beyond UniGrids
Grid projects exploiting UniGrids results
D-Grid (www.d-grid.de)
DEISA (www.deisa.org)
OMII-Europe (omii-europe.com)
A-WARE (www.a-ware.org)
Chemomentum (www.chemomentum.org)
EGEE-II (www.eu-egee.org)
UNICORE 6 Release Schedule
Alpha release ≡ UniGrids results: July 2006
Beta release: mid 2007
Final release: end 2007
27
A-WARE
Objectives
Simplify usage of Grid♦ Workflow management♦ User focused
workflow orchestration♦ Repository to store workflows
Fill an existing gap ♦ Between middleware and portals
28
Chemomentum
Objectives
Provide an integrated Grid solution for workflow-centric, complex applications;focus on the end users
Provide Grid-enabled applications, data services and knowledge management solutions
♦ Risk assessment♦ Toxicity prediction♦ Drug design
Test-drive developed services in the context of the European regulatory initiative “Registration and Evaluation of Chemicals” (REACH)
♦ Aimed at reducing animal testing, by developing in silico, Grid-based tools
29
OMII Europe
Objectives
Provide key software components for building e-infrastructures
Common interoperable services:♦ Database Access
♦ Virtual Organisation Management
♦ Portal
Infrastructure integration♦ Initial EGEE/UNICORE/Globus interoperability
♦ Interoperable security framework
♦ Accounting
♦ Job Submission
♦ Job Monitoring
30
Summary
UniGrids built Unicore/GS
Based on Unicore and the Open Grid Services Architecture
Compliant with the Web Services Resource Framework
UniGrids achieved interoperability
UniGrids developed generic software components
UniGrids results are the base of UNICORE 6
UniGrids results are exploited by several Grid projects