biodiversityworld grid workshop nesc, edinburgh – 30 june and 1 july 2005 resource wrappers, web...
TRANSCRIPT
BiodiversityWorld GRID WorkshopNeSC, Edinburgh – 30 June and 1 July 2005
Resource wrappers, web services, grid services
Jaspreet Singh School of Computer Science
Cardiff University
BiodiversityWorld GRID WorkshopNeSC, Edinburgh – 30 June and 1 July 2005
Overview
• BDWorld Project• Resource Wrappers• BDWorld Datatypes• BiodiversityWorld-GRID Interface (BGI)
Communications Layer• Web Services based BDW Wrappers• Globus Toolkit 4.0 & WSRF• Grid Services based BDW Wrappers• Further Work
BiodiversityWorld GRID WorkshopNeSC, Edinburgh – 30 June and 1 July 2005
BDWorld Project
• A flexible and extensible problem solving environment (PSE)• Brings together heterogeneous, globally distributed, biodiversity related
resources and analytical tools• The resources can be assembled into set of workflows to perform
complex scientific analysis• Provides consistent mechanisms across various facets to achieve
interoperability• The multi-layered architecture provides data related computation as part
of a separate layer • Provides uniform interfaces for heterogeneous resources• Provides a common resource access mechanism• Standard data flow mechanisms that use well defined data objects• Uses Triana Workflow System for assembling and executing workflows
BiodiversityWorld GRID WorkshopNeSC, Edinburgh – 30 June and 1 July 2005
BDWorld Wrappers
• A mechanism to provide consistent interface to resource users and a standard resource access/invocation mechanism
• Insulate the core BDWorld System from heterogeneous resource• Retains flexibility to use various operations supported by each resource• Provides a generic access mechanism• Wraps various kinds of resources and analytical tools• Can be deployed in Web Services/Grid environment• Provides interoperability mechanisms between client and heterogeneous
resources• Operations on remote resources is invoked via the invokeOperation()
method implemented by all the wrappers• Wrappers give consistent form to data retrieved from heterogeneous
resources by encapsulating them into BDWorld data types
BiodiversityWorld GRID WorkshopNeSC, Edinburgh – 30 June and 1 July 2005
InvokeOperation() method
• The parameters of invokeOperation()method identify:theResourceHandle: name of the resource to be invoked
theOperationHandle: name of the operation to be invoked on the remote resource/tool
theDataCollection is input data collection for the remote resource
BiodiversityWorld GRID WorkshopNeSC, Edinburgh – 30 June and 1 July 2005
Triana Workflow System
Wrapper Logic
Data TransformationConfiguration Tools
Data Transformation Tools
BDW Wrapper
Standard Operations
External Tools
BGI Communications Layer
BDW Wrapper Architecture
Remote Resource
Datatype Tools
Abstract Wrapper
BiodiversityWorld GRID WorkshopNeSC, Edinburgh – 30 June and 1 July 2005
BDWorld Datatypes
• Encapsulate different types of data and sub-datatypes for transporting data between end points
• Datatypes can be transformed into xml representations which can be easily serialised
• Xml schema for datatypes conform to W3C standards• Datatypes are flexible enough to encapsulate user defined
xml documents or data in string representation• They also act as parsing tool to extract data from raw xml• Higher level datatypes can hold several sub datatypes and
each sub datatype can hold several pieces of data• They are extensible and support incorporation of new
datatypes.
BiodiversityWorld GRID WorkshopNeSC, Edinburgh – 30 June and 1 July 2005
BDWorld-GRID Interface (BGI) Layer
• Provides standard mechanisms for invoking operations on heterogeneous resources
• provides seamless access to BDW resources via resource wrappers• Uses Xml/SOAP messaging system for invoking operations on resource
wrappers• Act as an integrated mechanism for accessing all resource wrappers• Isolates resources/resource wrapper implementation to enable use of
web services/grid technologies as part of a separate layer• Isolates users from grid/web service complexities• Helper Tool is provided to the user/triana for using the BGI layer• Can provide multiple users/workflows with access to multiple resources at
a time• Interoperable with other e-Science projects
BiodiversityWorld GRID WorkshopNeSC, Edinburgh – 30 June and 1 July 2005
BDW Web Services Architecture
• Web services is a mechanism of enabling distribute computing based on open standards
• Wrappers are currently deployed in a web service environment which can be accessed via the BGI Layer at servers in Cardiff and Reading
• Web services enabled wrappers can be invoked via the BGI Helper Tool
• Axis SOAP engine provides the WSDL that exposes wrapper operations to outside world
• The MetadataAgent WS will provide access to MDR via the BGI Layer
BiodiversityWorld GRID WorkshopNeSC, Edinburgh – 30 June and 1 July 2005
Unit A
Unit B
Unit C
Triana Workflow Units
Data Parsing Tools
BGI Communications Layer
BGI Helper Tool
Tomcat Servlet Container/ Axis Soap Engine
MDR
Metadata Agent WS
BDW Web Services Architecture
Remote ResourceExternal Tool
Java /non java resource
WS Interface
BDW Wrapper
WS Interface
BDW Wrapper
WS Interface
BDW Wrapper
BiodiversityWorld GRID WorkshopNeSC, Edinburgh – 30 June and 1 July 2005
Drawbacks of web services
• Web services are not stateful
• WS provide mechanisms for invoking remote operations but does not provide other services such as resource management, persistency, life cycle management, notification etc.
• Each web service needs to be deployed individually
BiodiversityWorld GRID WorkshopNeSC, Edinburgh – 30 June and 1 July 2005
GT4 Key Concepts
• Based on Open Grid Service Architecture (OGSA)• OGSA defines common, standard and open
architecture for grid-based applications• Standardises various services common to grid
applications (job management, resource monitoring and discovery, resource management, security services etc)
• Uses Web Services as underlying technology to enable distributed computing
• But Web Services are not stateful
BiodiversityWorld GRID WorkshopNeSC, Edinburgh – 30 June and 1 July 2005
Web Service Resource and Web Services Resource Framework (WSRF)
• WS-Resource construct expresses relationship between stateful resources and Web Services. It is pairing of web service with a resource
• WSRF is a set of web services specifications which define rendering WS-Resource approach in terms of specific message exchanges and related XML definitions
• It specifies how Web Services can be made stateful• It defines conventions for managing state for applications to discover,
inspect and interact with stateful resources in standard and interoperable way
• Using WSRF associations between a web service and one or more stateful resources can be established
• The state of the WS-Resource can be accessed through a web service interface by using endpoint references
• WSRF is the core of GT4
BiodiversityWorld GRID WorkshopNeSC, Edinburgh – 30 June and 1 July 2005
WSRF Specifications
• WS-ResourceProperties• WS-ResourceLifetime• WS-RenewableReferences• WS-ServiceGroup• WS-BaseFaultsRelated Specifications• WS-Notification• WS-Addressing
BiodiversityWorld GRID WorkshopNeSC, Edinburgh – 30 June and 1 July 2005
Applications
OGSA
Web ServicesWSRF
Stateful Web Services
Web Service
extends
OGSA WSRF
requires specifies
Relation between OGSA, WSRF and GT4 (source: [1])
BiodiversityWorld GRID WorkshopNeSC, Edinburgh – 30 June and 1 July 2005
WSRF – An approach to statefulness
• WSRF provides the mechanism to keep state information by keeping the Web Service and state information completely separate
• State information is stored in entity called resource• A resource is be identified via its unique key• When requiring stateful interaction, a web service
can be instructed to use a particular resource• The resources can be stored in memory or on
secondary storage
BiodiversityWorld GRID WorkshopNeSC, Edinburgh – 30 June and 1 July 2005
Client Web Service
A
B
C
Resources
Interaction of client with a stateful web service (source: [1])
Instruct client to use Resource A
request
response
Web service loads resource A
WS-Resource
Client provides EPR corresponding to Resource A
BiodiversityWorld GRID WorkshopNeSC, Edinburgh – 30 June and 1 July 2005
Grid enabled BDW Resource Wrappers
• Java WS Core component of GT4 being used• Prototype implementation of resource wrappers deployed in
the grid environment• Interaction of Grid enabled wrappers via the BGI Helper Tool• Single integrated mechanism of invoking all resource
wrappers in the grid environment• Grid services invoked via the dynamic loading of resource
wrappers • Resource properties provide access to memory resident
BDW cache data• Access to cache data via EPR from Triana/client without
invoking the resource behind again
BiodiversityWorld GRID WorkshopNeSC, Edinburgh – 30 June and 1 July 2005
Unit A
Unit B
Unit C
Triana Workflow Units
Data Parsing Tools
BGI Communications Layer
BGI Grid Helper Tool
BgiWrapperService
BgiWrapper BgiWrapperBgiWrapper
BDWCache BDWCache BDWCacheGT4
Container
MDR
Metadata Agent WS
Remote Resource
BiodiversityWorld GRID WorkshopNeSC, Edinburgh – 30 June and 1 July 2005
Further Work
• Move computation on the grid in addition to resource access mechanism
• BDW Resource pool, automatic update of resources
• Making resources persistent• BDW Heartbeat monitor• Security, Index Service, Monitoring and
Discovery System (MDS), OGSA-DAI for BDW resources
BiodiversityWorld GRID WorkshopNeSC, Edinburgh – 30 June and 1 July 2005
References
[1] Globus Tutorial http://gdp.globus.org/gt4-tutorial/
[2] Global Grid Forum http://www.ggf.org/ggf_abt_overview.htm
[3]The WS-Resource Framework http://www.globus.org/wsrf/specs/ws-wsrf.pdf