web & grid services track summary coordinators: brian wilson, paul davis contributors: mike...

Web & Grid Services Track Summary

Coordinators: Brian Wilson, Paul Davis

Contributors: Mike Smorul, Gary Jackson, Rudy Husar,Heather Weir, Michael Burnett, Yonsook Enloe, and many others

Web & Grid Services Track

Wilson, 01/04/2005 2ESIP Federation Meeting, Jan. 4-6, 2005

Carbon Cycle We’ll Answer These Provocative Questions:

Is SOAP the universal "glue" for distributed scientific data processing? The IT world seems to think so.

Will Web and Grid services ever converge (as in OGSI and WSRF)? Yes.

What exactly can one do with a Data Grid? Plenty.

Can I really use the Globus Toolkit to find and use compute resources on the Grid? Yes.

How do I do distributed computing and still sleep at night? (i.e., with security, authentication, authorization, single sign-on, reliability, traceability, data provenance, load balancing, etc.)

Promo



WS Summary

• SOAP, WS, and Service Collaborations– Significant interest in and use of Simple Object

Access Protocol (SOAP) for accessing services and WSDL for describing them.

– Significant work has already been done on creating a taxonomy of services and a service registry.

• GCMD SERF documents & services catalog• ECHO service taxonomy & UDDI registry

– Promote service collaborations (service chains) within the Federation by developing pilot projects.

– WSRF and WS-* Standards are still evolving.



WS Recommendations• Use the GCMD SERF catalog for Federation service

registry & discovery.– As opposed to creating an independent Federation survey of

available services.– Many ESIP’s have already written SERF documents.– Modify/add XML tags to the SERF document to support URL

links to WSDL and XML schema docs.– Promote the GCMD SERF Activity

• Have SERF Fest at the next ESIP Federation Meeting.

• Publish “TOP 20” list of services discovered and called via the GCMD services catalog.

– Coordinators: Comments/suggestions to Heather Weir at GCMD and/or Brian Wilson.



WS Recommendations

• Create a “test bed” for registry and discovery of Federation SOAP/WSDL services.– ECHO will support a UDDI registry in v7.– Mature our understanding of shared service

descriptions.– Promote interoperability and links between SERF

documents, service taxonomies, and UDDI registries.

– Coordinator: Michael Burnett.



WS Recommendations• Start a Federation “Service Collaboration” Cluster to create a

Federation Service-Oriented Architecture (SOA) built on contemporary Web Service technologies (SOAP/WSDL).– Create service collaborations (chains) within the Federation.– Develop a taxonomy of services (data discovery, query,

access, etc.) for SOAP purposes (leveraging work by ECHO and others).

– Use the taxonomy to design a standard set of Federation SOAP service interfaces.

– Explore the problems of tying together independently developed services with varying interfaces (semantic mediation).

– Demonstrate interoperability between OpenDAP, OGC WMS/WCS and SOAP services.

– Organizer: Brian Wilson.



Grid Services Summary• Data Grid

– Storage Resource Broker (SRB) is stable and usable.– A SRB-based Federation Data Grid exists (3 site consortium

led by U. Maryland)– Globus also provides “Data Grid” services (MCS, RLS, RFT).

• Grid Services– WS-* Standards are still evolving.– Globus Alliance Toolkit is evolving rapidly.

• Globus v2.4 in production use, but not WS based.

• Globus v3.2 works (OGSI spec.) but has been superseded.

• Globus v4 (WSRF spec.) is not available yet (beta, Mar 2005).

• Further technology exploration (testbed activity) will be required.



Grid Services Recommendations

• Expand participation in the SRB-based Federation Data Grid – Write a “white paper” describing the benefits of

joining the Fed. Data Grid (WHY-TO doc).– Create a HOW-TO document for installing SRB

and joining the Fed. Data Grid.– Review and report back about advances in Globus

services & maturity (v4).– Presentation, with a demo, at the next ESIP

Federation Meeting.– Coordinator: Paul Davis.



Grid Services Recommendations

• Investigate WS Choreography / Grid Workflow technologies.– One activity within the new Services Collaboration

cluster.– Join Grid Workflow activities & discussions at

Global Grid Forum (GGF) meetings.• NASA & Federation input to GGF world.• Earth Science is a new application domain for

GGF workflow efforts.– Coordinators: Brian Wilson, Yonsook Enloe.



Data Query/Access Services Services Chain goes beyond simple data access

– Time & Geolocation Query yielding data inventory– Query on Quality Flags– General Metadata Query catalog satisfying conditions– Data Access - by object or granule ID and variable name– Data Slicing - as in OpenDAP– Data Subsetting – by lat, lon, alt, & time ranges– Parameter Subsetting – select only desired physical variables– Data Reformatting – choose output format as in WMS/WCS– On-Demand Grid Computations – grid diff in GraDS/DODS– Variable Bundles – external metadata from scientist– Return Composite Data Objects

• 4DRectilinearGrid, 4DCurvilinearGrid (swath)• XML & Native Binary representations

– User Composites – add custom object to data model– Semantic Support

• Use generic variable names• Reason about variables to be bundled



SciFlo Data Query/Access (SOAP) Services

• QueryByGeoLocation(startTime, endTime, lat, lon, timeTolerance, distanceTolerace, variable, metadataGroups)– Returns granule ID’s and geolocation info. for AIRS L2 swaths that

intersect lat/lon point or are near enough.

• QueryByGeoRegion(startTime, endTime, lat/lon region, . . .)– Returns granule ID’s and geolocation info. for AIRS L2 swaths that

intersect the lat/lon region or are near enough.

• QueryByMetadata(variable, ListOfConstraints, groups)– Returns granule ID’s and selected metadata for AIRS granules that

satisfy the metadata constraint expression:(min1 <= field1 <= max1 and/or min2 <= field2 <= max2 . . .).

• FindDataById(IdList, UrlOptions)– Given list of unique ID’s, returns list of ftp, http, or DODS URL’s

pointing to the granules (files). Uses cache and redirection server.

• -- Other semantic interfaces possible



SciFlo Data Query/Access (SOAP) Services

• GetMetadata(type, groups)– Returns metadata describing scientific domain, dataset info, list of

metadata fields, generic name translation table, location of XML schema documents, location of related ontologies (OWL/RDF), etc.

• GetHelp(type, groups)– A SOAP service that itself returns help documentation describing

the available SOAP services.

• --These Services Combined Can Support:– Data Catalog, General Query, Data Access, Generic Names,– Data Discovery by Domain and Dataset Keywords– Type checking via XML schemas– Hooks to semantic web



Data Discovery to Inventory Level• SOAP Services just described provide:

– Domain description Metadata

– Dataset Info. and Keywords

– Generic VariableName Lookup Table

– Query by Metadata or just Time & GeoRegion

• Layer more semantics on top of these services– Dataset Discovery by keyword search

– Bind in additional metadata provided by scientist

– Tie ontologies into keywords and genericVariableNames

– Semantic Inference: AIRS and MODIS both provide atmosphericTemperature and cloudProperties. Hmmm. Compare them.

– XML/SOAP provides substrate; possibilities only limited by ontology development.

web & grid services track summary coordinators: brian wilson, paul davis contributors: mike...

Documents