Grid Standards & Grid Standards & Forums, SummaryForums, Summary

Sathish VadhiyarSathish Vadhiyar

Grid Standardization Efforts - Grid Standardization Efforts - OGSAOGSA

Web Service - SOAP, WSDL and UDDIWeb Service - SOAP, WSDL and UDDI

Open Grid Service Architecture (OGSA) – Open Grid Service Architecture (OGSA) – standard developed in 2002 by Globus standard developed in 2002 by Globus AllianceAlliance

A service oriented architecture – all A service oriented architecture – all entities are services and any operation is entities are services and any operation is performed due to message exchangeperformed due to message exchange

OGSA builds on web services – aligned OGSA builds on web services – aligned with industry-accepted web service with industry-accepted web service standardsstandards

Grid Standardization Efforts - Grid Standardization Efforts - OGSAOGSA

While web services can be permanent, many of While web services can be permanent, many of Grid components can be transient servicesGrid components can be transient services

Web service standards do not address issues Web service standards do not address issues related to basic service semantics: how services related to basic service semantics: how services are created, how long they live, how to manage are created, how long they live, how to manage faults etc.faults etc.

These are addressed by a core set of standard These are addressed by a core set of standard interfaces – initially OGSI (infrastructure), now interfaces – initially OGSI (infrastructure), now WSRF WSRF

Initially (July 2003) OGSA capabilities Initially (July 2003) OGSA capabilities implemented by OGSIimplemented by OGSI

ComponentsComponents

OGSI defines standard interfaces forOGSI defines standard interfaces for Describing and discovering service attributesDescribing and discovering service attributes Creating service instancesCreating service instances Managing service lifetimeManaging service lifetime NamingNaming fault modelfault model Service groupsService groups

Other issues addressed by OGSA servicesOther issues addressed by OGSA services AuthenticationAuthentication Monitoring service-level agreementsMonitoring service-level agreements Managing membershipsManaging memberships Integrating data resources into componentsIntegrating data resources into components

OGSA schemaOGSA schema For providing interoperability between componentsFor providing interoperability between components For describing properties of common Grid entitiesFor describing properties of common Grid entities

Core Components of OGSACore Components of OGSA

Web Services

Open Grid Service Infrastructure

OGSA services: registry,authorization, monitoring, data

access, management etc.

More specialized and domainspecific services

Hosting environment Protocol

OtherSchemas

OGSASchemas

OGSA Services – Some More OGSA Services – Some More DetailsDetails

Core ServicesCore Services SecuritySecurity PolicyPolicy Messaging, queuing and loggingMessaging, queuing and logging EventsEvents Metering and accountingMetering and accounting

Data and information servicesData and information services Data naming and accessData naming and access ReplicationReplication Metadata and provenanceMetadata and provenance

Resource and Service managementResource and Service management Administration and deploymentAdministration and deployment Service orchestrationService orchestration

WSRFWSRF

New alternative to OGSI – May 2004 developed New alternative to OGSI – May 2004 developed by OASIS groupby OASIS groupWeb Service Resource FrameworkWeb Service Resource FrameworkStates of web service – data values that persist States of web service – data values that persist across and evolve because of web service across and evolve because of web service interactionsinteractionsWSRF – representing relationships between WSRF – representing relationships between services and standards in an explicit and a services and standards in an explicit and a standard mannerstandard mannerStateless web service acting on stateful Stateless web service acting on stateful resourcesresources

More on WSRFMore on WSRF

Standardizing how state is represented and Standardizing how state is represented and manipulatedmanipulated

Answers questionsAnswers questions How WS-Resource – (web service + stateful How WS-Resource – (web service + stateful

resource) is composed?resource) is composed? How stateful resource is used in the execution of web How stateful resource is used in the execution of web

service message exchanges?service message exchanges? How WS-resources can be created and destroyed?How WS-resources can be created and destroyed? How to enable querying of WS-Resource via web How to enable querying of WS-Resource via web

service message exchanges?service message exchanges?

OGSI -> WSRFOGSI -> WSRF

OGSI did not work well with existing web OGSI did not work well with existing web servicesservicesOGSI models stateful resource as web OGSI models stateful resource as web service coupled with resource state.service coupled with resource state.WSRF separates service and stateful WSRF separates service and stateful entities acted upon by the service – WS entities acted upon by the service – WS Resource – better exploits existing XML Resource – better exploits existing XML standards and emerging web service standards and emerging web service standardsstandards

Illustration of Complexity – remote Illustration of Complexity – remote job execution with WSRFjob execution with WSRF

Global (Open) Grid ForumGlobal (Open) Grid Forum

Community-initiated forum of thousands of Community-initiated forum of thousands of individuals from industry and research leading individuals from industry and research leading the global standardization effort for grid the global standardization effort for grid computing.computing.Creation and documentation of "best practices" - Creation and documentation of "best practices" - technical specifications, user experiences, and technical specifications, user experiences, and implementation guidelines.implementation guidelines.Meets 3 times per year – started in March 2001 Meets 3 times per year – started in March 2001 – 15 GGFs have been held– 15 GGFs have been heldGGF is responsible for OGSAGGF is responsible for OGSA

GGF Areas and GroupsGGF Areas and Groups

9 different Areas9 different Areas2 kinds of groups – working groups and research 2 kinds of groups – working groups and research groupsgroupsApplications and Programming Models Applications and Programming Models Environment (APME)Environment (APME) GridCPR-WG, GridRPC-WG, Astro-RG etc.GridCPR-WG, GridRPC-WG, Astro-RG etc.

Architecture (ARCH)Architecture (ARCH) OGSA-WG, OGSI-WG, SEM-RG etc.OGSA-WG, OGSI-WG, SEM-RG etc.

Data (DATA)Data (DATA) DAIS-WG, GridFTP-WG, TM-RGDAIS-WG, GridFTP-WG, TM-RG

GGF Areas and GroupsGGF Areas and Groups

Grid Security (GRID SEC)Grid Security (GRID SEC) OGSA AUTHZ WGOGSA AUTHZ WG

Information Systems and Performance (ISP)Information Systems and Performance (ISP) GIR-WG, GB-RGGIR-WG, GB-RG

Peer-to-Peer (P2P)Peer-to-Peer (P2P) OGSA P2P-RGOGSA P2P-RG

Scheduling and Resource Management (SRM)Scheduling and Resource Management (SRM) DRMAA-WG, GESA-WG, JSDL-WG, GSA-RG, WFM-DRMAA-WG, GESA-WG, JSDL-WG, GSA-RG, WFM-

RGRG

GGF Market Awareness Committee (GMAC)GGF Market Awareness Committee (GMAC)Grid Research Oversight Committee (GROC)Grid Research Oversight Committee (GROC)

So, what is left in Grid?So, what is left in Grid?

Resource and Resource and Process Process ManagementManagement

This is the first area addressed by many projectsThis is the first area addressed by many projects

Addressed by Condor, Globus, Legion, NetSolve and Addressed by Condor, Globus, Legion, NetSolve and almost all Grid projectsalmost all Grid projects

Not much left in this areaNot much left in this area

One interesting area left to be addressed is how to One interesting area left to be addressed is how to combine batch and interactive systems for a single jobcombine batch and interactive systems for a single job

Information Information ServicesServices

Projects like NWS and Globus MDS address it with Projects like NWS and Globus MDS address it with LDAP mechanismsLDAP mechanisms

Number of projects on how to represent information Number of projects on how to represent information etc. – This area is loosing steam nowadaysetc. – This area is loosing steam nowadays

Questions regarding how to get the “very recent” Questions regarding how to get the “very recent” information and scalability still remaininformation and scalability still remain

How to get forecast for a certain period of time in the How to get forecast for a certain period of time in the future?future?

Scheduling – Scheduling – Problem mappingProblem mapping

Has been there for many many years – Number Has been there for many many years – Number of projects both within and outside Grid for of projects both within and outside Grid for mapping to heterogeneous resources – Condor, mapping to heterogeneous resources – Condor, AppLeS, NetSolve, APST and many othersAppLeS, NetSolve, APST and many others

Still an open problemStill an open problem

Many takers for the researchMany takers for the research

Heuristics is the best approach in GridsHeuristics is the best approach in Grids

ReschedulingRescheduling People agree on general strategies – Condor, People agree on general strategies – Condor, LSF, Dome, GrADSolve and other recent onesLSF, Dome, GrADSolve and other recent ones

The main area of interest is how to determine The main area of interest is how to determine various thresholds automaticallyvarious thresholds automatically

Lots of research need to be done regarding how Lots of research need to be done regarding how to orchestrate communications during reschedulingto orchestrate communications during rescheduling

Meta schedulingMeta scheduling Projects like Globus do deal with multiple local Projects like Globus do deal with multiple local scheduling mechanisms for a single jobscheduling mechanisms for a single job

Different meanings attached to the termDifferent meanings attached to the term

Data SchedulingData Scheduling The coscheduling of data and computation has The coscheduling of data and computation has been there for a long time and many more been there for a long time and many more methods will evolvemethods will evolve

Data replication, automatic choice of replicas, Data replication, automatic choice of replicas, collating different replica segments for a single collating different replica segments for a single problem – much more work to be done, Globus problem – much more work to be done, Globus provides only basic methodsprovides only basic methods

Performance Performance Modeling and Modeling and Prediction of Prediction of applicationsapplications

Has been there for a long time much before Has been there for a long time much before GridsGrids

Renewed interest due to running applications on Renewed interest due to running applications on Grid – AppLeS, GrADS etc.Grid – AppLeS, GrADS etc.

A very open area – any solution is not a good A very open area – any solution is not a good solutionsolution

Coupling performance prediction with scheduling Coupling performance prediction with scheduling mechanisms and improving over time – a very mechanisms and improving over time – a very interesting solutioninteresting solution

Latency tolerant Latency tolerant algorithmsalgorithms

Very few isolated projectsVery few isolated projects

Need automated techniques to translate a tightly Need automated techniques to translate a tightly coupled parallel application to a latency tolerant coupled parallel application to a latency tolerant codecode

Crying for helpCrying for help

Application Application Oriented GridsOriented Grids

Many isolated projects – AstroGrid, ClimateGrid, Many isolated projects – AstroGrid, ClimateGrid, GriPhyN etc.GriPhyN etc.

Can be many more for many other applicationsCan be many more for many other applications

Thrust should be on developing middleware Thrust should be on developing middleware layers for specific application classeslayers for specific application classes

Current focusCurrent focus

Simulation, Simulation, EmulationEmulation

Good starts by few projects – MicroGrid, Good starts by few projects – MicroGrid, SimGridm GridSimSimGridm GridSim

Need a standard and uniform simulation and Need a standard and uniform simulation and emulation frameworkemulation framework

A very important area in the coming daysA very important area in the coming days

Checkpointing, Checkpointing, fault tolerance and fault tolerance and migrationmigration

Many projects regarding checkpointing – Many projects regarding checkpointing – CoCheck, MPICH-v, libckpt, SRSCoCheck, MPICH-v, libckpt, SRS

May be some active research in checkpointing May be some active research in checkpointing when dealing with reconfigurable applications and when dealing with reconfigurable applications and algorithm adaptivityalgorithm adaptivity

Fault tolerance is assuming more and more Fault tolerance is assuming more and more importance.importance.

Many things have not been addressed including Many things have not been addressed including the main one – modeling and predicting faultsthe main one – modeling and predicting faults

Economy modelsEconomy models Nimrod-G, G-commerceNimrod-G, G-commerce

In its nascent stagesIn its nascent stages

There is a tacit understanding that it is important. There is a tacit understanding that it is important. But not sure if we need it now and how critical it is But not sure if we need it now and how critical it is going to begoing to be

Has started materializing, finallyHas started materializing, finally

ScheduleSchedule

Project Presentation and Report Project Presentation and Report Submission – December 18, Tuesday, Submission – December 18, Tuesday, 11:00 A.M., SERC 202.11:00 A.M., SERC 202.

Sessional Marks soonSessional Marks soon

Areas Not CoveredAreas Not CoveredGrid RPCGrid RPC

NinfNinf

Grid SecurityGrid Security SASLSASL

Gang scheduling, Coscheduling, LSF schedulingGang scheduling, Coscheduling, LSF schedulingApplication Oriented Grid FrameworksApplication Oriented Grid Frameworks

Scientific data federationScientific data federation world wide telescopeworld wide telescope medical data federationmedical data federation bioinformatics research networkbioinformatics research network

Grid ToolsGrid Tools AccessgridAccessgrid instrumentation and monitoringinstrumentation and monitoring

Network infrastructure, Production deploymentNetwork infrastructure, Production deploymentGrid economyGrid economy

Nimrod-GNimrod-G

Peer-to-Peer Grid ServicesPeer-to-Peer Grid Services Lessons can be learned in Grid regarding massive scalability, querying distributed Lessons can be learned in Grid regarding massive scalability, querying distributed

informationinformation

Semantic GridSemantic Grid Chimera-like, NVO-like approaches at a much bigger levelChimera-like, NVO-like approaches at a much bigger level

Current Working Grids, National InitiativesCurrent Working Grids, National Initiatives TeraGrid, EuroGrid, e-science etc.TeraGrid, EuroGrid, e-science etc.

Focus in this class has been more on concepts rather than on information dumping !

Junk!Junk!

Manages lifetimesManages lifetimes

Defines Grid Services – interactions with Defines Grid Services – interactions with clientsclients

Helped in managing long-lived states of Helped in managing long-lived states of resourcesresources

Extended WSDL and XML schemaExtended WSDL and XML schema

Grid Service Handle and Grid Service Grid Service Handle and Grid Service Reference to address Grid ServicesReference to address Grid Services

WSRFWSRF

Partitioning OGSI into a family of separate Partitioning OGSI into a family of separate specifications that allow for flexible specifications that allow for flexible compositionscompositions

To query about the servicesTo query about the services

Asynchronous notification of changes in Asynchronous notification of changes in the states of the resourcesthe states of the resources