community grids architecture discussion
DESCRIPTION
Community Grids Architecture Discussion. Internal Discussion August 20 2004 Geoffrey Fox Community Grids Lab Indiana University [email protected]. Philosophy of Web Service Grids. - PowerPoint PPT PresentationTRANSCRIPT
Community Grids Architecture Discussion
Internal Discussion
August 20 2004
Geoffrey Fox
Community Grids Lab
Indiana University
Philosophy of Web Service Grids• Much of Distributed Computing was built by natural
extensions of computing models developed for sequential machines
• This leads to the distributed object (DO) model represented by Java and CORBA– RPC (Remote Procedure Call) or RMI (Remote Method
Invocation) for Java• Key people think this is not a good idea as it scales badly
and ties distributed entities together too tightly– Distributed Objects Replaced by Services
• Note CORBA was considered too complicated in both organization and proposed infrastructure– and Java was considered as “tightly coupled to Sun”– So there were other reasons to discard
• Thus replace distributed objects by services connected by “one-way” messages and not by request-response messages
Web services• Web Services build
loosely-coupled, distributed applications, based on the SOA principles.
• Web Services interact by exchanging messages in SOAP format
• The contracts for the message exchanges that implement those interactions are described via WSDL interfaces.
Databases
Humans
ProgramsComputational resources
Devices
reso
urce
s
BP
EL,
Jav
a, .N
ET
serv
ice
logi
c
<env:Envelope> <env:Header> ... </env:header> <env:Body> ... </env:Body></env:Envelope> m
essa
ge p
roce
ssin
g
SO
AP
and
WS
DL
SOAP messages
Importance of SOAP• SOAP defines a very obvious message structure
with a header and a body• The header contains information used by the
“Internet operating system”– Destination, Source, Routing, Context, Sequence
Number …
• The message body is only used by the application and will never be looked at by “operating system” except to encrypt, compress it etc.
• Much discussion in field revolves around what is in header!– e.g. WSRF adds a lot to header
Web Services• Java is very powerful partly due to its many “frameworks” that
generalize libraries e.g.– Java Media Framework– Java Database Connectivity JDBC
• Web Services have a correspondingly collections of specifications that represent critical features of the distributed operating systems for “Grids of Simple Services”– Some 60 active WS-* specifications for areas such as– a. Core Infrastructure Specifications– b. Service Discovery– c. Security– d. Messaging– e. Notification– f. Workflow and Coordination– g. Characteristics– h. Metadata and State
WS-I Interoperability• Critical underpinning of Grids and Web Services is the
gradually growing set of specifications in the Web Service Interoperability Profiles
• Web Services Interoperability (WS-I) Interoperability Profile 1.0a." http://www.ws-i.org. gives us XSD, WSDL1.1, SOAP1.1, UDDI in basic profile and parts of WS-Security in their first security profile.
• We imagine the “60 Specifications” being checked out and evolved in the cauldron of the real world and occasionally best practice identifies a new specification to be added to WS-I which gradually increases in scope– Note only 4.5 out of 60 specifications have “made it” in this
definition
Web Services Grids and WS-I+• WS-I Interoperability doesn’t cover all the capabilities need to
support Grids• WS-I+ is designed to minimal extension of WS-I to support
“most current” Grids: it adds support for– Enhanced SOAP Addressing (WS-Addressing)– Fault tolerant (reliable) messaging– Workflow as in IBM-Microsoft standard BPEL
• Security and Notification best practice and support will probably get added soon– There are Web Service frameworks here but various IBM v Microsoft v
Globus differences to be resolved
• Portlet-based User Interfaces could be added• UK OMII Open Middleware Infrastructure Institute is adopting
this approach to support UK e-Science program – Currently UK e-Science largely either uses GT2 (as in EDG) or Simple
Web Services for “database Grids”– http://www.omii.ac.uk/
Bit levelInternet
Layered Architecture for Web Services and Grids
Base Hosting EnvironmentProtocol HTTP FTP DNS …
Presentation XDR …Session SSH …
Transport TCP UDP …Network IP …
Data Link / Physical
ServiceInternet
Application Specific GridsGenerally Useful Services and Grids
Workflow WSFL/BPELService Management (“Context etc.”)
Service Discovery (UDDI) / InformationService Internet Transport Protocol
Service Interfaces WSDL
ServiceContext
HigherLevelServices
Working up from the Bottom• We have the classic (CISCO, Juniper ….) Internet routing the
flood of ordinary packets• Web Services build the “Service Internet” with
– Fault Tolerance (WS-RM not TCP)– Security (WS-Security not IPSec/SSL etc.)– Information Services (UDDI/WS-Context not
DNS/Configuration files)– At message/web service level and not packet/IP address
level• Software-based Service Internet useful as computers “fast”• Familiar from Peer-to-peer networks and built as a software
overlay network defining Grid (analogy is VPN)• On top of “Service Internet”, one supports dynamic context or
the “shared memory” supporting groups (from 2 to more) of services
Information Grid
Enterprise Grid
Compute Grid
Campus Grid
R2R1
Teacher
Students
Dynamic light-weight Peer-to-peerCollaboration Training Grid
4 Overlay NetworksWith a 5th superimposed
Consequences of Rule of the Millisecond• Useful to remember critical time scales
– 1) 0.000001 ms – CPU does a calculation– 2) 0.001 to 0.01 ms – MPI latency– 3) 1 to 10 ms – wake-up a thread or process– 4) 10 to 1000 ms – Internet delay
• 4) implies geographically distributed metacomputing can’t in general compete with parallel systems (OK for some cases)
• 3) << 4) implies RPC not a critical programming abstraction as it ties distributed entities together and gains a time that is typically only 1% of inevitable network delay – However many service interactions are at their heart RPC but
implemented differently at times e.g. asynchronously• 2) says MPI is not relevant for a distributed environment as low
latency cannot be exploited• Even more serious than using RMI/RPC, current Object paradigms
also lead to mixed up services with unclear boundaries and autonomy• Web Services are only interesting model for services today
Linking Modules
From method based to RPC to message based to event-based
Module A
Module B
Method Calls.001 to 1 millisecond
Service A
Service B
Messages
0.1 to 1000 millisecond latency
Coarse Grain Service ModelClosely coupled Java/Python …
Service B Service A
PublisherPost Events
“Listener”Subscribe to Events
Message Queue in the Sky
What is a Simple Service?• Take any system – it has multiple functionalities
– We can implement each functionality as an independent distributed service
– Or we can bundle multiple functionalities in a single service• Whether functionality is an independent service or one of many
method calls into a “glob of software”, we can always make them as Web services by converting interface to WSDL
• Simple services are gotten by taking functionalities and making as small as possible subject to “rule of millisecond”– Distributed services incur messaging overhead of one (local) to
100’s (far apart) of milliseconds to use message rather than method call
– Use scripting or compiled integration of functionalities ONLY when require <1 millisecond interaction latency
• Apache web site has many projects that are multiple functionalities presented as (Java) globs and NOT (Java) Simple Services– Makes it hard to integrate sharing common security, user profile,
file access .. services
What is a Grid I?• You won’t find a clear description of what is Grid and how
does differ from a collection of Web Services– I see no essential reason that Grid Services have different
requirements than Web Services– Geoffrey Fox, David Walker, e-Science Gap Analysis, June 30
2003. Report UKeS-2003-01, http://www.nesc.ac.uk/technical_papers/UKeS-2003-01/index.html.
– Notice “service-building model” is like programming language – very personal!
• Grids were once defined as “Internet Scale Distributed Computing” but this isn’t good as Grids depend as much if not more on data as well as simulations
• So Grids can be termed “Internet Scale Distributed Simple Services” and represent a way of collecting services together in same way that program (package) collects methods and objects together.
What is a Grid II?
• So we build collections of Web Services which we package as component Grids– Visualization Grid– Sensor Grid– Utility Computing Grid– Person (Community) Grid– Earthquake Simulation Grid– Control Room Grid– Crisis Management Grid
• We build bigger Grids by composing component Grids using the Service Internet
Grids of Grids of Simple Services• Link via methods messages streams• Services and Grids are linked by messages• Internally to service, functionalities are linked by methods• A simple service is the smallest Grid• We are familiar with method-linked hierarchy
Lines of Code Methods Objects Programs Packages
Overlayand ComposeGrids of Grids
Methods Services Component Grids
CPUs Clusters ComputeResource Grids
MPPs
DatabasesFederatedDatabases
Sensor Sensor Nets
DataResource Grids
Critical Infrastructure (CI) Grids built as Grids of Grids
Gas Servicesand Filters
Physical Network
Registry Metadata
Flood Servicesand Filters
Flood CIGrid Gas CIGrid… Electricity CIGrid …
Data Access/Storage
Security WorkflowNotification Messaging
Portals Visualization GridCollaboration Grid
Sensor Grid Compute GridGIS Grid
Core Grid Services
Database Database
Analysis and VisualizationPortal
RepositoriesFederated Databases
Data Filter
Services
Field Trip DataStreaming Data
Sensors
?DiscoveryServices
SERVOGrid
ResearchSimulations
Research Education
CustomizationServices
From Research
to Education
EducationGrid ComputerFarmGeoscience Research and Education Grids
GISGrid
Sensor GridDatabase Grid
Compute Grid
Minicomputer
Firewall
ComputerServer
PDA
Modem
Laptop computerWorkstationPeers
Peers
Audio/VideoConferencing Client
Audio/VideoConferencing Client
NaradaBrokering BrokerNetwork
NaradaBrokering
Web Service B
Stream
Server-enhanced Messaging
NB supports messagesand streams
Queues
Multiple transport supportIn publish-subscribeParadigm with differentProtocols on each link
Transport protocols supported include TCP, Parallel TCP streams, UDP, Multicast, SSL, HTTP and HTTPS.Communications through authenticating proxies/firewalls & NATs. Network QoS based Routing
Subscription Formats Subscription can be Strings, Integers, XPath queries, Regular Expressions, SQL and tag=value pairs.
Reliable delivery Robust and exactly-once delivery of messages in presence of failures
Ordered delivery Producer Order and Total Order over a message type Time Ordered delivery using Grid-wide NTP based absolute time
Recovery and Replay Recovery from failures and disconnects.Replay of events/messages at any time.
Security Message-level WS-Security compatible security
Message Payload options
Compression and Decompression of payloadsFragmentation a nd Coalescing of payloads
Messaging Related Compliance
Java Message Service (JMS) 1.0.2b compliant Support for routing P2P JXTA interactions.
Grid Application Support NaradaBrokering enhanced Grid-FTP. Bridge to the Globus TK3.
Web Service reliability Prototype implementation of WS-ReliableMessaging
Current NaradaBrokering Features
IOI and CIE• Let us study the two layers IOI (Service Internet On the Bit Internet)
and CIE (Context and Information Environment)• IOI is most “straightforward” as it is providing reasonably well
understood capabilities at a new “level”• CIE is roughly the inter-service “shared memory” used to manage and
control them at “distributed operating system level– Critical is “shared” (a database service) versus message based CIE
IOI
Application Specific GridsGenerally Useful Services and Grids
Workflow WSFL/BPELService Management (“Context etc.”)
Service Discovery (UDDI) / InformationService Internet Transport Protocol
Service Interfaces WSDL
CIE
HigherLevelServices
NaradaBrokering and IOI• “Software Overlay Network” features• Support for Multiple Transport protocols• Support for multiple delivery mechanisms
– Reliable Delivery– Exactly-once Delivery– Ordered Delivery– Optional Delivery optimization modules for different modes
• Compression/Decompression of payloads with optional module
• Coalescing/Fragmentation of payloads with optional module
• NTP Time Service• Security Service with optional module• Virtual Private Grid
Virtualizing Communication Communication specified in terms of user goal and Quality of
Service – not in choice of port number and protocol Bit Internet Protocols have become overloaded e.g. MUST use
UDP for A/V latency requirements but CAN’t use UDP as firewall will not support ………
A given “Service Internet” communication can involve multiple transport protocols and multiple destinations – the latter possibly determined dynamically
Dial-upFilter
SatelliteUDP
FirewallHTTP
A
B1
Hand-HeldProtocol
FastLink
Software MulticastB2
B3NB Broker Client Filtering
NB Brokers
NaradaBrokering and IOI• Optimization of Messaging Environment (Broker set up
etc.) using CIE • Support of WS-FlexibleRepresentation (see later) in
transport of Streams• Performance Monitoring• Performance optimized routing with optional module• Ad-hoc Network Support• Broker Discovery --Find nearest broker• Topic discovery• SOAP Support and integration with WSIF, Axis, WSE,
gSOAP, Java WSDP• Support for WS-Reliability, WS-ReliableMessaging and
their Federation• Congestion control and other optimizing message delivery
strategies
Performance Monitoring Every broker incorporates a Monitoring service that
monitors links originating from the node. Every link measures and exposes a set of metrics
• Average delays, jitters, loss rates, throughput. Individual links can disable measurements for
individual or the entire set of metrics. Measurement
intervals can also be varied
Monitoring Service, returns measured metrics to Performance Aggregator.
BrokerNode
LinkData
BrokerNode
LinkData
Performance AggregationService
Control MessageExchange
Aggregates infofrom nodes in acertain domain
MonitoringService
NaradaBrokering Service IntegrationS1 S2P2P1
S1 S2
S1 S2
Proxy Messaging
Handler Messaging
Notification
S? Service P? Proxy
NB Transport
Internal to Service: SOAP Handlers/Extensions/Plug-ins Java (JAX-RPC) .NET Indigo and special cases: PDA's gSOAP, Axis C++
Standard SOAP Transport
Any Transport
Mechanisms for Reliable Messaging I
• There are essentially sequence numbers on each message• Unreliable transmission detected by non-arrival of a
message with a particular sequence number• Remember this is “just some TCP reliability” built at
application level• One can either use ACK’s – Receiver (service B) positively
acknowledges messages when received– Service A fully responsible for reliability
• Or NAK’s – Service B is partially responsible and tracks message numbers – sends a NAK if sequence number missing
Service B Service A
M(n+1)M(n)
Mechanisms for Reliable Messaging II• Each message has a retransmission time; messages are
retransmitted if ACK’s not received in time– Uses some increasing time delay if retransmit fails
• Note need to be informed (eventually) that OK to throw away messages at sender; pure NAK insufficient
• Note this is reliability from final end-point to beginning end-point: TCP reliability is for each link and has different grain size and less flexible reliability mechanisms
• There are several efficiency issues– Divide messages into groups and sequence within groups– Do not ACK each message but rather sequences of messages
• NAK based system attractive if high latency (some mobile devices) on messaging from receiver back to sender
Custom Message Reliability
NaradaBroker
Filter 1
Filter 2
WS-RM
WirelessOptimizedWS-RM
WS-Reliability
2 second PDA reply latency!
Different endpoints may well need different reliability schemes. Another reason to use application layer.Need to define easy touse “standard reliabilityprofiles
NaradaBrokering and Fault Tolerance• As well as reliable messaging, NaradaBrokering supports
performance based dynamic routing– Choose both route and protocol (UDP, Parallel TCP ..)
• It will also support automatic fail-over among replicated services subscribing to same message stream
• Provides scriptable control of streams for custom management schemes
• Saves ALL messages in faulttolerant storage for eithersession replay or recovery
• Will support reliable BitTorrent P2P file swapping model (better than GridFTP?)
GridFTP plus NaradaBrokering
Mirror Mirror on the wallWho is the fastest most reliable of them all?
Web Services!!!• Application layer “Internet” allows one to optimize message streams
and the cost of “startup time”, Web Services can deliver the fastest possible interconnections with or without reliable messaging
• Typical results from Grossman (UIC) comparing Slow SOAP over TCP with binary and UDP transport (latter gains a factor of 1000)
SOAP/XML WS-DMX/ASCII WS-DMX/Binary Record Count MB µ σ/µ MB µ σ/µ MB µ σ/µ
10000 0.93 2.04 6.45% 0.5 1.47 0.61% 0.28 1.45 0.38% 50000 4.65 8.21 1.57% 2.4 1.79 0.50% 1.4 1.63 0.27% 150000 13.9 26.4 0.30% 7.2 2.09 0.62% 4.2 1.94 0.85% 375000 34.9 75.4 0.25% 18 3.08 0.29% 10.5 2.11 1.11% 1000000 93 278 0.11% 48 3.88 1.73% 28 3.32 0.25% 5000000 465 7020 2.23% 242 8.45 6.92% 140 5.60 8.12%
Pure SOAP SOAP over UDP Binary over UDP
7020 5.60
SOAP Tortoise and UDP Hare II• Mechanism only works for streams – sets of related
messages• SOAP header in streams is constant except for
sequence number (Message ID), time-stamp ..• One needs two types of new Web Service
Specification• “WS-StreamNegotiation” to define how one can use
WS-Policy to send messages at start of a stream to define the methodology for treating remaining messages in stream
• “WS-FlexibleRepresentation” to define new encodings of messages
SOAP Tortoise and UDP Hare III• Then use “WS-StreamNegotiation” to negotiate stream in Tortoise
SOAP – ASCII XML over HTTP and TCP – – Deposit basic SOAP header through connection – it is part of
context for stream (linking of 2 services)– Agree on firewall penetration, reliability mechanism, binary
representation and fast transport protocol– Naturally transport UDP plus WS-RM
• Use “WS-FlexibleRepresentation” to define encoding of a Fast transport (On a different port) with messages just having “FlexibleRepresentationContextToken”, Sequence Number, Time stamp if needed– RTP packets have essentially this structure– Could add stream termination status
• Can monitor and control with original negotiation stream• Can generate different streams optimized for different end-points
CIE: Common Service Information and Metadata
• WS-RF and WS-GAF approach state with different approaches to contextualization – supplying a common “context” (Shared token or more generally (resource) metadata)
• One can supports such a common context either as pool of messages or as message-based access to a “database” (Context Service)
• We define a collection of services sharing information as a Gaggle
• Two services linked by a stream are perhaps simplest example of a Gaggle
CIE II• There is small amount of core shared information that must
be supported in the gaggle. This includes dynamic service metadata and the equivalent of configuration information.– We can put any “light-weight” information in the CIE even if it is
very application dependent• There are various metadata mechanisms suitable for “light-
weight dynamic situations”. WS-MetadataExchange, WS-Context and WSRF represent variants that can be supported.– UDDI, Globus MDS are “bigger” solutions
• The CIE stops short of supporting workflow; that would be built on top of CIE in analogous fashion to way that WS-CAF supports transactions as an extension of WS-Context.
• Note that there is a tension between storing metadata in messages and services. – This is shared versus distributed memory debate in parallel
computing
CIE III• We view the metadata as equivalent to some shared
memory for the services and whatever the actual implementation build a logical interface that is message based interactions with a simple database.
• There is a scripting environment that allows one to interact with administrative aspects of services and the support environment – it programs the Gaggle.
• Dennis Gannon notes that Linda (JavaSpaces) is an interesting model for the CIE at the Gaggle level.
• Notification Service is one simple CIE information tool
CIE in NaradaBrokering I• Scripting Management based on HPSearch technology• Topic discovery information repository• Broker discovery information repository• NaradaBrokering metadata information repository• Support of WS-FlexibleRepresentation information and
negotiation services• CIE Portal allowing access to all metadata, management of
deployment, firewall tunnels, performance info, error logs etc.– User and “programmatic” interfaces to set “defaults”
• Security Access and Authentication interface
CIE in NaradaBrokering II• Support of fault tolerant message storage, discovery and access
– Recovery– Replay or time difference sensitive recovery– Instant replay or replay of real-time streams– Support of finite state change architecture with major (complete) and minor
(partial) update events• Support of replicated services• Other core service fault tolerance mechanisms such as scalable
heartbeat• Support of multiple topic subscription types including string, integer,
XPath, Regular Expressions and <tag, value> tuples• JMS Compliance• Support of WS-Notification and WS-Eventing• Support of push and pull event models• Support for JXTA• Support for Gnutella• GridTorrent
CIE in NaradaBrokering III• Fault tolerant light weight metadata database (the core
metadata technology)• Support of WS-Discovery masquerading as dynamic UDDI• Support of WS-DM to manage services• Support of WSRF• Support of WS-Context• Support for WS-MetadataExchange• Support of session management aspects of XGSP, JXTA
Peer Groups etc.• GridFTP• FTHPIS
Web Service Metadata and State I• The Semantic Grid and Semantic Web are important
frameworks for metadata but handicapped by lack of “compelling” tools
• RDF Resource Description Framework (W3C) Set of recommendations expanded from original February 1999 standard http://www.w3.org/RDF/ and the heart of the Semantic Web and Grid http://www.semanticgrid.org
• DAML+OIL combining DAML (Darpa Agent Markup Language) and OIL (Ontology Inference Layer) (W3C) Note December 2001 http://www.w3.org/TR/daml+oil-reference
• OWL Web Ontology Language (W3C) Recommendation February 2004 http://www.w3.org/TR/2004/REC-owl-features-20040210/ More later!
Web Service Metadata and State II• WS-DistributedManagement Web Services Distributed Management
Framework with MUWS and MOWS below (OASIS) http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wsdm
• Management includes issues like monitoring quality of service, enforcing service level agreements, controlling tasks and managing life-cycles.
• WSDM-MUWS Web Services Distributed Management: Management Using Web Services (OASIS) V0.5 Committee Draft April 2004 http://www.oasis-open.org/committees/download.php/6234/cd-wsdm-muws-0.5.pdf
• WSDM-MOWS Web Services Distributed Management: Management of Web Services (OASIS) V0.5 Committee Draft April 2004 http://www.oasis-open.org/committees/download.php/6255/cd-wsdm-mows-0.5-20040402.pdf
• WS-MetadataExchange Web Services Metadata Exchange (BEA,IBM, Microsoft, SAP) March 2004 http://www-106.ibm.com/developerworks/library/specification/ws-mex/– Describes how metadata can be exchanged between services rather
than by looking it up in registries like UDDI or higher level metadata catalogs; the old OGSI standard used such service-resident metadata extensively
Web Service Metadata and State III• WS-RF Web Services Resource Framework including WS-
ResourceProperties, WS-ResourceLifetime, WS-RenewableReferences, WS-ServiceGroup, and WS-BaseFaults (OASIS) http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wsrf with Oasis TC set up April 2004 and V1.1 Framework March 2004 http://www-106.ibm.com/developerworks/library/ws-resource/ws-modelingresources.pdf – Uses rich metadata to define stateful interactions – its use of SOAP header
creates interoperability problems
• ASAP Asynchronous Service Access Protocol (OASIS)• http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=asap
with V1.0 working draft G June 2004 http://www.oasis-open.org/committees/download.php/7151/wd-asap-spec-01g.pdf
• WS-GAF Web Service Grid Application Framework (Arjuna, Newcastle University) http://www.neresc.ac.uk/ws-gaf/ – Uses WS-Context to provide “opaque” (don’t say much) stateful interactions
Metadata Catastrophe I• We keep finding places where metadata can be transmitted
to and from services• WS-Addressing and WS-RF specify metadata in SOAP
header of messages • WS-Context similarly specifies both SOAP header and WS-
Context context services as location of (temporary) metadata
• We have registries like UDDI of service data• WS-MetadataExchange covers metadata stored in services
– Service metadata is very common and often not explicitly called out e.g. WebDAV as in Apache Slide stores file metadata in addition to versioning information
• In addition, we have major source of one or more (federated) catalogs
• I think this confused situation will need to be addressed by some new dynamic metadata model
Metadata Catastrophe II• There are large long term metadata catalogs associated with
major applications/services– These are likely to remain as now based on traditional major
database technology like Oracle MySQLK and DB2• There are small but broadly available metadata catalogs
– Globus MDS and EDG RGMA roughly address these– Semantic Grid enriched Service catalogs as in UDDI
• We need to implement UDDI in a distributed (federated) fashion and work around its non-intuitive schema but this seems straightforward
• All the problems occur for local and highly dynamic data where key issues are:– Consistency: If metadata stored in messages flowing around, how
do we ensure consistency if it ever changes– Where is it: How do we decide where to look it up?
• My intuition is that best solution is highly dynamic lightweight database – doesn’t really fit any proposal yet!
Metadata and Semantic Grid• Can store in one catalog, multiple catalogs or in each service
– Not clear how a coherent approach will develop• Specialized metadata services like UDDI and MDS (Globus)
– Nobody likes UDDI– MDS uses old fashioned LDAP– RGMA is MDS with a relational database backend
• Some basic XML database (Oracle, Xindice …)• “By hand” as in current SERVOGrid Portal which is roughly same
as using service stored SDE’s (Service Data Elements) as in OGSI• Semantic Web (Darpa) produced a lot of metadata tools aimed at
annotating and searching/reasoning about metadata enhanced webpages– Semantic Grid uses for enriching Web Services– Implies interesting programming model with traditional analysis
(compiler) augmented by meta-data annotation
Database
SDE1SDE2
Service
SDE1SDE2
Service
SDE1SDE2
Service
SDE1SDE2
Service
SDE1SDE2
Service
SDE1SDE2
Service
SDE1SDE2
Service
Individual Services
Web Service Ports
Grid or Domain Specific Metadata Catalogs
System or Federated Registry or Metadata Catalog
Database1 Database2 Database3
Four Metadata Architectures
MessagesM M M M M M M M M M M M
Stateful Interactions• There are (at least) four approaches to specifying
state– OGSI use factories to generate separate services for each
session in standard distributed object fashion
– Globus GT-4 and WSRF use metadata of a resource to identify state associated with particular session
– WS-GAF uses WS-Context to provide abstract context defining state. Has strength and weakness that reveals less about nature of session
– WS-I+ “Pure Web Service” leaves state specification the application – e.g. put a context in the SOAP body
• I think we should smile and write a great metadata service hiding all these different models for state and metadata
Explicit and Implicit Factories• Stateful interactions are typified by amazon.com where messages carry
correlation information allowing multiple messages to be linked together– Amazon preserves state in this fashion which is in fact preserved in its
database permanently• Stateful services have state that can be queried outside a particular interaction• Also note difference between implicit and explicit factories
– Some claim that implicit factories scale as each service manages its own instances and so do not need to worry about registering instances and lifetime management
FACTORY
1
2
3
4
FACTORY
1
2
3
4
Explicit FactoryImplicit Factory
Hiddeninstances
Explicitinstances
Web Service Notification I• WS-Eventing Web Services Eventing (BEA, Microsoft,
TIBCO) January 2004 http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnglobspec/html/WS-Eventing.asp
• WS-Notification Framework for Web Services Notification with WS-Topics, WS-BaseNotification, and WS-BrokeredNotification (OASIS) OASIS Web Services Notification TC Set up March 2004 http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wsn and http://www-106.ibm.com/developerworks/library/specification/ws-notification/
• JMS Java Message Service V1.1 March 2002 http://java.sun.com/products/jms/docs.html
Notification Architecture• Point-to-Point
• Or Brokered
• Note that MOM (Message Oriented Middleware) uses brokered messaging for ALL transmission and not just “special” notification messages
Service B Service A
Broker PublishSubscribe
Queues MessagesSupports creationand subscription of topics
Service B Service APublishSubscribe
Classic Publish-Subscribe
Publisher 1
Subscriber 1 Subscriber 3Subscriber 2 Subscriber 5Subscriber 4
Topic BTopic A Topic C
Publisher 2
NotificationServicebroker
broker
brokerbroker
Publisher 1
Subscriber 1 Subscriber 3Subscriber 2 Subscriber 5Subscriber 4
Topic BTopic A Topic C
Publisher 2
NotificationServicebroker
broker
brokerbroker
Web Service Notification II• WS-Eventing is quite similar to WS-BaseNotification and
provides service to service notification• WS-Notification is similar to CORBA event service and
adds brokers to mediate notification which has several advantages– Don’t need queues and lists of subscribers on each
service– Solution scales to any number of publishers/subscribers
• JMS well known successful non Web Service brokered notification system
• Topics defined in WS-Topics can also provide contextualization
• Expect this area to clarify reasonably soon
FTHPIS: CIE for Applications I• CIE is designed to support metadata needed by the
“galactic distributed operating system” controlling services• It can used for cases where an application naturally is
supported by a Gaggle services with a modest amount of metadata
• A new approach to Configuration files• Maintaining user profiles and preferences• Maintaining application specific metadata such as GIS
metadata • Maintain “Grading System” for e-learning• Maintaining information regarding sessions and also the
state of entities in these sessions in GlobalMMCS• Enable playback/replay capabilities in GlobalMMCS or
other NaradaBrokering applications needing fault-tolerant event storage
FTHPIS: CIE for Applications II• Three key features of FTHPIS are
– Supported in Peer-to-peer or central server role– Fault Tolerance– Context (database) is lightweight and either files or real
databases• Fault Tolerance includes
– WS-ReliableMessaging– Replicated Storage of Messages to support reliable
messaging and permanent storage for replay. Also supports performance as data locality (caching)
– Fault Tolerance metadata stored in CIE (FTHPIS)– NaradaBrokering support of replicated services where
these services are discovery, access, storage etc for each FTHPIS (include caching support – location of nearest copy)
P2P and NaradaBrokering I Server/Broker-free version to support “immediate deployment” of NB-based
Community Grids using P2P versions of Grid applications Initially: Use a broker free version of NB and file-based Web services
• If successful, add brokers in the Grid sky to achieve better performance (if broker has better network link than clients)
Service providers and supercomputer/national grid centers could sell such Grid Farm services
P2P
Grid Farm in the Sky (clouds)
NaradaBrokers
P2P and NaradaBrokering II Global Information System could start with multiple
FTHPIS style lightweight Fault Tolerant Metadata Catalogs – subscribe multiple instances of metadata service to the MetadataCatalogN topics – publish queries to these replicated subscriber topics
Then one could add DHT (Distributed Hash Table) approach (used in latest JXTA) in NB. NB nodes will have Ids that can determine where a specific content would be stored.• Provides scalable location of content over LARGE numbers
of end-points with limited query capabilities Flooding to bridge gap between DHT and FTHPIS
where queries are broadcast to multiple gaggles and FTHPIS is used within each Gaggle
P2P and NaradaBrokering III GridTorrent: Merge NB-enhanced GridFTP and P2P
BitTorrent http://bitconjurer.org/BitTorrent/ to provide WSRM fault tolerant Parallel TCP P2P or Grid file transfer• BitTorrent supports fragmented distributed files which are
natural WSRM and NB architecture• Don’t really want GridFTP server; prefer to use fault tolerant
GridTorrent metadata service (implemented as a Gaggle FTPHIS)
GridTorrent file transfer automatically provides distributed fault tolerant caching – a better RLS?
P2P and NaradaBrokering IV We have proposed building a portlet based portal to the
IOI CIE and FTHPIS In the spirit of P2P, we can propose a model where one
exposes distributed files AND streams in same way as Napster or Gnutella – each resource (file or stream) is displayed in a “metadata enhanced” ls/Windows Explorer fashion• Metadata includes updating thumbnail for streams as in
GlobalMMCS or AccessGrid Note that “unit of resource storage” could be individual
end-point (client or server) or Gaggles or even better hierarchically endpoints Gaggles Gaggles of Gaggles