11

A High Level Pub/Sub Layer for A High Level Pub/Sub Layer for Open Distributed Heterogeneous Open Distributed Heterogeneous

EnvironmentsEnvironments

Universidad Nacional del Centro de la Provincia de Buenos Universidad Nacional del Centro de la Provincia de Buenos AiresAires

Facultad de Ciencias ExactasFacultad de Ciencias Exactas

Tandil, Argentina - Tandil, Argentina - FebruaryFebruary, 2004, 2004

Candidate:Candidate:Mario E. AntolliniMario E. Antollini

Thesis Supervisor:Thesis Supervisor:Mariano A. CiliaMariano A. Cilia

22

AgendaAgenda

MotivationMotivation Problem StatementProblem Statement Proposed ApproachProposed Approach Design & ImplementationDesign & Implementation ConclusionsConclusions

33

MotivationMotivation

Today: Client/Server (Request/Reply) Today: Client/Server (Request/Reply) predominancepredominance

Trend: Convergence of technologiesTrend: Convergence of technologies WWW, sensor networks, enterprise strategiesWWW, sensor networks, enterprise strategies

Data exchanged across traditional bordersData exchanged across traditional borders Information-drivenInformation-driven Large-scale distributed systemsLarge-scale distributed systems

Pull-based (Request/Reply) does not fitPull-based (Request/Reply) does not fit Polling implies resource wastePolling implies resource waste Leads to network saturation or server breakdown!Leads to network saturation or server breakdown!

New Paradigm for information-driven applicationsNew Paradigm for information-driven applications Push-based (event-based dissemination)Push-based (event-based dissemination)

44

Publish/Subscribe - IntroPublish/Subscribe - Intro Publish/Subscribe MiddlewarePublish/Subscribe Middleware

Reflects intrinsic behavior of Information-driven Reflects intrinsic behavior of Information-driven ApplicationApplication

Loosely-couplingLoosely-coupling Support asynchronous communicationSupport asynchronous communication ParticipantsParticipants

anonymous to each otheranonymous to each other can “dynamically” join and leavecan “dynamically” join and leave

Doesn’t require publishers and subscribersDoesn’t require publishers and subscribersat the same timeat the same time

Location TransparencyLocation Transparency It acts as an “intermediator” among It acts as an “intermediator” among

participantsparticipants

55

acting as producers

acting asconsumers

Pub/Sub – Intro (cont.)Pub/Sub – Intro (cont.) Independent Process Data Flow StyleIndependent Process Data Flow Style

Message Manager or Notification Service (JMS, Message Manager or Notification Service (JMS, Rebeca)Rebeca)

Clients (producers/consumers/both)Clients (producers/consumers/both) Communication linksCommunication links

Subsc ribes

Subsc ribes

P ublishesM s g

M s gM s g X

M s g X

M s g Y

M s g Y

Delivers

Delivers S ub X

S ub X

S ub Y

S ub Y

Client2

Client5

Client4

M s g X

M s g Y

Client1

Client3

Subsc ribes

P ublishes

Delivers

66

Problem StatementProblem Statement

Even when Notification Services Even when Notification Services support loosely-coupling and support loosely-coupling and anonymous participants, theyanonymous participants, they Do notDo not provide any support for Data provide any support for Data

Exchange among heterogeneous Exchange among heterogeneous participantsparticipants

Do notDo not support Event (message) support Event (message) CorrelationCorrelation

77

Problem Statement – Data Problem Statement – Data ExchangeExchange

Data from different sources/components is Data from different sources/components is represented differentlyrepresented differently

Different organizations/departments use different Different organizations/departments use different units and representation formatsunits and representation formats

Context information is usually left implicit and Context information is usually left implicit and consequently it is lost when crossing consequently it is lost when crossing component or institutional boundaries component or institutional boundaries

(date) 7/11/2003 Which one is the month?(date) 7/11/2003 Which one is the month? (price) 200 Currency? €?, U$S?…(price) 200 Currency? €?, U$S?…

Data from different apps needs to be Data from different apps needs to be interpreted by applications interpreted by applications

no cultural assumption!no cultural assumption!

To process events in a semantically To process events in a semantically meaningful way, meaningful way, explicit informationexplicit information about about semantics of data is requiredsemantics of data is required

88

Concept-based ApproachConcept-based Approach

Provides a higher level of abstraction to Provides a higher level of abstraction to describe the interests of publishers and describe the interests of publishers and subscriberssubscribers

Subscribers and Publishers can specify their Subscribers and Publishers can specify their assumptionsassumptions

Price < 100 Price < 100 [€][€] DeliveryDate := 7/11/2003 DeliveryDate := 7/11/2003 [dd/mm/yyyy][dd/mm/yyyy]

The notification service delivers The notification service delivers ready-to-ready-to-processprocess events to subscribers events to subscribers

No further processing is neededNo further processing is needed

99

Concept-based Approach – Concept-based Approach – OverviewOverview

Publisher1

Subscriber1

Subscriber3

Unsubsc ribe()

Notify()

Send() Publisher2

Subscriber2

Subsc ribe()

S end()

NS

1010

Design & ImplementationDesign & Implementation

MIXMIX Context TransformationContext Transformation Mapping Different Addressing ModelsMapping Different Addressing Models Notification Service IndependencyNotification Service Independency

1111

Model for Data Exchange - Model for Data Exchange - MIXMIX

MIX Data ModelMIX Data Model Allows the definition of concepts to describe Allows the definition of concepts to describe

data and metadata from a given domaindata and metadata from a given domain Event RepresentationEvent Representation

Concepts of the ontologyConcepts of the ontology Composed by a tree-shape object structureComposed by a tree-shape object structure

Subscription RepresentationSubscription Representation Boolean expressionsBoolean expressions Include contextual information for the correct Include contextual information for the correct

interpretation and matching of incoming eventsinterpretation and matching of incoming events

1212

Context TransformationContext Transformation

Adapters Adapters Applications’ contact point with the NSApplications’ contact point with the NS They realize mapping operations from local They realize mapping operations from local

context to the matching one and vice versacontext to the matching one and vice versa Context transformation occurs in case of both Context transformation occurs in case of both

event producers and event consumersevent producers and event consumers Before events arrive at subscriber’s side, Before events arrive at subscriber’s side,

adapters automatically convert message's adapters automatically convert message's context to the one specified by subscribers (at context to the one specified by subscribers (at subscription time)subscription time)

1313

Mapping Different Addressing Mapping Different Addressing ModelsModels

Data delivery problems is delegated to the underlying Data delivery problems is delegated to the underlying NSNS

The Concept-based approach is able to work on top of The Concept-based approach is able to work on top of NSs that can rely on different addressing modelsNSs that can rely on different addressing models

This flexibility requires different mapping strategiesThis flexibility requires different mapping strategies

1414

Mapping Different Adressing Mapping Different Adressing Models (cont.)Models (cont.)

ex pr pr edicat esub j ect pr edicat e (only =)

channel pr edicat e

msg’sub j ect + msgchannel + msg

cont ex tE valuat e r est of pat t er napply conver sion f unct ion f or cont ex t

E valuat e pat t er n+ apply conv. f unct f or cont ex t

ex pr + callback’sub j ect + callback’channel + callback ’

S ubscr iber

Cont entS ubj ectChannelA ddr essing model

msg’ E x t r act cont ent f r om msg

sub j ect msg(ident if ying at t r s)

channel msgPublisher

ex pr pr edicat esub j ect pr edicat e (only =)

channel pr edicat e

msg’sub j ect + msgchannel + msg

cont ex tE valuat e r est of pat t er napply conver sion f unct ion f or cont ex t

E valuat e pat t er n+ apply conv. f unct f or cont ex t

ex pr + callback’sub j ect + callback’channel + callback ’S ubscr iber

Cont entS ubj ectChannelA ddr essing model

msg’ E x t r act cont ent f r om msg

sub j ect msg(ident if ying at t r s)

channel msgPublisher

T opic

t opic

t opic + callback’

apply conver sionf unct ion f or

msg

t opic + msg

(ident if ying at t r s)

cont ex t

pr edicat et opic + select or

1515

Publisher CBPublisher

CBFactory CBM sg

M IX

NS IndependencyNS Independency

NS 2

NSM sg 2NSFactory 2

NS 1

NSM sg 1NSFactory 1

CBSubscriber

CBFactory

CBSubscription

CBListener

Subscriber

Listener

M IX

NS 1

NSFactory 1

NSSubscription 1

NSListener1

NS 2

NSFactory 2

NSSubscription 2

NSListener2

1616

NS Independency -NS Independency -Abstract Factory PatternAbstract Factory Pattern

1717

CBPublisher

JMS

Pub. clientJ2E E

MDB

CBSubscriber

Subs. client

Concept-based in J2EEConcept-based in J2EE

Concept-based through MDBsConcept-based through MDBs

S ub

M s g

1818

Problem Statement – Event Problem Statement – Event CorrelationCorrelation

Complex Event DetectionComplex Event Detection Involves de occurrence of two or more Involves de occurrence of two or more

primitive and/or composite events primitive and/or composite events Composite events are expressed using Composite events are expressed using

an event algebraan event algebra Order of occurrence of events is crucialOrder of occurrence of events is crucial

In open distributed environmentsIn open distributed environments Global time is not applicableGlobal time is not applicable

Total order of events cannot be guaranteedTotal order of events cannot be guaranteed

1919

Complex Event Detection Complex Event Detection PlatformPlatform

Proper Interpretation of TimeProper Interpretation of Time Timestamp representationTimestamp representation

Partial Order of EventsPartial Order of Events Situations of uncertain timestamp order are Situations of uncertain timestamp order are

detected. Action taken is exposed and well detected. Action taken is exposed and well defineddefined

Events are not erroneously orderedEvents are not erroneously ordered Transmission Delays, Failures, Order Transmission Delays, Failures, Order

and Uncertaintyand Uncertainty HeartbeatHeartbeat Window mechanismWindow mechanism Consumption modesConsumption modes

2020

Complex Event Detection Complex Event Detection Platform - EventListPlatform - EventList

Temporarily maintains event instances before Temporarily maintains event instances before compositioncomposition

time

sync -po int

failu

re h

andling

policy

selec tio

n polic

y

(sin

gle /multip

le)

u ncertain

ty

policy

event consumptioninterfac e

-ge t-ge tF rom R eference-consum e-cleanU p-em pty

M s gHeart-beat

2121

Complex Event Detection Complex Event Detection Platform – Event CompositorPlatform – Event Compositor

AND

SEQ

2222

ConclusionsConclusions Functionality of Notification Service was extended Functionality of Notification Service was extended

withwith Higher-level layerHigher-level layer

Common VocabularyCommon Vocabulary Contextual informationContextual information Mapping to different addressing modelsMapping to different addressing models NS API: independence of underlying Notification ServiceNS API: independence of underlying Notification Service Ready-to-process notificationsReady-to-process notifications Support meaningful data exchange among Support meaningful data exchange among

heterogeneous applicationsheterogeneous applications Complex Event DetectionComplex Event Detection

Component-container modelComponent-container model Easy to develop operatorEasy to develop operator

Definition is restricted to operator logicDefinition is restricted to operator logic Other aspects are handled with policiesOther aspects are handled with policies

J2EE IntegrationJ2EE Integration

2323

Future WorkFuture Work

Concept-based implementationConcept-based implementation Experiment with different addressing Experiment with different addressing

modelsmodels Remotely accessible operationRemotely accessible operation Hot-swapping of Notification ServiceHot-swapping of Notification Service

Other EnhancementsOther Enhancements Visualization tools to graphically trace Visualization tools to graphically trace

flow of event instances flow of event instances Trust and SecurityTrust and Security