content-based publish/subscribe: a re-assessment

OTM/DOA 2005 31 October 2005

Content-Based Publish/Subscribe

:A Re-Assessment

David S. RosenblumLondon Software Systems

University College London


Acknowledgments

Alexander L. WolfAntonio Carzaniga

Costin Raiciu

University of Lugano

University College London


The ‘Fire Hose’


Controlling the Fire Hose


Publish/Subscribe

symbol == MSFT&&

price > 30.00

symbol = MSFTprice = 29.34symbol = MSFTprice = 30.17symbol = IBMprice = 83.47

symbol = MSFTprice = 30.17


Publish/Subscribe Features Asynchronous delivery Multi-way delivery Content-driven interaction Anonymity Strong decoupling

Many applications are a natural fit


Some Ancient HistoryYEAST

Pub/sub for LANs of UNIX workstations

Centralised server implementation Novelty: Applications

Process awareness Office automation Telco feature deployment Many others .h

.cpp


Some More Recent HistorySIENA

Wide-area content-based publish/subscribe Decentralised overlay network ofpublish/subscribe ‘routers’

Routing and forwarding based onsubscription and notification content

Novelty:Algorithms, Protocols, ArchitecturesAssumed that the applications

would naturally appear!


Most RecentlyPreCache

Sony-funded startup to commercialise content-based publish/subscribe

Survived 2.5 years Successful technology development Less successful business development

Video-on-demand (???) Anti-virus updates Travel alerts


So What Are the Killer Applications? Many research projects

Many novel research results

No significant deployments yet

Need to take a closer lookat some proposed approaches


3

2

4

8

1

7

65

9

SIENA Content-Based RoutingSubscription Forwarding

as1

s1:a

s1:1

s1:2

s1:3

s1:2

s1:6

s1:3

s1:1

s1:5

s1: “price < 700”


s2: “price < 600”

3

2

4

8

7

65

9

SIENA Content-Based Routing Subscription Merging

s1:1

s1:2

s1:6

s1:3

s1:1

s2b

s1:3

s1:2

s1:5

s1:1s2:5

s1:2s2:8

s1:5s2:b

as1 covers s2

1

s1 covers s2

s1:as1:as2:2


3

2

4

8

1

7

65

9

SIENA Content-Based Routing Notification Delivery

b

s1:1s2:5

s1:2

s1:6

s1:3

s1:1s1:3

s1:2s2:8

a s1:as2:2

s1:5s2:b

n1: “price = 550”

n1


Implications of SIENA’s Design Notifications can be very frequent But subscriptions should be relatively infrequent

Yet there should be a lot of subscription variation

But there should be some similar subscriptions

And the similar subscriptions should come from the same part of the network Which applications are like this?


Other Approaches Gryphon

Subscription flooding over tree of clusters Applicable if subscriptions are few and stable

Hermes Rendezvous nodes allocated to content types

Applicable if load is spread evenly by type PreCache

Trie- and kd-tree-based subscription storage Applicable if unsubscription occurs very infrequently

All of these limit application suitability


Publish/Subscribe FeaturesConceptual Features

Asynchronous delivery

Multi-way delivery Content-driven interaction

Anonymity Strong decoupling

Few applications can naturally exploit these features

Infrastructure Features

Message flooding Subscription merging

Tree-based routing Localised forwarding

Content partitioning


ExampleStock Quotes vs Online Gaming

Stock Quotes

Message flooding? Subscription merging Tree-based routing Localised forwarding Content partitioning

Online Gaming

Message flooding? Subscription merging

Tree-based routing? Localised forwarding

Content partitioning

One size infrastructure does not fit all


???

Matching Applications with Infrastructures

Application Characteristics

Notification size Notification throughput

Notification latency Notification variability

Subscription selectivity

Subscription stability Locality …

Infrastructure Characteristics

Number of routers Number of routing hops

Path redundancy Subscription replication

Matching complexity Matching accuracy …


ExampleStock Quotes vs Online Gaming

Stock Quotes

Notification size Notification frequency Notification variability

Notification latency Subscription selectivity

Subscription stability Locality

Online Gaming

Notification size Notification frequency Notification variability

Notification latency Subscription selectivity

Subscription stability? Locality

How do we translate these to design decisions?


Additional Complications Mobility

Of publishers Of subscribers Of routers

Firewalls Edge Fanout

Security


The Value of Information

Can we do secure content-based routingover an untrusted infrastructure?


Security in Content-Based Publish/Subscribe Encryption used to implement many security goals Authentication Confidentiality Integrity

But content-based routing intrinsically requires some transparency of content Infrastructure must be able to determine if a

subscription matches notification Existing approaches have limited applicability

In large part due to need to secure multiple messages


A Cryptographic Protocol Based on Yao’s Garbled Circuits Subscriptions transformed to Boolean circuits and then garbled based on shared secret

Notifications encrypted with shared secret

Router evaluates circuit on encrypted notification

Router knows result but not content!Weak but inexpensive security


Better security but very expensive

A Cryptographic Protocol Based on PSM PSM = Private Simultaneous Messages (Feige et al.)

Subscription matching transformed to graph reachability

Notifications and subscriptions transformed to subgraphs and encrypted based on shared secret

Router sums adjacency matrices for subgraphs Router checks rank of resulting matrix for match

Router knows result but not content!


Inherent Security Limitations(1)

Must provide confidentiality of both notifications and subscriptions Range of plaintext notifications can be matched against confidential subscription

Range of plaintext subscriptions can be matched against confidential notification

Router must know outcome of match This alone can sometimes be useful information Example: Battlefield Awareness


Inherent Security Limitations(2)

Router can determine subscription coverage over time Again, this may be useful information

Router can determine Euclidean distance between notifications over time

Studied protocols require sharing of secret among potentially large number of publishers and subscribers


Inherent Limitations of Possible Security Solutions Cryptographic group membership protocols

Too expensive with high subscription volatility Padding notification stream with dummy messages Reduces throughput and increases latency of

infrastructure Defeats the whole purpose of the infrastructure!

Proxy publishers and subscribers Increases latency of messages

Trusted infrastructure Can be expensive to deploy for each application


A Generic Architecture for Content-Based Matching

Separates matching from routing Fully-connected mesh of N nodes in C clusters

Full connectivity simulated on DHT with minimal overhead

Choose 2 of 3 configuration parameters Subscription replication rate R (= N/C) Notification routing hops H (1 H C) Load-balancing factor B

Cluster 1

Cluster C

Cluster 3

Cluster 2

Cluster 4


ConclusionThe Past

There have been many innovations in wide-area content-based publish/subscribe

But researchers have ignored application characteristics for too long

A universal infrastructure shared by all applications is probably not feasible

Security is very difficult to achieve over an untrusted infrastructure


ConclusionThe Future

We need to understand better the relationship between application requirements and infrastructure design

And we need to explore further the limits of security in content-based publish/subscribe


Questions?

Prof. David S. RosenblumLondon Software SystemsUniversity College London

[email protected]://www.cs.ucl.ac.uk/staff/D.Rosenblum/

mailto:[email protected]

content-based publish/subscribe: a re-assessment

Documents