highly available publish/subscribe
TRANSCRIPT
Background & Motivation Handling Overload Evaluation Summary & Future Work 1 of 22 slides
Highly Available Publish/SubscribeDEBS 2006
Zbigniew Jerzak, Christof Fetzer
Dresden University of Technology
Monday 3rd July, 2006
{Zbigniew.Jerzak, Christof.Fetzer}@tu-dresden.de
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 2 of 22 slides
Motivation
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 2 of 22 slides
Motivation
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 2 of 22 slides
Motivation
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 2 of 22 slides
Motivation
Priorities?
Congestion?
Self-optimization?
Flexibility?
Fairness?
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 3 of 22 slides
Transient Congestion Effect
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 4 of 22 slides
Vicious Cycle
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 4 of 22 slides
Vicious Cycle
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 4 of 22 slides
Vicious Cycle
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 4 of 22 slides
Vicious Cycle
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 5 of 22 slides
Pub/Sub Invariants [EFGK03]
Space Decoupling
Time Decoupling
Synchronisation Decoupling
Acyclic/Cyclic graph network topology
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 6 of 22 slides
Solution Sketch
Priorities — value based approach
Congestion — load shedding (spanning tree)
Self-optimization — dynamically adapt to load
Flexibility — no fixed classes/priorities
Fairness — approach based on market economy
No violation of decoupling properties
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 6 of 22 slides
Solution Sketch
Priorities — value based approach
Congestion — load shedding (spanning tree)
Self-optimization — dynamically adapt to load
Flexibility — no fixed classes/priorities
Fairness — approach based on market economy
No violation of decoupling properties
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 7 of 22 slides
Subscription Routing
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 7 of 22 slides
Subscription Routing
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 7 of 22 slides
Subscription Routing
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 8 of 22 slides
Subscription Routing summary
Each subscription piggybacks client assigned value (Priorities,Flexibility)
Values are aggregated along with subscriptions
Small memory footprint and minimal bandwidth overhead
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 9 of 22 slides
Publication Forwarding — No Congestion/Overload
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 10 of 22 slides
Publication Forwarding — Link Overload
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 10 of 22 slides
Publication Forwarding — Link Overload
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 10 of 22 slides
Publication Forwarding — Link Overload
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 10 of 22 slides
Publication Forwarding — Link Overload
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 11 of 22 slides
Publication Forwarding — Router Overload
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 11 of 22 slides
Publication Forwarding — Router Overload
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 11 of 22 slides
Publication Forwarding — Router Overload
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 11 of 22 slides
Publication Forwarding — Router Overload
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 12 of 22 slides
Publication Forwarding Summary
Kill two birds with one stone (Congestion):router overloadlink overload
Re-integrate shedded links when load decreases (Flexibility)
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 13 of 22 slides
Why Coverage?
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 13 of 22 slides
Why Coverage?
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 13 of 22 slides
Why Coverage?
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 13 of 22 slides
Why Coverage?
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 13 of 22 slides
Why Coverage?
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 14 of 22 slides
Coverage Summary
Shed only as much as needed and only when needed(Self-optimization)
Charge for message delivery ⇒ prevent selfish behaviour(Fairness)
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 15 of 22 slides
Evaluation Environment
OMNeT++
20 node network – acyclic graph topology
Each run – 2 · 106 messagesNumber of publications > subscriptions ⇒ system stabilises
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 16 of 22 slides
Figure 1: Link marking and resulting matching times in the HAPS router
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 17 of 22 slides
Figure 2: Router revenue for different interface marking algorithms
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 18 of 22 slides
Figure 3: Backup link activation and resulting transmission times using the messageshedding
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 19 of 22 slides
Summary
Decoupled approach
Symmetrical algorithms
Fine grained approach
Self-optimizing strategy
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 20 of 22 slides
Current & Future Work
Formalise the market approachIntegration with FADS [FC99]upper bound on message transmission delayadditional parameter for overload manager
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 21 of 22 slides
Thank You for Your Attention
This presentation and other related resources:
http://wwwse.inf.tu-dresden.de/haps
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer
Background & Motivation Handling Overload Evaluation Summary & Future Work 22 of 22 slides
References
Antonio Carzaniga, Matthew J. Rutherford, and Alexander L. Wolf.A routing scheme for content-based networking.In Proceedings of IEEE INFOCOM 2004, Hong Kong, China, March 2004.
Antonio Carzaniga and Alexander L. Wolf.Forwarding in a content-based network.In Proceedings of ACM SIGCOMM 2003, pages 163–174, Karlsruhe, Germany,August 2003.
Patrick Th. Eugster, Pascal A. Felber, Rachid Guerraoui, and Anne-MarieKermarrec.The many faces of publish/subscribe.ACM Comput. Surv., 35(2):114–131, 2003.
Christof Fetzer and Flaviu Christian.A fail-aware datagram service.In Iain Bate and Alan Burns, editors, IEE Proceedings - Software Engineering,volume 146, pages 58–74. IEE, April 1999.
Highly Available Publish/Subscribe Zbigniew Jerzak, Christof Fetzer