enabling online analysis of notification services
DESCRIPTION
Enabling Online Analysis of Notification Services. Sebastian Salvucci. Advisor: Mariano Cilia Co-Advisor: Ricardo Orosco. Facultad de Ciencias Exactas Universidad Nacional del Centro de la Provincia de Buenos Aires. Tandil, August 2006. intro/motivation. proposed approach. architecture. - PowerPoint PPT PresentationTRANSCRIPT
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
Enabling Online Analysis of Notification ServicesSebastian Salvucci
Advisor: Mariano CiliaCo-Advisor: Ricardo Orosco
Facultad de Ciencias ExactasUniversidad Nacional del Centrode la Provincia de Buenos AiresTandil, August 2006
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
proposed approach
intro/motivation
architecture
implementation
conclusions/outlook
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
Pub/Sub Systems intro
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
NS characteristics intro
• Matching: is the process carried out by event brokers to check messages against subscriptions.• Routing: is in charge of forwarding delivering the messages to the subscribers. Different algorithms (flooding, simple routing, identity routing, covering, merging)• Other features: persistence, priorities, transactions, selectors, caching, expiration, filters, message types, …
Behavior (performance influenced by…)
• Research Projects: REBECA, Siena, Hermes, Jedi, Gryphon, Srcibe• Commercial Products: Tibco Rendezvous, WebSphereMQ, SonicMQ, Fiorano• JMS Standard API specification: Open Source Implementations (OpenJMS, JBossMQ, ActiveMQ, Joram)
Implementations (different features)
Adressing Models• channel-based• subject/topic-based• content-based
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
Centralized/Distributed intro
Centralized Distributed event broker network
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
Traditional Analysis Aproaches
Black-box analysis: data are observed and registeredat the clients, when messages are published and/or received
Tracing, log files: add extra source code finding first the right places and then modifying the code to write the relevant data into a text file.
intro
• enormous ad-hoc text files• source code abstraction level
• thick granularity does not allow to determine behavior details.
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
Motivation/Goal
Motivation
Goal
intro
• Today is difficult to analyze notification services focalizing individuals aspects of the behavior
• Few partials and proprietary solutions
• Should help to improve NS behavior and performance (research/development/tuning)
• Create an NS-independent analysis framework based on observations, metrics definitions and visual representations of them.
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
proposed approach
intro
architecture
implementation
conclusions/outlook
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
Proposed Approach proposed approach
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
Observations proposed approach
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
Observations (2) proposed approach
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
Following messages proposed approach
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
Metrics from observations proposed approach
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
Metrics Analysis
Online analysis
Post-mortem analysis
Offline analysis: replay of executionsStatic analysis: traditional queries, data mining, etc.
metric values external visualizationtechniques can be configured to represent metrics in different ways
proposed approach
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
proposed approach
intro/motivation
architecture
implementation
conclusions/outlook
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
Proposed Architecturearchitecture
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
Considerations
Visualization • Reusable and configurable visualization techniques• Metrics independence• Multi users (client)
Metrics Composition• Data in movement• Generic data transformation • High-level metrics definition• Post-mortem analysis• Global Time
Behavior Observation• Notification service independence• Influence over the NS under observation• Sending and generation mechanisms performance
Communication• Push-based communication• Many-to-many• Firewalls, security restrictions
Management• Configuration• Components orchestration
architecture
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
Behavior Observation Layer
• Adopting AOP to avoidintrusive modification ofsource code.
• AOP lets separate and modularize observation logic.
architecture
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
Behavior Observation Layer (2)
Implementation Alternatives
-static weaving
-dynamic weaving
• Enable/disable observations generation according metrics composition needs
• Sending frequency: (online, offline, time intervals, fixed amount, package size)
architecture
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
Metrics Composition Layerarchitecture
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
Compositors
time interval
observations
metric values
Generic transformation over data streams (configurable)
architecture
Implementation alternative:Stream Process Engines
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
Metrics Definition: Data Warehouse Metaphor
• High-level and intuitive modelling (from static analysis)• Generic and abstract data transformation• Different measures from the same observations• Decoupling metric composition output from visualization• Metadata for stored data
architecture
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
Metrics Composition
Configuring compositors from metrics definition
architecture
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
From Data to Visualizationarchitecture
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
Visualization
Visualization components
Visualization panel (web browser)
Implementation Alternatives
• Applets• DHTML/AJAX• SVG• Flash• VRML/X3D
architecture
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
Communication
centralized approach pub/sub approach
architecture
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
Components Orchestration
Test Run Definition• load generation scenario• metrics• visualization techniques
Test Run Recording1. Enable observations2. Deploy compositors (subs)3. Init visualization components (subs)4. Subscribe repository (all)5. Start Load generation
Test Run Reproduction1. Init visualization panel2. Stored data publication
architecture
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
proposed approach
intro
architecture
implementation
conclusions/outlook
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
Initial decisisons
HTTP-based publish/subscribe framework
Pushlets Characteristics• subject based• runs in a servlet container• very thin Java client• JavaScript client for browsers• modes stream/polling
Pushlets Limitations• scalability with mode stream• message type
HTTP streamingInstead of closing the HTTP connection after fetching an HTML page, the connection remains open while fresh data is pushed to the client
Communication Data representation
• XML data representation for observations and metric values
XML repository structure
Repository
• Adopting eXist XML native database (XQuery, XUpdate support)
implementation
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
Reusing Aspects
method signature & parameters
implementation
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
Compositors: Aggregator filter + primitives
Aggregator Filter
Primitives based filters
Enhancing Filtering in Notication Services
• Pushlet adapters• XPath primitive• Compositors creation (from metrics)• Compositors deploy/undeploy (pub/sub)
Implementations
implementation
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
Composing metrics: data transformation implementation
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
Visualization Panel
•Metrics and components from test run definition
implementation
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
Visualization Components
• Visualization components description.• Mapping between metrics and components
implementation
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
Components Orchestration implementation
Test Run Recording Test Run Reproduction
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
proposed approach
intro
architecture
implementation
conclusions/outlook
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
Conclusions
The utilization of three different kinds of components to observe the behavior(aspects), compose streams of observations (metrics) and graphically represent them (visualization techniques) as well as their loosely-coupled communication through a pub/sub service offer a high flexible framework to enable online analysis of notification services.
conclusions
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
Conclusions (2)
The use of a repository to store the data produced by aspects and metrics composition supports post-mortem (offline and static) analysis using the same architecture that was developed for online analysis.
conclusions
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
Conclusions (3)
• Independence of notification service implementations (by associating semantics of observations to behavior description instead source code)
• Not intrusive modification of the source code (by using AOP to generate observations)
• Generic and intuitive modeling of computations over data streams• Metrics decoupled from visualization • Metadata associated to stored executions (by data cube description)
conclusions
Enabling Online Analysis of Notification ServicesSebastian Salvucci - Tandil, Aug2006
Outlook
• Declarative description of observations and automatically generate the aspects involved letting choose different AOP alternatives.
• By seeking and improving the data warehouse metaphor could be possible to associate a complete and unique multidimensional analysis to data streams and historical data.
• Graphical wizards and interfaces upon XML descriptions towards a higher abstraction level for users, offering a complete analysis-oriented application (bookmarks, notes, etc..)
• Other domains (alternative data sources): software analysis, business activity monitoring, applications that rely on pub/sub systems, etc..
conclusions