summer school: events, publish/subscribe & systems
Post on 26-Jan-2016
29 Views
Preview:
DESCRIPTION
TRANSCRIPT
Summer School: Events, Publish/Subscribe & Systems
Hans-Arno Jacobsen
On Behalf of the Organizers:Frank Eliassen & Roman Vitenberg
http://canoe.ifi.uio.no/index.php/about-summer-school
VELKOMMEN
For Your Information
• Internet access• Breakfast, lunch, dinner and breaks– 7 AM – 9 AM (Breakfast)– 12:30 (Lunch everyday including Friday)– 7:30 - 9 Monday– 6:30 - 8 Tuesday & Thursday
• Viking Experience: Monday 6 PM at the reception• Wednesday afternoon boat trip (dinner included)– Leave the hotel at 3:45– Leave in front of the city hall in Oslo at 9:15
Around the Sundvolden Hotel
Quick Introductions
• In one minute, please tell us– Who you are– From where you are– Your main research interests– What you expect from the week– Where you heard about the Summer School
(DBWORLD, EPTS, TCCC, etc.)
CANOE
• Canada-Norway Partnership in Higher Education 2008-2011– http://canoe.ifi.uio.no/
• Collaborate in research and teaching on – Distributed systems, networking, and event
processing
• Jointly organize workshops, summer schools, student exchanges, and research visits
Summer School Objectives
• Learn about events, publish/subscribe and systems
• Interact on the above topics in discussions, brainstorming sessions, and poster session
• Network and have fun
Collaborate & Provide Feedback• Collaborate Wiki– http://debs.msrg.utoronto.ca/collaborate– Login via …
• Anonymous feedback form– Feedback URL– Please use as much and as often as you’d like– Many of the questions may only apply towards the
end of the week– Session specific forms available via the Collaborate
Wiki (login required)
THE PROGRAM
Organization & Structure
• Lectures and tutorials– Listen, learn and ask
• Panel, discussion and Q/A session– Ask and participate
• Break-out sessions and social events– Engage, collaborate, and contribute
Events, Publish/Subscribe & Systems
• Events– Are the problem
• Publish/Subscribe, Streams, Rule Engines, Event Processing– Are the abstractions & the paradigms towards a
solution
• Systems– Are the tools to build viable solutions
Applications
A
B C
D
E
F
RFID and sensor networksService oriented architecture
Workflows, business processes and job scheduling
Supply chain and logistics
Event-Based
Light on
Callback received Razor SKU
Transformed
Fault
Temperature high
Loan service invoked
Ordered
Delivered
ShippedJob A done
Triggered
Other Applications• Algorithmic & automated trading• Asset tracking• Audit and compliance checking• Automated security pricing• Business activity monitoring
(BAM)• Business intelligence (BI)• Business process execution• Business activity monitoring• Capital marker surveillance• Content distribution (stock)• Fraud detection and prevention
• Intrusion detection• Information dissemination &
filtering• Location-based services• Luggage handling at airports• Network management• Order management and
advanced pricing• Operational risk assessment• RFID-based systems• Resource management• Sensor networks• System monitoring and
diagnostics
Extracted from a series of use case presentations at EPTS meetings in 2007-present.
(alphabetical order)
Observations
Events in many verticals• Automotive• Banking & Finance• Insurance• Health-care• Transportation• Military• …
Core functions required• Monitor and observe• Detect• Filter• Correlate• Infer and predict• Disseminate
Event Processing Initiatives• ACM Distributed Event-based Systems Conference (DEBS)
– http://www.debs.org/2009
• Event-based Research Portal– http://www.event-based.org
• Complex Event Processing– http://complexevents.com/
• Event Processing Technical Society (EPTS)– http://www.ep-ts.com
OUR PROGRAM
Our Program
Monday• Towards a discipline of “smart” systems by
Mani Chandy, Caltech• Pub/Sub at Google by John Reumann, GoogleTuesday• Discussion & Q/A with John Reumann• Commercial Trends in Event Processing by
Paul Vincent, TIBCO
Our Program
Tuesday (cont.’d)• Poster presentations with all of us• Event Processing with the PADRES Pub/Sub
System by Hans-Arno Jacobsen, University of Toronto
• Break-out sessions with all of us
Our Program
Wednesday• High-performance Event Stream Processing
Infrastructures and Applications by Bugra Gedik, IBM
• Real-time Data Distribution by Angelo Corsaro, PrismTech
• The Next Generation Application Server – How Event-based Processing Yields Scalability by Guy Korland, Gigaspaces
• Field Trip
Our Program
Thursday• Content Routing and Filtering in the Large Scale,
Pascal Felber, University of Neuchatel• Discussions and Reflections with Mani Chandy• Mohammed and the Mountain: Distributing the
Data or Moving the Data by Eric Jul, University of Copenhagen
• Panel: Streams vs. Rules vs. Subscriptions by Bugra Gedik
• Break-out sessions
Our Program
Friday• Scalable Hostting of Web Applications by
Pierre Guillaume, Free University Amsterdam• Break-out Session Summaries & Results with
all of us
POSTER SESSION
Poster Session: Tuesday @ 2-3:30
• Abstracts are posted on our web page• Please set up just after lunch (around 1:45)• We may use walls with hotel provided tape in
the conference room• We start the session with one minute pitches
by authors (alphabetical order)• Please line-up at the front (see web site)• Pitches are performed without slides
BREAK-OUT SESSIONS
Break-outs Organization• Meet in small groups during two slots throughout the
week (evening slots on Tuesday and Thursday)
• Adjourn in plenum on Friday to present discussion and results
• Each group is given some guiding questions, but may define their own
• Groups should be self-organizing, i.e., designate a scribe and a presenter for Friday
http://debs.msrg.utoronto.ca/collaborate
Suggested Discussion Groups
• Simulation & evaluation• Applications• Research agenda & road map• Marketing & advocacy• Practical limitations• Collaborations• Interoperability• Management issues
Others ?
We will discuss the details in the plenum on Tuesdayevening at 8 PM, followed by the group discussions.
We would like tocharter at least
6 groups.
DISCUSSIONS
Discussion Sessions • Tuesday morning with John Reumann about
Events, Pub/Sub and Systems Research Challenges at Google
• Thursday morning with Mani Chandy as Discussion and Reflections about What we Heard and Learned so far
• Thursday afternoon with Bugra Gedic as panel discussion on Streams vs. Rules vs. Subscriptions
Fundamental Questions
• What abstractions support these core functions?• What models, formalisms techniques, algorithms
underlie these abstractions & core functions?• How are these abstractions used? • What abstractions can best be used to build event
processing requiring applications?– Databases? – Messaging middleware? – Transaction processing systems?– ...
• Is there a need for developing an Event Management System?
In analogy to a Database Management System for data.
My Advice
• A Summer School is not a conference
• Please ask questions, any questions
• We want to learn from each other
• Bring yourself in
What is an event?
• Etymology
– Latin: eventus;
• evenire to happen;
• e- + venire to come
Various Interpretations• A postulated outcome, something that follows as a
result or consequence, condition, or eventuality; “in the event that I am not there, you may …”
• Something that happens, a noteworthy happening, an occurrence
• Any of the contests in a program of sports, a social occasion or activity, a world event
• The fundamental entity of observed physical reality represented by a point designated by three coordinates of place and one of time in the space-time continuum postulated by the theory of relativity
• A subset of the possible outcomes of an experiment (probabilities)
Event – My Working Definition• An event is an instantaneous and asynchronous state transition in
the environment of interest to applications, systems, users.• Event versus event occurrence
– Like type or class versus instance• In my definition events do not extend over time
– Interval events are defined by starting and ending events (a.k.a. composite events)
• My definition does not capture world events– Maybe important for a theory of events, but less so for the
effective processing of events• My definition does not capture interval events
– Events associated with a time span, but those can be modeled as composite events
Synonymously Used
The terms• Event• Event occurrence• (Event) message• Event objectare often used synonymously in practice.• Also the pattern for detecting an event is
often simply referred to as event.
Similarly, …
• Event source• Event producer• Event generation• Event publisher
• Sink• Consumer• Capture• Subscriber
Event Processing• Event Processing is an emerging area and discipline that aims
to define and develop– Abstractions– Architectures– Systems – Languages– Patterns– Models– Standards
for the processing of events in systems, applications, and by end users.
A Note on CEP
• Complex Event Processing (CEP)• Event processing must not be complex– Or else, a marketing nightmare, I suspect
• Meant was really Complex-event Processing• Not the complex processing of events
• Let’s just keep to Event Processing and try to make it as simple as possible,
A.k.a.
• Event Processing• Complex Event Processing & CEP• Business Event Processing• Intelligent Event Processing• Event Stream Processing
Marketing Event Processing & CEP are widely used, as opposed to some of the other terms. I am sure there will be more.
Event Processing Models (1970-present)
• Rule-based event processing• Active databases• Publish/Subscribe• Event stream processing (ESP)• Programming language embedded & library
Event Processing Languages
• Rules • ECA rules • Database triggers • Subscriptions• XPath• SQL with stream processing extensions• Graphical notations
Emerging StandardsDissemination & Filtering• OMG DDS• WS Notifications• WS Eventing• INFO-D• AMQP• CORBA Events &
NotificationsAPIs & Formats• CEI & CBE• WSDM & WEF
Languages• StreamSQL• OMG Production Rule
Representation (PRR) and W3C RIF
• OMG Event Metamodel and Profile (EMP)
EP Initiatives• ACM Distributed Event-based Systems Conference (DEBS)
– http://www.debs.org/2009
• Event-based Research Portal– http://www.event-based.org
• Complex Event Processing– http://complexevents.com/
• Event Processing Technical Society (EPTS)– http://www.ep-ts.com
Events in Industry I
• Mostly pre-standardization today• Pre-competitive collaborations• Efforts revolve around activities of EPTS– Gathering of use cases & requirements– Definition of glossary (vocabulary)– Reference architecture– Event processing meta model– Event processing language
• In some related areas standardization started
Events in Industry II
• Predominant trends– Stream-based processing of events through
extensions of SQL• Most dominant database vendors
– Rule-based processing of events through various rule formalisms• Many traditional middleware vendors
– Programming language embedded– Scripted and proprietary
There is no one dominant approach today.
Events in Industry III• Relational algebra, SQL, and SQL extensions– EP uses and builds on stream processing efforts– Industry adoption of SQL is well known– Industry adoption of SQL extensions is not clear at this
state; no de facto standard yet either• Rules– Re-emerging interest in rule-based processing– Intense recent efforts in standardizing
• Flexible rule markup languages• Rule interchange formats
– Industry adoption of standards is not clear at this stage
Predication: We will see many more language dialects until in a few years timea de facto standard emerges. Most likely the approach taken by the surviving ordominant player.
Events in Industry IV
• Efforts underway at (alphabetical)– IBM, Oracle (& BEA), Tibco, – and many more, sorry for omissions
• Start-ups (alphabetical)– Coral8, EventZero, StreamBase– and many more, sorry for omissions
There was/is intense activity – let’s hope recent world events don’t interfere.
Events in Academia I
• Research on events in various sub-areas of computer science and others– Databases
• ECA rules & trigger management • Stream processing• Publish/Subscribe
– Networking• Event correlation and network management
– Systems• Publish/Subscribe• SEDA (staged event-driven architectures)
Events in Academia II
• Distributed systems– Event distribution and
dissemination– Content-based routing– Publish/Subscribe
• Software engineering– Publish/Subscribe
• Embedded systems– Formalization of event-driven
systems– Event-driven programming
• Programming languages– Programming with events– Events and types
• Philosophy– Notion of events
• Events are broadly recognized.
• There is no unifying theory of events.
Events in Academia III
• The DEBS Conference aims to establish a forum that brings together disparate research revolving around events.
• Started in 2002 as workshop until 2006• From 2007 until present an independent
conference.• Obtained ACM sponsorship in 2009.
ACM International Conference on Distributed Event-based Systems.
Data Management versus Event Processing
Data management• Relational algebra & theory• SQL• Started decades ago• 35+ year strong research
community (VLDB, SIGMOD, ICDE et al.)
• Industry community• Several key players and many
players catering to niches• Many textbooks
Event processing• Requires a foundation & theory• … a common language• Recently started• Young and vibrant research
community that’s growing (DEBS)
• EPTS• Interest from key players• Many small emerging players• Essentially none (emerging)
Summary
• Event processing cuts across domains
• Event processing is not new
• New are the concentrated efforts in industry and in academia to develop event processing systems
• Event processing as sub-area of computer science is young and needs time to mature
top related