![Page 1: MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Big Events Hans-Arno Jacobsen Middleware Systems Research Group MSRG.org](https://reader036.vdocuments.mx/reader036/viewer/2022062722/56649f315503460f94c4c82e/html5/thumbnails/1.jpg)
MIDDLEWARE SYSTEMSRESEARCH GROUP
MSRG.ORG
Big Events
Hans-Arno JacobsenMiddleware Systems Research GroupMSRG.org
![Page 2: MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Big Events Hans-Arno Jacobsen Middleware Systems Research Group MSRG.org](https://reader036.vdocuments.mx/reader036/viewer/2022062722/56649f315503460f94c4c82e/html5/thumbnails/2.jpg)
MIDDLEWARE SYSTEMSRESEARCH GROUP
MSRG.ORG
Big Event Data
![Page 3: MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Big Events Hans-Arno Jacobsen Middleware Systems Research Group MSRG.org](https://reader036.vdocuments.mx/reader036/viewer/2022062722/56649f315503460f94c4c82e/html5/thumbnails/3.jpg)
MIDDLEWARE SYSTEMSRESEARCH GROUP
MSRG.ORG
H.-A. Jacobsen
Traditional Big Data Domain vs. Rest of Universe
• There are other emerging domains with needs similar to Big Data– Smart grids– Smart cities …
My first message: There are other relevant Big
Data domains – beware!
![Page 4: MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Big Events Hans-Arno Jacobsen Middleware Systems Research Group MSRG.org](https://reader036.vdocuments.mx/reader036/viewer/2022062722/56649f315503460f94c4c82e/html5/thumbnails/4.jpg)
MIDDLEWARE SYSTEMSRESEARCH GROUP
MSRG.ORG
H.-A. Jacobsen
Smart Grids for Taming The Energy Problem
![Page 5: MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Big Events Hans-Arno Jacobsen Middleware Systems Research Group MSRG.org](https://reader036.vdocuments.mx/reader036/viewer/2022062722/56649f315503460f94c4c82e/html5/thumbnails/5.jpg)
MIDDLEWARE SYSTEMSRESEARCH GROUP
MSRG.ORG
H.-A. Jacobsen
Relevance of Smart Grids
• Increasing penetration of variable renewable energy sources like wind and solar et al.
• Paradigm shift from demand-following supply to supply-following demand
• Need for new large-scale information system infrastructure to control demand
![Page 6: MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Big Events Hans-Arno Jacobsen Middleware Systems Research Group MSRG.org](https://reader036.vdocuments.mx/reader036/viewer/2022062722/56649f315503460f94c4c82e/html5/thumbnails/6.jpg)
MIDDLEWARE SYSTEMSRESEARCH GROUP
MSRG.ORG
H.-A. Jacobsen
Distributed Generation, Flexible Loads and Energy Storage
• Come in big numbers• Show unique behavior (users, weather, equipment, …)• Have to be monitored and controlled
Big event data challenge
![Page 7: MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Big Events Hans-Arno Jacobsen Middleware Systems Research Group MSRG.org](https://reader036.vdocuments.mx/reader036/viewer/2022062722/56649f315503460f94c4c82e/html5/thumbnails/7.jpg)
MIDDLEWARE SYSTEMSRESEARCH GROUP
MSRG.ORG
H.-A. Jacobsen
Solar Photovoltaic Power Generation
• High frequency measurements required
• Several metrics of interest, many spatially distributed measurement points
1 127 253 379 505 631 757 883 1009113512611387-200
0
200
400
600
800
1000
1200
direct normal solar irradiance
minutes
watt
s/m
^2
1 121 241 361 481 601 721 841 961 10811201132105
10152025303540
air temperature
minutes
degr
ees c
elsiu
s
1 124 247 370 493 616 739 862 985 1108123113540
50
100
150
200
250
diffuse solar irradiance
minutes
watt
s/m
^2
Source: National Oceanic & Atmospheric Administration (U.S.)
~2.3 TB per year and 1k panels
![Page 8: MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Big Events Hans-Arno Jacobsen Middleware Systems Research Group MSRG.org](https://reader036.vdocuments.mx/reader036/viewer/2022062722/56649f315503460f94c4c82e/html5/thumbnails/8.jpg)
MIDDLEWARE SYSTEMSRESEARCH GROUP
MSRG.ORG
H.-A. Jacobsen
Use of PEVs as Grid Resource
-0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 0.6
-0.2
-0.15
-0.1
-0.05
0
0.05
GPS coordinates
Trip 1 Trip 2 Trip 3 Trip 4 Trip 5 Trip 6
delta longitude
dlet
a la
titud
e
• High frequency measurements required
• Important for SG applications: Continuous update of trip destination and energy level at destination
Source: Auto21 Project, University of Winnipeg
~ 0.5 TB per year and 1k vehicles
![Page 9: MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Big Events Hans-Arno Jacobsen Middleware Systems Research Group MSRG.org](https://reader036.vdocuments.mx/reader036/viewer/2022062722/56649f315503460f94c4c82e/html5/thumbnails/9.jpg)
MIDDLEWARE SYSTEMSRESEARCH GROUP
MSRG.ORG
H.-A. Jacobsen
Electric Power Consumption
0:00:00 3:39:00 7:18:00 10:57:00 14:36:00 18:15:00 21:54:00012345678
real power
time
kwatt
s
0:00:00 3:30:00 7:00:00 10:30:0014:00:0017:30:0021:00:000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
reactive power
time
kVAR
0:00:00 3:27:00 6:54:00 10:21:0013:48:0017:15:0020:42:00215220225230235240245250255
voltage
Time
Volts
• Very high frequency measurements required (e.g., for inferring device on/off events, grid stability, etc.)
• Several metrics of interest (household electricity meters, single devices, etc.)
Source: UCI Machine Learning Repository
~ 27.5 PB per year and 1k homes
![Page 10: MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Big Events Hans-Arno Jacobsen Middleware Systems Research Group MSRG.org](https://reader036.vdocuments.mx/reader036/viewer/2022062722/56649f315503460f94c4c82e/html5/thumbnails/10.jpg)
MIDDLEWARE SYSTEMSRESEARCH GROUP
MSRG.ORG
H.-A. Jacobsen
Traditional Big Data Domain vs. Rest of Universe
My second message: Detecting events in real-
time in the sea of Big Data is just as
important.
![Page 11: MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Big Events Hans-Arno Jacobsen Middleware Systems Research Group MSRG.org](https://reader036.vdocuments.mx/reader036/viewer/2022062722/56649f315503460f94c4c82e/html5/thumbnails/11.jpg)
MIDDLEWARE SYSTEMSRESEARCH GROUP
MSRG.ORGTowards Big Events
• Many non-traditional scenarios that require filtering of Big Events at large scales
• … scenarios that require filtering & storage of events at large scales
• Filtering & storage of “event streams”
• Filtering & storage of “event showers”H.-A. Jacobsen
![Page 12: MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Big Events Hans-Arno Jacobsen Middleware Systems Research Group MSRG.org](https://reader036.vdocuments.mx/reader036/viewer/2022062722/56649f315503460f94c4c82e/html5/thumbnails/12.jpg)
MIDDLEWARE SYSTEMSRESEARCH GROUP
MSRG.ORGEvent Showers vs. Event Streams
Event Showers• Partially ordered sets of events• No single event schema • Events vary in shape and size
from one to the next• Processing of many event
expressions• Tends to require support for
aggregation• Broader model & paradigm
(dissemination, matching, coordination)
Event Stream Processing• Linearly ordered event
sequences• Schema-based, single schema
per stream• Stream tuples follow schema• More single-expression
processing-based• Aggregation is a key requirement• Focused on processing
queries/expressions over event streams
H.-A. Jacobsen
![Page 13: MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Big Events Hans-Arno Jacobsen Middleware Systems Research Group MSRG.org](https://reader036.vdocuments.mx/reader036/viewer/2022062722/56649f315503460f94c4c82e/html5/thumbnails/13.jpg)
MIDDLEWARE SYSTEMSRESEARCH GROUP
MSRG.ORG
H.-A. Jacobsen
Conclusions
• Big Events are Big Data in motion
• Processing Big Data in real-time to detect events of interest is important as well
• There are other emerging application domains; let us watch out for themMy final message: Big
Data Benchmarking efforts should take this
into account.
![Page 14: MIDDLEWARE SYSTEMS RESEARCH GROUP MSRG.ORG Big Events Hans-Arno Jacobsen Middleware Systems Research Group MSRG.org](https://reader036.vdocuments.mx/reader036/viewer/2022062722/56649f315503460f94c4c82e/html5/thumbnails/14.jpg)
MIDDLEWARE SYSTEMSRESEARCH GROUP
MSRG.ORG
H.-A. Jacobsen
Acknowledgements
• C. Goebel for help with smart grid slides