connecting stream reasoners on the web
TRANSCRIPT
Connecting Stream Reasoners on the Web
Jean-Paul Calbimonte
Institute of Information Systems
University of Applied Sciences and Arts Western Switzerland (HES-SO Valais-Wallis)
Stream Reasoning Workshop
Berlin, 08.12.2016
@jpcik
HES-SO University of AppliedSciences and Arts
Western Switzerland
6
Streams with Linked Data?
e.g. publish data streams as RDF/Linked Data?
URIs as names of things
HTTP URIs
useful information when URI is dereferenced
Link to other URIs
users
applicationsWEB
static vs. streams
one-off vs. continuous
7
RDF Streams before RDF Streams
http://richard.cyganiak.de/2007/10/lod/
2011
Linked Sensor Data
MetOfficeAEMET
8
RDF Streams before RDF Streamsi.e. just use RDF
:observation1 rdf:type om-owl:Observation .
:observation1 om-owl:observedProperty weather:_AirTemperature .
:observation1 om-owl:procedure :sensor1 .
:observation1 om-owl:result :obsresult1 .
:observation1 om-owl:resultTime "2015-01-01T10:00:01"
:obsresult1 om-owl:floatValue 35.4 .
Plain triples
Where is the
timestamp?
:observation2 rdf:type om-owl:Observation .
:observation2 om-owl:observedProperty weather:_AirTemperature .
:observation2 om-owl:procedure :sensor1 .
:observation2 om-owl:result :obsresult2 .
:observation2 om-owl:resultTime "2015-01-01T10:00:02"
:obsresult2 om-owl:floatValue 36.4 .
What is the order
in the RDF graph?
Appended to a file?
Or to some RDF dataset?
How to store it?
9
The RSP CommunityResearch work
Many Papers
PhD Thesis
Datasets
Prototypes
Benchmarks
RDF Streams
Stream Reasoning
Complex Event Processing
Stream Query Processing
Stream Compression
Semantic Sensor Web
Man
y t
op
ics
To
ns o
f w
ork
http://www.w3.org/community/rspW3C RSP Community Group
Effort to our work on RDF stream processing
discussstandardizecombineformalizeevangelize
10
W3C RSP Documentshttps://www.w3.org/community/rsp/
http://streamreasoning.github.io/RSP-QL/RSP_Requirements_Design_Document/
Use cases
Implementations
State of the Art
Challenges & issues
Requirements
Design principles
• RDF Stream model
• RDF Stream query language
Abstract syntax RDF Streams
11
Connecting RSPs on the Web
MorphStreams
CSPARQL
s
Etalis
TrOWL
s
s SPARQL
CQELS
s
Minimal agreements: standards, serialization, interfaces
Formal models for RSPs and reasoning
Working prototypes/systems! Reactive RSPs
12
Connecting RSPs and Stream Reasoners
Stream Data Model
Stream Query Languages
Producing Streams
Consuming Streams
13
RDF Stream Data Model
14
RSP Data Model
https://github.com/streamreasoning/RSP-QL/blob/master/Semantics.md
Timestamped Graph
:g1 {:axel :isIn :RedRoom. :darko :isIn :RedRoom}
{:g1 prov:generatedAtTime "2001-10-26T21:32:52"}
Many/One-triple graphs
Multiple time predicates
Implicit timestamp
Different timestamp representations
Contemporaneity
Allows:
A RDF stream S consists of a sequence of timestamped graphs (with a partial order)
RDF Stream
:g1 {:axel :isIn :RedRoom. :darko :isIn :RedRoom} {:g1,prov:generatedAtTime,t1}
:g2 {:axel :isIn :BlueRoom. } {:g2,prov:generatedAtTime,t2}
:g3 {:minh :isIn :RedRoom. } {:g3,prov:generatedAtTime,t3} ...
Substream, time-bounded substream, window, window function, …
15
RDF Streams in W3C RSP
:g1 {:axel :isIn :RedRoom. :darko :isIn :RedRoom}
{:g1, prov:generatedAtTime, "2015-06-18T12:00:00Z"^^xsd:dateTime}
:g2 {:axel :isIn :BlueRoom. }
{:g2, prov:generatedAtTime, "2015-06-18T12:00:35"^^xsd:dateTime}
:g3 {:minh :isIn :RedRoom. }
{:g3, prov:generatedAtTime, "2015-06-18T12:02:07"^^xsd:dateTime}
...
https://www.w3.org/community/rsp/
http://streamreasoning.github.io/RSP-QL/RSP_Requirements_Design_Document/
Graph-based
Flexible time
property
RDF-friendly
Flexible
metadata
:g_1 :startsAt "2015-06-18T12:00:00"^^xsd:dateTime
:g_1 :endsAt "2015-06-18T13:00:00"^^xsd:dateTime
:g_2 :validBetween [:startsAt "2015-06-18T12:00:00"^^xsd:dateTime;
:endsAt "2015-06-18T13:00:00"^^xsd:dateTime]
Intervals
16
Query Languages
17
Similar (not equals!) query languagesSELECT ?sensor
FROM NAMED STREAM <http://www.cwi.nl/SRBench/observations> [NOW-3 HOURS SLIDE 10 MINUTES]
WHERE {
?observation om-owl:procedure ?sensor ;
om-owl:observedProperty weather:WindSpeed ;
om-owl:result [ om-owl:floatValue ?value ] . }
GROUP BY ?sensor HAVING ( AVG(?value) >= "74"^^xsd:float )
SELECT ?sensor
WHERE {
STREAM <http://www.cwi.nl/SRBench/observations> [RANGE 10800s SLIDE 600s] {
?observation om-owl:procedure ?sensor ;
om-owl:observedProperty weather:WindSpeed ;
om-owl:result [ om-owl:floatValue ?value ] .} }
GROUP BY ?sensor HAVING ( AVG(?value) >= "74"^^xsd:float )
SELECT ?sensor
FROM STREAM <http://www.cwi.nl/SRBench/observations> [RANGE 1h STEP 10m]
WHERE {
?observation om-owl:procedure ?sensor ;
om-owl:observedProperty weather:WindSpeed ;
om-owl:result [ om-owl:floatValue ?value ] . }
GROUP BY ?sensor HAVING ( AVG(?value) >= "74"^^xsd:float )
SPARQLStream
CQELS
C-SPARQL
18
RSP-QL
FROM NAMED WINDOW :win1 ON ex:social [RANGE PT10M]
FROM NAMED WINDOW :win2 ON ex:social [RANGE PT1M]
FROM NAMED WINDOW :win3 ON ex:sensors [RANGE PT1M]
SELECT ?room ?person
FROM NAMED WINDOW :win ON ex:social
WHERE {
WINDOW :win {?person :isIn ?room}
}
SELECT ?room
FROM NAMED WINDOW :win ON ex:social [RANGE PT10M SLIDE PT01M]
WHERE {
WINDOW :win { :axel :isIn ?room }
}
RSP-QL Semantics: a Unifying Query Model to Explain Heterogeneity
of RDF Stream Processing Systems. D. Dell’Aglio, E. Della Valle, J.-P.
Calbimonte, O. Corcho. International Journal on Semantic Web and
Information Systems IJSWIS, Volume 10(4). IGI Global, 2015.
Multiple windows
Extend SPARQL
Time based Windows
19
RSEP-QL
SELECT ?x ?v
FROM NAMED :s1 WIN [RANGE 9] AS :w1
FROM NAMED :s2 WIN [RANGE 2] AS :w2
EVENT ON :w1 { ?x :p ?y. } AS E1
EVENT ON :w1 { ?y :q ?z. } AS E2
EVENT ON :w1 { ?z :r ?v. } AS E3
WHERE {
WINDOW :w2 { ?x :r ?v. }
MATCH { (E1 SEQ E2) SEQ E3 }
}
A Query Model to Capture Event Pattern Matching in RDF Stream Processing
Query Languages. [pdf]
D. Dell’Aglio, M. Dao-Tran, J.-P. Calbimonte, D. Le Phuoc and E. Della Valle. In Proc. of
the 20th International Conference on Knowledge Engineering and Knowledge
ManagementEKAW 2016. Bologna, Italy, November 2016
Combine time
windows and CEP
Sequencing events
as in CEP
20
Producing RDF Streams
21
Feed an RDF Stream to a RSP engine
Ad-hoc
Conversi
on to
RDF
Live Non-RDF Streams
RDF
RDF datasets
RSP
Add (internal)
timestamp
on insertion
What is currently done in most RSPs
Continuous
additions
RDF +
timestamps
22
R2RML Mappings
2
2
:ObsValueMap
rr:subjectMap [
rr:template "http://opensense.epfl.ch/data/ObsResult_NO2_{sensor}_{time}"];
rr:predicateObjectMap [
rr:predicate qu:numericalValue;
rr:objectMap [ rr:column "no2"; rr:datatype xsd:float; ]];
rr:predicateObjectMap [
rr:predicate obs:uom;
rr:objectMap [ rr:parentTriplesMap :UnitMap; ]].
:ObservationMap
rr:subjectMap [
rr:template "http://opensense.epfl.ch/data/Obs_NO2_{sensor}_{time}"];
rr:predicateObjectMap [
rr:predicate ssn:observedProperty;
rr:objectMap [ rr:constant opensense:NO2]];
URI of subject
URI of predicate
Object: colum name
Column names in a template
Can be used for mapping both databases, CSVs, JSON, etc
23
Feed an RDF Stream to a RSP engine
Conversio
n to
RDF
Live Non-RDF Streams
RDF
RDF datasets
RSP
Add (internal)
timestamp
on insertion
Adding mappings to the data flow
Continuous
additions
RDF +
timestamps
24
An example: TripleWave
Running modesSources
TripleWave: Spreading RDF Streams on the Web. [pdf]
A. Mauri, J.-P. Calbimonte, D. Dell’Aglio, M. Balduini, M. Brambilla, E. Della Valle and K. Aberer. In Proc.
of the International Semantic Web Conference ISWC 2016. Kobe, Japan, October 2016.
25
An example: TripleWave{"type":"Feature",
"properties":{
"time":1388620046000,
"url":"http://earthquake.usgs.gov/earthquakes/eventpage/ak10992887",
"mag":1.1,
"magType":"ml",
"type":"earthquake",
"title":"M 1.1 - 117km NW of Talkeetna, Alaska"},
"geometry":{
"type":"Point",
"coordinates":[-151.6458,63.102,14.1]},
"id":"ak10992887"
}
{
"http://www.w3.org/ns/prov#generatedAtTime": "2015-06-30T16:44:59.587Z",
"@id": "http://streamreasoning.org/TripleWave/ak10992887",
"@graph": [
{ "@id": "http://streamreasoning.org/TripleWavee/ak10992887",
"@type": "http://example.org/onto/earth#Earthquake",
"url": "http://earthquake.usgs.gov/earthquakes/eventpage/ak10992887",
"location": {"@id": "http://streamreasoning.org/TripleWave/ak10992887Location"},
"description": "M 1.1 - 117km NW of Talkeetna, Alaska" },
{ "@id": "http://streamreasoning.org/TripleWave/ak10992887Location",
"@type": "https://schema.org/Place",
"longitude": "-151.6458", "latitude": "63.102" } ],
"@context": "https://schema.org/"
}
Original JSON Web
Earthquake Stream
JSON-LD RDF
Stream
26
Data stream characteristics
2
6
Data regularity• Raw data typically collected as time series
• Very regular structure.
• Patterns can be exploited
E.g. mobile NO2 sensor readings
29-02-2016T16:41:24,47,369,46.52104,6.63579
29-02-2016T16:41:34,47,358,46.52344,6.63595
29-02-2016T16:41:44,47,354,46.52632,6.63634
29-02-2016T16:41:54,47,355,46.52684,6.63729
...
Data order• Order of data is crucial
• Time is the key attribute for establishing an order among the data items.
• Important for indexing
• Enables efficient time-based selection, filtering and windowing
Timestamp Sensor Observed
Value
Coordinates
27
Consuming RDF Streams
28
Discoverable RDF Datasets
e.g. VOID, DCAT
Vocabularies
Dataset catalogs
Distribution
Endpoints
Schemas/Shapes
29
An example: TripleWave
Running modesSources
TripleWave: Spreading RDF Streams on the Web. [pdf]
A. Mauri, J.-P. Calbimonte, D. Dell’Aglio, M. Balduini, M. Brambilla, E. Della Valle and K. Aberer. In Proc.
of the International Semantic Web Conference ISWC 2016. Kobe, Japan, October 2016.
30
Stream Metadata in TripleWave
{
"@context":
{ "sld": "http://streamreasoning.org/ontologies/SLD4TripleWave#",
"generatedAt": {
"@id": "http://www.w3.org/ns/prov#generatedAtTime",
"@type": "http://www.w3.org/2001/XMLSchema#dateTime" }
},
"@type": "sld:sGraph",
"sld:streamLocation": "ws://localhost:8101/TripleWave/replay",
"sld:tBoxLocation": {"@id":"http://purl.oclc.org/NET/ssnx/ssn"},
"sld:contains": {"@list": [
{ "generatedAt": "2016-04-21T13:01:18.663Z", "@id": "tr:1461243678663" },
{ "generatedAt": "2016-04-21T13:01:19.784Z", "@id": "tr:1461243679784" }
]},
"sld:lastUpdated": "2016-04-21T13:02:06.575Z"
}
RSP vocabulary
Endpoint
Schema
Stream elements
31
Things to do…
Stream Catalog Vocabularies
Endpoint protocol interactions
• HTTP, WebSocket, MQTT
Efficient serialization
Federated query processing
Concrete implementations
Large scale evaluation
Muchas gracias!
Jean-Paul CalbimonteUniversity of Applied Sciences and Arts Western Switzerland
HES-SO Valais-Wallis
@jpcik