imars - incremental materialization for rdf streams (sr4ld2013)
DESCRIPTION
Presentation of IMaRS (an algorithm for the incremental maintenance of the window materialization) given at the SR4LD tutorial (ISWC 2013)TRANSCRIPT
Stream Reasoning
For Linked Data M. Balduini, J-P Calbimonte, O. Corcho,
D. Dell'Aglio, E. Della Valle, and J.Z. Pan
http://streamreasoning.org/sr4ld2013
IMaRS: Incremental
Materialization for RDF Streams Daniele Dell'Aglio – [email protected]
Emanuele Della Valle – [email protected]
http://streamreasoning.org/sr4ld2013
Share, Remix, Reuse — Legally
This work is licensed under the Creative Commons Attribution 3.0 Unported License.
Your are free:
to Share — to copy, distribute and transmit the work
to Remix — to adapt the work
Under the following conditions
Attribution — You must attribute the work by inserting – “[source http://streamreasoning.org/sr4ld2013]” at the end of
each reused slide
– a credits slide stating
- These slides are partially based on “Streaming Reasoning for Linked Data 2013” by M. Balduini, J-P Calbimonte, O. Corcho, D. Dell'Aglio, E. Della Valle, and J.Z. Pan http://streamreasoning.org/sr4ld2013
To view a copy of this license, visit http://creativecommons.org/licenses/by/3.0/
2
http://streamreasoning.org/sr4ld2013 3
Running Example – Data Model
Observation
Sensor
Person
Post
Room where
discusses
who
observes
subClassOf
subClassOf
posts
subPropOf
Streaming
information
Background
information
isWith
isIn
isConnectedTo
http://streamreasoning.org/sr4ld2013 4
Running Example – Data Model
Observation
Sensor
Person
Post
Room where
discusses
who
observes
subClassOf
subClassOf
posts
subPropOf
Streaming
information
Background
information
isWith
isIn
isConnectedTo 1 2 3
Instances
http://streamreasoning.org/sr4ld2013
What
Add reasoning in window-based RSPs
Naïve solution: materialize everything, every time
But windows slide: • The materialisation is executed every time the
window updates • Only part of data changes at each window update • Materialisation is (usually) an expensive task
4
http://streamreasoning.org/sr4ld2013
Naïve solution: an example
S 1
∈
2
2 ∈
∈
dom( ) ⊑
rng( ) ⊑
TBOX
1 ∈
5
http://streamreasoning.org/sr4ld2013
Naïve solution: an example
S 1
∈
2
2 ∈
∈
3
dom( ) ⊑
rng( ) ⊑
TBOX
1 ∈
5
http://streamreasoning.org/sr4ld2013
Naïve solution: an example
S 1 2 3
dom( ) ⊑
rng( ) ⊑
TBOX
5
http://streamreasoning.org/sr4ld2013
Naïve solution: an example
S 1 2
2 ∈
∈
3
dom( ) ⊑
rng( ) ⊑
TBOX
∈
3 ∈
5
http://streamreasoning.org/sr4ld2013
Incremental maintenance
Adopt an incremental approach
Compute only the differences that should be removed and added from the materialization
Explicit statements
Inferred statements
To
be re
mo
ved
To
be a
dd
ed
∆− ∆+
6
http://streamreasoning.org/sr4ld2013
Incremental maintenance: an example
S 1
∈
2
2 ∈
∈
dom( ) ⊑
rng( ) ⊑
TBOX
1 ∈
7
http://streamreasoning.org/sr4ld2013
Incremental maintenance: an example
S 1
∈
2
2 ∈
∈
3
dom( ) ⊑
rng( ) ⊑
TBOX
1 ∈
7
http://streamreasoning.org/sr4ld2013
Incremental maintenance: an example
S 1
∈
2
2 ∈
∈
3
dom( ) ⊑
rng( ) ⊑
TBOX
To be
deleted 1 ∈
7
http://streamreasoning.org/sr4ld2013
Incremental maintenance: an example
S 1
∈
2
2 ∈
∈
3
dom( ) ⊑
rng( ) ⊑
TBOX
To be
deleted 1 ∈
To be
added
3 ∈
7
http://streamreasoning.org/sr4ld2013
Incremental maintenance: an example
S 1
∈
2
2 ∈
∈
3
dom( ) ⊑
rng( ) ⊑
TBOX
To be
deleted 1 ∈
To be
added
3 ∈
To be
renewed
7
http://streamreasoning.org/sr4ld2013
Which technique?
The common problem in designing incremental maintenance techniques is in the management of deletions
In general it is not possible to foresee the statement deletions • DRed works with random insertions and deletions
In our setting it is possible • The window operator allow us to determine when
statements will be removed
8
http://streamreasoning.org/sr4ld2013
IMaRS
Variation of DRed for RDF streams
It pushes the maintenance algorithm in the window operator
An IMaRS window is a sliding window with four parameters: • ω: the size of the window • β: the slide of the window • T: the TBox that describes the data model • M: the mantinance program
One of the central IMaRS concepts is the expiration time
9
http://streamreasoning.org/sr4ld2013
Expiration time
Every time a statement is added to the window, it is annotated with an expiration time
The expiration time indicates when the statement should be removed from the materialization
S
2
2 ∈
∈
dom( ) ⊑
rng( ) ⊑
TBOX
∈
3 ∈
t
1
10
10
9 8 7 11 12 13
Current time
Window (2,1)
10
http://streamreasoning.org/sr4ld2013
Expiration time
Every time a statement is added to the window, it is annotated with an expiration time
The expiration time indicates when the statement should be removed from the materialization
S
2
2 ∈
∈
dom( ) ⊑
rng( ) ⊑
TBOX
∈
3 ∈
t
1
10 9 8 7 11 12 13
11
Current time
Window (2,1)
The statement
will exit at 13
10
http://streamreasoning.org/sr4ld2013
Expiration time
Every time a statement is added to the window, it is annotated with an expiration time
The expiration time indicates when the statement should be removed from the materialization
S
∈
2
2 ∈
∈
dom( ) ⊑
rng( ) ⊑
TBOX
1 ∈
∈
3 ∈
t
1
10 9 8 7 11 12 13
11
Current time
Window (2,1)
The statement
will exit at 13
The inferred
statements will
exit at 13
10
http://streamreasoning.org/sr4ld2013
Expiration time
Every time a statement is added to the window, it is annotated with an expiration time
The expiration time indicates when the statement should be removed from the materialization
S
∈
2
2 ∈
∈
dom( ) ⊑
rng( ) ⊑
TBOX
1 ∈
∈
3 ∈
t
1
10 9 8 7 11 12 13
12
Current time
Window (2,1)
10
http://streamreasoning.org/sr4ld2013
Expiration time
Every time a statement is added to the window, it is annotated with an expiration time
The expiration time indicates when the statement should be removed from the materialization
S
∈
2
2 ∈
∈
dom( ) ⊑
rng( ) ⊑
TBOX
1 ∈
∈
3 ∈
t
1
10 9 8 7 11 12 13
13
Current time
Window (2,1)
The inferred
statements
expire
10
http://streamreasoning.org/sr4ld2013
Expiration time
Every time a statement is added to the window, it is annotated with an expiration time
The expiration time indicates when the statement should be removed from the materialization
S
2
2 ∈
∈
dom( ) ⊑
rng( ) ⊑
TBOX
∈
3 ∈
t
1
10 9 8 7 11 12 13
13
Current time
Window (2,1)
The inferred
statements
expire
10
http://streamreasoning.org/sr4ld2013
At each window update
Expiration time generation
Expiration time
assignment
Expiration time
assignment
Explicit
statements
Inferred
statements
To
be
rem
oved
To
be
ad
ded
To be renewed
Expiration time
update Statement
deletion
The computation is done through the execution of a maintenance program
11
http://streamreasoning.org/sr4ld2013 25
IMaRS at a glance
Lookup
Insert + Renew
IMaRS
Delete
Rederive
Insert
DRed
http://streamreasoning.org/sr4ld2013
IMaRS maintenance program
The maintenance program computes the delta sets ∆− and ∆+
• It is a logic program
The program is executed every time the content changes • In our context, the program is executed every time
the window slides
The program is composed by maintenance rules • A maintenance rule adds a statement in a set
(context) if the preconditions are satisfied
13
http://streamreasoning.org/sr4ld2013
IMaRS contexts
The maintenance program uses four contexts to build the delta sets ∆− and ∆+
• Mat: the current materialization • Ins: the input that enters the stream and the
related inferred statements
Additionally, two support sets are used: • New: statements to be added to the
materialization • Ren: renewed statements
The new materialization is computed as
Mat U ∆+ \ ∆−
14
http://streamreasoning.org/sr4ld2013
IMaRS maintenance rules
Two examples of maintenance rules:
A triple is removed by the materialization when its expiration time expires
When a triple <s,p,o> enters the window, p is transitive and there is a triple <o,p,k> in the Ins context, then the triple <s,p,k> is a candidate for the addition in the materialization
Δ-(?s,?p,?o)[e] Mat(?s,?p,?o)[e] .
e < now
Ins(?x,?p,?z)[e] New(?x,?p,?y)[e1].
Ins(?x,?p,?y)[e2] .
Ins(?p,isA,TransitiveProperty)[e3].
e=min{e1,e2,e3}
15
http://streamreasoning.org/sr4ld2013
IMaRS maintenance program
The maintenance program is composed by two sets of maintenance rules: • One set of fixed maintenance rules • One dependent on the ontological language
The ontological language should be expressed as a set of inference rules, e.g. T(?x, ?p, ?z) :-
T(?p, rdf:type, owl:TransitiveProperty),
T(?x, ?p, ?y), T(?y, ?p, ?z)
It does not depend on the TBox!
16
http://streamreasoning.org/sr4ld2013 30
Generation of the maintenance program
Maintenance
program
generator
Ontological
language
Maintenance
program TBox
IMaRS Window
Rewriting
functions
(ω, β)
http://streamreasoning.org/sr4ld2013
Example: DRed rng( ) ⊑
tr( )
rng( ) ⊑
S
TBOX
t
Window (3,1)
10 9 8 7 11 12 13
Current time
2 1
2 1 4 1
3 2 3 4
2 1 1
2
11
Insert
DRed
18
http://streamreasoning.org/sr4ld2013
Example: DRed rng( ) ⊑
tr( )
rng( ) ⊑
S
TBOX
t
Window (3,1)
10 9 8 7 11 12 13
Current time
2 1
2 1 4 1
3 2 3 4
2 1
3 1
3 1
2
4
12
Insert
DRed
18
http://streamreasoning.org/sr4ld2013
Example: DRed rng( ) ⊑
tr( )
rng( ) ⊑
S
TBOX
t
Window (3,1)
10 9 8 7 11 12 13
Current time
2 1
2 1 4 1
3 2 3 4
2 1
3 1
3 1
2
4
13
Insert
DRed
18
http://streamreasoning.org/sr4ld2013
Example: DRed rng( ) ⊑
tr( )
rng( ) ⊑
S
TBOX
t
Window (3,1)
10 9 8 7 11 12 13
Current time
2 1
2 1 4 1
3 2 3 4
2 1
3 1
3 1
2
4
14
Insert
DRed
18
http://streamreasoning.org/sr4ld2013
Example: DRed rng( ) ⊑
tr( )
rng( ) ⊑
S
TBOX
t
Window (3,1)
10 9 8 7 11 12 13
Current time
2 1
2 1 4 1
3 2 3 4
2 1
3 1
3 1
2
4
14
Delete
Insert
DRed
18
http://streamreasoning.org/sr4ld2013
Example: DRed rng( ) ⊑
tr( )
rng( ) ⊑
S
TBOX
t
Window (3,1)
10 9 8 7 11 12 13
Current time
2 1
2 1 4 1
3 2 3 4
2 1
3 1
3 1
2
4
14
Delete
Insert
DRed
18
http://streamreasoning.org/sr4ld2013
Example: DRed rng( ) ⊑
tr( )
rng( ) ⊑
S
TBOX
t
Window (3,1)
10 9 8 7 11 12 13
Current time
2 1
2 1 4 1
3 2 3 4
2 1
3 1
3 1
2
4
14
Delete
Rederive
Insert
DRed
18
http://streamreasoning.org/sr4ld2013
Example: IMaRS rng( ) ⊑
tr( )
rng( ) ⊑
S
TBOX
t
Window (3,1)
10 9 8 7 11 12 13
11
Current time
2 1
2 1 4 1
3 2 3 4
2 1 1
2
19
http://streamreasoning.org/sr4ld2013
Example: IMaRS rng( ) ⊑
tr( )
rng( ) ⊑
S
TBOX
t
Window (3,1)
10 9 8 7 11 12 13
12
Current time
2 1
2 1 4 1
3 2 3 4
2 1
3 1
3 1
2
4
19
http://streamreasoning.org/sr4ld2013
Example: IMaRS rng( ) ⊑
tr( )
rng( ) ⊑
S
TBOX
t
Window (3,1)
10 9 8 7 11 12 13
13
Current time
2 1
2 1 4 1
3 2 3 4
2 1
3 1
3 1
2
4
19
http://streamreasoning.org/sr4ld2013
Example: IMaRS rng( ) ⊑
tr( )
rng( ) ⊑
S
TBOX
t
Window (3,1)
10 9 8 7 11 12 13
13
Current time
2 1
2 1 4 1
3 2 3 4
2 1
3 1
3 1
2
4
19
http://streamreasoning.org/sr4ld2013
Example: IMaRS rng( ) ⊑
tr( )
rng( ) ⊑
S
TBOX
t
Window (3,1)
10 9 8 7 11 12 13
13
Current time
2 1
2 1 4 1
3 2 3 4
2 1
3 1
3 1
2
4
19
http://streamreasoning.org/sr4ld2013
Example: IMaRS rng( ) ⊑
tr( )
rng( ) ⊑
S
TBOX
t
Window (3,1)
10 9 8 7 11 12 13
14
Current time
2 1
2 1 4 1
3 2 3 4
2 1
3 1
3 1
2
4
19
Stream Reasoning
For Linked Data M. Balduini, J-P Calbimonte, O. Corcho,
D. Dell'Aglio, E. Della Valle, and J.Z. Pan
http://streamreasoning.org/sr4ld2013
IMaRS: Incremental
Materialization for RDF Streams Daniele Dell'Aglio – [email protected]
Emanuele Della Valle – [email protected]