a workload-aware middleware for storing massive rdf graphs into nosql databases

A Workload-Aware Middleware for Storing Massive RDF Graphs into NoSQL Databases

Exame de Qualificação de Doutorado

Luiz Henrique Zambom Santana

Prof. Dr. Ronaldo dos Santos Melloorientador

UFSC/CTC/INE/PPGCC

Agenda● Introduction: Motivation, objectives, and contributions● Background

○ RDF○ NoSQL

● State of the Art○ Open Issues

● Rendezvous○ Storing: Fragmentation, Indexing, Partitioning, and Mapping○ Querying: Query decomposition and Caching

● Evaluation● Schedule

Introduction: Motivation● RDF is currently widespread:

○ Best buy:■ http://www.nytimes.com/external/readwriteweb/2010/07/01/01readwriteweb-how-best-bu

y-is-using-the-semantic-web-23031.html○ Globo.com:

■ https://www.slideshare.net/icaromedeiros/apresantacao-ufrj-icaro2013○ US data.gov:

■ https://www.data.gov/developers/semantic-web

https://www.slideshare.net/icaromedeiros/apresantacao-ufrj-icaro2013

https://www.slideshare.net/icaromedeiros/apresantacao-ufrj-icaro2013

https://www.data.gov/developers/semantic-web

https://www.data.gov/developers/semantic-web

Introduction: Motivation (LOD stats)

Introduction: Objectives

This PhD Thesis proposal presents Rendezvous, a middleware for storing massing RDF graphs. This

middleware includes a novel data partitioning approach, a fragmentation strategy that maps pieces of this RDF graph into NoSQL databases with different data models, and a caching structure that accelerate the querying response.

Introduction: Contributions● (i) a mapping of RDF data to the columnar, document, and key/value NoSQL

data models (SADALAGE; FOWLER, 2012)● (ii) a workload-aware partitioner based on the current graph structure and,

mainly, on the typical application workload ● (iii) a caching schema based on key/value databases for speeding up the

query response time ● (iv) an experimental evaluation that compares the current version of our

approach against two baselines (Rainbow (GU; HU; HUANG, 2015) and ScalaRDF (HU et al., )) by considering Redis, Apache Cassandra and MongoDB, the most popular key/value, columnar and document NoSQL databases, respectively


○ RDF○ NoSQL




Background: RDF and SPARQL

Background: NoSQL


○ RDF○ NoSQL




State of the Art - No NoSQL Triplestores

WARP (h-hop replication), YARS, Hexastore (multiple indexes), 4store, SPIDER, RDF-3X,

SHARD, SW-Store (vertical partition), SOLID, SPOVC (horizontal partition), and S2X

State of the Art - NoSQL Triplestores

State of the Art - NoSQL Triplestores

RDFJoin, RDFKB, Jena+HBase, Hive+HBase, CumulusRDF, Rya, Stratustore, MAPSIN, H2RDF, AMADA, Trinity.RDF,

H2RDF+, MonetDBRDF, xR2RML, W3C RDF/JSON, Rainbow, Sempala, PrestoRDF, RDFChain, Tomaszuk, Bouhali, and Laurent, Papailiou et al., and, ScalaRDF.

State of the Art - Categories

● RDF/NoSQL Converters● Polystores/Multimodel● In-memory

Rainbow (GU; HU; HUANG, 2015)

Amada (Aranda-Andújar, 2012)

State of the Art● BUGIOTTI, F. et al. Invisible glue:

scalable self-tuning multi-stores. In: Conference on Innovative Data Systems Research (CIDR). [S.l.: s.n.], 2015.

State of the Art - Open Issues● To avoid indexing all the triple component permutations● To consider workload and the usage of statistics for data partitioning● To exploit in-memory possibilities● To combine RDF storage with multiple NoSQL models


○ RDF○ NoSQL




Rendezvous: Architecture

Rendezvous: Storing● Fragmentation● Indexing● Partitioning● Mapping

Storing: Fragmentation

Storing: Indexing

Storing: Indexing - Simple queries and fragmentation

Storing: Partitioning● The partition is used when the dataset is bigger than each server capabilities

Storing: Partitioning

Rendezvous: Querying

● Query decomposition● Caching

Querying: Decomposition

Querying: DecompositionQ1: SELECT ?x WHERE {x? p5 y? . y? p2 z? .}

Q2: SELECT ?x WHERE {x? p9 y? . M p10 y? .}

D1: db.partition1.find({p5:{$exists:true}, p2:{$exists:true}}})

D2: db.partition1.find({p9:{$exists:true}, subject:M}})

Querying: DecompositionQ3: SELECT ?x WHERE { x? p1 y? . y? p2 z? . z? p3 w? . }

C1: SELECT S1,O1 FROM p1 SELECT S2,O2 FROM p2 WHERE O=S1 SELECT S3,O3 FROM p3 WHERE O=S2 AND S=D (C1)

Q4: SELECT ?x WHERE {x? p2 y? .y? p3 z? .x? p5 w? .w? p9 k? .L p11 k?.}

SQ5: SELECT ?x WHERE {x? p2 y?. y? p3 z?. x? p5 w?.}

SQ6: SELECT ?x WHERE {x? p5 w?. w? p9 k?. L p11 k?.}

Querying: DecompositionQ5: SELECT ?x WHERE { x? p1 y? . y? p2 z? . z? p3 w? . z? p5 ?k . k? p6 G . k? p7 I . k? p8 H }

P1: {k? p6 G . k? p7 I . k? p8 H }

P2: {x? p1 y? . y? p2 z? . z? p3 w?}

P3: { z? p5 ?k}

Querying: Caching

Querying: Caching (chain queries)


○ RDF○ NoSQL




Evaluation● LUBM: ontology for the University domain, synthetic RDF data scalable to any

size, and 14 extensional queries representing a variety of properties● Generated dataset with 4000 universities (around 100 GB and contains

around 500 million triples)● 12 queries with joins, all of them have at least one subject-subject join, and

six of them also have at least one subject-object join● Apache Jena version 3.2.0 with Java 1.8, and we use Redis 3.2, MongoDB

3.4.3, and Apache Cassandra 3.10● Amazon m3.xlarge spot with 7.5 GB of memory and 1 x 32 SSD capacity

Evaluation: Rendezvous vs. Rainbow

Evaluation: Rendezvous vs. ScalaRDF

Evaluation: Conclusions

● Fragments are scalable● Bigger boundaries are not necessarily related to bigger

storage size● Graph-aware partitions are better than NoSQL partitions● Near cache is fast but it makes more difficult to keep data

consistency

Evaluation: Future Work

● Compression of triples during the storage● Update and delete operations● Other NoSQL types (e.g., graph)● Better datasets


○ RDF○ NoSQL




Schedule

● Middleware development (continuously until 2018)○ Compression○ Graph database○ More complex and abstract workload awareness

● Submission of papers (continuously until 2018)○ Special Interest Group On Management of Data (SIGMOD)○ Very Large Databases (VLDB)○ IEEE Transactions on Knowledge and Data Engineering (TKDE)

● Defense of the PhD thesis (2019)

LUBM model

a workload-aware middleware for storing massive rdf graphs into nosql databases

Technology