semantic web - meetupfiles.meetup.com/14535342/semantic web … · ppt file · web view ·...
TRANSCRIPT
A Little SPARQL in your AnalyticsDr. Neil Brittliff
Introduction The Semantic Web, as originally envisioned, is a system that enables machines to "understand" and respond to complex human requests based on their meaning. Such an "understanding" requires that the relevant information sources be semantically structured.
Tim Berners-Lee originally expressed the vision of the Semantic Web as follows:
I have a dream for the Web [in which computers] become capable of analysing all the data on the Web – the content, links, and transactions between people and computers. A "Semantic Web", which makes this possible, has yet to emerge, but when it does, the day-to-day mechanisms of trade, bureaucracy and our daily lives will be handled by machines talking to machines.
Some other thoughts…Are you a member of
the SPARQL cultAlex Karp CEO Palantir
Its about graphs not trees
Pascal HitzlerProfessor and Director of Data Science at the
Department of Computer Science and Engineering at Wright State University in Dayton, Ohio
In the six degrees of separation, not all degrees
are equal. Malcolm Gladwell, The Tipping
Point: How Little Things Can Make a Big Difference
The Talk Structure Triples and RDF
◦ What is the Big Deal
Resource Description Framework (RDF)◦ A formal way to represent information◦ Some of Nomenclature ◦ Some data structures
SPARQL ◦ A language not dissimilar to SQL but can interrogate the Semantic Web
Ontological Representations◦ A formal way to describe structure
Analytics◦ Some SNA stuff
The Triple Store Implementations Some of my stuff – Dealing with Massive RDF Lists
Data Platforms
Lets look back…CODAYSL
Hierarchial
Relational
Columnar Key/ValueDocument
Graph
The Triple
Subject Predicate ObjectResource or Blank Resource Resource, Literal or Blank
Triple
Note: The URL can identify data in the cloudor on premise.
Note: No need for NULLs
RDF• RDF - Resource Description Framework and it is a flexible schema-
less data model
• Standards based – W3C
• RDF Syntaxo N3/Turtleo XML
• Predefined RDF Structureso Bago Seqo Alto List
These are the only data structures
RDF Representation@prefix p: <http://www.example.org/personal_details#> . @prefix m: <http://www.example.org/meeting_organization#> .@prefix g: <http://www.another.example.org/geographical#>
<http://www.example.org/people#fred> p:GivenName "fred"; p:hasEmail <mailto:[email protected]>; m:attending <http://meetings.example.com/cal#m1> .
<http://meetings.example.com/cal#m1> g:Location [ g:zip "02139"; g:lat "14.124425"; g:long "14.245" ];
<http://meetings.example.com/cal#m1> m:homePage <http://meetings.example.com/m1/hp>
“14.124425".
g:zip
g:lat
g:long
“02139".
“14.245".
http://meetings.example.com/cal#m1
g:location
CURIE
RDF Property Constants
Note: A Predicate is also referred to as property used when the object is a Literal
Property Description Usagerdf:first First Element in a list rdf:Property
rdf:rest Rest of the List rdf:Property
rdf:_i List Sequence rdf:Property
rdf:nil End of the List rdf:Resource
Note: If a Predicate is a number the number value is preceded by an underscore _
Nodes and the Blank Nodes
"2010-09-29".
genid:ARP11
sf:Phone-Mobile
sf:marial-status
sf:DOB
“single".sf:mode
“car".
sf:gender
“555-1278-4343".
“F".
"2010-09-29".
sf:Phone-Mobile
sf:marial-status
sf:DOB
“single".sf:mode
“car".
sf:gender
“555-1278-4343".
“F".
genid:ARP11 p:GivenName "fred"; p:hasEmail <mailto:[email protected]>; m:attending <http://meetings.example.com/cal#m1>
Note: Blank nodes are not idempotent !!!
_:a1
RDF Structures <rdf:Description rdf:about="http://science-fiction-book/uri"> <ex:authors>
<rdf:Bag><rdf:li "Asimov"><rdf:li "Dick"><rdf:li "Heinlein">
</rdf:Seq> </ex:authors></rdf:Description
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> . @prefix d: <http://learningsparql.com/ns/data#> . d:myList d:contents _:b1 . _:b1 rdf:first "one" . _:b1 rdf:rest _:b2 . _:b2 rdf:first "two" . _:b2 rdf:rest _:b3 . _:b3 rdf:first "three" . _:b3 rdf:rest _:b4 . _:b4 rdf:first "four" . _:b4 rdf:rest _:b5 . _:b5 rdf:first "five" . _:b5 rdf:rest rdf:nil .
“one"_:b1
“two"_:b2
“three"_:b3
“four"_:b4
_:b5 “five"
rdf:nil
d:mylistrdf:first
rdf:first
rdf:first
rdf:first
rdf:first
rdf:rest
rdf:rest
rdf:rest
rdf:rest
rdf:rest
d:contents
SPARQL SPARQL (pronounced "sparkle", a recursive acronym[2]
for SPARQL Protocol and RDF Query Language) is an RDF query language, that is, a semantic query language for databases, able to retrieve and manipulate data stored in Resource Description Framework (RDF) format. It was made a standard by the RDF Data Access Working Group (DAWG) of the World Wide Web Consortium, and is recognized as one of the key technologies of the semantic web. On 15 January 2008, SPARQL 1.0 became an official W3C Recommendation, and SPARQL 1.1 in March, 2013.
Note: Source Wikipedia
The Language StructurePREFIX abc: <nul://sparql/exampleOntology#> .
SELECT ?capital ?country WHERE {
?x abc:cityname ?capital ;
<nul://sparql/exampleOntology#isCapitalOf> ?y.
?y abc:countryname ?country ;
abc:isInContinent abc:Africa.
}
CURI
Resource
Note: It is a bit like an SQL Select Statement
FullyQualified
Triple
Variable
ConstructCONSTRUCT { ?article dwc:articleTaxonName ?name }WHERE { ?x txn:hasWikipediaArticle ?article. ?x txn:scientificName ?name. ?x a txn:SpeciesConcept. ?x txn:kingdom "Plantae". }
LIMIT 10
Note: Always returns RDF (triples) !!!
DescribeDESCRIBE ?x WHERE { ?x a txn:Occurrence. ?x dcterms:date "2010-09-29".}LIMIT 10
txn:Occurrence
"2010-09-29".
Note: Describe always returns RDF !
Node to describe
AskASK { ?x a txn:Occurrence. ?x dcterms:date "2010-09-29".}
Yes or No
SelectSELECT ?person ?name ?email WHERE {
?person foaf:email ?email.
?person foaf:name ?name.
?person foaf:skill "internet".
}LIMIT 50
Note: Results are returned in a tabular format
Results from the Selectperson name email<http://www.w3.org/People/karl/karl-foaf.xrdf#me> "Karl Dubost" <mailto:[email protected]>
<http://www.w3.org/People/card#amy> "Amy van der Hiel" <mailto:[email protected]>
<http://www.w3.org/People/card#edd> "Edd Dumbill" <mailto:[email protected]>
<http://www.w3.org/People/card#dj> "Dean Jackson" <mailto:[email protected]>
<http://www.w3.org/People/card#edd> "Edd Dumbill" <mailto:[email protected]>
<http://www.aaronsw.com/about.xrdf#aaronsw> "Aaron Swartz" <mailto:[email protected]>
<http://www.w3.org/People/card#i> "Timothy Berners-Lee" <mailto:[email protected]>
<http://www.w3.org/People/EM/contact#me> "Eric Miller" <mailto:[email protected]>
<http://www.w3.org/People/card#edd> "Edd Dumbill" <mailto:[email protected]>
<http://www.w3.org/People/card#dj> "Dean Jackson" <mailto:[email protected]>
<http://www.w3.org/People/card#libby> "Libby Miller" <mailto:[email protected]>
<http://www.w3.org/People/Connolly/#me> "Dan Connolly" <mailto:[email protected]>
Select - OptionalPREFIX mo: <http://purl.org/ontology/mo/> . PREFIX foaf: <http://xmlns.com/foaf/0.1/> .
SELECT ?name ?img ?hp ?loc WHERE { ?a a mo:MusicArtist ; foaf:name ?name . OPTIONAL { ?a foaf:img ?img } OPTIONAL { ?a foaf:homepage ?hp } OPTIONAL { ?a foaf:based_near ?loc }}
The Optional Clause Result
foaf:based_near
foaf:name
foaf:based_near
foaf:img
foaf:homepage
mo:MusicArtist
foaf:name
mo:MusicArtist
foaf:based_near
foaf:name
foaf:homepage
foaf:img
foaf:img
Select - FilterPREFIX prop: <http://dbpedia.org/property/> .
ASK WHERE { <http://dbpedia.org/resource/Amazon_River> prop:length ?amazon . <http://dbpedia.org/resource/Nile> prop:length ?nile .
FILTER(?amazon > ?nile) .
}Note: Filters are applied after the results are selected
Note: Filters can appear anywhere within the SELECT statement
Alternatives - UnionPREFIX go: <http://purl.org/obo/owl/GO#> .PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> .PREFIX odo: <http://www.obofoundry.org/ro/ro.owl#> .
SELECT DISTINCT ?label ?process COUNT(*) AS ?count
WHERE { { ?process obo:part_of go:GO_0007165 }
UNION { ?process rdfs:subClassOf go:GO_0007165 }
?process rdfs:label ?label} GROUP BY ?label ORDER BY DESC(COUNT(*))
GROUP BY ?interest ORDER BY DESC(COUNT(*))
COUNT(*) AS ?count
SPARQL - UpdatePREFIX prop: <http://dbpedia.org/property/> .PREFIX dc: <http://purl.org/dc/elements/1.1/> INSERT DATA {
<http://example/book1> dc:title "A new book" ; dc:creator "A.N.Other" . }
PREFIX dc: <http://purl.org/dc/elements/1.1/> DELETE DATA {
<http://example/book2> dc:title "David Copperfield" ; dc:creator "Edmund Wells" . }
Note: You can only insert and delete triplets
Quads – The GraphCONSTRUCT { GRAPH :g { ?s :p ?o }
{ ?s :p ?o }
<http://purl.org/obo/owl/GO#> { :s ?p :o }WHERE {. ..}
Note: Can be seen as a schema in a relational database
Default Graph
URI
SPARQL and Analytics
Property PathsSyntax Form Matches
uri A URI or a prefixed name. A path of length one.
^elt Inverse path (object to subject).
(elt) A group path elt, brackets control precedence.
elt1 / elt2 A sequence path of elt1, followed by elt2
elt1 ^ elt2Shorthand for elt1 / ^elt2, that is elt1 followed by the inverse of elt2.
elt1 | elt2 A alternative path of elt1, or elt2 (all possibilities are tried).
elt* A path of zero or more occurrences of elt.
elt+ A path of one or more occurrences of elt.
elt? A path of zero or one elt.
elt{n,m} A path between n and m occurrences of elt.
elt{n} Exactly n occurrences of elt. A fixed length path.
elt{n,} n or more occurrences of elt.
elt{,n} Between 0 and n occurrences of elt.
SELECT ?value WHERE {
:list rdf:rest* [][] rdf:first ?value
}
Note: Note the use [] this tellsthe SPARQL parser that the triples share a common resource.
Property Paths cont…PREFIX d: <http://learningsparql.com/ns/data#>
SELECT ?item WHERE {
d:myList d:contents/rdf:rest{2}/rdf:first ?item
}
----------- | item | =========== | "three" | -----------
SPARQL and Rinstall.packages(c('SPARQL','igraph','network','ergm'),dependencies=TRUE)
library(SPARQL)library(igraph) library(network) library(ergm)
endpoint <- "http://live.dbpedia.org/sparql"
sparql_prefix <- "PREFIX dbp: <http://dbpedia.org/property/> PREFIX dc: <http://purl.org/dc/terms/> PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> PREFIX xsd: <http://www.w3.org/2001/XMLSchema#> "
q <- paste(sparql_prefix, 'SELECT ?actor ?movie ?director ?movie_date WHERE { ?m dc:subject <http://dbpedia.org/resource/Category:American_films> . ?m rdfs:label ?movie . FILTER(LANG(?movie) = "en") ?m dbp:released ?movie_date . FILTER(DATATYPE(?movie_date) = xsd:date) ?m dbp:starring ?a . ?a rdfs:label ?actor . FILTER(LANG(?actor) = "en") ?m dbp:director ?d . ?d rdfs:label ?director . FILTER(LANG(?director) = "en") }')
res <- SPARQL(endpoint,q,ns=prefix,extra=options)
$results
Ego-centred network measures
Ontological Representations
RDFSRDF Schema (or RDFS) defines classes and properties.
The resources in the RDFS vocabulary have URIrefs beginning with http://www.w3.org/2000/01/rdf-schema#
ex:Vehicle rdf:type rdfs:Class. ex:Car rdfs:subClassOf ex:Vehicle . ex:Van rdfs:subClassOf ex:Vehicle . ex:Truck rdfs:subClassOf ex:Vehicle . ex:MiniVan rdfs:subClassOf ex:Van . ex:MiniVan rdfs:subClassOf ex:Car .
Class
Vehicle
Van
MiniVan
Truck Car
MiniVan
OWL The Ontology Web Language (OWL) is a family of knowledge representation languages for authoring ontologies. Ontologies are a formal way to describe taxonomies and classification networks, essentially defining the structure of knowledge for various domains: the nouns representing classes of objects and the verbs representing relations between the objects.<owl:Class>
<owl:intersectionOf rdf:parseType="Collection"> <owl:Class>
<owl:oneOf rdf:parseType="Collection">
<owl:Thing rdf:about="#Tosca" />
<owl:Thing rdf:about="#Salome" />
</owl:oneOf> </owl:Class>
<owl:Class> <owl:oneOf rdf:parseType="Collection">
<owl:Thing rdf:about="#Turandot" />
<owl:Thing rdf:about="#Tosca" />
</owl:oneOf> </owl:Class>
</owl:intersectionOf> </owl:Class>
OWL is intended to be used when the information contained in documents needs to be processed by applications, as opposed to situations where the content only needs to be presented to humans. OWL can be used to explicitly represent the meaning of terms in vocabularies and the relationships between those terms. OWL has more facilities for expressing meaning and semantics than XML, RDF, and RDFS, and thus OWL goes beyond these languages in its ability to represent machine interpretable content on the Web.
Reification @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .@prefix dc: <http://purl.org/dc/elements/1.1/> .@prefix : <http://example/ns#> .
_:a rdf:subject <http://example.org/book/book1> ._:a rdf:predicate dc:title ._:a rdf:book "SPARQL" ._:a :saidBy "Alice" .
_:b rdf:subject <http://example.org/book/book1> ._:b rdf:predicate dc:title ._:b rdf:book "SPARQL Tutorial" ._:b rdf:video "SPARQL Queries" ._:b :saidBy "Bob" .
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>PREFIX dc: <http://purl.org/dc/elements/1.1/>PREFIX : <http://example/ns#>
SELECT ?book ?titleWHERE{ ?t rdf:subject ?book . ?t rdf:predicate dc:title . ?t rdf:object ?title . ?t :saidBy "Bob" .}
book title<http://example.org/book/book1> "SPARQL Tutorial"<http://example.org/book/book1> "SPARQL Queries"
Ontologies The Friend Of A Friend (FOAF) ontology
Project homepage: http://www.foaf-project.org/Namespace: http://xmlns.com/foaf/0.1/Typical prefix: foaf:Documentation: http://xmlns.com/foaf/spec/
The Dublin Core (DC) ontology
Project homepage: http://dublincore.org/Namespace: http://purl.org/dc/elements/1.1/ and http://purl.org/dc/terms/Typical prefix: dc: and dcterm:Documentation: http://dublincore.org/specifications/Description: this is a light weight RDFS vocabulary for describing generic metadata.
Ontologies cont… VCARD
Project homepage: http://www.w3.org/TR/vcard-rdf/Namespace: http://www.w3.org/2006/vcard/ns#/Typical prefix: vcard:Documentation: hp://www.w3.org/TR/vcard-rdf/
<vcard:Individual rdf:about="http://example.com/me/corky"> <vcard:fn>Corky Crystal</vcard:fn> <vcard:nickname>Corks</vcard:nickname> <vcard:hasEmail rdf:resource="mailto:[email protected]"/> </vcard:Individual>
Implementations
Aduna SesameApache Jena
TDB
Aduna Sesame Aduna Sesame
◦ Multiple back-end relational database support◦ MYSQL◦ PostgresDB◦ Oracle (Provided by Oracle)◦ Various Third Party Implementations
◦ Limited support for the property path expression◦ Not great for massive graph retrievals◦ Comes with a complient REST interface◦ Excellent Management Console
Sesame Back-ends Ontotext GraphDB™ Ontotext GraphDB™ (formerly OWLIM) is a leading RDF Triplestore built on OWL (Ontology Web
Language) standards, and fully compatible with the Sesame APIs. GraphDB handles massive loads, queries and OWL inferencing in real time. Ontotext offers three versions: GraphDB™ Lite, GraphDB™ Standard and GraphDB™ Enterprise.
CumulusRDF
CumulusRDF is an RDF store on a cloud-based architecture, fully compatible with the Sesame APIs. CumulusRDF provides a REST-based API with CRUD operations to manage RDF data. The current version uses Apache Cassandra as storage backend.
Systap Blazegraph™
Blazegraph™ (formerly known as Bigdata) is an enterprise graph database by Systap, LLC that provides a horizontally scaling, fully Sesame-compatible, storage and retrieval solution for very large volumes of RDF.
Apache JenaApache Jena
◦ Multiple back-end relational database support◦ Does support property paths◦ OK on large graph retrievals
ARQ (SPARQL) Query your RDF data using ARQ, a SPARQL 1.1compliant engine. ARQ supports
remote federated queries and free text search.
Fuseki Expose your triples as a SPARQL end-point accessible over HTTP. Fuseki provides
REST-style interaction with your RDF data.
Inference API Reason over your data to expand and check the content of your triple store.
Configure your own inference rules or use the built-in OWL and RDFS reasoners.Note: Originally developed by Hewlett Packard
AllegroGraph AllegroGraph
◦ Utilises Aduna Sesame Infrastructure◦ Very Fast◦ Supports ‘free text’◦ Supports Geospatial search
OracleHas 2 implementations◦ Has two implementations
◦ A relation back-end (part of the Spatial Pack)Not so Good (not good for large Graphs)
◦ Built on Oracle Big Data/NoSQL technology◦ Utilises Apache Jena
◦ Get Neil’s thumbs up Oracle has nearly two decades of experience working with spatial and graph database technologies. We have combined this with cutting edge research from Oracle Labs to deliver advanced analytics for the NoSQL and Hadoop platform.
Oracle Big Data Spatial and Graph- Q&A with James Steiner, VP of product management
Melli Annamalai, PhD
Bench Mark – Load Times
20 60100
140180
2200
10
20
30
40
50
60
70
80
(Systap) Bigdata(Sesame) Postgres(Oracle) Spatial(Sesame) File
Time in Minutes
Bench Mark – Retrieval Times
20 60100
140180
2200
5
10
15
20
25
30
35
40
45
(Systap) Bigdata(Sesame) Postgres(Oracle) Spatial(Sesame) File
Trip
les R
etrie
ved
– 10
00s
Time in Minutes
My Stuff
Bongo’s Goals Focus on Tabular Data
◦ Develop an efficient RDF list structure◦ Creation and Extraction
◦ Integrates with other RDF Implementations◦ Property Path expression support◦ Nice Thin and Thick GUIs with an accompanying Command Line Interface
Subject Predicate Object
Triple
Key – only for literals values not resources
Retrieval PatternsSubject Object Predicate Description
○ Any Any Retrieve all the triplets for a given subject.
○ ○ Any Retrieve all the triplets for a given object.
Any ○ Any Retrieve all the triplets for a given predicate
Any Any ○ Retrieve all triplets for a given object
Any ○ ○ Retrieve all triplets for a specific object and predicate combination
○ Any ○ Return all the triplets for a given subject and object
Any Any Any Return all the triplets contained with in a graph
○ ○ ○ Determine if a specific triple pattern exists within a graph
Bongo ArchitectureSPARQL
Neils Stuff
ARQ
Cassandra
CassandraHBASEMapDB
BongoSPARQL Engine
Bongo – Thin Client
Bongo – Fat Client
Snail
Conclusion This talk covered: RDF RDF Structures The SPARQL language SPARQL Analytical Tools
Property Paths R Integration
Ontology and Ontological Support Triple Store Implementations My Stuff
Bongo Snail
Note: Read Foundations of Semantic Web Technologies Pacscal Hitzler, Markus Krotzsch and Sebastian Rudolph
Easter Egg…
Christmas Halloween
Answer
25dec 31oct