linked data, cultural heritage & the karma mapping software

Linked Data & Cultural Heritage

Pedro Szekely and Craig Knoblock USC/Information Sciences Institute [email protected], [email protected]

http://isi.edu/integration/karma

February 2015

Outline

•  Problem

•  Linked Data

•  Karma

•  Reconciliation

•  Next steps

CC-By 2.0 2 USC Information Sciences Institute

CURRENT STATE OF CULTURAL HERITAGE DATA


Humans Browsing the Web Crystal Bridges

Museum ofAmerican Art

Dallas Museum of Art

IndianapolisMuseum of Art

The Metropolitan Museum of Art

National Portrait Gallery

Smithsonian American Art Museum

USC Information Sciences Institute CC-By 2.0 4

WHAT WE SEE


blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah

blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah

blah blah blah blah

blah blah blah blah blah blah blah blah blah blah

blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah

blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah

WHAT THE COMPUTER SEES


WEB PAGES ARE UNUSABLE FOR CREATING INNOVATIVE APPLICATIONS

USING THE DATA


SOLUTION: Linked Open Data

“web pages for computers”

using W3C standards for publishing data


CC-By 2.0 9

Tim Berners Lee on Linked Open Data

USC Information Sciences Institute

http://youtu.be/OM6XIICm_qo

Humans Browsing the Web Crystal Bridges








CC-By 2.0 11

RAW DATA NOW


Publish Your Raw Data Crystal Bridges








CC-By 2.0 13

Examples of Raw Data Now


https://github.com/cooperhewitt/collection

https://github.com/IMAmuseum/ima-collection

Convert Data to CRM (2 star) Crystal Bridges








Linked Museum Data (3 star) Crystal Bridges








Linked Cultural Heritage Data (4 star)


Represent Resources Using URIs

h&p://szekelys.com/family#pedro

“Pedro”

h&p://xmlns.com/foaf/0.1/firstName


Represent Information as Triples

h&p://szekelys.com/family#pedro h&p://xmlns.com/foaf/0.1/firstName

Subject Predicate

Object

The resource being described

A property of the resource

The value of the property

“Pedro”


RDF Graphs

h&p://szekelys.com/family#pedro

“Pedro”

foaf:firstName

foaf:Person rdf:type

h&p://isi.edu/~szekely

foaf:homepage


Linked Open Data


Steps to Create Linked Open Data


Steps to Create Linked Open Data •  Publish the raw data

… get the data out of the proprietary database

•  Select ontologies … that define classes and properties for our data

•  Define URI scheme … identifiers of your resources

•  Convert data to RDF … from data sources to the ontologies

•  Identify links to other Linked Data datasets … aka reconciliation, entity resolution, …


CC-By 2.0 23

CIDOC CRM


http://www.cidoc-crm.org/


CC-By 2.0 24



CC-By 2.0 25

http://edan.si.edu/saam/person-institution/8 http://edan.si.edu/saam/person-institution/8/id http://edan.si.edu/saam/person-institution/8/appellation/displayname http://edan.si.edu/saam/object/12 http://edan.si.edu/saam/object/12/title http://edan.si.edu/saam/object/12/id http://edan.si.edu/saam/object/12/acquisition http://edan.si.edu/saam/object/12/production http://edan.si.edu/saam/object/12/production/date http://edan.si.edu/saam/thesauri/nationality/American http://edan.si.edu/saam/thesauri/classification/Photography



CC-By 2.0 26



RDF Mapping Tools


TOOL SHORTCOMINGS BENEFITS custom code

labor intensive w error prone

flexible

R2RML difficult to learn w only SQL databases

W3C standard w good documentation w multiple vendors

Open Refine

no guidance w only tabular data

graphical user interface w support for reconciliation w open source

Karma university product easy to use w flexible w multiple data formats w multiple deployment databases w scalable w R2RML compatible w open source

XML/JSON

Services

Karma

SQL/CSV

BigData

RDF

JSON

…

Interactive tool for rapidly extracting, cleaning, transforming, integrating & publishing

linked data in multiple formats 28 USC Information Sciences Institute

Ontology

KARMA DEMO


http://youtu.be/h3_yiBhAJIc

Easy To Use

CC-By 2.0 30

easy to use w flexible w multiple data formats w multiple deployment databases w scalable w R2RML compatible w open source

CLEAR DEPICTION OF MAPPING


CC-By 2.0 31


LEARNS TO MAP YOUR DATA


CC-By 2.0 32


SUGGEST CORRECT ADJUSTMENTS


CC-By 2.0 33


EMBEDDED PYTHON SCRIPTING


CC-By 2.0 34


IMPORT POPULAR DATA FORMATS


CC-By 2.0 35


OUTPUT RDF IN MULTIPLE FORMATS

ntriples

JSON

AVRO

SPARQL

ElasticSearch, GitHub, …

Hadoop, BigData USC Information Sciences Institute

CC-By 2.0 36


40 million documents 1 billion triples

larger than all AAC museums combined


CC-By 2.0 37


periodic update every hour, every day

continuous update as new records come in


CC-By 2.0 38


Karma compatible with R2RML tools


CC-By 2.0 39


Karma Is Open Souce USC Information Sciences Institute

CC-By 2.0 40

URI RECONCILIATION


Multiple “John Singer Sargent” ima:Singer_Sargent_John a aac:Person ; dct:date "1856-1925" ; foaf:name "John Singer Sargent" .

saam:person_4253 a aac:Person ; saam:associatedPlace saam:SaamPlace_1357324439768t1r13950_0, saam:SaamPlace_1357324439768t1r13951_0 ; saam:constituentId "4253" ; rdaGr2:biographicalInformation “Painter. Sargent traveled …" ; rdaGr2:dateAssociatedWithThePerson "1990-10-1”, "1995-5-8" ; rdaGr2:dateOfBirth "1856-1-12" ; rdaGr2:dateOfDeath "1925-4-15" ; rdaGr2:placeOfBirth saam:SaamPlace_1357324439768t1r13952_0 ; rdaGr2:placeOfDeath saam:SaamPlace_1357324439768t1r13953_0 ; skos:altLabel "John S. Sargent" ; skos:prefLabel "John Singer Sargent" .

cb:12_4567 a aac:Person ; ont0:dateOfBirth "1879", "1885" ; ont0:dateOfDeath "1925" ; skos:prefLabel "John Singer Sargent" .

met:person_1893_3819 a aac:Person ; ont0:placeOfResidence "North and Central America", "United States" ; foaf:name "John Singer Sargent" .

dma:person_John_Singer_Sargent a aac:Person ; ont0:dateOfBirth "1856" ; ont0:dateOfDeath "1925" ; foaf:name "John Singer Sargent" .

Pedro Szekely USC Information Sciences Institute CC-By 2.0 41

John Singer Sargent ima:SaamPerson_John_Singer_Sargent a aac:Person ; dct:date "1856-1925" ; foaf:name "John Singer Sargent" .

aac:Person_4253 a aac:Person ; saam:associatedPlace saam:SaamPlace_1357324439768t1r13950_0, saam:SaamPlace_1357324439768t1r13951_0 ; saam:constituentId "4253" ; rdaGr2:biographicalInformation “Painter. Sargent traveled …" ; rdaGr2:dateAssociatedWithThePerson "1990-10-1”, "1995-5-8" ; rdaGr2:dateOfBirth "1856-1-12" ; rdaGr2:dateOfDeath "1925-4-15" ; rdaGr2:placeOfBirth saam:SaamPlace_1357324439768t1r13952_0 ; rdaGr2:placeOfDeath saam:SaamPlace_1357324439768t1r13953_0 ; skos:altLabel "John S. Sargent" ; skos:prefLabel "John Singer Sargent" .

cb:SaamPerson_John_Singer_Sargent a aac:Person ; ont0:dateOfBirth "1879", "1885" ; ont0:dateOfDeath "1925" ; skos:prefLabel "John Singer Sargent" .

met:SaamPerson_John_Singer_Sargent a aac:Person ; ont0:placeOfResidence "North and Central America", "United States" ; foaf:name "John Singer Sargent" .

dallas:SaamPerson_John_Singer_Sargent a aac:Person ; ont0:dateOfBirth "1856" ; ont0:dateOfDeath "1925" ; foaf:name "John Singer Sargent" .


Reconciled “John Singer Sargent” URIs

saam:person_4253 owl:sameAs cb:12_4567 ; owl:sameAs dma:person_John_Singer_Sargent ; owl:sameAs ima:Singer_Sargent_John ; owl:sameAs met:SaamPerson_John_Singer_Sargent ; owl:sameAs dbpedia:John_Singer_Sargent ; owl:sameAs nytimes/N49129220686803623753 ; owl:sameAs w-flick/John_Singer_Sargent ; ....


URI Reconciliation In Karma


Results of Automatic Linking

Pedro Szekely

99% are correct 6% are missing


Steps to Create Linked Open Data •  Publish the raw data







CC-By 2.0 47

TMS to CRM easy?


CC-By 2.0 48

TMS to CRM easy?


NO

COMMUNITY EFFORT •  Publish the raw data







Radical Ideas

•  ULAN in Wikipedia or Wikidata •  ULAN in GitHub •  Collection data in GitHub •  Community created CRM mappings in GitHub •  CRM in JSON-LD in GitHub •  Tools to export from TMS to GitHub


STORING AND MAINTAINING

THE DATA CC-By 2.0 51 USC Information Sciences Institute

Deployment Options


Technology Shortcomings Benefits SPARQL endpoint

low reliability, esoteric, slow

sophisticated query language

RDF dump no query capability, esoteric

flexibility: clients can download and use in applications, easy to publish

JSON-LD + ElasticSearch

restricted query language

very high performance, mainstream technology, easy to publish

Karma supports the three options

CC-By 2.0 53

federation every publishes their data with

their own URIs

aggregation aggregator repulishes everyone’s

data with new URIs


thanks for your attention!

https://github.com/usc-isi-i2/Web-Karma!Open Source, Apache 2 License!


linked data, cultural heritage & the karma mapping software

Technology

publishing data cc

steps cc

examples of raw data

open data web pages

qo humans

w3c standards

tim berners lee

innovative applications