freebase workshop, december 2009
DESCRIPTION
Presented at the Freebase NYC Workshop, December 2009. Covers basic MQL query construction, basic service calls and eMQL adapter construction. PHP Code can be found at http://dev.mqlx.com/~jamie/nyc2009TRANSCRIPT
Freebase WorkshopJamie TaylorNew York City, December 2009
Code available at: http://dev.mqlx.com/~jamie/nyc2009
Building a MQL Query
MQL
[{ "name" : null, "type" : "/tv/tv_program"}]
"type":"/type/object" is assumed"name" = /type/object/name
http://www.freebase.com/app/queryeditor
MQL
{"id":"/en/ncis",
"name" : null, "type" : "/tv/tv_program"}
MQL
{"id":"/en/ncis",
"name" : null, "type" : "/tv/tv_program",
"program_creator":[ ]}
"/program_creator" = /tv/tv_program/program_creator
MQL
{"id":"/en/ncis",
"name" : null, "type" : "/tv/tv_program",
"program_creator":[{ "id":null, "name":null }]}
TVPerformance
Mark Harmon
Jethro Gibbs{
"id":"/en/ncis", "name" : null, "type" : "/tv/tv_program",
"program_creator":[{ "id":null, "name:null }],"regular_cast":[{
"actor":null, "character":null
}]}
{"id":"/en/ncis",
"name" : null, "type" : "/tv/tv_program",
"program_creator":[{ "id":null, "name:null }],"regular_cast":[{
"actor":null, "character":null
}]}
TVPerformance
Mark Harmon
Jethro Gibbs{
"id":"/en/ncis", "name" : null, "type" : "/tv/tv_program",
"program_creator":[{ "id":null, "name:null }],"regular_cast":[{
"actor": {"id":null, "name":null}, "character":null
}]}
{"id":"/en/ncis",
"name" : null, "type" : "/tv/tv_program",
"program_creator":[{ "id":null, "name:null }],"regular_cast":[{
"actor": {"id":null, "name":null}, "character":null
}],"spin_offs":[ ]
}
{"id":"/en/ncis",
"name" : null, "type" : "/tv/tv_program",
"program_creator":[{ "id":null, "name:null }],"regular_cast":[{
"actor": {"id":null, "name":null}, "character":null
}],"spin_offs":[{"id":null, "name":null,
"air_date_of_first_episode":null}]}
Accessing Freebase Services via PHP
Service Requests in PHP
• Using cURL to make external service requests
$topicid = "/en/ncis";$widgeturl = "http://www.freebase.com/widget/topic?id= $topicid&mode=i&panes=image,article_props";
$ch = curl_init();curl_setopt($ch, CURLOPT_URL, $widgeturl);curl_setopt($ch, CURLOPT_HEADER, 0);curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);$resultstr = curl_exec($ch);curl_close($ch);
print $resultstr;
Accessing External Services via MQL Extensions
A Graph of Entities
A Graph of Services
http://www.myspace.com/shakira
http://www.facebook.com/shakira
http://twitter.com/shakira
http://www.daylife.com/topic/Shakira
http://www.bestbuy.com/site/She+Wolf…
http://www.guardian.co.uk/music/shakira
http://www.last.fm/music/Shakira
http://www.netflix.com/RoleDisplay/Shakira/20046629
eMQL: MQL Extensions
• Request data from other service providers• Services accessed transparently within MQL query
• Use Freebase Keys/Properties in external service request
• Uses:
• real-time data services
• Stock quotes, sensor data
• premium services
• requests can specify private "api keys" for the service
• fine grained & authoritative data sources
eMQL Query
{ "id": "/en/ibm", "type": "/business/company", "ticker_symbol": [{ "stock_exchange": null, "ticker_symbol":null }]}
ticker
"ibm"
{ "id": "/en/ibm", "type": "/business/company", "ticker_symbol": [{ "stock_exchange": null, "ticker_symbol":null, "quote":null }]}
eMQL Query
ticker
"ibm"
{ "id": "/en/ibm", "type": "/business/company", "ticker_symbol": [{ "stock_exchange": null, "ticker_symbol":null, "quote":null }]}
eMQL Query
ticker
"ibm"
"pre"
"fetch"
eMQL Service Request• Pre
• Modify MQL query to provide additional information necessary to complete request
• e.g., retrieve specific namespace keys
• Fetch• Produce (retrieve, calculate, format) value for the
eMQL property
• Reduce (optional)• Create a result that looks across all results returned
in the query
• Help
• Document how the property is used
MQL Read Service http://api.freebase.com/api/service/mqlread?query= {"query":{"id":"/en/ncis", "name":null}}
http://api.freebase.com/api/service/mqlread?queries= {"q0":{"query":{"id":"/en/ncis", "name":null}}} {"q1":{"query":{"id":"/en/blade_runner", "name":null}}}
MQL Requests in PHP
$topicid = "/en/ncis";$simplequery = array('id'=>$topicid, 'name'=>null);$queryarray = array('q1'=>array('query'=>$simplequery)); #query envelope
$jsonquerystr = json_encode($queryarray);$mqlurl = "http://www.freebase.com/api/service/mqlread?queries=". $jsonquerystr;
$ch = curl_init();curl_setopt($ch, CURLOPT_URL, $mqlurl);curl_setopt($ch, CURLOPT_HEADER, 0);curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);$resultstr = curl_exec($ch);curl_close($ch); $resultarray = json_decode($resultstr, true); #true:give us an array
$topicname = $resultarray["q1"]["result"]["name"]
{"id":"/en/ncis", "name":null}
Demo eMQL Extensions
stock quote
@tag references
TV Episode Adapter
{ "id":"/en/ncis", "/user/jamie/extension/tv_episode": null}
"name":"Faith""airdate": "Dec/15/2009"
"id":"/authority/tvrage/episode/1064868143"
Query:
DesiredData:
TV Rage Episode Adapter
Show ID@4628Show Name@NCISShow URL@http://www.tvrage.com/NCISPremiered@2003Started@Sep/23/2003Ended@Latest Episode@07x09^Child's Play^Nov/24/2009Next Episode@07x10^Faith^Dec/15/2009RFC3339@2009-12-15T20:00:00-5:00GMT+0 NODST@1260921600Country@USAStatus@Returning SeriesClassification@ScriptedGenres@Action | Crime | Drama | Military/WarNetwork@CBSAirtime@Tuesday at 08:00 pmRuntime@60
http://services.tvrage.com/tools/quickinfo.php?show=NCIS
Tim
e
TV
Ep
iso
de
Ad
ap
ter
eM
QL D
rive
r
pre
updated MQLquery
MQ
L Q
ue
ry
run MQLquery
MQLresults
fetch
Extend MQL query to retrieveTV Rage Key (program identifier)
TV
Ra
ge
Se
rvic
e
get seriesdata
next episodenumber,
name,date
nextepisode
URL
get episodedata
query dispatched
format results
results
returnqueryresults
TV RageeMQL
AdapterLife Cycle
Using Foreign Identifiers to locate Freebase Topics
• /authority Namespace• Organizations managing stable entity identifiers
• /source - identifiers only appearing in URLs
• Keys can be for URLs to other sites• And vice-versa!
URLs and Freebase Keys
• http://www.imdb.com/title/tt0083658• /authority/imdb/title/tt0083658• http://www.rottentomatoes.com/alias?type=imdbid&s=0083658
• http://dbpedia.org/resource/Blade_Runner• /wikipedia/en/Blade_Runner• http://en.wikipedia.org/wiki/Blade_Runner
• http://musicbrainz.org/artist/2c4dae8c-e591-49e0-9c5a-62b310a15788.html• /authority/musicbrainz/2c4dae8c-
e591-49e0-9c5a-62b310a15788• http://www.bbc.co.uk/music/artists/2c4dae8c-
e591-49e0-9c5a-62b310a15788
Using Freebase RDF URIs
http://rdf.freebase.com/ns/<freebase-key>
• Performs content negotiation (HTTP ACCEPT Header)
• Produces HTML for standard browsers
• Produces RDF if you ask for it
• Easily translated to MQL style Freebase identifiers
http://rdf.freebase.com/ns/en.ncis = /en/ncis
Entity Extractors
• Zemanta• Freebase RDF URIs• Wikipedia Links• IMDB identifiers
• Orchestr8• Freebase RDF URIs• DBPedia RDF URIs• MusicBrainz identifiers• Crunchbase identifiers
• OpenCalais• Freebase RDF URIs• DBPedia RDF URIs
Getting Started++• Freebase Documentation Hub
• http://www.freebase.com/docs
• Developer Mailing List• http://freebase.markmail.org/search/?q=list:com.freebase.developers
• Schema/Modeling assistance• Data Modeling Mailing List
• http://freebase.markmail.org/search/?q=list:com.freebase.data-modeling
• Real Time help on IRC• Freenode #freebase
• Freebase Happenings• http://blog.freebase.com