notube slides-120130071559-phpapp01

1 New trends in television: social and semantic

Project overview and results

© NoTube project consortium. For re-use see notice at end.


TV on the Web: growing trend


TV on the Web: channel explosion


Source: Nielson Three Screen Report, March 2010


From „ Mobile Shopping Framework: The role of mobile devices in the

shopping process” by Yahoo! and the Nielson company, January 2011http://advertising.yahoo.com/industry-knowledge/mobile-shopping-insight.html

http://advertising.yahoo.com/industry-knowledge/mobile-shopping-insight.html








Including the Web in your TV

Yahoo! launches ConnectedTV platform for Web-based widgets on TV (e.g. Flickr, YouTube, facebook, twitter) – Jan 2009


Augmenting TV with the WebBlinkx BBTV makes

video information

and its textual

transcript clickable,

and links to Web

sources such as

IMDB and Wikipedia

www.blinkxbbtv.com

Also Mozilla has a

project on showing

content around

videos using HTML5

www.drumbeat.org

http://www.blinkxbbtv.com/

http://www.drumbeat.org/


Some Web-TV solutions today

Stand alone boxes such as

• TiVo – original DVR, added on-demand video, YouTube, music and photos from the Web

• Boxee – STB offering its own store of apps

• AppleTV – relaunched as $99 product tied to iTunes content, and iPhone/iPad integration

+ Hybrid boxes tied to specific IPTV providers

+ Games consoles (Sony, Microsoft, Nintendo) also adding Internet and video services to TV!


Some Web-TV solutions today

First TVs with

integrated Web

and individual

app platforms

in 2011.

Future TVs will

be „connected“

as standard. LG SmartTV, pic courtesy

http://www.wired.com/gadgetlab/2011/01/lg-smart-tv/


State of the art in TV

• TV content shifting to the Web as delivery platform

– An explosion in available content at any time

• Web content shifting to the TV as augmentation of the TV experience

– An explosion in additional content at any time


Limitations of today‘s TV

• Too much content in one place

– How to find what you want to watch? Sort between live TV, TV on demand, archives, video portals and P2P-TV

• Too much functionality at any one time

– The whole Internet while you watch TV. But what do viewers really want to be able to do additionally (parallel) to watching TV?


Social TV

• Integrate the TV experience with the so-called Social Web

– Who are my friends and what do they watch?

– What do my friends like -> maybe I‘d like it too

– Where are my friends now -> connect via the shared TV experience

• Key goal for social TV

– Enhance my TV experience through my friends‘ TV experience


Semantic TV

• Add formal semantic descriptions for

– TV programmes

– TV schedules (EPGs)

• Link those descriptions to other semantic data on the Web, cf. Linked Data

• Two key use cases for semantic TV:

– Filtering of TV content -> personalisation, recommendation

– Augmentation of TV content with Web data


NoTube project

• Integrating TV & Web with help of semantics

– Open and interlink TV content in a Web fashion with Linked Open Data

• Putting the user back in the driving seat

– Connect multitude of distributed personal data with explicit semantics

• TV is not bound to the device– Computer as TV & vice versa– Mobile device as remote control


NoTube partners


Bridging Web and TV cultures


Rest of this slideset

• Technological background (Semantic Web, Linked Data)

• Semantic annotations for TV data (semantic TV)

• Extracting knowledge from my activities and social graph (social TV)

• TV content recommendation (personalized TV)

• The further future: finally … interactive TV


“If computers can understand the

meaning behind the information

they can

learn what we are interested in

and

better help us find what we want.”*

* Source: http://www.slideshare.net/HatemMahmoud/web-30-the-semantic-web

(1) Semantic Web, Linked Data


The Semantic WebThe vision of what was termed the “Semantic Web“ first came to public

attention through an article in Scientific American in May 2001.

* Source: T. Berners-Lee, J. Hendler, O. Lassila; “The SemanticWeb”, Scientific American, 284(5):34–43, May 2001.

“The Semantic Web is not a separate

Web but an extension of the current one,

in which information is given well-defined

meaning, better enabling computers and

people to work in cooperation.”*

HTMLHTML was too limited for Web documents – it is purely a presentation

format. The tags in HTML have no meaning outside how content should be

rendered in the browser, and so the meaning of the content must be

interpreted by a human, hence excluding any possibility of machine

processing.

James Bond

MI5 

Her Majesty's Secret

Service 

Secret HQ 

007 England 

James Bond MI5

Her Majesty's

Secret

Service

Secret HQ

007 England

XML

<name>James Bond</name>

<company>

<shortname>MI5</shortname>

<fullname>Her Majesty's Secret

Service</fullname>

<address><street>Secret HQ</street>

<postcode>007</postcode>

<country>England</country>

</address>

</company>

The core idea of XML – Extensible Markup Language – is to provide for

definitions of markup which allows self-describing tags, i.e. tags which

describe the meaning of the content they mark up rather than its

presentation

James Bond MI5

Her Majesty's

Secret

Service

Secret HQ

007 England

RDFRDF provides a graph structure for making statements about things.

Individual things, and not just files, are given an URI identifier.

This is where the Semantic Web begins.

from is a child element of flight

(syntactic structure)

http://my.org/flightAI288

:from http://my.org/Vienna

:to http://my.org/Innsbruck

:dep 01-01-2009T12:00

:arr 01-01-2009T12:55

:price „88“

:currency http://my.org/euro

<flight>Flight AI288

<from>Vienna</from>-

<to>Innsbruck</to>

dep <dep>1.1. 1200</dep>

arr <arr>1.1. 1255</arr>

price <price>88€</price>

</flight>

from is a property of the resource


RDFSRDF Schema begins to formalise the meaning of things spoken about in

RDF on the basis of computational logic. RDFS permits simple ontologies

(models about concepts and their properties) to be defined, which can be

used to conclude new knowledge.

http://my.org/Vienna

is a http://my.org/City

http://my.org/City

subClass of http://my.org/PopulatedPlace


is a http://my.org/PopulatedPlace




:dep 01-01-2009T12:00

:arr 01-01-2009T12:55

:price „88“



OWLOWL broadens the possible expressivity of the ontology. This makes

richer models of knowledge about things possible, but at the cost of those

models being more complex for a computer to process.


isPlaceIn http://my.org/Austria

http://my.org/Austria

isPlaceIn http://my.org/Europe

isPlaceIn is a transitive property


isPlaceIn http://my.org/Europe




:dep 01-01-2009T12:00

:arr 01-01-2009T12:55

:price „88“


SPARQLThe final block of the Semantic Web that we will cover in this introduction is

SPARQL, the query language for semantic data using the RDF data model

(which includes OWL).




:dep 01-01-2009T12:00

:arr 01-01-2009T12:55

:price „88“


Is there a flight from Vienna to

somewhere in Austria for a price

under 100 euros?

SELECT ?flight

WHERE

?flight :from http://my.org/Vienna

?flight :to ?place

?place :isPlaceIn

http://my.org/Austria

?flight :price ?price

?flight :currency http://my.org/euro

FILTER

(?price < 100)‏


Semantic Web principles• Every concept can be identified with URIs

• Resources and relationships are typed semantically

• Partial information is acceptable

• Absolute truth is not necessary

• Evolution as a development principle


Linked Data principles

• Use URIs as names of things

• Use HTTP URIs so that people can look up those names

• When someone looks up an URI, provide useful information

• Include links to other URIs, so that they can discover more things


Semantic Web vs Linked Data

“In contrast to the full-fledged Semantic Web vision, linked data is mainly about publishing structured data in RDF using URIs rather than focusing on the ontological level or inference. This simplification - just as the Web simplified the established academic approaches of Hypertext systems -lowers the entry barrier for data providers, hence fosters a widespread adoption.”

vs

- Reference


Linked Data cloud


Linked Data for music & TV


DBPedia: Wikipedia as Linked Data


DBPedia Mobile

Pictures from revyu.com

Try yourself:

http://wiki.dbpedia.org/

DBpediaMobile


Resources and representations

http://dbpedia.org/resource/Berlin

.../page/Berlin .../data/Berlin

non-information resource

HTML representation RDF representation


Linking things, not documents

http://dbpedia.org/resource/ABBA

http://www.bbc.co.uk/music/artists/d87e52c5-

bb8d-4da8-b941-9f4928627dc8#artist

sameAs


Browsing things, not documents

http://dbpedia.org/resource/ABBA

http://dbpedia.org/resource/Knowing_Me%2C_

Knowing_You..._with_Alan_Partridge

themeMusicComposer


Asking for things, not documents

Which music artists have composed the theme music for a BBC comedy program?


(2) Semantic annotation for TV

• What can we annotate in TV?

– The program schedule

– The TV program

– TV program segments

• How can we annotate TV?

– Feature description (low level, analysis based)

– Metadata (date, creator, legal notice)

– Content description (title, summary, genre, concepts)


Why have metadata?

Archives from where content has to be found and retrieved have been the place where the need for accurate documentation first arose.


Broadcast metadata

• Data about data

– All digital resources (A/V, scripts, contracts, reports, pictures, etc.) are data

– Metadata is created at all stages in broadcasting from commissioning to playout

• Three main categories

– Administrative metadata

• Replacing project and asset management paperwork

– Technical metadata

• Format, processing, identification, location, database, network

– Descriptive metadata

• All asset related information, human readable


Need for common standards

Exchange of information

hampered by lots of proprietary

interfaces

TV Content

Creator 2

TV Content

Creator 3

TV Archive

1

TV Archive

2n+1

NoTubeBroadcaster

1

Broadcaster

2

Broadcaster

3

TV Content

Creator 1


EPGs

Screenshot http://www.ifanzy.nl

http://www.ifanzy.nl/


EPG data

• An EPG is composed of two parts: content descriptions and broadcast description

• Content descriptions contain static data about television programmes such as a brand name (e.g. EastEnders), description or plot summary, type of programme, (e.g. series, movie, news), genre(s) (e.g. drama) actors, directors, recording data, etc.

• Broadcast description is expressed by variable data, such as channel (e.g. BBC ONE), format (e.g. 16:9) and broadcast media (e.g. digital television)


TVAnytime (1/2)

• Unique document structure

– Program description

– Program location

– Program segmentation

– User description & personalisation

– System aspects

– Content rights


TVAnytime (2/2)

• Advantages of TV-Anytime

– It is network and middleware independent

– Supports related material, segmentation, locators, group information etc.

• Applications of TV-Anytime

– ARIB

– DVB (MHP, DVB GBS, DVB IPI, DVB CBMS)

– Asian User Groups, Korea

– US’ Consumer Electronic Association

– HbbTV


TVAnytime schema


Other models in use

• egtaMETA - a unique metadata exchange schema dedicated for the exchange of ads between ads agencies and broadcasters. NoTube was an early tester of the schema in its personalised advertisements use case.

• BMF – an abstract semantic model designed for metadata exchange in the professional media production domain. ARD in Germany is starting to use BMF.

• Presto Space – format generated by the project of the same name to provide for digital preservation of audiovisual collections. Used by NoTube partner RAI.


Metadata interoperability via NoTube

http://notube.tv/tv-metadata-interoperability/ for more information

http://notube.tv/tv-metadata-interoperability/






BBC /programmes

The BBC have made their EPG data machine-readable and published it on the Web


BBC /programmes: add .rdf

http://www.bbc.co.uk/program

mes/b00rl5y1

http://www.bbc.co.uk/program

mes/b00rl5y1.rdf


BBC /programmes ontology

From http://purl.org/ontology/po/

This may the first TV content

ontology, but certainly not the

last!

Key organisations in the TV

standards domain are exploring

the publication of metadata in

RDF or SKOS:

• EBU (Core)

• TV-Anytime

• IPTC (NewsML)

The final step must be a

common shared ontology

integrating the different

schemas (cf.W3C Media

Ontology and API)

http://www.bbc.co.uk/go/ontologies/programmes/2009-09-07.shtml/ext/_auto/-/http:/purl.org/ontology/po/

http://www.bbc.co.uk/go/ontologies/programmes/2009-09-07.shtml/ext/_auto/-/http:/purl.org/ontology/po/


Channel identifiers

• Collected resolvable channel identifiers together with relevant metadata in RDF, e.g. 1700+ channel identifiers of Freebase

http://www.cs.vu.nl/~ronny/notube/tv-channels.rdf





Genre taxonomies

• BBC, TV Anytime, YouTube, IMDB, tvgids.nl …

• Convert them into RDF concepts and define SKOS relations between them, e.g. EBU has done this for the TV Anytime Classification scheme


Concept extraction

• NLP tools identify named entities in text and attach an unique identifier to them

e.g. OpenCalais, Zemanta

• Focus on key classes of entity such as person, place or organisation

• Use of Linked Data for common concept identifiers

• Ontotext developed specifically for TV metadata the tool LUPedia


LUPedia (http://lupedia.ontotext.com)


Concept extraction for TV


Linking TV content to Web content

starring

David Dickinson

Tim Wonnacott

birthplace

Barnstaple


Pause


(3) Extracting knowledge about the user

Idea: generating user profiles from data the user creates on the Social Web, and in this way facilitating a personalised TV experience without an intrusive user profiling process.


Facebook, Twitter & co.


Activity Streams

• RSS/Atom feeds include a title, description, link and some other metadata;

• Activity Streams extend this with a verb and an object type

– to allow expression of intent and meaning

– to provide a means to syndicate user activities

• Supported by Facebook, MySpace, Windows Live, Google Buzz and…


Getting TV into the Social Network„ BBC iPlayer adds

Twitter and Facebook to socialise TV”

– Share what you are watching on iPlayer

– Sync viewing with friends

– Real time chat

Techcrunch Europe, May 26 2010

http://eu.techcrunch.com/2010/05/26/bbc-iplayer-adds-twitter-and-facebook-to-socialise-tv/






TV viewer actions

• Recorded

• Consumed

• Loved

• Bookmarked

• …


Twitter activity


Bringing it all together


Eurovision example

• Analyse tweets with the #eurovision tag over a set time period (during the program)

• Extract country and positive/negative remark


Getting the user‘s interests


Beancounter architecture


FOAF

• RDF based format

– Defines properties for describing a person and their relations to other people and objects

http://xmlns.com/foaf/spec/

http://xmlns.com/foaf/spec/


Weighted Interests

• Add weighting to the foaf:interest property

See http://xmlns.notu.be/wi/

http://xmlns.notu.be/wi/


FOAF as common vocabulary


Beancounter web UI


Collecting user streams


Viewer profile (1/2)


Viewer profile (2/2)


(4) TV content recommendation• Recommender strategy

– Collaborative recommendation

• You share interests with your friends

• Statistical analysis: what content is liked/watchedquantitively more by others with similar interests/history

– Content-based recommendation

• An interest in X means a potential interest in Y

• Pattern-based analysis: what content has related conceptsto the content liked/watched by you

– Hybrid recommendation

• Best of both!


NoTube recommendationapproach


Recommendation lifecycle

Graphic by Libby Miller, BBC


Linked Data recommendations

• The content-based approach:

– Identify weighted sets (patterns) of DBPediaresources from user activity objects

– Compute distance between DBPedia concepts in the user profile and in the program schedulethrough its SKOS-based categorisation scheme

– Choose the matches above a certain threshold forTV programme recommendation


User interests (DBPedia concepts)


Match user interest and TV subjects


N-Screen http://n-screen.notu.be


Get recommendations


TV recommendation calculation


So, is this the future of television?More: http://notube.tv/showcases/personalised-news/

http://notube.tv/showcases/personalised-news/




Or this?More: http://notube.tv/showcases/tv-guide-and-adaptive-ads/

http://notube.tv/showcases/tv-guide-and-adaptive-ads/










Or this?More: http://notube.tv/showcases/tv-and-the-social-web/

http://notube.tv/showcases/tv-and-the-social-web/










And in the farther future?


Interested in the project results?

Find out more online at www.notube.tv

All contents © NoTube project 2009-2012

No re-use of any slides or content of slideswithout explicit acknowledgement of:

NoTube project, www.notube.tv &

this slideset, www.notube.tv/slides

http://www.notube.tv/

http://www.notube.tv/

http://www.notube.tv/slides

notube slides-120130071559-phpapp01

Design

semantic tv

tv tv content

social tv

tv asaugmentation

augmenting tv

live tv

webtv solutions todaystand

limitations of todays