![Page 1: The LinkedTV Platform - Towards a Reactive Linked Media Management System](https://reader031.vdocuments.mx/reader031/viewer/2022030315/58812d0b1a28ab00438b5ce5/html5/thumbnails/1.jpg)
The LinkedTV PlatformTowards a Reactive Linked Media Management System
Jan Thomsen, Rolf Fricke, Ali Sarioglu
Condat AG, Berlin, Germany
LiME Workshop 2016,
2016/05/29 Heraklion, Greece
![Page 2: The LinkedTV Platform - Towards a Reactive Linked Media Management System](https://reader031.vdocuments.mx/reader031/viewer/2022030315/58812d0b1a28ab00438b5ce5/html5/thumbnails/2.jpg)
What is LinkedTV?
• EU FP7 Project 2012/10 – 2015/03
• „Web and TV seemlessly interlinked“
• Some technologies involved:• Video Analysis, ASR
• Media Fragments
• Annotations and enrichment
• Second Screen
• Personlization
• Partners: CERTH (GR), CONDAT (DE), CWI (NL), EURECOM (FR) , Fraunhofer (DE), Noterik (NL), STI/MODUL (AT), rbb (DE), Institut for Sound & Vision (NL), UEP (CZ), U Mons (BE), U St. Gallen (CH)
![Page 3: The LinkedTV Platform - Towards a Reactive Linked Media Management System](https://reader031.vdocuments.mx/reader031/viewer/2022030315/58812d0b1a28ab00438b5ce5/html5/thumbnails/3.jpg)
The basic LinkedTV workflow
Video SubtitleMetadata
SegmentationConcept detectionFace recognitionASRSpeaker Detection
Media Fragment URIRDF GenerationAnnotationsNER
Manual Curation(EditorTool)Adding automaticenrichments
Second ScreenHbbTV AppNews App
User TrackingUser ProfilingFiltering
![Page 4: The LinkedTV Platform - Towards a Reactive Linked Media Management System](https://reader031.vdocuments.mx/reader031/viewer/2022030315/58812d0b1a28ab00438b5ce5/html5/thumbnails/4.jpg)
IngestionService
S&VAVRO‘sTKK
rbb
Analysis
ShotChapter
Segment.
VideoOCR
ConceptDetection
ASR
FaceDetection
tv2rdfAnnotation
NERD
rbbaktuell
Alchemy
dbpediaspotlight
ExtractivLupedia Open
Calais
Saplo
SemiTags
TextRazor
Wikimeta
Yahoo!
Zemanta
tvenrich
MediaCollector
IRAPI
Europeana
Tracking UserModeling
StreamingServer
HTMLPlayer
HbbTV
Filtering
SpringfieldMultiscreen
Toolkit
AttentionTracker
LUMO
SimpleLearner
EasyMiner
LUMELiFR
Recommender
LSF
SpeakerRecognition
FaceTracking
KeywordExtraction
ObjectRe-detection
THD
InBeat/GAIN
LISp-Miner
ContextTracker
Kinect
LinkedProfiler
EXMARaLDACreation
EditorTool
2ndScreen
LeapMotion
The Linked Media Ecosystem ofLinkedTV
LinkedTVPlatform
SRTSerilazer
TVAnytimeSerilazer
ExmaraldaSerilazer
LinkedCulture
EntityExpansion
TVNewsEnricher
Topic labeling
![Page 5: The LinkedTV Platform - Towards a Reactive Linked Media Management System](https://reader031.vdocuments.mx/reader031/viewer/2022030315/58812d0b1a28ab00438b5ce5/html5/thumbnails/5.jpg)
Linked Media Management in LinkedTV
Main functional features
• Generation of Audio Visual Profiles by analyzing video and audio tracks (e.g. chapter, scenes, objects)
• Generation of addressable Media Fragments (URIs)
• Generation of structured annotations describing these media fragments
• Named Entity Recognition and linking to LOD resources
• Enrichment
• Editing
Main technical features
• Ingestion of Media resources and associated files (e.g. subtitle files)
• Workflow Management
• Linked Data Repository
• REST API for interaction and development of Linked Media Applications for Web and TV (e.g. personalized Second Screen applications)
• Framework for easy integration of further services
• Tools and APIs for administration and configuration
Linked Media Management
![Page 6: The LinkedTV Platform - Towards a Reactive Linked Media Management System](https://reader031.vdocuments.mx/reader031/viewer/2022030315/58812d0b1a28ab00438b5ce5/html5/thumbnails/6.jpg)
Linked Media Management in context
Linked MediaManagement
Media Resources
Linked Data The Web
Content Management
Content DeliveryStreaming
Linked MediaApplications
Productionand
Publishing
![Page 7: The LinkedTV Platform - Towards a Reactive Linked Media Management System](https://reader031.vdocuments.mx/reader031/viewer/2022030315/58812d0b1a28ab00438b5ce5/html5/thumbnails/7.jpg)
The LinkedTV Platform: Version 1
• LinkedTV Platform Version 1 supports the general LinkedTV workflow
• Implements a very basic form of Linked Media Management
• Only a linear workflow
• Not scalable
• Not suitable for configuration of different workflows and smooth integration of further Linked Media services
• Not reliable and robust enough for production use
• Motivation for a redesign of the LinkedTV Platform
![Page 8: The LinkedTV Platform - Towards a Reactive Linked Media Management System](https://reader031.vdocuments.mx/reader031/viewer/2022030315/58812d0b1a28ab00438b5ce5/html5/thumbnails/8.jpg)
LinkedTV Platform Version 2 – Main Properties
• Network of indepedent microservices
• Each microservice acts as a consumer and/or producer
• Most microservices connect external REST services
• Operated by a message driven architecture (AMQP)
• Built on top of RabbitMQ
dispatcher.mediaresource.update_event_status(uuid, 'transferred', dispatcher.STATUS_FINISHED)
utilities.writelog(logfile, ' [Dispatcher] Transferred ...')
message = profile + ' ' + uuid + ' ' + video
ch.basic_ack(delivery_tag = method.delivery_tag)
channel.basic_publish(exchange='',
routing_key=queue_name_to,
body=message,
properties=pika.BasicProperties(
delivery_mode = 2,
))
utilities.writelog(logfile, "
[Dispatcher] INFO Sent to RabbitMQ Exchange %r:%r:%r" % (profile, message, queue_name_to))
![Page 9: The LinkedTV Platform - Towards a Reactive Linked Media Management System](https://reader031.vdocuments.mx/reader031/viewer/2022030315/58812d0b1a28ab00438b5ce5/html5/thumbnails/9.jpg)
The LinkedTV „Tube Map“
![Page 10: The LinkedTV Platform - Towards a Reactive Linked Media Management System](https://reader031.vdocuments.mx/reader031/viewer/2022030315/58812d0b1a28ab00438b5ce5/html5/thumbnails/10.jpg)
LinkedTV Platform Version 2 – Main Benefits
• Parallel processing
• Scalable and elastic: microservices can be multiplied and distributed among different servers
• Reliable: RabbitMQ takes care of message delivery
• Extendable: very easy to add new microservices without interfering with existent ones
• Easy to customize by selecting the microservices and defining routes
• In accordance to reactive design principles: Responsiveness, Resilience, Elasticity, Message Driven*
*reactivemanifesto.org
![Page 11: The LinkedTV Platform - Towards a Reactive Linked Media Management System](https://reader031.vdocuments.mx/reader031/viewer/2022030315/58812d0b1a28ab00438b5ce5/html5/thumbnails/11.jpg)
REST API data.linkedtv.eu
![Page 12: The LinkedTV Platform - Towards a Reactive Linked Media Management System](https://reader031.vdocuments.mx/reader031/viewer/2022030315/58812d0b1a28ab00438b5ce5/html5/thumbnails/12.jpg)
![Page 13: The LinkedTV Platform - Towards a Reactive Linked Media Management System](https://reader031.vdocuments.mx/reader031/viewer/2022030315/58812d0b1a28ab00438b5ce5/html5/thumbnails/13.jpg)
Summary and Discussion
• Linked Media needs new kind of „Linked Media Management Systems“
• The LinkedTV Platform is an example of such a system
• Still lacking a lot of features necessary for production use
• Does not extend to external services
• Currently only for the production part; to be extended into the publishing and consumption areas
• Next step: test runs for getting quantitave data