virtual campfire –a hero’s journey into multimedia storytelling

Lehrstuhl Informatik 5(Informationssysteme)

Prof. Dr. M. JarkeI5-RK-0909-1

TeLLNet

Ralf KlammaKolloquium Universität Klagenfurt,

29. Januar, 2010

Virtual Campfire –A Hero’s Journey

into Multimedia Storytelling

With thanks to Joseph Campbell: The Hero with a Thousand Faces. Novato, California: New World Library, 2008



TeLLNet

Agenda

UMIC

at R

WTH

Aac

hen

Stor

ytellin

g

Virtu

al Ca

mpfire

Conc

lusion

s and

Outl

ook

WALL·E – Der Letzte räumt die Erde auf

WALL-E: A Hero's Journey.



TeLLNet

RWTH Aachen University

• 1,250 spin-off businesses have created around 30,000 jobs in the greater Aachen region over the past 20 years.

• IDEA League

• Germany’s Excellence Initiative: 3 clusters of excellence, a graduate school and the institutional strategy “RWTH Aachen 2020: Meeting Global Challenges”

• 260 institutes in 9 faculties as Europe’s leading institutions for science and research

• Currently around 31,400 students are enrolled in over 100 academic programs

• Over 5,000 of them are international students hailing from 120 different countries



TeLLNet

Ultra high-speed Mobile Information and Communication (UMIC)

Excellence Cluster funded by DFG (2007-2011)

More than 15 institutes fromElectrical Engineering andComputer Science

New professorships andresearch groups

Own new building

http://www.umic.rwth-aachen.de



TeLLNet

Community Information Systems Research Group

Established at DBIS chair, RWTH Aachen University 9 Phd students & researchers 10-15 paid student workers & thesis workers



TeLLNet

Storytelling

Ancient concept to stimulate

knowledge sharing and learning

How can storytelling be utilized for

knowledge sharing in mobile

professional communities?

• Different roles• Different contexts• Different

semantics



TeLLNet

Why Virtual Campfire?

Media is witness of cultural heritage

– Heterogeneous communities– Media becomes only research

source when the physical object asprimary source gets lost

Reconstruction? How?

Knowledge may get lost, if no community / individual is telling stories any moreCommunities of practice are groups of people who share a concern or a passion for something they do and who interact regularly to learn how to do it better (Wenger, 1998)



TeLLNet

Non-linear Multimedia Storytelling

Creation of non-linear multimedia storiesDefinition of story paths compliant to Movement Oriented Design (Victoria University)Linkage of story elements with multimedia contents



TeLLNet

UMIC Subproject Virtual Campfire

Context-aware Semantic Multimedia Services for Mobile Communities

1.Context

awareness of mobile

communities via location

based multimedia

services

2.A bridge between

multimedia semantics

and multimedia context via metadata

management

3.Monitoring of

mobile communities

4.Mobile

multimedia webservices

5.Social

network analysis of multimedia interaction



TeLLNet

Context awareness of mobile communities

via location based multimedia services



TeLLNet

Car Race ScenarioContext-Aware Multimedia Sharing

Communities produce and consume media at different places, times, etc. via LBS Semantic mapping of context data on semantic base types needed

– GPS data -> Tribunes, Curves, etc. (Places)– Time data -> Lap, Start, Finish, Box stop, etc. (Events)– Racing car data –> Ferrari, McLaren-Mercedes, BMW-Sauber Community, etc. (Agents)

Multimedia queries are dependent on context!

IJCSSE, May 2010



TeLLNet

Multimedia Semantics and Multimedia Context

Multimedia semantics

Meaning

Richness

Precision

Machine readability

Multimedia context

Computing context

User context

Physical context

Time/ location

Community Context

Context in computer science: situational or environmental information [Abowd et al., 1999 ]

Context awareness– For multimedia input– For multimedia search– For multimedia retrieval

Information richness [Daft & Lengel, 1986]

Problem: multimedia semantics and multimedia context are not well combined for mobile multimedia communities



TeLLNet

The Multimedia Community Processing Model

MPEG-7

Dublin Core

TV-Anytime

EXIF

...

Ontology A(Commsonomy)

RDF

RDFS

OWLOntology B

SPARQL

Domain Information

Multimedia resources

More precise multimedia

query results

Community of Practice (CoP)

Context

Yiwei Cao, Ralf Klamma, Dejan Kovachev: Multimedia Processing on Multimedia Semantics and Multimedia Context , in Proc. SeMuDaTe'09,, Graz, Austria, December, 2009



TeLLNet

A bridge between multimedia semantics and multimedia context

via metadata management



TeLLNet

Cross-Media Annotation Services Collaborative indexing based on:

AgentEventConceptObjectPlaceTimeState

– Free text annotations (a lá Flickr)

– Semantic MPEG-7 Basetypes annotations

– Image annotations– Community aware

annotations: folksonomies



TeLLNet

Concepts of CA3M Community-Aware Static Multimedia

Adaptation based on Clustering Algorithms [KSCa06]

Context-Aware Adaptation based on Semantic Einrichment of Multimedia Information [CKHJ08]

How to support MPEG-7/MPEG-21 Semantic Enrichment for further Semantic Processing?

How to systematically handle Uncertain Information in furtherSemantic Processing?

Context-AwareAdaptation

System(OWL)

[CKHJ08, MDM‘08]

Media AdaptationSystem

(MPEG-7/21)

[KSCa06, ECTEL‘06]

MPEG-7ToRDF

Context Reasoner[CKHJ08, MDM‘08]

Uncertainty Management



TeLLNet

Context Reasoner Reasoning on context information SPARQL

– RDF Query Language– Ontology Reasoning Language– Implementation using the Jena Framework


Context Reasoner [CKHJ08]

MPEG-7ToRDF

Media Adaptation

System(MPEG-7/21)

[KSCa06]

Context-Aware

AdaptationSystem(OWL)

[CKHJ08]

PREFIX mpeg7: <http://manet.informatik.rwth-aachen.de/~khodaei/mpeg7-v2.owl#>PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>SELECT ?agent ?placeWHERE {

?agent mpeg7:name “FirstName LastName"?agent mpeg7:hasGeographicPosition ?loc1?loc1 mpeg7:longitude ?lon1?loc1 mpeg7:latitude ?lat1?place mpeg7:hasGeographicPosition ?loc2?place mpeg7:definition “curve-n”?loc2 mpeg7:longitude ?lon2?loc2 mpeg7:latitude ?lat2FILTER Distance(?loc1, ?loc2) <= distance

}



TeLLNet

Uncertainty Management Clustering Algorithms based on User Preferences

– Calculation Euclidean Distance of Communities– Clusters are recalculated when new users come

Rating/Ranking as User Query Feedback


Context Reasoner [CKHJ08]

MPEG-7ToRDF

Media Adaptation

System(MPEG-7/21)

[KSCa06]

Context-Aware

AdaptationSystem(OWL)

[CKHJ08]



TeLLNet

Virtual Campfire ExtentionsAn interactive semantic video annotation platformMetadata standard based video tagging on various levels MPEG-7 basedGIS enabledGranulation from frames to videos(Semi-) professional communities



TeLLNet

Monitoring of mobile communities



TeLLNet Data SourcesContext-Aware Services

Community Engine

How Virtual Campfire Works –Lightweight Application Server LAS

WWW

MPEG-7 Services

Context Services

UserManager

Map Services

StorytellingService

Object Manager

(Mobile) Interfaces

SessionManager

SNA Services

Multimedia Processor

MultimediaAnnotation

MultimediaExtractor

MultimediaAdaptation

MultimediaPlayer

Metadata

DatabasesConnectors: H

TTP, SO

AP, XM

PP

MultimediaRepository

MediaCreation

MediaSearch

MediaTagging

Semantic Browsing

Mashups

Automatic Discovery &

Configuration

MultimediaInput

Data Access

MultimediaRepositoryMultimediaRepository

Invoking services Data flows



TeLLNet

Mobile Multimedia Community Service Testbed

Dominik Renzel, Ralf Klamma :Semantic Monitoring and Analyzing Context-aware Collaborative Multimedia Services in Proceedings of the 2009 IEEE International Conference on Semantic Computing, 14-16 September 2009 / Berkeley, CA, USA



TeLLNet

youTell: A Web 2.0 Service for Collaborative Storytelling

Ralf Klamma, Yiwei Cao, Matthias Jarke: Storytelling on the Web 2.0 as a New Means of Creating Artsin Borko Furht (Ed.): Handbook of Multimedia for Digital Entertainment and Arts, Springer, pp. 623-650, 2009



TeLLNet

Story-tellling Expert Finding

KeywordsExpert values

Knowledge Value of Community sorted by keywords# R

ecom

mend

ation

s

ExpertAmateur

Measuring Development of Knowledge Value over Time



TeLLNet

Mobile multimedia web services



TeLLNet

Mobile Interfaces toMultimedia Web Services



TeLLNet

Alternative System Architecture

LAS access with Mobile Web Services

LAS access via Enterprise ServiceBus (ESB)

Yiwei Cao, Matthias Jarke, Ralf Klamma, O. Mendoza, Satish Srirama: Mobile Access to MPEG-7 Based Multimedia Services in Proc. of the Tenth International Conference on Mobile Data Management, May 18-20, 2009, Taipei, Taiwan, pp. 102-111



TeLLNet

Evaluation Results Multimedia files are retrieved on search request on mobile devices MPEG-7 based multimedia services are monitored by MobSOS



TeLLNet

Lessons learnt for Mobile Multimedia Webservices

Design and implementation of alternative architecture

Interoperability and scalability are crucial for mobile multimedia services across various mobile platforms

Load balancing can reduce wireless data communication cost and improve mobile multimedia service quality

Quality of services can be traced by service monitoring

Factors Mobile Web Services ESB as middlewarePerformance Response time is proportional to multi-

media file size and increases exponentiallyResponse time increases linearly at a low rate

Scalability Not scalable, only one LAS server is connected

Scalable, distributed access to LAS services

Developmentcost

Details must be known by clients WSDL is used to publish and find Web services on the ESB

Reliability Long response time by Web Services overload

Short response time



TeLLNet

Social network analysis of multimedia interaction



TeLLNet

Reflective Information Systems as an Architectural Foundation

Communities of Practice Media Networks

Community Information Systems

Agent-oriented RE

Community IS Design

Participatory Design

CommunitySelf-Observation

CommunitySelf-Modeling

Operational SupportCan we support CoPs with the collaborative creation of complex multimedia objects?

Can CoPs make use of metadata over the frontiers of media and standards?

Can we support CoPs by personalized knowledge management and networking strategies in Social Software?

How do adaptive, mobile web-based interfaces for CoPs look like?

Reflective SupportCan CoPs continuously elicitate and implement requirements? How much computer science support is needed?

Can CoPs learn meaningful social interaction and make use of disturbances?

How can CoPs record their complex their complex media learning traces and how they can deal with them?

Can CoPs maintain or even improve their agency (Learning, Researching, Working) in the Web 2.0?

Actor-Network Theory

SocialNetwork Analysis

Game Theory



TeLLNet

Solution idea for Reflective Support:Cross-Media Social Network Analysis Interdisciplinary multidimensional model of digital networks

– Social network analysis (SNA) is defining measures for social relations

– Actor network theory (ANT) is connecting human and media agents– I* framework is defining strategic goals and dependencies– Theory of media transcriptions is studying cross-media knowledge

social softwareWiki, Blog, Podcast, IM, Chat, Email, Newsgroup, Chat …

i*-Dependencies(Structural, Cross-media)

Members(Social Network Analysis: Centrality,

Efficiency)

network of artifactsMicrocontent, Blog entry, Message, Burst, Thread,

Comment, Conversation, Feedback (Rating)

network of members

Communities of practice

Media Networks



TeLLNet

MediaBase Collection of Social Software

artifacts with parameterized PERL scripts– Mailing lists– Newsletter– Web sites– RSS Feeds– Blogs

Database support by IBM DB2, eXist, Oracle, ...

Web Interface based on Plone/Zope, LAS, ...

Strategies of visualization– Tree maps– Cross-media graphs

Klamma et al.: Pattern-Based Cross Media Social Network Analysis for Technology Enhanced Learning in Europe, EC-TEL 2006



TeLLNet

Evolution of a Database Research Community

VLDB 1990 VLDB 1995

VLDB 2000 VLDB 2006



TeLLNet

Conclusions & OutlookMobile Multimedia Communities

InternationalCommunity

GPS-enabledMedia production

Mobile Media Management

Context through storytelling

Community Facilitation

Use and Creation inCommunity

Media Organisation

Structure Media

Settings

Mobile ServerStandard-conform Web ServicesFurther protocolsP2P data management

Non-linear storiesMPEG-7 & MPEG-21Multimedia AdaptationContext Reasoning