research group 'multimedia communication' presentation (march 2015)
TRANSCRIPT
Univ.-Prof. Hermann Hellwagner
Assoc.Prof. Christian Timmerer
Institute of Information Technology (ITEC)
Faculty of Technical Sciences (TEWI)
Alpen-Adria-Universität Klagenfurt (AAU)
Austria
Research Group Multimedia Communication (MMC)
1
Research Group
Multimedia Communication (MMC)
Research Group Multimedia Communication (MMC)
2
Alpen-Adria-Universität Klagenfurt (AAU)
> 10,000 students, > 800 employees
(> 500 scientific staff members, 70 full
professors)
Studies: cultural sciences, economics,
psychology, mathematics, computer
science, information technology, …
Research Group Multimedia Communication (MMC)
3
Faculty of Technical Sciences (TEWI)Application EngineeringIntelligent Systems and Business InformaticsSystemsicherheitInformation and Communication SystemsInteractive SystemsSoftware EngineeringDistributed Multimedia Systems
Multimedia Communication
Mobile SystemsPervasive ComputingEmbedded SystemsSmart GridsTransportation InformaticsSensory and ActuatorsControl and Mechatronic Systems
Applied AnalysisDiscrete Mathematics and OptimizationEducation and Philosophy of Mathematics
Biometrics and Clinical StatisticsSpatial and Financial StatisticsBiometrics and Educational Statistics
Full Professor
• Dr. Hermann Hellwagner
Associate Professor
• Dr. Christian Timmerer
Secretary
• Margit Pertl, BA
University Assistants
• DI Daniel Posch
• DI Florian Bacher
System Engineer
• Rudolf Messner
Research Group Multimedia Communication (MMC)
4
Research Group Staff
Project Assistants
• DI Christian Kreuzberger
• DI Daniela Pohl
• DI Christian Raffelsberger
• DI Benjamin Rainer
• Sandra Bergmann
Multimedia Communication and QoS/QoE Provisioning
Adaptive Video Streaming
Dynamic Adaptive Streaming over HTTP (DASH)
Use of Scalable Video Coding (SVC) Technology
Multimedia Content Distribution in Information-Centric Networks (ICN)
Quality of Experience Beyond Audio-Visual Perception
Sensory Experience
Standardization within ISO/IEC MPEG
Multimedia in Disaster Management
Research Group Multimedia Communication (MMC)
5
Research Areas
European Commission (IST FP6 /FP7):
• DANAE (STREP; 01/2004-06/2006): „Dynamic and distributed Adaptation of scalable multimedia
coNtent in a context-Aware Environment“
• ENTHRONE II (IP; 09/2006-11/2008): „End-to-End QoS through Integrated Management of
Content, Networks and Terminals“
• INTERMEDIA (NoE; 10/2006-09/2010): „Interactive Media with Personal Networked Devices“
• P2P-Next (IP; 01/2008-12/2011): „Next Generation Peer-to-Peer Content Delivery Platform“
• ALICANTE (IP; 03/2010-08/2013): „Media Ecosystem Deployment Through Ubiquitous Content-
Aware Network Environments“
• BRIDGE (IP; 04/2011-03/2015): „Bridging Resources and Agencies in Large-Scale Emergency
Management“
• SOCIAL SENSOR (IP; 10/2011-10/2014): „Sensing User Generated Input for Improved Media
Discovery and Experience“
Industry (partial funding):
• MPEG-21 standardization: in cooperation with Siemens CT IC 2, Munich
• SVC adaptation: in cooperation with ST Microelectronics, Milan
Research Group Multimedia Communication (MMC)
6
Research Projects and Funding
COST (European Cooperation in Science and Technology):
QUALINET (ICT Action IC1003; 11/2010 – 11/2014): „European Network on Quality
of Experience in Multimedia Systems and Services“
Austrian Science Fund (FWF) – Basic Research:
• Quality Adaptive Video Caching and Transport (QAVCT)
• Adaptive Streaming of Secure Scalable Wavelet-based Video (ASSSV)
• A Context-Adaptive Content Ecosystem Under Uncertainty (CONCERT)
Austrian Research Promotion Agency (FFG) and CELTIC:
• Scalable Video Based Provisioning of TV and VoD Services over the Internet
(SCALIPTV)
Klagenfurt University:
• Independent PhD Theses
Research Group Multimedia Communication (MMC)
7
Research Projects and Funding
www.p2p-next.org
General goal: development of the open-source, BitTorrent-based,
next-generation P2P system NextShare
Approach: research, prototyping, and trials
in Klagenfurt and around the world
Specific goal of MMC:
streaming of scalable media in P2P mode
Research Group Multimedia Communication (MMC)
8
Example of Previous Work: P2P-NextPiece-Picking for Layered Content in BitTorrent-based P2P Networks
When streaming layered videos in a BitTorrent-based P2P network,
the piece-picking algorithm needs to decide which piece to download
at which point in time.
The main goal is to provide the best
possible quality with the available
bandwidth while ensuring continuous
playback and minimizing changes
in quality.
Research Group Multimedia Communication (MMC)
9
P2P-NextPiece-Picking for Layered Content in BitTorrent-based P2P Networks
Research Group Multimedia Communication (MMC)
10
P2P-NextSliding Window and Utility Calculation
𝑼𝒊 𝒋 𝒌 =𝐝𝐣 × 𝐝𝐩𝐢 𝐣 𝐤
(𝐢𝐩𝐢 − 𝐢𝐩𝐤)𝛂
ipi: start of the ith time slot in intervals
ipk: kth decision point in intervals
lj: layer of the piece
dj: distortion reduction importance of the piece
dpijk: probability to receive the useful piece in time
: urgency weighting
The support for layered content and the piece-picking algorithm for
layered content was integrated into the NextShare P2P system.
The performance of the newly developed piece-picking algorithm was
evaluated using the Omnet++/Oversim simulation framework as well
as by using real system tests.
Compared to distributing single-layer content, the support for layered
content can significantly increase the performance when realistic peer
churn rates and/or less than optimal network conditions are applied.
Compared to previously published piece-picking algorithms for
layered content, the newly developed piece-picking algorithm
provides a very good performance over all tested scenarios.
Research Group Multimedia Communication (MMC)
11
P2P-NextResults
www.bridgeproject.eu
General goal: development of technical solutions for support of
emergency agencies that help increase safety of EU citizens in case
of large-scale emergencies
Technical solutions are developed with a focus on:
Interoperability: interoperability between heterogeneous (ad-hoc)
networks, different data formats and existing contextual information
Situational awareness: common operational picture through
developing intelligent user interfaces, data collecting, filtering, and
information sharing mechanisms
Collaboration: inter-agency workflow creation and composition
Research Group Multimedia Communication (MMC)
12
Examples of Current Work: BRIDGEBridging Resources and Agencies in Large-Scale Emergency
Management
Technical solution to improve situational awareness in emergency
management through sub-event detection from multimedia data (especially,
social media data)
Sub-events are specific hotspots originating in the context of an emergency
(disaster, crisis) that are of importance for planning the response activities
(e.g., flooding in a specific district, collapsed buildings in another area)
Sub-events are recognized via identifying multimedia items describing the
same incident
Sub-event detection algorithms are examined, created, and evaluated
covering offline (retrospective) and online (just-in-time) detection algorithms
Important techniques: machine learning (i.e., feature selection, clustering, and
classification), active learning, natural language processing
Research Group Multimedia Communication (MMC)
13
BRIDGESub-Event Detection from/with Multimedia Data
Multimedia (Detection) Framework that allows to include multimedia data from
different sources (e.g., on-site through an Android app, specific social media
platforms, etc.) in the sub-event detection process
Transmission of identified sub-events to other BRIDGE related sub-systems
to support a common
operational picture
Training support for
emergency management
through the development
of a simulation tool to
simulate social media
usage during an exercise
Research Group Multimedia Communication (MMC)
14
BRIDGESub-Event Detection from/with Multimedia Data
Multimedia (Detection) Framework
Processing
(Aggregation/Detection & Filtering)
BRIDGESub-Systems
Social Media Platforms
Android-App
Simulation Tool
Result Monitoring
Result Visualization
Identifying & transmitting important information
BRIDGEMiddleware
Simulation scenario examples
Technical solution to improve communication in ad-hoc networks for
emergency response operations that are prone to disruptions
Development of hybrid routing approaches that combine end-to-end and
delay-tolerant routing in order to mitigate effects of network disruptions and
provide communication in partitioned networks
Providing disruption-tolerant communication services to other BRIDGE
systems such as electronic triage
or S.O.S. smartphone app
Simulation-based evaluations
of hybrid approach in
realistic emergency
response networking
scenarios
Research Group Multimedia Communication (MMC)
15
BRIDGECommunication in Disrupted Networks
Use-case example
Multimedia communication in disrupted networks
Videos from incident site are important to improve situational awareness and
provide a common operational picture
State-of-the art multimedia delivery protocols cannot cope with disruptions
Development of a multimedia
delivery system that allows to
exchange multimedia data
between the incident site and
the incident command center
in partitioned ad-hoc networks
Research Group Multimedia Communication (MMC)
16
BRIDGEMultimedia Communication in Disrupted Networks
www.concert-project.org
International basic research project, cooperation partners:
University College London, Dept. of Electronic and Electrical Engineering
University of Surrey, Centre of Communication Systems Research
École Polytechnique Fédérale de Lausanne, Signal Processing Lab.
General goal: to develop a content ecosystem with all relevant
“players“ which will be able to perform intelligent content and network
adaptation in highly dynamic conditions under uncertainty
Basis: Information-Centric Networks (ICN) with Content- and Context-
aware Routers (CCR)
Research Group Multimedia Communication (MMC)
17
CONCERTA Context-Adaptive Content Ecosystem Under Uncertainty
CONCERTA Context-Adaptive Content Ecosystem Under Uncertainty
Research Group Multimedia Communication (MMC)
18
ContentServer
MultimediaContent
CCR – Content- and Context-aware Router
Cached and adapted multimedia content
ICN is a revolutionary network architecture which changes the current
host-centric paradigm to a content-centric paradigm
Specific ICN under investigation: Named Data Network (PARC, UCLA)
General goal: improve the effectiveness of content dissemination and
provide the best possible Quality of Experience (QoE) for consumers
Scenarios: commercial content distribution; user-generated content;
multi-view streaming
NDN‘s security model enables network-inherent caching, but prohibits
in-network content processing/adaptation in the first place
In-network adaptation is being integrated based on layered encodings
Research Group Multimedia Communication (MMC)
19
CONCERTEffective Content Dissemination and In-Network Adaptation in ICN
dash.itec.aau.at (Dynamic Adaptive Streaming over HTTP – DASH)
Goal: efficient, adaptive multimedia streaming over HTTP/TCP
Approach: research and development, prototyping, measurements,
and standardization of DASH concepts
Research Group Multimedia Communication (MMC)
20
Adaptive Media Streaming
selab.itec.aau.at
What comes after 3D Video? Sensory Effects
Goal: to improve the user‘s QoE by means of sensory effects
Approach: annotate videos with additional effects (wind, light,
explosion, etc.) which are rendered on specific devices (fan, vibration
chair, etc.), subjective tests, QoE models
Research Group Multimedia Communication (MMC)
21
Sensory Experience Lab
… giving her/him the sensation of being part of the particular media➪ worthwhile, informative user experience
www.socialsensor.eu
Snychronized playback of Internet video streams with concurrent real-
time communication channels
Inter-Destination Media Synchronization (IDMS)
Research Group Multimedia Communication (MMC)
22
Social TV
Live Sport Events- Olympic Games- NFL Superbowl- FIFA Worldcup- …
SatelliteBroadcast
(Semi-)Professional&
AmateurContent Acquisition
Consumer
Consumer
Consumer
Heterogeneous Broadband Networks
(wired, wireless)
Geographically distributed consumers accessing the same content while communicating among them.
Syn
chro
niz
ed
Pla
you
t
Online Community
www.qualinet.eu (COST Action QUALINET)
Factors impacting Quality of Experience
Research Group Multimedia Communication (MMC)
23
Quality of Experience (QoE)
Quality of Experience
(QoE)
DeviceNetwork
Content Format
EnvironmentContent
User Expectation
TaskApplication
Technical Factors
Social andPsychological
Factors
User
Context
www.bitmovin.com
Spin-off from MMC and research projects ALICANTE, SocialSensor
Founders: Stefan Lederer, Christopher Müller, Christian Timmerer
Investors: Speed Invest GmbH, Constantia Industries AG
Research Group Multimedia Communication (MMC)
24
bitmovin
MPEG-4 ISO Base File Format: timed metadata
MPEG-21 Multimedia Framework (2002-2007)
Editor of the standard
Major contributions to Digital Item Adaptation
Various contributions to Vision, Technologies and Strategy, Digital Item Declaration,
File Format, Reference Software, Conformance, Event Reporting, Binary Format
MPEG-V Media Context and Control (2007-2010)
Editor of the standard
Development of Sensory Effect Description Language (SEDL); reference impl.
Multimedia Extensible Middleware: MXM (2008-2010)
Co-editor of the standard
Contributions related to adaptation of multimedia content according to the usage
environment context
Research Group Multimedia Communication (MMC)
25
MPEG Standardization Activities
MPEG Media Transport (2010)
Chair of the AhG and the evaluation of the responses to the call for proposals
MPEG-M Advanced Media Platform Technologies (2010-2012)
Co-editor of the standard
Various contributions in the area of content adaptation
Dynamic Adaptive Streaming over HTTP (2010 – now)
Chair of the AhG and the evaluation of the responses to the call for proposals
Contributions related to the composition of media presentations, quality metrics, and
conformance / reference implementation (co-editor of the standard)
Head of the Austrian Delegation
Research Group Multimedia Communication (MMC)
26
Recent MPEG Standardization Activities
Pohl D., Bouchachia A.: Information Propagation in Social Networks during Crises: A Structural
Framework. In: D. Krol, D. Fay, B. Gabrys (eds.): Propagation Phenomena in Real World Networks.
Springer 2015 (to appear).
Timmerer C., Waltl M., Rainer B., Murray N.: Sensory Experience: Quality of Experience Beyond
Audio-Visual. In: S. Möller, A. Raake (eds.): Quality of Experience: Advanced Concepts, Applications
and Methods. Springer 2014.
Hellwagner H.: The Interplay of Technology Development and Media Convergence: Examples. In: S.
Diel, M. Karmasin (eds.): Media and Convergence Management. Springer 2013.
Hellwagner H., Kofler I., Eberhard M., Kuschnig R., Ransburg M., Sablatschan M.: Scalable Video
Coding: Techniques and Applications for Adaptive Streaming. In: C. Zhu, Y. Li, X. Niu (eds.): Streaming
Media Architectures, Techniques and Applications: Recent Advances. IGI Global 2011.
Grafl M., Timmerer C., Hellwagner H., Negru D., Borcoci E., Renzi D., Mevel A., Chernilov A.: Scalable
Video Coding in Content-Aware Networks: Research Challenges and Open Issues. In: N. Blefari-
Melazzi, G. Bianchi, L. Salgarelli (eds.): Trustworthy Internet. Springer 2011.
Hellwagner H., Timmerer C.: MPEG-21 Applications and Use Cases. In: M.C. Angelides, H. Agius
(eds.): The Handbook of MPEG Applications: Standards in Practice. Wiley 2010.
Research Group Multimedia Communication (MMC)
27
Relevant Recent PublicationsBooks and Book Chapters
Pohl D., Bouchachia A., Hellwagner H.: Online Indexing and Clustering of Social Media Data for
Emergency Management. Neurocomputing, 2015 (to appear).
Rainer B., Timmerer C.: A Generic Utility Model Representing the Quality of Sensory Experience. ACM
Transactions on Multimedia Computing, Communications, and Applications (TOMM), 11(1s):14:1-
14:17, Sept. 2014.
Ghinea G., Timmerer C., Lin W., Gulliver S.: Mulsemedia: State of the Art, Perspectives, and
Challenges. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM),
11(1s):17:1-17:23, Sept. 2014.
Timmerer C., Rainer B.: The Social Multimedia Experience. IEEE Computer, 47(3):67-69, March 2014.
Pohl D., Bouchachia A., Hellwagner H.: Social Media for Crisis Management: Clustering Approaches
for Sub-Event Detection. Multimedia Tools and Applications, Dec. 2013.
Grafl M., Timmerer C., Hellwagner H., Xilouris G., Gardikis G., Renzi D., Battista S,. Borcoci E., Negru
D.: Scalable Media Coding Enabling Content-Aware Networking. IEEE MultiMedia, 20(2):30-41, 2013.
Waltl M., Rainer B., Timmerer C., Hellwagner H.: An End-to-End Tool Chain for Sensory Experience
based on MPEG-V. Signal Processing: Image Communication, 28(2):136-150, Feb. 2013.
Hellwagner H., Hofbauer H., Kuschnig R., Stütz T., Uhl, A.: Secure Transport and Adaptation of MC-
EZBC Video Utilizing H.264-based Transport Protocols. Signal Processing: Image Communication,
27(2):192-207, Feb. 2011.
Lopez F., Jannach D., Martínez J., Timmerer C., Garciá N., Hellwagner H.: Bounded Non-deterministic
Planning for Multimedia Adaptation. Journal of Applied Intelligence, 36(1):29-60, Jan. 2010.
Research Group Multimedia Communication (MMC)
28
Relevant Recent PublicationsInternational Journals
Raffelsberger C., Hellwagner H.: A Multimedia Delivery System for Delay-/Disruption-Tolerant
Networks. In: Proc. 13th IEEE Int‘l. Conf. on Pervasive Computing and Communications Workshops
(PerCom Workshops) 2015 (to appear).
Kreuzberger C., Posch D., Hellwagner H.: A Scalable Video Coding Dataset and Toolchain for
Dynamic Adaptive Streaming over HTTP. In: Proc. 6th ACM Int‘l. Conf. on Multimedia Systems
(MMSys) 2015.
Rainer B., Timmerer C.: Self-Organized Inter-Destination Multimedia Synchronization For Adaptive
Media Streaming. In: Proc. 22st ACM Int‘l. Conf. on Multimedia (MM) 2014.
Rainer B., Timmerer C.: A Quality of Experience Model for Adaptive Media Playout. In: Proc. 6th Int‘l.
Workshop on Quality of Multimedia Experience (QoMEX) 2014.
Raffelsberger C., Hellwagner H.: Combined Mobile Ad-Hoc and Delay/Disruption-Tolerant Routing. In:
Proc. 13th Int‘l. Conf. on Ad-hoc, Mobile, and Wireless Networks (ADHOC-NOW) 2014.
Posch D., Kreuzberger C., Rainer B., Hellwagner H.: Client Starvation: A Shortcoming of Client-driven
Adaptive Streaming in Named Data Networking. In: Proc. 1st ACM Int‘l. Conf. on Information-Centric
Networking (ICN) 2014.
Pohl D., Bouchachia A., Hellwagner, H.: Online Processing of Social Media Data for Emergency
Management, In: Proc. 12th Int‘l. Conf. on Machine Learning and Applications (ICMLA) 2013.
Lederer S., Mueller C., Rainer B., Timmerer C., Hellwagner H.: An Experimental Analysis of Dynamic
Adaptive Streaming over HTTP in Content Centric Networks. In: Proc. IEEE Int‘l. Conf. on Multimedia
and Expo (ICME) 2013.
Research Group Multimedia Communication (MMC)
29
Relevant Recent PublicationsInternational Conferences