chapter 2 media format interoperability. section 2.1 background

Chapter 2

MEDIA FORMAT INTEROPERABILITY

Section 2.1

Background

2.1 Background Digital media has become an integral part of

our everyday lives, and consumers have access to content through a wide array of media services.

Part of the problem is that the content is compressed in a variety of different media formats making it difficult to transfer and consume content on different devices.

2.1 Background Content enters the home through a variety of

ways.(Broadcast, Internet, CD/DVD…)

Interoperability between different systems and different networks is becoming more difficult

The aim is to provide a seamless interaction between content that is authored for one purpose and consumed in a different way.

2.1 Background

本圖取自 "Technologies for Home Networking". Edited by Sudhir Dixit and Ramjee Prasad, published by John Wiley, 2008.

Section 2.2

MEDIA FORMATS

2.2.1 Image and Video Formats Image Formats: (1)JPEG(2)JPEG 2000(3)GIF(4)PNG(5)TIFF

Video Formats:(1)MPEG-1, MPEG-2, MPEG-3, MPEG-4 (2)H.261, H.262, H.263, H.264(3)VCEG(4)VC-1

2.2.1 Image and Video Formats

本圖取自 "Technologies for Home Networking". Edited by Sudhir Dixit and Ramjee Prasad, published by John Wiley, 2008.

2.2.2 Audio Formats

Audio Formats:(1)LPCM(2)MPEG-1(MP3)(3)MPEG-2(AAC-LC) (4)MPEG-4(HE-AAC)(5)AC-3(Dolby Digital) (6)WMA

2.2.2 Audio Formats

本圖取自 "Technologies for Home Networking". Edited by Sudhir Dixit and Ramjee Prasad, published by John Wiley, 2008.

2.2.3 Transport and File Formats

The main task is focused on multiplexing and synchronizing of these coded media streams into a single bitstream or multiple bitstreams.

MPEG-2 also referred to as H.222, specifies two formats: <1>Transport Stream (TS) <2> Program Stream (PS).

Any system-level stream is being able to retrieve the coded media from the transport stream, decode it, and present the decoded results.

The system-level stream is first demultiplexed, and relevant data including both system- and media-related data would then be decoded and presented in a synchronized manner.

2.2.3 Transport and File Formats Transport Stream (TS) : (1)Designed for communicating , storing programs of

coded video, audio. (2)Combine programs with time bases into a single stream.(3) Some noise must be tolerated.(4)Ex: Over-the-air broadcasting.

Program Stream (PS):(1)Designed for multiplexing of audio ,video.(2) error-free environment(3)Ex:DVD,other digital storage applications.

Overhead: TS > PS.

2.2.3 Transport and File Formats MP4 file format:(1)Store multiple variations of the same content and to

select an appropriate version for delivery.

(2)Supporting multiple transport formats (Ex:Real Time Transport Protocol (RTP) )

(3)Audiovisual media data is stored separately from its metadata.( timing information, video frame, file location…etc)

2.2.4 Profiles and Levels Critical concepts regarding media format

interoperability.

Profiles essentially limit the set of tools that need to be implemented from a decoder’s point of view.

Levels define the complexity ,maximum bit-rates, maximum frame sizes , maximum decoded picture buffer sizes.

Section 2.3

METADATA FORMATS

2.3 METADATA FORMATS

Two class of metadata:

(1)Content descriptions : Include Coding format, attributes, parameters

of the data.

(2)Usage environment descriptions: Content is processed and characterized by

terminal capabilities and network conditions.

2.3.1 Content Descriptions MPEG-7:alternative metadata formats do exist,

especially within specialized content domains.

Among the many MPEG-7 tools available, a subset of these tools is particularly targeted toward supporting the distribution to a variety of end terminals considering other user or networking constraints.

2.3.1.1 Media FormatAchieve interoperability.

Coding syntax required care because: (1) There are many conformance points that

could potentially be described. (2) The number of conformance points is

growing, so the specification should be extensible.

(3) A generic mechanism to specify coding syntax of any media type, whether it be audio, video, or graphics, would be desirable.

2.3.1.1 Media Format Media coding syntax is using a controlled term

list and classification schemes.

Classification schemes defined in MPEG-7 include controlled term lists for visual, audio, and graphics formats.

2.3.1.2 Data AbstractionThe Summary Description Schemes (DS) in MPEG-7

enables two types of navigation modes: (1)Hierarchical mode: Information is organized into successive levels. Each describing the audiovisual content at a

different level of detail. Closer to the root more coarse summary(2)Sequential mode: Provides a sequence of images or video frames. Possibly synchronized with audio.Adaptive delivery of content: e.g. highlight only

2.3.1.2 Data Abstraction The summary description would essentially

provide pointers into the main content to enable efficient transmission and navigation to the scenes of interest.

The metadata may be generated during recording or shortly after recording has completed.

2.3.1.3 Multiple Variations MPEG-7’s Variation DS is to allow a server or

proxy to select the most suitable variation of the content for delivery according to the capabilities of terminal devices,network conditions, or user preferences.

A variation fidelity value gives the quality of the variation compared with the original.

2.3.1.4 Transcoding HintsImprove the quality and reduce the complexity for

transcoding.

Difficulty Hint: (a)Coding complexity of the original source content

(b)Used for improved bit-rate control and bit-rate conversion. (CBR&VBR)

Motion Hint: (a)The motion range.

(b)The motion uncompensability. (c)The motion intensity.

Coding Hint: (a)Provides high-level information about the video bitstream was

coded.

2.3.2 Usage Environment Descriptions

Cover a wide range of factors that might affect the optimal way that content is ultimately consumed.

Emphasize the importance of metadata that describes the terminal capabilities and network characteristics.

2.3.2.1 Terminal Capabilities Described in receiving and transmitting

capabilities.

Mainly includes codec capabilities, input–output characteristics, CPU characteristics.

2.3.2.2 Network Characteristics

Capabilities: Static. Maximun capacity ,minimum guarantee

bandwidth In-sequence packet delivery. Deal erroneous packets.

Conditions: Dynamic. Describe the available bandwidth,

error(Ex:packet loss rate & Bit error rate), delay (one-way and two-way packet delay).

2.3.3 User Preferences Suggest a preferred format for different classes

of devices.

UserInteraction Description Scheme (DS) defined by MPEG-7 describe preferences of users pertaining to consumption of content and the usage history.

The UsageHistory DS describes the history of actions carried out by a user of a multimedia system.

2.3.4 Electronic Program Guide(EPG)

Description of programming information.

(Live television programs->Program title, Channel,Time)

Such metadata is being delivered as part of an integrated content service or provided as a separate service to users.

Section 2.4

MEDIA ADAPTATION

2.4 MEDIA ADAPTATION

Transcoding research focused on two key aspects:

(1) Keeping the complexity as low as possible

(2) Minimizing signal degradation.

Transcoding method: Cascaded approach(a) Simply decodes the original signal, performs any

intermediate processing, and re-encodes to the target format.

(b) Advantage: Minimize losses in quality, uses conventional.(c) Disadvantage: Computationally intensive, costly.

2.4 MEDIA ADAPTATION

本圖取自 "Technologies for Home Networking". Edited by Sudhir Dixit and Ramjee Prasad, published by John Wiley, 2008.

Section 2.5

MANDATORY MEDIA FORMAT PROFILES

2.5 MANDATORY MEDIA FORMAT PROFILES

Digital Living Network Alliance (DLNA) A platform of interoperability

An example profile would define the AV media formats as well as the encapsulation or system-layer format.

DLNA specifies rules about conversion between optional and mandatory formats to ensure that content can be enjoyed on all compliant devices.

Section 2.6

MEDIA FORMAT INTEROPERABILITY: AN EXAMPLE

2.6 MEDIA FORMAT INTEROPERABILITY: AN EXAMPLE

本圖取自 "Technologies for Home Networking". Edited by Sudhir Dixit and Ramjee Prasad, published by John Wiley, 2008.

Section 2.7

CONCLUSIONS

2.7 CONCLUSIONS The importance of transcoding between

different media formats has been highlighted, and some of the key technologies for computationally efficient transcoding of video have been covered.