lec 17 - mpeg video system ii: adv. dash · 2020-04-01 · lec 17 - mpeg video system ii: adv. dash...

ECE 5578 Multimedia Communication

Lec 17 - MPEG Video System II: Adv. DASH

Zhu LiDept of CSEE, UMKC

Office: FH560E, Email: [email protected], Ph: x 2346.http://l.web.umkc.edu/lizhu

ECE 5578: Multimedia Communciation, 2018 p.1

slides created with WPS Office Linux and EqualX LaTex equation editor

Outline

ReCap Lecture 16 Advanced DASH Topics DASH FDH: DASH over Full Duplex HTTP/WebSocket DASH SAND: QoS, QoE and Traffic Coordination

Summary


MPEG File Format (FF)

Chocolate box approach Abstract the content into a hierarchical structure,

meta data, media data Hiding the compression tech details

The Solution – ISOBMFF ISO Based Media File Format – aka, Mp4 moov mdat moof hint …


ISOBMFF file types Plain File:

Simple recording of plain media data, data first, header last “mdat” box, then “moov” box, e.g, foreman_320kbps_nf120.mp4

Progressive File: For progressive download or streaming Header first then media data Interleaved Chunks

Fragmented File: Multiple moof segments followed by moov box Good for continous recording

Segmented File: Self contained and playable fragments in signle file or in separate files For HTTP streaming (DASH) Tools: segment file, indexing


DASH in a Nutshell


Dynamic, Adaptive, Streaming over HTTP: An OTT solution HTTP De-facto Internet Transport Infrastructure

Media Presentation Data Model MPD: a manifest of content available on HTTP server Accessible segments and their timing As a .xml file to be retrieved by clients at the start of DASH session


Credits to figures on following slides: Christian and Ali, Over the Top Content Delivery: State of the Art and Challenges Ahead, ICME 2015 Tutorial

Understanding DASH structure

Three types of files Manifest (.mpd)

o XML file describing the segments Initialization file

o Contains headers needed to decode bytes in segments Segment Files

o Contains playable mediao Includes:

• 0…many video tracks• 0…many audio tracks


DASH Manifest

Manifest contains: Program Info (including title) 1 or more periods

o Periods contain 1 adaptation set per video stream and o Periods contain 1 adaptation set per audio streamo Adaptation Sets contain:

• Content Composition nodes (for each video or audio track)• 1 or more Representation node

• Each representation describes a single bitrate• Representations contain data on finding the actual segments• Different ways a representation can describe segments


Describing Representations

SegmentBase Describes a stream with only a single Segment per bitrate Can be used for Byte Range Requests

SegmentList A SegmentList will contain a specific list of each SegmentURL

(individual HTTP packet with media data) Can be used for Byte Range Requests

SegmentTemplate Defines a known url for the fragment with wildcards resolved at

runtime to request a segments (see bbb.mpd) Alternatively, can specify a list of segments based on duration


SegmentList<Representation id="h264bl_hd" mimeType="video/mp4" codecs="avc1.42c01f" width="1280" height="720" startWithSAP="1" bandwidth="514864"> <SegmentList timescale="1000" duration="10000"> <Initialization sourceURL="mp4-main-multi-h264bl_hd-.mp4"/> <SegmentURL media="mp4-main-multi-h264bl_hd-1.m4s"/> <SegmentURL media="mp4-main-multi-h264bl_hd-2.m4s"/> <SegmentURL media="mp4-main-multi-h264bl_hd-3.m4s"/> <SegmentURL media="mp4-main-multi-h264bl_hd-4.m4s"/> <SegmentURL media="mp4-main-multi-h264bl_hd-5.m4s"/> <SegmentURL media="mp4-main-multi-h264bl_hd-6.m4s"/> <SegmentURL media="mp4-main-multi-h264bl_hd-7.m4s"/> <SegmentURL media="mp4-main-multi-h264bl_hd-8.m4s"/>


SegmentTemplate fixed segment duration

<AdaptationSet> <ContentComponent id="1" contentType="video"/> <SegmentTemplate initialization="BigBuckBunny_720p_1800kbps_44khz_track1_dash.mp4"/> <Representation id="1" mimeType="video/mp4“ codecs="avc1.64001f" width="1280" height="720“ startWithSAP="1" bandwidth="1809954"> <SegmentTemplate timescale="1000" duration="13809" media="bbb_seg_BigBuckBunny_720p_1800kbps_44khz_track1$Number$.m4s" startNumber="1"/> </Representation></AdaptationSet>


SegmentTemplate variable segment duration

<AdaptationSet group="2" mimeType="video/mp4" par="16:9“ minBandwidth="475000“ maxBandwidth="6589000" minWidth="176" maxWidth="1680" minHeight="99" maxHeight="944“ segmentAlignment="true“ startWithSAP="1"> <SegmentTemplate timescale="1000"

initialization="dash/ateam-video=$Bandwidth$.dash"media="dash/ateam-video=$Bandwidth$-$Time$.dash">

<SegmentTimeline> <S t="0" d="4171" /> <S d="2503" /> <S d="2961" /> <S d="2461" /> <S d="2127" r="2" />…


Understanding MP4 – H.264 structure

Each segment is binary, with a series of boxes contained withinFirst 8 bytes of every box defines Box type Number of bytes for the box.


Initialization Segment

FTYP – file type and compatibilityFREE – free spaceMOOV – container of all metadata MVHD – movie header 1.* TRAK – container for individual stream data

o TKHD – track headero MDIA – container for all media info

• Lots of child boxes which contain more detailed stream info


.m4s Content Segments

STYP - segment typeSIDX - segment indexMOOF -movie fragment MFHD - movie fragment header TRAF - track fragment

o TFHD - track fragment headero TFDT - track fragment decode timeo TRUN - track fragment run

MDAT - media data container (actual media data) Contains one or more samples (smallest described piece of content)


Preparing DASH content with MP4Box

Download you favorable video from youtube https://addons.mozilla.org/en-

US/firefox/addon/video-downloadhelper/

Encode for on-demand DASH content MP4Box –dash 10000 –out mydash/my.mpd

–dash-profile on-demand –segment-name mydash-video/seg myvideo.mp4

It will create A MPD file listing all segments A collection of video segment files in .m4s, the

segmented file.


Download Helper

MPD


DASH.js To playback DASH video is very simple, just use <video> tagCheck out: http://l.web.umkc.edu/lizhu/teaching/2016sp.video-

communication/DASH_js_demo.html


<script src="http://cdn.dashjs.org/latest/dash.all.min.js"></script><h4> DASH.js Player Demo </h4><style> video { width: 640px; height: 360px; }</style>.<div> <video data-dashjs-player src="http://dash.edgesuite.net/envivio/EnvivioDash3/manifest.mpd" controls></video> </div>

Outline

ReCap Lecture 15 Advanced DASH Topics DASH FDH: DASH over Full Duplex HTTP/WebSocket DASH SAND: Traffic Coordination MPEG MMT

Summary


DASH FDH

Introducing Push Based Operation

Benefit: Reduce the HTTP GET overhead Avoid TCP transmission window collapse


…

Client Serverreq seg 1

seg 1

seg 2

seg n

req seg 2

req seg n

Client Serverreq seg 1 to k

seg 1

…seg k

req seg (n-k+1) to n

seg (n-k+1)…

seg n

…

(a) Regular HTTP (b) HTTP/2 Server Push

HTTP 2.0

IN a NutShell One TCP connection Multiple streams – requests are in streams, can be parallelized Binarized framing, streams can have flow control All for Push Service


HTTP 2.0 Streams

Streams are multiplexed into frames Frames are interleaved Can have flow control on streams E.g give main.css with priority 1, give me fig1.jpg with priority 5.


DASH FDH with HTTP 2.0

Example of “push-next” Client request pushing 2 next segments


DASH FDH over HTTP 2.0 Server Response: Push promise: 2 segments over

stream 2 and 3 Segment 1 data


DASH FDH over HTTP 2.0

Server Response: Segment 2, 3 over a

different stream


Response [Stream ID = 2]:HEADERS

+ END_STREAM+ END_HEADERS:status = 200

DATA + END_STREAM

{binary data for segment 2} Response [Stream ID = 3]: HEADERS

+ END_STREAM+ END_HEADERS:status = 200

DATA + END_STREAM

{binary data for segment 3}

WebSockets

Motivation – Evolution of Web Applications Early days no real time requirement Now Dynamic, Media, and Real time applications like chat, streaming,

collaboration

HTTP is not designed for web apps– Large overhead– Hanging-GET is necessary for real-time server push Hence, the new solution, WebSockets Full Duplex Small overhead Multiplexing over a single TCP Easy to use API

Specs RFC 6455: https://tools.ietf.org/html/rfc6455


Establishing WebSocket Connection

“Upgrade” over HTTP1.1


WebSocket Frames

Support binary data channel Opcode:

o 0x0 Continuation frame, 0x1 Connection close• 0x2 Ping, 0x3 Pong, 0x4 Text frame, 0x5 Binary frame• 0x7-0xF Reserved


FDH DASH over WebSocket

Message flow


WebSocket Sub-Protocol for DASH

DASH WebSocket Header as part of WebSocket Payload STREAM_ID: 8 bits, identify current stream MSG_CODE: 8 bits:

o 1- get_mpd, 2 – get_segment, 3-new_mpd, 4: new_segmento 255: cancel

F: 3 bits: flags to signal different commands EXT_LENT: extension header for JSON coded parameters


0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+| STREAM_ID | MSG_CODE | F | EXT_LENGTH |+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+| Extension (JSON encoded parameters) |+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+| Application Data || ... |+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

WebSocket DASH FDH Ops

Request New MPD


Client Request:STREAM_ID : 1MSG_CODE: 1EXT_LENGTH: 27EXT: {“mpd_uri”:“./example.mpd”} Server Response:STREAM_ID : 1MSG_CODE: 3EXT_LENGTH: 0{binary data with example.mpd}

WebSocket DASH FDH Ops

Request a new segment, followed by 2 pushed segmetns


Client Request:STREAM_ID : 1MSG_CODE: 2EXT_LENGTH: 98E X T : {“segment_uri”:“./rep1/segment1.mp4”,”push_directive”:”type=\”urn:mpeg:dash:fdh:2015:push-next\”;2”}

Server Response:STREAM_ID : 1MSG_CODE: 4EXT_LENGTH: 59E X T : {”push_policy”:”type=\”urn:mpeg:dash:fdh:2015:push-next\”;2”}{binary data with segment1.mp4}

Server Response:STREAM_ID : 1MSG_CODE: 4EXT_LENGTH: 0{binary data with segment2.mp4} STREAM_ID : 1MSG_CODE: 4EXT_LENGTH: 0{binary data with segment3.mp4}

Outline

ReCap Lecture 15 Advanced DASH Topics DASH FDH: DASH over Full Duplex HTTP/WebSocket DASH SAND: Traffic Coordination MPEG MMT

Summary


DASH SAND

DASH SAND Server and Network Assisted DASH (SAND)

Objectives: Unidirectional/bidirectional, point-to-point/multipoint communication

with and without session (management) between servers/CDNs and DASH clients, Mechanisms for providing content-awareness and service-awareness

towards the underlying protocol stack including server and/or network assistance, Various impacts on elements of the existing Internet infrastructure such

as servers, proxies, caches and CDNs, QoS and QoE support for DASH-based services, Scalability in general and specifically for logging interfaces, and Analytics and monitoring of DASH-based services.


DASH SAND DASH Boxes Client, Server, DANE.

SAND Messages Parameters Enhancing Delivery (PED) messages that are exchanged

between DANEs, Parameters Enhancing Reception (PER) messages that are sent from

DANEs to DASH clients, Metrics and Status messages that are sent from DASH clients to DANEs.


SAND Operations

Where SAND sits


QoS Signalling with SAND PER:QoSInformation


Resource Coordination Metics: Client to DANE Allows for cross traffic sharing of common resources allocationStrategy: weighted, pricing….etc


Parameter Type Cardinality DescriptionSharedResourceAllocation object 1 operationPoints array 1..N List of suitable operation points for current

play time.

bandwidth

integer 1 A bandwidth value expressed in a number of bits per second. This value shall be computed from the MPD by summing bandwidths of all components the client would use for working at this operation point. If playback rate is not 1, this bandwidth value shall be modified accordingly.

quality integer 0..1 An optional value describing the quality of the current operation point.

minBufferTime

integer 0..1 An optional value in milliseconds extracted from the MPD regarding a minimal buffer time of the current operation point.

weight integer 0..1 A user allocated optional value which indicates a weight of the request in the present message for the resource allocation process. The exact use of this value depends on the allocation strategy indicated by allocationStrategy.

allocationStrategy urn 0..1 An optional identifier to indicate the resource allocation strategy preferred by the client for resource sharing.

mpdUrl string 0..1 If present, an URL to the MPD related to the present message.

Resource Allocation PER:sharedResourceAllocation , DANE to Client Results of optimization process

Resource Pricing in bottleneck coordination:

Ying Li, Zhu Li, Mung Chiang, A. Robert Calderbank:Content-Aware Distortion-Fair Video Streaming in Congested Networks. IEEE Trans. Multimedia 11(6): 1182-1193 (2009)

Z. Li, Z. Li, S. Zhao, D. Medhi, and I. Bouazizi, "Wireless Video Traffic Bottleneck Coordination with DASH SAND Framework", IEEE Visual Communication and Image Processing (VCIP) Conf, Chengdu, China 2016.


Parameter Type Cardinality DescriptionSharedResourceAssignment object 1

Response message from the coordinator that indicates the results of the bandwidth sharing operation.

clientID string 1The clientID identifies the target receiver of this message. This field shall use the same value as the senderID of the SharedResourceAllocation message sent by the client.

resourcePrice int 0..1A price for the bandwidth resource to be used by the receiver in its utility to price tradeoff to determine an optimal operation point. The operation point selected will maximize the utility to price ratio.

bandwidth int 1This fields contains the assigned bandwidth to the identified client. The unit is defined in bits per second.

Summary

MP4Box/DASH.js MP4Box is a versatile tool for media file manipulation, it has

functionality built in for preparing DASH streaming content DASH.js is a reference javascript DASH client implementation, by the

DASH IF, and easy to modify and good for research PoC

DASH FDH Improve throughput, efficiency by utilizing new transport HTTP 2.0/WebSocket has different APIs but fundamentally the same

implementation Ref: DASH FDH specs

DASH SAND Allows server and other DANE middle box to play a role New messaging to report QoS, QoE, and coordinate, accelerate traffic Ref: DASH SAND specs


lec 17 - mpeg video system ii: adv. dash · 2020-04-01 · lec 17 - mpeg video system ii: adv. dash...

Documents