video streaming concepts
DESCRIPTION
Video Streaming Concepts. Reading: John G. Apostolopoulos, Wai- tian Tan, Susie J. Wee, “Video Streaming: Concepts, Algorithms, and Systems”, HP Laboratories Palo Alto, 2002. Classification of video apps. Point2point vs multicast vs broadcast Is there a “reverse channel”? - PowerPoint PPT PresentationTRANSCRIPT
Video Streaming ConceptsVideo Streaming Concepts
Reading:John G. Apostolopoulos, Wai- tian Tan, Susie J. Wee,“Video Streaming: Concepts, Algorithms, and Systems”, HP Laboratories Palo Alto, 2002.
1
Classification of video appsClassification of video apps
Point2point vs multicast vs broadcast Is there a “reverse channel”? Pros and cons of reverse channel
Real-time vs pre-encorded(stored) video What about “almost real-time”?
Interactive vs non-interactive video Static vs dynamic channels
Bandwidth, delay, and loss are static or dynamic
CBR vs VBR channels Do not confuse with CBR/VBR encoding
Packet-switched vs circuit-switched channels QoS support?
Video compression standardsVideo compression standards What do the standards specify?
Encoder and decoder implementations? Bit stream syntax? Decoding process?
Video streaming challengesVideo streaming challenges
Video delivery via file download vs streaming ? Characteristics of Internet are unknown or time-
varying Bandwidth rate control Delay jitter playout buffer Loss error control
Transport and Rate Control Transport and Rate Control for overcoming Time-varying BWfor overcoming Time-varying BW Objectives
How to estimate the appropriate transmission rates dynamically at the time of streaming ? – available BW estimation
How media coding has evolved to support such dynamic changes in transmission rates ? – coding rate control
Rate control for video streaming over TCP TCP is a window (not rate) based transport protocol TCP adjusts the window size based on AIMD congestion-control
algorithm varying throughput (saw-tooth pattern) Retransmission enlarge delay jitter Nevertheless, TCP is often used in video streaming, especially
when receiver window is appropriately sized Stable and scalable TCP rate control TCP guarantees delivery How would you calculate the receiver window size to set the
streaming rate at a certain value?
Transport and Rate Control Transport and Rate Control for overcoming Time-varying BWfor overcoming Time-varying BW Rate control for video streaming over UDP
Separation of error control and rate control TCP-friendly rate control
Mimic average throughput of TCP = f(RTT, p) Smooth the instantaneous fluctuations of TCP’s AIMD algorithm
Rate 1/RTT : video streaming 에 적합한가 ?
Some media streaming system do not perform rate control Multicasting: Identical stream is transmitted to all recipients via
channels of different levels of congestion No feedback channel
Receiver-driven rate control Layered multicast of scalable or layered compressed video
Meeting Transmission BW ConstraintsMeeting Transmission BW Constraints If channel transmission rate < media bit rate, then ?? Transcoding
Decode and re-encode to the desired bit rate Include bit rate reduction, spatial downsampling, frame rate
reduction, changing compression formats Requires application-layer gateways
Multiple file switching Multiple copies of the same content at different bit-rates. Client
chooses the appropriate media rate. Multi-rate switching: enables dynamic switching between
different rates within a session No recompression penalty, but BW waste for multiple copies of
the same media
Scalable compression (layered coding)
7
Evolving Approaches in InternetEvolving Approaches in Internet
IntServ model QoS guarantees BW, packet loss rate, delay on per
flow basis Explicit resource reservation via RSVP
DiffServ model Classify and differentiate among classes based on a
tag(code-point) in each packet
8
Playout Buffer for Overcoming JitterPlayout Buffer for Overcoming Jitter
Solution: Playback buffering Key question: how long
should the playback buffer (or playback delay) be?
Playback buffers have additional advantages: Error recovery through ReTx Error resilience through interleaving Smooth throughput variations
(e.g., due to TCP)
Adaptive Media Playout
Error Control for Overcoming Channel Error Control for Overcoming Channel LossesLosses Bit error vs packet loss
In wired network, almost packet loss due to congestion In wireless network, bit errors or burst errors may cause
To be passed up to the appl. Layer Or, discarded (packet loss)
Solutions? Retransmissions
Commonly used, but constrained by delay budget and playback delay Requires back-channel
Forward Error Correction Send N packets (K data packets + K-N redundant packets). If received
packets >= K correctlry, data packets are recovered. Introduce bandwidth overhead (and potentially delay) Or, cannot recover.
Loss/error concealment Loss/error resilient video coding
Loss/Error ConcealmentLoss/Error Concealment
Pixels, MBs, slices, and entire frame may be lost.
Exploits the correlation along Spatial interpolation Temporal extrapolation (freeze frame) Motion-compensated temporal extrapolation
Motion-compensated block + estimation of lost MV from ??
Error concealment is performed at the decoder
11
Error Resilience Video Coding: Error Resilience Video Coding: Overcoming Loss of Bitstream Synch.Overcoming Loss of Bitstream Synch. Resynch Marker
Put the Resync markers (placed where?)
Reversible VLCs (MPEG-4) Less efficient than VLS
Data Partitioning (MPEG-4) Place most important data just after markers
Application Level Framing (ALF) Design the packet(frame) payload to minimize the effect of loss
12
Error Resilience Video Coding: Error Resilience Video Coding: Overcoming Incorrect State and Error PropagationOvercoming Incorrect State and Error Propagation Overcome error propagation
Use periodic I-pictures (GOP) Use periodic intra-coding of MBs
In point-to-point comm. with back channel (short RTT) Both the encoder and decoder store multiple previously coded
frames Decoder notifies the packet error to the encoder The encode tells which picture should be used as the reference
for the next prediction (Reference Picture Selection in H.263) 13
Scalable (Layered) Video Coding Scalable (Layered) Video Coding for Lossy Networksfor Lossy Networks SVC implicitly assumes broadcast or multicast of
video steams (including overlay multicasting) Base layer and several enhancement layers
need different treatments unequal error protection (UEP), prioritized transmission
Useful if network provides several classes of service or priorities (DiffServ, IntServ)
Under the best effort Internet ?
Multiple Description Video CodingMultiple Description Video Coding Multiple description coding (MDC) video
Several “descriptions” of the same video The more descriptions you receive, the better Significant redundancy among descriptions What if all descriptions are subject to simultaneous losses?
MDC video with path diversity
Media Streaming Protocols and StandardsMedia Streaming Protocols and Standards Protocols for video streaming over the Internet
Media delivery: RTP/RCTP RTP does not provide real-time delivery, neither support any
QoS Only provides time stamps, sequence numbering, video
source identification, payload specification RTCP provides QoS feedback in terms of # of los packets,
delay, inter-arrival jitter, etc.
Media Session Control RTSP (Realtime Streaming Protocol) SIP (Session Initiation Protocol)
Media Description and Announcement SDP (Session Description Protocol) SAP (Session Announcement Protocol)
16
Additional Video Streaming TopicsAdditional Video Streaming Topics
Multicast Should solve heterogeneity problem
Network heterogeneity: different channel conditions (BW, error protection)
Receiver heterogeneity Approaches
Different multicasts for different ranges of intended bit-rates: e.g) simulcast
Different multicasts can contain incremental information (SVC): e.g) layered multicast
End-to-end security and Transcoding Streaming over wired and wireless Links
Loss differentialtion: Loss due to congestion or noise ?
Streaming media CDN17
Streaming Media CDNStreaming Media CDN
Conventional CDN vs streaming media CDN Issues
Application-layer multicast (overlay multicast) Seamless hand-off
MD-CDN Achieve path diversity from the infrastruture of CDN
18