Streaming Stored Streaming Stored AudioAudio

and Video (1)and Video (1)

Advanced Multimedia

University of PalestineUniversity of Palestine Eng. Wisam ZaqootEng. Wisam Zaqoot October 2010October 2010

Ref: Computer Networking: A Top Down Approach, 4th ed., Kurose & Ross

A few words about audio A few words about audio compressioncompression

analog signal sampled analog signal sampled at constant rateat constant rate telephone: 8,000 telephone: 8,000

samples/secsamples/sec CD music: 44,100 CD music: 44,100

samples/secsamples/sec each sample each sample

quantized, i.e., quantized, i.e., roundedrounded e.g., e.g., 2288=256 possible =256 possible

quantized valuesquantized values each quantized value each quantized value

represented by bitsrepresented by bits 8 bits for 256 values8 bits for 256 values

example: 8,000 example: 8,000 samples/sec, 256 samples/sec, 256 quantized values --> quantized values --> 64,000 bps*64,000 bps*

receiver converts bits back receiver converts bits back to analog signal:to analog signal: some quality reductionsome quality reduction

Example ratesExample rates CD: 1.411 Mbps**CD: 1.411 Mbps** MP3: 96, 128, 160 kbps***MP3: 96, 128, 160 kbps*** Internet telephony: Internet telephony:

5.3 kbps and up5.3 kbps and up GSM (13kbps), G.729 GSM (13kbps), G.729

(8kbps)****(8kbps)****

A few words about video A few words about video compressioncompression

video: sequence of video: sequence of images displayed at images displayed at constant rateconstant rate e.g. 24 images/sece.g. 24 images/sec

digital image: array digital image: array of pixelsof pixels each pixel each pixel

represented by bitsrepresented by bits redundancyredundancy

spatial (within image)spatial (within image) temporal (from one temporal (from one

image to next)image to next)

Examples:Examples: MPEG 1 (CD-ROM) 1.5 MPEG 1 (CD-ROM) 1.5

MbpsMbps MPEG2 (DVD) 3-6 MPEG2 (DVD) 3-6

MbpsMbps MPEG4 (OO video MPEG4 (OO video

compression, often compression, often used in Internet, < 1 used in Internet, < 1 Mbps)Mbps)

Research:Research: layered (scalable) layered (scalable)

video*video* adapt layers to adapt layers to

available bandwidthavailable bandwidth

* Layered (scalable) video* Layered (scalable) video A scalable solution to the problem of available A scalable solution to the problem of available

bandwidth variation is to use multi-layered bandwidth variation is to use multi-layered video. A multi-layered video encoder encodes video. A multi-layered video encoder encodes raw video data into one or more streams, or raw video data into one or more streams, or layers, of differing priority. The layer with the layers, of differing priority. The layer with the highest priority, called the highest priority, called the base layerbase layer, contains , contains the most important portions of the video stream. the most important portions of the video stream. One or more One or more enhancement layersenhancement layers with with progressively lower priorities may then be progressively lower priorities may then be encoded to further refine the quality of the base encoded to further refine the quality of the base layer stream.layer stream.

The source must respond to constantly changing The source must respond to constantly changing

network conditions by dynamically adjusting the network conditions by dynamically adjusting the number of video layers it generates. The source number of video layers it generates. The source have to receive congestion feedback from the have to receive congestion feedback from the destinations and the network. destinations and the network.

Streaming Stored Streaming Stored MultimediaMultimedia

application-level application-level streaming techniques streaming techniques for making the best for making the best out of best effort out of best effort service:service: client-side bufferingclient-side buffering use of UDP versus use of UDP versus

TCPTCP multiple encodings of multiple encodings of

multimediamultimedia

jitter removaljitter removal decompressiondecompression error concealmenterror concealment graphical user graphical user

interface interface with controls for with controls for interactivityinteractivity

Media Player

Internet multimedia: simplest approach

audio, video not streamed:• no, “pipelining,” long delays until playout!

• audio or video stored in file• files transferred as HTTP object

– received in entirety at client– then passed to player

Internet multimedia: streaming approach

• browser GETs metafile• browser launches player, passing metafile• player contacts server• server streams audio/video to player

Metafile ExampleMetafile Example<title>Twister</title> <title>Twister</title> <session> <session> <group language=en lipsync> <group language=en lipsync> <switch> <switch> <track type=audio <track type=audio e="PCMU/8000/1" e="PCMU/8000/1" src= src=

""rtsprtsp://audio.example.com/twister/audio.en/lofi"> ://audio.example.com/twister/audio.en/lofi"> <track type=audio <track type=audio e="DVI4/16000/2" pt="90 DVI4/8000/1" e="DVI4/16000/2" pt="90 DVI4/8000/1"

src="src="rtsprtsp://audio.example.com/twister/audio.en/hifi"> ://audio.example.com/twister/audio.en/hifi"> </switch> </switch> <track type="video/jpeg" <track type="video/jpeg"

src="src="rtsprtsp://video.example.com/twister/video"> ://video.example.com/twister/video"> </group> </group> </session> </session>

Internet multimedia: Streaming from a streaming server

• allows for non-HTTP protocol between server and media player

• UDP or TCP for step (3), more shortly

Streaming Multimedia: Client Buffering

constant bit rate videotransmission

Cum

ula

tive

data

time

variablenetworkdelay

client videoreception

constant bit rate video playout at client

client playoutdelay

bu

ffere

dvid

eo

• client-side buffering, playout delay compensate for network-added delay, delay jitter

Streaming Multimedia: Client Buffering

• client-side buffering, playout delay compensate for network-added delay, delay jitter

Streaming Multimedia: UDP Streaming Multimedia: UDP or TCP?or TCP?

UDP:UDP: server sends at rate appropriate for client (Do not server sends at rate appropriate for client (Do not

care about network congestion !)care about network congestion !) often send rate = encoding rate = constant rateoften send rate = encoding rate = constant rate then, fill rate = constant rate - packet lossthen, fill rate = constant rate - packet loss

short playout delay (2-5 seconds) to remove short playout delay (2-5 seconds) to remove network jitternetwork jitter

error recover: time permittingerror recover: time permittingTCP:TCP: send at maximum possible rate under TCP send at maximum possible rate under TCP

(careful about network congestion)(careful about network congestion) fill rate fluctuates due to TCP congestion controlfill rate fluctuates due to TCP congestion control larger playout delay: smooth TCP delivery ratelarger playout delay: smooth TCP delivery rate HTTP/TCP passes more easily through firewallsHTTP/TCP passes more easily through firewalls

Streaming Multimedia: client rate(s)

Q: how to handle different client receive rate capabilities?– 28.8 Kbps dialup– 100 Mbps Ethernet

A: server stores, transmits multiple copies of video, encoded at different rates

1.5 Mbps encoding

28.8 Kbps encoding