1 multimedia networking r. yang. 2 outline r admin. and review introduction to multimedia...
TRANSCRIPT
2
Outline
Admin. and review Introduction to multimedia
networkingAn architecture of stored multimediaNetwork support of multimedia app.Application adaptation
3
Preview: Multimedia NetworkingOur goals:
Understand the service requirements of networked multimedia applications
Learn the high level structures designed for multimedia networking
Deal with multimedia in a best-effort network
How the Internet might evolve to better support multimedia?
Schedule of the two classes:
Class 1: Multimedia applications; application adaptation in best-effort networks
Class 2: New architectures: IntServ and DiffServ
4
MM Networking Applications
Fundamental characteristics:
Typically delay sensitive end-to-end delay delay jitter
But loss tolerant: infrequent losses cause minor glitches
Antithesis of data, which are loss intolerant but delay tolerant.
Classes of MM applications:
1) Streaming stored audio and video
2) Streaming live audio and video
3) Real-time interactive audio and video
5
Streaming Stored Multimedia
Streaming: media stored at source transmitted to client streaming: client playout begins
before all data has arrived
timing constraint for still-to-be transmitted data: in time for playout
6
Streaming Stored Multimedia: Interactivity
VCR-like functionality: client can pause, rewind, FF, push slider bar 10 sec initial delay OK 1-2 sec until command effect
OK
7
Streaming Live Multimedia
Examples: Internet radio talk show Live sporting event
Streaming play can lag tens of seconds still have timing constraint (but shorter delay
required)Interactivity fast forward impossible rewind, pause possible!
8
Interactive, Real-Time Multimedia
end-end delay requirements: audio: < 150 msec good, < 400 msec OK
• includes application-level (packetization) and network delays• higher delays noticeable, impair interactivity
session initialization callee be able to advertise its IP address, port number, encoding
algorithms.
applications: IP telephony, video conference, distributed interactive worlds
9
Outline
Admin. and reviewA brief introduction to the physical
layerIntroduction to multimedia
networking An architecture of stored multimediaNetwork support of multimedia app.Application adaptation
10
Streaming from Web Server (1) Audio and video files
stored in Web servers Browser requests file with
HTTP request message Web server sends file in
HTTP response message Content-type header line
indicates an audio/video encoding
Browser launches media player, and passes file to media player
Media player renders file
• Major drawback: media playerinteracts with server throughintermediary of a Web browser
11
Streaming from Web Server (2)Alternative: set up
connection between server and player
Web browser requests and receives a meta file (a file describing the object) instead of receiving the file itself
Content-type header indicates specific audio/video application
Browser launches media player and passes it the meta file
Player sets up a TCP connection with server and sends HTTP request
Some concerns: Media player
communicates over HTTP, which is not designed with pause, ff, rewind commands
May want to stream over UDP
12
RTSP/RTCP/RTP
HTTP GET
mediaplayer
Webserver
mediaserver
Webbrowser
client server
presentation desc.
RTSP
RTCP
RTP
RTSP: Real-Time Signaling ProtocolRTCP: Real-Time Control ProtocolRTP: Real-Time Transport Protocol
13
Start: Using HTTP to get Presentation File
<title>Twister</title> <session> <group language=en lipsync> <switch> <track type=audio e="PCMU/8000/1" src =
"rtsp://audio.example.com/twister/audio.en/lofi"> <track type=audio e="DVI4/16000/2" pt="90 DVI4/8000/1"
src="rtsp://audio.example.com/twister/audio.en/hifi"> </switch> <track type="video/jpeg" src="rtsp://video.example.com/twister/video"> </group> </session>
Example presentation file
14
Control: RTSP, Out of Band ControlRTSP messages are sent out-of-band:
The RTSP control messages use a different port number (554) than the media stream, and are therefore sent out-of-band
RTSP can be sent over UDP or TCP Each RTSP message can be sent over a separate TCP
connection The media stream, whose packet structure is not defined
by RTSP, is considered “in-band” If the RTSP messages were to use the same port number
as the media stream, then RTSP messages would be said to be “interleaved” with the media stream
15
RTSP: Initialization and Control
HTTP GET
SETUP
PLAY
media stream (RTP)mediaplayer
Webserver
mediaserver
Webbrowser
clientserver
presentation desc.
TEARDOWN
PAUSE
RTCP
16
RTSP: Exchange Example C: SETUP rtsp://audio.example.com/twister/audio RTSP/1.0 Transport: rtp/udp; compression; port=3056; mode=PLAY
S: RTSP/1.0 200 1 OK Session 4231
C: PLAY rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0 Session: 4231 Range: npt=0-
C: PAUSE rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0 Session: 4231 Range: npt=37
C: TEARDOWN rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0 Session: 4231
S: 200 3 OK
17
Data: Real-Time Protocol (RTP)
RTP runs in the end systems RTP specifies a packet structure for packets
carrying audio and video data: RFC 1889 RTP packets are encapsulated in UDP segments Interoperability: If two Internet phone
applications run RTP, then they may be able to work together
18
RTP Header
- Payload Type - e.g. type 0: (Pulse Coded Modulation) PCM mu-law, 64 Kbps; type 3, GSM, 13 Kbps; type 33, MPEG2 videoQuestion: why include payload type in each packet?
- Sequence Number
- SSRC field- Identifies the source of the RTP stream. Each stream in a RTP session should have a distinct SSRC
- Timestamp field - Reflects the sampling instant of the first byte in the RTP data packet
8 16 32 32
19
RTP Header (2)
Timestamp field Reflects the sampling instant of the first byte in the RTP
data packet The timestamp is derived from a sampling clock at the
sender Example
• audio stream sampled every 125 usecs • audio application generates chunks consisting of 160 encoded
samples• then the timestamp increases by 160 for each RTP packet
when the source is active• the timestamp clock continues to increase at a constant rate
even when the source is inactive
8 16 32 32
20
H.323 Overview
Foundation for audio and video conferencing across IP networks
Targets real-time communication (rather than on-demand)
Umbrella recommendation from the ITU Broad in scope:
stand-alone devices (e.g., Web phones) applications in PCs
23
Gateway
H.3
23 te
rmin
als
Gatekeeper
Router Internet
LAN = “Zone”
RAS
Gateway
PSTN
• Bridge between IP Zone and PSTN (or ISDN) network.• Terminals communicate with gateways using H.245 and Q.931
24
Gatekeeper
• The gatekeeper is optional. Can provides to terminals:• address translation to IP addresses• bandwidth management: can limit amount of bandwidth consumed by real-time conferences
• Optionally, H.323 calls can be routed through gatekeeper. Useful for billing.• RAS protocol (over TCP) for terminal-gatekeeper communication.
H.3
23 te
rmin
als
Gatekeeper
Router
Internet
LAN = “Zone”
RAS
26
H.323 Endpoints Must Support:
G.711 - ITU standard for speech compression
RTP - protocol for encapsulating media chunks into packets
H.245 - “Out-of-band” control protocol for controlling media between H.323 endpoints.
Q.931 - A signalling protocol for establishing and terminating calls.
RAS (Registration/Admission/Status) channel protocol - Protocol for communicating with a gatekeeper (if gatekeeper is present)
27
H.323 Encoding
Audio: H.323 endpoints must
support G.711 standard for speech compression. G.711 transmits voice at 56/64 kbps.
H.323 is considering requiring G.723 = G.723.1, which operates at 5.3/6.3 kbps.
Optional: G.722, G.728, G.729
Video Video capabilities for an
H.323 endpoint are optional.
Any video-enabled H.323 endpoint must support the QCIF H.261 (176x144 pixels).
Optionally supports other H.261 schemes: CIF, 4CIF and 16CIF.
H.261 is used with communication channels that are multiples of 64 kbps.
28
Generating Audio Packet Stream in H.323
AudioSource
Encoding:e.g., G.711 or G.723.1
RTP packetencapsulation
UDP socket
Internet orGatekeeper
29
H.245 Control Channel
H.323 stream may contain multiple channels for different media types.
One H.245 control channel per H.323 session.
H.245 control channel is a reliable (TCP) channel that carries control messages.
Principle tasks: Open and closing
media channels. Capability
exchange: before sending media, endpoints agree on encoding algorithm
Note: H.245 for multimedia conferencing is analogous with RTSP for media streaming
30
Information flows
H.323Terminal
H.323Terminal
Media Channel1
Media ControlChannel
Media Channel2
Call SignalingChannel
Call ControlChannel
TCP
UDP
31
Gatekeeper 2/2
H.323 terminal must register itself with the gatekeeper in its zone. When the H.323
application is invoked at the terminal, the terminal uses RAS to send its IP address and alias (provided by user) to the gatekeeper.
If gatekeeper is present in a zone, each terminal in zone must contact gatekeeper to ask permission to make a call.
Once it has permission, the terminal can send the gatekeeper an e-mail address, alias string or phone extension. The gatekeeper translates the alias to an IP address. If necessary, a
gatekeeper will poll other gatekeepers in other zones to resolve an IP address. Process varies by vendor.
32
Data Control: Real-Time Control Protocol (RTCP)
Works in conjunction with each real-time RTP session
Each participant in an RTP session periodically transmits RTCP control packets to all other participants, e.g., Receiver-report packets: fraction of packets lost, last
sequence number, average inter-arrival jitter Sender-report packets: the number of packets sent, the
timestamp and the wall-time of the last sent packet
RTCP attempts to limit its traffic to 5% of the session bandwidth
33
OutlineAdmin. and reviewA brief introduction to the physical
layerIntroduction to multimedia
networkingAn architecture of stored multimedia Network support of multimedia app.Application adaptation in best-effort
Internet
34
Multimedia in Networks:Fundamental Characteristics
Many have requirements on rate, i.e. long-term average bandwidth requirement (e.g. 13kbps phone conversion)
Many are delay and jitter sensitive jitter is the variability of packet delays within the same
packet stream. Many are loss tolerant
infrequent losses cause minor glitches that can be concealed but high loss rate makes the application useless
Antithesis of data (programs, banking info, etc.), which are loss intolerant but delay tolerant; multimedia is also called “continuous media”
rate rate
utility
35
Why Best-effort Internet May not Work Well?
Variable bandwidth dynamic bandwidth sharing
Potentially long delays Assume simple arrival and service processes
(Poisson). Suppose the utilization of a link is (<1), the delay is
Variable delay jitters
1
1
36
Multimedia Networking: Two Types of Approaches
Laissez-faire philosophy No reservations, no
change of the services provided by IP
Several tools As demand increases,
provision more bandwidth
Application adaptation Place stored content at
edge of network
Integrated/Differentiated services philosophy:
Extends the service provided by the Internet
Two approaches IntServ
• New protocols and router mechanisms so that each flow can reserve end-to-end bandwidth
DiffServ• Datagrams are
marked• User pays more to
send/receive 1st class packets
37
Outline
Admin. and review A brief introduction to the physical layer Introduction to multimedia networking An architecture of stored multimedia Network support of multimedia app. Application adaptation in best-effort
Internet dealing with variable delay dealing with packet loss dealing with variable bandwidth
39
Playout Buffer
Basic idea: delay playing out packets to accommodate delay jitter
Discussion: For what types of applications will playout buffer be the most effective?
packetsnumber
time
packetsgenerated
p p'
delay0
1
r
packets received
40
Example: Adaptive Playout Delay Estimation
TCP-like delay and timeout estimation; adjust playout delay at the beginning of each check point
41
OutlineAdmin. and reviewA brief introduction to the physical
layerIntroduction to multimedia
networkingAn architecture of stored multimediaApplication adaptation in best-effort
Internet dealing with variable delay dealing with packet loss dealing with variable bandwidth
42
Forward Error Correction (FEC): Piggyback A Lower Quality Stream on Next Packet
Send a lower resolution audio stream as theredundant information on the next packet, e.g. nominal stream at 64 kbps and redundant stream GSM at 13 kbps
2 3
43
Outline Admin. and review A brief introduction to the physical layer Introduction to multimedia networking An architecture of stored multimedia Application adaptation in best-effort
Internet dealing with variable delay dealing with packet loss dealing with variable bandwidth
44
Basic Idea: Change the Rate of an MM Application
Video and audio can generate different encoding rates by
controlling the encoding process Question: how do you change the rate of a
multimedia stream?
(262 KB)
45
Video Encoding: Block Transform Encoding
DCT
Zig-zag Quantize
Run-length Code
Huffman Code
011010001011101...
46
Discrete Cosine Transform
F[u,v] = 4C(u)C(v)
n2n-1 n-1
j=0 k=0
f(j,k) cos(2j+1)u
2n
(2k+1)v
2ncos
where
C(w) =
1
21
for w=0
for w=1,2,…,n-1
- Convert from pixel domain to frequency domain- DCT better at reducing redundancy than Discrete Fourier Transform but it is more computationally expensive
48
Example: Block Encoding
139 144 149 153144 151 153 156150 155 160 163159 161 162 160
original image
DCT1260 -1 -12 -5-23 -17 -6 -3-11 -9 -2 2-7 -2 0 1
DC component
AC components
Quantize
79 0 -1 0-2 -1 0 0-1 -1 0 00 0 0 0
zigzag79 0 -2 -1 -1 -1 0 0 -1 0 0 0 0 0 0 0
run-lengthcode
0 791 -20 -10 -10 -12 -10 0
Huffmancode 10011011100011...
coded bitstream < 10 bits (0.55 bits/pixel)
49
Result of Coding/Decoding
139 144 149 153144 151 153 156150 155 160 163159 161 162 160
144 146 149 152156 150 152 154155 156 157 158160 161 161 162
original block reconstructed block
-5 -2 0 1-4 1 1 2-5 -1 3 5-1 0 1 -2
errors
51
Encode Rate According to Available Bandwidth
Steps Estimate available bandwidth (how?) Encode the stream according to the
estimation of available bandwidth (how?)
53
Layering
Put the stream into multiple layers Each layer is a refinement of the previous
layers Example:
Zigzag of the DCT matrix, put the coefficients into different layers