06 - transfer and control protocols
TRANSCRIPT
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 1Chapter 3.2: Transfer and Control Protocols
3.2: Transfer and Control Protocols
• The H.x Protocols• Session Initiation Protocol SIP• Streaming Multimedia Data
• Transport Protocols: RTP and RTCP
• VoIP Example
Chapter 2: Basics
Chapter 3: Multimedia Systems – Communication Aspects and Services• Multimedia Applications and Communication• Multimedia Transfer and
Control Protocols
• Quality of Service and Resource Management
• Synchronization
• Multimedia Operating Systems
Chapter 4: Multimedia Systems – Storage Aspects
Chapter 5: Multimedia Usage
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 2Chapter 3.2: Transfer and Control Protocols
Transfer and Control Protocols
A main protocol family is the H.x standards by the ITU• H.261 and H.263 define video coding for video conferences, similar to MPEG
• H.323 is a control protocol for cooperative computing (session management)• Developed by ITU, driven by telecommunication needs
Alternative for session management: Session Initiation Protocol (SIP)• Only one protocol, not a protocol family• Developed by IETF: integrated with the Internet
Additionally: RTP/RTCP as transfer protocols
• H.x and SIP both are not defining transport subsystems• RTP as an addition to UDP
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 3Chapter 3.2: Transfer and Control Protocols
Standards of ITU
User Interface
Network Interface
H.225.0 Layer
Audio CodecsG.711G.722
…
H.245
H.450
H.235
Audio Video Configuration
Video CodecsH.261H.263
…
H.323
The ITU has standardized everything needed in cooperative computing:
• G.711, G.722, G.723, G.728, G.729 for audio coding with 5.3 – 64 kbit/s
• H.261, H.263, H.264, … for video coding similar to MPEG
• H.245 for controlling media streams
• H.450 for negotiation of communication resources
• H.235 for authentication and ciphering
• H.225.0 for connection setup and termination, packetizing of data streams, signaling, …
• H.323 for controlling and coordination
• … and several more, e.g. T.x for data transfer
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 4Chapter 3.2: Transfer and Control Protocols
H.261
For video conferencing systems, coding/decoding in real-time is required
• H.261 was designed for ISDN• It is a video codec for audiovisual services at p·64 Kbit/s (p = 1, 2, 3, ..., 30,
referring to ISDN)
→ H.261 can be denoted as “px64”• Real-time processing requirement of encoding and decoding considered in this
standard: maximum signal delay ≤ 150 ms (this requirement is a kind of limitation concerning coding and decoding procedures)
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 5Chapter 3.2: Transfer and Control Protocols
Properties of H.261
• Image format precisely defined• Image refresh frequency at input: 29.97 frames/sec
• Image encoded as luminance signal Y and chrominance difference signals CB, CR (according to a 4:1:1 subsampling scheme, later adopted by MPEG) → 3 basic information from which the full color may be constructed
• 2 resolution formats (each with 4:3 aspect ratio):
� Common Intermediate Format (CIF)• luminance 288 lines × 352 pixels (8 bit per pixel)
• chrominance 144 × 176 � Quarter-CIF (QCIF)
• luminance 144 × 176 • chrominance 72 × 88
QCIF is mandatory for all H.261 implementations, CIF is optional
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 6Chapter 3.2: Transfer and Control Protocols
Image Preparation
• Image is subdivided into blocks of size 8 lines × 8 pixels (luminance & chrominance)
• Macro blocks consists of 4 luminance blocks and 2 corresponding chrominance blocks• Group of blocks (GOB) = combination of 33 macro blocks
→ QCIF image consists of 3 GOBs (= 3 × 33 × 4 × 8 × 8 = 144 × 176 pixels for luminance), CIF image of 12 GOBs (= 12 × 33 × 4 × 8 × 8 = 288 × 352 pixels for luminance)
• Note: color difference samples placed such that their block boundaries coincide with luminance block boundaries:
Luminance sampleChrominance sampleBlock edge
Block
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 7Chapter 3.2: Transfer and Control Protocols
Data Amount
Uncompressed QCIF:
• Data rate = 29.97 frames/sec × (144 × 176 + 2 × 72 × 88) bytes/frame ≈ 9.115 Mbit/sec
Uncompressed CIF:
• Data rate ≈ 36.45 Mbit/sec (= 4 × 9.115 Mbit/sec)
Compressed QCIF:• Needs only 10 frames per second (instead of 29.97 frames/sec), i.e. three times less:
3.041 Mbit/sec are required
→ Compression ratio in order to transmit 3.041 uncompressed Mbit/sec via a 64 Kbit/sec line: 64/3.041 ≈ 1 : 47.5 � This is possible for today's technology, but only for slow moving pictures
• Compressed CIF would need 4 - 6 ISDN B-channels for the same purpose
• Coding Algorithms: 2 different methods of coding (choice up to the coding control strategy): � Intraframe coding: like in JPEG with DCT, quantization and entropy encoding
� Interframe coding: use of information from previous frame (P frames in MPEG)
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 8Chapter 3.2: Transfer and Control Protocols
Prediction for each macro block by motion compensation and spatial filter• Motion compensation (similar to MPEG):
� Comparison of macro blocks from previous and current image→ motion vector defined by relative position of previous and current macro block
� One motion vector per macro block, used for all luminance and chrominance blocks• Simple implementations just compare previous and actual macro blocks at the same
position. In such case, the motion vector is a zero vector
• Optionally (but rarely used) a low pass filter between DCT and entropy encoding can be used for deleting any remaining high-frequency noise
• Linear quantization (step size adjusted according to data amount in transformation buffer)
� Constant data rate at encoder output enforced� Quality of encoded video data depends on image contents and motion within scene
Interframe Coding
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 9Chapter 3.2: Transfer and Control Protocols
Data Streams in H.261
Characteristics of Data Stream for H.261:• Data stream produced by H.261 has a hierarchical structure→ several layers, like in MPEG (bottom layer containing compressed picture)
• Data stream includes information for error correction (18 parity bits for 492 data bits)• 5-bit image number as temporal reference for each image
• Freezing of image which was shown last is possible by an application command; this allows the application at the decoding station to stop and start a video scene in a convenient way
• Switching between still images and moving images possible (by encoder command!)
Conclusion:• Suited for applications which do not require too much quality and where the content
doesn‘t move too fast (video conferencing)
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 10Chapter 3.2: Transfer and Control Protocols
H.263 and H.264
H.263 is similar to H.261, but…• … defines 5 image formats (sub-QCIF, QCIF, CIF, 4CIF, 16CIF)
• … error correction is optional• … consideration of GOBs and Slices like in MPEG
H.264 additionally …
• … allows variable block size (16×16, 8×16, 16×8, 8×8)• … uses a very simple 4×4 transform instead of DCT (astonishingly with negligible loss in
quality!)
• … allows to use any frame as a reference for prediction• … also allows bi-directional prediction (B-frames)
• … allows mixed frames – slices of one frame can be coded independently as I-slices, P-slices and B-slices!
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 11Chapter 3.2: Transfer and Control Protocols
H.323
A video conference is not only transferring video…• Audio transmission (G.7xx), synchronization with video stream
• Exchange of configuration data, signalling (H.225, H.245)• Whiteboard, chat, application sharing, data, fax … (T.x)• Transport subsystem (TCP, UDP, RTP, RTCP)
• …→ H.323 for coordination
Not only client terminals (telephones, video phones, NetMeeting, …) “speak” H.323, but also other system components:
• Gatekeeper: address translation (phone numbers to IP addresses), admissioncontrol and bandwidth management for multipoint connections, call authorization, call signal routing
• Gateway: integration with other voice networks• Multipoint control unit (MCU): coordinates several terminals taking part in a
conference• Proxy: e.g. used to pass a firewall
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 12Chapter 3.2: Transfer and Control Protocols
The ITU Family and the OSI Reference Model
NETWORK
DATA LINK
PHYSICAL
TRANSPORT
SESSION
PRESENTATION
APPLICATION
Supplementary Services
Audio Signal
Video Signal Data
Control
G.711 G.728
H.261 H.263 T.127
T.126
T.124
T.125/T.122
G.722 G.729
G.723.1
RTCP RAS RTP
H.450.3 H.450.2
H.450.1H.235
H.245 H.225UDP TCP
X.224.0
H.323
• Transfer of multimedia data uses UDP, transfer of control information uses TCP
• H.323 is an “umbrella” standard comprising all the other functionality
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 13Chapter 3.2: Transfer and Control Protocols
H.323 Network Components
• H.323 terminal can be workstations as well as more specalized end systems, e.g. IP phones
• The gateway enables an integration with existing systems like ISDN or older POTS (Plain Old Telephony System)
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 14Chapter 3.2: Transfer and Control Protocols
H.323 Components and Signaling
• H.245 – A protocol for capabilities advertisement, media channel establishment and conference control.
• H.225 - Call Control• Q.931 – A protocol for call control and call setup.
• RAS – Registration, admission and status protocol used for communicating between an H.323 endpoint and a gatekeeper.
POTS
Gatekeeper
Terminal
H.225/RAS messages
GatewayH.245 messages over call control channel
H.225/Q.931 messages over call signaling channel
H.225/RAS messages
H.225/Q.931 (optional) H.225/Q.931 (optional)
H.245 messages (optional) H.245 messages (optional)
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 15Chapter 3.2: Transfer and Control Protocols
Process for Establishing Communication
Establishing communication using H.323 occurs in five steps:
1. Call setup2. Initial communication and capabilities exchange
3. Audio/video communication establishment4. Call services
5. Call termination
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 16Chapter 3.2: Transfer and Control Protocols
Simplified H.323 Call Setup• Both endpoints have previously
registered with the gatekeeper• Terminal A initiate the call to the
gatekeeper
• The gatekeeper provides information for Terminal A to contact Terminal B
• Terminal A sends a SETUP message to Terminal B
• Terminal B responds with a Call Proceeding message and also contacts the gatekeeper for permission
• Terminal B sends a Alerting and Connect message
• Terminal B and A exchange H.245 messages to determine master/slave, terminal capabilities, and open logical channels
• The two terminals establish RTP media paths for data transmission
Terminal A Gatekeeper Terminal B
RAS messagesCall Signaling Messages
1. ARQ2. ACF
5. ARQ6. ACF
3. SETUP4. Call Proceeding
7.Alerting8.Connect
H.245 MessagesRTP Media Path
Note: This diagram only illustrates a simple point-to-point call setup where call signaling is not routed to the gatekeeper. Refer to the H.323 recommendation for more call setup scenarios.
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 17Chapter 3.2: Transfer and Control Protocols
Session Initiation Protocol SIP
• Defined by IETF• SIP long-term vision
� All telephone calls and video conference calls take place over the Internet� People are identified by names or e-mail addresses, rather than by phone
numbers
� You can reach the callee, no matter where the callee roams, no matter what IP device the callee is currently using
• SIP is an application layer signaling protocol that defines initiation, modification and termination of interactive multimedia communication sessions between multiple users
• Bases upon HTTP concepts (message syntax, SIP URLs, responses, …)
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 18Chapter 3.2: Transfer and Control Protocols
SIP Services
Setting up a call• Provides mechanisms for caller to let callee know he wants to establish a call
• Provides mechanisms so that caller and callee can agree on media type and encoding• Provides mechanisms to end call
Determine current IP address of callee• Maps mnemonic identifier to current IP address
Call management• Add new media streams during call• Change encoding during call
• Invite others • Transfer and hold calls
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 19Chapter 3.2: Transfer and Control Protocols
Setting up a Call to a known IP Address
• Alice’s SIP invite message indicates her port number & IP address. Indicates encoding that Alice prefers to receive (PCM µlaw)
• Bob’s 200 OK message indicates his port number, IP address & preferred encoding (GSM)
• SIP messages can be sent over TCP or UDP; here sent over RTP/UDP
• Default SIP port number is 506.
µ
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 20Chapter 3.2: Transfer and Control Protocols
Call Setup
• Codec negotiation� Suppose Bob doesn’t have PCM µlaw encoder� Bob will instead reply with 606 Not Acceptable Reply and list encoders he
can use� Alice can then send a new INVITE message, advertising an appropriate
encoder
• Rejecting the call� Bob can reject with replies “busy,” “gone,” “payment required,” “forbidden”
• Media can be sent over RTP or some other protocol.
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 21Chapter 3.2: Transfer and Control Protocols
Example of SIP message
INVITE sip:[email protected] SIP/2.0
Via: SIP/2.0/UDP 167.180.112.24
From: sip:[email protected]
To: sip:[email protected]
Call-ID: [email protected]
Content-Type: application/sdp
Content-Length: 885
c=IN IP4 167.180.112.24
m=audio 38060 RTP/AVP 0
Notes:
• HTTP message syntax• sdp = session description protocol
• Call-ID is unique for every call.
Here we don’t know Bob’s IP address. Intermediate SIPservers will be necessary.
Alice sends and receives SIP messages using the SIP default port number 506.
Alice specifies in Via:header that SIP client sends and receives SIP messages over UDP
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 22Chapter 3.2: Transfer and Control Protocols
Name Translation and User Location
• Caller wants to call callee, but only has callee’s name or e-mail address
• Need to get IP address of callee’s current host:� User moves around
� DHCP protocol� User has different IP devices (PC, PDA, car device)
• Result can be based on:
� Time of day (work, home)� Caller (don’t want boss to call you at home)
� Status of callee (calls sent to voicemail when callee is already talking to someone)
Service provided by SIP servers:
• SIP registrar server• SIP proxy server
• SIP redirect server• SIP location server
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 23Chapter 3.2: Transfer and Control Protocols
Redirect Server
SIP Distributed Architecture
Location Server
Registrar Server
User Agent
Proxy Server
Gateway
PSTN
SIP Components
Proxy Server
• User Agent Client (UAC) – An entity that initiates a call
• User Agent Server (UAS) – An entity that receives a call
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 24Chapter 3.2: Transfer and Control Protocols
SIP Registrar
• REGISTER sip:domain.com SIP/2.0
• Via: SIP/2.0/UDP 193.64.210.89 • From: sip:[email protected]
• To: sip:[email protected]• Expires: 3600
• When Bob starts SIP client, the client sends SIP REGISTER message to Bob’s registrar server
Register Message:
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 25Chapter 3.2: Transfer and Control Protocols
SIP Proxy
• Alice sends invite message to her proxy server� contains address sip:[email protected]
• Proxy responsible for routing SIP messages to callee� possibly through multiple proxies
• Callee sends response back through the same set of proxies
• Proxy returns SIP response message to Alice
� contains Bob’s IP address
• Interprets, rewrites or translates a request message before forwarding it
• Note: proxy is analogous to local DNS server
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 26Chapter 3.2: Transfer and Control Protocols
Example
Caller [email protected] places a call to [email protected]
(1) Jim sends INVITE message to umass SIP proxy
(2) Proxy forwards request to upenn registrar server
(3) upenn server returnsredirect response,indicating that it should try [email protected]
(4) umass proxy sends INVITE to eurecom registrar
(5) eurecom regristrar forwards INVITE to 197.87.54.21, which is running keith’s SIP client
SIP client217.123.56.89
SIP client197.87.54.21
SIP proxyumass.edu
SIP registrarupenn.edu
SIPregistrareurecom.fr
1
2
34
5
6
7
8
9
Note: also a SIP ack message, which is not shown
(6-8) SIP response sent back(9) Messages sent directly between clients
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 27Chapter 3.2: Transfer and Control Protocols
Comparison with H.323
• H.323 is a complete, vertically integrated suite of protocols for multimedia conferencing: signaling, registration, admission control, transport and codecs
• SIP is a single component. Works with RTP, but does not mandate it. Can be combined with other protocols and services.
• H.323 comes from the ITU (telephony)• SIP comes from IETF: Borrows much of its
concepts from HTTP. SIP has a Web flavor, whereas H.323 has a telephony flavor
H.323 is complex – SIP uses the KISS principle: Keep it simple and stupid
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 28Chapter 3.2: Transfer and Control Protocols
Transport Subsystem
How to transfer multimedia data in the Internet?
• TCP/UDP/IP: “best-effort service”• No guarantees on delay, loss
→ Today’s Internet multimedia applications use application-level techniques to mitigate (as best as possible) effects of delay and loss
E.g. streamed stored multimedia
• Application-level streaming techniques for making the best out of best effort service:� Client side buffering
� Use of UDP versus TCP� Multiple encodings of multimedia
But: what protocols on lower layers are suitable to support such application-level streaming?
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 29Chapter 3.2: Transfer and Control Protocols
Internet Multimedia: Simplest Approach
Audio and video are not really streamed:• Long delays until playout!
• Audio or video stored in files
• Files are transferred as HTTP object� Received in entirety at client
� Then passed to player
First: how can application level streaming be realized?
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 30Chapter 3.2: Transfer and Control Protocols
Internet Multimedia: Streaming Approach
• Browser GETs metafile with server contact information
• Browser launches player, passing metafile• Player contacts server
• Server streams audio/video to player
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 31Chapter 3.2: Transfer and Control Protocols
Streaming from a Streaming Server
• Separation of web server and streaming• This architecture allows for non-HTTP protocol between server and media player
• Can also use UDP instead of TCP
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 32Chapter 3.2: Transfer and Control Protocols
Streaming Multimedia: Client Buffering
• In streaming, data can arrive with variable rate by network delay and jitter• Thus: client-side buffering for playout delay for compensation of these problems
bufferedvideo
variable fillrate x(t)
constantdrain
rate d
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 33Chapter 3.2: Transfer and Control Protocols
Streaming Multimedia
What transport protocol to use for such an approach?
UDP • Server sends at rate appropriate for client (oblivious to network congestion !)
• Often send rate = encoding rate = constant rate• Then: fill rate = constant rate - packet loss• Short playout delay (2-5 seconds) to compensate for network jitter
• Error recovery: if time permits
TCP
• Send at maximum possible rate under TCP• Fill rate fluctuates due to TCP congestion control
• Larger playout delay: smooth TCP delivery rate• HTTP/TCP passes more easily through firewalls
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 34Chapter 3.2: Transfer and Control Protocols
Solution: RTSP
HTTP• Does not target multimedia content
• No commands for fast forward, etc.
Real-time Streaming Protocol RTSP
• Client-server application layer protocol• For user to control display: rewind, fast forward, pause, resume, repositioning, etc…
What it doesn’t do:• Does not define how audio/video is encapsulated for streaming over network• Does not restrict how streamed media is transported; it can be transported over UDP
or TCP• Does not specify how the media player buffers audio/video
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 35Chapter 3.2: Transfer and Control Protocols
RTSP: Out of Band Control
FTP uses an “out-of-band” control channel:• A file is transferred over one TCP connection
• Control information (directory changes, file deletion, file renaming, etc.) is sent over a separate TCP connection
• The “out-of-band” and “in-band” channels use different port numbers
RTSP messages are also sent out-of-band:• RTSP control messages use different port numbers than the media stream
(Port 554): out-of-band
• The media stream is considered “in-band”
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 36Chapter 3.2: Transfer and Control Protocols
RTSP Example
Scenario:• Metafile communicated to web browser• Browser launches player
• Player sets up an RTSP control connection and a data connection to streaming server
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 37Chapter 3.2: Transfer and Control Protocols
Metafile Example
<title>Twister</title>
<session> <group language=en lipsync>
<switch>
<track type=audio e="PCMU/8000/1"
src = "rtsp://audio.example.com/twister/audio.en/lofi"> <track type=audio
e="DVI4/16000/2" pt="90 DVI4/8000/1" src="rtsp://audio.example.com/twister/audio.en/hifi">
</switch> <track type="video/jpeg"
src="rtsp://video.example.com/twister/video"> </group>
</session>
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 38Chapter 3.2: Transfer and Control Protocols
RTSP Exchange Example
C: SETUP rtsp://audio.example.com/twister/audio RTSP/1.0
Transport: rtp/udp; compression; port=3056; mode=PLAY
S: RTSP/1.0 200 1 OK
Session 4231
C: PLAY rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0
Session: 4231 Range: npt=0-
C: PAUSE rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0 Session: 4231
Range: npt=37
C: TEARDOWN rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0
Session: 4231
S: 200 3 OK
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 39Chapter 3.2: Transfer and Control Protocols
Example: Internet Phone
Introduce Internet Phone by way of an example
• Speaker’s audio: alternating talk spurts, silent periods� 64 kbit/s during talk spurt
• Packets are generated only during talk spurts
� 20 msec chunks at 64 kbit/sec: 160 bytes data• Application-layer header added to each chunk
• Chunk and header are encapsulated into a UDP segment.• Application sends UDP segments into socket every 20 msec during talkspurt
Required:
• Network loss: IP datagram lost due to network congestion (router buffer overflow)• Delay loss: IP datagram arrives too late for playout at receiver
� Delays: processing, queueing in network; end-system (sender, receiver) delays� Typical maximum tolerable delay: 400 ms
• Loss tolerance: depending on voice encoding, losses concealed, packet loss rates between 1% and 10% can be tolerated
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 40Chapter 3.2: Transfer and Control Protocols
constant bit rate
transmission
Cum
ulat
ive
data
time
variablenetwork
delay(jitter)
clientreception
constant bit rate playout
at client
client playoutdelay
buffe
red
data
Jitter
• Consider the end-to-end delays of two consecutive packets: difference can be more or less than 20 msec
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 41Chapter 3.2: Transfer and Control Protocols
Internet Phone: Fixed Playout Delay
Receiver attempts to playout each chunk exactly q msecs after chunk was generated
• chunk has timestamp t: play out chunk at t + q
• chunk arrives after t + q: data arrives too late for playout, data “lost”
Tradeoff for q:• large q:
less packet loss• small q: better
interactive experience
20 msec
r: receiving of first packetp: first playout schedulep’: second playout schedule
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 42Chapter 3.2: Transfer and Control Protocols
Adaptive Playout Delay
i
i
i
*i i i
i
t timestamp of the ith packet
r the time packet i is received by receiver
p the time packet i is played at receiver
d = r t network delay for ith packet
d estimate of average network delay after r
===
− == eceiving ith packet
Dynamic estimate of average delay at receiver: *i i 1 id (1 u ) d u d−= − ⋅ + ⋅
where u is a fixed constant (e.g., u = .01)
• Goal: minimize playout delay, keeping late loss rate low
• Approach: adaptive playout delay adjustment:� Estimate network delay, adjust playout delay at beginning of each talk spurt� Silent periods compressed and elongated
� Chunks still played out every 20 msec during talk spurt.
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 43Chapter 3.2: Transfer and Control Protocols
Also useful to estimate the average deviation of the delay vi (jitter):
*i i 1 i iv (1 u ) v u | d d |−= − ⋅ + ⋅ −
The estimates di and vi are calculated for every received packet, although they are only used at the beginning of a talk spurt.
For first packet in talk spurt, playout time is:
i i i ip t d Kv= + +
where K is a positive constant.
Remaining packets in talkspurt are played out periodically
Adaptive Playout Delay
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 44Chapter 3.2: Transfer and Control Protocols
Recovery from Packet Loss
Forward error correction (FEC): simple scheme• For every group of n chunks create a redundant chunk by exclusive OR-ing the n
original chunks
• Send out n+1 chunks, increasing the bandwidth by factor 1/n.• Can reconstruct the original n chunks if there is at most one lost chunk from the n+1
chunks
• Playout delay needs to be fixed to the time to receive all n+1 packets• Tradeoff:
� increase n, less bandwidth waste� increase n, longer playout delay
� increase n, higher probability that 2 or more chunks will be lost
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 45Chapter 3.2: Transfer and Control Protocols
Recovery from Packet Loss
Other FEC scheme:• “Piggyback lower
quality stream”
• Send lower resolutionaudio stream as theredundant information
• For example, nominal stream PCM at 64 kbpsand redundant streamGSM at 13 kbps
• Whenever there is non-consecutive loss, the receiver can conceal the loss
• Can also append (n-1)st and (n-2)nd low-bit rate chunk
lower quality
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 46Chapter 3.2: Transfer and Control Protocols
Recovery from Packet Loss
Interleaving
• Chunks are broken up into smaller units• For example, 45 msec units per chunk• Packet contains small units from
different chunks
• If packet is lost, still have most of every chunk
• Has no redundancy overhead
• But adds to playout delay
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 47Chapter 3.2: Transfer and Control Protocols
Summary: Internet Multimedia: Bag of Tricks
• Use UDP to avoid TCP congestion control (delays) for time-sensitive traffic• Client-side adaptive playout delay to compensate for network delay
• Server side matches stream bandwidth to available client-to-server path bandwidth� Chose among pre-encoded stream rates
� Dynamic server encoding rate• Error recovery (on top of UDP)
� FEC, interleaving� Retransmissions, time permitting� Conceal errors: repeat nearby data
→ Provide a standardized transport protocol which supports such tricks: RTP
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 48Chapter 3.2: Transfer and Control Protocols
Real-Time Protocol (RTP)
RTSP still would have to use the unreliable UDP or the “slow” TCP – better define a “new” transport protocol for combining speed with reliability:Real-Time Transport Protocol (RTP)
• RTP specifies a packet structure for packets carrying audio and video data• RTP packet provides
� Payload type identification� Packet sequence numbering
� Timestamping• RTP runs in the end systems
• RTP packets are encapsulated in UDP segments• Interoperability: if two Internet phone applications run RTP, then they may be able to
work together
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 49Chapter 3.2: Transfer and Control Protocols
RTP libraries provide a transport-layer interface that extend UDP: • Port numbers, IP addresses• Payload type identification
• Packet sequence numbering• Time-stamping
RTP runs on Top of UDP
Transport Layer
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 50Chapter 3.2: Transfer and Control Protocols
RTP Header
• Ver.: Version number of the RTP protocol in use• P: packet size was padded to a multiple of 32 bit
• X: an extension header is used• CC: indicates the number of sources
• M: User-specific mark. Can e.g. mark the beginning of a word on an audio channel.• Contributing Source Identifier: used by mixers in the studio. The mixed flows are listed here.
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 51Chapter 3.2: Transfer and Control Protocols
RTP Header
Payload Type (7 bits)Indicates type of encoding currently being used. If the sender changes encoding in
middle of transmission, it informs the receiver through this payload type field• Payload type 0: PCM µ-law, 64 kbps
• Payload type 3, GSM, 13 kbps• Payload type 26, Motion JPEG• Payload type 31, H.261
• Payload type 33, MPEG2 video
Sequence Number (16 bits)Increments by one for each RTP packet sent, and may be used to detect packet loss and
to restore packet sequence
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 52Chapter 3.2: Transfer and Control Protocols
RTP Header
Timestamp field (32 bits long)• Reflects the sampling instant of the first byte in the RTP data packet• For audio, timestamp clock typically increments by one for each sampling period (for
example, each 125 µsecs for a 8 KHz sampling clock)
• If application generates chunks of 160 encoded samples, then timestamp increases by 160 for each RTP packet when source is active. Timestamp clock continues to increase at constant rate when source is inactive.
• The timestamp gives the receiver the relative time (with respect to the first data) when to playout the data
Synchronization Source Identifier field (32 bits long)• Identifies the source of the RTP stream• Each stream in a RTP session should have a distinct identifier
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 53Chapter 3.2: Transfer and Control Protocols
RTP and QoS
• RTP only adds some information to the UDP header needed for kind of reliability
• RTP does not provide any mechanism to ensure timely delivery of data or provide other quality of service guarantees
• RTP encapsulation is only seen at the end systems: it is not seen by intermediate routers. � Routers providing best-effort service do not make any special effort to ensure
that RTP packets arrive at the destination in a timely matter.
• Usage of (and reaction to) the information in the RTP header are left over to the application
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 54Chapter 3.2: Transfer and Control Protocols
Real-Time Control Protocol (RTCP)
• Works in conjunction with RTP
• Each participant in RTP session periodically transmits RTCP control packets to all other participants
• Each RTCP packet contains sender and/or receiver reports
� report statistics useful to application• Statistics include number of packets sent, number of packets lost, interarrival jitter,
etc.
• Feedback can be used to control performance� Sender may modify its transmissions based on feedback
• To limit traffic, each participant reduces his RTCP traffic as the number of conference participants increases
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 55Chapter 3.2: Transfer and Control Protocols
RTCP
RTCP controls the data flow:
• Feedback to the sender about QoS on receiver side• Data losses, delay and jitter are reported
• Note: RTCP does not provide corrective actions - this is left over to the application
Application
RTP / RTCP
UDP
IP
Application
RTP / RTCP
UDP
IP
RTP RTP RTP RTP RTPRTP
RTCP RTCP RTCPRTCP
Sender Receiver
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 56Chapter 3.2: Transfer and Control Protocols
Application Example: Voice over IP (VoIP)
Telephony using an IP network with standardized protocols: VoIP• Transferring speech and
signaling information
• Not only internally in a IP network, also integration with “normal”telephony systems
IP network(Internet/Intranet)IP phones
IP terminal
ISDN phone
VoIPGateway
Telecommunicationnetwork
Phone numbers
IP addresses and virtual phone numbers
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 57Chapter 3.2: Transfer and Control Protocols
VoIP-based Telephony System
R
VoIP-Gateway
ISDNPTSS
Branch Company central
ISDNPTSS
ISDN
R
VoIP-Gateway
IP network(Internet)
TeleworkingPCs
R = RouterPTSS = Private Telecommunications Switching System
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 58Chapter 3.2: Transfer and Control Protocols
Realization with H.323
H.323terminal
H.323terminalGate-
keeper MCU
Gateway
Networkwithout QoS guarantees
H.323 zone
H.324
POTS
H.321
ATM network
H.320
ISDN
H.323 gives us all functionality we need to realize an IP-based telephony integrated with conventional solutions
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 59Chapter 3.2: Transfer and Control Protocols
Call Setup
IP phone
Call isinitiated
Ringing
Pick up receiver
Dialing tone
Connection
PTSS
ISDN phoneVoIP
Gateway
Intranet
GK
ISDN
IP[TCP[SETUP[...] ]]
IP[TCP[Alerting [...]]]
IP[TCP[Connectt[...] ]]
B channelRTP channel
Voice IP[UDP[RTP[Voice]]]
Connect
Alerting
CallProceeding
Phone nr => IP-Adr.?
SETUP[...]
GK: Gatekeeper
Lehrstuhl für Informatik 4
Kommunikation und verteilte Systeme
Page 60Chapter 3.2: Transfer and Control Protocols
VoIP Future
At the moment, VoIP products based on H.323 are most popular
• But: complex, and telecommunication networks tend to converge with IP networks→ Use protocols better integrated with the IP world
→ SIP together with a MGCP (Media Gateway Control Protocol) gets more and more significance
→ Better integration with web applications
→ SIP seems to be the multimedia signaling protocol for the future
Still a problem: quality of an IP transmission; how to improve QoS in the Internet?