06 - transfer and control protocols

15
3.2: Transfer and Control Protocols The H.x Protocols Session Initiation Protocol SIP Streaming Multimedia Data Transport Protocols: RTP and RTCP • VoIP Example Chapter 2: Basics Chapter 3: Multimedia Systems – Communication Aspects and Services Multimedia Applications and Communication Multimedia Transfer and Control Protocols Quality of Service and Resource Management • Synchronization Multimedia Operating Systems Chapter 4: Multimedia Systems – Storage Aspects Chapter 5: Multimedia Usage Transfer and Control Protocols A main protocol family is the H.x standards by the ITU H.261 and H.263 define video coding for video conferences, similar to MPEG H.323 is a control protocol for cooperative computing (session management) Developed by ITU, driven by telecommunication needs Alternative for session management: Session Initiation Protocol (SIP) Only one protocol, not a protocol family Developed by IETF: integrated with the Internet Additionally: RTP/RTCP as transfer protocols H.x and SIP both are not defining transport subsystems RTP as an addition to UDP Standards of ITU User Interface Network Interface H.225.0 Layer Audio Codecs G.711 G.722 H.245 H.450 H.235 Audio Video Configuration Video Codecs H.261 H.263 H.323 The ITU has standardized everything needed in cooperative computing: G.711, G.722, G.723, G.728, G.729 for audio coding with 5.3 – 64 kbit/s H.261, H.263, H.264, … for video coding similar to MPEG H.245 for controlling media streams H.450 for negotiation of communication resources H.235 for authentication and ciphering H.225.0 for connection setup and termination, packetizing of data streams, signaling, … H.323 for controlling and coordination … and several more, e.g. T.x for data transfer H.261 For video conferencing systems, coding/decoding in real-time is required H.261 was designed for ISDN It is a video codec for audiovisual services at p 64 Kbit/s (p = 1, 2, 3, ..., 30, referring to ISDN) H.261 can be denoted as “px64” Real-time processing requirement of encoding and decoding considered in this standard: maximum signal delay 150 ms (this requirement is a kind of limitation concerning coding and decoding procedures)

Upload: others

Post on 03-Feb-2022

3 views

Category:

Documents


0 download

TRANSCRIPT

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 1Chapter 3.2: Transfer and Control Protocols

3.2: Transfer and Control Protocols

• The H.x Protocols• Session Initiation Protocol SIP• Streaming Multimedia Data

• Transport Protocols: RTP and RTCP

• VoIP Example

Chapter 2: Basics

Chapter 3: Multimedia Systems – Communication Aspects and Services• Multimedia Applications and Communication• Multimedia Transfer and

Control Protocols

• Quality of Service and Resource Management

• Synchronization

• Multimedia Operating Systems

Chapter 4: Multimedia Systems – Storage Aspects

Chapter 5: Multimedia Usage

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 2Chapter 3.2: Transfer and Control Protocols

Transfer and Control Protocols

A main protocol family is the H.x standards by the ITU• H.261 and H.263 define video coding for video conferences, similar to MPEG

• H.323 is a control protocol for cooperative computing (session management)• Developed by ITU, driven by telecommunication needs

Alternative for session management: Session Initiation Protocol (SIP)• Only one protocol, not a protocol family• Developed by IETF: integrated with the Internet

Additionally: RTP/RTCP as transfer protocols

• H.x and SIP both are not defining transport subsystems• RTP as an addition to UDP

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 3Chapter 3.2: Transfer and Control Protocols

Standards of ITU

User Interface

Network Interface

H.225.0 Layer

Audio CodecsG.711G.722

H.245

H.450

H.235

Audio Video Configuration

Video CodecsH.261H.263

H.323

The ITU has standardized everything needed in cooperative computing:

• G.711, G.722, G.723, G.728, G.729 for audio coding with 5.3 – 64 kbit/s

• H.261, H.263, H.264, … for video coding similar to MPEG

• H.245 for controlling media streams

• H.450 for negotiation of communication resources

• H.235 for authentication and ciphering

• H.225.0 for connection setup and termination, packetizing of data streams, signaling, …

• H.323 for controlling and coordination

• … and several more, e.g. T.x for data transfer

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 4Chapter 3.2: Transfer and Control Protocols

H.261

For video conferencing systems, coding/decoding in real-time is required

• H.261 was designed for ISDN• It is a video codec for audiovisual services at p·64 Kbit/s (p = 1, 2, 3, ..., 30,

referring to ISDN)

→ H.261 can be denoted as “px64”• Real-time processing requirement of encoding and decoding considered in this

standard: maximum signal delay ≤ 150 ms (this requirement is a kind of limitation concerning coding and decoding procedures)

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 5Chapter 3.2: Transfer and Control Protocols

Properties of H.261

• Image format precisely defined• Image refresh frequency at input: 29.97 frames/sec

• Image encoded as luminance signal Y and chrominance difference signals CB, CR (according to a 4:1:1 subsampling scheme, later adopted by MPEG) → 3 basic information from which the full color may be constructed

• 2 resolution formats (each with 4:3 aspect ratio):

� Common Intermediate Format (CIF)• luminance 288 lines × 352 pixels (8 bit per pixel)

• chrominance 144 × 176 � Quarter-CIF (QCIF)

• luminance 144 × 176 • chrominance 72 × 88

QCIF is mandatory for all H.261 implementations, CIF is optional

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 6Chapter 3.2: Transfer and Control Protocols

Image Preparation

• Image is subdivided into blocks of size 8 lines × 8 pixels (luminance & chrominance)

• Macro blocks consists of 4 luminance blocks and 2 corresponding chrominance blocks• Group of blocks (GOB) = combination of 33 macro blocks

→ QCIF image consists of 3 GOBs (= 3 × 33 × 4 × 8 × 8 = 144 × 176 pixels for luminance), CIF image of 12 GOBs (= 12 × 33 × 4 × 8 × 8 = 288 × 352 pixels for luminance)

• Note: color difference samples placed such that their block boundaries coincide with luminance block boundaries:

Luminance sampleChrominance sampleBlock edge

Block

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 7Chapter 3.2: Transfer and Control Protocols

Data Amount

Uncompressed QCIF:

• Data rate = 29.97 frames/sec × (144 × 176 + 2 × 72 × 88) bytes/frame ≈ 9.115 Mbit/sec

Uncompressed CIF:

• Data rate ≈ 36.45 Mbit/sec (= 4 × 9.115 Mbit/sec)

Compressed QCIF:• Needs only 10 frames per second (instead of 29.97 frames/sec), i.e. three times less:

3.041 Mbit/sec are required

→ Compression ratio in order to transmit 3.041 uncompressed Mbit/sec via a 64 Kbit/sec line: 64/3.041 ≈ 1 : 47.5 � This is possible for today's technology, but only for slow moving pictures

• Compressed CIF would need 4 - 6 ISDN B-channels for the same purpose

• Coding Algorithms: 2 different methods of coding (choice up to the coding control strategy): � Intraframe coding: like in JPEG with DCT, quantization and entropy encoding

� Interframe coding: use of information from previous frame (P frames in MPEG)

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 8Chapter 3.2: Transfer and Control Protocols

Prediction for each macro block by motion compensation and spatial filter• Motion compensation (similar to MPEG):

� Comparison of macro blocks from previous and current image→ motion vector defined by relative position of previous and current macro block

� One motion vector per macro block, used for all luminance and chrominance blocks• Simple implementations just compare previous and actual macro blocks at the same

position. In such case, the motion vector is a zero vector

• Optionally (but rarely used) a low pass filter between DCT and entropy encoding can be used for deleting any remaining high-frequency noise

• Linear quantization (step size adjusted according to data amount in transformation buffer)

� Constant data rate at encoder output enforced� Quality of encoded video data depends on image contents and motion within scene

Interframe Coding

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 9Chapter 3.2: Transfer and Control Protocols

Data Streams in H.261

Characteristics of Data Stream for H.261:• Data stream produced by H.261 has a hierarchical structure→ several layers, like in MPEG (bottom layer containing compressed picture)

• Data stream includes information for error correction (18 parity bits for 492 data bits)• 5-bit image number as temporal reference for each image

• Freezing of image which was shown last is possible by an application command; this allows the application at the decoding station to stop and start a video scene in a convenient way

• Switching between still images and moving images possible (by encoder command!)

Conclusion:• Suited for applications which do not require too much quality and where the content

doesn‘t move too fast (video conferencing)

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 10Chapter 3.2: Transfer and Control Protocols

H.263 and H.264

H.263 is similar to H.261, but…• … defines 5 image formats (sub-QCIF, QCIF, CIF, 4CIF, 16CIF)

• … error correction is optional• … consideration of GOBs and Slices like in MPEG

H.264 additionally …

• … allows variable block size (16×16, 8×16, 16×8, 8×8)• … uses a very simple 4×4 transform instead of DCT (astonishingly with negligible loss in

quality!)

• … allows to use any frame as a reference for prediction• … also allows bi-directional prediction (B-frames)

• … allows mixed frames – slices of one frame can be coded independently as I-slices, P-slices and B-slices!

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 11Chapter 3.2: Transfer and Control Protocols

H.323

A video conference is not only transferring video…• Audio transmission (G.7xx), synchronization with video stream

• Exchange of configuration data, signalling (H.225, H.245)• Whiteboard, chat, application sharing, data, fax … (T.x)• Transport subsystem (TCP, UDP, RTP, RTCP)

• …→ H.323 for coordination

Not only client terminals (telephones, video phones, NetMeeting, …) “speak” H.323, but also other system components:

• Gatekeeper: address translation (phone numbers to IP addresses), admissioncontrol and bandwidth management for multipoint connections, call authorization, call signal routing

• Gateway: integration with other voice networks• Multipoint control unit (MCU): coordinates several terminals taking part in a

conference• Proxy: e.g. used to pass a firewall

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 12Chapter 3.2: Transfer and Control Protocols

The ITU Family and the OSI Reference Model

NETWORK

DATA LINK

PHYSICAL

TRANSPORT

SESSION

PRESENTATION

APPLICATION

Supplementary Services

Audio Signal

Video Signal Data

Control

G.711 G.728

H.261 H.263 T.127

T.126

T.124

T.125/T.122

G.722 G.729

G.723.1

RTCP RAS RTP

H.450.3 H.450.2

H.450.1H.235

H.245 H.225UDP TCP

X.224.0

H.323

• Transfer of multimedia data uses UDP, transfer of control information uses TCP

• H.323 is an “umbrella” standard comprising all the other functionality

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 13Chapter 3.2: Transfer and Control Protocols

H.323 Network Components

• H.323 terminal can be workstations as well as more specalized end systems, e.g. IP phones

• The gateway enables an integration with existing systems like ISDN or older POTS (Plain Old Telephony System)

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 14Chapter 3.2: Transfer and Control Protocols

H.323 Components and Signaling

• H.245 – A protocol for capabilities advertisement, media channel establishment and conference control.

• H.225 - Call Control• Q.931 – A protocol for call control and call setup.

• RAS – Registration, admission and status protocol used for communicating between an H.323 endpoint and a gatekeeper.

POTS

Gatekeeper

Terminal

H.225/RAS messages

GatewayH.245 messages over call control channel

H.225/Q.931 messages over call signaling channel

H.225/RAS messages

H.225/Q.931 (optional) H.225/Q.931 (optional)

H.245 messages (optional) H.245 messages (optional)

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 15Chapter 3.2: Transfer and Control Protocols

Process for Establishing Communication

Establishing communication using H.323 occurs in five steps:

1. Call setup2. Initial communication and capabilities exchange

3. Audio/video communication establishment4. Call services

5. Call termination

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 16Chapter 3.2: Transfer and Control Protocols

Simplified H.323 Call Setup• Both endpoints have previously

registered with the gatekeeper• Terminal A initiate the call to the

gatekeeper

• The gatekeeper provides information for Terminal A to contact Terminal B

• Terminal A sends a SETUP message to Terminal B

• Terminal B responds with a Call Proceeding message and also contacts the gatekeeper for permission

• Terminal B sends a Alerting and Connect message

• Terminal B and A exchange H.245 messages to determine master/slave, terminal capabilities, and open logical channels

• The two terminals establish RTP media paths for data transmission

Terminal A Gatekeeper Terminal B

RAS messagesCall Signaling Messages

1. ARQ2. ACF

5. ARQ6. ACF

3. SETUP4. Call Proceeding

7.Alerting8.Connect

H.245 MessagesRTP Media Path

Note: This diagram only illustrates a simple point-to-point call setup where call signaling is not routed to the gatekeeper. Refer to the H.323 recommendation for more call setup scenarios.

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 17Chapter 3.2: Transfer and Control Protocols

Session Initiation Protocol SIP

• Defined by IETF• SIP long-term vision

� All telephone calls and video conference calls take place over the Internet� People are identified by names or e-mail addresses, rather than by phone

numbers

� You can reach the callee, no matter where the callee roams, no matter what IP device the callee is currently using

• SIP is an application layer signaling protocol that defines initiation, modification and termination of interactive multimedia communication sessions between multiple users

• Bases upon HTTP concepts (message syntax, SIP URLs, responses, …)

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 18Chapter 3.2: Transfer and Control Protocols

SIP Services

Setting up a call• Provides mechanisms for caller to let callee know he wants to establish a call

• Provides mechanisms so that caller and callee can agree on media type and encoding• Provides mechanisms to end call

Determine current IP address of callee• Maps mnemonic identifier to current IP address

Call management• Add new media streams during call• Change encoding during call

• Invite others • Transfer and hold calls

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 19Chapter 3.2: Transfer and Control Protocols

Setting up a Call to a known IP Address

• Alice’s SIP invite message indicates her port number & IP address. Indicates encoding that Alice prefers to receive (PCM µlaw)

• Bob’s 200 OK message indicates his port number, IP address & preferred encoding (GSM)

• SIP messages can be sent over TCP or UDP; here sent over RTP/UDP

• Default SIP port number is 506.

µ

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 20Chapter 3.2: Transfer and Control Protocols

Call Setup

• Codec negotiation� Suppose Bob doesn’t have PCM µlaw encoder� Bob will instead reply with 606 Not Acceptable Reply and list encoders he

can use� Alice can then send a new INVITE message, advertising an appropriate

encoder

• Rejecting the call� Bob can reject with replies “busy,” “gone,” “payment required,” “forbidden”

• Media can be sent over RTP or some other protocol.

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 21Chapter 3.2: Transfer and Control Protocols

Example of SIP message

INVITE sip:[email protected] SIP/2.0

Via: SIP/2.0/UDP 167.180.112.24

From: sip:[email protected]

To: sip:[email protected]

Call-ID: [email protected]

Content-Type: application/sdp

Content-Length: 885

c=IN IP4 167.180.112.24

m=audio 38060 RTP/AVP 0

Notes:

• HTTP message syntax• sdp = session description protocol

• Call-ID is unique for every call.

Here we don’t know Bob’s IP address. Intermediate SIPservers will be necessary.

Alice sends and receives SIP messages using the SIP default port number 506.

Alice specifies in Via:header that SIP client sends and receives SIP messages over UDP

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 22Chapter 3.2: Transfer and Control Protocols

Name Translation and User Location

• Caller wants to call callee, but only has callee’s name or e-mail address

• Need to get IP address of callee’s current host:� User moves around

� DHCP protocol� User has different IP devices (PC, PDA, car device)

• Result can be based on:

� Time of day (work, home)� Caller (don’t want boss to call you at home)

� Status of callee (calls sent to voicemail when callee is already talking to someone)

Service provided by SIP servers:

• SIP registrar server• SIP proxy server

• SIP redirect server• SIP location server

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 23Chapter 3.2: Transfer and Control Protocols

Redirect Server

SIP Distributed Architecture

Location Server

Registrar Server

User Agent

Proxy Server

Gateway

PSTN

SIP Components

Proxy Server

• User Agent Client (UAC) – An entity that initiates a call

• User Agent Server (UAS) – An entity that receives a call

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 24Chapter 3.2: Transfer and Control Protocols

SIP Registrar

• REGISTER sip:domain.com SIP/2.0

• Via: SIP/2.0/UDP 193.64.210.89 • From: sip:[email protected]

• To: sip:[email protected]• Expires: 3600

• When Bob starts SIP client, the client sends SIP REGISTER message to Bob’s registrar server

Register Message:

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 25Chapter 3.2: Transfer and Control Protocols

SIP Proxy

• Alice sends invite message to her proxy server� contains address sip:[email protected]

• Proxy responsible for routing SIP messages to callee� possibly through multiple proxies

• Callee sends response back through the same set of proxies

• Proxy returns SIP response message to Alice

� contains Bob’s IP address

• Interprets, rewrites or translates a request message before forwarding it

• Note: proxy is analogous to local DNS server

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 26Chapter 3.2: Transfer and Control Protocols

Example

Caller [email protected] places a call to [email protected]

(1) Jim sends INVITE message to umass SIP proxy

(2) Proxy forwards request to upenn registrar server

(3) upenn server returnsredirect response,indicating that it should try [email protected]

(4) umass proxy sends INVITE to eurecom registrar

(5) eurecom regristrar forwards INVITE to 197.87.54.21, which is running keith’s SIP client

SIP client217.123.56.89

SIP client197.87.54.21

SIP proxyumass.edu

SIP registrarupenn.edu

SIPregistrareurecom.fr

1

2

34

5

6

7

8

9

Note: also a SIP ack message, which is not shown

(6-8) SIP response sent back(9) Messages sent directly between clients

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 27Chapter 3.2: Transfer and Control Protocols

Comparison with H.323

• H.323 is a complete, vertically integrated suite of protocols for multimedia conferencing: signaling, registration, admission control, transport and codecs

• SIP is a single component. Works with RTP, but does not mandate it. Can be combined with other protocols and services.

• H.323 comes from the ITU (telephony)• SIP comes from IETF: Borrows much of its

concepts from HTTP. SIP has a Web flavor, whereas H.323 has a telephony flavor

H.323 is complex – SIP uses the KISS principle: Keep it simple and stupid

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 28Chapter 3.2: Transfer and Control Protocols

Transport Subsystem

How to transfer multimedia data in the Internet?

• TCP/UDP/IP: “best-effort service”• No guarantees on delay, loss

→ Today’s Internet multimedia applications use application-level techniques to mitigate (as best as possible) effects of delay and loss

E.g. streamed stored multimedia

• Application-level streaming techniques for making the best out of best effort service:� Client side buffering

� Use of UDP versus TCP� Multiple encodings of multimedia

But: what protocols on lower layers are suitable to support such application-level streaming?

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 29Chapter 3.2: Transfer and Control Protocols

Internet Multimedia: Simplest Approach

Audio and video are not really streamed:• Long delays until playout!

• Audio or video stored in files

• Files are transferred as HTTP object� Received in entirety at client

� Then passed to player

First: how can application level streaming be realized?

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 30Chapter 3.2: Transfer and Control Protocols

Internet Multimedia: Streaming Approach

• Browser GETs metafile with server contact information

• Browser launches player, passing metafile• Player contacts server

• Server streams audio/video to player

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 31Chapter 3.2: Transfer and Control Protocols

Streaming from a Streaming Server

• Separation of web server and streaming• This architecture allows for non-HTTP protocol between server and media player

• Can also use UDP instead of TCP

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 32Chapter 3.2: Transfer and Control Protocols

Streaming Multimedia: Client Buffering

• In streaming, data can arrive with variable rate by network delay and jitter• Thus: client-side buffering for playout delay for compensation of these problems

bufferedvideo

variable fillrate x(t)

constantdrain

rate d

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 33Chapter 3.2: Transfer and Control Protocols

Streaming Multimedia

What transport protocol to use for such an approach?

UDP • Server sends at rate appropriate for client (oblivious to network congestion !)

• Often send rate = encoding rate = constant rate• Then: fill rate = constant rate - packet loss• Short playout delay (2-5 seconds) to compensate for network jitter

• Error recovery: if time permits

TCP

• Send at maximum possible rate under TCP• Fill rate fluctuates due to TCP congestion control

• Larger playout delay: smooth TCP delivery rate• HTTP/TCP passes more easily through firewalls

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 34Chapter 3.2: Transfer and Control Protocols

Solution: RTSP

HTTP• Does not target multimedia content

• No commands for fast forward, etc.

Real-time Streaming Protocol RTSP

• Client-server application layer protocol• For user to control display: rewind, fast forward, pause, resume, repositioning, etc…

What it doesn’t do:• Does not define how audio/video is encapsulated for streaming over network• Does not restrict how streamed media is transported; it can be transported over UDP

or TCP• Does not specify how the media player buffers audio/video

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 35Chapter 3.2: Transfer and Control Protocols

RTSP: Out of Band Control

FTP uses an “out-of-band” control channel:• A file is transferred over one TCP connection

• Control information (directory changes, file deletion, file renaming, etc.) is sent over a separate TCP connection

• The “out-of-band” and “in-band” channels use different port numbers

RTSP messages are also sent out-of-band:• RTSP control messages use different port numbers than the media stream

(Port 554): out-of-band

• The media stream is considered “in-band”

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 36Chapter 3.2: Transfer and Control Protocols

RTSP Example

Scenario:• Metafile communicated to web browser• Browser launches player

• Player sets up an RTSP control connection and a data connection to streaming server

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 37Chapter 3.2: Transfer and Control Protocols

Metafile Example

<title>Twister</title>

<session> <group language=en lipsync>

<switch>

<track type=audio e="PCMU/8000/1"

src = "rtsp://audio.example.com/twister/audio.en/lofi"> <track type=audio

e="DVI4/16000/2" pt="90 DVI4/8000/1" src="rtsp://audio.example.com/twister/audio.en/hifi">

</switch> <track type="video/jpeg"

src="rtsp://video.example.com/twister/video"> </group>

</session>

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 38Chapter 3.2: Transfer and Control Protocols

RTSP Exchange Example

C: SETUP rtsp://audio.example.com/twister/audio RTSP/1.0

Transport: rtp/udp; compression; port=3056; mode=PLAY

S: RTSP/1.0 200 1 OK

Session 4231

C: PLAY rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0

Session: 4231 Range: npt=0-

C: PAUSE rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0 Session: 4231

Range: npt=37

C: TEARDOWN rtsp://audio.example.com/twister/audio.en/lofi RTSP/1.0

Session: 4231

S: 200 3 OK

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 39Chapter 3.2: Transfer and Control Protocols

Example: Internet Phone

Introduce Internet Phone by way of an example

• Speaker’s audio: alternating talk spurts, silent periods� 64 kbit/s during talk spurt

• Packets are generated only during talk spurts

� 20 msec chunks at 64 kbit/sec: 160 bytes data• Application-layer header added to each chunk

• Chunk and header are encapsulated into a UDP segment.• Application sends UDP segments into socket every 20 msec during talkspurt

Required:

• Network loss: IP datagram lost due to network congestion (router buffer overflow)• Delay loss: IP datagram arrives too late for playout at receiver

� Delays: processing, queueing in network; end-system (sender, receiver) delays� Typical maximum tolerable delay: 400 ms

• Loss tolerance: depending on voice encoding, losses concealed, packet loss rates between 1% and 10% can be tolerated

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 40Chapter 3.2: Transfer and Control Protocols

constant bit rate

transmission

Cum

ulat

ive

data

time

variablenetwork

delay(jitter)

clientreception

constant bit rate playout

at client

client playoutdelay

buffe

red

data

Jitter

• Consider the end-to-end delays of two consecutive packets: difference can be more or less than 20 msec

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 41Chapter 3.2: Transfer and Control Protocols

Internet Phone: Fixed Playout Delay

Receiver attempts to playout each chunk exactly q msecs after chunk was generated

• chunk has timestamp t: play out chunk at t + q

• chunk arrives after t + q: data arrives too late for playout, data “lost”

Tradeoff for q:• large q:

less packet loss• small q: better

interactive experience

20 msec

r: receiving of first packetp: first playout schedulep’: second playout schedule

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 42Chapter 3.2: Transfer and Control Protocols

Adaptive Playout Delay

i

i

i

*i i i

i

t timestamp of the ith packet

r the time packet i is received by receiver

p the time packet i is played at receiver

d = r t network delay for ith packet

d estimate of average network delay after r

===

− == eceiving ith packet

Dynamic estimate of average delay at receiver: *i i 1 id (1 u ) d u d−= − ⋅ + ⋅

where u is a fixed constant (e.g., u = .01)

• Goal: minimize playout delay, keeping late loss rate low

• Approach: adaptive playout delay adjustment:� Estimate network delay, adjust playout delay at beginning of each talk spurt� Silent periods compressed and elongated

� Chunks still played out every 20 msec during talk spurt.

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 43Chapter 3.2: Transfer and Control Protocols

Also useful to estimate the average deviation of the delay vi (jitter):

*i i 1 i iv (1 u ) v u | d d |−= − ⋅ + ⋅ −

The estimates di and vi are calculated for every received packet, although they are only used at the beginning of a talk spurt.

For first packet in talk spurt, playout time is:

i i i ip t d Kv= + +

where K is a positive constant.

Remaining packets in talkspurt are played out periodically

Adaptive Playout Delay

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 44Chapter 3.2: Transfer and Control Protocols

Recovery from Packet Loss

Forward error correction (FEC): simple scheme• For every group of n chunks create a redundant chunk by exclusive OR-ing the n

original chunks

• Send out n+1 chunks, increasing the bandwidth by factor 1/n.• Can reconstruct the original n chunks if there is at most one lost chunk from the n+1

chunks

• Playout delay needs to be fixed to the time to receive all n+1 packets• Tradeoff:

� increase n, less bandwidth waste� increase n, longer playout delay

� increase n, higher probability that 2 or more chunks will be lost

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 45Chapter 3.2: Transfer and Control Protocols

Recovery from Packet Loss

Other FEC scheme:• “Piggyback lower

quality stream”

• Send lower resolutionaudio stream as theredundant information

• For example, nominal stream PCM at 64 kbpsand redundant streamGSM at 13 kbps

• Whenever there is non-consecutive loss, the receiver can conceal the loss

• Can also append (n-1)st and (n-2)nd low-bit rate chunk

lower quality

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 46Chapter 3.2: Transfer and Control Protocols

Recovery from Packet Loss

Interleaving

• Chunks are broken up into smaller units• For example, 45 msec units per chunk• Packet contains small units from

different chunks

• If packet is lost, still have most of every chunk

• Has no redundancy overhead

• But adds to playout delay

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 47Chapter 3.2: Transfer and Control Protocols

Summary: Internet Multimedia: Bag of Tricks

• Use UDP to avoid TCP congestion control (delays) for time-sensitive traffic• Client-side adaptive playout delay to compensate for network delay

• Server side matches stream bandwidth to available client-to-server path bandwidth� Chose among pre-encoded stream rates

� Dynamic server encoding rate• Error recovery (on top of UDP)

� FEC, interleaving� Retransmissions, time permitting� Conceal errors: repeat nearby data

→ Provide a standardized transport protocol which supports such tricks: RTP

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 48Chapter 3.2: Transfer and Control Protocols

Real-Time Protocol (RTP)

RTSP still would have to use the unreliable UDP or the “slow” TCP – better define a “new” transport protocol for combining speed with reliability:Real-Time Transport Protocol (RTP)

• RTP specifies a packet structure for packets carrying audio and video data• RTP packet provides

� Payload type identification� Packet sequence numbering

� Timestamping• RTP runs in the end systems

• RTP packets are encapsulated in UDP segments• Interoperability: if two Internet phone applications run RTP, then they may be able to

work together

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 49Chapter 3.2: Transfer and Control Protocols

RTP libraries provide a transport-layer interface that extend UDP: • Port numbers, IP addresses• Payload type identification

• Packet sequence numbering• Time-stamping

RTP runs on Top of UDP

Transport Layer

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 50Chapter 3.2: Transfer and Control Protocols

RTP Header

• Ver.: Version number of the RTP protocol in use• P: packet size was padded to a multiple of 32 bit

• X: an extension header is used• CC: indicates the number of sources

• M: User-specific mark. Can e.g. mark the beginning of a word on an audio channel.• Contributing Source Identifier: used by mixers in the studio. The mixed flows are listed here.

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 51Chapter 3.2: Transfer and Control Protocols

RTP Header

Payload Type (7 bits)Indicates type of encoding currently being used. If the sender changes encoding in

middle of transmission, it informs the receiver through this payload type field• Payload type 0: PCM µ-law, 64 kbps

• Payload type 3, GSM, 13 kbps• Payload type 26, Motion JPEG• Payload type 31, H.261

• Payload type 33, MPEG2 video

Sequence Number (16 bits)Increments by one for each RTP packet sent, and may be used to detect packet loss and

to restore packet sequence

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 52Chapter 3.2: Transfer and Control Protocols

RTP Header

Timestamp field (32 bits long)• Reflects the sampling instant of the first byte in the RTP data packet• For audio, timestamp clock typically increments by one for each sampling period (for

example, each 125 µsecs for a 8 KHz sampling clock)

• If application generates chunks of 160 encoded samples, then timestamp increases by 160 for each RTP packet when source is active. Timestamp clock continues to increase at constant rate when source is inactive.

• The timestamp gives the receiver the relative time (with respect to the first data) when to playout the data

Synchronization Source Identifier field (32 bits long)• Identifies the source of the RTP stream• Each stream in a RTP session should have a distinct identifier

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 53Chapter 3.2: Transfer and Control Protocols

RTP and QoS

• RTP only adds some information to the UDP header needed for kind of reliability

• RTP does not provide any mechanism to ensure timely delivery of data or provide other quality of service guarantees

• RTP encapsulation is only seen at the end systems: it is not seen by intermediate routers. � Routers providing best-effort service do not make any special effort to ensure

that RTP packets arrive at the destination in a timely matter.

• Usage of (and reaction to) the information in the RTP header are left over to the application

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 54Chapter 3.2: Transfer and Control Protocols

Real-Time Control Protocol (RTCP)

• Works in conjunction with RTP

• Each participant in RTP session periodically transmits RTCP control packets to all other participants

• Each RTCP packet contains sender and/or receiver reports

� report statistics useful to application• Statistics include number of packets sent, number of packets lost, interarrival jitter,

etc.

• Feedback can be used to control performance� Sender may modify its transmissions based on feedback

• To limit traffic, each participant reduces his RTCP traffic as the number of conference participants increases

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 55Chapter 3.2: Transfer and Control Protocols

RTCP

RTCP controls the data flow:

• Feedback to the sender about QoS on receiver side• Data losses, delay and jitter are reported

• Note: RTCP does not provide corrective actions - this is left over to the application

Application

RTP / RTCP

UDP

IP

Application

RTP / RTCP

UDP

IP

RTP RTP RTP RTP RTPRTP

RTCP RTCP RTCPRTCP

Sender Receiver

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 56Chapter 3.2: Transfer and Control Protocols

Application Example: Voice over IP (VoIP)

Telephony using an IP network with standardized protocols: VoIP• Transferring speech and

signaling information

• Not only internally in a IP network, also integration with “normal”telephony systems

IP network(Internet/Intranet)IP phones

IP terminal

ISDN phone

VoIPGateway

Telecommunicationnetwork

Phone numbers

IP addresses and virtual phone numbers

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 57Chapter 3.2: Transfer and Control Protocols

VoIP-based Telephony System

R

VoIP-Gateway

ISDNPTSS

Branch Company central

ISDNPTSS

ISDN

R

VoIP-Gateway

IP network(Internet)

TeleworkingPCs

R = RouterPTSS = Private Telecommunications Switching System

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 58Chapter 3.2: Transfer and Control Protocols

Realization with H.323

H.323terminal

H.323terminalGate-

keeper MCU

Gateway

Networkwithout QoS guarantees

H.323 zone

H.324

POTS

H.321

ATM network

H.320

ISDN

H.323 gives us all functionality we need to realize an IP-based telephony integrated with conventional solutions

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 59Chapter 3.2: Transfer and Control Protocols

Call Setup

IP phone

Call isinitiated

Ringing

Pick up receiver

Dialing tone

Connection

PTSS

ISDN phoneVoIP

Gateway

Intranet

GK

ISDN

IP[TCP[SETUP[...] ]]

IP[TCP[Alerting [...]]]

IP[TCP[Connectt[...] ]]

B channelRTP channel

Voice IP[UDP[RTP[Voice]]]

Connect

Alerting

CallProceeding

Phone nr => IP-Adr.?

SETUP[...]

GK: Gatekeeper

Lehrstuhl für Informatik 4

Kommunikation und verteilte Systeme

Page 60Chapter 3.2: Transfer and Control Protocols

VoIP Future

At the moment, VoIP products based on H.323 are most popular

• But: complex, and telecommunication networks tend to converge with IP networks→ Use protocols better integrated with the IP world

→ SIP together with a MGCP (Media Gateway Control Protocol) gets more and more significance

→ Better integration with web applications

→ SIP seems to be the multimedia signaling protocol for the future

Still a problem: quality of an IP transmission; how to improve QoS in the Internet?