Multimedia Communications over the Internet

IP Packet-Switching Networks

• Packet-switching protocols based on the Internet Protocol (IP) generally consist of a variety of different subnetworks of various technologies – The datalink and physical layer are largely outside the scope of the Internet efforts• IP is a connectionless (datagram) network layer – Different packets may traverse various routes between source and destination (they may arrive out of order or be duplicated) – Packets are transmitted on a best effort basis (no guarantee that a packet will arrive at destination

IP Networks (cont’d)

• IP provides for the transmission of blocks of data called datagrams from source to destination, where source and destination are hosts identified by fixed-length addresses • Datagrams can be as large as 64 KB, but usually the are ~1500 bytes long. • IP also provides for fragmentation and reassembly of long datagrams, if necessary, for transmission through small packet networks. • IP basically runs over any Media Access Control (MAC) protocol

Internet Protocol Relationships (Non Real-Time)

• At the highest level, the users invoke application programs that access services available across the Internet. An application interacts with the transport-level protocol to send or receive data.• Each application program chooses the size of transport needed, which can be either a sequence of individual messages or a continuous stream of bytes.

IP Relationships (cont’d)

• Above IP, there are two transport layer options: TCP and UDP. – TCP provides end-to-end communication. It takes care of reliable, error-free transfer of data, and in- sequence delivery. UDP has less overhead compared to TCP, but does not guarantee transfers.

– Both protocols support multiplexing, i.e. they allow several distinct streams of data between two hosts • Streams are labeled by source and destination port numbers

UDP

• The User Datagram Protocol (UDP) offers only minimal services beyond those provided by the network layer – Multiplexing via port number – Checksum for the payload in the packet • IP provides only a checksum for the IP header

TCP

• The Transmission Control Protocol (TCP) offers a reliable, sequenced byte stream service between two Internet hosts• Reliability is achieved by retransmission of lost or erroneous packets – Requires acknowledgment of received data• Flow control is achieved by means of a sliding window mechanism – Also used for congestion control: the sender reduces the window size in the case of traffic congestion • The sender probes for the currently available bandwidth by gradually increasing its window size until it senses packet loss - at which point it quickly reduces the window size

TCP (cont’d)

• TCP is less suited to multimedia data than UDP – It trades reliability for delay jitter

– Since it enforces in-order delivery, a single lost packet can hold up packets arrived after it

– To improve delay behavior, an implementation is free to accept out-of-order packets and to acknowledge packets that have not been received but are excessively delayed

• However, this may interfere with the TCP congestion control

– The TCP congestion control mechanism imposes throughput limitations that may change on very short time scales

Connectivity Requirements

• Connectivity requirements examples – Accommodating a listening audience of 25,000 streams daily with an average listening time of 6 minutes per stream requires, on average, only 100 concurrent streams

– Accommodating a daily listening audience of 250,000 streams, with an average listening time of 20 minutes per stream, requires an average of 6,000 concurrent streams

Connectivity Requirements (cont’d)

• Due to variable traffic patterns, load is not constant throughout the day, therefore the peak number of streams required may be twice or three times the average

• When implemented, broadcast protocols (to an entire subnetwork) or multicast protocols (to members of a multicast group in various scattered subnetworks) allow to reduce the load on the network by allowing many user to share a single stream from the server

IP Multicast

• IP Multicast allows a sender to transmit an IP packet to multiple receivers. Three possible ways for multicast: – Setting up multiple virtual circuits (not possible in Internet) – Including list of addresses in packet header (what happens if 1,000,000 addresses?)

– Radio-like approach (used by IP Multicast) • Sender chooses an IP address from the class-D set (244.0.0.0 to 239.255.255.255). Receivers subscribe to multicast finding out the multicast address. • An IP multicast group can have any number of senders and receivers. Membership is dynamic.

• IP Multicast is independent of transport mechanism(but TCP cannot be used)

HTTP

‧ HTTP (Hyper-Text Transfer Protocol) is built on top of TCP/IP ‧ For transmission through firewalls, often multimedia must be encapsulated in a HTTP envelope, lacking most of the required real-time features ‧ Multimedia data is downloaded on the client

computer. Fast-start techniques can be used tobegin playback as soon as enough of the content has been downloaded to the client

• UDP and TCP protocols are otherwise used

– UDP is sometimes blocked by firewalls

Requirements for Real-Time Multimedia

• Sequencing. Packets must be reordered in real time at the receiver. If a packet is lost, must be detected and compensated without retransmission • Intramedia synchronization. Need some form of “time stamping” to know when to playback packets – Very important for VBR traffic

• Intermedia synchronization. E.g., audio must be synchronized with video (lip-sync).• Payload identification. E.g., for media filtering• Frame indication. For synchronized delivery, it is useful to indicate when a video frame (or audio segment) begins or end.

RTP

‧ RTP (Real-Time Protocol) is a transport protocol for audio and videoconferences and other multiparticipant real-time applications. ‧ Designed to run over multicast IP ‧ Light-weight protocol, without error correction, flow control, or guaranteed time delivery functionality.

‧ Offers services such as playout synchronization, demultiplexing, media identification, and active- party identification.

Functionalities of RTP

• Re-sequencing and loss detection• Multicast-friendly• Media-independent (voice, video,…)• Explicit support for mixers and translator – Mixers: take media from several users and mix them into one media stream (e.g., conference bridge) – Translators: take a single media stream, convert it to another format, and send it out (e.g. media filtering)

Functionalities of RTP (cont’d)

• QoS feedback (via RTCP) – RTP sources can use this information to adjust their data rate (media scaling)

• Loose session control – Using RTCP, participants can periodically distribute identification information (name, e-mail address,…) – Provides awareness of who is participating in a session without maintaining a centralized conference participant registry

• Encryption

RTP (cont’d)

• RTP specifies a packet structure for packets carrying audio and video data – Payload type identification – Packet sequence numbering – Time-stamping

• RTP runs on top of UDP (can be viewed as a sub-layer of the transport layer)

• RTP encapsulation is only seen at the end systems (not at the intermediate routers)

RTP - Example

• Consider sending 64 Kb/s PCM-encoded voice over RTP

• Application collects the encoded data in chunks, e.g. every 20 ms = 160 bytes/chunk

• The audio chunk, along with the RTP header, forms the RTP packet, which is encapsulated into a UDP packet

• The RTP header indicates the type of audio encoding in each packet. Sender can change encoding during a conference

• RTP header also contains sequence number and timestamps

RTP Streams• RTP allows each source (e.g., a camera or a microphone) to be assigned its own independent stream of packets – E.g., for a videoconference between two participants, 4 streams could be opened: two streams for audio (one in each direction) and two for video

• However, MPEG-1 and MPEG-2 bundle audio and video into a single stream during the encoding process - then only one RTP stream is generated in each direction.

Real-Time Control Protocol (RTCP)

• Works in conjunction with RTP• Each participant in a RTP session periodically transmits RTCP control packets to all other participants. – Each RTCP packet contains sender and/or receiver reports with statistics useful to the application – E.g.: number of packets sent, number of packets lost, inter-arrival jitter, etc.

• This feedback information can be used to control performance and diagnostic purposes – E.g. the sender may modify its transmission based on feedback

RTCP (cont’d)

• RTCP can be used to synchronize different media stream within a RTP session

– E.g.: videoconference where each sender generates one RTP stream for video and one for audio – Timestamps in RTP packets are tied to video/audio sampling clocks (not wall-clock time, ie real time) – Each RTCP sender-report packet contains, for the most recently generated packet in the associated stream, the timestamp of the RTP packet and the wall-clock time of when the packet was created – Receivers can use this association to synchronize the playout of audio and video

Protocols for Real-Time (cont’d)

• RTSP (Real-Time Streaming Protocol) provides methods to realize commands (play, fast-forward, fast-rewind, pause, stop) similar to the functionality provided by CD players or VCRs. – Can control either a single or several time- synchronized streams of continuous media. – Can act as a network remote control for multimedia servers and can run over TCP or UDP

Live vs. Video-On-Demand

• Live streaming is a live broadcast which allows users to join a session in which real time media is being sent over a network – Because these streams are live, users are not allowed to jump around to any point in time – Live streaming can exploit multicast or broadcast protocols• Video-on-demand represents content stored on a streaming server which can be viewed at any time – User is allowed to jump to any point in time in the media – Only unicast protocols are allowed