multimedia concepts 1 multimedia concepts. multimedia concepts 2 topics multimedia basic terms ...

64
1 MULTIMEDIA CONCEPTS Multimedia Concepts

Upload: nathan-snow

Post on 25-Dec-2015

321 views

Category:

Documents


6 download

TRANSCRIPT

Page 1: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

1

MULTIMEDIA CONCEPTS

Multimedia Concepts

Page 2: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

2

MULTIMEDIA CONCEPTS

Topics

Multimedia Basic Terms Multimedia Compression/Decompression Multimedia Protocols

Page 3: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

3

MULTIMEDIA CONCEPTS

Basic Terms

Multimedia Definition The term multimedia usually implies that at

least one of text ( structured/unstructured, hypertext, etc), graphics (drawings), or image (discrete media) is associated with either audio or motion video information (continuous media).

Page 4: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

4

MULTIMEDIA CONCEPTS

Basic Terms

Multimedia streaming Multimedia streaming is the overlapping

the playout of the data at the receiver with the transmission by the sender.

A video stream consists of a sequence of images or frames.

• A frame consists of a grid of pixels. (Table 1)

An audio stream consists of a sequence of audio samples.

Page 5: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

5

MULTIMEDIA CONCEPTS

Basic Terms

Term Definition

Pixel

Frame

Stream

Session

Presentation

Picture element

Two-dimensional grid of pixels

Sequence of frames over time

Synchronized set of streams

Set of multimedia sessions

Table 1. Hierarchy of multimedia content.

Page 6: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

6

MULTIMEDIA CONCEPTS

Basic Terms

The advantage of streaming is that it can enable easier access to multimedia resources.

Another possibility is the integration of video and audio with other web-based applications, such as chat and other real-time collaboration tools.

Page 7: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

7

MULTIMEDIA CONCEPTS

Basic Terms Streaming vs. downloading What Is The Difference Between Downloading

and Streaming?When you download a video, you have to copy the

entire file to your hard disk before you can play it.

Page 8: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

8

MULTIMEDIA CONCEPTS

Basic TermsWhen the video is streamed, there is a small wait as the stream 'buffers' but there is no need to save the file. Streaming is the act of sending media files (audio and/or video) over the Internet from one computer to another computer so that the media plays as it is being delivered.

Page 9: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

9

MULTIMEDIA CONCEPTS

Basic Terms

Multimedia Streaming

MediaEncoding

AudioVideoAnimation

Clients

Send RequestTo ServersWeb

Server

Send Request to Media Server

MediaServer

ProprietaryFormat

•Multicast capable•More Robust•Access to Storage•Relieves Web Server

Send StreamTo Clients

•Standalone player•Java based player•Browser plug-in player•Appliance

•Decode•Buffer•Sync.

Figure 2. To hear or view a media file without downloading it

Page 10: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

10

MULTIMEDIA CONCEPTS

Basic Terms

A media stream proceeds through the following stages before it is displayed to a recipient:

Capturing The audio or video stream must be captured

from an analog device, such as a microphone or a video camera, and converting to a digital form.

25 fps (frame per second) for video and 16-bit for audio is suitable.

Page 11: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

11

MULTIMEDIA CONCEPTS

Basic Terms Encoding

An encoder converts the raw digital data into a particular audio or video format.

Storing A server may store the encoded stream for

future transmission.

Delivering The stream is transmitted to one or more

recipients. A live stream may be transmitted as it is captured and encoded, whereas a prerecorded stream is transmitted by a server.

Page 12: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

12

MULTIMEDIA CONCEPTS

Basic Terms Decoding

The receiver decodes and displays the data as they arrive. Alternatively, the receiver may store the entire stream before initiating playback.

(Figure 3)

Page 13: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

13

MULTIMEDIA CONCEPTS

Basic Terms

Figure 3. Capturing Video

Page 14: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

14

MULTIMEDIA CONCEPTS

Basic Terms

There are two different types of streaming: Progressive download

The client begins playback of the multimedia file as it is delivered. The file is ultimately stored on the client computer.

Use standard web server Quality is better than real-time streaming

Real-time streaming The multimedia file is delivered to the client

computer but the file is not stored on the client computer.

Require a special streaming server

Page 15: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

15

MULTIMEDIA CONCEPTS

Basic Terms two different types of real-time streaming:

Live streaming used to deliver a live event while it is

occurring.

Examples: live soccer game, live concerts, live radio, and videoconferences.

On-demand streaming used to deliver archived media streams.

Examples: video clips, movies, and

lectures.

Page 16: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

16

MULTIMEDIA CONCEPTS

Basic Terms Why Streaming Media?

No waiting for complete downloads. Streamed files are not written to disk. Presentation of live events is possible.

Page 17: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

17

MULTIMEDIA CONCEPTS

Basic Terms Major streaming formats

Microsoft Windows Media Real Networks Apple Quicktime

Page 18: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

18

MULTIMEDIA CONCEPTS

Basic Terms How does streaming work?

Figure 4. Streaming media development process

Page 19: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

19

MULTIMEDIA CONCEPTS

Basic Terms How does streaming work?

Figure 5. Streaming media playback

Page 20: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

20

MULTIMEDIA CONCEPTS

Basic Terms

Figure 6. Streaming media from a conventional Web server

Page 21: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

21

MULTIMEDIA CONCEPTS

Basic Terms

Figure 7. Realtime Streaming protocol

Page 22: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

22

MULTIMEDIA CONCEPTS

Basic Terms

RTSP States SETUP - the server allocates resources

for a client session. PLAY - the server delivers a stream to a

client session. PAUSE - the server suspends delivery of

a stream. TEARDOWN - the server breaks down

the connection and releases the resources allocated for the session.

Page 23: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

23

MULTIMEDIA CONCEPTS

Basic Terms RTSP State Machine

Figure 8. RTSP state machihne

Page 24: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

24

MULTIMEDIA CONCEPTS

Basic Terms

RTSP operation

Page 25: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

25

MULTIMEDIA CONCEPTS

Basic Terms

Clip Clip is a media file that contains audio,

video, or both.

Page 26: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

26

MULTIMEDIA CONCEPTS

Basic Terms

Webcast A webcast uses streaming media

technology to take a single content source and distribute it to many simultaneous listeners/viewers by broadcasting over the Internet.

Page 27: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

27

MULTIMEDIA CONCEPTS

Basic Terms Three general methods for delivering content

from a server to a client across a network: Unicasting

The server delivers the content to a single client. 

Broadcasting The server delivers the content to all clients,

regardless whether they want the content or not. 

Multicasting The server delivers the content to a group

of receivers who indicate they wish to receive the content.

Page 28: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

28

MULTIMEDIA CONCEPTS

Basic Terms

Broadcasting Broadcast means a piece of information is

sent or transmitted from one point to all other points.

There is just one sender, but the information is simultaneously sent to all connected receivers.

In telecommunications, broadcasting means propagation of a flow of information from one source to all potential recipients.

Page 29: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

29

MULTIMEDIA CONCEPTS

Basic Terms

Broadcasting In networking, a distinction is made

between broadcasting and multicasting. Broadcasting sends a message to everyone

on the network. Multicasting sends a message to a select

list of recipients.

Page 30: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

30

MULTIMEDIA CONCEPTS

Basic Terms

Figure 10. Unicast delivering

Page 31: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

31

MULTIMEDIA CONCEPTS

Basic Terms

Figure 11. Multicast delivering

Page 32: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

32

MULTIMEDIA CONCEPTS

Basic Terms Unicast/Multicast

Host

Router

Unicast

Host

Router

Multicast

Figure 12. Unicast/Multicast delivering

Page 33: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

33

MULTIMEDIA CONCEPTS

Basic Terms Bit rate

Bit rate is amount of data that can be carried from one point to another in a given time period (usually a second).

Bit rate is sometimes called data rate or transfer rate or bandwidth.

Multiple Bit Rate Encoding Combine several streams with different bit rate

into a single file The appropriate bit rate stream is automatically

selected

(Figure 13)

Page 34: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

34

MULTIMEDIA CONCEPTS

Basic Terms

Figure 13. Multiple bit rate encoding

Page 35: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

35

MULTIMEDIA CONCEPTS

Basic Terms

Aspect Ratio This is the ratio of width to height that the

encoded video will be. This information is present in the output

video stream and used by the decoder to display the video at the correct aspect ratio.

The computer display is designed for an aspect ratio of 1.33:1, which means that the width of the display area is only 1.33 times the height, almost square.

Page 36: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

36

MULTIMEDIA CONCEPTS

Basic Terms

Frame Frame means one still picture. By changing still pictures (frames) quickly,

human eye "thinks" that the video is smooth and can't separate pictures from each others and instead sees smooth video.

Page 37: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

37

MULTIMEDIA CONCEPTS

Basic Terms

Frame rate Frame rate is the number of video frames

(complete pictures) that will be presented to the viewer each second.

Human eye can sees smooth video with the frame rate more than ~24 fps (frames per second).

In American TV system, NTSC, the frame rate is approximately 29.97 fps.

In European PAL system the frame rate is 25fps.

Page 38: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

38

MULTIMEDIA CONCEPTS

Basic Terms

Frame Buffer Frame buffer is a special memory to hold

the complete digital representation of the frame to be displayed on a computer screen.

The frame buffer is scanned line by line by the digital-to-analog converter system of the display.

Page 39: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

39

MULTIMEDIA CONCEPTS

Basic Terms

Color depth Color depth or bit depth is the number of

bits used to represent the color of a single pixel in a bitmapped image or video frame buffer.

It is known as bits per pixel (bpp). Higher color depth gives a broader range

of distinct colors. (Table 2)

Page 40: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

40

MULTIMEDIA CONCEPTS

Basic Terms

Bit-Depth Number of Colors1 2

(monochrome)

2 4(CGA)

4 16(EGA)

8 256(VGA)

16 65,536(High Color, XGA)

24 16,777,216(True Color, SVGA)

32 16,777,216(True Color + Alpha Channel)

Table 2. Color depth chart.

Page 41: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

41

MULTIMEDIA CONCEPTS

Basic Terms

Jitter In transmission technology, jitter refers to

the variation of the delay generated by the transmission equipment.

In data communications, jitter refers to the variation over time of the network transit delay.

Page 42: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

42

MULTIMEDIA CONCEPTS

Multimedia Compression

Lossless compression Lossy compression Compression/Coding Standards Decompression Codec

Page 43: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

43

MULTIMEDIA CONCEPTS

Multimedia Compression

Compression is the process of eliminating redundant information to decrease file size.

Compression converts frames and pixels to mathematical algorithms that the computer can understand.

Decompression converts mathematical algorithms back to frames and pixels for playback.

Page 44: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

44

MULTIMEDIA CONCEPTS

Multimedia Compression

Two compression methods are: Lossless compression

• Run-Length coding• Huffman coding• Arithmetic coding

Lossy compression

• Transform coding

Page 45: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

45

MULTIMEDIA CONCEPTS

Lossless compression

Lossless compression retains all of the data of the original file as it's converted to a smaller file size.

In lossless compression the information is recovered without any alteration after the decompression stage.

When a lossless file is opened, algorithms restore all compressed information, creating a duplicate of the source file.

Page 46: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

46

MULTIMEDIA CONCEPTS

Lossless compression

It generally preferred for creating high-quality or professional applications.

Lossless compression is applied where the accuracy of the information is essential, such as in medical imaging where it's important to retain fine detail.

Lossless compression is also called bit-preserving compression.

Page 47: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

47

MULTIMEDIA CONCEPTS

Lossy compression

Lossy compression refers to the case where the decompressed information is different from the original uncompressed information.

With this kind of compression, some of the source file's information is discarded to conserve space.

Page 48: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

48

MULTIMEDIA CONCEPTS

Lossy compression

When the file is decompressed, this information is reconstructed through algorithms.

This method results in some loss of sound quality or image detail when compared to the original.

This mode is suitable for most continuous media such as sound and motion video as well as for many images.

Page 49: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

49

MULTIMEDIA CONCEPTS

Compression/Coding Standards

The MPEG Standards MPEG standards developed and managed

by Motion Picture Experts Group (MPEG) MPEG-1: VCD MPEG-2: DVD, HDTV MPEG-4: Content-based video coding MPEG-7: Multimedia indexing and retrieval MPEG-21: Multimedia delivery and

consumption

Page 50: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

50

MULTIMEDIA CONCEPTS

Compression/Coding Standards

The MPEG-1 Released in 1992 A standard for coded representation of

• Moving pictures• Audio• Combination of above

Typical application – video CD (VCD)

Page 51: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

51

MULTIMEDIA CONCEPTS

Compression/Coding Standards

The MPEG-2 Standard Released in 1994 A standard to provide video quality not lower than

NTSC/PAL with bit rates target between 2-10 Mbit/s

Applications Digital cable TV distribution Networked database service via ATM Digital video tape recorder (VTR) Satellite and terrestrial digital broadcasting

distribution

It also supports HDTV applications, and so pre-emptied MPEG-3 standard

Page 52: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

52

MULTIMEDIA CONCEPTS

Compression/Coding Standards

The MPEG-4 Standard First released in 1998, and targeted for

content-based multimedia applications and low bit-rate video coding.

Algorithms and tools for coding and flexible representation of audio/video to meet the challenges of multimedia applications.

The objective of low bit-rate video coding was later accomplished by H.264, the convergence of ITU-T H.263 and MPEG-2.

Page 53: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

53

MULTIMEDIA CONCEPTS

Compression/Coding Standards

The MPEG-7 First release in 2001 Official name:

Multimedia Content Description Interface

Objective: To allow efficient search for multimedia

content using standardized descriptors

The main research issues: Optimum search engine Feature analysis & query design

Page 54: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

54

MULTIMEDIA CONCEPTS

Compression/Coding Standards

The MPEG-21 Standard Aim at defining a normative open

framework for multimedia delivery and consumption for use by all the players in the delivery and consumption chain.

Page 55: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

55

MULTIMEDIA CONCEPTS

Decompression

Decompression is the process by which compressed information is expanded by addition of the redundant information eliminated at the compression stage.

After decompression, the resulting information may be identical to the original – lossless compression – or be different – lossy compression.

Page 56: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

56

MULTIMEDIA CONCEPTS

Codec

Codec stands for Coder/Decoder or Compression/Decompression.

Codec is a piece of software or a driver that is mostly for compression to reduce file size but may also do some formatting.

Compression is the primary function of the Codec.

Page 57: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

57

MULTIMEDIA CONCEPTS

Codec

With codec, your system recognizes the encoded video/audio format and allows you to play (decode) the audio/video file in a particular format.

Page 58: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

58

MULTIMEDIA CONCEPTS

Codec Examples

Video MPEG1, MPEG2, DIVX, WMV(WINDOWS

MEDIA VIDEO), MPEG4-H264, RealVideo

Audio MP3,ATRAC, AAC, WMA (WINDOWS

MEDIA AUDIO), DTS, RealAudio

Image JPEG, JPEG2000, PNG, GIF

Data ZIP, STUFFIT

Page 59: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

59

MULTIMEDIA CONCEPTS

Multimedia Protocols

TCP (Transmission Control Protocol) UDP (User Datagram Protocol) RTP (Real-time Transport Protocol) RSVP (Resource ReSerVation Protocol) (Table 3)

Page 60: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

60

MULTIMEDIA CONCEPTS

Multimedia Protocols

MULTIMEDIA CONCEPTS

Network Protocol Advantage Disadvantage

TCP · Dominate protocol for data

transfer of data over the Internet

· Streaming through firewall

· Reliable

· Typically need large buffer to

handle data rate variation

· Loss recovery needs retransmission

causing further jitter or skew

· No support for multicast

UDP · Suitable for streaming

· Allows packet drops; if packets

arrive late or damaged,

streaming will continue

· No retransmission needed

· Many network firewalls block

UDP data

· Need error concealment for video

packet loss

· No support for congestion control

· Cannot be played using popular

stream players such as QuickTime

RTP/RTCP · Support real-time transmission

· Provide timing reconstruction,

loss detection, security and

content identification

· Allows retrieval of very

interesting network statistics

· No guarantee for QoS

· Header is larger than UDP

· More complicated that UDP

· No support for congestion control

RSVP · Reliable connection

· Receiver can obtain different

levels of service

· Complicated request mechanism

· Receivers may experience random

packet loss for small reservation

Table 3. Multimedia protocols.

Page 61: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

61

MULTIMEDIA CONCEPTS

References1) B.Krishnamurthy, J. Rexford. “Web Protocols and

Practice”, 2001

2) A.Silberschatz, P.Garvin, G.Gange. “ operating system concepts”, 2005

3) http://www.micc.unifi.it/delbimbo/documents/prog_prod_multimediale/2

4) H.Sun, A Vetro, J. Xin. “ An Overview of Scalable Video Streaming”. 2007

5) Z.Li and M.S.Drew. “Fundamentals of Multimedia”, Prentice-Hall 2004

6) R.Steinmetz, K.Nahrstedt, “Multimedia: Computing, Communications and Applications”, Prentice-Hall 1995

Page 62: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

62

MULTIMEDIA CONCEPTS

References7) T.Vaughan, “Multimedia: Making it Work, 7th Edition”,

McGraw-Hill College 2006

8) L.Guan, S.Y.Kung, J.Larsen “Multimedia image and video processing”

Page 63: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

63

MULTIMEDIA CONCEPTS

ReferencesImage Compression

JPEG: http://www.faqs.org/faqs/jpeg-faq/part1/

JPEG: http://en.wikipedia.org/wiki/JPEG

Video Compression

MPEG-4 pt. 2: http://en.wikipedia.org/wiki/MPEG-4_Part_2

MPEG-4 pt. 10/H.264/AVC: http://en.wikipedia.org/wiki/H.264/MPEG-4_AVC

File FormatsGeneral info: http://en.wikipedia.org/wiki/File_format

Containers: http://en.wikipedia.org/wiki/Container_format

MPEG-4 pt. 14: http://en.wikipedia.org/wiki/.m4a

Codec list: http://www.fourcc.org/

Audio

MP3: http://en.wikipedia.org/wiki/MP3

Page 64: MULTIMEDIA CONCEPTS 1 Multimedia Concepts. MULTIMEDIA CONCEPTS 2 Topics  Multimedia Basic Terms  Multimedia Compression/Decompression  Multimedia Protocols

64

MULTIMEDIA CONCEPTS

References

DCT Basis Function Image: http://en.wikipedia.org/wiki/Image:Dctjpeg.png, GNU licensed

JPEG Example Image: http://en.wikipedia.org/wiki/Image:Phalaenopsis_JPEG.jpg , by Ilmari Karonon at Wikipedia, Creative Commons Attribution-ShareAlike 2.5 License

MP3 File Image: http://en.wikipedia.org/wiki/Image:Mp3filestructure.jpg