CYH/MMT/Rep/p.1

Multimedia informationrepresentation

2.1 Introduction

• All types of multimedia information are stored andprocessed within a computer in a digital form.

• They can be integrated together and transmitted over asingle all-digital communications network.

2.2 Digitization principles

• A signal whose amplitude varies continuously with timeis known as an analog signal.

• Techniques involved in analog-to-digital conversioninclude sampling and quantization.

• The range of frequencies of the sinusoidal componentsthat make up a signal is called the signal bandwidth.

• Any signal transmitted over a channel must have a signalbandwidth less than the channel bandwidth.

CYH/MMT/Rep/p.2

CYH/MMT/Rep/p.3

• Nyquist sampling theorem: The amplitude of a signalmust be sampled at a minimum rate that is equal to orgreater than twice the bandwidth of the signal.

CYH/MMT/Rep/p.4

• Quantization is the process that confines the amplitudeof a signal into a finite number of values.

• The difference between the actual signal amplitude andthe corresponding nominal amplitude is called thequantization error.

• The ratio of the peak amplitude of a signal to itsminimum amplitude is known as the dynamic range.

dBinVVD )/(log20 minmax10=

CYH/MMT/Rep/p.5

• Quantization interval : )2/( 1max

−= nVqwhere n is the number of bits used and Vmax is themaximum positive and negative signal amplitude.

CYH/MMT/Rep/p.6

CYH/MMT/Rep/p.7

2.3 Text

• There are 3 types of text that are used to produce pagesof documents:

• Unformatted text:(=plaintext)

• comprise strings of fixed-size characters from alimited character set such as ASCII code set.

• e.g. .txt file

• Formatted text: (=richtext)

• comprise of strings of characters of different styles,size, and shape with tables, graphics, and images.

• e.g. word .doc file

CYH/MMT/Rep/p.8

• Hypertext:

• integrated set of documents (each comprisingformatted text) to be created which have definedlinkages (hyperlinks) between them.

• e.g. .htm file

CYH/MMT/Rep/p.9

2.4 Images

• Images include computer-generated images (referred toas computer graphics) and digitized images ofdocuments and pictures.

• All images are displayed in the form of a two-dimensional matrix of individual picture elements calledpixels.

2.4.1 Graphics

• Graphics are composed of all kinds of visual objectssuch as lines, arcs, squares, circles and so on, as well asany form of hand-drawn objects.

• Each object has a number of associated attributes such ascolor, shape, size, shadow and so on.

• There are 2 forms of representation of a computergraphic:

• The representation of a graphic is analogous to thestructure of a program written in a high-levelprogramming language, which consists of a set ofcommands that are necessary to draw the differentobjects that make up the graphic.

• Another form of representation is the actual pixel-image of the graphic (bit-map format).

CYH/MMT/Rep/p.10

CYH/MMT/Rep/p.11

2.4.2 Digitized pictures

• A whole spectrum of colors can be produced by usingdifferent proportions of the 3 primary colors red, greenand blue.

• There are 2 mixing techniques: additive color mixing andsubtractive color mixing.

CYH/MMT/Rep/p.12

• Raster-scan principles:

• The frame refresh rate must be high enough to ensurethe eye is not aware the display is continuously beingrefreshed.

• A low refresh rate leads to what is called flicker.

CYH/MMT/Rep/p.13

• The number of bits per pixel is known as the pixel depthand determines the range of different colors that can beproduced.

• The aspect ratio is the ratio of the number of pixels perscanned line and the number of lines per frame.(common ratios: 4:3, 16:9)

• Example display resolutions and memory requirementsStandard Resolution Number

of colorsMemory requiredper frame (bytes)

VGA 640X480X8 256 307.2kBXGA 640X480X16 64k 614.4kB

1024X768X8 256 786.432kBSVGA 800X600X16 64k 960kB

1024X768X8 256 786.432kB1024X768X24 16M 2359.296kB

CYH/MMT/Rep/p.14

CYH/MMT/Rep/p.15

2.5 Audio

• There are 2 types of audio signal: speech and music-quality audio.

• Audio can be produced either naturally by means of amicrophone or electronically using some form ofsynthesizer.

• Output of all digitized audio signals the stream ofdigitized values must be converted back into its analogform as loudspeakers operate using an analog signal.

• Summary of characteristicsSpeech Music

Typical bandwidth 50Hz - 10kHz 15Hz - 20kHzSampling rate 20kHz 40kHzBits per sample 12 16No. of channels usually mono usually stereo

CYH/MMT/Rep/p.16

CYH/MMT/Rep/p.17

2.5.1 PCM

• The bandwidth of a speech circuit in a PSTN was limitedto 200Hz through to 3.4kHz. ⇒Sampling rate 8 kHz

• The digitization procedure is known as pulse codemodulation (PCM) and the international standardrelating to this is defined in ITU-T RecommendationG.711.

• North America & Japan: 7 bits per sample, use µ-lawcompression-expansion characteristics

• Europe: 8 bits per sample, use A-law compression-expansion characteristics

CYH/MMT/Rep/p.18

CYH/MMT/Rep/p.19

2.5.2 CD-quality audio

• Sampling rates 44.1kHz

• 16 bits per sample

• 2 channels

CYH/MMT/Rep/p.20

2.5.3 Synthesized audio

• Synthesized audio is often used in multimediaapplications since

(i) the amount of memory required can be 2 to 3 ordersof magnitude less than that required to store theequivalent digitized waveform version,

(ii) it is much easier to edit synthesized audio and to mixseveral passages together.

• The computer takes input commands from the keyboardand outputs these to the sound generators, which, in turn,produce the corresponding sound wave forms to drivethe speakers.

• A standardized set of messages have been defined forboth input and for output to the corresponding set ofsound generators.

• These are defined in a standard called Music InstrumentDigital Interface (MIDI).

CYH/MMT/Rep/p.21

2.6 Video

• The quality of the video required varies considerablyfrom one type of application to another.

2.6.1 Broadcast television:

• Interlaced scanning: Each frame is divided into 2 fields.One comprises the odd scan lines and one comprises theeven scan lines. They are transmitted one by one andintegrated together to form a complete frame in thetelevision receiver.

CYH/MMT/Rep/p.22

• The refresh rate can be halved with interlaced scanning.

• Progressive scanning: Frames are not split into fieldsand each of them is transmitted as a whole.

• Two types of system (based on line resolution):• NTSC: 525 lines per frame• PAL/CCIR/SECAM: 625 lines per frame

• For historical reason, a different set of color signals fromR, G and B were selected for color television broadcasts.

• 1 luminance and 2 chrominance components are used todescribe the color of each pixel.

• Luminance components describe the brightness of pixels.

• Chronminance components carry the color informationof pixels.

• PAL system:Y = 0.299R + 0.587 G + 0.114 BU = 0.493 (B-Y)V = 0.877 (R-Y)

• NTSC system:Y = 0.299R + 0.587 G + 0.114 BI = 0.74 (R-Y) – 0.27 (B-Y)Q = 0.48 (R-Y) + 0.41 (B-Y)

CYH/MMT/Rep/p.23

2.6.2 Digital video

• These are intended for use with standard televisionreceivers.

• The 3 component signals are usually digitized separatelyprior to their transmission.

• The resolution of the eyes is less sensitive for color thanit is for luminance.

• The International Telecommunications Union -Radiocommunications Branch (ITU-R) - formerly knownas Consultative Committee for InternationalRadiocommunications (CCIR) - defined a standard forthe digitization of video pictures known asRecommunication CCIR-601.

• Variants of this standard have been defined for use inother application domains such as digital televisionbroadcasting, digital telephony and videoconferencing.

• Collectively these are known as digitization formats.

CYH/MMT/Rep/p.24

• 4:2:2 format

• original digitization format used in RecommendationCCIR-601 for use in television studios.

• Line sampling rate: 13.5MHz for luminance and6.75MHz for chrominance signals.

• The number of bits per sample is 8 for all 3 signals.

CYH/MMT/Rep/p.25

• 4:2:0 format

• used in digital video broadcast applications.

• Interlaced scanning is used

• Absence of chrominance samples in alternative lines isthe origin of the term 4:2:0

CYH/MMT/Rep/p.26

• SIF (Source intermediate format)

• Give a picture quality comparable with that obtainedwith VCRs

• CIF: common intermediate format

• Used in videoconferencing applications

• 4:1:1 digitization format

• spatial resolution: Y=360x288, Cb=Cr=180x144

• temporal resolution 30Hz using progressive scanning

• Derivatives of CIF: 4CIF & 16CIF

CYH/MMT/Rep/p.27

• QCIF:

• for video telephony applications.

• Quarter of CIF

CYH/MMT/Rep/p.28

2.6.3 PC video

• PC video digitization formats:

Digitizationformat

System Spatial resolution Temporalresolution

525-line Y= 640x480Cb=Cr= 320x240

60Hz4:2:0

625-line Y= 768x576Cb=Cr= 384x288

50Hz

525-line Y= 320x240Cb=Cr= 160x240

30HzSIF

625-line Y= 384x288Cb=Cr= 192x144

25Hz

CIF Y= 384x288Cb=Cr= 192x144

30Hz

QCIF Y= 192x144Cb=Cr= 96x72

15/7.5Hz

• All PC monitors use progressive (non-interlaced)scanning rather than interlaced scanning.