Digital Representations

Digital Video Special EffectsDigital Video Special EffectsFall 2006

Analog-to-Digital (A-D) Conversion Sampling Quantization Coding

Sampling -- Analog to Discrete Analog signal to discrete-time signal

x(t) --> x[n] Sampling procedure

= dtnTtftxnTx )()(][ f(t) is the sampling function

Simple sampling x[n] = x(t=n), i.e., f(t)=(t)

= dtnTtftxnTx )()(][

Reconstruction: Discrete to Analog Can we reconstruct analog signal from its discrete

time samples? x[n] --> x(t) ? Generally not.

Nyquist (Shannon) sampling theorem for Nyquist (Shannon) sampling theorem for bandlimited signals If the simple sampling rate is at least twice bandwidth of

the analog signal, the analog signal can be perfectly reconstructed:

)(sinc)()( T nTtnTxtxn

=

=

Quantization -- Digitization

Discrete-time signal digital signal Quantization error Quantization level Quantization level

How many bits to represent one sample? Trade-off between error and bit rate (communication band

width) Nonlinear quantization

Pre-compression and de-compression ( law and A law) Vector quantization

Raw Data Rate Sampling frequency= f (Hz) Each sample represented by R bits Raw data rate (bit rate):

T = f x R (bits per second, or bps)T = f x R (bits per second, or bps)

Digital Audio Signals Frequency band of sound: human hearing

frequency range: 20Hz-20 KHz. Sampling rate > 40 KHz (Actual sampling

rate of CD-Audio = 44.1 KHz)rate of CD-Audio = 44.1 KHz) Bit rate for CD quality audio signal (44.1 KHz,

Quantization:16 bits, 2 channels):T = 44100 x 16 x 2 (bits per second, or bps)

CD quality stereo sound 10.6 MB / min

Examples

Sampling Rate (KHz)

Quantization level (bits)

Bit Rate (Kbps)

Telephone 8 8 64 AM Radio 16 16 256 FM Radio 22.05 16 352.8 FM Radio 22.05 16 352.8 CD Stereo 44.1 16 1411.2 DAT 48 16 1536 DVD (Stereo)

192 24 9216

Speech Signals Properties

Human ear: most sensitive to 600Hz-6000Hz Quasi-stationary for around 30 ms Characteristic maxima -- formantsSpeech analysis and synthesis Speech analysis and synthesis Speech components, e.g., vowels and consonants

MIDI A protocol that enables computer,

synthesizers, keyboards, and other musical device to communicate with each other.

Bit rate: 31.25Kbps Bit rate: 31.25Kbps A MIDI file stores the messages regarding

specific musical actions. Commands, instead of actual waveforms, are

saved. One minute of MIDI: 4KB storage.

Digital Image Representation Picture elements (pixels)

Sampling, quantization Higher dimensional image -- voxels Bi-level images (black/0 or white/1) Grayscale images

1 byte/pixel: 256 gray levels

Color images True color: RGB 24bits/pixel

Image size, e.g. VGA 640x480 Grayscale image: 307,200 bytes True color image: 921,600 bytes

Graphics Format Graphics primitives and attributes

2-D objects: lines, rectangles, circles, ellipses, text strings, etc.

Attributes: line style, line width, color, etc. Attributes: line style, line width, color, etc. High-level representation: structured, object-

based Low-level representation: bitmap

Computer Graphics Computer animation Computer Generated Images (CGI) Photo-realistic rendering

Video Signal Requirements Aspect ratio: TV 4/3; HDTV16/9 Luminance and chrominance Continuity of motion > 15 frames/s

TV 30 or 25 frames/s, movie 24 frames/s TV 30 or 25 frames/s, movie 24 frames/s Flicker. Marginal at least 50 refresh cycles/s

Movie: 2x24=48 TV: Half picture by line-interleaving

Scanning rate: at lease 25Hz, finish one frame in 1/25s

Color Representation in Video RGB, normalized R=G=B=1 -- white color YUV signal

Y=0.30R+0.59G+0.11B (Luminance) U=(B-Y) x 0.493, V=(R-Y) x 0.877 (Chrominance channels) Example: PAL, CD-I and DVI (Digital Video Interactive) video. Example: PAL, CD-I and DVI (Digital Video Interactive) video.

YIQ signal Y=0.30R+0.59G+0.11B (Luminance) I=0.60R-0.28G-0.32B, Q=0.21R-0.52G+0.31B Example: NTSC

Avoid cross talk between luminance and colors: S-Video video signals separate the luminance and chrominance information into two separate analog signals.

Subsampling in Video Different spatial sampling rates for different

chrominance channels Human beings are more sensitive to

luminance (using more samples) while less luminance (using more samples) while less sensitive to colors (using less samples).

Different resolution for different components Y:C1:C2 -- 4:2:2

Subsampling and upsampling techniques

Computer Video Format CGA (Color Graphics Adapter): 4 colors,

320x200x2bits = 16,000 bytes EGA: 640x350x4bits = 112,000 bytes

VGA: 640x480x8bits = 307,000 bytes VGA: 640x480x8bits = 307,000 bytes SVGA: 800x600 pixels XGA: 1024x768 pixels SXGA: 1280x1024 pixels

Video Quality VCR Quality -- SIF (MPEG1)

NTSC: 240x352; PAL: 288x352 per frame Videoconferencing quality

CIF (Common Interchange Format) -- H.261 288x352, subsampling 4:1:1(halving both 288x352, subsampling 4:1:1(halving both

direction) Q: what is the raw bit rate of CIF video

(30frames/s)? QCIF (Quarter CIF)

144x176, subsampling 4:1:1(halving both direction) Q: what is the raw bit rate of QCIF video (30frames/s)

Super-CIF: 576x704, subsampling 4:1:1(halving both direction)

The Need for Compression Take, for example, a video signal with

resolution 320x240 and 256 (8 bits) colors,30 frames per second

Raw bit rate = 320x240x8x30 Raw bit rate = 320x240x8x30 = 18,432,000 bits = 2,304,000 bytes = 2.3 MB

A 90 minute movie would take 2.3x60x90 MB = 12.44 GB

Without compression, data storage and transmission would pose serious problems!

Data Compression Data compression requires the identification

and extraction of source redundancy. In other words, data compression seeks to

reduce the number of bits used to store or reduce the number of bits used to store or transmit information.

Lossless Compression Lossless compression can recover the exact

original data after compression. It is used mainly for compressing database

records, spreadsheets or word processing records, spreadsheets or word processing files, where exact replication of the original is essential.

Examples: Run Length Encoding (RLE), Lempel Ziv Welch (LZW), Huffman Coding.

Lossy Compression Result in a certain loss of accuracy in

exchange for a substantial increase in compression.

More effective when used to compress More effective when used to compress images and voice where losses outside visual or aural perception can be tolerated.

Most lossy compression techniques can be adjusted to different quality levels.

Example: DCT(JPEG), MPEG

Compression Ratio Compression ratio

original data size ------------------------- : 1------------------------- : 1compressed data size