concepts of multimedia processing and transmission it 481, lecture 2 dennis mccaughey, ph.d. 29...

Concepts of Multimedia Concepts of Multimedia Processing and TransmissionProcessing and Transmission

IT 481, Lecture 2Dennis McCaughey, Ph.D.

29 January, 2007

01/22/2007Dennis McCaughey, IT 481, Spring 20072

Course Web SiteCourse Web Site

http://teal.gmu.edu/~dgm/sp07/IT481-s07.htm

WebCt site will be set up this week


OverviewOverview

Need for an understanding and ability to apply top level signal/image processing concepts and algorithms– As a communication tool to aid in understanding

the course material– To allow the class to implement and observe the

results of the key processing/compression required for the efficient storage and communication of multimedia data

Not a course in DSP but a basic expertise is required

Exercises will be confined to home work and not on the mid-term or final


Required Signal Processing ConceptsRequired Signal Processing Concepts

Continuous-time Signal Processing– Linear Filtering and Convolution– Fourier Transform– Relationship between the Fourier Transform and

Convolution– Extensions to Image Processing

Discrete-Time Signal Processing– Shannon’s Sampling Theorem– Discrete Fourier Transform– Linear Filtering and Convolution– Relationship between the Fourier Transform and

Convolution– Extensions to Image Processing


Basic ToolsetsBasic Toolsets

Linear Algebra– Vector Spaces– Linear Operators– Matrix and Vector Algebra

Matlab– Programming tool for signal/image processing– Allows “hands-on” demonstration of signal/image

processing algorithms– Linear algebra intensive


Importance of Linear SystemsImportance of Linear Systems

A great deal of engineering situations are linear, at least within specified ranges

Exact solutions of the behavior of linear systems can be usually found by standard techniques

The techniques remain the same irrespective of whether the problem at hand is one on electrical circuits, mechanical vibration, heat conduction, motion of elastic beams or diffusion of liquids etc.

Except for a very few special cases, there are no exact methods for analyzing nonlinear systems


Matrix Algebra and Linear SystemsMatrix Algebra and Linear Systems

Every Linear operator on a finite dimensional vector space has a matrix representation– Matrix representation provides a useful tool for examining

the properties of a linear operator, even if the implementation does not explicitly employ a matrix

– In fact, a direct matrix implementation is often computationally inefficient

What is a vector space? What is a finite dimensional vector space? We will define both and develop applicability

through a simple electrical circuits example


Linear Vector SpaceLinear Vector Space

Definition– A vector space V is a set of elements called vectors with two operations,

called addition (designated by +) and multiplication by scalars (designated by juxtaposition), such that the following axioms or conditions are satisfied:

1 :M5

i.e. :additionscalar respect to with vedistributi is scalarsby tionsMultiplica :M4

i.e. :addition vector respect to with vedistributi is scalarsby tionsMultiplica :M3

s i.e. :eassociativ is scalarsby tionsMultiplica :M2

andr ofproduct e th called r vector unique a associated is thereV and Revery For :M1

i.e. :ecommutativ isAddition :A5

0-

such that inverse) additive the(called - vector unique a associated is thereVeach With :A4

0 with 0 vector,zero a exists There:A3

i.e. :eassociativ isAddition :A2

sum their called vector unique a exists therepair every with :A1

srαsr

rrr

rsr

r

Vα,β


ExamplesExamples

The sets of real and complex numbers The system of directed line segments in 3-

space The set of a real polynomials in a variable t The set of all n-tuples of real numbers


Linear System Example From CircuitsLinear System Example From Circuits

R1

V1 R2

R3

i1

i3

i2

Kirchhoff's Laws:

1. The algebraic sum of the voltages around a loop equal zero

2. The algebraic sum of the currents at a node equal zero


Derivation of the Relevant EquationsDerivation of the Relevant Equations

32221

222111

32212

221111

213

3223

23111

0

:gRearrangin

0

lawcurrent sKirchoff'

0

law voltagesKirchoff'

RRiRi

RiRRiV

RiRii

RiiRiV

iii

RiRi

RiRiV

R1

V1 R2

R3

i1

i3

i2

0

NotationMatrix In

1

2

1

322

221 V

i

i

RRR

RRR


Adding a Second Voltage SourceAdding a Second Voltage Source

322212

222111

322122

221111

213

32232

23111

:gRearrangin

lawcurrent sKirchoff'

law voltagesKirchoff'

RRiRiV

RiRRiV

RiRiiV

RiiRiV

iii

RiRiV

RiRiV

R1

V1 R2

R3

i1

i3

i2

V2

2

1

2

1

322

221

NotationMatrix In

V

V

i

i

RRR

RRR


SuperpositionSuperposition

2

1

322

2211

1

322

221

2

1

2

1

1

322

221

2

1

1

0

0 VRRR

RRRV

RRR

RRR

i

i

V

V

RRR

RRR

i

i

vi

vi

R

R

The output is the sum of the response to the sum the separate inputs

The superposition theorem states that the response in any element of a linear network containing two or more sources is the sum of the responses obtained by each source acting separately and with all other sources set equal to zero


Matrix AlgebraMatrix Algebra

BA

ABC

BAC

BA

BA

ofdimension column theequalmust ofdimension row The

:re Whe

:tionMultiplica

equal bemust matricesboth of dimensionscolumn theand row The

:Addition

: as Symbolized

1,,,

,

,

,,

,1,

,11,1

,1,

,11,1

l

kjkkiji

nmji

nmji

nljilmji

nlnll

l

lmlmm

l

bac

c

c

ba

bb

bb

aa

aa


Example (Multiplication)Example (Multiplication)

43

4223

81149711396112951119

8745773567255715

8341733163215311

8765

4321

119

75

31

C

BA


Matrix InversionMatrix Inversion

For the inverse to exist the matrix determinant must be non zero– The matrix must be square, i.e. the row and column dimensions

must be equal– Examples for some small matrices

3,31,22,12,33,21,11,32,23,1

1,33,22,12,31,23,13,32,21,1

1,22,12,21,1

1,1

det

matrix 3x3 a is If

det

matrix 2x2 a is If

det

matrix 1x1 a is If

aaaaaaaaa

aaaaaaaaa

aaaa

a

A

A

A

A

A

A


Matrix DeterminantMatrix Determinant

It is also possible to expand a determinant along a row or column using Laplace's formula, which is efficient for relatively small matrices. To do this along row i, say, we write

Where the Ci,j represent the matrix cofactors, i.e. Ci,j is

( − 1)i + j times the minor Mi,j, which is the determinant of the

matrix that results from A by removing the i-th row and the j-th column.


Matrix Classical AdjointMatrix Classical Adjoint

jiji

ji M

adj

adj

,,

'

1

1

of Cofactors ofmatrix theis

det

C

AC

CA

A

AA

It may (or may not) be helpful to attach names to the steps in the process. You can let M~ij be the (n-1) x (n-1) matrix minor, that is, the matrix that results from deleting row i and column j of A. Then Mij = det( M~ij). Let cof(A) be the cofactor matrix mentioned above. Then adj(A) = transpose of cof(A).

2,21,2

2,11,1

3,21,2

3.11,1

3,22,2

3.12,1

2,31,3

2,11,1

3,31,3

3,11,1

3,32,3

3,12,1

2,31,3

2,21,2

3,31,3

3,21,2

3,32,3

3,22,2

3,32,31,3

3,22,21,2

3,12,11,1

aa

aa

aa

aa

aa

aa

aa

aa

aa

aa

aa

aa

aa

aa

aa

aa

aa

aa

aaa

aaa

aaa

C

A


ExampleExample

6det 6

600

060

006

120

210

112

240

420

333

240

420

333

243

423

003

10

12

20

12

21

1120

12

10

12

12

1120

10

10

20

12

21

120

210

112

AIAA

A

A

adj

adj

C

Useful for 2x2 matrices


Matlab “Codelet”Matlab “Codelet”

% column delimiter =; row delimiter = ;A=[2,1,1;0,-1,2;0,2,-1]d = det(A)adjA = d*inv(A)


Return to Circuit ExampleReturn to Circuit Example

212 844 212

2

2

10

10

5

1

10

10

32

23

5

1

32

23

5

1

32

23

10

10

32

23

10-

1 ,2 ,1

32311212111

2

1

2

1

1

1

2

1

1

222

221

2

1

21

321

RIeRIIIeRIe

I

I

I

I

V

V

RRR

RRR

I

I

VV

RRR

R1

V1 R2

R3

V2

e1 e3

e2

I1 I2


Linear Systemh(t)

f(t) g(t)

Linear System RepresentationLinear System Representation


Linear System DefinitionLinear System Definition

tftbhtftahtg

tbftafthtg

SystemLinearafor

tfthtg

21

21

:

additionan or tion multiplica a

ynecessarilnot operation, general a denotes


Linear System Response to a Series of Linear System Response to a Series of Sampled data InputsSampled data Inputs

0 2 T N TT

(2 )Tf T ( )Tf T (0)Tf

( ) ( )Th t T f T

( ) (0)Th t f

( ) ( )Th t N T f N T

( 2 ) (2 )Th t T f T

t

( )Tf N T

1

0

( ) ( ) ( )N

Nn

g t T f n T h t n T


Linear System Input/OutputLinear System Input/Output

1

0

0

( ) ( ) ( )

( ) lim

( ) ( )

N

Nn

NN

T

t

g t T f n T h t n T

g t g t

f h t d

This is denoted as the convolution of f(t) and h(t)


Convolution Sum ExampleConvolution Sum Example

0

0 , 1 , 2

0 , 1 , 2

0 0 0

1 0 1 1 0

2 0 2 1 1 2 0

3 1 2 2 1

4 2 2

n

k

f f f f

h h h h

g n f k h n k

g f h

g f h f h

g f h f h f h

g f h f h

g f h

ng = nf + nh -1

f(k) = h(k) =0 for k >2


Integer Arithmetic ExampleInteger Arithmetic Example

Multiplication of 2 Integers is a form of discrete convolution

12345

111

*

12345

111

12345

12345

12345

1370295

x

y

z x y


Fourier Transform - Non-periodic SignalFourier Transform - Non-periodic Signal

Let x(t) be a non-periodic function of t

The Fourier Transform of x(t) is

The Inverse Fourier Transform is

dtetxfX ft2

dfefXtx tfj 2


Fourier Transform ExampleFourier Transform Example

22

0

)2(

0

)2(

0

22

2

2

2

1

2

1

)()(

)()(

fa

fja

fja

efja

dte

dteedtetuefX

tuetx

tfjatfja

tfjattfjat

at


Relationship Between the Fourier Relationship Between the Fourier Transform and ConvolutionTransform and Convolution

responses system andinput theof TransformsFourier theof

product theisoutput theof TransformFourier The

:Then

If

fXfHfY

dthxty


Very Important PropertiesVery Important Properties

dffXdttx2

-

2

-

:Theorem sParcival'

ffXdttx as 0Then If

22

-


Important Fourier Transform PropertiesImportant Fourier Transform Properties

a

fX

a

dueuxa

adtduatu

dteatxfX

a

fX

aatxF

fXtxF

Scaling

ta

uj

ftj

1

1

,let

1)(then

if

:

2

2

0

0

00

0

2

22

0

220

2

20

20

,let

)(

:

ftj

fujftj

ftjftjftj

ftj

ftj

efX

dteuxefX

dudttt

dteettxe

dtettxfX

efXttxF

Shifting


Combined Shifting and ScalingCombined Shifting and Scaling

0

0

0

0

2

2

0

00

2

20

20

2exp1

,let

2exp1

)(

:Scaling and Shifting

ftj

ftj

ftj

ftj

ftj

ea

fX

duua

fjux

aefX

adtduttau

adta

ttafjttax

ae

dtettaxfX

efXttxF


Discrete Time SystemsDiscrete Time Systems

Computer applications deal with discrete time or sampled data systems

Need a theory that connects sampled data and continuous time systems

This is provided by Shannon’s Sampling Theorem


Signal Sampling and RecoverySignal Sampling and Recovery

Sampler(Rate 1/T)

Low Pass Filter

s(t)s(n)

s(t)

Shannon’s sampling theorem states that the original signal s(t) can be recovered from its sampled version if the sampling rate, 1/T is greater than 2B where B is the one sided bandwidth of the signal


Sampling Theorem DemonstrationSampling Theorem Demonstration

-B B

S(f)

f

Ss(f)

0 1/(2T)1/T 1/(3T)-1/T-1/(2T)-1/(3T)f

Original Spectrum

Sampled Signal Spectrum

Low Pass Filter


Idealized Discrete-Time System Processing FlowIdealized Discrete-Time System Processing Flow

x(n) y(n)x(t) y(t)D/ADigital Filterh(n)

A/D

•Assume x(t) is band limited•Implicit in the D/A converter is an ideal LPF•What forms can the Digital Filter employ?

h(n) is the “impulse or characteristic” response of the filter.

It is given by the sequence h(n) ={y(0), y(1), y(2)…….} when the input sequence x(n) = {1, 0, 0,…….}


Digital Filter FormsDigital Filter Forms

Finite Impulse Response (FIR)

Infinite Impulse Response (IIR)

n

k

knxkNny0

m

j

n

k

jnyjD

knxkNny

1

0

All of the D's are zero for an FIR filter. The main advantage of IIR filters is that they can produce a steeper slope for a given number of coefficients. The main advantage of FIR filters is that the group delay is constant. This provides the capability of obtaining both a steep cutoff and perfect phase response. This is impossible to achieve with an analog filter.


Z-TransformZ-Transform

zYzknyZ

zuyzzuyz

ukkkikikiu

zkiyzzkiy

ziynyZY(z)

k

u

uk

ku

uk

i

kik

i

i

i

i

0

00

0

000 ,let

:property useful a

transform- Z theDefine


Z-Transform and Discrete ConvolutionZ-Transform and Discrete Convolution

zXzH

zixzkhzknxzkh

zknxzkh

zknxzkhzz

zzknxkhz

knxkhzzY

nnxnyknxkhny

k i

ik

k n

knk

on k n

knk

on k

knknn

knkn

on k

n

on k

n

n

k

0 00 0

0 0

0

0

0

0

0for 0 ,

Z-Transform of the output is the product if the Z-Transforms of the input and the filter response

X(z) Y(z)H(z)

x(n) y(n)Digital Filter


Calculating the Filter Impulse Response Calculating the Filter Impulse Response from its Z-Transformfrom its Z-Transform

ipzii

n

i i

in

ii

zHzpr

zp

rzHzH

H

zHzXzD

zNzY

zD

zNzH

zX

Otherwise

nnx

1

11

1

1

1

z ofExpansion Fraction Partial

1)(

0

0 ,1

:function Impulse

N

i

nii

niii

i

ii

i

prnh

nprnh

zp

rzH

pmZ-Transfro

1

1

,2,1,0 ,

1

unique are : Inverse


IIR-ExampleIIR-Example

nn

z

z

nh

zz

zz

zz

zzCheck

zz

zzr

zzzr

z

r

z

rzz

ZH

5.056.06

6.015.01

1

6.015.01

6.0155.016

6.01

5

5.01

6 :

65.06.0

6.0

6.05.01

1

6.015.01

16.01

56.05.0

5.0

5.06.01

1

6.015.01

15.01

6.015.01

03.01.11

1

11

11

11

11

6.0

11

11

2

5.011

11

12

11

21


Matlab “Codelet”Matlab “Codelet”

n =[0:20]y= 6*(0.6).^n-5*(0.5).^nbar(n,y,.01)


Impulse ResponseImpulse Response

0 5 10 15 20 250

0.2

0.4

0.6

0.8

1

1.2

1.4Impulse Response

n

h(n)


Determine k for Unity GainDetermine k for Unity Gain

nxnynyny

zXzz

zY

k

k

z

z

zz

kzz

zz

kZH

z

2.023.011.13.01.11

2.0

2.0

13.01.1113.01.1

1

3.01.11

21

12

2

21


Filter ResponseFilter Response

0 5 10 15 20 250

0.2

0.4

0.6

0.8

1Filter Response to a Unit Step

n

y(n

)


Flow ChartFlow Chart

Y(n) Unit Delay Unit DelayX(n)

Y(n-2)

+

a

b

Y(n-1)

+


Matrix RepresentationMatrix Representation

1 2 1

1 1 2

1 1

2 2

1

Let , 1

1 1

1

11 0 0

Shorthand:

nn n

y n y n y n y n

y n x n ay n by n

y n y na b x n

y n y n

y y x

AThe filter behavior can be determined from the characteristics of A


Observations on the Z-TransformObservations on the Z-Transform

Useful tool for implementing convolutions– We can develop a recursion

relationship for y(n) given a filter impulse (characteristic) response h(n) and an input sequence x(n).

– Recursions often provide very advantageous implementations

So far the development has been as an “algebraic” tool with no physical basis– What are the frequency

response characteristics of a digital filter described by H(z)?

This will require the development of the Discrete Fourier Transform (DFT)

N

ii

M

ii

N

ii

M

ii

M

iii

N

ii

N

ii

M

ii

inyDinxNny

Inverting

zDzYzNzXzY

zNzXzDzY

zXzD

zNzXzHzY

00

0

1

0

1

1

0

1

0

1

0

1

:

)(

)(1

),(1

Recursion


The Discrete Fourier TransformThe Discrete Fourier Transform

Let xp(t) be a periodic signal with property, xp(t) = xp(t+T0) where T0 is the signal period.

– Note: for the purposes if this discussion, any signal observed over a finite window (nT0 <t<(n+1)T0) can be considered periodic outside it.

nN

kjN

n

nTNT

kjN

ns

s

s

fnTjN

ns

p

fnTj

nsp

ssfnj

np

enxkXenTxkXNT

kX

NkNT

kfenTxfX

Nnnxnx

enTxfX

TwnTxnxenxfX

ss

s

s

21

0

21

0

21

0

2

0

2

0

12,1,0for at evaluating

Otherwise 0

12,1,0

rate sampling theis here that implicit


Relationship Between the DFT and the Z-TransformRelationship Between the DFT and the Z-Transform

22

2

1

0

0

cos21

1

1

11

1

1

11

1 ,

1

1

DFT the

aaaeae

aeaeH

aezH

azzH

Example

HenhzH

znhzH

jj

jj

jez

n

nj

ez

n

n

j

j


Frequency ResponseFrequency Response

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-80

-60

-40

-20

0

Normalized Frequency ( rad/sample)

Ph

as

e (

de

gre

es

)

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-10

0

10

20

Normalized Frequency ( rad/sample)

Ma

gn

itu

de

(d

B)


The Discrete Cosine TransformThe Discrete Cosine Transform

The general equation for a 1D (N data items) DCT is defined by the following equation:

and the corresponding inverse 1D DCT transform is simple F-1(u), i.e.:

where

The general equation for a 2D (N by M image) DCT is defined by the following equation:

and the corresponding inverse 2D DCT transform is simple F-1(u,v), i.e.:

where


DCT as It Applies to Images/VideoDCT as It Applies to Images/Video

The discrete cosine transform (DCT) helps separate the image into parts (or spectral sub-bands) of differing importance (with respect to the image's visual quality).

The DCT is similar to the discrete Fourier transform: it transforms a signal or image from the spatial domain to the frequency domain


SummarySummary

Shannon’s Sampling Theorem Fourier Transform Linear Systems Digital Filters Utility of Matrix Representations

concepts of multimedia processing and transmission it 481, lecture 2 dennis mccaughey, ph.d. 29...

Documents