signals and linear and time-invariant systems in discrete timegik2/teach/linear_systems-dt-2.pdf ·...

Signals and Linear and Time-Invariant Systems in Discrete Time

• Properties of signals and systems (di↵erence equations)

• Time-domain analysis

– ZIR, system characteristic values and modes

– ZSR, unit-pulse response and convolution

– stability, eigenresponse and transfer function

• Frequency-domain analysis

c�2016 George Kesidis

1

Time-domain analysis of discrete-time LTI systems

• Discrete-time signals

• Di↵erence equation single-input, single-output systems in discrete time

• The zero-input response (ZIR): characteristic values and modes

• The zero (initial) state response (ZSR): the unit-pulse response, convolution

• System stability

• The eigenresponse and (zero state) system transfer function


2

Discrete-time signal by sampling a continuous-time signal

• Consider a continuous-time signal x : R ! R sampled every T > 0 seconds

x(kT + t0) =: x[k] for k 2 Z,where

– t0 is the sampling time of the 0th sample, and

– T is assumed less than the Nyquist sampling period of x, and

– x[k] (with square brackets) is the kth sample itself.

• Here x[·] is a discrete-time signal defined on Z.


3

Example of sampling with t0 = 0 and positive signal x


4

Introduction to signals and systems in discrete time

• A discrete-time function (or signal) x : A ! B is one with countable (time) domain A.

• We will take the range B = R or B = C.

• Typically, we will herein take domain A = Z or Z��n for some (finite) integer n � 0.

• Some properties of signals are as in continuous time: e.g., periodic, causal, bounded, evenor odd.

• Similarly, some signal operations are as in continuous time: e.g., spatial shift/scale, super-position, time reflection, and (integer valued) time shift.


5

Time scaling: decimation and interpolation

• Time scaling can be implemented in continuous time prior to sampling at a fixed rate,

• or the sampling rate itself could be varied (again recall the Nyquist sampling rate).

• In discrete time, a signal x = {x[k] | k 2 Z} can be decimated (subsampled) by aninteger factor L 6= 0 to create the signal xL defined by

xL[k] = x[kL], 8k 2 Z,i.e., xL is defined only by every Lth sample of x.

• A discrete-time signal x can also be interpolated by an integer factor L > 0 to create xL

satisfying

xL[kL] = x[k], 8k 2 Z.

• For an interpolated signal xL, the values of xL[r] for r not a multiple of L (i.e., 8k 2 Zs.t. r 6= kL) can be set in di↵erent ways, e.g., between consecutive samples:

– (piecewise constant) hold: xL[r] = xL[L br/Lc] = x[br/Lc]

– linear interpolation:

xL[r] = x[br/Lc] +r � L br/Lc

L(x[br/Lc+1]� x[br/Lc])


6

Time scaling: decimation and interpolation - Questions

• Is the functional mapping x ! xL causal for linear interpolation?

• Is the hold causal?

• Exercise: Show that if a periodic, continuous-time signal x(t), with period T0, is period-ically sampled every T seconds, then the resulting discrete-time signal x[k] is periodic ifand only if T/T0 is rational.


7

Unit pulse �, unit step u, unit delay �, and convolution *

• Some important signals in discrete time are as those in continuous time, e.g., polynomials,exponentials, unit step.

• In discrete time, rather than the (unit) impulse, there is unit pulse (Kronecker delta):

�[k] =

⇢1 if k = 00 else

• Any discrete-time signal x can thus be written as

x[k] =1X

r=�1x[r]�[k � r] =

1X

r=�1x[k � r]�[r]

= (x ⇤ �)[k]

• or just x = x ⇤ �, i.e., the unit pulse � is the identity of discrete-time convolution.

• Define the operator � as unit delay (time-shift), i.e., 8 signals y and 8k, r 2 Z,(�ry)[k] := y[k � r].

• The discrete-time unit step u satisfies � = u��u, equivalently: 8k 2 Z,

�[k] = u[k]� u[k � 1] and u[k] =1X

r=0

(�r�)[k] =1X

r=0

�[k � r].

8

Unit pulse and unit step functions

• Exercise: For any signal causal f ({f [k], k � 0}), show that

8k � 0, (f ⇤ u)[k] =kX

r=0

f [r].


9

Exponential signals in discrete time

• Real-valued exponential (geometric) signals have the form x[k] = A�k, k 2 Z, whereA, � 2 R.

• Consider the scalar z = �ej⌦ 2 C with � > 0,⌦ 2 R, where again j :=p�1.

• Generally, complex-valued exponential signals have the (polar) form

x[k] = Aej�zk = A�kej(⌦k+�), k 2 Z,where w.l.o.g. we can take

�⇡ < ⌦,� ⇡ and real A > 0.

• Exercise: Show this complex-valued exponential is periodic if and only if ⌦/⇡ is rational.

• By the Euler-De Moivre identity,

x[k] = A�kej(⌦k+�) = A�k cos(⌦k + �) + jA�k sin(⌦k + �), k 2 Z.


10

Systems - single input, single output (SISO)

• In the figure, f is an input signal that is being transformed into an output signal, y, by thedepicted system (box).

• To emphasize this functional transformation, and clarify system properties, we will writethe output signal (i.e., system “response” to the input f) as

y = Sf,where, again, we are making a statement about functional equivalence:

8k 2 Z, y[k] = (Sf)[k].

• Again, Sf is not S “multiplied by” f , rather a functional transformation of f .


11

SISO systems (cont)

• The n signals {x1, x2, ..., xn} are the internal states of the system.

• The states can be taken as outputs of unit-delay operators, �, i.e.,

8k 2 Z, (�y)[k] = y[k � 1].

• Some properties of systems are as in continuous time: e.g., linear, time invariant, causal,memoryless, stable (with di↵erent conditions for stability as we shall see).


12

Di↵erence equation for an discrete time, LTI, SISO system

• For linear and time-invariant systems in discrete time, relate output y to input f via di↵er-ence equation in standard (time-advance operator) form:

8k � �n, y[k + n] + an�1y[k + n� 1] + ...+ a1y[k +1] + a0y[k]= bmf [k +m] + bm�1f [k +m� 1] + ...+ b1f [k +1] + b0f [k],

given

– scalars ak for 0 k n, with an := 1, and scalars bk for 0 k m,

– a0 6= 0 or b0 6= 0 (so that P,Q are of minimal degree), and

– initial conditions y[�n], y[�n+1], ..., y[�2], y[�1].

• Compact representation of the above di↵erence equation:

Q(��1)y = P (��1)f, where polynomials

Q(z) = zn +n�1X

k=0

akzk, P (z) =

mX

k=0

bkzk, and

��1 is the unit time-advance operator: (��1y)[k] ⌘ y[k+1], (��ry)[k] ⌘ y[k+r]


13

Discussion: conditions for causality and di↵erence equation in �

• Exercise: Show that the di↵erence equation Q(��1)y = P (��1)f is not causal ifdeg(P ) = m > n = deg(Q), i.e., the system is not proper.

• A not anti-causal di↵erence equation can be implemented simply using memory to store asliding window of prior values of the input f and delaying the output.

• Example: Decoding B (bidirectional) frames of MPEG video.


14

Numerical solution to di↵erence equation by recursive substitution

• Given the system Q(��1)y = P (��1)f , the input f (f [k] for k � 0), and initialconditions y[�n], ..., y[�1],

• one can recursively solve for y (y[k] for k � 0) by rewriting the system equation as

y[k + n] = �n�1X

r=0

ary[k + r] +mX

r=0

brf [k + r] for k � �n

) y[k] = �n�1X

r=0

ary[k + r � n] +mX

r=0

brf [k + r � n] for k � 0.

• For example, the di↵erence equation in standard form,

y[k +1] + 3y[k] = 7f [k +1] for k � �1,

can be rewritten as

y[k] = �3y[k � 1] + 7f [k] for k � 0.

• So, given f and y[�1] we can recursively compute

y[0] = �3y[�1] + 7f [0], y[1] = �3y[0] + 7f [1], y[2] = �3y[1] + 7f [2], etc.

• Exercise: If f = u and y[�1] = 7 then find y[3] for this example.

15

Approach to closed-form solution: ZIR and ZSR

• The total response y of P (��1)f = Q(��1)y to the given initial conditions and inputf is a sum of two parts:

– the ZSR, yZS, which solves

P (��1)f = Q(��1)yZS with zero i.c.’s, i.e., with 0 = y[�n] = ... = y[�1];

– the ZIR, yZI, which solves

0 = Q(��1)yZI with the given initial conditions.

• The total response y of the system to f and the given initial conditions is, by linearity,

y = yZI + yZS.

• We will determine the ZIR by finding the characteristic modes of the system.

• We will determine the ZSR by convolution of the input with the (zero state) unit-pulseresponse, the latter also in terms of characteristic modes.


16

Total response - example (cont)

• Consider again the di↵erence equation:

8k � �1, y[k +1] + 3y[k] = 7f [k +1],

• i.e., Q(z) = z +3 with degree n = 1, and P (z) = 7z with degree m = 1,

• Exercise: Show that the following system corresponds to this di↵erence equation.


17


• By recursive substitution, the total response is, 8k � �1:

y[k] = �3y[k � 1] + 7f [k]= �3(�3y[k � 2] + 7f [k � 1]) + 7f [k]= (�3)2y[k � 2]� 3 · 7f [k � 1] + 7f [k]= ...

= (�3)k+1y[�1] +kX

r=0

7(�3)k�rf [r]

=: (�3)k+1y[�1] +1X

r=0

h[k � r]f [r] =: (�3)k+1y[�1] + (h ⇤ f)[k],

• where h[k] := 7(�3)ku[k] is the (zero state) unit-pulse response,

• y[�1] is the given (n = 1) initial condition, and

• we have defined the discrete-time convolution operator withP�1

r=0(...) := 0.


18


• Exercise: Prove by induction this expression for y[k] for all k � �1.

• Exercise: Prove convolution is commutative: h ⇤ f = f ⇤ h.

• So, we can write the total response y = yZI + yZS starting from the time of oldest initialcondition:

8k � �1, yZI[k] = (�3)k+1y[�1]

8k � �1, yZS[k] = u[k]kX

r=0

7(�3)k�rf [r] = u[k](h ⇤ f)[k]

where yZS[k] = 0 when k < 0.

• Obviously, this example involves a linear, time-invariant and causal system as described bythe di↵erence equation above.


19

Total response - discussion

• Note that in CMPSC 360, we don’t restrict our attention to linear and time-invariantdi↵erence equations.

• We use recursive substitution to guess at the form of the solution and then verify our guessby an inductive proof.

• In this course, we will describe a systematic approach to solve any LTIC di↵erence equation,

• i.e., to solve for the output of a DT-LTIC system given the input and initial conditions.

• And again as in continuous time, we will see important insights about discrete-time signalsand LTIC systems through frequency-domain representations and analysis.


20

ZIR - the characteristic values

• Note that 8k, ��rzk = zk+r = zrzk, i.e., the r-units time-advance operator, ��r, isreplaced by the scalar zr for all r 2 Z.

• Our objective is to solve for the ZIR, i.e., solve

Q(��1)y ⌘ 0 given y[�n], y[�n+1], ..., y[�2], y[�1].

• Note that exponential (or “geometric”) functions, {zk | k 2 Z} for z 2 C, are eigenfunc-tions of time-shift operators of the form Q(��1) for a polynomial Q.

• That is, for any non-zero scalar z 2 C, if we substitute y[k] = zk 8k 2 Z we get:

8k 2 Z, (Q(��1)y)[k] = Q(��1)zk = Q(z)zk.

• So, to solve Q(z)zk ⌘ 0 for all time k � 0, when z 6= 0 we require

Q(z) = 0, the characteristic equation of the system.


21

ZIR - the characteristic values (cont)

• If z is a root of the characteristic polynomial Q of the system, then

– z would be a characteristic value of the system, and

– the signal {zk}k�0 is a characteristic mode of the system when z 6= 0, i.e.,Q(��1)zk = 0 8k � 0.

• Since Q has degree n, there are n roots of Q in C, each a system characteristic value.


22

ZIR - the characteristic values (cont)

• Let n0 n be the number of non-zero roots of Q, i.e., Q(z) = Q(z)/zn�n0is a

polynomial satisfying Q(0) 6= 0.

• Though there may be some repeated roots of the characteristic polynomial Q, there willalways be n0 di↵erent, linearly independent characteristic modes, µk, i.e.,

8k � �n,

n0X

r=1

crµr[k] = 0 , 8r, scalars cr = 0.

• When n = n0, by system linearity, we will be able to write

8k � �n, yZI[k] =nX

r=1

crµr[k],

for scalars cr 2 C that are found by considering the given initial conditions

y[k] =nX

r=1

crµr[k] for k 2 {�n, ...,�2,�1},

i.e., n equations in n unknowns (cr).

• The linear independence of the modes implies linear independence of these n equations incr, and so they have a unique solution.

23

ZIR - the case of di↵erent, non-zero, real characteristic values

• If there are n di↵erent non-zero roots of Q in R, z1, z2, ..., zn, then there are n character-istic modes: for r 2 {1,2, ...n},

8 time k, µr[k] = zkr .

• Therefore,

8k � �n, yZI[k] =nX

r=1

crzkr .

• The n unknown scalars ck 2 R can be solved using the n equations:

y[k] =nX

r=1

crzkr , for k 2 {�n,�n+1, ...,�2,�1}.


24

ZIR - the case of di↵erent, non-zero, real characteristic values

• Example: Consider the di↵erence equation:

8k � �3, 2y[k +3]� 10y[k +2] + 12y[k +1] = 3f [k +2],

with y[�2] = 1 and y[�1] = 3.

• That is, Q(z) = z2 � 5z + 6 = (z � 3)(z � 2) and n = 2, P (z) = (3/2)z andm = 1.

• So, the n = 2 characteristic values are z = 3,2 and the ZIR is

8k � �n = �2, yZI[k] = c13k + c22

k

• Using the initial conditions to find the scalars c1, c2:

1 = y[�2] = c13�2 + c22

�2 and3 = y[�1] = c13

�1 + c22�1.

• Exercise: Now solve for c1 and c2.

• Note: When a coe�cient c is worked out to be zero, it may not be exactly zero in practice,and the corresponding characteristic mode zk will increasingly contribute to ZIR yZI overtime if |z| > 1 (i.e., an “unstable” mode in discrete time).


25

ZIR - the case of not-real characteristic values

• The characteristic polynomial Q may have non-real roots, but such roots come in complex-conjugate pairs because Q’s coe�cients ak are all real.

• For example, if the characteristic polynomial is

Q(z) = (z � 1)(z2 � 2z � 2)

then the characteristic values (Q’s roots) are

�1, 1± jp3 again recalling j =

p�1.

• Because we have three di↵erent characteristic values 2 C, we can specify three correspond-ing characteristic modes,

(�1)k, (1 + jp3)k, (1� j

p3)k, 8k � 0,

and construct the ZIR as

8k � �n = �3, yZI[k] = c1(�1)k + c2(1 + jp3)k + c3(1� j

p3)k

= c1(�1)k + c22kekj⇡/3 + c32

ke�kj⇡/3

where

– c1 2 R and c2 = c3 2 C so that yZI is real-valued, and again,

– these scalars are determined by the n = 3 given (real) initial conditions: y[�3], y[�2], y[�1].

26

ZIR - not-real characteristic values with real characteristic modes

• By the Euler-De Moivre identity for the previous example,

8k � �3, yZI[k] = c1(�1)k + (c2 + c3)2k cos(k⇡/3) + j(c2 � c3)2

k sin(k⇡/3)

= c1(�1)k +2Re{c2}2k cos(k⇡/3)� 2Im{c2}2k sin(k⇡/3)

• Again, because all initial conditions are real and Q has real coe�cients, yZI is real valuedand so c3 = c2 ) c2 + c3, j(c2 � c3) 2 R.

• In general, consider two complex conjugate characteristic values v ± jq corresponding totwo complex-valued characteristic modes |z|ke±jk\z, where |z| =

pv2 + q2 and

\z = arctan(q/v).

• One can use Euler’s identity to show that the corresponding real-valued characteristic modesare

|z|k cos(k\z), |z|k sin(k\z)


27

ZIR - the case of repeated characteristic values

• Consider the case where at least one characteristic value is of order > 1, i.e., there arerepeated roots of the characteristic polynomial, Q.

• For example, Q(z) = (z+0.75)3(z�0.5) has a triple (twice repeated) root at �0.75and a single root at 0.5.

• Again, {(�0.75)k} is a characteristic mode because Q(��1)(�.75)k ⌘ 0 follows from

(��1 + .75)(�.75)k = ��1(�.75)k + .75(�.75)k

= (�.75)k+1 + .75(�.75)k

= 0.

• Similarly, (0.5)k is a characteristic mode since (��1 � 0.5)(0.5)k ⌘ 0.

• Also, {k(�.75)k} is a characteristic mode because Q(��1)k(�.75)k ⌘ 0 follows from

(��1 + .75)2k(�.75)k

= (��2 + 1.5��1 + (.75)2)k(�.75)k

= ��2k(�.75)k +1.5��1k(�.75)k + (.75)2k(�0.75)k

= (k +2)(�.75)k+2 + 1.5(k +1)(�.75)k+1 + (.75)2k(�0.75)k

= (�.75)k+2((k +2)� 2(k +1)+ k)= 0.

28

ZIR - the case of repeated characteristic values (cont)

• Similarly, {k2(�.75)k} is also a characteristic mode because(��1 + .75)3k2(�.75)k = 0.

• Note that without three such linearly independent characteristic modes

{(�.75)k, k(�.75)k, k2(�.75)k ; k � 0}for the twice-repeated (triple) characteristic value -.75, the initial conditions will create an“overspecified” set of n equations involving fewer than n “unknown” coe�cients (ck) ofthe linear combination of modes forming the ZIR.

• For this example,

yZI[k] = c0(�0.75)k + c1k(�0.75)k + c2k2(�0.75)k + c3(0.5)

k, k � �4.

• If the given initial conditions are, say,

y[�4] = 12, y[�3] = 6, y[�2] = �5, y[�1] = 10,

the four equations to solve for the four unknown coe�cients ck are:

yZI[�4] = (�.75)�4c0 + (�4)(�.75)�4c1 + (�4)2(�.75)�4c2 + (.5)�4c3 = 12yZI[�3] = (�.75)�3c0 + (�3)(�.75)�3c1 + (�3)2(�.75)�3c2 + (.5)�3c3 = 6yZI[�2] = (�.75)�2c0 + (�2)(�.75)�2c1 + (�2)2(�.75)�2c2 + (.5)�2c3 = �5yZI[�1] = (�.75)�1c0 + (�1)(�.75)�1c1 + (�1)2(�.75)�1c2 + (.5)�1c3 = 10

29

ZIR - general case of repeated, non-zero characteristic values

• In general, a set of r linearly independent modes corresponding to a non-zero characteristicvalue z 2 C repeated r � 1 times are

kr�1zk, kr�2zk, ..., kzk, zk, for k � 0.

• Also, if v ± jq are characteristic values repeated r � 1 times, with v, q 2 R and q 6= 0,we can use the 2k real-valued modes

ka|z|k cos(k\z), ka|z|k sin(k\z), for a 2 {0,1,2, ..., r � 1},

where |z| =p

v2 + q2 and \z = arctan(q/v).


30

ZIR - when some characteristic values are zero

• Again let n0 n be the number of non-zero roots of Q (characteristic values),

• i.e., r := n� n0 � 0 is the order (1+repetition) of the characteristic value 0, and

• r � 0 is the smallest index such that (the coe�cient of Q) ar 6= 0.

• So, there is a polynomial Q such that Q(z) = zrQ(z) and Q(0) 6= 0.

• Because the constant signal zero cannot be a characteristic mode, we add r = n � n0

time-advanced unit-pulses:

8k � �n, yZI[k] =rX

i=1

Ci�[k + i] + yN[k]

= Cr�[k + r] + Cr�1�[k + r � 1] + ...+ C1�[k +1] + yN[k]

where yN is a “natural response” (linear combination of n0 characteristic modes).

• The n initial conditions are then met by the r coe�cients Ci of the advanced unit pulsestogether with the n0 = n� r coe�cients of the characteristic modes in yN.


31

ZIR - when some characteristic values are zero - example

• Consider a fourth-order system with characteristic polynomialQ(z) = z2(z +1)2.

• Thus the poles are 0,�1 each repeated and the (non-zero) characteristic modes are(�1)k, k(�1)k.

• So, the ZIR is, for k � �4:

yZI[k] = C2�[k +2] + C1�[k +1] + c1(�1)k + c2k(�1)k

• That is, the ZIR has four unknown coe�cients C2, C1, c1, c2 to account for the four (given)initial conditions y[�4], y[�3], y[�2], y[�1].


32

Zero State Response - the unit-pulse response

• Recall the LTIC system

nX

r=0

ar��ry =: Q(��1)y = P (��1)f :=

mX

r=0

br��rf

with an ⌘ 1, a0 6= 0 or b0 6= 0, m n.

• We can express any input signal

f [k] =1X

r=0

f [r]�[k � r] 8k � 0, i.e., 8f , f = f ⇤ �.

• So the unit pulse � is the identity of the convolution operator in discrete time.

• Thus, by LTI, the ZSR yZS is the convolution of input f and ZSR h to unit pulse �,

yZS[k] =1X

r=0

f [r]h[k � r] = (f ⇤ h)[k], 8k � 0,

• h is called the unit-pulse response of the LTIC system, i.e.,

Q(��1)h = P (��1)� s.t. h[k] = 0 8k < 0.

33

Computing an LTIC system’s unit-pulse response, h

• For the LTIC system in standard form, if a0 6= 0 then

h = (b0/a0)� + yNu

where yN is a natural response of the system (linear combination of characteristic modes).

• Note that h[k] = 0 for all k < 0 owing to the unit step u.

• The n scalars of the natural response yN component of h are solved using

(Q(��1)h)[k] = (P (��1)�)[k] for k 2 {�n,�n+1, ...,�2,�1}


34

Unit-pulse response when zero is a characteristic value

• If r � 0 is the smallest index such that ar 6= 0 (0 is a char. mode of order r), then mayneed to add r delayed unit-pulse terms to h:

h =r�1X

i=0

Ai�i� + (b0/ar)�

r� + yNu,

where

– by definition of the standard form of the di↵erence equation, if r > 0, a0 = 0 sob0 6= 0, and

– r n since 0 6= an := 1.

• So if r = 0 (i.e., a0 6= 0), then A0 = b0/a0 as above, whereP�1

i=0(...) := 0.

• Exercise: Prove Ar = b0/ar for 0 r n.

• Thus, zero is a characteristic value of degree r � 0, and

• there are r characteristic modes that will all be zero.

• The additional unit-pulse terms introduce r degrees of freedom in the form of the coe�cientsA0, A1, ..., Ar�1 to accommodate the n = r + n0 initial conditions of the unit-pulseresponse: h[�n] = h[�n+1] = ... = h[�2] = h[�1] = 0.

35

Computing the ZSR - example 1

• Recall that the di↵erence equation y = 7f � 3�y corresponds to the above system; instandard form:

8k � �1, y[k +1] + 3y[k] = 7f [k +1].

with Q(z) = z +3, P (z) = 7z and n = 1 = m.

• Since the system characteristic value is�3 and b0 = 0, the (zero state) unit-pulse responsehas the form h[k] = c(�3)ku[k].

• The scalar c is solved by evaluating the above di↵erence equation at time k = �1:

(Q(��1)h)[�1] = (P (��1)�)[�1]i.e., h[0] + 3h[�1] = 7�[0]

) c+3 · 0 = 7 · 1, c = 7

36

Computing the ZSR - example 1 (cont)

• So, h[k] = 7(�3)ku[k].

• If the input is f [k] = 4(0.5)ku[k], the system ZSR is, for all k � 0,

yZS[k] =kX

r=0

h[r]f [k � r] =kX

r=0

7(�3)r4(0.5)k�r

= 28(0.5)kkX

r=0

(�6)r = 28(0.5)k(�6)k+1 � 1

�6� 1u[k]

= (24(�3)k +4(0.5)k)u[k].

• Note how the ZIR yZI has a term that is a characteristic mode (excited by the input f)and a term that is proportional to the input f (this forced response is an eigenresponse).

• Exercise: For the di↵erence equation, y[k + 1] + 3y[k] = 7f [k] 8k � �1: draw theblock diagram, show that h[k] = 21(�3)k�1u[k] + (7/3)�[k], and find the ZSR tothe above input f .

• Exercise: Read “sliding tape” method to compute convolution in Lathi, p. 595.


37

Computing the unit pulse response - example 2

• Find the ZSR of the following system to input f [k] = 2(�5)ku[k]:

• Exercise: show the di↵erence equation for this system (in direct canonical form) is:

8k � 0, y[k +2]� 5y[k +1] + 6y[k] = 1.5f [k +1]

• That is, Q(z) = z2 � 5z + 6 = (z � 3)(z � 2) and n = 2, P (z) = 1.5z andm = 1.

• So, the n = 2 characteristic values are z = 3,2 and b0 = 0 so the unit-pulse response

h[k] = (c13k + c22

k)u[k].


38

Computing the unit pulse response - example 2 (cont)

• To find the constants, evaluate the di↵erence equation at k = �1:

2h[1]� 10h[0] + 12h[�1] = 3�[0]) 2h[1]� 10h[0] = 3

) (2 · 3� 10 · 1)c1 + (2 · 2� 10 · 1)c2 = 3) �4c1 +�6c2 = 3

and at k = �2:

2h[0]� 10h[�1] + 12h[�2] = 3�[�1] ) 12h[0] = 0 ) h[0] = 0) c1 + c2 = 0.

• Thus, c2 = �1.5 = �c1 so that h[k] = (�1.5(3)k +1.5(2)k)u[k] and for k � 0

yZS[k] = (h ⇤ f)[k] =kX

r=0

h[r]f [k � r].

• Exercise: Write the ZSR as a sum of system modes 2k and 3k and a (force) term like theinput, here taken as f [k] = 4(�5)ku[k].


39

Convolution - other important properties

• Again, for a LTI system with impulse response h and input f , the ZSR is yZS = f ⇤ h,where

(f ⇤ h)[k] =1X

r=�1f [r]h[k � r]

• By simply changing the dummy variable of summation to r0 = h�r, can show convolutionis commutative: f ⇤ h = h ⇤ f .

• One can directly show that convolution f ⇤ h is a bi-linear mapping from pairs of signals(f, h) to signals (yZS), consistent with convolution’s commutative property and the (zerostate) system with impulse response h being LTI;

• that is, 8 signals f, g, h and scalars ↵,� 2 C,(↵f + �g) ⇤ h = ↵(f ⇤ h) + �(g ⇤ h)

• By changing order of summation (Fubini’s theorem), one can easily show that convolutionis associative, i.e., 8 signals f, g, h,

(f ⇤ g) ⇤ h = f ⇤ (g ⇤ h).


40

Convolution - other important properties (cont)

• We’ll use these properties when composing more complex systems from simpler ones.

• By just changing variables of integration, we can show how to exchange time-shift withconvolution, i.e., 8 signals f, h : Z ! C and times k 2 Z,

(�kf) ⇤ h = �k(f ⇤ h);

recall how convolution represents the ZSR of linear and time-invariant systems.

• By the ideal sampling property, recall that the identity signal for convolution is the unitpulse �, i.e., 8 signals f ,

f ⇤ � = � ⇤ f = f

• Exercise: Adapt the proofs of these properties in continuous time to this discrete-timecase.

• Exercise: In particular, show that if f and h are causal signals, then y = f ⇤ h is causal;i.e., if the unit-pulse response h of a system is a causal signal, then the system is causal.


41

System stability - ZIR - asymptotically stable

• Consider a SISO system with input f and output y.

• Recall that the ZIR yZI is a linear combination of the system’s characteristic modes, wherethe coe�cients depend on the initial conditions, possibly including some initial unit-pulseterms if zero is a characteristic value (system pole).

• A system is said to be asymptotically stable if for all initial conditions,

limk!1

yZI[k] = 0.

• So, a system is asymptotically stable if and only if all of its characteristic values havemagnitude less than 1.


42

System stability - ZIR - asymptotically stable: Example

• If the characteristic polynomial Q(z) = (z � 0.5)(z2 + 0.0625), then

• the system’s characteristic values (roots of Q) are 0.5,±0.25j each with magnitude lessthan one,

• and the ZIR is of the form,

yZI[k] =�c1(0.5)

k + c2(0.25j)k + c2(�0.25j)k

�u[k]

=�c1(0.5)

k +2Re{c2}(0.25)k cos(k⇡/2)� 2Im{c2}(0.25)k sin(k⇡/2)�u[k],

• recalling that jk = ejk⇡/2.

• So, yZI[k] ! 0 as k ! 1 for all c1, c2 (i.e., for all initial conditions), and

• hence is asymptotically stable.

43

System stability - bounded signals

• A signal y is said to be bounded if

9 M < 1 s.t. 8k 2 Z, |y[k]| M ;

otherwise y is said to be unbounded.

• For example, y[k] = 0.25(1+jp3

2)ku[k] is bounded (can use M = 0.25).

• Also, 3cos(5k) is bounded (can use M = 3).

• But both 2k cos(5k) and 3 · (�2)k are unbounded.


44

System stability - ZIR - marginally stable

• A system is said to be marginally stable if it is not asymptotically stable but yZI is always(for all initial conditions) bounded.

• A system is marginally stable if and only if

– it has no characteristic values with magnitude strictly greater than 1,

– it has at least one characteristic value with magnitude exactly 1, and

– all magnitude-1 characteristic values are not repeated.

• That is, a marginally stable system has

– some characteristic modes of the form cos(⌦k) or sin(⌦k),

– while the rest of the modes are all of the form kr|z|k cos(⌦k) or kr|z|k sin(⌦k),with |z| < 1 and integer degree r � 0.

– Exercise: Explain why we can take ⌦ 2 (�⇡,⇡] without loss of generality.

– Note: the dimension of frequency ⌦ is [⌦] = radians per unit time.


45

System stability - ZIR - marginally stable: Example

• The characteristic polynomial is Q(z) = z(z2+1)(z�0.25) gives characteristic values0,0.25,±j,

• then the system is marginally stable with modes (0.25)k, cos(k⇡/2), sin(k⇡/2),

• the last two of which are bounded but do not tend to zero as time k ! 1.


46

System stability - ZIR - unstable

• A system that is neither asymptotically nor marginally stable (i.e., a system with unboundedmodes) is said to be unstable.

• For example, the system with Q(z) = (z2 � 0.5)(z + 3) is unstable owing to thecharacteristic value �3 with unbounded mode (�3)k.

• For another example, if the characteristic polynomial is Q(z) = (z2+1)2(z�0.5) thenthe purely imaginary characteristic values ±j are repeated, and hence the two additionalmodes k sin(k⇡/2), k cos(k⇡/2) are unbounded, so this system is unstable.

• Similarly, if Q(z) = (z2 � 1)2(z � 0.5) then the characteristic values ±1 are repeatedand the modes k and k(�1)k are unbounded, so this system is unstable too.


47

ZIR stability - stability of poles


48

System stability - ZSR - BIBO stable

• A SISO system is said to be Bounded Input, Bounded Output (BIBO) stable if8 bounded input signals f , the ZSR yZS is bounded.

• A su�cient condition for BIBO stability is absolute summability of the unit-pulse response,

1X

k=0

|h[k]| < 1.

• To see why: If the input f is bounded (by Mf with 0 Mf < 1) then 8k � 0:

|yZS[k]| = |(f ⇤ h)[k]|

=

��

kX

r=0

f [k � r]h[r]

��

kX

r=0

|f [k � r]h[r]| (by the triangle inequality)

kX

r=0

Mf |h[r]|

Mf

1X

r=0

|h[r]| =: My < 1,

49

System stability - ZSR - BIBO stable

• The condition of absolute summability of the unit-pulse response,

1X

r=0

|h[r]| < 1,

is also necessary for, and hence equivalent to, BIBO stability.

• If any component characteristic mode of h is unbounded, then h will not to be absolutelysummable.

• Thus, if the system (ZIR) is asymptotically stable it will be BIBO stable; the converse isalso true.


50

ZSR - the transfer function, H

• Recall that for any polynomial Q and z 2 C (including s = jw, w 2 R),

Q(��1)zk = Q(z)zk, 8k � 0.

• So, if we guess that a “particular” solution of the system Q(��1)y = P (��1)f withinput f [k] = Azku[k] is of the form yp[k] = AH(z)zk = H(z)f [k], k � 0, then weget by substitution that 8k � 0, z 2 C,

(Q(��1)yp)[k] = (P (��1)f)[k] ) AH(z)Q(z)zk = AP (z)zk

) H(z) = P (z)/Q(z).

• The “rational polynomial” H = P/Q is known as the system’s transfer function and willfigure prominently in our study of frequency-domain analysis.

• So, the ZSR (forced response + characteristic modes) would be of the form:

yZS[k] = (AH(z)zk + linear combination of char. modes)u[k].

• Recall that for the example with Q(z) = z + 3 and P (z) = 7z, we computed theunit-pulse response h[k] = 7(�3)ku[k] and the ZSR to input f [k] = 4(0.5)ku[k] asyZS[k] = (24(�3)k +4(0.5)k)u[k].

• Here, note that H(0.5) = P (0.5)/Q(0.5) = 1, i.e., the forced response componentof yZS is H(0.5)f [k] = 1 · 4(0.5)ku[k] = 4(0.5)ku[k].

51

ZSR - unit-pulse response h, transfer function H, and eigenresponse

• yZS[k] = (H(z)Azk + linear combination of char. modes )u[k] is the ZSR to inputf [k] = Azku[k], where H(z) = P (z)/Q(z).

• The eigenresponse is a special case of the forced response for exponential inputs.

• If |z| = 1, i.e., z = ej⌦ for some⌦ 2 (�⇡,⇡] (w.l.o.g.), and the system is asymptoticallystable, then the ZSR tends to the steady-state eigenresponse of the system:

y[k] ! AH(ej⌦)ej⌦k as k ! 1.

• Since y = h ⇤ f , we get that as k ! 1 for a LTIC and asymptotically stable system,

yZS[k] =kX

r=0

h[r]Aej⌦(k�r)

= Aej⌦kkX

r=0

h[r]e�j⌦r ! Aej⌦kH(ej⌦),

)1X

r=0

h[r]e�j⌦r = H(ej⌦), 8⌦ 2 (�⇡,⇡].

52

ZSR - transfer function H and eigenresponse (cont)

• The LTIC system transfer function H is the z-transform of the system unit-pulse responseh:

H(z) =1X

k=0

h[k]z�k,

where z 2 C is in H’s “region of convergence”.

• Note that H(ej⌦) is periodic since H(ej⌦) ⌘ H(ej⌦+2⇡k) for any integer k.

• For the z-transform (and DTFS) we will use this notation for H, but for the DTFT we willwrite H(⌦) instead of H(ej⌦).


53

Frequency-domain methods for discrete-time signals

• Discrete-Time Fourier Series (DTFS) of periodic signals

• Discrete-Time Fourier Transform (DTFT)

• sampled data systems

* DFT & FFT

• z-transform for (complete) transient response

• eigenresponse

• canonical system realization of a di↵erence equation


54

Discrete-time Fourier series of periodic signals

• For all r,N 2 Z, note that the signal {exp(jr2⇡Nk) | k 2 Z} “repeats itself” everyN > 0

units of (discrete) time k, in particular

8r 2 Z, ejr2⇡

Nk|k=0 = 1 = ejr

2⇡

Nk|k=N

• Also the signals {exp(jr2⇡Nk) | k 2 Z} ⌘ {exp(jr02⇡

Nk) | k 2 Z} whenever r0 = r

mod N .

• Suppose N is the period of periodic signal x = {x[k] | k 2 Z} and⌦o = 2⇡/N be the fundamental frequency of x (recall [⌦o] = radians/unit-time).

• We can write x as a Discrete-Time Fourier Series (DTFS):

8k 2 Z, x[k] =N�1X

r=0

Drejr⌦ok.

where r indexes x’s N harmonics.

• Note that the DTFS can also be written for any discrete-time signal x : A ! R definedover any finite interval of time, e.g., A = {0,1,2, ..., N � 1} orA = {�N,�N +1, ...,�1} for integer N < 1.


55

Discrete-time Fourier series of periodic signals (cont)

• Consider the N signals ⇠r[k] := ejr⌦ok over any time-interval A of length N .

• Equivalently consider these N signals ⇠r as N -vectors in RN , i.e., the kth entry of vector⇠r is ⇠r[k].

• If these signals/vectors {⇠r}N�1r=0 are linearly independent, then they will form a basis span-

ning all other signals x : A ! R, equivalently all other vectors x 2 RN ,

• i.e., any such x can be written as a linear combination of the {⇠r}N�1r=0 giving the DTFS of

x:

xr =N�1X

r=0

Dr⇠r.

• If we show that these signals/vectors {⇠r}N�1r=0 are orthogonal then

– linear independence follows

– the rth coordinate Dr (DTFS coe�cients) is found by simply projecting x onto thevector ⇠r:

Dr = hx, ⇠ri/||⇠r||2.


56

DTFS - coe�cients (cont)

• Consider any period of x : Z ! R, say {0,1,2, ..., N � 1}.

• First note that for any v 2 Z that is not a multiple of N (so ejv⌦o = ejv(2⇡/N) 6= 1), thegeometric series

N�1X

k=0

ejv⌦ok =N�1X

k=0

⇣ejv2⇡/N

⌘k=

ejv(2⇡/N)·N � ejv(2⇡/N)·0

ejv2⇡/N � 1= 0.

• Thus, for any r 6= v 2 Z such that N 6 |(v � r), the inner product h⇠r, ⇠vi =

h{ejr(2⇡/N)k}, {ejv(2⇡/N)k}i :=N�1X

k=0

ejr(2⇡/N)kejv(2⇡/N)k =N�1X

k=0

ej(r�v)(2⇡/N)k = 0,

recalling that the inner product is conjugate-linear in the second argument so that< x, x >= ||x||2 when x is C-valued.

• So, these signals are orthogonal and the DTFS coe�cients of N -periodic x are

Dr =1

N

N�1X

k=0

x[k]e�jr⌦ok =hx, {ejr⌦ok}i||{ejr⌦ok}||2

, where ⌦o =2⇡

N.


57

DTFS - checking coe�cients

• Let’s now compute the inner product of ⇠v, for any v 2 {0,1, ..., N �1}, with the DTFSof N -periodic x:

hx, {ejv⌦ok}i =N�1X

k=0

x[k]e�jv⌦ok =N�1X

k=0

N�1X

r=0

Drejr⌦oke�jv⌦ok

=N�1X

r=0

Dr

N�1X

k=0

ej(r�v)⌦ok =N�1X

r=0

DrN�(r � v) = DvN

• Again, we have verified the DTFS coe�cients is

Dr =1

N

N�1X

k=0

x[k]e�jr⌦ok =hx, {ejr⌦ok}i||{ejr⌦ok}||2

, ⌦o =2⇡

N


58

DTFS - summary for N-periodic signal x


59

DTFS - example

Problem:Identify the DTFS coe�cients (if they exist) for

x[k] = 7 sin(5.7⇡k) + 2cos(3.2⇡k), k 2 Z.Solution:

• First note that the two components of x are periodic, so their sum is periodic.(Why is this so in discrete time?)

• Since sin and cos have period 2⇡, we can subtract integer multiples of 2⇡ to get

x[k] = 7 sin(1.7⇡k) + 2cos(1.2⇡k).

• 1.7⇡k is an integer multiple of 2⇡ when (integer) k = 20, and when k = 5 for 1.2⇡k,so least common multiple of these periods is k = 20.

• (Show that one can alternatively find the greatest common divisor of the component fre-quencies.)


60

DTFS - example (cont)

• Thus, the period of x is N = 20 and the fund. frequ. is ⌦o = 2⇡/N = 0.1⇡.

• By Euler’s identity and adding 2⇡k to the negative exponents,

x[k] =7

2jej1.7⇡k �

7

2je�j1.7⇡k + ej1.2⇡k + e�j1.2⇡k

= �3.5jej1.7⇡k +3.5jej0.3⇡k + ej1.2⇡k + ej0.8⇡k.

• So, the DTFS of x[k] =P19

r=0Drejr0.1⇡k with

D17 = �3.5j = 3.5e�j⇡/2, D3 = 3.5j = 3.5ej⇡/2, D12 = 1, and D8 = 1;

else Dr = 0 (incl. the fundamental r 2 {1,19} & DC r = 0 components).


61

DTFS - example and exercise

• Example: The DTFS of an even rectangle wave with period N = 6 and duty cycle 3:

x[k] =1X

`=�1

�6`(��1u��2u)[k] =1X

`=�1

(u[k +1� 6`]� u[k � 2� 6`])

is =5X

r=0

Drejr⌦ok,

where the fund. freq. ⌦o = 2⇡/6 and, 8r 2 Z,

Dr =1

6

2X

k=�3

x[k]e�jr⌦ok =1

6

1X

k=�1

1 · e�jr(2⇡/6)k =1

6(1+ 2cos(r(2⇡/6)k)).

• Exercise: Plot x[k] as a function of time k and plot its (periodic) spectrum:8r 2 {0,1,2, ...,5}, ` 2 Z,

X(r2⇡/6+ 2⇡`) = Dr.

62

DTFS - Parseval’s theorem

• The average power of the N -periodic discrete-time signal x is

Px =1

N

N�1X

k=0

|x[k]|2 =1

N

N�1X

k=0

x[k]x[k];

equivalently, the sum could be taken over any interval of length N 2 Z>0.

• Substituing the Fourier series of x separately for x[k] and x[k] (using a di↵erent summation-index variable for each substitution), leads to Parseval’s theorem

Px =N�1X

r=0

|Dr|2.

• Parseval’s theorem can be used to determine the amount of periodic signal x’s power residesin a given frequency band [⌦1,⌦2] ⇢ [0,2⇡] radians/unit-time:

1. determine the harmonics r⌦o of x that reside in this band, i.e., integers r 2 [⌦1/⌦o,⌦2/⌦o]where x’s fundamental frequency ⌦o = 2⇡/N .

2. sum just over these harmonics to get the answer,P

d⌦1/⌦oerb⌦2/⌦oc |Dr|2.


63

DTFS - Parseval’s theorem example

• Find the fraction of x’s average power in the frequency band [0.4⇡,1.1⇡] where

8k 2 Z, x[k] =1X

v=�1(3�[k � 4v]� 4�[k � 1� 4v])

• Solution: x has period N = 4 and average power

Px =1

N

N�1X

k=0

|x[k]|2 =1

4

3X

k=0

|x[k]|2 =1

4(32 + (�4)2 + 02 + 02) =

25

4

• x has fundamental frequency ⌦o = 2⇡/N = ⇡/2 radians/unit-time and discrete-timeFourier coe�cients

Dr =1

N

N�1X

k=0

x[k]e�j(r⌦o)k =1

4

⇣3� 4e�jr⇡/2

⌘, 0 r N � 1 = 3.

• The harmonics r of x that reside in [0.4⇡,1.1⇡] satisfy 0.4⇡ r⌦o = r⇡/2 1.1⇡,i.e., r 2 {1,2}.

• So, by Parseval’s theorem, the answer is (|D1|2 + |D2|2)/Px.


64

Periodic extensions

• Consider signal x : Z ! R having finite support {�M,�M + 1, ...,0, ...,M � 1,M}for 0 < M < 1; i.e., 8|k| > M, x[k] = 0.

• For N � M , define 2N -periodic x(N) such that

x(N)[k] =

⇢x[k] if |k| M0 if M < |k| N

• x(N) is a periodic extension of the finite-support signal x, where again x(N)’s period is 2Nand

limN!1

x(N) = x.


65

DTFS of periodic extension leading to DTFT

• For r 2 {�N +1,�N +2, ..., N � 1, N}, the DTFS of x(N) has coe�cients

D(N)r =

1

2N

NX

k=�N+1

x(N)[k]e�jr 2⇡

2Nk

=1

2N

MX

k=�M

x[k]e�jr 2⇡

2Nk

=1

2N

1X

k=�1

x[k]e�jr 2⇡

2Nk

=:1

2NX

✓r2⇡

2N

◆,

where the Discrete-Time Fourier Transform (DTFT) of (aperiodic) x : Z ! R isX : R ! C:

X(⌦) :=1X

k=�1

x[k]e�j⌦k =: (Fx)(⌦), ⌦ 2 R

• Note that Fourier integrals (spectra of discrete-time signals) are periodic, repeating them-selves every 2⇡ radians: 8⌦ 2 R, ` 2 Z,

X(⌦) = X(⌦+ `2⇡).

66

Inverse DTFT by Fourier Integral

• Thus, 8k 2 Z,

x[k] = limN!1

x(N)[k]

= limN!1

NX

r=�N+1

D(N)r ejr

2⇡

2Nk

= limN!1

NX

r=�N+1

X

✓r2⇡

2N

◆ejr

2⇡

2Nk 1

2N·2⇡

2⇡

=1

2⇡

Z ⇡

�⇡

X(⌦)ej⌦kd⌦

where the last equality is by Riemann integration with 2⇡/(2N) ! d⌦.

• Thus, we have derived the inverse DTFT by Fourier integral of X giving (aperiodic) x,

8k 2 Z, x[k] =1

2⇡

Z ⇡

�⇡

X(⌦)ej⌦kd⌦ =: (F�1X)[k].


67

DTFT Examples - exponential signal

• If x = � then obviously X ⌘ 1.

• The geometric signal x[k] = �ku[k] for scalar � s.t. |�| < 1 has DTFT

X(⌦) =1X

k=0

�ke�j⌦k =1X

k=0

(�e�j⌦)k

=1

1� �e�j⌦=

1

(1� �) cos(⌦) + j� sin(⌦)

• Note that

|X(⌦)| =1

(1� � cos(⌦))2 + �2 sin2(⌦)=

1

1+ �2 � 2� cos(⌦)

\X(⌦) = � arctan

✓� sin(⌦)

1� � cos(⌦)

◆


68

DTFT Examples - exponential signal (cont)

• The plots above are for � = 0.5.

• Note how X has period 2⇡.

• Exercise: What are the maximum and minimum values of |X|, i.e., how would this plotdepend on � > 0? Plot x and \X. How do these plots di↵er when �1 < � < 0?

• Exercise: Find the DTFT of anticausal signal x[k] = �ku[�k] for scalar � s.t. |�| > 1.

• Exercise: Find the DTFT of x[k] = �|k|, k 2 Z, for scalar � s.t. |�| < 1.


69

DTFT Examples - Square and Triangle Pulse

• For T 2 Z>0, the even rectangle pulse with support 2T +1,x = ��Tu��T+1u (i.e., x[k] = u[k + T ]� u[k � (T +1)]), has DTFT

X(⌦) =TX

k=�T

1e�j⌦k = 1+ 2TX

k=1

cos(k⌦), ⌦ 2 R.

• Exercise (even rectangle pulse in frequency domain):Show that for fixed ⌦0 s.t. 0 < ⌦0 < ⇡,

F�1{��⌦0 u��⌦0 u}[k] =⌦0

⇡sinc(⌦0k), k 2 Z.

• For T 2 Z>0, the odd triangle pulse with support 2T +1,x[k] ⌘ k(��Tu[k]��T+1u[k]) has DTFT

X(⌦) =TX

k=�T

ke�j⌦k = � 2jTX

k=1

k sin(k⌦), ⌦ 2 R.


70

DTFT Examples - exponential sinusoid

• For fixed time K0, clearly

F{�[k �K0]}(⌦) = ejK0⌦,

where here � is the unit pulse.

• Note that ejK0⌦ is a sinusoidal function of ⌦ with period 2⇡/K0 radians.

• Exercise: For fixed frequency ⌦0 radians/unit-time, show that

F{e�j⌦0k}(⌦) = 2⇡1X

v=�1�(⌦�⌦0 + 2⇡v),

where here � is the Dirac impulse (in the frequency domain ⌦ 2 R). Hint: work with F�1.

• So, the DTFT of a N -periodic signal with Fourier series

N�1X

r=0

Drejr2⇡

Nk F! 2⇡

1X

v=�1

N�1X

r=0

Dr�(⌦� r2⇡

N+2⇡v)


71

DTFT - Time shift and frequency shift properties

• If fixed K0 2 Z and X = F{x} then

F{�K0x}(⌦) =1X

k=�1

(�K0x)[k]e�j⌦k

=1X

k=�1

x[k �K0]e�j⌦k

=1X

k0=�1

x[k0]e�j(k0+K0)⌦

= e�jK0⌦X(⌦),

i.e., shift in time by K0 corresponds to product with sinusoid of period 2⇡/K0 (linearphase shift) in frequency domain.

• Exercise: Prove the dual property that if fixed ⌦0 2 R and X = F{x} then

F{x[k]ej⌦0k}(⌦) = X(⌦�⌦0),

i.e., modulation (multiplication by a sinusoid) in time domain results in frequency shift.


72

DTFT - convolution properties

• Let Xr = F{xr} for r 2 {1,2}.

F{x1 ⇤ x2}(⌦) :=1X

k=�1

(x1 ⇤ x2)[k]e�j⌦k

:=1X

k=�1

1X

l=�1

x1[l]x2[k � l]e�j(k�l)⌦e�jl⌦ i.e., ⇥ejl⌦e�jl⌦ = 1

=1X

k0=�1

1X

l=�1

x1[l]x2[k0]e�jk0⌦e�jl⌦ where k0 = k � l

=1X

l=�1

x1[l]e�jl⌦1X

k0=�1

x2[k0]e�jk0⌦ =: X1(⌦)X2(⌦)

• Exercise: Prove the dual property that

F{x1x2}(⌦) =1

2⇡(X1 ⇤X2)(⌦) :=

1

2⇡

Z

2⇡X1(v)X2(⌦� v)dv.

• Exercise: Use the convolution properties to prove the time and frequency shift properties.Hint: (�Ko�) ⇤ x = �Kox.

• Exercise: Show that DTFT is a linear operator.

73

DTFT - Parseval’s Theorem

• The energy of a signal DT x is

Ex :=1X

k=�1

|x[k]|2 =1X

k=�1

x[k]x[k] =1X

k=�1

(F�1X)[k](F�1X)[k]

=1X

k=�1

1

2⇡

Z

2⇡X(⌦0)ej⌦

0kd⌦0 1

2⇡

Z

2⇡X(⌦)ej⌦kd⌦

=1X

k=�1

1

2⇡

Z

2⇡X(⌦0)ej⌦

0kd⌦0 1

2⇡

Z

2⇡X(⌦)e�j⌦kd⌦

=1

2⇡

Z

2⇡X(⌦0)

Z

2⇡X(⌦)

1

2⇡

1X

k=�1

ej⌦0ke�j⌦k

!d⌦d⌦0

=1

2⇡

Z

2⇡X(⌦0)

Z

2⇡X(⌦)

1

2⇡

�2⇡�(⌦�⌦0)

�d⌦d⌦0

=1

2⇡

Z

2⇡X(⌦0)X(⌦0)d⌦0 =

1

2⇡

Z

2⇡|X(⌦0)|2d⌦0, recalling that for

fixed ⌦0: F�1{2⇡�(⌦�⌦0)}[k] = e�j⌦0k &R2⇡ X(⌦)�(⌦�⌦0)d⌦ = X(⌦0).


74

DTFT - Parseval’s Theorem - example

• The even rectangle pulse with support 2T +1, x = ��Tu��T+1u has energy

Ex =1X

k=�1

|x[k]|2 =TX

k=�T

12

= 2T +1.

• Recall its DTFT is X(⌦) =PT

k=�T e�j⌦k, so

1

2⇡

Z

2⇡|X(⌦)|2d⌦ =

1

2⇡

Z

2⇡X(⌦)X(⌦)d⌦ =

1

2⇡

Z

2⇡

TX

k=�T

e�j⌦kTX

k0=�T

e+j 0⌦kd⌦

=1

2⇡

Z

2⇡

TX

k=�T

1+X

k<k0

ej(k0�k)⌦

!d⌦

=TX

k=�T

1

2⇡

Z

2⇡1d⌦+

X

k<k0

1

2⇡

Z

2⇡ej(k

0�k)⌦d⌦ =TX

k=�T

1 + 0

= 2T +1.

75

DTFT - Parseval’s Theorem - exercises

• Exercise: Repeat this calculation using X(⌦) = 1+ 2PT

k=1 cos(⌦k).

• Exercise: Compute amount of energy of x in the frequency band [�⇡/6,⇡/6] radians/unit-time, i.e.,

1

2⇡

Z ⇡/6

�⇡/6|X(⌦)|2d⌦


76

Analysis of Stable DT LTI Systems in Steady-State

• Consider a SISO, DT-LTIC system described by the di↵erence equation

Q(��1)y = P (��1)f,

where f is the input and y is the ZSR (output).

• Recall that by the time-shift property,

Q(ej⌦)YZS(⌦) = P (ej⌦)F (⌦) ) YZS(⌦) = H(⌦)F (⌦).

• We now re-derive from first principles the eigenresponse by first recalling that the ZSRyZS = f ⇤ h where h is the unit-pulse response.

• Taking DTFTs, YZS = HF where H = Fh is the transfer function.

• Suppose the system is BIBO/asymptotically stable, i.e., the n roots of Q (system char.modes/poles) z all have modulus |z| < 1.

• The ZSR will consist of a forced response plus characteristic modes, where the latter will! 0 over time (our stability assumption) so that the forced response becomes the steady-state response.


77

Analysis of Stable DT LTI Systems in Steady-State (cont)

• The forced response to a persistent sinusoidal input

f [k] = Afej(⌦ok+�f)

will be of the form

yss[k] = Ayej(⌦ok+�y)

where (for k � 0),

Q(ej⌦o)yss[k] = (Q(��1)yss)[k] = (P (��1)f)[k] = P (ej⌦o)f [k].

) yss[k] =P (ej⌦0)

Q(ej⌦0)f [k]

• Also, the ZSR yZS = h ⇤ f , i.e., for all time k � 0:

yZS[k] =kX

v=0

h[v]Afej(⌦o(k�v)+�f) = f [k]

kX

v=0

h[v]e�j⌦ov

! f [k]H(⌦o) =: yss[k] as k ! 1.


78

Transfer Function and Eigenresponse in Discrete Time (cont)

• Equating the forced responses (steady-state response for a stable system), we again getthat the system transfer function is

H(⌦) =P (ej⌦)

Q(ej⌦)= (Fh)(⌦).

• Note that 8k 2 Z, H(⌦) = H(⌦+2⇡k).

• Also, we write H(⌦) not H(ej⌦) for the DTFT.

• So, the eigenresponse of a BIBO/asymptotically stable SISO, DT-LTIC system is the steady-state response to a sinusoid:

f [k] = Afej(⌦ok+�f) ! H(⌦o)f [k] = Ayej(⌦ok+�y) =: yss[k]

• The system magnitude response (gain) is |H(⌦)| = |P (ej⌦)|/|Q(ej⌦)|,i.e., Ay = Af |H(⌦0)|.

• The system phase response is \H(⌦) = \P (ej⌦)� \Q(ej⌦),i.e., �y = �f + \H(⌦0).


79

Eigenresponse - example

• Problem: For the system 2y[k] = 0.6y[k � 1]� 7f [k] find the steady-state response(if it exists) to f [k] = 4cos(5k)u[k].

• Solution: The di↵erence equation in standard form is(Q(��1)y)[k] = y[k + 1] � 0.3y[k] = �3.5f [k + 1] = (P (��1)f)[k], whereQ(z) = z � 0.3 and P (z) = �3.5z.

• The sole system characteristic value (root of Q, system pole) is 0.3, hence the system isBIBO/asymptotically stable.

• By Euler’s identity f [k] = (2ej5k +2ej(�5)k)u[k]

• By linearity, the eigenresponse is therefore

2H(5)ej5k +2H(�5)ej(�5)k,

where H(⌦) = P (ej⌦)/Q(ej⌦) = �3.5ej⌦/(ej⌦ � 0.3) = H(�⌦), so that

|H(⌦)| =3.5p

(cos(⌦)� .3)2 + sin2(⌦), \H(⌦) = ⇡ +⌦� arctan

✓sin(⌦)

cos(⌦)� .3

◆

• Exercise: Show that the eigenresponse is also simply |H(5)|4cos(5k + \H(5)).

80

2D Image Processing Example

• Apply 1-dimensional filtering to a 2-dimensional (2D) image by separately performing rowand column operations.

• For 256⇥ 256 pixel (2D) image,

f =

2

664

f [1,1] f [1,2] ... f [1,256]f [2,1] f [2,2] ... f [2,256]

... ... ... ...f [256,1] f [256,2] ... f [256,256]

3

775

• If f [k, i] represents the 8-bit (grey) intensity of the pixel in row k and column i (i.e., 8bits per pixel or bpp), then the “raw” image size will be 2563bits = 16Mb = 2MB.

• Each of f ’s rows of pixels can be processed by a system with unit-pulse response h toobtain a new row of pixels, and thus a new image y:

8k, f [k, ·] ! h ! y[k, ·]

• Alternatively, each of f ’s columns of pixels can be processed by a system with unit-pulseresponse h to obtain a new column of pixels, and thus a new image y:

8i, f [·, i] ! h ! y[·, i]


81

Image Processing: High-Pass and Low-Pass Filtering

• The system h may have a specific signal processing objective.

• The output pixels y[k, i] may be quantized to fewer bpp than those of the input, thusachieving image compression.

• The simple low-pass filter (L)

h[k] =1

2(�[k] + �[k � 1])

✓) y[k] =

1

2(f [k] + f [k � 1])

◆

can capture shading and texture in the image.

• The simple high-pass filter (H)

h[k] =1

2(�[k]� �[k � 1])

✓) y[k] =

1

2(f [k]� f [k � 1])

◆

can capture edges in the image.

• Typically more compression possible in higher-frequency bands (H).


82

Image Processing: Tandem Row and Column Filtering

f ! rowfiltering

! columnfiltering

! y

• Define yLH as the output of

f ! L ! H ! yLH

• Similarly define yLL, yHH and yHL.

• The y images are downsampled by a factor of four (two in each direction).

• The yLL image will have a lot of energy while yHH will have the least energy.

• This motivates non-uniform quantization (bit allotment per pixel) of these images.

• Together with a coding strategy for the quantized images (particularly for the regions of zeropixel-values), this is the basic approach used in JPEG leading to very good compression,e.g., from 8 bpp to 0.2-0.5 bpp.


83

Sampling Continuous-Time Signals (A/D)

• Consider continuous-time signal x with X = Fx.

• Recall that by sampling at period T with impulses in continuous time t 2 R, we get

xT(t) :=1X

k=�1

x(kT )�(t� kT )F!

1X

k=�1

x(kT )e�jkTw =: XT(w),

equivalently, XT(w) =1

T

1X

v=�1X

✓w � v

2⇡

T

◆.

• Now define the sampled process in discrete-time k 2 Z and its DTFT,

x[k] := x(kT )F! X(⌦) =

1X

k=�1

x[k]e�j⌦k.

• Substituting w = ⌦/T we get

X(⌦) = XT

✓⌦

T

◆=

1

T

1X

v=�1X

✓⌦� v2⇡

T

◆.

• Exercise: Read decimation (downsampling) and interpolation (upsampling) of Lathi Figs.8.17 & 10.9.

84

Sampling Continuous-Time Signals - example

• We are particularly interested in the case where

– the continuous-time signal x is band-limited, i.e., 9w0 > 0 s.t. X(w) = 0 for|w| > w0, and

– the sampling frequency is greater than Nyquist’s, i.e., 2⇡/T > 2w0 ) w0T < ⇡.

• Example: For fixed w0 > 0, consider the cts-time signal x(t) = Asinc(w0t) with FT

X(w) =A⇡

w0 (u(w + w0)� u(w � w0)).

• Sampling x at period T < ⇡/w0 we get the discrete-time signal x[k] = Asinc(w0kT ).

• Using inverse DTFT, recall that we can easily check that the DTFT of x is,

X(⌦) =1X

v=�1

A⇡

w0T(u(⌦+ w0T � 2⇡v)� u(⌦� w0T � 2⇡v))

=1X

v=�1

1

TX

✓⌦� 2⇡v

T

◆,

noting 8T > 0, u(⌦T± w0) = u(1

T(⌦± w0T )) = u(⌦± w0T ), ⌦ := ⌦� 2⇡v.

85

Sampling Continuous-Time Signals - example (cont), w0T < ⇡


86

Sampled Data Systems: A/D (analog-to-digital conversion)

• Suppose the signal f is sampled every Ts seconds, i.e., at sampling frequencyws := 2⇡/Ts.

• Recall Poisson’s identity (the Fourier series of the picket-fence function)

pTs(t) =

1X

k=�1

�(t� kTs) =1

Ts

1X

k=�1

ejkwst

• Let’s rederive the relationship between the spectrum of a sampled continuous-time signaland its discrete-time counterpart by first defining the discrete-time signal

8k 2 Z, f [k] = f(kTs).

• We want to relate the (continuous-time) Fourier transform of f to the (discrete-time)Fourier transform of f ,

F (⌦) :=1X

k=�1

f [k]e�j⌦k =1X

k=�1

f(kTs)e�j⌦k.


87

Sampled Data Systems: A/D (cont)

• To this end, recall

f(t)pTs(t) =

1

Ts

1X

k=�1

f(t)ejkwst F!1

Ts

1X

k=�1

F (w � kws), and also

f(t)pTs(t) =

1X

k=�1

f(kTs)�(t� kTs)F!

1X

k=�1

f(kTs)e�jwkTs = F (wTs).

• Equating these two expressions for F{fpTs} we get,

F (wTs) =1

Ts

1X

k=�1

F (w � kws).

• Substituting w = ⌦/Ts we get,

F (⌦) =1

Ts

1X

k=�1

F

✓⌦� k2⇡

Ts

◆.


88

Sampled Data Systems: D/A (digital to analog conversion)

• Now consider a discrete time signal y[k].

• We implement at D/A with a Ts-second hold, i.e., construct the continuous-time signal

y(t) :=1X

k=�1

y[k]rTs(t� kTs), where

rTs(t) := u(t)� u(t� Ts)

F! Tssinc(wTs/2)e�jwTs/2 =: RTs(w).

• Note that y is in the form of a convolution, so:

Y (w) =1X

k=�1

y[k]RTs(w)e�jwkTs

= RTs(w)Y (wTs)


89

Sampled Data Systems: equivalent cts-time transfer function

• Consider a digital system H(⌦) (or H(ej⌦) depending on notation),whose (ZS) output is y when the input is f , i.e., Y = HF .

• The equivalent continuous-time transformation of the tandem system

f ! A/D (Ts-sample)f! H(⌦)

y! D/A (Ts-hold) ! y

with input f has (ZS) output

Y (w) = RTs(w)Y (wTs) = RTs

(w)H(wTs)F (wTs)

= RTs(w)H(wTs)

1

Ts

1X

k=�1

F (w � kws).

• Exercise: Show that if f is band-limited by ws/2 (i.e., ws is greater than f ’s Nyquistfrequency) and the previous sampled data system is followed by an ideal low-pass filter withbandwidth ws/2, then the equivalent (continuous-time) transfer function is

H(w) = H(wTs)T�1s RTs

(w)(u(w + ws/2)� u(w � ws/2))


90

Sampled Data Systems: equivalent cts-time transfer function -discussion

• Note that the term in the transfer function H,

T�1s RTs

(w)(u(w + ws/2)� u(w � ws/2)) = sinc(⌦/2)(u(⌦+ ⇡)� u(⌦� ⇡))

is not a constant function of ⌦ = wTs.

• This distortion due to the hold function R can be reduced by putting in tandem with Han equalizer system with transfer function approximately

dR�1(⌦) :=1X

k=�1

u(⌦+ ⇡ � k2⇡)� u(⌦� ⇡ � k2⇡)

sinc((⌦� k2⇡)/2)

i.e.,

H(⌦) ! dR�1(⌦)


91

Sampled Data Systems: equalization of hold sinc(⌦/2) by dR�1(⌦)

• the hold (at left, R) distorts the signal by attenuating its higher frequency components

• the equalizer (at right, R�1) amplifies at the higher frequencies to cancel out this distortion


92

DFT and FFT - Reading Exercise on Computational Issues

• Read Lathi Sec. 5.2 and 5.3 re. continuous-time FS, FT

• Read Lathi Sec. 10.6 re. DTFS, DTFT


93

Transient analysis in discrete time by unilateral z-transform

• z-transform definition and region of convergence.

• Basic z-transform pairs and properties.

• Inverse z-transform of rational polynomials by Partial Fraction Expansion (PFE).

• Total transient response of SISO DT LTIC systems Q(��1)y = P (��1)f .

• The steady-state eigenresponse revisited.

• System composition and canonical realizations.


94

The unilateral z-transform & region of convergence

• The z-transform of a signal x = {x[k]}k�0 is

X(z) = (Zx)(z) =1X

k=0

x[k]z�k := limK!1

KX

k=0

x[k]z�k,

where z 2 C.

• If the signal x is bounded by an exponential (geometric), i.e.,

9 M, � 2 R>0 such that 8k 2 Z�0, |x[k]| M�k (i.e., �M�k x[k] M�k)

then the series X(z) converges in the region outside of a disk centered 0 2 C,

{z 2 C | |z| > �}.

• To see why bounded by an exponential su�ces, recall absolute convergence ) convergence:

8k � 0, |x[k]z�k| = |x[k]| · |z|�k M�k|z|�k = M(�/|z|)k

)1X

k=0

|x[k]z�k| M

1X

k=0

(�/|z|)k which converges if �/|z| < 1.


95

Basic z-transform pairs and RoCs

�[k]Z! 1, z 2 C

u[k]Z!

1X

k=0

z�k =1

1� z�1=

z

z � 1, |z| > 1

�ku[k]Z!

1X

k=0

�kz�k =1

1� �z�1=

z

z � �, |z| > |�|

{�k�1u[k � 1]})(z) Z!1X

k=1

�k�1z�k = z�11X

k0=0

�k0z�k0

= z�1 1

1� �z�1=

1

z � �, |z| > |�|

ej⌦oku[k]Z!

1X

k=0

ej⌦okz�k =1

1� ej⌦oz�1, |z| > 1 (� = ej⌦o)

k�ku[k]Z!

1X

k=0

k�kz�k = �d

d�

1X

k=0

�kz�k = �d

d�

1

1� �z�1=

�z�1

(1� �z�1)2,

|z| > |�|Exercise: Find Z{A cos(⌦ok + �)u[k]} and Z{A sin(⌦ok + �)u[k]}.


96

Basic z-transform properties: linearity

• The z-transform is a linear operator: for all scalars a1, a2 2 C and all signalsx1, x2 : Z�0 ! C with respective ROCs C1, C2 ⇢ C,

(Z{a1x1 + a2x2})(z) = a1(Zx1)(z) + a2(Zx2)(z), z 2 C1

\C2.

• Note that

{z | |z| > �1}\

{z | |z| > �2} = {z | |z| > max{�1, �2}} ⇢ C.


97

Basic z-transform properties: advance time shift

• Advance time shift (no change in RoC): Let X = Zx.

��1xZ!

1X

k=0

x[k +1]z�k = �zx[0] +1X

k=�1

x[k +1]z�k

= �zx[0] + z

1X

k=�1

x[k +1]z�(k+1)

= �zx[0] + z

1X

k0=0

x[k0]z�k0

= �zx[0] + zX(z)

• Exercise: For v 2 Z>0 show by induction that

(Z{��vx})(z) = �vX

k=1

zkx[v � k] + zvX(z)


98

Basic z-transform properties: delay time shift

• Delay time shift (no change in RoC): For v 2 Z>0,

�v(xu)Z!

1X

k=0

x[k � v]u[k � v]z�k

=1X

k=v

x[k � v]z�k =1X

k0=0

x[k0]z�k0�v

= z�vX(z).

• So in the “zero-state” (input-output) context (i.e., x[k]u[k] = 0 for k < 0), we identifymultiplying by z�1 in complex-frequency domain with the unit delay � in thetime domain.

• Delay v 2 Z>0 of non-causal x:

�vxZ!

1X

k=0

x[k � v]z�k =1X

k0=�v

x[k0]z�k0�v

=�1X

k0=�v

x[k0]z�k0�v + z�vX(z).

99

Basic z-transform properties: frequency shift & convolution

• Let X = Zx with RoC C(�) := {z 2 C | |z| > �}.

�kx[k]Z!

1X

k=0

�kx[k]z�k =1X

k=0

x[k](z/�)�k = X(z/�), z 2 C(�|�|).

i.e., ⇥�k in the time-domain is dilation by � in the z-domain.

• For signals x1, x2 : Z�0 ! C (x1[k], x2[k] = 0 for k < 0), with respective ROCsC1, C2 ⇢ C,

x1 ⇤ x2Z!

1X

k=0

(x1 ⇤ x2)[k]z�k =

1X

k=0

kX

v=0

x1[v]x2[k � v]z�(k�v)z�v

=1X

v=0

x1[v]z�v

1X

k=v

x2[k � v]z�(k�v)

=1X

v=0

x1[v]z�v

1X

k0=0

x2[k0]z�k0

= X1(z)X2(z), z 2 C1

\C2.


100

Basic z-transform properties: convolution, IVT & FVT

• So convolution in the time-domain is multiplication in the frequency domain.

• The converse is also true.

• Directly by definition of X = Zx, we get the initial value theorem

limz!1

X(z) = x[0].

• There is also a “final value” theorem for limk!1 x[k].


101

Total response of SISO LTIC systems

• We now study transient analysis of LTI di↵erence equations using z-transforms.

• Recall our system is defined given polynomials P,Q, input f and initial conditions:

– Q(��1)y = P (��1)f , where y is the (total) output and

– input f [k] = 0 for k < 0,

– degree of polynomial Q = n � m = degree of polynomial P (causal system),

– Q(z) = zn +Pn�1

v=0 avzv (i.e., an = 1) and P (z) =

Pmv=0 bvz

v,

– an 6= 0 or bn 6= 0 for poly’ls Q,P of minimum degree,

– n initial conditions y[�n], y[�n+1], ..., y[�2], y[�1].

• We can restate the di↵erence equation in terms of delays by delaying both sides by ntime-units (i.e., applying with �n), to get

�nQ(��1)y = �nP (��1)f

) Q(�)y :=nX

v=0

av�n�vy =

mX

v=0

bv�n�vf =: P (�)f

102

Total response of SISO LTIC systems (cont)

• So, taking the z-transform of the (delay) di↵erence equation, we get by the (delay) time-shift and linearity properties that

nX

v=0

av

�1X

k=�v

y[k]z�k�v + Q(z�1)Y (z) = P (z�1)F (z)

• So, solving for the total response Y we get

Y (z) =P (z�1)

Q(z�1)F (z)�

Pnv=0 av

P�1k=�v y[k]z

�k�v

Q(z�1)= YZS(z) + YZI(z)

• where the ZIR and ZSR in the complex-frequency (z) domain respectively are

YZI(z) := �Pn

v=0 avP�1

k=�v y[k]z�k�v

Q(z�1)= �

Pnv=0 av

P�1k=�v y[k]z

n�k�v

Q(z)

YZS(z) :=P (z�1)

Q(z�1)F (z) =

P (z)

Q(z)F (z) = H(z)F (z) (transfer function H).

• Regarding this total transient response, note how

– the z-transform’s unilateral aspect captures the impact of initial conditions (ZIR), and

– a greater range of inputs f than under DTFT through RoC ⇢ C (not just |z| = 1).

103

Total response of SISO LTIC systems - example

• Suppose i.c. y[�1] = �1, input f [k] = 2(�3)ku[k] and output y s.t.

8k � �1, 2y[k +1] + 2y[k] = 3f [k +1] + 2f [k].

• To find the total response y, we take the z-transform of the equivalent system: 8k � 0,

2y[k] + 2y[k � 1] = 3f [k] + 2f [k � 1]) 2Y (z) + 2(z�1Y (z) + y[�1]) = 3F (z) + 2z�1F (z).

• So by the delay property for non-causal signals (y), the total response

Y (z) =3+ 2z�1

2 + 2z�1F (z) +

�2y[�1]

2 + 2z�1

= H(z)F (z) +�y[�1]

1 + z�1

= YZS(z) + YZI(z)

with RoC for Y being the intersection of those of F and H.

104

Total response of SISO LTIC systems - example

• Here F (z) = Z{2(�3)k} = 2/(1 + 3z�1), y[�1] = �1, so

Y (z) =3+ 2z�1

2 + 2z�1·

2

1+ 3z�1+

1

1+ z�1

=3+ 2z�1

(1 + z�1)(1 + 3z�1)+

1

1+ z�1

=

✓3.5

1+ 3z�1+

�0.5

1+ z�1

◆+

1

1+ z�1

where for the last equality see PFE below (here in z�1).

• Understanding that the ZIR begins at k = �1 (initial condition) and the ZSRat time k = 0, we get:

8k � �1, y[k] = (3.5(�3)k � 0.5(�1)k)u[k] + (�1)k = yZS[k] + yZI[k],

where we minded the ambiguity Zx = Zxu.

• Exercise: Verify this solution using time-domain methods, i.e.,y = yZI + yZS = yZI + h ⇤ f , where h and yZI consist of char. modes.


105

Inverse z-transform of proper rational polynomials

• We now describe how to find Z�1X of causal signal X that is rational polynomial in z,i.e., X(z) = M(z)/N(z) where M(z) and N(z) are polynomials in z.

• If deg(M) = deg(N) + 1, we perform long division to write X = c + M/N wheredeg(N) = deg(M) and Z�1X = c� + Z�1{M/N}.

• If deg(M) = deg(N) and M(0) = 0 (so z�1M(z) is a polynomial), we can factor zfrom M to get

X(z) = zz�1M(z)

N(z).

• We will find Z�1X using PFE of the strictly proper rational polynomial z�1M(z)/N(z).

• Alternatively, we could apply PFE on strictly proper rational polynomials in z�1,z�KM(z)/(z�KN(z)) where K := deg(N), as in the previous example.


106

Partial Fraction Expansion (PFE) example in z (not z�1)

• For example, suppose

X(z) :=z(3z +2)

z2 � 0.64= z

(3z +2)

(z +0.8)(z � 0.8)

= z

✓0.25

z +0.8+

2.75

z � 0.8

◆= 0.25

z

z +0.8+ 2.75

z

z � 0.8

where PFE (below) gave the numerators (residues) 0.25 and 2.75.

• So,

(Z�1X)[k] = 0.25(�0.8)ku[k] + 2.75(0.8)ku[k]

• Note that the associated RoC of X is {z 2 C | |z| > 0.8}.


107

Partial Fraction Expansion (PFE) - preliminaries

• Let K = deg(N) = deg(M) so that we can factor

N(z) =KY

k=1

(z � pk),

where the pk are the roots of N (poles of M/N).

• We assume M and N have no common roots, i.e., no “pole-zero cancellation” issue toconsider, so that the pk are the poles of M/N .

• Again, we assume M(0) = 0 (0 is a zero of M/N) and so z�1M(z) is a polynomial ofdegree K � 1.

• Note that the RoC for M(z)/N(z) is {z 2 C | |z| > maxk |pk|}.


108

PFE - the case of no repeated poles

• Suppose there are no repeated poles for M/N , i.e., 8k 6= l, pk 6= pl.

• In this case, we can write the PFE of z�1M(z)/N(z) as

z�1M(z)

N(z)=

KX

l=1

cl

z � pl

)M(z)

N(z)= z

z�1M(z)

N(z)=

KX

l=1

clz

z � pl=

KX

l=1

cl1

1� plz�1

where the scalars (Heaviside coe�cients) cl 2 C are

cl =z�1M(z)Qk 6=l(z � pk)

��z=pl

= limz!pl

z�1M(z)

N(z)(z � pl) =

z�1M(z)

N(z)(z � pl)

��z=pl

.

• That is, to find the Heaviside coe�cient ck over the term z � pk in the PFE, we haveremoved (covered up) the term z � pk from the denominator N(z) and evaluated theremaining rational polynomial at z = pk.

• This approach, called the Heaviside cover-up method, works even when p is C-valued.

• Given the PFE of z�1M/N , (Z�1M/N)[k] =PK

l=1 clpkl u[k].

109

PFE - proof of Heaviside cover-up method

• To prove that the above formula for the Heaviside coe�cient cl is correct, note that theclaimed PFE of z�1M(z)/N(z) is

KX

l=1

cl

z � pl=

PKl=1 cl

Qk 6=l(z � pk)

N(z)

• Thus, the PFE equals z�1M(z)/N(z) if and only if the numerator polynomials are equal,i.e., i↵

z�1M(z) =KX

l=1

clY

k 6=l

(z � pk) =: M(z).

• Again, two polynomials are equal if their degrees, L, are equal and either:

– their coe�cients are the same, or

– they agree at L + 1 (or more) di↵erent points, e.g., two lines (L = 1) are equal ifthey agree at 2 (= L+1) points.

• Since z�1M(z) is a polynomial of degree< K, it su�ces to check that whether z�1M(z) =M(z) for all z = pk, k 2 {1,2, ...,K}, i.e., this would create K equations in < K un-knowns (the coe�cients of M).

110

PFE - proof of Heaviside cover-up method (cont)

• But note that any pole pr of z�1M(z)/N(z) is a root of all but the rth term in M , sothat

M(pr) = crY

k 6=r

(pr � pk)

=

0

@ z�1M(z)Qk0 6=r(z � pk0)

��z=pr

1

AY

k 6=r

(pr � pk)

=p�1r M(pr)Q

k0 6=r(pr � pk0)

Y

k 6=r

(pr � pk)

= p�1r M(pr).

• Q.E.D.


111

PFE - the case of no repeated poles - example

• To find the inverse z-transform of a proper rational polynomial X = M/N with M(0) =0, first factor its denominator N and factor z from M , e.g.,

X(z) =z3 + 5z2

z3 + 9z2 + 26z +24= z

z2 + 5z

(z +4)(z +3)(z +2), for |z| > 4.

• So, by PFE

X(z) = z

⇣c4

z +4+

c3

z +3+

c2

z +2

⌘= z

z�1M(z)

N(z))

z�1M(z) = 1z2 + 5z +0 = c4(z +3)(z +2)+ c3(z +4)(z +2)+ c2(z +4)(z +3) =: M(z).

• We can solve for the 3 constants ck by comparing the 3 coe�cients of quadratic M andM .

• The Heaviside cover-up method suggests we try z = �2,�3,�4 to solve for c2, c3, c4:

c4 =z2 + 5z

(z +3)(z +2)

��z=�4

= �2, c3 =z2 + 5z

(z +4)(z +2)

��z=�3

= 6, c2 =z2 + 5z

(z +4)(z +3)

��z=�2

= �3

• Thus, x[k] = (Z�1X)[k] = (�2(�4)k +6(�3)k � 3(�2)k)u[k].


112

PFE - the case of a non-repeated, complex-conjugate pair of poles

• Again, recall that for polynomials with all coe�cients 2 R, all complex poles will come incomplex-conjugate pairs, p1 = p2.

• The case of non-repeated poles p1, p2 = ↵±j� (↵,� 2 R, j :=p�1) that are complex-

conjugate pairs can be handled as above, leading to corresponding complex-conjugate Heav-iside coe�cients c1, c2, i.e., c1 = c2.

• In the PFE, we can alternatively combine the terms

c1

z � p1+

c2

z � p2=

r1z + r0

(z � ↵)2 + �2

where by equating the two numerator polynomials’ coe�cients,

r0 = � c1p2 � c2p1 = � 2Re{c1p2} 2 R and r1 = c1 + c2 = 2Re{c1} 2 R.

• Exercise: Show that

2|c| · |p|k cos(k\p + \c) Z!cz

z � p+

cz

z � p


113

PFE - non-repeated complex-conjugate pair of poles - example

• To find the inverse z-transform of

X(z) =3z2 + 2z

z3 + 5z2 + 10z +12,

first factor the denominator and divide the numerator by z to get

X(z) = z3z +2

(z2 + 2z +4)(z +3).

• Note that the poles of X are �3 and �1± jp3 (so X’s RoC is |z| > 3).

• So, we can expand X to

X(z) = zr1z + r0

z2 + 2z +4+ z

c3

z +3,

where by the Heaviside cover-up method,

c3 =3z +2

z2 + 2z +4

��z=�3

= � 1.


114

PFE - non-repeated complex-conjugate pair of poles - example (cont)

• To find r1, r0, we will compare coe�cients of the numerator polynomials of X (actuallyz�1X) and its PFE, i.e.,

0z2 + 3z +2 = (r1z + r0)(z +3)+ c3(z2 + 2z +4) (⇤)

= (r1 � 1)z2 + (3r1 + r0 � 2)z +3r0 � 4.

• Thus, by comparing coe�cients

0 = r1 � 1, 3 = 3r1 + r0 � 2, 2 = 3r0 � 4

we get

r0 = 2 and r1 = 1.

• Note how z = �3 in (⇤) gives c3 = �1 as Heaviside cover-up did.


115

PFE - non-repeated complex-conjugate pair of poles - example (cont)

• Thus by substituting, we get

X(z) = zz +2

z2 + 2z +4+ z

�1

z +3

• Exercise: Show that

x[k] = (Z�1X)[k]

=⇣p

4/3 2k cos(k2⇡/3� ⇡/6)� (�3)k⌘u[k].


116

PFE - the general case of repeated poles

• If a particular pole p of z�1M(z)/N(z) is of order r � 1, i.e.,N(z) has a factor (z�p)r,then the PFE of z�1M(z)/N(z) has the terms

c1

z � p+

c2

(z � p)2+ ...+

cr

(z � p)r=

rX

k=1

ck

(z � p)k=

z�1M(z)

N(z)��(z)

with ck 2 C 8k 2 {1,2, ..., r}, where�(z) represents the other PFE terms of z�1M(z)/N(z)(i.e., corresponding to poles 6= p).

• Note that equating z�1M(z)/N(z) to its PFE and multiplying by (z � p)r gives

z�1M(z)

N(z)(z � p)r = cr +

r�1X

k=1

ck(z � p)r�k +�(z)(z � p)r

)z�1M(z)

N(z)(z � p)r

��z=p

= cr,

i.e., Heaviside cover-up (of the entire term (z � p)r) works for cr.


117

PFE - the general case of repeated poles (cont)

• To find cr�1, we di↵erentiate the previous display to get

d

dz

z�1M(z)

N(z)(z � p)r =

r�1X

k=1

ck(r � k)(z � p)r�1�k +d

dz�(z)(z � p)r

= cr�1 +r�2X

k=1

ck(r � k)(z � p)r�1�k +d

dz�(z)(z � p)r

) cr�1 =

✓d

dz

z�1M(z)

N(z)(z � p)r

◆��z=p

• If we di↵erentiate the original display k 2 {0,1,2, ..., r � 1} times and then substitutez = p, we get (with 0! := 1)

✓dk

dzkz�1M(z)

N(z)(z � p)r

◆��z=p

= k!cr�k

) cr�k =1

k!

✓dk

dzkz�1M(z)

N(z)(z � p)r

◆��z=p

.


118

PFE - the general case of repeated poles - example

• To find the inverse z-transform of

X(z) =z(3z +2)

(z +1)(z +2)3,

write the PFE of X as

X(z) = z

✓c1

z +1+

c2,1

z +2+

c2,2

(z +2)2+

c2,3

(z +2)3

◆,

so clearly the RoC of causal X is |z| > 2.

• By Heaviside cover-up

c1 =3z +2

(z +2)3

��z=�1

= �1 and c2,3 =3z +2

z +1

��z=�2

= 4.


119

PFE - the general case of repeated poles - example (cont)

• Also,

c2,2 =1

1!

✓d

dz

3z +2

z +1

◆��z=�2

=1

1!

1

(z +1)2

��z=�2

= 1

c2,1 =1

2!

✓d2

dz23z +2

z +1

◆��z=�2

=1

2!

�2

(z +1)3

��z=�2

= 1

• Thus,

X(z) = z�1

z +1+ z

1

z +2+ z

1

(z +2)2+ z

4

(z +2)38|z| > 2

) x[k] = (Z�1X)[k] =

✓�(�1)k + (�2)k + k(�2)k�1 + 4

k(k � 1)

2(�2)k�2

◆u[k].

• Exercise: Show by induction and integration by parts that: 8m 2 Z>0,⇣ km

⌘�k�mu[k]

Z!z

(z � �)m

• Exercise: Find the ZSR y to input f [k] = 2jku[k] = 2ejk⇡/2u[k] of the marginallystable system H(z) = 4/(z2 + 1).

120

PFE of M/N when M(0) 6= 0

• If M(0) 6= 0 (so cannot factor z from M(z)), then just perform long division ifdeg(M) � deg(N) to get a strictly proper rational polynomial, factor N to find thepoles, and find the PFE as before.

• When taking inverse z-transform, recall the z-transform pair

�k�1u[k � 1]Z!

1

z � �, |z| > |�|


121

PFE without factoring z from the numerator first

• For example, to find the ZSR to f [k] = 2(�1)ku[k] of the system

y[k +1]� 4y[k] = 5f [k],

take the z-transform to get

YZS(z) = H(z)F (z) =5

z � 4F (z) =

10z

(z � 4)(z +1)

=8

z � 4+

2

z +1(by PFE)

) yZS[k] = 8(4)k�1u[k � 1] + 2(�1)k�1u[k � 1]

• Note that the unit-pulse response is

h[k] = Z�1(H)[k] = 5(4)k�1u[k � 1],

and that, by delaying the di↵erence equation to get

y[k] = �4y[k � 1] + 5f [k � 1],

we see that (the ZSR) yZS[0] = 0.

• Exercise: First factor z from the numerator of YZS before PFE to show that

yZS[k] = 2(4)ku[k]� 2(�1)ku[k].

Is this result di↵erent? Check for k = 0 and k > 0.

122

PFE and eigenresponse for asymptotically stable systems

• The total response of a SISO LTI system to input f is of the form

Y (z) = H(z)F (z) +P1(z)

Q(z)=

P (z)

Q(z)F (z) +

P1(z)

Q(z)= YZS(z) + YZI(z).

where P1 depends on the initial conditions and the RoC is the intersection of that of inputF = Zf and the system characteristic modes.

• Unlike for DTFT notation, here write H(z) = P (z)/Q(z) = (Zh)(z).

• Suppose the system is BIBO/asymptotically stable and the input is a sinusoid atfrequency ⌦o radians per unit time, f [k] = Aej(⌦ok+�)u[k] = Aej�(ej⌦o)ku[k] withA > 0) F (z) = Aej�z/(z � ej⌦o) with RoC |z| > 1.

• Since ej⌦o cannot be a system pole (owing to asymptotic stability all poles have modulusstrictly less than one), we can use Heaviside cover-up on

YZS(z) = H(z)F (z) = zP (z)

Q(z)(z � ej⌦o)Aej� to get

YZS(z) = zH(ej⌦o)

z � ej⌦o

Aej� + char. modes = H(ej⌦o)F (z) + char. modes.


123

PFE and eigenresponse for asymptotically stable systems (cont)

• Thus, the total response of an asymptotically stable system to a sinusoidal input f atfrequency ⌦o is

y[k] = H(ej⌦o)f [k] + linear combination of characteristic modes.

• So by asymptotic stability, the steady-state response is the eigenresponse, i.e., as k ! 1,

y[k] ! H(ej⌦o)f [k] = H(ej⌦o)Aej(⌦ok+�) = |H(ej⌦o)|Aej(⌦ok+�+\H(ej⌦o)),

where again,

• H = P/Q is the system’s transfer function,

• |H(ej⌦o)| is the system’s magnitude response at frequency ⌦o radians/unit-time, and

• \H(ej⌦o) is the system’s phase response at ⌦o.


124

PFE and eigenresponse for asymptotically stable systems (cont)

• Laplace’s approximation: the rate at which the total response converges to the eigenre-sponse response is according to the characteristic value of largest modulus,

– which will be < 1 owing to the stability assumption,

– i.e., giving the modes(s) that ! 0 slowest.

• In continuous-time systems, it’s the characteristic value of largest real part, which will benegative owing to stability assumption.


125

Canonical (ZS) system-realizations - direct form

• Consider the proper (m n) transfer function

H(z) =P (z)

Q(z)=

bmzm + bm�1zm�1 + ...+ b1z + b0

zn + an�1zn�1 + ...+ a1z + a0=

Y (z)

F (z)

• The direct-form realization employs the interior system state X := F/Q,i.e., F = QX and Y = PX where the former implies (with an = 1),

F (z) =nX

r=0

arzrX(z) ) znX(z) = F (z)�

n�1X

r=0

arzrX(z).

• For n = 2, there are two “system states” (outputs of unit delays), X and zX (respectively,x[k] and (��1x)[k] = x[k +1]):

126

Canonical system-realizations - direct form (cont)

• Now adding Y = PX, we finally get the direct-form canonical system-realization of H:

• Again, state variables taken as outputs of unit delays, here: x, ��1x, ... ��(n�1)x.

• If bn = b2 6= 0, there is direct coupling of input and output, H is proper but not strictlyso, h = Z�1H has a unit-pulse component b2�.


127

Canonical system realizations - direct form (cont)

• Note that this n = 2 example above can be used to implement a pair of complex-conjugatepoles as part of a larger PFE-based implementation (with otherwise di↵erent states); e.g., forn = 2, H(z) = P (z)/Q(z) where

Q(z) = z2 + a1z + a0 = (z � ↵)2 + �2

for ↵,� 2 R, so the poles are ↵± j�.


128

Canonical system realizations by PFE

• In the general case of a proper transfer function, we can use partial-fraction expansion

– grouping the terms corresponding to a complex-conjugate pair of poles, i.e., a second-order denominator, and

– using a direct-form realization for these terms.

• Besides the PFE-based and direct-form realizations, there are other (zero-state) systemrealizations, e.g., “observer” canonical.

• For proper rational-polynomial transfer functionsH = P/Q, all of these realizations involven (= degree of Q) unit delays, the output of each being a di↵erent interior state variableof the system.


129

Canonical system realizations by PFE - example

H(z) =.3z2 � .1

z2 � 0.1z � .3= .3+

.3z � .01

(z � 0.6)(z +0.5)= .3+

.17/1.1

z � 0.6+

.16/1.1

z +0.5

Note that one cannot factor z from the numerator of H.

Exercise: Find a realization for this transfer function H by

1. splitting/forking the input signal F ,

2. using the direct canonical form for each of these 3 terms of H found by long division andPFE, and

3. summing three resulting output signals to get the (ZS) output Y = HF .


130

Digital Proportional-Integral (PI) system

• Consider a continuous-time signal x sampled every T seconds,

8k 2 Z+, x[k] = x(kT ),

and its integral y(t) =R t

0 x(⌧)d⌧ .

• The sampled integral can be approximated, y(kT ) ⇡ y[k], by the trapezoid rule,

y[k] = y[k � 1] +x[k � 1] + x[k]

2T.

• In the complex-frequency domain,

Y (z) = Y (z)z�1 +X(z)z�1 +X(z)

2T

)Y (z)

X(z)=

T

2·1+ z�1

1� z�1.


131

Digital PI system (cont)

• So, a digital PI transfer function would be of the form,

G(z) = Kp

+K

i

T

2·1+ z�1

1� z�1.

for constants Kp

,Ki

.

• In practice, PID or PI systems G are commonly used to control a plant H, where G maybe in series with H or in the feedback branch.

• Exercises:

– Draw the direct-form canonical realization for G.

– Draw the block diagram for the closed-loop system with negative feedback:Y = HX and X = F �GY where H is the (open-loop) system.

– Find the closed-loop transfer function Y/F and recall the pole placement problem tostabilize H.


132

Recursive Least Squares (RLS) Filter - Introduction

• Consider a LTI system with input f and output y,

y[k] =KX

r=0

h[k � r]f [r] + v[k], k 2 Z,

where v is an additive noise process and K is the maximum system order.

• The system (unit-pulse response) h is not known.

• Past values of the output y are observed (known).

• At time k, the objective is to forecast the next output y[k + 1], based on the assumedknown/observed quantities:

– the next input f [k +1],

– the past R input-output pairs {f [r], y[r]}k�R+1rk.


133

RLS objective and Rth-order linear tap filter

• The output of an Rth-order RLS tap-filter at time k is,

yk[i] =iX

r=i�R+1

⌘k[i� r]f [r], i k +1.

• The objective of this filter at time k is to accurately estimate the system output y[k+1]with yk[k +1] by choosing the R filter coe�cients

⌘k[k �R+1], ..., ⌘k[k � 1], ⌘k[k]

that minimize the following sum-of-square-error objective:

Ek =kX

r=k�R+1

�k�r|y[r]� yk[r]|2 =kX

r=k�R+1

�k�r|ek[r]|2

where

– � > 0 is a forgetting factor and

– error ek[r] := y[r]� yk[r].


134

RLS filter

• So, to minimize Ek, substitute yk[r] into Ek and solve

0 =@Ek

@⌘k[i]for i 2 {k �R+1, ..., k � 1, k}.

• That is, R equations in R unknowns: for i 2 {k �R+1, ..., k � 1, k},

0 =kX

r=k�R+1

2�k�rek[r]@ek[r]

@⌘k[i]

=kX

r=k�R+1

2�k�r(y[r]� yk[r])

✓�@yk[r]

@⌘k[i]

◆

=kX

r=k�R+1

2�k�r(yk[r]� y[r])f [r � i]

• Exercise: Prove the last equality.


135

RLS filter (cont)

• Substituting yk[r], rewrite these equations to get the following R equations in R unknowns⌘k[i] that are Ek-minimizing: for i 2 {k �R+1, ..., k � 1, k},

kX

r=k�R+1

�k�rf [r � i]rX

`=r�R+1

f [`]⌘k[r � `] =kX

r=k�R+1

�k�ry[r]f [r � i]

• Exercise: Prove the last equality and write it in matrix form.

• Exercise: Research how the Ek-minimizing filter parameters ⌘k can be computed recur-sively, i.e., using ⌘k�1.

• The filter order R can also be “trial adapted” to discover the system order K so that theerror-minimizing filter parameters ⌘k “track” the system unit-pulse response h over time k.

• Note the required initial “warm-up” period of R time-units where the outputs of system hare simply observed and recorded and no estimates are made.

• Exercise: If there was no additive noise process v and the system unit-pulse response hhad finite support (i.e., a FIR system with K < 1), show how h can be deduced frominput-output (f, y) observations.


136

signals and linear and time-invariant systems in discrete timegik2/teach/linear_systems-dt-2.pdf ·...

Documents