Lab 7: Filters and convolution

One of the most remarkable properties of a linear dynamical system is itsresponse to sinusoidal inputs. As we learned in lecture, a sinusoidal inputto a linear dynamical system produces a sinusoidal output having the samefrequency as the input. However, the output sinusoid can differ in two waysfrom the input sinusoid: 1) its amplitude, and 2) its phase. Measurementsof the phase and amplitude of the output to a sinusoidal input can be usedto identify the differential equation which best captures the input-output re-lationships of the dynamical system. The graphical technique for estimatingthe transfer function from these experimental observations is the Bode plot.Once the transfer function has been determined the response of the dynam-ical system to any realizable input can be predicted. The purpose of thislaboratory is to use computer experiments to verify these observations andto introduce the concepts of a filter and convolution.

1 Background

The lecture briefly reviews the concept of an impulse function, filtering andconvolution. A practical, but important, detail concerns the representationof a sine wave wave in a computer program. We can write a simple programthat plots a 10 second interval of a sine wave for a given frequency, f , and agiven time step.

import numpy as npimport matplotlib.pyplot as plt

f=0.1N=11NN=101

1

start=0.0stop=100.0t=np.linspace(start,stop,num=N,endpoint=True)tt=np.linspace(start,stop,num=NN,endpoint=True)plt.plot(t,np.sin(2*np.pi*f*t),’k-o’)plt.plot(tt,np.sin(2*np.pi*f*tt),’r-’)plt.axis([0,stop,-1.1,1.1])plt.show()

The period, T , is 1f, so when f = 0.1, we have T = 10. If we choose start

and stop to be, respectively, 0 and 100, then this would draw ten cycles of a0.1Hz provided that N is chosen appropriately (red line in figure produced bythis program). The positive integer, N , is the number of equally spaced timesamples between, in this case, 0 and 100. It is important to remember thatby choosing the endpoint=True option for np.linspace() both the startingand the end point are included, N points altogether1. The time step, dt is

dt =stop-start

Ntimes per period

What is a good choice of the number of time steps, N? If we chooseN = 10, then dt = 1 per period and we would obtain 10 identical xalues ofx since we are sampled once per period and the rtime series is 10 periodslong (compare black and red lines). This is not a good choice of N sincewe cannot uniquely represent a function using a single value. Next we couldtry N = 20 which would give us two point per period. In principle we canuniquely represent a sine wave if we sample at twice the frequency (this isthe concept of the Nyquist frequency that we will discuss in the next lab).However, x would not look like a nice sine wave. So we must pick N larger:the larger N , the smaller dt, and hence the more frequently the sine waveis sampled. However, it is also true that the larger N the more computermemory it takes to store in this case x and t, and the greater the number ofcalculations that the computer must make. Thus there is a practical trade–off between the accuracy by which functions are to be represented and thetime it takes the computer to complete its calculations. The bottom lineis that as we change f we need to adjust dt. A reasonable choice might beto sample the sine wave 10− 100 times per period. Ideally we would like tokeep the number of sample points per period the same as we change f .

1Note that to get say 10 evenly spaced samples you need to set num= N + 1.

2

Browser use: Have available the document intro_python.pdf on yourbrowser for review, in particular, the sections that deal with how to de-clare a function. As always, information concerning the use of Python andmatplotlib functions can be obtained from the Internet using a browser.

Housekeeping: Labs 8 and 9 use some of the programs and results thatwe develop in this lab. Thus it is useful to put today’s material in a sub-directory of ~\pyprogs called, for example, convol, namely ~\pyprogs\convol.This will make it easier for you to find these results when we need them.

Exercise 1: Bode plots: low–pass filter

1. Write a computer program to integrate

dx

dt= k1x+ sin 2πft , (1)

where k1 is a constant.

2. Take k1 = −1.

3. Vary frequency over a large range (say f = 0.01 to f = 100) andmeasure the amplitude. Plot the amplitude versus frequency.

4. Vary the frequency and measure the shift between the sine wave andthe solution of the equation. For example, we can plot the input andoutput sine wave n the same figure. Plot the phase shift versus thefrequency.

5. Now change a to, let’s say, k1 = 10 and then k1 = 0.1. What happens?

6. Construct the Bode plot by choosing a suitable range of f . For eachchoice of f enter the amplitude and phase shift of the steady–statesolution and write these values in a *.tsv file. Use this *.tsv file toconstruct the phase part of the Bode plot.

7. You have just constructed your first low pass filter. Answer the follow-ing:

• The frequency on the input sine wave and the output sine wave isthe same. True of false?

3

• The output amplitude changes as a function of frequency. Trueor false?

• The output sine wave is phase shifted compared to the input sinewave. True or false?

• Why is this type of filter called a low pass filter?

8. What is the transfer function for (1)?

9. Show that the impulse function, I(t, t′), is

I(t, t′) = k1 exp(k1t) .

(Remember that the impulse function corresponds to the case when theinput is a delta function.) Is this the same as the answer we derived inclass (Section 7.3)?

10. The transfer function, C(s), for the thermometer is of the form

C(s) =1

1 + αs.

Sketch the Bode plot. What is the corner frequency?

11. Consider the transfer function

C(s) =10

s(1 + 0.5s)(1 + 0.1s).

Sketch the Bode plot for the amplitude showing the contribution ofeach term.

Exercise 2: The convolution integral:The concept of convolution is one of the most important concepts in

science: every time one uses a device to make a measurement, oneessentially performs a convolution. All physical observations are limitedby the resolving power of instruments, and for this reason alone convolution isubiquitous. This principle does not only apply to laboratory instruments: oursenses, e.g. vision, touch, smell, audition, proprioception, are also convolutedby the sensory receptors which translate physical stimuli into neural spikestrains. This convolution is mathematically as the operation

y(t) =∫ ∞−∞

x(t)h(t− u)du , (2)

4

where x(t) describes the input to the dynamical system and h(t − u) de-scribes the impulse function. In practice it is very easy to perform a convolu-tion numerically. In Python this is accomplished using the single commandnp.convolve() (similar commands exist, for example, in MATLAB). In or-der to use the convolution integral to solve (1) we need to know that theimpulse function.

Convolution: Graphical interpretationAlthough the mechanics of evaluating the convolution integral seem straight-

forward, it is quite difficult to understand in a simple way the process of evalu-ating the convolution integral. The purpose of this exercise is to demonstratea graphical interpretation of convolution [1]. This graphical method is veryuseful and, for example, we can use it to understand issues related to zero-padding and to appreciate the difference between convolution and correlation(Lab 14).

Figure 1: Graphical method for convolving a delta–function input (verticaldashed line in ’Multiplication panel’) with an alpha–type impulse response(solid line). See text for discussion.

The graphical convolution method divides the process of convolution into

5

four steps [1]:

1. Folding: Reflect the impulse response, I(t), about the ordinate axis toobtain I(−t).

2. Displacement Shift I(−t) by an amount ∆t to obtain I(−t+ ∆t).

3. Multiplication: Multiply I(−t+ ∆t) by the input, b(t).

4. Integration: Determine the area under the curve produced by I(−t +∆t) and b(t).

Figure 1 illustrates this procedure in the evaluation of the alpha functionresponse, I(t), of a neuron to a single delta-function input, b(t). The step‘Folding’ means that we reflect the impulse response about the y-axis. Thestep ‘Displacement’ means that we slowly shift the reflected impulse in thepositive direction along the x-axis. The step ‘Multiplication’ means that wemultiple the reflected impulse with the input. It is easiest to understand thisstep iteratively: for each time step do the multiplication (and then Step 4),then displace the reflected impulse response another time step to the rightand repeat the process. In general this corresponds to the shaded regionunder the two curves for which both functions are non-zero. Finally the step‘Integration’ calculates the area under the the two curves. In this examplethe area is easy to determine since by the shifting property of the delta-function (Section 6.3), the area is just the value of the alpha function at thatpoint. The result of convolution is the area computed in Step 4 as a functionof time, namely as the reflected impulse response is shifted iteratively to theright.

Questions to be answered:

1. Use the graphical technique discussed in Section 1 to compute the con-volution for the following choices of b(t) and I(t)

• b(t) = constant and I(t) = e−t

• b(t) is a single square wave pulse and I(t) = e−t (This is animatedon the Wikipedia site for Convolution.)

• Both b(t) and I(t) are square wave pulses (This is animated onthe Wikipedia site for Convolution.)

6

Exercise 3: Computing the convolutionNumpy has a function called np.convolve(). Although it is tempting

to think that it would be easy to calculate the convolution of any function;there are some subtleties.

The function np.convolve(r,s,mode=’full’) calculates the convolu-tion integral, denoted (r ∗ s), as

(r ∗ s)j =N/2∑

k=−N/2+1

sj−krk

where s is the discretely sampled signal and r is the discretely sampled im-pulse function [2]. Unfortunately this approximation assumes that the lengthof r is the same as that of s and that s is periodic. Thus we need to figureout how to get around these constraints. The solution involves padding thedata and impulse response with zeros. This procedure is called zero padding.

The first constraint is easily dealt with since it is typically the case thatthe length of impuse response, M , is much shorter than the length of thesignal, N . Thus we need to pad the impulse function with zero is makeM = N . The second problem is more subtle. Not only must the signal beconsidered to be periodic, but also the impulse function. Therefore it happensthat a portion at each end of the original signal is erroneously wrappedaround by the convolution with the response function. In other words, thefirst output channel (s∗r)0 is contaminated wuth some wrapped-around datafrom the far end of the data stream sN−1, sn−2, · · ·. This we need to set up abuffer zone of zero-padded values at the end of s in order to make the effectsof this contamination equal to zero. How many zeros do we need top add?We need to pad the signal function on one end with the number of zerosequal to the length of the response function.

The numpy function, np.pad(), is very useful for handling issues relatedto zero padding

np.pad(array,pad_width,mode=None).

A useful feature of np,pad() is that we can pad to the left and/or to theright of the array. For example, if

x=[1,2,3]

then the command

7

X=np.pad(x,(0,3),’constant’)

yields

X=[1,2,3,0,0,0]

and the command

X=np.pad(x,(3,0),’constant’)

yields the array

X=[0,0,0,1,2,3]

However, we are still not done. In terms of our fold-displacement-multiplication-integration analogy for convolution, convolve() does not include the inte-gration step. Thus we need to multiple the output of np.convolve() by thetime step.

Questions to be answered:

1. Write a computer program that uses np.convolve() to construct theBode plot for (1). Again for each choice of F , measure the amplitudeand phase shift using the steady state solution.

Exercise 4: Neural inputs: The program convolve_neural_spikes_alpha.py

uses the convolution integral to estimate the sub-threshold membrane poten-tial when the impulse function is given by the alpha function

Iα(t, t′) =

{±k2

m(t− t′)e−km(t−t′) if t ≥ t′ ,0 t < t′ .

(3)

The input spike train is modeled as a sequence of delta functions. Figure 2shows how the membrane potential changes for a given periodic excitatoryinput.

Hints:

1. A common use of def is to define and introduce functions that are usedfor calculations in the program. The command r=impulse_response(t)

evaluates the function for all of the values of t. Note that the commandsin the def are indented by usually 4 spaces

8

Figure 2: The membrane potential of a neuron (bottom panel) in responseto a pulsatile, excitatory input whose timing corresponds to the timing ofspikes in the spike train shown in the top panel. The changes in the neuron’smembrane potential following the arrival of each EPSP is described by (3)with km = 0.5.

2. Useful array manipulations:

[1:5] means from 1 to 5

[1:] means 1 to end

[len(a):1] means from length of a to end

[::n] means each n-th item in entire sequence

3. The function plt.subplot() has the form

\verb+plt.subplot(row,column,panel number)

Which panel would be drawn using the command plt.subplot(223)?

4. The package matplotlib.ticker includes a number of useful functionsthat make it possible to, for example, change the spacing of the numberon the axis, etc. Labeling the subplots ax, bx, .. is necessary in order

9

to use the functions within matplotlib.ticker. In the case of a figurethat has only one panel, we can use the command plt.subplot(111).

Question to be answered:An impulse function can be used to model the changes in blood concentra-

tion of a drug ingested orally. For example, if we take (3) as a simple impulsefunction for drug ingestion, then the sharp rising portion indicates the timecourse of the drug going from the stomach to the blood (more typically min-utes), and the falling phase describes the clearance of drug from the body bythe actions of the liver and kidneys (typically hours to days). Consider thata doctor prescribes a drug for which the e−1 time is ∼ 6 hours and asks thatthe patient take the medication 4 times a day. Most patients do not takethe medication every 6 hours, but more often at 8am–noon–5pm–midnight.Write a computer program to look at the effects of different dosing scheduleson blood levels to answer the following questions.

• Would it be better for the patient to take the medication three timesa day, i.e. every 8 hours?

• If the e−1 time is ∼ 4 hours, what would be the difference between thetwo dosing schedules?

• If the e−1 time is ∼ 12 hours, what would be the difference betweenthe two dosing schedules?

Low-pass filters with two-polesHere we discuss the transfer function for a second-order linear ordinary

differential equation. As we discussed in lecture it is traditional and conve-nient to write the differential equation in the form

1

ω2n

d2x

dt2+

2ε

ωn

dx

dt+ x = b(t) , (4)

where ωn = 2πfn is the natural frequency of the associated harmonic oscil-lator (Note that ωn = 2πfn). The damping ratio is ε:

ε > 1 : Overdamped ,

ε = 1 : Critically damped ,

1 > ε > 0 : Underdamped ,

ε = 0 : Undamped .

10

For the homogeneous ordinary differential equation

d2x

dt2+ 2εωn

dx

dt+ ω2

nx = 0 , (5)

the two eigenvalues are

λ1,2 =2εωn ±

√(2εωn)2 − 4ω2

n

2,

= −εω2 ± ωn√ε2 − 1 .

When we determine the impulse function we need to remember that theλ1,2 are either be real negative numbers (the over-damped case) or a pair ofcomplex numbers with negative real parts (the under-damped case). This isbecause we can only define the impulse function for a dynamical system whichin the absence of an input has a stable fixed–point. For the over–dampedcase we have

I(t, t′) =ωn

2√ε2 − 1

[eλ1t − eλ2t

]. (6)

For the under-damped case we have

I(t, t′) =σ2 + ω2

ωe−σt sin(ωt) , (7)

where σ is the absolute value of the real part of the complex eigenvalue andω is the imaginary part.

Questions to be answered:Consider the second–order linear differential equation

d2x

dt2+ k1

dx

dt+ k2x = sin 2πft , (8)

where k1, k2 are constants to be determined.

1. Choose the values of k1, k2 such that both eigenvalues are real andnegative. In particular, pick the parameters so that one eigenvalues,λ1, is equal to −1. Construct the Bode plot when λ2 is smaller thanλ1 and another when λ2 is larger. Compare these Bode plots to theone we obtained for (1). You should be able to sketch these Bode plotswithout using computer simulations (Of course you are welcome to usea computer program to confirm the validity of your sketches.) Whatare the differences?

11

2. Pick values of k1, k2 such that the fixed–point of the homogeneous equa-tion (i.e., the right–hand side is 0) is a stable spiral point.

• For these choice of parameters, what is the natural frequency?

• Write a computer program to show that as k1 approaches zero,(8) exhibits resonance as f is closed to the natural frequency, fn.Your figure should show the amplified solution (x versus t).

• Sketch the Bode plot for this case.

Deliverables:Assignment should be submitted as a *.pdf prepared using LaTeX. An-

swer the questions. In those cases in which the figures are sketched, attachthe sketches to the LaTeX document.

References

[1] E. O. Brigham. The fast Fourier transform and its applications. PrenticeHall, Englewood, N. J., 1988.

[2] W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery. Nu-merical recipes: The art of scientific computing, third edition. CambridgeUniversity Press, New York, 2007.

12