Lesson 3 Approximating Fourier series

1

2

• Last lecture, we saw that the trapezoidal rule was an effective method for calculatingintegrals of periodic functions

We used the Euler–McLaurin formula to prove that the error decayed fasterthan O

�1

n�

�for any �

• In this lecture, we will apply this to calculating Fourier coefficients and Fourier series

• This is practical function approximation

3

• Given an integrable function f defined on T = [��, �), the following exist forevery integer k:

f̂k =1

2�

� �

��f(�) � k� x

• We can formally define the Fourier series of f :

f(�) ���

k=��f̂k

k�

This sum will converge uniformly to f for periodic smooth functions

4

-3 -2 -1 0 1 2 3

• Define the m evenly spaced points on the periodic interval � = (�1, . . . , �m):

� :=

���,

�2

m� 1

��, . . . ,

�1 � 2

m

��

�=

• We can approximate the Fourier coefficients using the m point trapezoidal rule

f̂k =1

2�

� �

��f(�) � � 1

m

m�

j=1

f(�j)�k�j =: f̂m

k

5

-3 -2 -1 0 1 2 3

• Define the m evenly spaced points on the periodic interval � = (�1, . . . , �m):

� :=

���,

�2

m� 1

��, . . . ,

�1 � 2

m

��

�=

• We can approximate the Fourier coefficients using the m point trapezoidal rule

f̂k =1

2�

� �

��f(�) � � 1

m

m�

j=1

f(�j)�k�j =: f̂m

k

6

• Using these, and truncating the Fourier sum between integers � and � we obtainan approximate Fourier series

f(�) � f�,�,m(�) :=��

k=�

f̂mk

k�

• Big question: how to choose �, � and m?

• When we specify just � and � , we will choose m to be the same as the numberof coefficients

f�,�(�) := f�,�,���+1(�)

• When we specify just m, we will choose roughly equal number of negative andpositive coefficients:

fm(�) :=

�f 1�m

2 , m�12 ,m(�) m odd

f� m2 , m

2 �1,m(�) m even

7

• Using these, and truncating the Fourier sum between integers � and � we obtainan approximate Fourier series

f(�) � f�,�,m(�) :=��

k=�

f̂mk

k�

• Big question: how to choose �, � and m?

• When we specify just � and � , we will choose m to be the same as the numberof coefficients

f�,�(�) := f�,�,���+1(�)

• When we specify just m, we will choose roughly equal number of negative andpositive coefficients:

fm(�) :=

�f 1�m

2 , m�12 ,m(�) m odd

f� m2 , m

2 �1,m(�) m even

8

• Using these, and truncating the Fourier sum between integers � and � we obtainan approximate Fourier series

f(�) � f�,�,m(�) :=��

k=�

f̂mk

k�

• Big question: how to choose �, � and m?

• When we specify just � and � , we will choose m to be the same as the numberof coefficients

f�,�(�) := f�,�,���+1(�)

• When we specify just m, we will choose roughly equal number of negative andpositive coefficients:

fm(�) :=

�f 1�m

2 , m�12 ,m(�) m odd

f� m2 , m

2 �1,m(�) m even

Experimental results

9

10

�

-3 -2 -1 1 2 3q

5

10

m = 5

11

�

-3 -2 -1 1 2 3q

5

10

m = 5

-3 -2 -1 1 2 3q

0.5

1.0

1.5

2.0

2.5

3.0

m = 5

|� � .1|

12

-3 -2 -1 1 2 3q

-1.0

-0.5

0.5

1.0

m = 5

�

-3 -2 -1 1 2 3q

5

10

m = 5

20

��

-3 -2 -1 1 2 3q

0.5

1.0

1.5

2.0

2.5

3.0

m = 5

|� � .1| (� � .1)

-3 -2 -1 1 2 3q

-1.0

-0.5

0.5

1.0

1.5m = 5

-3 -2 -1 1 2 3q

-1.0

-0.5

0.5

1.0m = 5

5�

-3 -2 -1 1 2 3q

-0.5

0.5

1.0m = 5

2 �

13

�20

��

|� � .1| (� � .1)

5�

-3 -2 -1 1 2 3q

-0.4

-0.2

0.2

0.4

0.6

0.8

1.0m = 10

-3 -2 -1 1 2 3q

-1.0

-0.5

0.5

1.0m = 10

2 �

-3 -2 -1 1 2 3q

-1.0

-0.5

0.5

1.0

m = 10

-3 -2 -1 1 2 3q

0.5

1.0

1.5

2.0

2.5

3.0

m = 10

-3 -2 -1 1 2 3q

-1.0

-0.5

0.5

1.0m = 10

-3 -2 -1 1 2 3q

5

10

15

m = 10

14

�20

��

|� � .1| (� � .1)

5�

-3 -2 -1 1 2 3q

-0.4

-0.2

0.2

0.4

0.6

0.8

1.0m = 100

-3 -2 -1 1 2 3q

-1.0

-0.5

0.5

1.0m = 100

2 �

-3 -2 -1 1 2 3q

-1.0

-0.5

0.5

1.0

m = 100

-3 -2 -1 1 2 3q

0.5

1.0

1.5

2.0

2.5

3.0

m = 100

-3 -2 -1 1 2 3q

-1.0

-0.5

0.5

1.0m = 100

-3 -2 -1 1 2 3q

5

10

15

20

25m = 100

15

• Observed convergence properties:

Fast convergence for periodic functions, just like trapezoidal ruleSlow convergence for non-periodic functions away from singularitiesNo convergence in neighbourhood of jump singularities (including ±�)

• We also observed interpolation at the quadrature points �

• To understand this, we need to related the approximate Fourier coefficients f̂mk

to the true Fourier coefficients f̂k

• This will follow naturally from orthogonality properties of �

Orthogonality of complex exponentials

16

17

The L2 inner product and norm• We define the 2 inner product (on T) by

�f, g� =1

2�

� �

��f̄(�)g(�) �

• Associated with this inner product is the 2 norm:

�f� =

�1

2�

� �

��|f(�)|2 �

• 2 space is all integrable functions f such that �f� < �

Exercise: verify 2 is a vector space and �f, g� is an inner product on 2

18

• A set of nonzero vectors v1, . . . , vn in a vector space V are called orthogonal if

�vi, vj� = 0 whenever i �= k.

• They are called orthonormal if they are orthogonal and all vectors are of unit norm:

1 = �vi� , or equivalently, �vi, vi� = 1.

• For orthonormal vectors vk , we can construct a projection of a vector f � V into{v1, . . . , vn} by

Pf :=n�

k=1

�vk, f� vk

If f � {v1, . . . , vn} then f is equal to its projection:

f = Pf

In other words P2f = Pf

19

• A set of nonzero vectors v1, . . . , vn in a vector space V are called orthogonal if

�vi, vj� = 0 whenever i �= k.

• They are called orthonormal if they are orthogonal and all vectors are of unit norm:

1 = �vi� , or equivalently, �vi, vi� = 1.

• For orthonormal vectors vk , we can construct a projection of a vector f � V into{v1, . . . , vn} by

Pf :=n�

k=1

�vk, f� vk

If f � {v1, . . . , vn} then f is equal to its projection:

f = Pf

In other words P2f = Pf

20

• A set of nonzero vectors v1, . . . , vn in a vector space V are called orthogonal if

�vi, vj� = 0 whenever i �= k.

• They are called orthonormal if they are orthogonal and all vectors are of unit norm:

1 = �vi� , or equivalently, �vi, vi� = 1.

• For orthonormal vectors vk , we can construct a projection of a vector f � V into{v1, . . . , vn} by

Pf :=n�

k=1

�vk, f� vk

If f � {v1, . . . , vn} then f is equal to its projection:

f = Pf

In other words P2f = Pf

21

• A set of nonzero vectors v1, . . . , vn in a vector space V are called orthogonal if

�vi, vj� = 0 whenever i �= k.

• They are called orthonormal if they are orthogonal and all vectors are of unit norm:

1 = �vi� , or equivalently, �vi, vi� = 1.

• For orthonormal vectors vk , we can construct a projection of a vector f � V into{v1, . . . , vn} by

Pf :=n�

k=1

�vk, f� vk

If f � {v1, . . . , vn} then f is equal to its projection:

f = Pf

In other words P2f = Pf

22

• We have�

k�, k��

=1

2�

� �

��� = 1

and for k �= j

�k�, j�

�=

1

2�

� �

��

(j�k)� � =(j�k)� � � (j�k)�

2� (j � k)= 0

• In other words, the complex exponentials are orthonormal!

• Thus we can think of the Fourier series as an infinite projection

f(�) ���

k=��

�k�, f

�k�

Since this sum is infinite, we cannot appeal to the simple argument of equalityfrom the last slide

Discrete orthogonality of complex exponentials

23

24

• We have shown that the complex exponentials are orthogonal with respect to theinner product

�f, g� =1

2�

� �

��f̄(�)g(�) �

• A remarkable fact we now show is that they are also orthogonal with respect tothe following discrete semi-inner product:

�f, g�m =1

m

m�

j=1

f̄(�j)g(�j) =f(�)�g(�)

m

where � = (�1, . . . , �m) are again evenly spaced points:

� =

���,

�2

m� 1

��, . . . ,

�1 � 2

m

��

�= -3 -2 -1 0 1 2 3

25

Evenly spaced points on the unit circle

ei✓

-1.0 -0.5 0.5 1.0

-1.0

-0.5

0.5

1.0

-3 -2 -1 0 1 2 3

� = (�1, . . . , �m) z = (z1, . . . , zm)

26

Some identities (shown for even m):

z

m�

j=1

ei�j =m�

j=1

zj = 0-1.0 -0.5 0.5 1.0

-1.0

-0.5

0.5

1.0

27

Some identities (shown for even m):

z

m�

j=1

ei2�j =m�

j=1

z2j = 0

-1.0 -0.5 0.5 1.0

-0.5

0.5

28

:

m�

j=1

k�j = (�)km for k = . . . , �2m, �m, 0, m, 2m. . .

m�

j=1

k�j = 0 for all other integer k

• Note thatzj = � 2� (j�1)/m = ��j�1

for � = 11/m = 2� /m

• Therefore,m�

j=1

zkj = (�)k

m�1�

j=0

�kj

• If k = �m is a multiple of m, then we have

m�1�

j=0

�kj =m�1�

j=0

(�m)�j =m�1�

j=0

1�j = m

• Note thatzj = � 2� (j�1)/m = ��j�1

for � = 11/m = 2� /m

• Therefore,m�

j=1

zkj = (�)k

m�1�

j=0

�kj

• If k = �m is a multiple of m, then we have

m�1�

j=0

�kj =m�1�

j=0

(�m)�j =m�1�

j=0

1�j = m

• Note thatzj = � 2� (j�1)/m = ��j�1

for � = 11/m = 2� /m

• Therefore,m�

j=1

zkj = (�)k

m�1�

j=0

�kj

• If k = �m is a multiple of m, then we have

m�1�

j=0

�kj =m�1�

j=0

(�m)�j =m�1�

j=0

1�j = m

• Note thatzj = � 2� (j�1)/m = ��j�1

for � = 11/m = 2� /m

• Therefore,m�

j=1

zkj = (�)k

m�1�

j=0

�kj

• If k = �m is a multiple of m, then we have

m�1�

j=0

�kj =m�1�

j=0

(�m)�j =m�1�

j=0

1�j = m

• For k not a multiple of m, recall the geometric sum formula:

m�1�

j=0

zj =zm � 1

z � 1

• Thusm�1�

j=0

�kj =m�1�

j=0

(�k)j =�km � 1

�k � 1

• Because k is not a multiple of m, the denominator is nonzero

• Because �km = (�m)k = 1k , the numerator is zero

• Note thatzj = � 2� (j�1)/m = ��j�1

for � = 11/m = 2� /m

• Therefore,m�

j=1

zkj = (�)k

m�1�

j=0

�kj

• If k = �m is a multiple of m, then we have

m�1�

j=0

�kj =m�1�

j=0

(�m)�j =m�1�

j=0

1�j = m

• For k not a multiple of m, recall the geometric sum formula:

m�1�

j=0

zj =zm � 1

z � 1

• Thusm�1�

j=0

�kj =m�1�

j=0

(�k)j =�km � 1

�k � 1

• Because k is not a multiple of m, the denominator is nonzero

• Because �km = (�m)k = 1k , the numerator is zero

• Note thatzj = � 2� (j�1)/m = ��j�1

for � = 11/m = 2� /m

• Therefore,m�

j=1

zkj = (�)k

m�1�

j=0

�kj

• If k = �m is a multiple of m, then we have

m�1�

j=0

�kj =m�1�

j=0

(�m)�j =m�1�

j=0

1�j = m

• For k not a multiple of m, recall the geometric sum formula:

m�1�

j=0

zj =zm � 1

z � 1

• Thusm�1�

j=0

�kj =m�1�

j=0

(�k)j =�km � 1

�k � 1

• Because k is not a multiple of m, the denominator is nonzero

• Because �km = (�m)k = 1k , the numerator is zero

��

k�, j��

m= (�1)j�k JSV j � k = . . . , �2m, �m, 0, m, 2m, . . .

�k�, j�

�m

= 0 SXLIV[MWI

�

� *SPPS[W�JVSQ�TVIGIHMRK�XLISVIQ�

�k�, j�

�m

=1

m

m�

j=1

(j�k)�j