1Orthogonal functions

Function approximation: Fourier, Chebyshev, Lagrange

Orthogonal functionsFourier SeriesDiscrete Fourier SeriesFourier Transform: propertiesChebyshev polynomialsConvolutionDFT and FFT

Scope: Understanding where the Fourier Transform comes from. Moving from the continuous to the discrete world. The concepts are the basis for pseudospectral methods and the spectral element approach.

2Orthogonal functions

Fourier Series: one way to derive them

The Problem

we are trying to approximate a function f(x) by another function gn (x) which consists of a sum over N orthogonal functions Φ(x) weighted by some coefficients an .

)()()(0

xaxgxfN

iiiN ∑

=

Φ=≈

3Orthogonal functions

... and we are looking for optimal functions in a least squares (l2 ) sense ...

... a good choice for the basis functions Φ(x) are orthogonal functions. What are orthogonal functions? Two functions f and g are said to be

orthogonal in the interval [a,b] if

∫ =b

a

dxxgxf 0)()(

How is this related to the more conceivable concept of orthogonal vectors? Let us look at the original definition of integrals:

The Problem

{ } !Min)()()()(2/1

2

2=⎥

⎦

⎤⎢⎣

⎡−=− ∫

b

aNN dxxgxfxgxf

4Orthogonal functions

Orthogonal Functions

... where x0 =a and xN =b, and xi -xi-1 =Δx ...If we interpret f(xi ) and g(xi ) as the ith components of an N component

vector, then this sum corresponds directly to a scalar product of vectors. The vanishing of the scalar product is the condition for orthogonality of

vectors (or functions).

⎟⎠

⎞⎜⎝

⎛Δ= ∑∫

=∞→

N

iii

b

aN

xxgxfdxxgxf1

)()(lim)()(

figi

0==• ∑ ii

iii gfgf

5Orthogonal functions

Periodic functions

-15 -10 -5 0 5 10 15 200

10

20

30

40

Let us assume we have a piecewise continuous function of the form

)()2( xfxf =+ π

2)()2( xxfxf ==+ π

... we want to approximate this function with a linear combination of 2π periodic functions:

)sin(),cos(),...,2sin(),2cos(),sin(),cos(,1 nxnxxxxx

{ }∑=

++=≈⇒N

kkkN kxbkxaaxgxf

10 )sin()cos(

21)()(

6Orthogonal functions

Orthogonality

... are these functions orthogonal ?

0,00)sin()cos(

0

0,,0)sin()sin(

002

0)cos()cos(

>≥=

>=

>≠

⎪⎩

⎪⎨

⎧=

>===

≠

⎪⎩

⎪⎨

⎧=

∫

∫

∫

−

−

−

kjdxkxjx

kj

kjkjdxkxjx

kjkj

kjdxkxjx

π

π

π

π

π

π

π

ππ

... YES, and these relations are valid for any interval of length 2π.Now we know that this is an orthogonal basis, but how can we obtain the

coefficients for the basis functions?

from minimising f(x)-g(x)

7Orthogonal functions

Fourier coefficients

optimal functions g(x) are given if

{ } 0)()(!Min)()(22=−=− ∂

∂ xfxgorxfxg nan k

leading to

... with the definition of g(x) we get ...

{ }⎥⎥⎦

⎤

⎢⎢⎣

⎡⎥⎦

⎤⎢⎣

⎡−++

∂∂

=−∂∂

∫ ∑− =

dxxfkxbkxaaa

xfxga

N

kkk

kn

k

2

10

2 )()sin()cos(21)()(

2

π

π

{ }

∫

∫

∑

−

−

=

==

==

++=

π

π

π

π

π

π

Nkdxkxxfb

Nkdxkxxfa

kxbkxaaxg

k

k

N

kkkN

,...,2,1,)sin()(1

,...,1,0,)cos()(1

with)sin()cos(21)(

10

8Orthogonal functions

Fourier approximation of |x|

... Example ...

.. and for n<4 g(x) looks like

leads to the Fourier Serie

⎭⎬⎫

⎩⎨⎧ +++−= ...

5)5cos(

3)3cos(

1)cos(4

21)( 222

xxxxgπ

π

ππ ≤≤−= xxxf ,)(

-20 -15 -10 -5 0 5 100

1

2

3

4

9Orthogonal functions

Fourier approximation of x2

... another Example ...

π20,)( 2 <<= xxxf

.. and for N<11, g(x) looks like

leads to the Fourier Serie

∑= ⎭

⎬⎫

⎩⎨⎧ −+=

N

kN kx

kkx

kxg

12

2

)sin(4)cos(43

4)( ππ

-10 -5 0 5 10-10

0

10

20

30

40

10Orthogonal functions

Fourier - discrete functions

iN

xiπ2

=

.. the so-defined Fourier polynomial is the unique interpolating function to the function f(xj ) with N=2m

it turns out that in this particular case the coefficients are given by

,...3,2,1,)sin()(2

,...2,1,0,)cos()(2

1

*

1

*

==

==

∑

∑

=

=

kkxxfN

b

kkxxfN

a

N

jjj

N

jjj

k

k

{ } )cos(21)sin()cos(

21)( *

1

1

****0 kxakxbkxaaxg m

m

km kk

+++= ∑−

=

... what happens if we know our function f(x) only at the points

11Orthogonal functions

)()(*iim xfxg =

Fourier - collocation points

... with the important property that ...

... in our previous examples ...

-10 -5 0 50

0.5

1

1.5

2

2.5

3

3.5

f(x)=|x| => f(x) - blue ; g(x) - red; xi - ‘+’

12Orthogonal functions

Fourier series - convergencef(x)=x2 => f(x) - blue ; g(x) - red; xi - ‘+’

13Orthogonal functions

Fourier series - convergencef(x)=x2 => f(x) - blue ; g(x) - red; xi - ‘+’

14Orthogonal functions

Gibb’s phenomenonf(x)=x2 => f(x) - blue ; g(x) - red; xi - ‘+’

0 0.5 1 1.5-6

-4

-2

0

2

4

6

N = 32

0 0.5 1 1.5-6

-4

-2

0

2

4

6

N = 16

0 0.5-6

-4

-2

0

2

4

6

0 0.5 1 1.5-6

-4

-2

0

2

4

6

N = 128

0 0.5 1 1.5-6

-4

-2

0

2

4

6

N = 256

The overshoot for equi- spaced Fourier

interpolations is ≈14% of the step height.

15Orthogonal functions

Chebyshev polynomials

We have seen that Fourier series are excellent for interpolating (and differentiating) periodic functions defined on a regularly spaced grid. In many circumstances physical phenomena which are not periodic (in space) and occur in a limited area. This quest leads to the use of Chebyshev polynomials.

We depart by observing that cos(nϕ) can be expressed by a polynomial in cos(ϕ):

1cos8cos8)4cos(cos3cos4)3cos(

1cos2)2cos(

24

3

2

+−=

−=

−=

ϕϕϕ

ϕϕϕ

ϕϕ

... which leads us to the definition:

16Orthogonal functions

Chebyshev polynomials - definition

NnxxxTTn nn ∈−∈=== ],1,1[),cos(),())(cos()cos( ϕϕϕ

... for the Chebyshev polynomials Tn (x). Note that because of x=cos(ϕ) they are defined in the interval [-1,1] (which - however - can be extended to ℜ). The first polynomials are

0

244

33

22

1

0

and]1,1[for1)(where188)(

34)(

12)(

)(1)(

NnxxTxxxT

xxxT

xxT

xxTxT

n ∈−∈≤

+−=

−=

−=

==

17Orthogonal functions

Chebyshev polynomials - Graphical

The first ten polynomials look like [0, -1]

The n-th polynomial has extrema with values 1 or -1 at

0 0.2 0.4 0.6 0.8-1

-0.5

0

0.5

1

x

T_n(

x)

nkn

kx extk ,...,3,2,1,0),cos()( ==

π

18Orthogonal functions

Chebyshev collocation points

These extrema are not equidistant (like the Fourier extrema)

nkn

kx extk ,...,3,2,1,0),cos()( ==

π

k

x(k)

19Orthogonal functions

Chebyshev polynomials - orthogonality

... are the Chebyshev polynomials orthogonal?

02

1

1

,,0

02/0

1)()( Njk

jkforjkforjkfor

xdxxTxT jk ∈

⎪⎭

⎪⎬

⎫

⎪⎩

⎪⎨

⎧

==>=

≠=

−∫− π

π

Chebyshev polynomials are an orthogonal set of functions in the interval [-1,1] with respect to the weight functionsuch that

21/1 x−

... this can be easily verified noting that

)cos()(),cos()(sin,cos

ϕϕϕϕϕ

jxTkxTddxx

jk ==−==

20Orthogonal functions

Chebyshev polynomials - interpolation

... we are now faced with the same problem as with the Fourier series. We want to approximate a function f(x), this time not a

periodical function but a function which is defined between [-1,1]. We are looking for gn (x)

)()(21)()(

100 xTcxTcxgxf

n

kkkn ∑

=

+=≈

... and we are faced with the problem, how we can determine the coefficients ck . Again we obtain this by finding the extremum

(minimum)

{ } 01

)()(2

1

1

2 =⎥⎦

⎤⎢⎣

⎡

−−

∂∂∫− x

dxxfxgc n

k

21Orthogonal functions

Chebyshev polynomials - interpolation

... to obtain ...

nkx

dxxTxfc kk ,...,2,1,0,1

)()(2 1

12

=−

= ∫−π

... surprisingly these coefficients can be calculated with FFT techniques, noting that

nkdkfck ,...,2,1,0,cos)(cos2

0

== ∫ ϕϕϕπ

π

... and the fact that f(cosϕ) is a 2π-periodic function ...

nkdkfck ,...,2,1,0,cos)(cos1== ∫

−

ϕϕϕπ

π

π

... which means that the coefficients ck are the Fourier coefficients ak of the periodic function F(ϕ)=f(cos ϕ)!

22Orthogonal functions

Chebyshev - discrete functions

iN

xiπcos=

... leading to the polynomial ...

in this particular case the coefficients are given by

2/,...2,1,0,)cos()(cos21

* NkkfN

cN

jjjk

== ∑=

ϕϕ

∑=

+=m

kkkm xTcTcxg

1

**0

* )(21)( 0

... what happens if we know our function f(x) only at the points

... with the property

N0,1,2,...,jj/N)cos(xat)()( j* === πxfxg m

23Orthogonal functions

Chebyshev - collocation points - |x|

f(x)=|x| => f(x) - blue ; gn (x) - red; xi - ‘+’

-1 -0 .8 -0 .6 -0 .4 -0 .2 0 0 .2 0 .4 0 .6 0 .80

0 .2

0 .4

0 .6

0 .8

1N = 8

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.80

0.2

0.4

0.6

0.8

1N = 16

8 points

16 points

24Orthogonal functions

Chebyshev - collocation points - |x|

f(x)=|x| => f(x) - blue ; gn (x) - red; xi - ‘+’

32 points

128 points

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.80

0.2

0.4

0.6

0.8

1N = 32

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.0

0.2

0.4

0.6

0.8

1N = 128

25Orthogonal functions

Chebyshev - collocation points - x2

f(x)=x2 => f(x) - blue ; gn (x) - red; xi - ‘+’

8 points

64 points

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.80

0.2

0.4

0.6

0.8

1

1.2N = 8

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1

1.2N = 64

The interpolating function gn (x) was shifted by a small amount to be visible at all!

26Orthogonal functions

Chebyshev vs. Fourier - numerical

f(x)=x2 => f(x) - blue ; gN (x) - red; xi - ‘+’

This graph speaks for itself ! Gibb’s phenomenon with Chebyshev?

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1 N = 16

0 2-5

0

5

10

15

20

25

30

35

N =

Chebyshev Fourier

27Orthogonal functions

Chebyshev vs. Fourier - Gibb’s

f(x)=sign(x-π) => f(x) - blue ; gN (x) - red; xi - ‘+’

Gibb’s phenomenon with Chebyshev? YES!

Chebyshev Fourier

-1 -0.5 0 0.5 1-1.5

-1

-0.5

0

0.5

1

1.5 N = 16

0 2-1.5

-1

-0.5

0

0.5

1

1.5N = 16

28Orthogonal functions

Chebyshev vs. Fourier - Gibb’s

f(x)=sign(x-π) => f(x) - blue ; gN (x) - red; xi - ‘+’

Chebyshev Fourier

-1 -0.5 0 0.5 1-1.5

-1

-0.5

0

0.5

1

1.5 N = 64

0 2 4-1.5

-1

-0.5

0

0.5

1

1.5N = 6

29Orthogonal functions

Fourier vs. Chebyshev

Fourier Chebyshev

iN

x iπ2

= iN

xiπcos=

periodic functions limited area [-1,1]

)sin(),cos( nxnxϕϕ

cos),cos()(

==

xnxT n

{ }

)cos(21

)sin()cos(

21)(

*

1

1

**

**0

kxa

kxbkxa

axg

m

m

k

m

kk

+

++

=

∑−

=

∑=

+=m

kkkm xTcTcxg

1

**0

* )(21)( 0

collocation points

domain

basis functions

interpolating function

30Orthogonal functions

Fourier vs. Chebyshev (cont’d)

Fourier Chebyshev

coefficients

some properties

∑

∑

=

=

=

=

N

jjj

N

jjj

kxxfN

b

kxxfN

a

k

k

1

*

1

*

)sin()(2

)cos()(2

∑=

=N

jjj kf

Nc

k1

* )cos()(cos2 ϕϕ

• Gibb’s phenomenon for discontinuous functions

• Efficient calculation via FFT

• infinite domain through periodicity

• limited area calculations

• grid densification at boundaries

• coefficients via FFT

• excellent convergence at boundaries

• Gibb’s phenomenon

31Orthogonal functions

The Fourier Transform Pair

∫

∫∞

∞−

−

∞

∞−

=

=

ωω

πω

ω

ω

deFtf

dtetfF

ti

ti

)()(

)(21)( Forward transform

Inverse transform

Note the conventions concerning the sign of the exponents and the factor.

32Orthogonal functions

Some properties of the Fourier Transform

Defining as the FT: )()( ωFtf ⇒

Linearity

Symmetry

Time shifting

Time differentiation

)()()()( 2121 ωω bFaFtbftaf +⇒+

)(2)( ωπ −⇒− Ftf

)()( ωω Fettf ti Δ⇒Δ+

)()()( ωω Fit

tf nn

n

−⇒∂

∂

33Orthogonal functions

Differentiation theorem

Time differentiation )()()( ωω Fit

tf nn

n

−⇒∂

∂

34Orthogonal functions

Convolution

∫∫∞

∞−

∞

∞−

−=−=∗ ')'()'(')'()'()()( dttgttfdtttgtftgtf

The convolution operation is at the heart of linear systems.

Definition:

Properties: )()()()( tftgtgtf ∗=∗

)()()( tfttf =∂∗

∫=∗ dttftHtf )()()(

H(t) is the Heaviside function:

35Orthogonal functions

The convolution theorem

A convolution in the time domain corresponds to a multiplication in the frequency domain.

… and vice versa …

a convolution in the frequency domain corresponds to a multiplication in the time domain

)()()()( ωω GFtgtf ⇒∗

)()()()( ωω GFtgtf ∗⇒

The first relation is of tremendous practical implication!

36Orthogonal functions

Summary

The Fourier Transform can be derived from the problem of approximating an arbitrary function.

A regular set of points allows exact interpolation (or derivation) of arbitrary functions

There are other basis functions (e.g., Chebyshev polynomials, Legendre polynomials) with similar properties

These properties are the basis for the success of the spectralelement method