CHAPTER 10

Fast Direct Solution of a Large Linear System

Exercise 10.5: Fourier matrix

The Fourier matrix FN has entries

(FN)j,k = !(j�1)(k�1)N , !N := e�

2⇡N i = cos

✓2⇡

N

◆� i sin

✓2⇡

N

◆.

In particular for N = 4, this implies that !4 = �i and

F4 =

2

664

1 1 1 11 �i �1 i1 �1 1 �11 i �1 �i

3

775 .

Computing the transpose and Hermitian transpose gives

FT4 =

2

664

1 1 1 11 �i �1 i1 �1 1 �11 i �1 �i

3

775 = F4, FH4 =

2

664

1 1 1 11 i �1 �i1 �1 1 �11 �i �1 i

3

775 6= F4,

which is what needed to be shown.

Exercise 10.6: Sine transform as Fourier transform

According to Lemma 10.2, the Discrete Sine Transform can be computed from theDiscrete Fourier Transform by (Smx)k =

i2(F2m+2z)k+1, where

z = [0, x1, . . . , xm, 0,�xm, . . . ,�x1]T.

For m = 1 this means that

z = [0, x1, 0,�x1]T and (S1x)1 =

i

2(F4z)2.

Since h = 1m+1

= 12for m = 1, computing the DST directly gives

(S1x)1 = sin(⇡h)x1 = sin⇣⇡2

⌘x1 = x1,

while computing the Fourier transform gives

F4z =

2

664

1 1 1 11 �i �1 i1 �1 1 �11 i �1 �i

3

775

2

664

0x1

0�x1

3

775 =

2

664

0�2ix1

02ix1

3

775 = �2i

2

664

0x1

0�x1

3

775 = �2iz.

Multiplying the Fourier transform with i2, one finds i

2F4z = z, so that i

2(F4z)2 = x1 =

(S1x)1, which is what we needed to show.

64

Exercise 10.7: Explicit solution of the discrete Poisson equation

For any integer m � 1, let h = 1/(m + 1). For j = 1, . . . ,m, let �j = 4 sin2�j⇡h/2

�,

D = diag(�1, . . . ,�m), and S = (sjk)jk =�sin(jk⇡h)

�jk. By Section 10.2, the solution

to the discrete Poisson equation is V = SXS, where X is found by solving DX+XD =4h4SFS. Since D is diagonal, one has (DX+XD)pr = (�p + �r)xpr, so that

xpr = 4h4 (SFS)pr�p + �r

= 4h4

mX

k=1

mX

l=1

spkfklslr�p + �r

so that

vij =mX

p=1

mX

r=1

sipxprsrj = 4h4

mX

p=1

mX

r=1

mX

k=1

mX

l=1

sipspkslrsrj�p + �r

fkl

= h4

mX

p=1

mX

r=1

mX

k=1

mX

l=1

sin�

ip⇡m+1

�sin

�pk⇡m+1

�sin

�lr⇡m+1

�sin

�rj⇡m+1

�

sin2⇣

p⇡2(m+1)

⌘+ sin2

⇣r⇡

2(m+1)

⌘ fkl,

which is what needed to be shown.

Exercise 10.8: Improved version of Algorithm 10.1

Given is that

(?) TV +VT = h2F.

Let T = SDS�1 be the orthogonal diagonalization of T from Equation (10.4), andwrite X = VS and C = h2FS.

(a) Multiplying Equation (?) from the right by S, one obtains

TX+XD = TVS+VSD = TVS+VTS = h2FS = C.

(b) Writing C = [c1, . . . , cm], X = [x1, . . . ,xm] and applying the rules of blockmultiplication, we find

[c1, . . . , cm] = C

= TX+XD

= T[x1, . . . ,xm] +X[�1e1, . . . ,�mem]

= [Tx1 + �1Xe1, . . . ,Txm + �mXem]

= [Tx1 + �1x1, . . . ,Txm + �mxm]

= [(T+ �1I)x1, . . . , (T+ �mI)xm],

which is equivalent to System (10.9). To find X, we therefore need to solve the mtridiagonal linear systems of (10.9). Since the eigenvalues �1, . . . ,�m are positive,each matrix T + �jI is diagonally dominant. By Theorem 1.24, every such matrixis nonsingular and has a unique LU factorization. Algorithms 1.8 and 1.9 then solvethe corresponding system (T + �jI)xj = cj in O(�m) operations for some constant �.Doing this for all m columns x1, . . . ,xm, one finds the matrix X in O(�m2) operations.

(c) To find V, we first find C = h2FS by performing O(2m3) operations. Next wefind X as in step b) by performing O(�m2) operations. Finally we compute V = 2hXSby performing O(2m3) operations. In total, this amounts to O(4m3) operations.

65

(d) As explained in Section 10.3, multiplying by the matrix S can be done inO(2m2 log2 m) operations by using the Fourier transform. The two matrix multiplica-tions in c) can therefore be carried out in

O(4�m2 log2 m) = O(4�n log2 n1/2) = O(2�n log2 n)

operations.

Exercise 10.9: Fast solution of 9 point scheme

Analogously to Section 10.2, we use the relations between the matrices T,S,X,D torewrite Equation (9.18).

TV +VT� 1

6TVT = h2µF

() TSXS+ SXST� 1

6TSXST = h2µF

() STSXS2 + S2XSTS� 1

6STSXSTS = h2µSFS

() S2DXS2 + S2XS2D� 1

6S2DXS2D = h2µSFS

() DX+XD� 1

6DXD = 4h4µSFS = 4h4G

Writing D = diag(�1, . . . ,�m), the (j, k)-th entry of DX + XD � 16DXD is equal to

�jxjk + xjk�k � 16�jxjk�k. Isolating xjk and writing �j = 4�j = 4 sin2(j⇡h/2) then

yields

xjk =4h4gjk

�j + �k � 16�j�k

=h4gjk

�j + �k � 23�j�k

, �j = sin2

✓j⇡h

2

◆.

Defining ↵ := j⇡h/2 and � = k⇡h/2, one has 0 < ↵, � < ⇡/2. Note that

�j + �k � 2

3�j�k > �j + �k � �j�k

= 2� cos2 ↵� cos2 � � (1� cos2 ↵)(1� cos2 �)

= 1� cos2 ↵ cos2 �

� 1� cos2 �

� 0.

Let A = T ⌦ I + I ⌦ T � 16T ⌦ T be as in Exercise 9.13.(b) and si as in Section

10.2. Applying the mixed-product rule, one obtains

A(si ⌦ sj) = (T⌦ I+ I⌦T)(si ⌦ sj)� 1

6(T⌦T)(si ⌦ sj) =

(�i + �j)(si ⌦ sj)� 1

6�i�j(si ⌦ sj) = (�i + �j � 1

6�i�j)(si ⌦ sj).

The matrix A therefore has eigenvectors si ⌦ sj, and counting them shows that thesemust be all of them. As shown above, the corresponding eigen values �i + �j � 1

6�i�j

are positive, implying that the matrix A is positive definite. It follows that the System(9.17) always has a (unique) solution.

66

Exercise 10.10: Algorithm for fast solution of 9 point scheme

The following describes an algorithm for solving System (9.17).

Algorithm 1 A method for solving the discrete Poisson problem (9.17)

Require: An integer m denoting the grid size, a matrix µF 2 Rm,m of function values.Ensure: The solution V to the discrete Poisson problem (9.17).1: h 1

m+1

2: S �sin(jk⇡h)

�mj,k=1

3: � �sin2

�j⇡h2

��mj=1

4: G SµFS

5: X ⇣

h4gi,j�i+�j� 2

3�i�j

⌘m

j,k=1

6: V SXS

For the individual steps in this algorithm, the time complexities are shown in thefollowing table.

step 1 2 3 4 5 6

complexity O(1) O(m2) O(m) O(m3) O(m2) O(m3)

Hence the overall complexity is determined by the four matrix multiplications andgiven by O(m3).

Exercise 10.11: Fast solution of biharmonic equation

From Exercise 9.14 we know that T 2 Rm⇥m is the second derivative matrix. Accordingto Lemma 1.31, the eigenpairs (�j, sj), with j = 1, . . . ,m, of T are given by

sj = [sin(j⇡h), sin(2j⇡h), . . . , sin(mj⇡h)]T,

�j = 2� 2 cos(j⇡h) = 4 sin2(j⇡h/2),

and satisfy sTj sk = �j,k/(2h) for all j, k, where h := 1/(m + 1). Using, in order, thatU = SXS, TS = SD, and S2 = I/(2h), one finds that

h4F = T2U+ 2TUT+UT2

() h4F = T2SXS+ 2TSXST+ SXST2

() h4SFS = ST2SXS2 + 2STSXSTS+ S2XST2S

() h4SFS = S2D2XS2 + 2S2DXS2D+ S2XS2D2

() h4SFS = ID2XI/(4h2) + 2IDXID/(4h2) + IXID2/(4h2)

() 4h6G = D2X+ 2DXD+XD2,

where G := SFS. The (j, k)-th entry of the latter matrix equation is

4h6gjk = �2jxjk + 2�jxjk�k + xjk�

2k = xjk(�j + �k)

2.

Writing �j := sin2(j⇡h/2) = �j/4, one obtains

xjk =4h6gjk

(�j + �k)2=

4h6gjk�4 sin2(j⇡h/2) + 4 sin2(k⇡h/2)

�2 =h6gjk

4(�j + �k)2.

67

Exercise 10.12: Algorithm for fast solution of biharmonic equation

In order to derive an algorithm that computes U in Problem 9.14, we can adjustAlgorithm 10.1 by replacing the computation of the matrix X by the formula fromExercise 10.11. This adjustment does not change the complexity of Algorithm 10.1,which therefore remains O(�n3/2). The new algorithm can be implemented inMatlab asin Listing 10.1.

function U = simplefastbiharmonic(F)

m = length(F);

h = 1/(m+1);

hv = pi

*

h

*

(1:m)’;

sigma = sin(hv/2). 2;

S = sin(hv

*

(1:m));

G = S

*

F

*

S;

X = (h 6)

*

G./(4

*

(sigma

*

ones(1,m)+ones(m,1)

*

sigma’). 2);

U = zeros(m+2,m+2);

U(2:m+1,2:m+1) = S

*

X

*

S;

end

Listing 10.1. A simple fast solution to the biharmonic equation

Exercise 10.13: Check algorithm for fast solution of biharmonic equation

The Matlab function from Listing 10.2 directly solves the standard form Ax = bof Equation (9.21), making sure to return a matrix of the same dimension as theimplementation from Listing 10.1.

function V = standardbiharmonic(F)

m = length(F);

h = 1/(m+1);

T = gallery(’tridiag’, m, -1, 2,-1);

A = kron(T 2, eye(m)) + 2

*

kron(T,T) + kron(eye(m),T 2);

b = h. 4

*

F(:);

x = A\b;

V = zeros(m+2, m+2);

V(2:m+1,2:m+1) = reshape(x,m,m);

end

Listing 10.2. A direct solution to the biharmonic equation

After specifying m = 4 by issuing the command F = ones(4,4), the com-mands simplefastbiharmonic(F) and standardbiharmonic(F) both returnthe matrix

2

666664

0 0 0 0 0 00 0.0015 0.0024 0.0024 0.0015 00 0.0024 0.0037 0.0037 0.0024 00 0.0024 0.0037 0.0037 0.0024 00 0.0015 0.0024 0.0024 0.0015 00 0 0 0 0 0

3

777775.

68

For large m, it is more insightful to plot the data returned by our Matlab functions.For m = 50, we solve and plot our system with the commands in Listing 10.3.

F = ones(50, 50);

U = simplefastbiharmonic(F);

V = standardbiharmonic(F);

surf(U);

surf(V);

Listing 10.3. Solving the biharmonic equation and plotting the result

0

20

40

60

0

20

40

600

1

2

3

4

5

x 10−3

simplefastbiharmonic

0

20

40

60

0

20

40

60

0

1

2

3

4

5

x 10−3

standardbiharmonic

On the face of it, these plots seem to be virtually identical. But exactly how close arethey? We investigate this by plotting the di↵erence with the command surf(U-V),which gives

0

20

40

60

0

20

40

600

0.5

1

1.5

2

x 10−14

simplefastbiharmonic minus standardbiharmonic

We conclude that their maximal di↵erence is of the order of 10�14, which makes themindeed very similar.

69