Eigen-decomposition of a classofInfinite dimensional tridiagonal matrices

G.V. Moustakides: Dept. of Computer Engineering, Univ. of Patras, GreeceB. Philippe: IRISA-INRIA, France

22

Outline

Definition of the problem.

From finite to infinite dimensions.

Reduction of the problem to a finite dimensional d.e. and eigen-decomposition problem, using Fourier transform.

Numerical aspects regarding the solution of the d.e. and the computation of the final (infinite dimensional) eigenvectors.

Conclusion.

33

Definition of the problem

2

2

t

t

t

t

t

t

rr

rr

AA D I A 0

A D I AQ A D A

A D I A0 A D I A

A

I:D:A:

r:

identity matrix diagonal matrix general matrix

real scalar

real matrices of dimensions NN}

44

Eigen-decompositionEigen-decompositionEigenvalues:Eigenvalues: There is an infinite number.

Eigenvectors:Eigenvectors: There is an infinite number and each eigenvector is of infinite size.

Goal:Goal: To reduce the infinite dimensional eigen-decomposition problem into a finite one.

55

From finite to infinite dimensionst

t

t

tK

t

t

Kr

r

r

Kr

D I AA A 0

A D I AQ A D A

A D I A0 A A

A D I

QK has dimensions: (2K+1)N (2K+1)N, therefore we have (2K+1)N eigenvalue-eigenvector pairs.

Typical values: N = 100-1000, K = 5-10.

66

( ) ( ) ( )K i i ik k k QQK has dimensions: (2K+1)N (2K+1)N

i(k) has dimensions: (2K+1)N 1.

k = -K,…,K, i = 1,…,N.( , ) ( , )

( ,1) ( ,1)( )( ,0) ( ,0)

( , 1) ( ,1)

( , ) ( , )

i it

i i

ii it

i i

i i

k K k KKr

k kkk k

k kKr

k K k K

D I A 0A

DA

0 A D I

i(k,l) has dimensions: N 1. k,l= -K,…,K, i=1,…,N.

77

( 1) ( , 1)( , )

( 1) ( , 1)

( , 1)( ) ( , )

( , 1)

ti

ti

ti

i

i i

i

l r k llr k l

l r k l

k lk k l

k l

A D I A 00 A D I A 0

0 A D I A

Consider now the infinite dimensional problem by letting K

Ai (k,l+1) + (D+lrI)i (k,l) + Ati (k,l-1) = i(k)i (k,l)

Ai (k,l+1) + Di (k,l) + Ati (k,l-1) = (i(k) -lr)i (k,l)

88

Reduction to finite dimensionsReduction to finite dimensionsAi(k,l+1)+Di(k,l)+Ati(k,l-1) = (i(k)-lr)i(k,l)

A,D: NNi(k,l): N1i=1,…,N, k,l= -,…,

Key IdeaKey Ideai(k) = i + kr without loss of generality assume 0 i ri(k,l) = i(l-k)

Ai(l-k+1)+Di(l-k)+Ati(l-k-1) = (i-(l-k)r)i(l-k)

Ai(n+1)+(D-iI)i(n)+Ati(n-1) = -nri(n)

99

Ai(n+1)+(D-iI)i(n)+Ati(n-1) = -nri(n)

(2)(1)

, ;(0)( 1)( 2)

i

i

i i

i

i

i , {i(n)}, i=1,…,N, 0 i r

(1) (0)(0) ( 1)

( ) , ; ( 2 ) , ;( 1) ( 2)( 2) ( 3)( 3) ( 4)

i i

i i

i ii i

i i

i i

r r

1010

Fourier TransformFourier Transform

( ) ( ) jn

n

X x n e

Let …, x(-2), x(-1), x(0), x(1), x(2),… be a real sequence. Then we define its Fourier Transform as

ImportanImportantt

( )( ) jn

n

dXnx n e jd

( 2 ) ( )X X

( ) ( )jn jk

n

x n k e e X

1111

Ai(n+1)+(D-iI)i(n)+Ati(n-1) = -rni(n)

( ) ( ) jni i

n

n e

( )( ) ( ) ( ) ( )j t j ii i i i

de e jrd

A D I A

1( ) ( )j t jii i

d jr e ed

D A A I

1212

1( ) ( )j t jii i

d jr e ed

D A A I

1( ) ( ), (0)j t jd jr e ed

Ψ D A A Ψ Ψ I

1

( ) ( ) (0)ijri ie

Ψ

i() as being the Fourier transform of a (vector) sequence isnecessarily periodic with period 2.

We need i and i(0) to solve it.

1313

TheoremTheorem

0( ) ( ) ( ), (0)

( 2 ) ( ).

dX X X Xd

B

B B

Consider the following linear system of d.e.

Let Z() be the transition matrix of the d.e., that is

( ) ( ) ( ), (0)dd

Z B Z Z I

then we know that X()= Z()X0.

0 0(2 )X X ZThe solution X() is periodic if and only if X(2)=X(0)

1414

1 2(2 ) (0) (2 ) (0) (0)ijri i ie

Z Ψ

1 2(2 ) (0) (0)ijri ie

Ψ

1( ) ( )j t jii i

d jr e ed

D A A I

1( ) ( ), (0)j t jd jr e ed

Ψ D A A Ψ Ψ I

1

( ) ( )ijre Z Ψ

1515

Steps to obtain Steps to obtain ((i i ,{,{i i ((nn)}), )}), i=i=1,…,1,…,NN

Compute the transition matrix () from the d.e.

1( ) ( ), (0)j t jd jr e ed

Ψ D A A Ψ Ψ I

Find the eigenvalue-eigenvector pairs i, i(0) of

(2 ) (0) (0), 1, ,i i i i N Ψ

Form the desired eigenvalue-FT(eigenvector) pairs as

1

, ( ) ( ) (0)2

ijrii i ir e

Ψ

Use Inverse Fourier Transform to recover the final infinite eigenvector {i(n)} from i().

1616

Numerical aspectsNumerical aspects Numerical solution of the d.e.

1( ) ( ), (0)j t jd jr e ed

Ψ D A A Ψ Ψ I

Eigen-decomposition of (2).

Computation of the Inverse Fourier Transform of i() where

1

( ) ( ) (0)ijri ie

Ψ

1717

Numerical solution of the d.e.Numerical solution of the d.e.

1( ) ( ), (0)j t jd jr e ed

Ψ D A A Ψ Ψ I

( ) ( ) ( ), (0) , ( ) Hermitiand jd

Ψ B Ψ Ψ I B

One can show that () is unitary, therefore any numerical solution should respect this structure. A possible scheme is

1

( / 2)

( ) ( , ) ( ), (0)

( , )n n n

je

B

Ψ M Ψ Ψ I

M

1818

( / 2)( , ) je BM

1

3 3

( ) (1 ) ,

, (1 2 )( , ) ( )

1 2 2 2 1.35123

n n

n

n n

Ψ M

MM Ψ

3 Step Integration. Yoshida scheme

1( ) ( , ) ( ), (0)n n n Ψ M Ψ Ψ I 1 Step Integration

1

1

12 2

2

1 1( )2 2

1 1 1 1( )2 12 2 12

e P

P

X X I X I X

X I X X I X X

Pade 1

Pade 2

1919

Pade 1, 1 step intgr.

Pade 2, 1 step intgr.

Pade 2, 3 step intgr.

Pade 1, 3 step intgr.

2020

Eigen-decomposition of Eigen-decomposition of (2(2))

Since (2) is unitary there are special eigen-decomposition algorithms that require lower computational complexity than the corresponding algorithm for the general case.

1

( ) ( ) (0)i njri n n ie

Ψ

From this problem we obtain the pairs i, i(0), i=1,…,N.

Using the solution () of the differential equation we can compute the Discrete Fourier Transform of the eigenvectors

Notice that we obtain a sampled version of the required Fourier transform.

2121

Inverting the Fourier TransformInverting the Fourier Transform

( ) ( ) jn

n

X x n e

Let …, x(-2), x(-1), x(0), x(1), x(2),… with Fourier Transform

If x(n)=0 for n < 0 and n M, then the Fourier Transform is equal

1

0

( ) ( )M

jn

n

X x n e

then the finite sequence x(n), n =0,…, M-1, can be completely recovered from a sampled version of the Fourier transform. Specifically we need only the samples

2 , 0,..., 1X n n MM

2222

21

0

1 2( ) , 0,..., 1M jnk

M

k

x n X k e n MM M

Complexity O(M 2).

For M=2m popular Fast Fourier Transform (FFT).Complexity O(M log(M)).

Apply Inverse Discrete Fourier Transform to i(n), this will yield the desired vectors i(n).

If only a small number of i(n) is significant, then we apply Inverse Discrete Fourier Transform only to a subset of the vectors i(n) produced by the solution of the d.e.

Inverce discrete Fourier Inverce discrete Fourier TransformTransform

2323

ConclusionConclusion

We have presented as special infinite dimensional eigen- decomposition problem.

With the help of the Fourier Transform this problem was transformed into a d.e. followed by an eigen-decomposition both of finite size.

We presented numerical techniques that efficiently solve all subproblems of the proposed solution.

2424

E n D

Questions please ?Questions please ?