EIGENVALUES & EIGENVECTORS

We say λ is an eigenvalue of a square matrix A if

Ax = λx

for some x �= 0. The vector x is called an eigenvector

of A, associated with the eigenvalue λ. Note that if

x is an eigenvector, then any multiple αx is also an

eigenvector.

EXAMPLES: −7 13 −16

13 −10 13−16 13 −7

1−11

= −36

1−11

1 0 −1

1 −1 0−1 1 0

1

11

= 0

1

11

Knowing the eigenvalues and eigenvectors of a matrix

A will often give insight as to what is happening when

solving systems or problems involving A.

THE CHARACTERISTIC POLYNOMIAL

For an n × n matrix A, solving Ax = λx for a vector

x �= 0 is equivalent to solving the homogeneous linear

system

(A − λI)x = 0

This has a nonzero solution if and only if

det (A − λI) = 0

fA(λ) ≡ det

a1,1 − λ · · · a1,n

... . . . ...an,1 · · · an,n − λ

= 0

We can expand this determinant by minors, obtaining

fA(λ) =(a1,1 − λ

)· · · (an,n − λ)

+ terms of degree ≤ n − 2

= (−1)nλn

+(−1)n−1(a1,1 + · · ·+ an,n

)λn−1

+ terms of degree ≤ n − 2

We call fA(λ) the characteristic polynomial of A; and

fA(λ) = 0

is called the characteristic equation for A. Since fA(λ)

is a polynomial of degree n:

1. The matrix A has at least one eigenvalue.

2. A has at most n distinct eigenvalues.

The multiplicity of λ as a root of fA(λ) = 0 is called

the algebraic multiplicity of λ. The number of inde-

pendent eigenvectors associated with λ is called the

geometric multiplicity of λ.

EXAMPLES : [1 00 1

]

has the eigenvalue λ = 1 with both the algebraic and

geometric multiplicity equal to 2.[1 10 1

]

has the eigenvalue λ = 1 with algebraic multiplicity 2

and geometric multiplicity 1.

PROPERTIES

Let A and B be similar, meaning that for some non-

singular matrix P ,

B = P−1AP

Being similar means that A and B represent the same

linear transformation of the vector space Rn or Cn,

but with respect to different bases. Then

fB(λ) = det (B − λI)

= det(P−1AP − λI

)= det

(P−1 [A − λI]P

)=

[detP−1

]det (A − λI) [detP ]

= det (A − λI) = fA(λ)

since detP−1 detP = det(P−1P

)= 1. This says

that the similar matrices have the same eigenvalues.

For the eigenvectors, write

Ax = λx(P−1AP

)P−1x = λP−1x

Bz = λz with z = P−1x

Thus there is a simple connection with the eigenvec-

tors of similar matrices.

Since fB(λ) = fA(λ) for similar matrices A and B, we

have that their coefficients are invariant. In particular,

note that

fA(0) = detA

which is the constant term in fA(λ). Thus similar

matrices have the same determinant. In particular,

introduce

traceA = a1,1 + · · ·+ an,n

It is the coefficient of λn−1, except for sign. Similar

matrices have the same trace and determinant.

Let λ1, ..., λn be the eigenvalues of A, repeated ac-

cording to their multiplicity. Then

fA(λ) = (λ1 − λ) · · · (λn − λ)

= (−1)nλn + (−1)n−1 (λ1 + · · ·λn)λn−1

+ · · ·+ (λ1 · · ·λn)

Thus

traceA = λ1 + · · ·λn, detA = λ1 · · ·λn

EXAMPLE : The eigenvalues of

A =

[1 23 4

]

satisfy

λ1 + λ2 = 5, λ1λ2 = −2

CANONICAL FORMS

By use of similarity transformations, we change a ma-

trix A into a similar matrix which is “simpler” in its

form. These are often called canonical forms; and

there is a large literature on such forms. The ones

most used in numerical analysis are:

The Schur normal form

The principal axes form

The Jordan form

The singular value decomposition

The Schur normal form is less well-known, but is a

powerful tool in deriving results in matrix algebra.

SCHUR’S NORMAL FORM. Let A be a square ma-

trix of order n with coefficients from C. Then there

is a nonsingular unitary matrix U for which

U∗AU = T, U∗U = I

with T an upper triangular matrix. Since U is unitary,

U∗ = U−1, and thus T and A are similar.

A proof by induction on n is given in the text, on page

474.

Note that for a triangular matrix T , the characteristic

polynomial is

fT (λ) = det

t1,1 − λ t1,2 · · · t1,n0 . . .... . . . ...0 · · · 0 tn,n − λ

=(t1,1 − λ

)· · · (tn,n − λ)

Thus the diagonal elements{ti,i

}are the eigenvalues

of T .

PRINCIPAL AXES THEOREM. Let A be Hermitian

(or self-adjoint), meaning

A∗ = A

Then there is a nonsingular unitary matrix U for which

U∗AU = D

a diagonal matrix with only real entries. If the matrix

A is real (and thus symmetric), the matrix U can be

chosen as a real orthogonal matrix.

PROOF: Using the Schur normal form,

U∗AU = T

Now form the conjugate transpose of both sides

T ∗ = (U∗AU)∗= U∗A∗ (U∗)∗= U∗A∗U since (B∗)∗ = B in general= U∗AU since A∗ = A= T

Since T ∗ =(T

)T, the diagonal elements of T are

real. Moreover, T = T ∗ implies T must be a diagonal

matrix, as asserted, and we write

D = diag [λ1, ..., λn]

I will not show here that A real implies U real.

Consequences: Again, assume A (and U) are n × n;

and let V = Rn or Cn, depending on whether A is

real or complex. Write

U = [u1, ..., un]

with u1, ..., un the orthogonal columns of U . Since

these are elements of the n-dimensional space V,

{u1, ..., un} is an orthogonal basis of V. In addition,

re-write U∗AU = D as

AU = UD

A [u1, ..., un] = [u1, ..., un]

λ1 0 · · · 00 · ...... · 00 · · · 0 λn

Matching corresponding columns on the two sides of

this equation, we have

Auj = λjuj, j = 1, ..., n

Thus the columns u1, ..., un are orthogonal eigenvec-

tors of A; and they form a basis for V.

For a Hermitian matrix A, the eigenvalues are all real;

and there is an orthogonal basis for the associated

vector space V consisting of eigenvectors of A.

In dealing with such a matrix A in a problem, the basis

{u1, ..., un} is often used in place of the original basis

(or coordinate system) used in setting up the problem.

With this basis, the use of A is clear:

A (c1u1 + · · ·+ cnun) = c1Au1 + · · ·+ cnAun

= c1λ1u1 + · · ·+ cnλnun

SINGULAR VALUE DECOMPOSITION. This is a canon-

ical form for general rectangular matrices. Let A have

order n × m. Then there are unitary matrices U and

V , of respective orders m × m and n × n, for which

V ∗AU = F

with F of order n × m and of the form

F =

µ1 0 · · ·0 µ2 0 · · ·... . . .

µr

0. . .

The numbers µi are all real, with

µ1 ≥ µ2 ≥ · · · ≥ µr > 0

This is proven in the text (p. 478).

JORDAN CANONICAL FORM

This is probably the best known canonical form of lin-

ear algebra textbooks. Begin by introducing matrices

Jm(λ) =

λ 1 0 · · ·0 λ . . . . . .... . . . . . . 10 · · · 0 λ

The order is m×m, and all elements are zero except

for λ on the diagonal and 1 on the superdiagonal. We

refer to this matrix as a Jordan block.

Let A have order n, with elements chosen from C.

Then there is a nonsingular matrix P for which

P−1AP =

Jn1(λ1) 0 · · · 00 Jn2(λ2)

. . . ...... . . . . . . 00 · · · 0 Jnr(λr)

The numbers λ1, ..., λr are the eigenvalues of A, and

they need not be distinct.

Note that the Jordan block Jm(λ) satisfies

Jm(0)m = 0

Such a matrix is called nilpotent. We sometimes write

the Jordan canonical form as

P−1AP = D + N

with D containing the diagonal elements of P−1AP

and N containing the remaining nonzero elements. In

this case, N is nilpotent with

Nq = 0

with q = max {n1, ..., nr} ≤ n.

The proof of the Jordan canonical form is very com-

plicated, and it can be found in more advanced level

books on linear algebra.