Week 11

Section 5.5: Similarity & Diagonalization Revisited

Definition:

If A and B are n n matrices, then we say A and B are similar, denoted A B if B = P1AP .Note: A is diagonalizable iff A is similar to a diagonal matrix D since D = P1AP .

Definition:

If A is an n n matrix, then the trace of A denoted tr(A) is the sum of the entries along the maindiagonal of A.ie. If aij denotes the (i, j)-entry of A, then tr(A) = a11 + a22 + . . .+ ann.

Lemma:

If A and B are n n matrices, then tr(AB) = tr(BA).

Proof:

Let aij and bij denote the (i, j)-entries of A and B, respectively.

tr(AB) = a11b11 + a12b21 + . . .+ a1nbn1

+ a21b12 + a22b22 + . . .+ a2nbn2

+ . . .

+ an1b1n + an2b2n + . . .+ annbnn

tr(BA) = b11a11 + b12a21 + . . .+ b1nan1

+ b21a12 + b22a22 + . . .+ b2nan2

+ . . .

+ bn1a1n + bn2a2n + . . .+ bnnann

Comparing the terms in the first column sum of tr(AB) and the first row sum of tr(BA) we have thesame entries. Similarly, the second column sum of tr(AB) is equal to the second row sum of tr(BA)and so on.

Lemma:

If A is an n n matrix and B and C are invertible n n matrices, then Col(AB) = Col(A) andRow(CA) = Row(A) and thus Rank(AB) = Rank(A) = Rank(CA).

Proof:

Denote the columns of A by ~c1,~c2, . . . ,~cn and let ~x be the jth column of B.

Then column j of AB is A~x = x1~c1 + x2~c2 + . . .+ xn~cn Col(A).Therefore Col(AB) Col(A).

1

Also, Col(A) = Col(ABB1) = Col((AB)B1) Col(AB).Thus Col(A) = Col(AB).

Row(CA) = Col((CA)T ) = Col(ATCT ) Col(AT ) = Row(A).Also, Row(A) = Row(C1CA) = Row(C1(CA)) Row(CA).Thus, Row(CA) = Row(A).

This tells us that Rank(AB) = Rank(A) and Rank(CA) = Rank(A).

Theorem

If A and B are similar, then A and B have the same determinant, rank, trace, characteristic polyno-mial and eigenvalues.

Proof

Let B = P1AP .Then |B| = |P1AP | = 1|P | |A||P | = |A|.Rank(B) = Rank(P1AP ) = Rank(P1(AP )) = Rank(P1A) = Rank(A), by the Lemma above.tr(B) = tr(P1AP ) = tr(P1(AP )) = tr((AP )P1) = tr(AI) = tr(A).The characteristic polynomial for B is

cB() = det(I B)= det(I P1AP )= det(P1IP P1AP )= det(P1(IP AP ))= det(P1(I A)P )= det(P1) det(I A) det(P )=

1

det(P )det(I A) det(P )

= det(I A)= cA()

Since A and B have the same characteristic polynomial they must have the same solutions to thisequation, hence they have the same eigenvalues .

Recall: Eigenvectors ~x of an n n matrix A corresponding to an eigenvalue are the non-trivialsolutions to (I A)~x = ~0. If we include the trivial solution, we will have a subspace called theeigenspace of A corresponding to , denoted E. We can see that we have a subspace since it issimply the Null Space of the matrix (I A).Recall that A is diagonalizable iff there exists an invertible matrix P such that P1AP = D, adiagonal matrix. We found P by placing the basic eigenvectors as columns of P . In order for P tobe invertible, these vectors we now know must be linearly independent and in fact form a basis forRn. This gives us the following result:

An n n matrix A is diagonalizable iff A has n linearly independent eigenvectors. Furthermore,these eigenvectors form a basis for Rn.

2

We know that the basic eigenvectors corresponding to a particular eigenvalue will be independent.But what about the eigenvectors corresponding to different eigenvalues?

Theorem:

Let {~v1, ~v2, . . . , ~vk} be eigenvectors corresponding to distinct eigenvalues 1, 2, . . . , k of an n nmatrix A. Then {~v1, ~v2, . . . , ~vk} is linearly independent.

Proof:

We will prove this indirectly or by contradiction.Assume the set {~v1, ~v2, . . . , ~vk} is linearly dependent.Then at least one of these vectors can be written as a linear combination of the others.Let ~vp+1 be the first vector in the set that makes the set linearly dependent.That is, in the set {~v1, ~v2, . . . , ~vp, ~vp+1, . . . , ~vk}, the first p vectors are independent, and ~vp+1 is in thespan{~v1, ~v2, . . . , ~vp}.Thus,

~vp+1 = c1~v1 + c2~v2 + . . .+ cp~vp (1)

Multiplying both sides of this equation by A, we have

A~vp+1 = Ac1~v1 + Ac2~v2 + . . .+ Acp~vp

Using our definition of eigenvectors and eigenvalues of A, we can replace A~vi by i~vi.

p+1~vp+1 = c11~v1 + c22~v2 + . . .+ cpp~vp (2)

Multiplying (1) by p+1 and subtracting from (2), we have

~0 = c1(1 p+1)~v1 + c2(2 p+1)~v2 + . . .+ cp(p p+1)~vpBut since {~v1, ~v2, . . . , ~vp} is linearly independent, we must have ci(ip+1) = 0 for all i = 1, 2, . . . , p.But i p+1 6= 0 since the is are distinct.Thus ci = 0 for all i = 1, 2, . . . p.Substituting this into equation (1), we have

~vp+1 = 0~v1 + 0~v2 + . . .+ 0~vp

= ~0

This is a contradiction since by definition, eigenvectors are non-zero vectors.Thus, our assumption was incorrect and {~v1, ~v2 . . . , ~vk} is linearly independent if they correspond todistinct eigenvalues.

Corollary

If A is an n n matrix with n distinct eigenvalues, then A is diagonalizable.Why? We can choose 1 eigenvector from each of the n distinct eigenvalues. This will form anindependent set of vectors which we can then place in an invertible matrix P .We could not end up with more than 1 independent eigenvector from each since this would give usmore than n independent vectors in Rn which is a contradiction of the Fundamental Theorem.

3

Example

Suppose A =

5 6 164 1 84 4 11

and B = 2 1 12 1 21 0 2

. Which matrix is diagonalizable?Solution:

4

8.2 Orthogonal Diagonalization

An n n matrix A is diagonalizable iff the eigenvectors of A form a basis for Rn.An n n matrix A is orthogonally diagonalizable if the eigenvectors of A form an orthonormalbasis for Rn. That is, there exists an invertible matrix Q such that A = QDQ1, where the columnsof Q are an orthonormal basis for Rn.

Definition:

An nn matrix Q is orthogonal if Q is invertible and Q1 = QT . In other words, QQT = I = QTQ.

Theorem:

The columns of an n n orthogonal matrix Q form an orthonormal basis for Rn.

Proof:

Example:

Verify that Q =

3/7 2/7 6/76/7 3/7 2/72/7 6/7 3/7

is an orthogonal matrix.

5

If we can write A = QDQ1, where Q is orthogonal, then the columns of Q form an orthonormalbasis for Rn. (Place eigenvectors in Q like as usual).

Definition:

An n n matrix A is orthogonally diagonalizable if there exists an orthogonal matrix Q suchthat A = QDQ1 or A = QDQT , where D is a diagonal matrix containing the eigenvalues of A.

Theorem:

A is orthogonally diagonalizable if and only if A is symmetric.

Proof:

We will prove this theorem in one direction.Assume A is orthogonally diagonalizable.Then we can write A = QDQT .Then AT = (QDQT )T = (QT )TDTQT = QDQT = A.Thus, A is symmetric.

In fact, this theorem holds in the other direction as well. That is, if A is symmetric, then it isorthogonally diagonalizable. This direction is shown in the text.

Theorem:

If A is a symmetric nn matrix then the eigenvectors corresponding to distinct eigenvalues of A areorthogonal.

Proof:

First we note the following: If ~x and ~y are column vectors, then ~x ~y = ~xT~y.Let ~v1 and ~v2 be eigenvectors corresponding to distinct eigenvalues 1 and 2, respectively.Suppose A~v1 = 1~v1 and A~v2 = 2~v2, where 1 6= 2.A~v1 ~v2 = 1~v1 ~v2 = 1(~v1 ~v2)

A~v1 ~v2 = (A~v1)T~v2= ~vT1 A

T~v2

= ~vT1 A~v2

= ~v1 A~v2= ~v1 2~v2= 2(~v1 ~v2)

Setting these two expressions for A~v1 ~v2 equal to each other, we have 1(~v1 ~v2) = 2(~v1 ~v2) = 0 or(12)(~v1 ~v2) = 0, and since 1 6= 2, we have ~v1 ~v2 = 0 and so these eigenvectors are orthogonal.Thus, if A is symmetric and has n distinct eigenvalues, the set of n corresponding eigenvectors willbe orthogonal.

6

How to Orthogonally Diagonalize a Symmetric Matrix A

1. Find the eigenvalues of A from the characteristic equation. (det(A I) = 0.)2. Find the basic eigenvectors corresponding to each eigenvalue of A.

If A has n distinct eigenvalues, then the n basic eigenvectors will form an orthogonal basisfor Rn.

If some eigenvalue has multiplicity k > 1, then E = span{~v1, ~v2, . . . , ~vk}, where{~v1, ~v2, . . . , ~vk} forms a regular basis for E. Convert this basis to an orthogonal basisusing the Gram-Schmidt Algorithm.

3. Normalize each orthogonal eigenvector and place in Q.

Example:

Orthogonally diagonalize A =

8 2 22 5 42 4 5

Solution:

7