eigen values, eigen vectors_afzaal_1.pdf

7/27/2019 Eigen Values, Eigen Vectors_afzaal_1.pdf

1/21

Linear Algebra

Eigenvalues, Eigenvectors

2

Eigenvalues and Eigenvectors

If A is an n x n matrix and x is a vector in , then

there is usually no general geometric relationship

between the vector x and the vector Ax. However,

there are often certain nonzero vectors x such that x

and Ax are scalar multiples of one another. Such

vectors arise naturally in the study of vibrations,

electrical systems, chemical reactions, quantum

mechanics, mechanical stress, economics and

geometry.

nR

3


IfA is an n n, then a non zero vectorx in Rn

is called an eigenvectorofA ifAx is a scalarmultiple ofx; that is

for some scalar . The scalar is called aneigenvalue of A, and x is called theeigenvector ofA corresponding to .

Eigen values are also called the propervalues or characteristic values they arealso called the Latent roots.

x x

4


The set of the eigenvalues is called the

spectrum ofA.

The largest of the absolute values of the

eigenvalue ofA is called the spectral radius

ofA.

The set of all eigenvectors corresponding to

an eigenvalue of A, together with 0,forms a

vector space called the eigenspace of A

corresponding to this eigenvalue.
http://www.pdfonline.com/easypdf/?gad=CLjUiqcCEgjbNejkqKEugRjG27j-AyCw_-AP


2/21

5

Graphical Interpretation

Eigen values and Eigen vectors have also a useful

graphical interpretation in R2 and R3. If is the

eigenvalue of A corresponding to x then Ax=x so

that the multiplication by A dilates x, contracts x, or

reverses the direction of x, depending upon the value

.

Dilation(>1) Contraction(0<


3/21

9

Example 1(Eigenvectors)

The solution of this equation are =1 and =2;

these are the eigenvalues ofA.

Since Ax=x

Eigenvector corresponding to =1

Putting =1 gives

1

2

3 20

1

x

x

1

2

2 20

1 1

x

x

10


The above matrix gives two equations

which gives the general solution in the form

1 2x x

1 2

1 2

2 2 0

0

x x

x x

2 1x 1 1 xLet

1

Then

The eigenvector corresponding to is

1

1

11


Similarly 12

1

2

1 2

1 2

1 2

2 1

3 20

1

P u t t i n g 2

1 20 g i v e s e q u a t i o n s

1 2

2 0

2 0

T h e s o lu t io n o f a b o v e e q u a t io n s i s

2

1 2E ig en v e c t o r c o rr e sp p o n d in g t o 2 i s

2

1

x

x

x

x

x x

x x

x x

l e t x t h e n x

12

Multiple Eigenvalues

The orderM of an eigenvalue as a root of

characteristic polynomial is called the

algebraic multiplicity of.

The number m of linearly independent

eigenvectors corresponding to is called the

geometric multiplicity of.

Thus m is the dimension of the eigenspace

corresponding to this .


4/21

13

Multiple Eigenvalues

The characteristic polynomial has a degree

n, the sum of all algebraic multiplicity must

equal n.

In general m M

The defect of is defined as

Example :Find the eigenvalues and

eigenvectors of the matrix

m

2 2 3

2 1 6

1 2 0

A

14

Example 2(1/7)

Eigenvalues

The characteristic polynomial

have three latent root (eigenvalues) as

2 2 3

2 1 6

1 2

d e t( ) 0

A I

A I

3 2 21 45 0

1 2 35 , 3

15

Example 2(2/7)

The algebraic multiplicity of =-3 is 2

M-3 =2

and that of=5 is M5 =1

Eigenvectors: To find the eigenvectors apply

the gauss elimination to the system (A-I)x=0

first =5 and =-3.

Putting =5 gives

1

2

3

7 2 3( ) 2 4 6 0

1 2 5

x

A I x x

x 16

Example 2(3/7)

Performing Gauss elimination on the matrix

above gives1 2 5

0 1 2

0 0 0

A I

1 2 3

2 3

2 5 0

2 0

x x x

x x


5/21

17

Example 2(4/7)

The above matrix can be written in the

equation form which gives

1 2 3

2 3

3 2 1

1

(2 5 )

2

Let 1 then 2 and 1

T he e igen vector c orresponding to 5

1

2

1

x x x

x x

x x x

18

Example 2(5/7)

Putting =-3 and performing Gauss

Elimination gives

1 2 3

3 0 0 0

0 0 0

A I

1 2 3

1 2 3

2 3 0

2 3

x x x

o r

x x x

19

Example 2(6/7)

Or

So there are two linearly independent vectors

corresponding to

1

0

3

0

1

2

32

3

2

1

xx

x

x

x

3

20

Example 2(7/7)

Since there are two Linear Independent

vectors corresponding to =-3 the geometric

multiplicity of

m-3 =2

m5 =1

The defect of=-3 is

-3 = M-3m-3

-3=05=0


6/21

21

Example:

Find the Eigenvalues and Eigenvectors of the matrix

Solution:

The algebraic multiplicity is

00

10A

0

1

10

01

00

10IA

00

1det 2

IA

021

20 MM 22

Example (Contd)

To find Eigenvector

The solution is

The Eigenvector is

The geometric multiplicity is

Thus

0

0

00

10

2

1

x

x

12 .0 xx

0

11

2

1x

x

x

10 mm

00 Mm

23

Theorems

If A is an nn triangular matrix (upper

triangular, lower triangular or diagonal) then

the eigenvalues of A are entries on main

diagonal of A.

Example :

1 2 3

10 0

2

21 0

3

15 8

4T h e E i g e n v a lu e s a r e

1 2 1, ,

2 3 4

A

24

Theorems


7/21

25

Theorems

26

Example

27

Theorems

If k is a positive integer, is an eigenvalue of

a matrix A, and x is a corresponding

eigenvector, then k is an eigenvalue of Ak

and x is a corresponding eigenvector.

28

Example:

Eigenvalues of are:

1282,11 73271

1 2 3

0 0 2

If 1 2 1

1 0 3

thenfindEigenvaluesfor if eigenvalues

for are 1, 2

A

7A

A

7


8/21

29

Theorems

30

Theorems

If x is an eigenvector of a matrix A to an

eigenvalue ,so is kx with any k 0.

31

Example

Let

The eigenvalues of A are 3, 0, and 2. Theeigenvalues of B are 4 and 1.

What does it mean for a matrix to have an eigenvalueof 0 ?

This happens if and only if the equation hasa nontrivial solution. Equivalently, Ax=0 has anontrivial soltion if and only if A is not invertible. Thus

0 is an eigenvalue of A if and only if A is notinvertible.

435

012

004

and

200

600

863

BA

0xx A

32

Diagonalization

The Eigenvector Problem:

Given an matrix A, does there exist abasis for consisting of eigenvectors of A?

The Diagonalization Problem:

Given an matrix A, does there exist aninvertible matrix P such that isa diagonal matrix?Apparently above two problems are different butthey are equivalent thats why diagonalization

problem is considered in the discussion ofeigenvectors.

n nn

R

n n1D P AP


9/21

33

Diagonalizable Matrix

Definition:

A square matrix A is said to bediagonalizable if there exist an invertiblematrix P such that is a diagonalmatrix.

Theorem:

If A is an matrix then the following areequivalent:

A is diagonalizable

A has n linearly independent eigenvectors.

1D P AP

n n

34

Procedure for Diagonalization

1. Find eigenvalues of A.

2. Find n linearly independent eigenvectors

of A, say,

3. Form matrix P having as

its columns.

4. Construct . The matrixD will be

diagonal with as its

successive diagonal entries, where is the

eigenvalue corresponding to .

1 2 3 4, , , ,..., .np p p p

1 2 3 4, , , ,..., np p p p

1D P AP

1 2 3 4, , , ,..., n

i

, 1, 2,...,i

i n

35

Example 1 (1/10)

Find a matrix P that diagonalizes

0 0 2

1 2 1

1 0 3

A

Step 1

Find the eigenvalues of A

Solution:

1 0 0 0 0 2

0 1 0 1 2 1

0 0 1 1 0 3

I A

36

0 0 0 0 2

0 0 1 2 1

0 0 1 0 3

0 2

1 2 1

1 0 3

0 2

det( ) 1 2 1

1 0 3

I A

Example 1 (2/10)


10/21

37

The characteristic equation of A is3 25 8 4 0

In factored form

2

1 2 0 Verify!

Thus, the eigenvalues of A are

1 2 31, 2

Example 1 (3/10)

38

Step 2

Find the corresponding eigenvectors

By definition 1

2

3

x

x

x

x

is an eigenvector of A corresponding to if and

only ifx is a non-trivial solution of

that is

0I A x

1

2

3

0 2 0

1 2 1 0

1 0 3 0

x

x

x

Example 1 (4/10)

39

If , then2

1

2

3

2 0 2 0

1 0 1 0

1 0 1 0

x

x

x

Solving this system yields

1 3 x x x2,,x3 (free variables)

The eigenvectors of A corresponding to

are nonzero vectors of the form2

0 1 0

0 0 1

0 1 0

s s

t t s t

s s

x

Example 1 (5/10)

40

As

are linearly independent, so they form the basis

for eigenspace corresponding to 2

1 0

0 , 1

1 0

If , then1

1

2

3

1 0 2 0

1 1 1 0

1 0 2 0

x

x

x

Example 1 (6/10)


11/21

41

Solving this system yields

1 32 x x x3 (free variable)

The eigenvectors corresponding to are

nonzero vectors of the form1

2 2

1

1

s

s s

s

x So2

1

1

2 3x x

=>

1 2 3

1 0 2

0 , = 1 , 1

1 0 1

p p p

Example 1 (7/10)

42

Step 3

It is easy to verify that are

linearly independent so that

diagonalizes A.

Step 4

The resulting diagonal matrix is

1 2 3{ , , }p p p

1 0 2

0 1 1

1 0 1

P

1

2 0 0

0 2 0

0 0 1

D P AP

Example 1 (8/10)

43

To verify that

1

2 0 0

0 2 0

0 0 1

A P P

We can compute

PA DP Verify it for the given case

Example 1 (9/10)

44

Remark:

There is no preferred order for the columns of

P. Since the ith diagonal entry of is an

eigenvalue corresponding to ith column vector

of P, changing the order of columns of P will

change the order of eigenvalues on the

diagonal of .

1P AP

1P AP

1 2 0

0 1 1

1 1 0

P

For example

12 0 00 1 0

0 0 2

D P AP=>

Example 1 (10/10)


12/21

45

Example 2 (1/3)

Diagonalize

The characteristic equation of A is

Thus, is the only eigenvalue of A, and

corresponding eigenvectors are the solutions of

3 2

2 1

A

1

( ) 0 I A x

2( 1) 0.

46

1 2

1 2

2 2 0

2 2 0

1

1

x x

x x

x t

That is

This system has eigenspace consisting of

vectors of the form

1

1

tt

t

Example 2 (2/3)

47

Example (3/3)

Since A does not have two linearly

independent eigenvectors i.e. the eigenspace

is 1-dimensional, therefore, A is not

diagoalizable.

48

Conditions for Diagonalizability

Theorem 1:

If are eigenvectors of A

corresponding to distinct eigenvalues

,then

are linearly independent.

Theorem 2:

If an matrix A has n distinct eigenvalues,

then A is diagonalizable.

1 2 3 4, , , ,..., nv v v v v

n n

1 2 3 4, , , ,..., n 1 2 3, , ,..., nv v v v


13/21

49

Example

Whether A is

diagonalizable or not?0 1 0

0 0 1

4 17 8

A

The eigenvalues of A are

1 2 34, 2 3, 2 3

Since A has three distinct eigenvalues, so A

can be diagonalized. We can verify that

1

4 0 0

0 2 3 0

0 0 2 3

D P AP

50

LU-Factorization

With Gaussian elimination and Gauss-Jordan

elimination, a linear system is solved by

operating systematically on the augmented

matrix.

Another approach is based on factoring the

coefficient matrix into a product of lower (L)

and upper (U) triangular matrix (LU-

Decomposition).

LU-decomposition speeds up the solution of

Ax b

51

Solving Linear Systems by

LU-Factorization

If an matrix can be factored into the product of L

and U matrices, then the linear system can be solved as:

Step 1

Rewrite the system as

Step 2

Define a new matrix y by

n n

Ax b

)(1LUx =b

1n

( )2U x = y52

Step 3

Use (2) to rewrite (1) as and solve fory

Step 4

Substitute y in (2) and solve forx

Ly=b

x b

y

Multiplication by A

Multip

licationb

yLMultipl

icationbyU

Solving Linear Systems by

LU-Factorization


14/21

53

LU Factorization

The LU factorization is motivated by the fairlycommon industrial and business problem of solving asequence of equations, all with the same coefficientmatrix A:

(1)

When A is invertible, one could compute and thencompute and so on. However, it is moreefficient to solve the first equation in (1) by rowreduction and obtain an LU factorization of A at thesame time. Thereafter, the remaining equations in (1)are solved with LU factorization.

n21 bx,,bx,bx AAA 1

,, 21

1

1bAbA

54

Example Solving Linear

Systems by LU-Factorization Solve the system

1

2

3

2 6 2 2

-3 -8 0 2

4 9 2 3

x

x

x

Given

2 6 2 2 0 0 1 3 1

-3 -8 0 3 1 0 0 1 3

4 9 2 4 3 7 0 0 1

A=LU

55

1

2

3

2 0 0 1 3 1 2

3 1 0 0 1 3 2

4 3 7 0 0 1 3

x

x

x

Step 1

Rewriting the system as

Step 2

Defining vectory as y=[y1, y2, y3]T

1 1

2 2

3 3

1 3 1

0 1 3

0 0 1

y

y

y


Systems by LU-Factorization

56

Equivalently1

1 2

1 2 3

2 2

3 2

4 3 7 3

y

y y

y y y

Solving using forward substitution, we get

1 2 31, 5, 2y y y

Step 3

1

2

3

1 3 1 1

0 1 3 5

0 0 1 2

x

x

x




15/21

57

Equivalently1 2 3

2 3

3

3 1

3 5

2

x x x

x x

x

Solving using back substitution, we get

1 2 32, x 1, x 2x

Step 4



58

LU decomposition

Theorem:

If A is a square matrix that can be reduced to

a row-echelon form U without using row

interchanges, then A can be factored as

A = LU, where L is a lower triangular matrix.

59

Procedure for LU decomposition

Step 1 : Reduce A to a row-echelon form U without

row interchanges, keeping track of the multipliers

used to introduce the leading 1s and the multipliers

used to introduce the zeros below the leading 1s.

Step 2 : In each position along the main diagonal of

L, place the reciprocal of the multiplier that introduced

the leading 1 in that position in U.

Step 3 : In each position below the main diagonal ofL, place the negative of the multiplier used to

introduce the zero in that position in U. Step 4: Form the decomposition A=L U

60

LU Decomposition

Example

Find an LU-decomposition of

Solution: We begin by reducing A to row-echelon form,

keeping track of all multipliers.

573

119

026

A


16/21

61

LU Decomposition

Example

573

119

026

573

119

03

116

1Multiplier

580

120

03

11

3Multiplier

9Multiplier

5802

110

03

11

2

1Multiplier

1002

110

03

11

8Multiplier

1002

110

03

11

1Multiplier

62

LU Decomposition

Example

Constructing L from the multipliers yields the LU-

decomposition

1002

110

03

11

183

029

006

LUA

63

Iterative Solution of Linear

Systems

Although Gaussian elimination or Gauss-

Jordan elimination is generally the method of

choice for solving a linear system of n

equations in n unknowns, there are other

approaches to solving linear systems, called

iterative orindirect methods, that are better in

certain situations.

The Gauss-Seidel method is the most

commonly used iterative method.

64

Gauss-Seidel Method

Basic Procedure:

Algebraically solve each linear equation for xi

Assume an initial guess

Solve for each xi and repeat

Use absolute relative approximate error after each

iteration to check if error is within a prespecified

tolerance.


17/21

65

A set ofn equations in n unknowns:

11313212111 ... bxaxaxaxa nn

22323222121 bxa...xaxaxa nn

nnnnnnn bxaxaxaxa ...332211

. .

. .

. .

Gauss-Seidel Method (Algorithm)

The system Ax=b is reshaped by solving the firstequation forx1, the second equation forx2, and thethird forx3, and n

th equation forxn.

66

Rewriting each equation

11

131321211

a

xaxaxabx nn

nn

nn,nnnn

n

,nn

n,nnn,nn,n,nn

n

nn

a

xaxaxabx

a

xaxaxaxabx

a

xaxaxabx

112211

11

12212211111

1

22

232312122


67

Solve for the unknowns

Assume an initial guess forx

n

-n

2

x

x

x

x

1

1

Use rewritten equations to

solve for each value of xi.

Important: Remember to

use the most recent value

of xi. Which means to

apply values calculated to

the calculations remaining

in the current iteration.


68

Calculate the Absolute Relative Approximate Error

100new old

i i

x newi

i

x x

x

So when has the answer been found?

The iterations are stopped when the absolute relativeapproximate error is less than a pre-specified tolerance for all

unknowns.



18/21

69

Gauss-Seidel Method: Pitfall

2 5.81 34

45 43 1

123 16 1

A

Diagonally dominant: The coefficient on the diagonal

must be at least equal to the sum of the other

coefficients in that row and at least one row with a

diagonal coefficient greater than the sum of the other

coefficients in that row.

124 34 56

23 53 5

96 34 129

B

Which coefficient matrix is diagonally dominant?

Most physical systems do result in simultaneous linear

equations that have diagonally dominant coefficientmatrices.70

Gauss-Seidel Method: Example 1

Given the system of equations

15312 321 x-xx

2835 321 xxx

761373 321 xxx

1

0

1

3

2

1

x

x

x

With an initial guess of

The coefficient matrix is:

12 3 5

1 5 3

3 7 13

A

Will the solution converge

using the Gauss-Seidel

method?

71


12 3 5

1 5 3

3 7 13

A

Checking if the coefficient matrix is diagonally dominant

43155 232122 aaa

10731313 323133 aaa

8531212 131211 aaa

The inequalities are all true and hence the matrix A is

strictlydiagonally dominant.

Therefore: The solution should converge using the Gauss-

Seidel Method

72


1

2

3

12 3 5 1

1 5 3 28

3 7 13 76

x

x

x

Rewriting each equation

12

531 321

xxx

5

328 312

xxx

13

7376 213

xxx

With an initial guess of

1

0

1

3

2

1

x

x

x

50000.0

12

150311

x

9000.4

5

135.0282

x

0923.313

9000.4750000.03763 x


19/21

73


The absolute relative approximate error

1

0.50000 1.0000100 67.662%

0.50000x

2

4.9000 0100 100.00%

4.9000x

3

3.0923 1.0000100 67.662%

3.0923x

The maximum absolute relative error after the first

iteration is 100%74


8118.3

7153.3

14679.0

3

2

1

x

x

x

After Iteration #1

14679.0

12

0923.359000.4311

x

7153.3

5

0923.3314679.0282

x

8118.3

13

900.4714679.03763

x

Substituting the x values into the equations

After Iteration #2

0923.3

9000.4

5000.0

3

2

1

x

x

x

75


Iteration #2 absolute relative approximate error

x 1

0.14679 0.50000100 240.62%

0.14679

x 2

3.7153 4.9000100 31.887%

3.7153

x 3

3.8118 3.0923100 18.876%

3.8118

The maximum absolute relative error after the first iteration

is 240.62%This is much larger than the maximum absolute relative

error obtained in iteration #1. Is this a problem?76


Repeating more iterations, the following values are obtained

1x

2x

3x

67.662

18.876

4.0042

0.65798

0.07499

0.00000

3.0923

3.8118

3.9708

3.9971

4.0001

4.0001

100.00

31.887

17.409

4.5012

0.82240

0.11000

4.900

3.7153

3.1644

3.0281

3.0034

3.0001

67.662

240.62

80.23

21.547

4.5394

0.74260

0.50000

0.14679

0.74275

0.94675

0.99177

0.99919

1

2

3

4

5

6

x3x2x1Iteration

4

3

1

3

2

1

x

x

x

0001.4

0001.399919.0

3

2

1

x

xx

The solution obtained

is close to the exact solution of


20/21

77


Solve the following linear system by Gauss-

Seidel method.

1 2 3

1 2 3

1 2 3

3 2

6 4 11 1

5 2 2 9

x x x

x x x

x x x

Solution

We can verify that the system is not strictly

diagonally dominant, so the Gauss-Seidel

method does not guaranteed to converge.

1 3 1

6 4 11

5 2 2

A

78

Definiteness of Matrices

A matrix is Positive Definite iff xTAx>0 for all

x.

A matrix is Negative Definite iff xTAx0 for some x

and xTAx


21/21

81

Checking for Definiteness -Example

Check the definiteness of the matrix.

12 2

2 10A

Solution

12 2

2 10

12 2

2903

1 2

1

6E E

Since the two diagonal elements are +ve,

therefore matrix A is +ve definite.

eigen values, eigen vectors_afzaal_1.pdf

Documents