lecture 15, 16: diagonalizationgraphics.ics.uci.edu/ics6n/newlectures/lecture15,16.pdf · ·...
TRANSCRIPT
Lecture 15, 16: Diagonalization
• Suppose 𝐴 is a 𝑛 × 𝑛 matrix and has 𝑛 linearly independent eigenvectors 𝒑1…𝒑𝑛, put them into columns of eigenvector matrix 𝑃.
• Put eigenvalues into a diagonal matrix 𝐷. 𝐷𝑖𝑖 has the eigenvalue corresponding to the eigenvector in the 𝑖-th column of 𝑃.
𝐴𝑃 = 𝐴 𝒑1 … 𝒑𝑛 = 𝜆1𝒑1 … 𝜆𝑛𝒑𝑛 = 𝑃𝐷.
• From above equation it is implied:• 𝑃−1𝐴𝑃 = 𝐷 or 𝑃𝐷𝑃−1 = 𝐴• The square matrix 𝐴 is diagonalizable if it has 𝑛 distinct eigenvalues or if its 𝑛 eigenvectors are linearly independent.
1
Motivation: Eigenvalues and Eigenvectors are easy to compute for diagonal matrices. Hence, we would like (if possible) to convert matrix A into a diagonal matrix.
Powers of A
• Suppose 𝐴 is factored to 𝑃𝐷𝑃−1.
• To compute 𝐴2 we multiply its factored equivalent:
• 𝐴2 = 𝑃𝐷𝑃−1 𝑃𝐷𝑃−1 = 𝑃𝐷2𝑃−1.
• We can imply that 𝐴𝑘 = 𝑃𝐷𝑘𝑃−1
• 𝐷𝑘 is the diagonal matrix 𝐷 where its entries are to the power of 𝑘, and is very easy to calculate.
Powers of A
Steps for Matrix Diagonalization
• Diagonalize the following matrix:
𝐴 =1 3 3−3 −5 −33 3 1
• Find the eigenvalues 𝜆1, 𝜆2, 𝜆3.
• Find three linearly independent eigenvectors of A.
• Construct P = [v1, v2, v3]
• Construct D.
• Check 𝐴𝑃 = 𝑃𝐷 𝑎𝑛𝑑 𝐴 = 𝑃𝐷𝑃−1
Not all Matrices are Diagonalizable
Application: Steady State
• Suppose there is a system such that 𝒖𝑘+1 = 𝐴𝒖𝑘, where 𝑢 is the state vector and its subscripts denote its order in a time series.
• If 𝒖0 is the initial condition of the system at beginning, we can find 𝒖𝑘, the state at the time 𝑛, using 𝒖𝑘 = 𝐴𝑘𝒖0.
• We know that 𝐴𝑘 = 𝑋𝐷𝑘𝑋−1.
• So if 𝑛 → ∞:• If 𝜆𝑖 < 1 then 𝜆𝑖
𝑘 → 0.• If 𝜆𝑖 = 1 then 𝜆𝑖
𝑘 = 1.• If 𝜆𝑖 > 1 then 𝜆𝑖
𝑘 → ∞.
• So, stable systems 𝐴 should not have eigenvalues 𝜆𝑖 > 1 .
Application: Steady State
• Example: Suppose 10% of residents of the city A move to city B in a year. During the same time 5% of the city B residents move to city A. If initial residents of cities A and B are 10,000 and 100,000 respectively, what will happen in a long run ?
• We formulate the system as 𝒖 =𝑟𝑒𝑠𝑖𝑑𝑒𝑛𝑡𝑠 𝑜𝑓 𝑐𝑖𝑡𝑦 𝐴𝑟𝑒𝑠𝑖𝑑𝑒𝑛𝑡𝑠 𝑜𝑓 𝑐𝑖𝑡𝑦 𝐵
• If no other thing affect the population of the cities A and B, the matrix 𝐴 is:
• 𝒖1 = 𝐴𝒖0 → 𝒖1 =.9 .05.1 .95
𝒖0
• Eigenvalues of 𝐴, are 𝜆1 = 0.85 and 𝜆2 = 1.
• Eigenvectors of 𝐴 are 𝒙1 =−11
, 𝒙2 =−1−2
Application: Steady State
• Example Cont’d:• In a long run 𝒖𝑘 = 𝐴𝑘𝒖0
• 𝐴𝑘 = 𝑋𝐷𝑘𝑋−1 =−1 −11 −2
0.85 00 1
𝑘 −2
3
1
3
−1
3−
1
3
• If 𝑘 → ∞ then 𝜆1𝑘 → 0.
• Only the 𝜆2 will be effective in 𝐴𝑘 when 𝑘 → ∞
• lim𝑘→∞
𝐴𝑘 =−1 −11 −2
0 00 1
−2
3
1
3
−1
3−
1
3
=
1
3
1
32
3
2
3
• 𝒖∞ = 𝐴∞𝒖0 =
1
3
1
32
3
2
3
10,000100,000
~36,66773,333
Application: Steady State
• Example Cont’d:• Another approach:• Recall that if 𝒚 = 𝛼1𝒙1 + 𝛼2𝒙2 +⋯+ 𝛼𝑛𝒙𝑛 then 𝐴𝒚 = 𝛼1𝜆1𝒙1 + 𝛼2𝜆2𝒙2 +⋯+ 𝛼𝑛𝜆𝑛𝒙𝑛
• Therefore, 𝐴𝑘𝒚 = 𝛼1𝜆1𝑘𝒙1 + 𝛼2𝜆2
𝑘𝒙2 +⋯+ 𝛼𝑛𝜆𝑛𝑘𝒙𝑛
•10,000100,000
~−1 −11 −2
26667−36667
• 𝐴𝑘10,000100,000
= 26667(0.85)𝑘−11
− 36667(1)𝑘−1−2
• 𝐴∞10,000100,000
= −36667−1−2
~36,66773,333
Application: Discrete Dynamic Systems
Application: Discrete Dynamic Systems
Application: Fibonacci Series
• A Fibonacci series: 1, 1, 2, 3, 5, 8, 13 ……
• How to find the 100th element x100 quickly?
Application: Fibonacci Series
Application: Fibonacci Series
Diagonalization of Symmetric Matrices
• Symmetric matrices always have real eigenvalues and are diagonalizable.
• What is special about 𝐴𝑥 = 𝜆𝑥, when 𝐴 is symmetric (𝐴 = 𝐴𝑇)?
• 𝐴 = 𝐴𝑇 → 𝑃𝐷𝑃−1 = (𝑃𝐷𝑃−1)𝑇= (𝑃−1)𝑇𝐷𝑃𝑇
• Above equation is correct only if 𝑃−1 = 𝑃𝑇, and this means 𝑃 is orthonormal.
• Therefore, every symmetric matrix 𝐴 = 𝑄𝐷𝑄𝑇, where 𝑄 is the normalized eigenvector matrix, and 𝐷 is the eigenvalue diagonal matrix.
• 𝑄 is an orthonormal matrix and 𝑄−1 = 𝑄𝑇.
Diagonalization of Symmetric Matrices
Example: re-write matrix 𝐴 =1 22 −2
, as factorization 𝐴 = 𝑄𝐷𝑄𝑇.
• 𝐴 − 𝜆𝐼 =1 − 𝜆 22 −2 − 𝜆
• 1 − 𝜆 −2 − 𝜆 − 4 = 0 → 𝜆2 + 𝜆 − 6 = 0 → 𝜆 + 3 𝜆 − 2 = 0; 𝜆1 = −3 and 𝜆2 = 2
𝐴 − 𝜆1𝐼 𝒙1 = 0 →4 22 1
= 𝟎 → 𝒙1 =1−2
𝐴 − 𝜆2𝐼 𝒙2 = 0 →−1 22 −4
= 𝟎 → 𝒙2 =2−1
• 𝒙1 and 𝒙2 are orthogonal, but not orthonormal. If we normalize them, they will be orthonormal.
• 𝑄 =1
5
1 2−2 −1
, 𝐷 =−3 00 2
Diagonalization of Symmetric Matrices
Example: Diagonalize
Diagonalization of Symmetric Matrices
Spectrum Theorem
• If A is a n x n symmetric matrix
• All eigenvalues of A are real.
• A has exactly n real eigenvalues (counting for multiplicity). But this does not mean they are distinct.
• The geometric multiplicity of 𝜆 = dimension of Null(A - 𝜆I) = the algebraic multiplicity of 𝜆.
• The eigenspaces are mutually orthogonal.
• If 𝜆1 ≠ 𝜆2 are two distinct eigenvalues, then their corresponding eigenvectors v1 and v2 are orthogonal.
Proof