matrices a brief introduction - polito.it · 2015. 9. 30. · matrices a brief introduction basilio...

MatricesA brief introduction

Basilio Bona

DAUIN – Politecnico di Torino

Semester 1, 2014-15

B. Bona (DAUIN) Matrices Semester 1, 2014-15 1 / 41

Definitions

Definition

A matrix is a set of N real or complex numbers organized in m rows and ncolumns, with N = mn

A =

a11 a12 · · · a1na21 a22 · · · a2n· · · · · · aij · · ·am1 am2 · · · amn

≡

[aij]

i = 1, . . . ,m j = 1, . . . , n

A matrix is always written as a boldface capital letter viene as in A.

To indicate matrix dimensions we use the following symbols

Am×n Am×n A ∈ Fm×n A ∈ Fm×n

where F = R for real elements and F = C for complex elements.


Transpose matrix

Given a matrix Am×n we define a transpose matrix the matrix obtainedexchanging rows and columns

ATn×m =

a11 a21 · · · am1

a12 a22 · · · am2...

.... . .

...a1n a2n · · · amn

The following property holds (AT)T = A


Square matrix

A matrix is said to be square when m = n

A square n× n matrix is upper triangular when aij = 0, ∀i > j

An×n =

a11 a12 · · · a1n0 a22 · · · a2n...

.... . .

...0 0 · · · ann

If a square matrix is upper triangular its transpose is lower triangular andviceversa

ATn×n =

a11 0 · · · 0a12 a22 · · · 0...

.... . .

...a1n a2n · · · ann


Symmetric matrix

A real square matrix is said to be symmetric if A = AT, or

A−AT = O

In a real symmetric matrix there are at leastn(n+ 1)

2independent

elements.

If a matrix K has complex elements kij = aij + jbij (where j =√−1) its

conjugate is K with elements k ij = aij − jbij .

Given a complex matrix K, an adjoint matrix K∗ is defined, as the

conjugate transpose K∗ = KT= KT

A complex matrix is called self-adjoint or hermitian when K = K∗. Sometextbooks indicate this matrix as K† or KH


Diagonal matrix

A square matrix is said to be diagonal if

aij = 0 for i 6= j

An×n = diag(ai) =

a1 0 · · · 00 a2 · · · 0...

.... . .

...0 0 · · · an

A diagonal matrix is always symmetric.


Matrix Algebra

Matrices form an algebra, i.e., a vector space endowed with the productoperator . The main operations are: product by a scalar, sum, matrixproduct

Product by a scalar

αA = α

a11 a12 · · · a1na21 a22 · · · a2n...

.... . .

...am1 am2 · · · amn

=

αa11 αa12 · · · αa1nαa21 αa22 · · · αa2n...

.... . .

...αam1 αam2 · · · αamn

Sum

A+B =

a11 + b11 a12 + b12 · · · a1n + b1na21 + b21 a22 + b22 · · · a2n + b2n

......

. . ....

am1 + bm1 am2 + bm2 · · · amn + bmn


Sum

Properties

A+O = A

A+ B = B+ A

(A +B) + C = A+ (B+ C)

(A+ B)T = AT + BT

The neutral element O is called null or zero matrix . The matrix differenceis defined introducing the scalar α = −1:

A−B = A+ (−1)B.


Matrix Product

Matrix product

The operation follows the rule “row by column”: the generic cij element ofthe product matrix Cm×p = Am×n · Bn×p is

cij =n∑

k=1

aikbkj

The following identity holds:

α(A ·B) = (αA) ·B = A · (αB)


Product

Properties

A ·B · C = (A · B) · C = A · (B · C)A · (B + C) = A ·B+ A · C(A+ B) · C = A · C+ B · C(A · B)T = BT ·AT

In general:

the matrix product is NOT commutative: A · B 6= B ·A, except someparticular case;

A ·B = A · C does not imply B = C, except some particular case;

A ·B = O does not imply A = O or B = O, except some particularcase.


Identity Matrix

The neutral element wrt the matrix product is calledidentity matrix usuallywritten as In or I when there are no ambiguities on the dimension.

Identity matrix

I =

1 0 · · · 00 · · · · · · 0...

.... . .

...0 0 · · · 1

Given a rectangular matrix Am×n the following hold

Am×n = ImAm×n = Am×nIn


Matrix Power

Given a square matric A ∈ Rn×n, the k-th power is

Ak =

k∏

ℓ=1

A

One matrix is said to be idempotent iff

A2 = A → Ak = A.


Matrix Trace

Trace

The trace of a square matrix An×n is the sum of its diagonal elements

tr (A) =

n∑

k=1

akk

Trace satisfy the following properties

tr (αA + βB) = α tr (A) + β tr (B)tr (AB) = tr (BA)tr (A) = tr (AT)tr (A) = tr (T−1AT) for T non-singular (see below for explanation)


Row/column cancellation

Given the square matrix A ∈ Rn×n, we call A(ij) ∈ R

(n−1)×(n−1) the matrixobtained deleting the la i -the row and the j-the columns of A.

Example: given

A =

1 −5 3 2

-6 4 9 -7

7 −4 -8 2

0 −9 -2 −3

deleting row 2, column 3 we obtain

A(23) =

1 −5 27 −4 20 −9 −3


Minors and Determinant

A minor of order p of a generic matrix Am×n is defined as the ildeterminant Dp of a square sub-matrix obtained selecting any p rows andp columns of Am×n

There exist as many minors as the possible choices of p on m rows and pon n columns

The definition of determinant will be given soon.

Given a matrix Am×n its principal minors of order k are the determinantsDk , with k = 1, · · · ,min{m, n}, obtained selecting the first k rows and kcolumns of Am×n.


Example

Given the 4× 3 matrix

A =

1 −3 57 2 4−1 3 28 −1 6

we compute a generic minor D2, for example that obtained selecting thefirst and rows 1 and 3 and columns 1 and 2 (in red).

First we form the submatrix

D =

[1 −3−1 3

]

and then we compute the determinant

D2 = det(D) = 3× 1− (−3×−1) = 0


Example


A =

1 −3 57 2 4−1 3 28 −1 6

we compute the principal minors minors Dk , k = 1, 2, 3,

D1 = 1

D2 = det

[1 −37 2

]

= 23

D3 = det

1 −3 57 2 4−1 3 2

= 161


Complement

We call the complement Crc of a generic (r , c) element of a square matrixAn×n the determinant of the matrix obtained deleting its r -the row and

the c-th column, i.e., detA(rc)

Drc = detA(rc).

The cofactor of the (r , c) element of a square matrix An×n is the signedproduct

Crc = (−1)r+cDrc


Example


A =

1 2 34 5 67 8 9

some of the cofactors are

C11 = (−1)2(45 − 48) = −3

C12 = (−1)3(36 − 42) = 6

C31 = (−1)4(12 − 15) = −3


Adjugate/Adjunct/Adjoint

The cofactor matrix of A is the n× n matrix C whose (i , j) entry Cij is the(i , j) cofactor of A

Cij = (−1)i+jDij

The adjugate or adjunct or adjoint of a square matrix A is the transposeof C, that is, the n × n matrix whose (i , j) entry is the (j , i) cofactor of A,

Aadjij = Cji = (−1)i+jDji

The adjoint matrix of A is the matrix X that satisfies the following equality

AX = XA = det(A)I


Example


A =

1 3 24 6 57 9 8

its adjoint is

Aadj =

3 −6 33 −6 3−6 12 −6


Determinant

The determinant of a square matrix Ax×n can be computed in differentways.

Choosing any row i , the definition “by row” is:

det (A) =

n∑

k=1

aik(−1)i+k det (A(ik)) =

n∑

k=1

aikAik

Choosing any column j , the definition “by column” is::

det (A) =

n∑

k=1

akj (−1)k+j det (A(kj)) =

n∑

k=1

akjAkj

Since these definitions are recursive, involving the determinants ofincreasingly smaller minors, we define the determinant of a 1× 1 matrixA = a, simply as detA = a.


Properties

The determinant has the following properties:

det(A · B) = det(A) det(B)

det(AT) = det(A)

det(kA) = kn det(A)

if one exchanges s rows or columns of A, obtaining As , thendet(As) = (−1)s det(A)

if A has two or more rows/columns equal or proportional, thendet(A) = 0

if A has a row/column that can be obtained as a linear combinationof other rows/columns, then det(A) = 0

if A is triangular, then det(A) =∏n

i=1 aii

if A is block-triangular, with p blocks Aii on the diagonal, thendet(A) =

∏pi=1 detAii


Rank and Singularity

A matrix A is singular if det(A) = 0.

The rank (or characteristic) of a matrix Am×n is the number ρ(Am×n),defined as the largest integer p for which at least a minor Dp is non-zero.

The following properties hold:

ρ(A) ≤ min{m, n}if ρ(A) = min{m, n}, A is said to be full rank

if ρ(A) < min{m, n}, the rank of the matrix is said to drop

if An×n and detA < n the matrix is not full rank, or is rank deficient

ρ(A ·B) ≤ min{ρ(A), ρ(B)}ρ(A) = ρ(AT)

ρ(A ·AT) = ρ(AT ·A) = ρ(A)


Invertible Matrix

A square matrix A ∈ Rn×n it is said to be invertible or non singular if the

inverse A−1n×n exists, such that

AA−1 = A−1A = In

A matrix is invertible iff ρ(A) = n, i.e., it is full-rank; this is equivalent tohave a non zero determinant det(A) 6= 0.

The inverse is computed as:

A−1 =1

det(A)Aadj

The following properties hold: (A−1)−1 = A; (AT)−1 = (A−1)T.

The inverse, if exists, allows to solve the following matrix equation

y = Ax

with respect to the unknown x, as

x = A−1y.


Matrix derivative

If a square matrix An×n(t) has elements function of a variable (e.g., thetime t) aij(t), then the matrix derivative is

d

dtA(t) = A(t) =

[d

dtaij(t)

]

= [aij(t)]

If A(t) rank is full, ρ(A(t)) = n for every t, then the derivative of theinverse is

d

dtA(t)−1 = −A−1(t)A(t)A(t)−1

Notice that, since the inverse is a nonlinear function, the derivative of theinverse is in general different from the inverse of the derivative

[dA(t)

dt

]−1

6= d

dt

[A(t)−1

]


Example

Given the square matrix

A(t) =

[cos θ(t) − sin θ(t)sin θ(t) cos θ(t)

]

we haved

dtA(t) = A(t) =

[− sin θ(t) − cos θ(t)cos θ(t) − sin θ(t)

]

θ

The inverse of A is

A(t)−1 =

[cos θ(t) sin θ(t)− sin θ(t) cos θ(t)

]

= A(t)T

and in this particular case the two inverses are equal

[dA(t)

dt

]−1

=d

dt

[A(t)−1

]=

[cos θ(t) sin θ(t)− sin θ(t) cos θ(t)

]


Matrix Decomposition

Given a real matrix A ∈ Rm×n, the following products give symmetric

matricesATA ∈ R

n×n

AAT ∈ Rm×m

Given a square matrix A, it is always possible to decompose it in a sum oftwo matrices

A = As + Ass

where

As =1

2(A +AT)

is symmetric , and

Ass =1

2(A− AT)

is skew-symmetric .


Similarity Transformation

Similarity transformation

Given a square matrix A ∈ Rn×n and a square nonsingular matrix

T ∈ Rn×n, the matrix B ∈ R

n×n obtained as

B = T−1AT or B = TAT−1

is called similar to A, and the transformation T is called similaritytransformation.


Eigenvalues and Eigenvectors

If it is possible to find a nonsingular matrix U such that A is similar to thediagonal matrix Λ = diag(λi )

A = UΛU−1 → AU = UΛ

and if we call ui the i -th column of U,

U =[u1 u2 · · · un

]

we haveAui = λiui

This relation is the well known formula defining eigenvectors andeigenvalues of A.

The λi are the eigenvalues of A and the ui are the eigenvectors of A.


Eigenvalues and Eigenvectors

Eigenvalues and eigenvectors

Given a square matrix An×n, the matrix eigenvalues λi are the (real orcomplex) solutions of the characteristic equation

det(λI− A) = 0

det(λI−A) is a polynomial in λ, called the characteristic polynomial of A.

If the eigenvalues are all distinct, we call eigenvectors the vectors ui thatsatisfy the following identity

Aui = λiui


Geometrical interpretation

If the eigenvalues are not all distinct, we obtain the generalizedeigenvectors, whose characterization goes beyond the scope of thispresentation.

From a geometrical point of view, the eigenvectors represent thoseparticular “directions” in the R

n space, i.e., the domain of the lineartransformation represented by A, that remain invariant under thetransformation, while the eigenvalues give the scaling constants alongthese same directions.

The set of the matrix eigenvalues is usually indicated as Λ(A) or {λi (A)};the set of the matrix eigenvectors is indicated as {ui (A)}. In general, theyare normalized, i.e., ‖{ui(A)}‖ = 1


Eigenvalues Properties

Given a square matrix A and its eigenvalues, {λi (A)}, the followingproperties hold true

{λi (A+ cI)} = {(λi (A) + c)}

{λi (cA)} = {(cλi (A)}

Given a triangular matrix

a11 a12 · · · a1n0 a22 · · · a2n...

.... . .

...0 0 · · · ann

,

a11 0 · · · 0a21 a22 · · · 0...

.... . .

...an1 an2 · · · ann

its eigenvalues are the elements on the main diagonal {λi (A)} = {aii}; thesame is true for a diagonal matrix.


Other properties

Given a square matrix An×n and its eigenvalues {λi (A)}, the followinghold true

det(A) =

n∏

i=1

λi

and

tr (A) =n∑

i=1

λi

Given any invertible similarity transformation T,

B = T−1AT

the eigenvalues of A are invariant to it, i.e.,

{λi (B)} = {λi (A)}


Modal matrix

If we build a matrix M whose columns are the normalized eigenvectorsui(A)

M =[u1 · · · un

]

then the similarity transformation with respect to M results in the diagonalmatrix

Λ =

λ1 0 · · · 00 λ2 · · · 0...

.... . .

...0 0 · · · λn

= M−1AM

M is the modal matrix .

If A is symmetric, all its eigenvalues are real and we have

Λ = MTAM

In this case M is orthonormal.


Singular Value decomposition (SVD)

Given a matrix A ∈ Rm×n, having rank r = ρ(A) ≤ s, with

s = min{m, n}, it can be decomposed (factored) in the following way:

A = UΣVT =

s∑

i=1

σiuivTi (1)

The decomposition is characterized by three elements:

σi

ui

vi

as follows.


SVD Characterization

σi(A) ≥ 0 are the singular values and are equal to the non-negative

square roots of the eigenvalues of the symmetric matrix ATA:

{σi (A)} = {√

λi (ATA)} σi ≥ 0

ordered in decreasing order

σ1 ≥ σ2 ≥ · · · ≥ σs ≥ 0

if r < s there are r positive singular values; the remaining ones arezero

σ1 ≥ σ2 ≥ · · · ≥ σr > 0; σr+1 = · · · = σs = 0

U is a orthonormal (m ×m) matrix

U =[u1 u2 · · · um

]

containing the eigenvectors ui of AAT


SVD Characterization

V is a (n × n) orthonormal matrix

V =[v1 v2 · · · vn

]

whose columns are the eigenvectors vi of the matrix ATA

Σ is a (m × n) matrix, with the following structure

if m < n Σ =[Σs O

]

if m = n Σ = Σs

if m > n Σ =

[Σs

O

]

.

where Σs = diag(σi ) is diagonal with dimension s × s, having thesingular values on the diagonal.


Example

Given

A =

[1 3 24 6 5

]

ρ(A) = 2

its SVD isA = UΣVT

where

U =

[−0.3863 −0.9224−0.9224 0.3863

]

Σ =

[9.5080 0 0

0 0.7729 0

]

V =

−0.4287 0.8060 −0.4082−0.7039 −0.5812 −0.4082−0.5663 0.1124 0.8165


Alternative SVD Decomposition

Alternately, we can decompose the A matrix as follows:

A =[P P

]

︸︷︷︸

U

[Σr OO O

]

︸︷︷︸

Σ

[QT

QT

]

︸︷︷︸

VT

= PΣrQT (2)

where

P is an orthonormal m × r matrix,

P is an orthonormal m × (m − r) matrix;

Q is an orthonormal n × r matrix, QTis an orthonormal n × (n − r)

matrix;

Σr is an diagonal r × r matrix, whose diagonal elements are thepositive singular values σi > 0, i = 1, · · · , r .


Rank

The rank r = ρ(A) of A is equal to the number r ≤ s of nonzero singularvalues.

Given any matrix A ∈ Rm×n, both ATA and AAT are symmetric, with

identical positive singular values and differ only for the number of zerosingular values.


matrices a brief introduction - polito.it · 2015. 9. 30. · matrices a brief introduction basilio...

Documents