applied geometrical matrix computations

Applied GeometricalApplied GeometricalMatrix ComputationsMatrix Computations

Alan EdelmanAlan EdelmanDept of Mathematics: MITDept of Mathematics: MIT

MIT Laboratory for Computer ScienceMIT Laboratory for Computer Science

Householder Symposium XV

June 21, 2002

OutlineOutline

• Geometrical Matrix ComputationsGeometrical Matrix Computations• Illustration with 2x2 matrices:Illustration with 2x2 matrices:• Excursions into eigenland (or why tangency Excursions into eigenland (or why tangency

and curvature matter!!)and curvature matter!!)• Where do matrix factorizations come from?Where do matrix factorizations come from?• Application to Color ScienceApplication to Color Science• Matrix AnimationsMatrix Animations

Working definition:Working definition:•Concerns geometry of matrix space Concerns geometry of matrix space

(n(n2 2 dimensions rather than n)dimensions rather than n)•Involves numerical computation (probably MATLAB)Involves numerical computation (probably MATLAB)•Relates to an NLA problemRelates to an NLA problem

Geometrical Matrix Geometrical Matrix ComputationsComputations

Some Other GMC PeopleSome Other GMC People

Absil, Demmel, Elmroth, Huhtanen, Kagstrom, Absil, Demmel, Elmroth, Huhtanen, Kagstrom, Kahan,Kahan,

Lippert, Ma, Mahony, Malyshev, Sepulchre, Lippert, Ma, Mahony, Malyshev, Sepulchre, Tisseur, Trefethen, Van DoorenTisseur, Trefethen, Van Dooren

Vector Space DiagramsVector Space Diagrams•Points are vectors Points are vectors (not matrices!)(not matrices!)•Geometric relationships for vectors, Geometric relationships for vectors, subspaces, and linear transformationssubspaces, and linear transformations

OutlineOutline• Geometrical Matrix ComputationsGeometrical Matrix Computations• Illustration with 2x2 matrices:Illustration with 2x2 matrices:• Excursions into eigenland (or why tangency Excursions into eigenland (or why tangency


Eigenland (in 2d)Eigenland (in 2d)

2 2x z

x -zM = z -x

xx

zz

Isoeig surfaces are hyperbolasIsoeig surfaces are hyperbolas

The Eigenvalue MapThe Eigenvalue Map

/2/2

0000

zz

xx

Zero MatrixZero Matrix

x -zM = z -x

xx

zz2 2x z

cos z x Eigvec Angle


/2/2

0000

zz

xx


x -zM = z -x

xx

zz2 2x z


•MM• MM


/2/2

0000

zz

xx


x -zM = z -x

xx

zz2 2x z


• MM •MM??Uniformly

Pseudospectra (Trefethen)Pseudospectra (Trefethen)

2( ) { : eig( ), with || || }A z z A E E C :““z is an eigenvalue of a matrix near A”z is an eigenvalue of a matrix near A”

min{ : ( - ) }z zI A CPseudoportraits = pictures of contours of zPseudoportraits = pictures of contours of z

zzzz

A = 1 -1 0 0 0 1 1 -1 0 0 1 1 1 -1 0 1 1 1 1 -1 0 1 1 1 1 Random PointsRandom PointsPseudoportraitsPseudoportraits

pseudospectra & geometrypseudospectra & geometry

2nC

1A X X

X

ProjectProject

matrix space eig (w/singularity)spectral portrait2nC

OutlineOutline



Circle/Hyperbola Tangency = High DensityCircle/Hyperbola Tangency = High Density

have eigenvalue distributions with

2 spikes.

eigenvalueeigenvalue

frequency

frequency


4 spikes.


frequency

frequency


3 spikes.

frequency

frequency


•Circles tangent to 2 hyperbolas…

* *


*

**

*


*

* *

Radius of Curvature = Highest Radius of Curvature = Highest DensityDensity•Circles are tangent to 3

hyperbolas when two tangency points collide

•The circle also shares a radius of curvature with

the hyperbola at this point

•This is even better than tangency, which means a

higher spike


frequency

frequency

frequency

frequency

eigenvalueeigenvaluefr

equency

frequency


*

* *

OutlineOutline



Where do Matrix Factorizations Where do Matrix Factorizations Come From?Come From?

A=UV’

Classical Classical Answer:Answer:

Representation Representation Theory of Theory of

Semisimple Semisimple GroupsGroups

Semisimple group recipeSemisimple group recipe

•Nicely links factorizationsNicely links factorizations•Three ExamplesThree Examples

NonsingularsNonsingulars UnitaryUnitary OrthogonalOrthogonal

SVDSVD UeUeiiV’V’ CS decompCS decomp

SPDSPD Eigen Eigen UeUeiiU’U’ Essentially Sym Essentially Sym OrthOrth

One more exampleOne more example

Hyperbolic Svd as in last talkHyperbolic Svd as in last talk

Group = SO(p,q) ( XJ=JX)Group = SO(p,q) ( XJ=JX)

Matrix FactorizationsMatrix Factorizations

Where can we look for new factorizations?Where can we look for new factorizations?

• The Mathematics LiteratureThe Mathematics Literature– Lie Algebra: Cartan, Iwasawa, BruhatLie Algebra: Cartan, Iwasawa, Bruhat

– Representation Theory: QuiversRepresentation Theory: Quivers

• Nearness ProblemsNearness Problems

• ApplicationsApplications– Engineering: A factorization is useful if someone can use itEngineering: A factorization is useful if someone can use it

– Mathematics: The useful factorizations are characterized by an Mathematics: The useful factorizations are characterized by an abstract criterionabstract criterion

Ideas to GeneralizeIdeas to Generalize

E = (antisymmetric) + (symmetric)E = (antisymmetric) + (symmetric)

M= Q * S [polar]M= Q * S [polar]

expmexpmexpmexpmexpm

Non-singularNon-singular OrthogonalOrthogonal Pos DefinitePos Definite

1: Cartan Decomposition1: Cartan Decomposition







2:KAK Decomposition2:KAK DecompositionPositive Diagonals = Maximal Group Positive Diagonals = Maximal Group

M=UM=UV’V’

Conjugates give S=QConjugates give S=QQ’Q’








M=UM=UV’V’

Conjugates give S=QConjugates give S=QQ’Q’3:Iwasawa, Bruhat3:Iwasawa, Bruhat

Above not unique at I. GivesAbove not unique at I. Gives

M=M=LU, other permutations, totally LU, other permutations, totally positive, etcpositive, etc








M=UM=UV’V’

Conjugates give S=QConjugates give S=QQ’Q’3:Iwasawa, Bruhat3:Iwasawa, Bruhat

Above not unique at I. GivesAbove not unique at I. Gives

M=M=LU, other permutations, totally LU, other permutations, totally positive, etcpositive, etc4:Eigenvalue, Jordan 4:Eigenvalue, Jordan SchurSchur

Step 1:Cartan DecompositionStep 1:Cartan Decomposition•Group: Group: non-singular matricesnon-singular matrices

•Involution: Involution: (((M))=M(M))=M (M(M11MM22)= )= (M(M11))(M(M22))

(M)=M(M)=M-T-T

•Fixed Points Fixed Points (M)=M are a group (M)=M are a group KK

KK = orthogonal matrices = orthogonal matrices•Near INear I

M = (antisymmetric) + (symmetric)M = (antisymmetric) + (symmetric)•Cartan: expmCartan: expm

M = QS (S>0) (polar)M = QS (S>0) (polar)

Step 1:Cartan Decomposition (U/O)Step 1:Cartan Decomposition (U/O)•Group: Group: unitary matricesunitary matrices•Near INear I

M = (antisymmetric) + (i*symmetric)M = (antisymmetric) + (i*symmetric)•Cartan:Cartan:

M= (real orth)(unitary symmetric) M= (real orth)(unitary symmetric)

Step 2:KAK DecompositionStep 2:KAK Decomposition

P P = sym pos def= sym pos def• A A = biggest group inside= biggest group inside P P (abelian)(abelian)

e.g. e.g. diagonal > 0, or conjugates U diagonal > 0, or conjugates UU’ U’ (fix U)(fix U)•KAKKAK

M=UM=UV’V’•P P = union of conjugates= union of conjugates

S=QS=QQ’Q’

Step 2:KAK Decomposition (U/Q)Step 2:KAK Decomposition (U/Q)

P P = unitary symmetric = unitary symmetric • A A = biggest group inside= biggest group inside P P (abelian)(abelian)

e.g. diagonals (ee.g. diagonals (eii) or conjugates ) or conjugates •KAKKAK

M=UeM=UeiiV’ (U, V real orthogonal)V’ (U, V real orthogonal)•P P = union of conjugates= union of conjugates

S=QeS=QeiiQ’ (Q real orthogonal)Q’ (Q real orthogonal)

Step 2:KAK Decomposition (OStep 2:KAK Decomposition (Onn/O/Op p XX OOq q ))

P P = matrices orthogonally similar to ( = matrices orthogonally similar to ( ) )

• A A = biggest group inside= biggest group inside P P (abelian)(abelian)

e.g. e.g. =( ) or conjugates =( ) or conjugates •KAKKAK

The CS DecompositionThe CS Decomposition

CC SS

-S-S CC

CC SS

-S-S CC

MissingMissing•The constructible decompositionsThe constructible decompositions

Tridiagonalization, BidiagonalizationTridiagonalization, Bidiagonalization•The NNMF (Lee, Seung 1999)The NNMF (Lee, Seung 1999)

•V V WH Input: V WH Input: Vijij>0>0

Output: WOutput: Wijij>0 H>0 Hijij>0 (low >0 (low rank)rank)

Algorithm: H Algorithm: H H .* H .* (W’(W’VV)./(W’)./(W’WHWH))

W W W .* W .* ((VVH’)./(H’)./(WHWHH’)H’)

Original Application: EigenfacesOriginal Application: Eigenfaces

Another Example: Color ScienceAnother Example: Color Science

OutlineOutline



Color Science: Light Spectra from Color Science: Light Spectra from filmfilm

Reds Greens Blues

Grays

wavelength vs densitywavelength vs density

Film Recording and Film Recording and measurementsmeasurements

Reds

• Solid colors sent to film recorder, e.g. redsSolid colors sent to film recorder, e.g. reds

• Negative is produced: film appears as cyansNegative is produced: film appears as cyans

• Negative sent through projector to spectrometerNegative sent through projector to spectrometer

• Energy data at each Energy data at each wavelengthwavelength• Log ratio with no film (only Log ratio with no film (only bulb)bulb)

film density =film density =

log(no film / with film)log(no film / with film)

The DataThe Data• Inputs (r,g,b) for 1Inputs (r,g,b) for 1r,g,b r,g,b 10 scaled (1000 10 scaled (1000 frames)frames)• Output Space: Densities at 400:3:700 nm’sOutput Space: Densities at 400:3:700 nm’s• Data Structure: 101 x 1000 matrix “A”Data Structure: 101 x 1000 matrix “A”• Compute SVD(A)Compute SVD(A)

indexindex

svd

svd

•Project onto best 3 spaceProject onto best 3 space

Three significantThree significant singular valuessingular values

SVD Basis = no physical SVD Basis = no physical meaningmeaning

The NNMF Basis = primary colorsThe NNMF Basis = primary colors

OutlineOutline• Geometrical Matrix ComputationsGeometrical Matrix Computations• Illustration with 2x2 matrices:Illustration with 2x2 matrices:• Excursions into eigenland (or why tangency Excursions into eigenland (or why tangency


Singular 2x2 Matrices (by svd)Singular 2x2 Matrices (by svd)sin

cos 0 sin co

cos cos

s

sinA

sin

and scales cone

_

A=A=(( ))cos cos cos cos cos cos sin sin -sin -sin cos cos -sin -sin sin sin

All isoeig surfaces are translates of the All isoeig surfaces are translates of the =0 surface!=0 surface!

hyperpolas and hyperboloids are cross sections!hyperpolas and hyperboloids are cross sections!

Torus!Torus!

Torus Cone?Torus Cone?

Bohemian DomeBohemian Dome

Linear Algebra with moviesLinear Algebra with movies

A=QA=QQQTT A=Q A=QQ A=QRQ A=QRHopf FibrationHopf Fibration

Horizontal Vertical Horizontal Vertical VillarceauVillarceau

Incorporate 3d graphics tools directly into Matrix Incorporate 3d graphics tools directly into Matrix computations. Include geometry of matrix space.computations. Include geometry of matrix space.

How should this look?How should this look?

Generalize everything and incorporate into softwareGeneralize everything and incorporate into software

ChallengesChallenges

applied geometrical matrix computations

Documents