advances in random matrix theory (stochastic eigen analysis)

04/19/23 1

Advances in Random Matrix Theory(stochastic eigenanalysis)

Alan Edelman

MIT: Dept of Mathematics,

Computer Science AI Laboratories

04/19/23 2

Stochastic EigenanalysisCounterpart to stochastic differential equationsEmphasis on applications to engineering & financeBeautiful mathematics:

Random Matrix TheoryFree Probability

Raw Material fromPhysicsCombinatoricsNumerical Linear AlgebraMultivariate Statistics

3

Scalars, Vectors, Matrices Mathematics: Notation = power & less ink! Computation: Use those caches! Statistics: Classical, Multivariate, Modern Random Matrix Theory

The Stochastic Eigenproblem * Mathematics of probabilistic linear algebra * Emerging Computational Algorithms

* Emerging Statistical Techniques

Ideas from numerical computation that stand the test of time are right for mathematics!

4

Open Questions

Find new applications of spacing (or other) statistics

Cleanest derivation of Tracy-Widom? “Finite” free probability? Finite meets infinite

Muirhead meets Tracy-Widom Software for stochastic eigen-analysis

5

Wigner’s Semi-Circle The classical & most famous rand eig theorem Let S = random symmetric Gaussian

MATLAB: A=randn(n); S=( A+A’)/2; S known as the Hermite Ensemble Normalized eigenvalue histogram is a semi-circle

Precise statements require n etc.

6




n x n iid standard normals

7




8

Wigner’s original proof Compute E(tr A2p) as n∞ Terms with too many indices, have some element with

power 1. Vanishes with mean 0. Terms with too few indices: not enough to be relevant as

n∞ Leaves only a Catalan number left: Cp=(2p)/(p+1) for the

moments when all is said and done Semi-circle only distribution with Catalan number

moments

p

9

Finite Versions of semicirclen=2; n=4;

n=3; n=5;

10

Finite Versionsn=2; n=4;

n=3; n=5;Area under curve (-∞,x): Can be expressed as sums of probabilities that certain tridiagonal determinants are positive.

11

Wigner’s Semi-Circle Real Numbers: x β=1 Complex Numbers: x+iy β=2 Quaternions: x+iy+jz+kw β=4 β=2½? x+iy+jz β=2½? Defined through joint eigenvalue density:

const x ∏|xi-xj|β ∏exp(-xi2 /2)

β=repulsion strengthβ=0 “no interference” spacings are Poisson

Classical research only β=1,2,4 missing the link to Poisson, continuous techniques, etc

12

Largest eigenvalue

“convection-diffusion?”

13

Haar or not Haar?“Uniform Distribution on orthogonal matrices”Gram-Schmidt or [Q,R]=QR(randn(n))

14

Haar or not Haar?“Uniform Distribution on orthogonal matrices”Gram-Schmidt or [Q,R]=QR(randn(n))

Eigenvalues Wrong

15

Longest Increasing Subsequence(n=4) (Baik-Deift-Johansson)

(Okounkov’s proof)

1 2 3 4 2 1 3 4 3 1 2 4 4 1 2 3

1 2 4 3 2 1 4 3 3 1 4 2 4 1 3 2

1 3 2 4 2 3 1 4 3 2 1 4 4 2 1 3

1 3 4 2 2 3 4 1 3 2 4 1 4 2 3 1

1 4 2 3 2 4 1 3 3 4 1 2 4 3 1 2

1 4 3 2 2 4 3 1 3 4 2 1 4 3 2 1

Green: 4 Yellow: 3 Red: 2 Purple: 1

16

Bulk spacing statistics

Bus wait times in Mexico Energy levels of heavy atoms Parked Cars in London Zeros of Riemann zeta Mice Brain Wave Spikes

Telltale Sign: Repulsion + optimality

“convection-diffusion?”

17

“what’s my β?”web page

http://people.csail.mit.edu/cychan/BetaEstimator.html

Cy’s tricks:• Maximum Likelihood Estimation• Bayesian Probability• Kernel Density Estimation

• Epanechnikov kernel• Confidence Intervals

http://people.csail.mit.edu/~cychan/BetaEstimator.html

18

Open Questions

Find new applications of spacing (or other) distributions



04/19/23 19

Everyone’s Favorite Tridiagonal

-2 1

1 -2 1

1

1 -2

… …

…

…

…1n2

d2

dx2

04/19/23 20

Everyone’s Favorite Tridiagonal

-2 1

1 -2 1

1

1 -2

… …

…

…

…1n2

d2

dx2

1(βn)1/2+

G

G

G

dWβ1/2+

21

Stochastic Operator Limit

,

N(0,2)χ

χN(0,2)χ

χN(0,2)χ

χN(0,2)

nβ2

1 ~ H

β

β2β

2)β(n1)β(n

1)β(n

βn

,dW β

2 x

dxd

2

2

, Gβ

2 H H nn

βn

… … …

Cast of characters: Dumitriu, Sutton, Rider

22

Open Questions




23

Is it really the random matrices?

The excitement is that the random matrix statistics are everyhwere

Random matrices properly tridiagonalized are discretizations of stochastic differential operators!

Eigenvalues of SDO’s not as well studied Deep down this is what I believe is the important

mechanism in the spacings, not the random matrices! (See Brian Sutton thesis, Brian Rider papers—connection to Schrodinger operators)

Deep down for other statistics, though it’s the matrices

24

Open Questions




26

Open Questions




27

Free Probability

Free Probability (name refers to “free algebras” meaning no strings attached)

Gets us past Gaussian ensembles and Wishart Matrices

28

The flipping coins example

Classical Probability: Coin: +1 or -1 with p=.5

-1.5 -1 -0.5 0 0.5 1 1.5 -1.5 -1 -0.5 0 0.5 1 1.5

y:

-1 +1 -1 +1

x+y:

50% 50% 50% 50%

x:

-2 0 +2

29

The flipping coins example

Classical Probability: Coin: +1 or -1 with p=.5

-1.5 -1 -0.5 0 0.5 1 1.5 -1.5 -1 -0.5 0 0.5 1 1.5

eig(B):

-1 +1 -1 +1

eig(A+QBQ’):

50% 50% 50% 50%

eig(A):

-2 0 +2

Free

30

From Finite to Infinite

31


Gaussian (m=1)

32


Gaussian (m=1)

Wiggly

33


Gaussian (m=1)

Wiggly

Wigner

34

Semi-circle law for different betas

35

Open Questions




36

Matrix Statistics

•Many Worked out in 1950s and 1960s•Muirhead “Aspects of Multivariate Statistics”

•Are two covariance matrices equal?•Does my matrix equal this matrix?•Is my matrix a multiple of the identity?

•Answers Require Computation of•Hypergeometrics of Matrix Argument

•Long thought Computationally Intractible

37

The special functions of multivariate statistics

Hypergeometric Functions of Matrix Argument β=2: Schur Polynomials Other values: Jack Polynomials Orthogonal Polynomials of Matrix Argument

Begin with w(x) on I ∫ pκ(x)pλ(x) Δ(x)β ∏i w(xi)dxi = δκλ Jack Polynomials orthogonal for w=1 on the unit circle.

Analogs of xm

Plamen Koev revolutionary computation Dumitriu’s MOPS symbolic package

38

Multivariate Orthogonal Polynomials&

Hypergeometrics of Matrix Argument

The important special functions of the 21st century Begin with w(x) on I

∫ pκ(x)pλ(x) Δ(x)β ∏i w(xi)dxi = δκλ

Jack Polynomials orthogonal for w=1 on the unit circle. Analogs of xm

39

Smallest eigenvalue statistics

A=randn(m,n); hist(min(svd(A).^2))

40

Multivariate Hypergeometric Functions

41

Multivariate Hypergeometric Functions

42

Open Questions




43

Plamen Koev’s clever idea

44

A=randn(n); S=(A+A’)/2; trace(S^4)

det(S^3)

Symbolic MOPS applications

45

Mops (Ioana Dumitriu) Symbolic

46

Random Matrix Calculator

47

Encoding the semicircleThe algebraic secret

f(x) = sqrt(4-x2)/(2π) m(z) = (-z + i*sqrt(4-z2))/2 L(m,z) ≡ m2+zm+1=0

m(z) = ∫ (x-z)-1f(x) dx Stieltjes transform

-3 -2 -1 0 1 2 30

0.05

0.1

0.15

0.2

0.25

0.3

0.35

x

Pro

ba

bili

ty

Practical encoding: Polynomial L whose root m is Stieltjes transform

48

The Polynomial Method

RMTool http://arxiv.org/abs/math/0601389

The polynomial method for random matrices

Eigenvectors as well!

49

Plus

-3 -2 -1 0 1 2 30

0.05

0.1

0.15

0.2

0.25

0.3

0.35

x

Pro

babi

lity

0 0.5 1 1.5 2 2.5 30

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

x

Pro

babi

lity

+

X =randn(n,n)A=X+X’

m2+zm+1=0

Y=randn(n,2n)B=Y*Y’

zm2+(2z-1)m+2=0

-2 -1 0 1 2 3 40

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

x

Pro

babi

lity

A+Bm3+(z+2)m2+(2z-1)m+2=0

50

Times

-3 -2 -1 0 1 2 30

0.05

0.1

0.15

0.2

0.25

0.3

0.35

x

Pro

babi

lity

0 0.5 1 1.5 2 2.5 30

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

x

Pro

babi

lity

*

X =randn(n,n)A=X+X’

m2+zm+1=0

Y=randn(n,2n)B=Y*Y’

zm2+(2z-1)m+2=0

-2 -1 0 1 2 3 40

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

x

Pro

babi

lity

A*Bm4z2-2m3z+m2+4mz+4=0

-3 -2 -1 0 1 2 30

0.1

0.2

0.3

0.4

0.5

0.6

0.7

x

Pro

ba

bili

ty

51

Open Questions




52

Matrix Versions of Classical Stats

Hermite Sym Eig eig(A+A’)Normal

Laguerre SVD eig(A*A’)Chi-squared

Jacobi GSVD gsvd(A,B)Beta

Fourier Eig [U,R]=qr(A+i*B)

Orthog Matrix MATLAB (A=randn(n) B=randn(n))

53

The big structure

Hermite Sym Eig exp(-x2) Normal Complete Graph

non-compactA,AI,AII

Laguerre SVD xαe-xChi-squared

Bipartite Graph

non-compactAIII,BDI,CII

Jacobi GSVD(1-x)α x

(1+x)βBeta Regular

Graph

compactA, AI, AII, C, D, CI, D, DIII

Fourier Eig eiθcompactAIII, BDI, CDI

Orthog Matrix Weight Stats Graph Theory SymSpace

54

Summary

Stochastic Eigenanalysis Emerging Techniques Open Problems

advances in random matrix theory (stochastic eigen analysis)

Documents

randnn s

semicircleprecise statements

x iy jz

famous rand eig theoremlet

dept of mathematics

financebeautiful mathematics

uniform distribution

joint eigenvalue density