imperial college london, july 2016 khalil elkhalil khalil.elkhalil ... · khalil elkhalil...

39
Gram Matrices and Statistical Signal Processing Imperial College London, July 2016 Khalil Elkhalil [email protected] Computer, Electrical, Mathematical Sciences and Engineering (CEMSE) 1/34

Upload: others

Post on 11-Aug-2020

4 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

Gram Matrices and Statistical Signal Processing

Imperial College London, July 2016

Khalil [email protected]

Computer, Electrical, Mathematical Sciences and Engineering (CEMSE)

1/34

Page 2: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

2/34

A work done with

▸ Dr Abla Kammoun (Research scientist, KAUST).

▸ Prof. Tareq Y. Al-Naffouri (Associate Professor, KAUST).

▸ Prof. Mohamed-Slim Alouini (Professor, KAUST).

Page 3: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

3/34

Outline

Some history on RMT

Background on Random Matrix Theory (RMT)

Inverse Moments of One-Sided Correlated Gram Matrices

Blind Measurement Selection

Summary

Page 4: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

4/34

Some history on RMT

Background on Random Matrix Theory (RMT)

Inverse Moments of One-Sided Correlated Gram Matrices

Blind Measurement Selection

Summary

Page 5: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

4/34

▸ All started with the work of E. Wigner in the 1950’s in nuclear physics.

▸ High dimensional probability and statistics (Tao, Marcenko, Pastur...)

▸ Electrical Eng. at the end of the 1990’s (Verdu, Tse, Debbah,...)

▸ Recently applied in the field of statistical signal processing and robustestimation (Mestre, Loubaton, Couillet,...)

▸ Applied in Machine Learning (2016, Couillet and Kammoun).

▸ The journey continues ...

Page 6: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

5/34

Some history on RMT

Background on Random Matrix Theory (RMT)

Inverse Moments of One-Sided Correlated Gram Matrices

Blind Measurement Selection

Summary

Page 7: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

5/34

Random matrices

XN =⎡⎢⎢⎢⎢⎢⎣

X1,1 ⋯ X1,N

⋮ ⋮XN,1 ⋯ XN,N

⎤⎥⎥⎥⎥⎥⎦.

where the entries Xi,j are random variables.Usually we’re interested in the following quantities

▸ The spectrum (The distribution of the λi ’s, λmin and λmax ).

▸ Some useful statistics involving the eigenvalues of XN

E∑i

f (λi)

f (x) ∶1

xp(p ∈ Z), log x, ...

▸ Eigenvectors, ⋯Two approaches to obtain the statistics of interest

▸ Exact Approach (usually hard /).

▸ Asymptotic Approach (usually feasible and leads to simple expressions ,).

m,n →∞,m

n→ c ∈ (0,∞) .

Page 8: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

6/34

Marcenko-Pastur’s Theorem (Wishart matrices)Let X ∈ Cm×n a Gaussian random matrix with i.i.d zero mean unit variance entries andW = 1

nXX∗ with spectral measure

E (W) =1

m∑i

δλi (W).

0 0.5 1 1.5 2 2.5 3spectrum

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Distribution

HistogramMP(x)

m = 1200n = 3000c = 0.4

Let λ− = (1 −√c)2 and λ+ = (1 +

√c)2. Then as (m,n)→∞ with m

n→ c ∈ (0,∞)

E (W) dÐÐÐÐÐ→m,n→+∞

PMP(dx) = (1 −1

c)+δ0(dx) +

√(λ+ − x)(x − λ−)

2πxc1[λ−,λ+](x)dx .

Page 9: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

7/34

Some history on RMT

Background on Random Matrix Theory (RMT)

Inverse Moments of One-Sided Correlated Gram Matrices

Blind Measurement Selection

Summary

Page 10: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

7/34

Gram random matrices

S = H∗H

= (Λ12 X)

∗(Λ

12 X)

= X∗ΛX.

where X ∈ Cn×m with i.i.d zero mean unit variance Gaussian entries and Λ is positivedefinite matrix with distinct eigenvalues θ1, θ2,⋯, θn.This kind of matrices may arise in the following context

y = Hv + z, m < n.

The error covariance matrix after applying least squares (LS)

S−1 = (H∗H)−1.

The mean square error is given by

MSE = E trace (H∗H)−1

= E∑i

λ−1i (H∗H)

f (x) =1

x.

Can we derive the MSE in closed form ? (Exact Approach)

Page 11: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

8/34

In this particular case: Yes, we can ,Mellin transform approach

Mfλ(s) ≜ ∫∞

0ξs−1fλ (ξ)dξ. (1)

E trace (H∗H)−p = lims→0Mfλ(s − p + 1)

Lemma 1[1]

Mfλ(s) = L

m

∑j=1

m

∑i=1

D (i, j)Γ (s + j − 1)⎛⎝θn−m+s+j−2n−m+i

−n−m∑l=1

n−m∑k=1

[Ψ−1]

k,lθn−m+s+j−2l

θk−1n−m+i

⎞⎠,

(2)

with L = det(Ψ)m∏n

k<l (θl−θk )∏m−1l=1

l!, Γ(.) the Gamma function and Ψ is the (n −m)× (n −m) Vandermonde matrix

given by

Ψ =⎡⎢⎢⎢⎢⎢⎣

1 θ1 ⋯ θn−m−11

⋮ ⋮ ⋱ ⋮1 θn−m ⋯ θn−m−1

n−m

⎤⎥⎥⎥⎥⎥⎦

and D (i, j) is the (i, j)−cofactor of the (m ×m) matrix C whose (l, k)−th entry is given by

[C]l,k = (k − 1)!⎛⎝θn−m+k−1n−m+l −

n−m∑p=1

n−m∑q=1

[Ψ−1]

p,qθp−1n−m+lθ

n−m+k−1q

⎞⎠.

[1] G. Alfano et al “Capacity of MIMO Channels with One-Sided Correlation,” ISSSTA, 2004.

Page 12: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

9/34

▸ p ≤ 0: straightforward.

▸ p > 0: singularity issues arise.

→ Divide and conquer

Mfλ(s − p + 1) =M1 (s − p + 1) +M2 (s − p + 1) , (3)

where

M1 (s) = Lp

∑j=1

m

∑i=1

D (i , j)Γ (s + j − 1)⎛⎝θn−m+s+j−2n−m+i

−n−m∑l=1

n−m∑k=1

[Ψ−1]k,lθn−m+s+j−2l

θk−1n−m+i

⎞⎠.

M2 (s) = Lm

∑j=p+1

m

∑i=1

D (i , j)Γ (s + j − 1)⎛⎝θn−m+s+j−2n−m+i

−n−m∑l=1

n−m∑k=1

[Ψ−1]k,lθn−m+s+j−2l

θk−1n−m+i

⎞⎠.

Page 13: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

10/34

Good news

lims→0M2(s − p + 1) = 0.

Proof

lims→0M2 (s − p + 1)

= Lm

∑j=p+1

m

∑i=1

D (i , j)Γ (−p + j)

×⎛⎝θn−m−p+j−1n−m+i −

n−m∑l=1

n−m∑k=1

[Ψ−1]k,lθn−m−p+j−1l

θk−1n−m+i

⎞⎠

= Lm

∑j=p+1

m

∑i=1

[D]i,j [C]i,j−p

= Lm

∑j=p+1

[DtC]j,j−p ,

where D and C are as defined in Lemma 1. Since D is the cofactor of C, thenDtC = det (C) Im. Therefore, [DtC]j,j−p = 0 for j = p + 1,⋯,m.

Page 14: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

11/34

The first moment is more involved!

aj = [θn−m−p+j−11 , θn−m−p+j−1

2 ,⋯, θn−m−p+j−1n−m ]

t.

Di = diag

⎡⎢⎢⎢⎢⎣log(

θn−m+iθ1

) , log(θn−m+iθ2

) ,⋯, log (θn−m+iθn−m

)⎤⎥⎥⎥⎥⎦.

bi ≜ [1, θn−m+i ,⋯, θn−m−1n−m+i ]

t.

Proposition [1] Let q = min (m,n −m), then for 1 ≤ p ≤ q we have

lims→0M1 (s − p + 1)

= Lp

∑j=1

m

∑i=1

D (i , j)(−1)p−j

(p − j)!bti Ψ

−1Diaj .

[1] K. Elkhalil et al “Analytical Derivation of the Inverse Moments of One-Sided Correlated Gram Matrices,” IEEETransactions on Signal Processing, 2016.

Page 15: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

12/34

Elements of the Proof

▸ Some useful notations

Ψs ≜⎡⎢⎢⎢⎢⎢⎣

θs1 θ1+s1 ⋯ θn−m+s−1

1⋮ ⋮ ⋱ ⋮

θsn−m θ1+sn−m ⋯ θn−m+s−1

n−m

⎤⎥⎥⎥⎥⎥⎦,Ψs = Ψ + o(s).

as,j ≜⎡⎢⎢⎢⎢⎣θn−m+s−p+j−1

1 , θn−m+s−p+j−12 ,⋯, θn−m+s−p+j−1

n−m

⎤⎥⎥⎥⎥⎦

t

.

bs,i ≜ [θsn−m+i , θ1+sn−m+i ,⋯, θ

n−m+s−1n−m+i ]t .

bi ≜ [1, θn−m+i ,⋯, θn−m−1n−m+i ]

t.

ek ≜ [01×(n−m−k−1),1,01×k] t ,

k = 0,⋯,n −m − 1,

Page 16: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

13/34

Main idea: It starts with a simple observation

Ψs ≜⎡⎢⎢⎢⎢⎢⎣

θs1 θ1+s1 ⋯ θn−m+s−p+j−1

1 θn−m+s−11

⋮ ⋮ ⋱ ⋮ ⋮θsn−m θ1+s

n−m ⋯ θn−m+s−p+j−1n−m θn−m+s−1

n−m

⎤⎥⎥⎥⎥⎥⎦,

as,j ≜⎡⎢⎢⎢⎢⎣θn−m+s−p+j−1

1 , θn−m+s−p+j−12 ,⋯, θn−m+s−p+j−1

n−m

⎤⎥⎥⎥⎥⎦

t

.

bs,i ≜ [θsn−m+i , θ1+sn−m+i ,⋯, θ

n−m+s−1n−m+i ]t .

▸ Ψsep−j = as,j and bts,iep−j = θ

n−m+s−p+j−1n−m+i Ô⇒ Ψ−1

s as,j = ep−j and consequently

bts,iΨ

−1s as,j = θn−m+s−p+j−1

n−m+i .

Therefore,

θn−m+s−p+j−1n−m+i − bt

s,iΨ−1s as,j = 0.

Page 17: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

14/34

M1 (s − p + 1) = Lp

∑j=1

m

∑i=1

D (i, j)Γ (s − p + j)⎛⎝θn−m+s−p+j−1n−m+i −

n−m∑l=1

n−m∑k=1

[Ψ−1]

k,lθn−m+s−p+j−1l

θk−1n−m+i

⎞⎠

= Lp

∑j=1

m

∑i=1

D (i, j)Γ (s − p + j)⎛⎝θn−m+s−p+j−1n−m+i − b

ti Ψ−1

as,j⎞⎠

Substract and add bti Ψ−1s as,j

= Lp

∑j=1

m

∑i=1

D (i, j)Γ (s − p + j)⎛⎝θn−m+s−p+j−1n−m+i − b

ti Ψ−1s as,j

⎞⎠+ L

p

∑j=1

m

∑i=1

D (i, j)Γ (s − p + j)

× bti (Ψ

−1s −Ψ

−1) as,j

Substract and add bts,iΨ−1s as,j

= Lp

∑j=1

m

∑i=1

D (i, j)Γ (s − p + j)⎛⎝θn−m+s−p+j−1n−m+i − b

ts,iΨ

−1s as,j

⎞⎠+ L

p

∑j=1

m

∑i=1

D (i, j)Γ (s − p + j)

× (bts,i − b

ti )

´¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¸¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¶Ψ−1s as,j + L

p

∑j=1

m

∑i=1

D (i, j)Γ (s − p + j) bti (Ψ

−1s −Ψ

−1)´¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¸¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¶

as,j

Page 18: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

15/34

▸ θs − 1 =s→0

s log θ + o(s).

bs,i − bi = s

⎡⎢⎢⎢⎢⎣log (θn−m+i) , θn−m+i log (θn−m+i) ,⋯, θn−m−1

n−m+i log (θn−m+i)⎤⎥⎥⎥⎥⎦

t

+ o (s)

= s log (θn−m+i)bi + o (s) ,

▸ For non positive arguments −k, k = 0,1,⋯,

Γ(s − p + j) =s→0

(−1)p−j

s(p − j)!+ o(s).

Γ (s − p + j) (bts,i − bt

i )Ψ−1s as,j

=(−1)p−j log (θn−m+i)

(p − j)!bti Ψ

−1aj + o(s)

▸ The resolvent identity

Ψ−1s −Ψ−1 = Ψ−1

s (Ψ −Ψs)Ψ−1.

▸ Finally,

(Ψ −Ψs) =s→0

−sΦΨ + o(s), Φ = diag (log θi , i = 1,⋯n −m) .

Page 19: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

16/34

Possible Applications

▸ Performance of linear estimators such as the LS (correlated channel), or thebest linear unbiased estimator (BLUE) when the noise in correlated.

y = Hv + z, (4)

vblue = (H∗Σ−1z H)−1

H∗Σ−1z y

= v + (H∗Σ−1z H)−1

H∗Σ−1z z

= v + eblue ,

(5)

EH{∥vblue − v∥2} = EHtrace (Σe,blue)

= EHtrace ((H∗Σ−1z H)−1)

= mµΛ (−1) ,

(6)

where Λ = Σ−1z .

Page 20: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

17/34

▸ Approximation of the LMMSE error

EH{∥xlmmse − x∥2} = EHtrace (Σe,lmmse)

= EHtrace (Σ−1x +H∗Σ−1

z H)−1.

(7)

TheoremLet Λ = Σ−1

z and Σx = σ2x I. Then, the LMMSE average estimation error at both the

high SNR regime (σ2x ≫ 1) and the low SNR regime (σ2

x ≪ 1) is given by

1. High SNR regime

EH{∥xlmmse − x∥2}

= mp

∑k=0

(−1)k

σ2kx

µΛ (−k − 1) + o (σ−2px ) ,

(8)

where p ≤ q − 1 with q = min(m,n −m).

2. Low SNR regime

EH{∥xlmmse − x∥2} = m∞∑k=0

(−1)k σ2k+2x µΛ (k) . (9)

Page 21: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

18/34

σx2 in dB

-50 -40 -30 -20 -10 0 10 20 30

LMM

SE

err

or v

aria

nce

in d

B

-100

0

100

200

300

400

500Montecarlo simulation (104 realizations)Theoretical approximation (Low SNR)Theoretical approximation (High SNR)

Figure: LMMSE mean square error with Σz modeled as Bessel correlationmatrix: Montecarlo simulation versus theoretical approximations.

σx

2 in dB

-50 -40 -30 -20 -10 0 10 20 30

LM

MS

E e

rror

variance in d

B

-50

0

50

100

150

200

250

300

350

400

Montecarlo simulation (104 realizations)

Theoretical approximation (Low SNR)

Theoretical approximation (High SNR)

Figure: LMMSE mean square error with Σz modeled as randomcorrelation matrix: Montecarlo simulation versus theoreticalapproximations.

Page 22: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

19/34

▸ Sample Correlation Matrix (SCM)

R = E [uu∗] ,

where u (k) = R12 x(k) and x(k) ∼ CN (0, Im).

To estimate R, we refer to the SCM as

R (n) =1

n

n

∑k=1

u (k)u∗ (k)

*The SCM is the maximum likelihood (ML) estimator of R.* For fixed m and large n

∥R − R (n)∥→ 0, a.s.

The SCM may not be robust for finite dimensions !Exponentially weighted SCM

R (n) = (1 − λ)n

∑k=1

λn−ku (k)u∗ (k)

= R12 XΛ (n)X∗R

12 ,

where Λ(n) = (1 − λ)diag (λn−1, λn−2,⋯,1).

Loss (n) ≜ E ∥R12 R−1 (n)R

12 − Im∥

2

Fro.

Page 23: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

20/34

LemmaLet Sn = XΛ (n)X∗. Then, the loss can be expressed as a function of inverse momentsof Sn as

Loss (n) = m (1 + µΛ(n) (−2) − 2µΛ(n) (−1)) . (10)

How to choose λ ?

λ

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

Lo

ss in

dB

15

20

25

30

35

40

45

50

55

60m=3, n=5

m=3, n=6

m=3, n=7

Figure: The estimation loss as a function of the forgetting factorλ.

Page 24: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

21/34

Large RMT (Asymptotic Approach)The Stieltjes Transform (ST)For a probability measure P, the associated ST is defined as

SP(z) = ∫R

P(dλ)λ − z

, z ∈ C+

Spectral Measure

E (A) =1

m∑i

δλi.

Then,

SE(z) =1

m∑i

1

λi − z

=1

mtrace (A − zI)−1 .

It follows that,

(∂kSE(z)∂zk

)z=0

=k!

mtr (A−(k+1)) .

Page 25: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

22/34

Theorem (Silverstein and Bai) Consider the Gram matrix S = 1m

X∗ΛX with ST s(z).

▸ The entries of X are Gaussian i.i.d with zero mean and unit variance.

▸ m,n →∞ with mn→ c ∈ (0,∞).

▸ Λ is nonnegative definite matrix with

E(Λ) n→∞ÐÐÐ→ dH(τ).

Then, the ST of S, s(z) satisfies

s(z) = (−z + c ∫τdH(τ)

1 + τs(z))−1

(11)

≈ (−z + c.trace [Λ (I + s(z)Λ)−1])−1(12)

=1

−z + 1m ∑

nk=1

λk1+λk s(z)

. (13)

s(k)0 = (

∂k s(z)∂zk

)z=0

=k!

mtr (S−(k+1)) .

How to obtain the higher order moments ?

Page 26: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

23/34

Higher order moments

Let p ≥ 1 and fk (z) = − 11+[D]k,k s(z)

. Denote by f(p)k

the p-th derivative of fk(z) at

z = 0. Then, the following relations hold true:

ps(p−1)0 +

s(p)0

m

n

∑k=1

[D]k,k f(0)k

1 + [D]k,k s (0)

+1

m

n

∑k=1

p−1

∑l=1

(p

l)[D]k,k s

(l)0 f

(p−l)k

1 + [D]k,k s (0)= 0,

(14)

f(p)k

+[D]k,k s

(p)0 f

(0)k

1 + [D]k,k s (0)+

p−1

∑l=1

(p

l)[D]k,k s

(l)0 f

(p−l)k

1 + [D]k,k s (0)= 0. (15)

Algorithm 1 Asymptotic inverse moments computation

1: Compute s (0) using (22)2: Compute fk (0) = − 1

1+[D]k,k s(0)3: for i = 1→ p do

4: compute s(i)0 using (14)

5: compute f(i)k

using (15)

Page 27: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

24/34

Numerical validation

n

7 8 9 10 11 12

µΛ(r)in

dB

-9

-8

-7

-6

-5

-4r = −1

Theoretical Formula (Theorem 1)Monte Carlo simulationNormalized asymptotic moments (Theorem 2)

n

7 8 9 10 11 12

µΛ(r)in

dB

-16

-14

-12

-10

-8

-6

-4r = −2

Theoretical Formula (Theorem 1)Monte Carlo simulationNormalized asymptotic moments (Theorem 2)

n

7 8 9 10 11 12

µΛ(r)in

dB

-25

-20

-15

-10

-5

0r = −3

Theoretical Formula (Theorem 1)Monte Carlo simulationNormalized asymptotic moments (Theorem 2)

n

7 8 9 10 11 12µΛ(r)in

dB

-30

-20

-10

0

10r = −4

Theoretical Formula (Theorem 1)Monte Carlo simulationNormalized asymptotic moments (Theorem 2)

Figure: Inverse moments for Λ modeled as Bessel Correlation matrix: Acomparison between theoretical result (Theorem 1), normalizedasymptotic moments (Theroem 2) and Monte Carlo simulations (105

realizations).

Page 28: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

25/34

Some history on RMT

Background on Random Matrix Theory (RMT)

Inverse Moments of One-Sided Correlated Gram Matrices

Blind Measurement Selection

Summary

Page 29: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

25/34

Consider

y = Hx + z, z ∼ CN (0,R) .

The covariance of the estimation error vector is given by

Σ = (H∗R−1H)−1.

Popular measures for the quality of estimation

▸ Mean square error

MSE = trace (Σ) , f(x) =1

x.

▸ The log volume of the concentration ellipsoid (VCE)

VCE ∝ − log det (Σ) , f(x) = − log x.

▸ Worst case error variance (WEV)

WEV = max∥q∥=1

qT Σq = maxiλi (Σ) = λmax (Σ) ,

Page 30: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

26/34

Measurement selection▸ Antenna selection in massive MIMO systems.

BS

1

2

n

user 2

user m

user 1

Hy x

Antenna selection

▸ Sensor selection.

-100 -50 0 50 100

-100

-80

-60

-40

-20

0

20

40

60

80

100

▸ ⋯

Page 31: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

27/34

y = Hx + z.

yS = Sy, (16)

where S is defined as follows

[S]i,j = {1 j = S [i]0 otherwise

, i = 1,⋯, k. (17)

Ee have the following properties

▸ SST = Ik .

▸ ST S = diag (s).

where s = {si}i=1,⋯,n.The resultant error covariance matrix, which we denote by ΣS , easily writes as

ΣS = (HHST (SRST )−1SH)

−1

= (WHR12 diag (s)R

12 W)

−1.

(18)

How to choose s ?

Page 32: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

28/34

Mathematically speaking, a general formulation of the selection problem can beillustrated as follows

S∗ = argminS

f (ΣS)

s.t. S ⊆ {1,⋯,n}∣S ∣ = k.

(19)

where f = MSE, VCE, WEV.→ NP hard problem (we don’t know an algorithm that can solve the problem in apolynomial time).Convex Relaxation (Joshi and Boyd)

s = argmins

f (s)

s.t. 1T s = k

0 ≤ si ≤ 1, i = 1,⋯,n.

(20)

→ polynomial complexity of O (n3)What if the channel H is time-varying ? Best we can do is NO (n3) so far!

Page 33: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

29/34

Blind Measurement Selection via Large RMTidea: Solve an approximate problemWe solve the following optimization problem

S = argminS

f (ΣS)

s.t. S ⊆ {1,⋯,n}∣S ∣ = k.

(21)

where f (ΣS) is an approximation of f (ΣS) (Deterministic equivalent, DE).Some useful DEs

▸ MSE

s(0) =1

1m ∑

nk=1

λk1+λk s(0)

. (22)

▸ VCE

I = log det(In +c

δD) +m log (δ) − nc, δ =

1

ntr [D(In +

c

δD)

−1

] .

D = diag (θ1,⋯, θn) .

▸ WEV

λ = (−1

m+ ∫

t

1 + cmtν (t)dt) , m2 =

1

c(∫

t2

(1 + cmt)2ν (t)dt)

−1

.

Page 34: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

30/34

Low complexity search algorithm (Greedy approach)

Algorithm 2 Greedy Approach for Antenna Selection

1: Initialize S = randsample(n, k) ▷ randomly generate a pattern of size k2: Compute metric∗ = f (H,S)3: for i = 1→#iterations do4: S = {1,⋯,n} /S5: j ← 16: while j ≤ n − k do7: p ← S [j]8: I ← S9: table← zeros (k,1)

10: for l = 1→ k do11: I [l]← p12: table [l]← f (H,I)13: I ← S14: if min (table) < metric∗ then15: metric∗ ← min (table)16: S [arg min (table)]← p

Complexity of O (n2).

Page 35: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

31/34

Performance

▸ Antennas selection in Massive MIMO: Ri,j = λ∣i−j ∣.

km

2 3 4 5

MSEin

dB

-16

-14

-12

-10

-8

-6

-4

-2Greedy (H = H)

Greedy (H = ΘR)

Convex Optimization

km

2 3 4 5Run-T

ime(secon

ds)

10-2

100

102

104Greedy (H = H)

Greedy (H = ΘR)

Convex Optimization

Figure: Average MSE (over N = 100 channel realizations) and Run-time vs. the ratio km

for ΘR =Toeplitz (λ = 0.75). m = 10 users, n = 1.25k and ρ = 10 dB.

Page 36: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

32/34

▸ Sensor selection:

θ/2

Sensor node

Correlation range

Correlation exists

SiSj

di,j = ‖Si − Sj‖

Ri,j =cos−1(

di,jθ)

π−

di,j

πθ2

√θ2 − d2

i,j, 0 ≤ di,j ≤ θ

= 0 di,j > θ.

15 20 25 30 35# of active sensors (k)

-7.2

-7

-6.8

-6.6

-6.4

-6.2

-6

VCE

15 20 25 30 35# of active sensors (k)

-30

-28

-26

-24

-22

MSE

indB

15 20 25 30 35# of active sensors (k)

-40

-35

-30

-25

WEV

indB Exact (CVX)

DE (G)DE (CVX)

Page 37: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

33/34

Some history on RMT

Background on Random Matrix Theory (RMT)

Inverse Moments of One-Sided Correlated Gram Matrices

Blind Measurement Selection

Summary

Page 38: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

33/34

▸ Exact derivation of the inverse moments of Gram matrices with one-sidedcorrelation.

▸ Correlation can be exploited to figure out the potential measurements in a linearsystem.

▸ Good performances in some practical scenarios (massive MIMO and WSN).

▸ The case of identical eigenvalues is still an open question.

Page 39: Imperial College London, July 2016 Khalil Elkhalil khalil.elkhalil ... · Khalil Elkhalil khalil.elkhalil@kaust.edu.sa Computer, Electrical, Mathematical Sciences and Engineering

34/34

Thank you for your attention