lecture #7.1 random vectors: 2 moment theory, continued d. …dvanalp/ece...

D. van Alphen 1

ECE 650 – Lecture #7.1

Random Vectors: 2nd Moment Theory,

continued

D. van Alphen

[Ref: Lecture Notes of Dr. Robert Scholtz, USC]

D. van Alphen 2

Lecture Overview

• Review of 2nd-Moment Theory from Lecture 7

• Spectral Decomposition Example

• Aside - Signal Space Concepts

• Communication Applications for 2nd Moment Theory

– Binary Signal Detection in Additive White Noise

– Binary Signal Detection in Colored Noise

• Case 1: Noise with a Singular Covariance Matrix

• Case 2: Noise with a Non-singular Covariance Matrix

D. van Alphen 3

Lecture 7 - Review

• Linear Transformation matrix H is causal if it is lower triangular

– Can use Cholesky Factorization to obtain a causal H for C

= HHT if C is positive definite.

• Spectral Resolution (or Decomposition, or “Mercer’s

Theorem”) for Covariance Matrix:

CY =

• Mean squared length of R. Vector Y:

Ti

n

1iii ee

n

1ii

2)R(tr}{E YY

D. van Alphen 4

Lecture 7 – Review, continued

• The mean-squared length of the projection of Y0 onto b is:

• The Preferred direction of R. Vector Y is given by the e-

vector emax (corresponding to the largest eigenvalue of CY.)

• Scholtz Peanut: plot of the RMS length of random vectors with

a given covariance matrix, in various directions specified by

unit-length vectors.

– Corresponds to basic shape of scatter plot for the random

vectors, showing the directional preference of the vectors

• Whitening Filter for colored noise: G = H-1 = L-1/2 ET

bT CY b (b: unit-length)

D. van Alphen 5

Preliminary Definitions for Signal Space

- Orthogonalilty -

• Two functions y1(t) and y2(t) are orthogonal on the interval

(0, T) if:

• Example: Let y1(t) and y2(t) be the waveforms shown below,

on the interval (0, 2):

yyT

021 0dt)t()t(

t

y1(t) y2(t)

0 1 2 0 1 2t

1 1

D. van Alphen 6

Preliminary Definitions for Signal Space

• A set of functions {yi(t)} is orthogonal on the interval (0, T) if

any pair of distinct functions in the set are orthogonal:

• A set of functions {yi(t)} is orthonormal on the interval (0, T) if:

yyT

0ji ji,0dt)t()t(

yyT

0jkji dt)t()t(

1 if j = k

0 if j k

This means that the functions are orthogonal on (0, T) and that

each function has unit-energy.

D. van Alphen 7

“Easy” Orthogonal Signals

- Sufficient but not Necessary Conditions -

• Claim 1: Any two signals that do not overlap in the time domain

are orthogonal.

• Claim 2: Any two signals that do not overlap in the frequency

domain are orthogonal.

• s1(t) = cos(2pf0t)

• s2(t) = cos(2pf1t)orthogonal if f0 f1

y1(t)

t0 1

y1(t)

t

y2(t)

t0 1 2

Note: these are not

pulsed sinusoids

D. van Alphen 8

Signal Space and Basis Functions

• A set of functions, {yi(t)}, is linearly independent if no function in the set can be written as a linear combination of the others.

• An N-dimensional signal space is a vector space with “vectors” {si(t)}, and characterized by a set of N functions, {yi(t)}, called the basis functions for the space, defined on (0, T), such that:

1. Every function, sk(t), in the signal space can be written as a linear combination of the basis functions, {yi(t)}; and

2. The basis functions, {yi(t)}, are linearly independent and orthonormal on (0, T).

D. van Alphen 9

Signal Space Claim [Ref: Sklar]

• Any set of physically realizable waveforms, {si(t)}, i = 1, 2, …,

M, each of duration T, can be written as a linear combination of

N orthonormal functions, {yi(t)}, i = 1, …, N (N M):

s1(t) = s11y1(t) + s12y2(t) + … + s1N yN(t)

s2(t) = s21y1(t) + s22y2(t) + … + s2N yN(t)

sM(t) = sM1y1(t) + sM2y2(t) + … + sMN yN(t)

...

Generalized _____________ ______________

D. van Alphen 10

Signal Space Concepts

• How do we find the coefficients, sij ?

• Writing signals as vectors – an example

– Say s3(t) = s31y1(t) + s32y2(t) + … + s3N yN(t)

S3 = [s31 s32 s33 … s3N ]

yT

0jiij dt)t()t(ss

Coefficient of yj(t), in

expansion of si(t)

• Every signal si(t) can be written as an N-dimensional vector, Si.

• Energy in si(t) = , the length-squared of the vector Si.

N

1j

2ij

s

D. van Alphen 11

Signal Space Example 1

Basis Functions Orthogonal on (0, 1)? Unit-energy?

y2(t)

t0 .5 1

2

y1(t)

t0 .5 1

2

D. van Alphen 12


y1(t)

t0 .5 1

2

y2(t)

t0 .5 1

2

Basis Functions Signals Vectors (??)

s1(t)

t0 .5 1

2

t

s2(t)

0 .5 , 1

1

-1

t

s3(t)

0 .5 1

32

S1 = [ ____ ____ ]

S2 = [ ____ ____ ]

S3 = [ ____ ____ ]

D. van Alphen 13

Signal Vectors for Example 1

Signals Vectors

s1(t)

t0 .5 1

2

t

s2(t)

0 .5 , 1

1

-1

t

s3(t)

0 .5 1

32

S1 = [ ____ ____ ]

S2 = [ ____ ____ ]

S3 = [ ____ ____ ]

y1

y2

Distance from origin

to ith signal vector:

sqrt(Ei), where Ei is

the energy in the ith

signal

D. van Alphen 14


- Common Basis Functions for IQ Modulation -

• Basis Functions:

• y1(t) =

• y2(t) =

• Orthogonal on (0, T)? (Complete for Homework Set 5)

• Unit-energy?

Tt0),tcos(T

20

Tt0),tsin(T

20

D. van Alphen 15


- QPSK -

• Basis Functions:

– y1(t) =

– y2(t) =

• Signals ??:

Tt0),tcos(T

20

Tt0),tsin(T

20

Signal Vectors ??:

x

xx

xS1S2

S3 S4

Ey1

y2

D. van Alphen 16

2-Dimensional Modem Constellations,

sin and cos basis functions, circa 1990’s

V.32 V.32 bis*

32 Signal Vectors, TCM 128 Signal Vectors, TCM

4 data bits. 1 parity.

9600 bps.

6 data bits. 1 parity.

14.4 kbps.

* bis = repeated, or 2nd version

y1

y2

y1

y2

http://www.computing.dcu.ie/~humphrys/Notes/Networks/tanenbaum/2-26.jpg

http://www.computing.dcu.ie/~humphrys/Notes/Networks/tanenbaum/2-26.jpg

D. van Alphen 17

Communications Application

- Signal Detection in Additive White Noise

• Consider two equally-likely signal vectors, S1 and S2, in

additive white noise ( no directional preference)

• Let ai denote the action of choosing hypothesis Hi (that signal

Si was sent), i = 1, 2

• Receiver’s task: choose between 2 hypotheses in some

optimal way, given any received vector, Y = Si + N

S1

S2

I1I2

radius = RMS proj. of length

of noise in any direction

D. van Alphen 18



• Let Ii denote the decision regions:

– Take action ai if received vector Y falls in region Ii

• Decision boundary, line l : (for this problem): perpendicular

bisector of line connecting S1, S2

S1

S2

I1I2

radius = RMS proj. of length

of noise in any direction

l

D. van Alphen 19



• Minimum Distance Receiver Rule (appropriate for 2 equally-

likely signals in additive white noise):

d(Y) = a1 iff |Y – S1| < |Y – S2|

S1

S2

I1 I2

l

(Maps received vector Y to whichever possibly tx’d signal it

is closest to)

Problem: computing distances is algebraically intense

D. van Alphen 20



• To obtain an algebraically equivalent decision rule:

d(Y) = a1 iff |Y – S1|2 < |Y – S2|

2

iff (Y – S1)T (Y – S1) < (Y – S2)

T (Y – S2)

iff |Y|2 – S1T Y – YT S1 + |S1|

2 < |Y|2 – S2T Y – YT S2 + |S2|

2

iff |S1|2 - 2 {YT S1} < |S2|

2 – 2 {Y T S2}

iff 2 {YT [S1 - S2 ] > |S1|2 - |S2 |2

iff

(M + M* = 2 Re(M)

21

22

21

21

21T

2

||||

||

)(

SS

SS

SS

SSY

Dividing through

by 2|S1-S2|

D. van Alphen 21



• Algebraic equivalent to Min. Distance decision rule, so far:

d(Y) = a1 iff ()

• Notation:

– Let b = , (unit-length) be the normalized signal

difference.

– Let Tth = denote the

threshhold on the RHS of ().

• Min. Distance Rule, from (): d(Y) = a1 iff {YT b} > Tth

||

)(

21

21

SS

SS

21

22

21

2

||||

SS

SS

21

22

21

21

21T

2

||||

||

)(

SS

SS

SS

SSY

D. van Alphen 22



• Again: d(Y) = a1 iff : YT b > Tth

• Recall engineering vocabulary: ( YT b) b = [Y1 Y2] is

the projection of Y in the direction of b

– Projection coefficient (YT b = bT Y = Y b) is the dot

product or inner product or correlation of Y and b

• Block Diagram: Correlation Receiver, Vector Form

(appropriate for equally likely signals in additive white noise)

b

2

1

b

b

Tth

a1

a2

Comparator

Correlator

.

b

Yai

Summary: Correlation

Detection = Min. Dist.

Detection (appropriate

for equally-likely signals

in additive white noise)

D. van Alphen 23

Example: BPSK Signaling in AWGN,

Given Noise Power s2 = N0/2 = 1

• Consider the case of 2 equally-likely signals:

s1(t) = 20 cos(2p200t), 0 < t < .01.

s2(t) = 20 cos(2p200t+90), 0 < t < .01.

• Signal energies*:

E1 = E2 = S T = (A2/2) T = (400/2) (.01) = 2

• Signal Space Diagram:

– Decision boundary, l

– Decision regions, Ii

orthogonal

* Here S denotes average signal power; T denotes signal duration.

2

y2

y1

2

D. van Alphen 24

Example: BPSK Signaling in AWGN

2

y2

y1

2

P(error) = P(err.|S1) P(S1) + P(err.|S2) P(S2)

= .5 [P(error|S1) + P(error|S2)]

= P(error|S1), by symmetry

Total Prob. Equation:

n

Dist. between signal points: 2

P(error) = P(error|S1) = P(n > dist/2) = P(n > 1)

= P(n > 1)= 1 – normcdf(1) = .1587

Note: Performance depends only on the geometry, not the actual

signals transmitted.

- The actual signals transmitted depend upon the choice of

basis functions.

D. van Alphen 25

Example: BPSK Signaling in AWGN

2

y2

y1

2

To find the simplified algebraic decision

rule for the correlation receiver:

Note Tth = whenever

the signals are equal-energy.

02

||||

21

22

21

SS

SS

2/1

2/1

2

2

2

2

0

0

2

||

)(and

2121

21

SSSS

SSb

Decision Rule: d(Y) = a1 iff 02/1

2/1YRe T

Aside: Karhunen-Loeve (K-L) Expansion

for Random Vectors

• Consider a N-dimensional random vector Y with covariance

matrix CY and mean E[Y].

• Then any realization of Y can be written as

Y = E[Y] +

where

i and ei are eigenvalues and corresponding eigenvectors of

CY; and

Wi are uncorrelated random variables.

• Think of

• the ei’s as orthogonal unit-length axes for a coordinate system

(bases);

• the values ( sqrt(i) Wi ) as giving the coordinates for Y on

each of the orthogonal unit-length axes; or

• the values Wi as giving the coordinates for Y on the scaled

orthogonal axes, sqrt(i) ei.

ii

N

1ii eW

Picturing the Case: N = 2

• Say the vector Y0 = ; say CY has eigenvectors e1 = ,

e2 = and eigenvalues 1 = 4, 2 = 9;

Then Y0 =

11 e

22 e

0

Y0

1

2

2/1

2/1

2/1

2/1

2211222111 eW3eW2eWeW

Assume Y is 0-mean.

Review: Spectral Resolution (Mercer’s Thm)

vs. K-L Expansion

• From Lecture 7, p. 12 - Mercer: Let C be a covariance matrix

for some real R Vector, with eigenvectors ei and

corresponding eigenvalues i.

– Then C = a projection matrix

• K-L Expansion: Any realization of an N-dimensional random

vector Y with covariance matrix CY and mean E[Y] can be

written:

– Y = E[Y] +

where

i and ei are e-values and corresponding e-vectors of CY;

and

Wi are uncorrelated random variables.

–

:e Tii

Ti

N

1iii eee

ii

N

1ii eW

D. van Alphen 29


- Signal Detection in Additive -

• Let Y = Si + N; N: 0-mean, covariance matrix CN (not white)

• Two (Major) Cases - consider Scholtz peanut:

Non-singular CN Singular CN

min > 0 min = 0

maxmax e

minmin e

Si

maxmax e

Si

0-length proj. in

direction of emin = enull

Since CN

is NNDSince CN

is NND

* Note: singular matrices have det = 0 at least one e-value is 0.

D. van Alphen 30


- Signal Detection in Additive Colored Noise

• Case 1: Let CN be singular

• Example: CN =

11

11

2

12

1

1

1

2

1,2 11 e

null12

2

12

1

1

1

2

1,0 ee

S1

2

S2

2

Line of possible

rcvd points if S1 tx’d

Line of possible


D. van Alphen 31



• Case 1: Singular Example: CN =

• Verifying claim that rcvd signal lies

on one of two dotted lines shown

• Consider K-L Expansion of rcvd Y,

assuming S1 was tx’d:

Y = S1 + N = S1 +

= S1 +

11

11

S1

2

S2

2

Line of possible


Line of possible


j

2

1jjj W e

0W2 11 e

1

1W

1

1

2

1W2 1111 SS

W1: 0-mean, unit-var.

R. variable (scalar)

D. van Alphen 32



Notes for Case 1: Singular CN

• Perfect (error-free) detection is possible if the two parallel

lines are distinct:

S2 S1 + k emax

• Correlating Y with enull yields:

YT enull = [Si + N]T enull = null

T

1i1

1W eS

(KL Exp. on N)

nullTinull1

Ti ]W]11[[ eSeS ( e-vectors)

projection coefficient of signal in

noise-free direction

D. van Alphen 33



Notes for Case 1: Singular CN , continued

• Repeating - correlating Y with enull yields:

YT enull (No noise!)

• Decision Rule for Additive Colored Noise, Case 1

(Singular cov matrix):

d(Y) = ai iff YT enull = Si enull

Perfect Correlation Detector in Singular Colored Noise, if :

S2 S1 + k emax

nullTi eS

D. van Alphen 34



• Case 2: Let CN be non-singular; Y = Si + N

– E-values i > 0, det CN 0 No noise-free direction

– Approach: Find linear transformation to make the colored

noise vector white; then use “min. distance rcvr = corr. rcvr”

on whitened, received vectors

– Caveat: We can’t separate the signal component Si from

the noise component N (both in Y), so whitening N will

effectively change the transmitted signals

• Recall: If covariance matrix CN = HHT, where H = EL1/2, then

the whitening filter is G = H-1 = L1/2 ET

D. van Alphen 35



• Case 2, continued

• Applying Whitening to Y = Si + N:

• Now apply min. distance rule, with modified signals H-1S1 and

H-1S2:

• Equiv. algebraic version of decision rule, next page:

G = H-1Y

= Si + NV = H-1 Y = H-1(Si + N) = H-1Si + W

one of 2 possible

modified signalsadditive

white noise

d(Y) = a1 iff |H-1Y – H-1S1| < |H-1Y – H-1S2|

D. van Alphen 36



• Case 2, continued; whitening filter applied to Y

• Algebraic Decision Rule :

• Note: decision rules and above both require that Y be pre-

filtered by H-1.

21

11

21

11

T1

HH

HH]H[

SS

SSY

d(Y) = a1 iff [H-1Y]T [H-1S1– H-1S2] > ½ [ |H-1S1|2 – |H-1S2|

2 ]

mod. threshhold, Tth’’

iff

21

11

22

121

1

HH2

|H||H|

SS

SS

mod. b, say b’’

Whitened Y

D. van Alphen 37

Example: Signal Detection in Colored Noise

• Design an opt. (minimum probability of error) receiver for

detection of equally likely signals S1 and S2 (below), received

in additive colored noise with covariance CN (below):

• Received signal: Y = Si + N

42

29C,

4

4,

4

4N21 SS

MATLAB:

s1 = [4; 4]; s2 = [-4; -4];

C = [9 2; 2 4];

[E, lambda] = eig(C)

G = (lambda^(-1/2)) * E’

1063.3030.

5196.1823.G

the required whitening filter

D. van Alphen 38

Example, continued

More MATLAB (Code)

s1_new = G*s1; s2_new = G*s2; % modified signals

b_new = (s1_new – s2_new)/norm(s1_new – s2_new)

Th_new_num = ( (norm(s1_new))^2 – (norm(s2_new))^2 ) / …

(2*norm(s1_new – s2_new))

More MATLAB (Results):

b_new = [-.6361; -.7716]

Th_new = 0

0

a1

a2

.Y

11.30.

52.18.G

77.

64.

ai

lecture #7.1 random vectors: 2 moment theory, continued d. …dvanalp/ece...

Documents