BLIND SOURCE SEPARATION BY KURTOSIS MAXIMIZATION WITH APPLICATIONS IN WIRELESS

COMMUNICATIONS

Chong-Yung Chi (祁忠勇 )

Institute of Communications Engineering &Department of Electrical Engineering

National Tsing Hua UniversityHsinchu, Taiwan 30013, R.O.C.

Tel: +886-3-5731156, Fax: +886-3-5751787E-mail: cychi@ee.nthu.edu.tw

http://www.ee.nthu.edu.tw/cychi/

Acknowledgments: The viewgraphs were prepared through Chun-Hsien Peng’s helps.

Invited talk at I2R, Singapore, July 18, 2006.

OUTLINE

1. Introduction to Blind Source Separation (BSS)

2. FKMA and MSC Procedure

3. Turbo Source Extraction Algorithm (TSEA)

4. Non-cancellation Multistage Source (NCMS) Separation Algorithms

NCMS-FKMA

NCMS-TSEA

5. Simulation Results --- Part 1

6. Turbo Space-time Receiver for CCI/ISI Reduction

7. Simulation Results --- Part 2

8. Conclusions

1

FKMA: Fast Kurtosis Maximization AlgorithmMSC: Multistage Successive Cancellation

1. Blind Source Separation (BSS)

Instantaneous Mixture of Sources

][1 nx

][nxP

][1 ns

][nsK

Noise P Output Measurements

Unknown

mixing matrix KP

][1 nw

][nwP

][ns ][nx

(Mutually Indep. but Colored)

A

2

Applications: array signal processing, wireless communications and biomedical signal processing, etc [1-3].

1

[ ] [ ] [ ] [ ] [ ] K

i ii

n n n s n n

x A s w a w

GOAL Extract all the source signals with only measurements .

][nx][nsi

(Memoryless channel)

is the basis vector that spans the subspace of

][nsi (ith column of i A)a

Algorithm

Whitening

Multistage

AMUSE 1. Statistically mutually uncorrelated2. Zero mean3. Temporal colored with distinct power spectra

1. Zero mean

2. Gaussian

SOBI

FOBI 1. Statistically mutually independent2. Zero mean3. Distinct fourth-order moments

EFOBI

FastICA1. Statistically mutually independent2. Zero mean 3. Non-Gaussian (Non-zero fourth-order cumulants, e.g., kurtosis)

MSC-FKMA

MSC-TSEA

NCMS-FKMA

NCMS-TSEA

P KA

P K

P K

],[nsk ][nw

Statistically independent

Kk ,...,1

SOS

HOS

Existing BSS Algorithms

SOS: Second-order Statistics

HOS: Higher-order Statistics

3

AMUSE: Algorithm for Multiple Unknown Signals Extraction (Tong et al., 1990 [1])

SOBI: Second-order Blind Identification (Belouchrani et al., 1997 [2])

FOBI: Fourth-order Blind Identification (Cardoso, 1989 [12])

EFOBI: Extended Fourth-order Blind Identification (Tong et al., 1991 [1])

FastICA: Fast Independent Component Analysis (Hyvarinen et al., 1997 [13, 14])

MSC: Multistage Successive Cancellation

NCMS: Non-cancellation Multistage

FKMA: Fast Kurtosis Maximization Algorithm

TSEA: Turbo Source Extraction Algorithm

4

AMUSE and SOBI Algorithm Using SOS:

Step 1: Prewhitening by Eigenvalue Decomposition (EVD)

]}[][{ H nnE xxRx

K ..., , , 21

Kfff ..., , , 21

: largest K eigenvalues of xR

: associated K eignevectors of

2ˆw : average of the other (smallest) P-K eigenvalues of (assuming that )

xRIwwRw

2H ]}[][{ wnnE

(PxP matrix)

EVDxR

H1

2 21

ˆ [ , ..., ]ˆ ˆ

K

w K w

f fD

][ˆ][][ˆ][ nnnn wDUsxDz

U : KxK unitary matrix

(dimension-reduced whitening spatial processing)

(whitening matrix)

AMUSE: Algorithm for Multiple Unknown Signals Extraction (Tong et al., 1990 [1])SOBI: Second-order Blind Identification (Belouchrani et al., 1997 [2])

5

{

]}[][{][ knnEk HzzRz

Step 2: Estimation of the Unitary Matrix fromU

Prewhitening by EVD

][nx ][nz EVD of

U2/])[][( kk H

zz RR

U Joint Diagonalization of }1|][{ Jiki ..., ,Rz

(AMUSE)

(SOBI)

(KxK matrix)

][ˆ][ˆ H nn zUs

(demixing matrix)

(spatial processing for simultaneous extraction of all the K sources)

Step 3: Source Separation and Channel Estimation

# #ˆ ˆ ˆ A W D U ( : pseudo-inverse)#

6

Hˆ ˆW U D

(mixing matrix estimate)

(A2) are modeled as

: zero-mean non-Gaussian independent identically distributed (i.i.d.) process with ; is statistically independent of for all .

Assumptions:(A1) The unknown mixing matrix is of full column rank with . P K

][nw(A3) is zero-mean Gaussian and statistically independent of .

}z ,z ,z ,z{cum 4321 : fourth-order joint cumulant of random variables

KP A

][ns

][][][ nbnuns iii ][nui

Stable LTI System

][nbi

} ..., ,2 ,1{ ],[ Kinsi

][nui

0]}[{4 nuC i][nui ][nu j

ji

2. FKMA and MSC Procedure

(referred to as kurtosis of )

FKMA: Fast Kurtosis Maximization Algorithm (Chi and Chen, 2001 [4,5])

MSC: Multistage Successive Cancellation 7

4321 z ,z ,z ,z

}z ,z ,z ,z{cum}z{C4 z

where

is the characteristic function of random variables

01

11

1

),,(ln

)(},,{cum

M

M

MM

MM jxx

}{ )(1

11),,( MM xxjM eE

.,,1 Mxx

Definition of Definition of HOS (i.e., (i.e., Cumulants):): (Bartlett, 1955, Brillinger, 1975, etc)

Assume that are zero-mean random variables. Then

}{},{cum 2121 xxExx

}{}{}{}{

}{}{}{},,,{cum

}{},,{cum

32414231

432143214321

321321

xxExxExxExxE

xxExxExxxxExxxx

xxxExxx

4321 , , , xxxx

22224**4 }{}){(2}{},,,{cum}{ xExExExxxxxC

(referred to as kurtosis of )x

8

}][{

]}[{])[()( 22

4

neE

neCneJJ v

Maximization

][][][ T nsnne kk xv

( is an unknown complex scale

factor and )

k},,1{ Kk

(noise-free case(noise-free case))

Fast Kurtosis Maximization Algorithm (FKMA) (Chi et al., 2001 [4, 5])

Optimum

9

v

Closed-form solution for : Not existent

Gradient-type iterative algorithms for finding a local optimum : Not very computationally efficient

v

)( )1()1()1()( iiii J vQvv

v

where Q is a positive-definite matrix depending on the algorithm used, and μ is the step size such that

)()( )1()( ii JJ vv

magnutude of normalized kutorsis of [ ]e n

Criterion [7]:

Criterion [7]:

Fast Kurtosis Maximization Algorithm (FKMA) (Chi et al., 2001 [4, 5])

9

)1(1

)1(1)(

i

ii

dR

dRv

Compute

At the th iterationi

][nx

][)1( ne i

YesSuper-expoAlgorithm(SEA)

nential

No

( ) ( 1)( ) ( )i iJ J v v?

][)( ne i

][)( ne i

To the thiteration

)1( i

Update through a gradient type optimization algorithm such that

)(iv

)()( )1()( ii JJ vv

T* *{ [ ] [ ]}E n n XR R x x

( 1) ( 1) ( 1) ( 1) * *cum{ [ ], [ ], ( [ ]) , [ ]}i i i ie n e n e n n d x

(PxP matrix)

Algorithm:

}][{

]}[{])[()( 22

4

neE

neCneJJ v

Maximization

][][][ T nsnne kk xv

( is an unknown complex scale

factor and )

k},,1{ Kk

(noise-free case(noise-free case))

Optimumv

magnutude of normalized kutorsis of [ ]e n

can be thought of as a measure of distance of from a Gaussian process, implying that the performance of the FKMA (which requires to be non-Gaussian [6]), depends on .

( [ ])is n ][nsi

( [ ])is n][nsi

10

Observations: The FKMA itself is an exclusive spatial processing algorithm.

The smaller the value of , the worse the performance of the FKMA for finite SNR and finite data length .

( [ ])is nN

By (A2)

( [ ]) ( [ ]) ( [ ])i i iJ s n s n J u n

where

(absolute normalized kurtosis of )][nsi

(entropy measure of the stable sequence ) ][nbi

(equality holds only as , i.e., minimum entropy of ) ][][ nnbi ][nbi

4

22

[ ]( [ ])

0 ( [ ]) 1( [ ])

[ ]

ii k

ii

ik

b kJ s n

s nJ u n

b k

MSC Procedure

][nx

][ˆ nsk

Estimate One Source Signal Using FKMA

Obtain

}{}{

2|][ˆ|

][ˆ][ˆ

nsE

nsnE

k

kk

x

a

Update by][nx

][ˆˆ][ nsn kkax Next Stage

Each Stage of the Multistage Successive Cancellation (MSC)

Procedure

( : th column of )ka k A

11

The estimated sources and columns of obtained at later stages in the MSC procedure may become less accurate due to error propagation effects from stage to stage [6].

][ˆ nsk ka A

NOTE

ka

][nx

][nx

][][TSEA nvn vv

: vector for extracting a colored source signal , i.e.,

removing spatial interference due to the mixing matrix . ((spatial spatial

filterfilter))

][nskv 1 PA

: single-input single-output (SISO) deconvolution (or higher-order whitening) filter of order to restore from . ((temporal temporal

filterfilter))

L][nv

][nuk ][nsk

where

T TTSEA TSEA[ ] [ ] [ ] [ ] [ ]

k

n n n k n k

v x v x

3. Turbo Source Extraction Algorithm

Source Separation Filter:

12

( [ ])J nMaximization

Design Criterion:

T TTSEA

T

[ ] [ ] [ ] [ ] [ ] [ ]

ˆ[ ] [ ] [ ] (spatial processing)

[ ] [ ] [ ] (temporal processing)k

n n n n e n v n

e n s n n

n v n n

v x v y

v x

y x

(A bank of same temporal filters)

( [ ]) ( [ ])

( [ ]) ( [ ])kn J u n

J e n e n

(Extracted Source)(Extracted Source)

Turbo Source Extraction Algorithm (TSEA) (Chi et al., 2003 [3])

Signal processing procedure at the th cyclei

Step 1

FKMA(s)][ˆ][ )1( nvnv i

)(ˆ iv

Temporal ProcessingTemporal Processing Spatial ProcessingSpatial Processing

][)ˆ(

][)1(T)( n

nii yv

][][

][)1(

nvn

ni

x

y

][nx

Step 1

(a)(a) (b)(b)

13

][ˆ][

][)( nvne

ni

FKMA(t) )(ˆ ivv

][ˆ )( nv i

][ˆ

][][ T

ns

nne

k

xv

Step 2

][nxStep 2

(b)(b) (a)(a)

(Extracted Source)(Extracted Source)

T [ ] e n

T[ [0], [1], , [ ]]v v v L T]][ , ],1[ ],[[][ Lnenenen e

][][][])[][(

][][ˆ][][][)(

)()(

ngnunvnbnu

nvnsnvnen

kkki

kkk

ik

i

Interpretations:

14

Why? Performance of TSEA is superior to FKMA.

][][][ )( nvnbng ikk

Increasing is equivalent to increasing

( [ ]) ( [ ]) ( [ ])kJ n n J u n

1) 1) TemporaTemporal l ProcessiProcessing:ng:

T)(1 ])[~ , ],[][][][~ , ],[~(][~ nsnnvnsnsnsn K

ikk s

][~][~][][][ )( nwnnvnn i sAxy

( [ ]) ( [ ]) ( [ ]) and ( [ ]), k k ln s n s n s n l k

2) Spatial 2) Spatial Processing:Processing:

4

22

[ ]( [ ]) ( [ ]),

[ ]

kmk

km

g mn s n

g m

(b)(b)

(b)(b)

Remarks:

15

The performance gain of the TSEATSEA reaches the maximum as long as the order LL (a parameter under our choice) of the temporal filter is sufficiently large. On the other hand, the asymptotic performance of FKMA approaches that of the TSEA as and .

All the sources can be extracted through the MSC MSC procedure. The resultant BSS algorithm that uses the TSEA, is referred to as MSC-TSEA,MSC-TSEA, also outperforms the MSC-FKMAMSC-FKMA, at the extra expense of the temporal processing at each stage.

TSEA is computationally efficient with super-exponential convergence rate and P parameters for spatial processingP parameters for spatial processing and L+1 parameters for temporal processing,L+1 parameters for temporal processing, respectively.

N SNR

Constrained

Criterion: ˆ ˆ ˆ, , , , 2, 3, , K a a a1 2C -1

4. Non-Cancellation Multistage Source Separation Algorithms NCMS-FKMA

16

T T Targ max{ ( ) ( [ ]) : [ ] [ ], }J J e n e n n v

v v v x v C 0 -1

where

Theorem 1: Let be the set of all the extracted source signals up to stage . With (A1), (A2), and the noise-free assumption, the optimum where is an unknown non-zero constant and .

T T[ ] [ ] [ ] [ ]k ke n n n s n x v x 1

S

k[ ]ks n S

Constraint

( ) v C (unconstrained optimization (unconstrained optimization problem)problem) Targ max{ ( ) ( [ ]) : [ ] [ ]}J J e n e n n x

vv

[ ] [ ]n nx C x

C: projection matrixP P

Unconstrained Criterion:

v

CObtain by

SVD of and][nx

17

C][][ nn xCx ˆ

-1a

[ ]nx

][ˆ nsk

(F-a)(F-a)Estimate One Source Signal

Using FKMA

Obtain

}{}{

2|][ˆ|

][ˆ][

nsE

nsnE

k

k

x

][nx

][nx

P/)1, ,1 ,1( T)0( v(Initial (Initial Condition)Condition) (0) v v

Good Good Initial Initial

ConditionCondition

Good Good Initial Initial

ConditionCondition

ˆaˆa

[ ]e n

Signal Processing Procedure of NCMS-FKMANCMS-FKMA

v(F-b)(F-b)

Estimate One Source Signal

Using FKMA

Remarks:

18

The constrained source extraction filter obtained in (F-a)(F-a) provides a suitable initial conditionsuitable initial condition for the unconstrained source extraction filter in (F-b),(F-b), which accordingly leads to one distinct source estimatedistinct source estimate obtained at each stage neither involving cancellation nor imposing any constraints on the source extraction filter, as well as faster convergence than (F-a).(F-a). Therefore, unlike the MSC-FKMA, the NCMS-FKMA is free fromfree from the error propagation effects the error propagation effects at each stage.

v

v[ ]e n

As the MSC-TSEAMSC-TSEA performs better than the MSC-FKMAMSC-FKMA, the NCMS-TSEANCMS-TSEA also performs better than the NCMS-FKMANCMS-FKMA at the moderate expense of extra computational load for the temporal processing of the TSEA.

CObtain by

SVD of and][nx

19

C][][ nn xCx ˆ

-1a

][][ˆ nensk

(T-a)(T-a)Estimate One Source Signal

Using TSEA

Obtain

}{}{

2|][ˆ|

][ˆ][

nsE

nsnE

k

k

x

][nx

][nx

P/)1, ,1 ,1( T)0( v

(Initial (Initial Condition)Condition) (0) v v

Good Good Initial Initial

ConditionCondition

Good Good Initial Initial

ConditionCondition ˆa

[ ]e n

Signal Processing Procedure of NCMS-TSEA NCMS-TSEA

v

(T-b)(T-b)Estimate One Source Signal

Using TSEA

(0)[ ] [ ]v n v n

[ ]v n[ ]nx

ˆa

5. Simulation Results --- Part 1

Parameters Used:

: zero-mean, independent binary sequence of with equal probability

: generated by filtering through the chosen FIR filters

: real white Gaussian noise vector

SNR:

50 independent runs

}][{

}][][{SNR

2

2

nE

nnE

w

wx

][nui

][nw

}1{

][nsi ][nui ][nbi

20

Output (extracted) signal to interference-plus-noise ratio (Output SINR)

K

iiK 1

SINR1

SINROutput

Four cases are considered as follows:

Part A: Performance of NCMS-FKMA and NCMS-TSEA mixing matrix (taken from Chang et al., 1998 [9]) (P=5, K=4)

4 5

5 ..., 1, ,0 )10

1(exp][

nn

nbi

i

4593.04807.01983.05731.0

6640.04216.02644.03558.0

2504.02661.04959.06107.0

2097.01157.07494.03397.0

4914.07120.02887.02380.0

A

A

21

Case 1: Output SINR versus SNR for different data length .

Case 2: Output SINR versus different data length .

Case 3: Output SINR versus (or ) for all .

Case 4: (a) Output SINR versus L.

(b) versus L.

N

( [ ])is n i i

N

K

kk nJK

1

])[()1(

(or ) for all , 5.0i ( [ ]) 0.2368i is n i 5L

Figure 1. Simulation results (Output SINR versus SNR) of Case 1.

22

5 10 15 20 25 300

5

10

15

20

25

30

SNR (dB)

OU

TP

UT

SIN

R (

dB

)NCMS-TSEA, N=1500NCMS-TSEA, N=1000NCMS-TSEA, N=500 NCMS-FKMA, N=1500NCMS-FKMA, N=1000NCMS-FKMA, N=500

(or ) for all , , and SNR=30 dB1i ( [ ]) 0.1856i is n

Figure 2. Simulation results (Output SINR versus data length ) of Case 2.

N

i 5L

23

103

104

105

106

14

16

18

20

22

24

26

28

30

32

N

OU

TP

UT

SIN

R (

dB

)

NCMS-TSEANCMS-FKMA

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 110

12

14

16

18

20

22

24

26

28

30

32

OU

TP

UT

SIN

R (

dB

)

NCMS-TSEANCMS-FKMA

0.745 0.345 0.240 0.183 0.144 0.115 0.091 0.068 0.01

SNR=30 dB, , and2000N

Figure 3. Simulation results (Output SINR versus ) of Case 3.

5L

24

0 1 2 3 4 5 6 7 8 910

15

20

25

30

35

L

OU

TP

UT

SIN

R (

dB

)

NCMS-TSEA, =0.5 (or =0.2368)NCMS-TSEA, =1 (or =0.1856)

and (i.e., and ) for all

SNR=30 dB, 1i 5.0 ( [ ]) 0.1856is n 2368.0 i

Figure 4a. Simulation results (Output SINR versus the order of

the temporal filter ) of Case 4 (a).L

2000N

25

0 1 2 3 4 5 6 7 8 90.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

L

NCMS-TSEA, =0.5 (or =0.2368)NCMS-TSEA, =1 (or =0.1856)

J(

[n

]) k=

1

K

(1/ K

)k

Figure 4b. Simulation results (Output SINR versus the order of

the temporal filter ) of Case 4 (b).L

26

and (i.e., and ) for all

SNR=30 dB, 1i 5.0 ( [ ]) 0.1856is n 2368.0 i

2000N

Part B: Performance Comparison The same mixing matrix in Part A and

Data length = 2000 and = 5

Comparison with the MSC-FKMA, AMUSE (Tong et al. 1990 [1]) and SOBI algorithm (Belouchrani et al. 1997 [2])

4 5

5 ..., 1, ,0 ),10

1(exp][

nn

nbi

i

A

N L

)1(

)2.0 ,3.0 ,4.0 , 1( 4321

4644.0 ])[( ,3335.0 ])[( ,2706.0 ])[( , 1856.0])[( 4321 nsnsnsns

)3,2,1 , ( iii

Three cases are considered as follows: Case A:Case A: Output SINR1 versus SNR for and .

Case B:Case B: Output SINR versus SNR for and .

Case C:Case C: Output SINR versus for SNR = 20 dB and .

2000N 5L

2000N 5L

5LN

27

5 10 15 20 25 300

5

10

15

20

25

30

35

SNR (dB)

OU

TP

UT

SIN

R 1 (d

B)

NCMS-TSEAMSC-TSEANCMS-FKMAMSC-FKMAFastICA SOBI ALGORITHMAMUSE

Figure 5. Simulation results (Output SINR1 versus SNR) of Case ACase A.

28

5 10 15 20 25 300

5

10

15

20

25

30

SNR (dB)

OU

TP

UT

SIN

R (

dB

)

NCMS-TSEAMSC-TSEANCMS-FKMAMSC-FKMAFastICA SOBI ALGORITHMAMUSE

Figure 6. Simulation results (Output SINR versus SNR) of Case BCase B.

29

500 1000 1500 2000 2500 3000 3500 4000 4500 50006

8

10

12

14

16

18

20

22

N

OU

TP

UT

SIN

R (

dB

)

NCMS-TSEAMSC-TSEANCMS-FKMAMSC-FKMAFastICA SOBI ALGORITHMAMUSE

Figure 7. Simulation results (Output SINR versus data length ) of Case CCase C.

N

30

: a 3x2 mixing matrix by removing the last two rows and columns of

the mixing matrix in Part A. (P=3, K=2)

Data length =1000, SNR=30 dB and =3.

Comparison with the MSC-FKMA, AMUSE and SOBI algorithm

Case D:Case D: Output SINR versus

1 1 11(z) (1 0.5z )(1 0.8z )(1 4z )B

1 1 12 (z) [1 (0.5 )z ][1 (0.8 )z ][1 (4 )z ]B

A

4959.06107.0

7494.03397.0

2887.02380.0

A

N L

)1( )3 ,2 ,1 ,( iii

40.005.0

31

0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.45

10

15

20

25

30

OU

TP

UT

SIN

R (

dB

)

NCMS-TSEAMSC-TSEANCMS-FKMAMSC-FKMAFastICA SOBI ALGORITHMAMUSE

Figure 8. Simulation results (Output SINR versus ) of Case Case D.D.

32

f1

f3

f5

f4f6

f2

f1

f7

CCICCI

GOALGOAL Enhance data rate, link quality, capacity, and coverage.

CCI:CCI: Co-channel Interference

ISI:ISI: Intersymbol Interference (due to multipath)

Space-time processingSpace-time processing using an antenna array has been used for

combating CCI and ISIcombating CCI and ISI in the receiver design [15-16].

][nh][nu

][nx

(Nois(Noise)e)

(Multipath (Multipath channel)channel)

][nw

Problem Statement: CCICCI and ISIISI Suppression in TDMA Cellular Wireless Communications

6. Turbo Space-time Receiver for CCI/ISI Reduction

33

][1 nx

][2 nx

][nxP

][1 nu

)],[( 1111 nh

)],[( 1212 nh

)],[( 2121 nh

)],[( 2222 nh

][2 nu

Consider the scenario where the base station is equipped with Consider the scenario where the base station is equipped with multiple antennas,multiple antennas, and the signal of interest and CCI are received and the signal of interest and CCI are received fromfrom multiple distinct directions of arrival (DOA), with a frequency- multiple distinct directions of arrival (DOA), with a frequency-selective fading channel for each DOAselective fading channel for each DOA. . ((a general scenarioa general scenario))

Signal Model:

34

][][][ nnn wAsx

““ISI-distorted’’ signal ISI-distorted’’ signal (colored signal)(colored signal) from jth DOA of user from jth DOA of user kk

( is no. of DOAsDOAs of user k)

th-order channel impulse response of jthjth DOADOA of user k

where

1 2{ } ( , ,..., )KP A A A A 1 2{ } ( ( ), ( ),..., ( ))k kk P p k k kp A a a a

T1 2[ ] ( [ ], [ ],..., [ ])s

kk k k kpn s n s n s n

[ ] [ ] [ ] 1, 2,..., kj kj k ks n h n u n j p

( ) : kja steering vector of jthjth DOADOA of user k

[ ] :kjh n kjL

The received signal from the desired user and CCIs (users) can be The received signal from the desired user and CCIs (users) can be expressed as expressed as anan instantaneous mixture of multiple sourcesinstantaneous mixture of multiple sources

pk

1K

1

K

kk

p (total no. of DOAs or ’’sources(total no. of DOAs or ’’sources”)”)

35

T T T T1 2[ ] ( [ ], [ ],..., [ ])s s s sKn n n n

][][1

nnK

kkk wsA

(A1) The unknown DOA matrix is of full column rank and

P

36

(A3) is zero-mean Gaussian, and statistically independent of for all .

Assumptions: A

(A2) The data sequence of user 1 (the desired signal)(the desired signal) is i. i. d. zero-mean non-Gaussian with , and meanwhile statistically independent of the other ( ) zero-mean i. i. d. data sequences (of CCI).(of CCI). k

][nuk

][1 nu0]}[{C 14 nu

][nw ][nuk

1K

1

K

kk

p (total no. of DOAs or ’’sources(total no. of DOAs or ’’sources”)”)

P

block mutually independent colored sourcesK

T1 2[ ] ( [ ], [ ],..., [ ])s

kk k k kpn s n s n s n

correlated colored non-Gaussian sourcespk

T T T T1 2[ ] ( [ ], [ ],..., [ ])s s s sKn n n n

block mutually independent colored sources; mutually independent random variables for each n

Mutually independentcolored sources

Case II: Each user has multiple DOAs with disjoint domains of support of multipath channel impulse responses, i.e.,

Case I: Each user has a single DOA with multiple paths (Venkataraman et al., 2003), i.e.,

1, 1, 2,...,kp k K

11 1( ( ), , ( ))K A a a T11 1[ ] ( [ ], , [ ])Kn s n s ns

1 1[ ] [ ] [ ]k k ks n u n h n

2* *[ ] [ ] [ ] [ ] [ ] 0, ki kj ki kj kl

E s n s n h l h l E u n i j

T T T T1 2[ ] ( [ ], [ ],..., [ ])s s s sKn n n n

[ ] [ ] 0 and 1ki kjh n h n i j k K

37

K

1[ ] [ ]je n s n ][][ 1 nun Temporal Filter

Spatial Filter

Conventional Cascade Space-Time Receiver (CSTR) (Jelitto and Fettweis, 2002)

][nv][nx

v

Space-time Processor

For Cases I and II, the conventional CSTR has been reported for CCI and ISI suppression

In CAMSAP-06, we proposed two space-time receivers based on kurtosis maximization for these two cases and a discussion of the proposed space-time receivers for the general scenario.general scenario.

Other existing structures: full-dimension (joint) ST processing, reduced dimension ST processing (prewhitening followed by joint ST processing).

38

Closed-form solution for : Not existent Gradient-type iterative algorithms for finding a local optimum : Not very computationally efficient Applicable not only for Case I but also for Case II (Peng et al., ICICS 2005)

39

}][{

]}[{])[()( 22

4

neE

neCneJJ v

MaximizationT[ ] [ ] [ ]

[ ] [ ]

kj kj

kj k kj

e n n s n

u n h n

v x

(noise-free case)

Kurtosis Maximization (Ding ad Nguyen, 2000):

v

v

is an unknown complex scale factorkj

} , ,2 ,1{ Kk {1, 2, , }kj p and

magnutude of normalized kutorsis of [ ]e n

)1(1

)1(1)(

i

ii

dR

dRv

Compute

At the th iterationi

][nx

][)1( ne i

YesSuper-expoAlgorithm(SEA)

nential

No

)( )( )1()( ii JJ vv

?

][)( ne i

][)( ne i

To the thiteration

)1( i

Update through a gradient type optimization algorithm such that

)(iv

)( )( )1()( ii JJ vv

]}[][{ T* nnE xxR

]}[ ,])[( ],[],[{cum **)1()1()1()1( nnenene iiii xd

(PxP matrix)

Fast Kurtosis Maximization Algorithm (FKMA) (Chi and Chen, 2001):

40

Blind CSTR Using FKMA

T1 1 1[ ] [ ] [ ] [ ] [ ] j je n n s n u n h n v x

Spatial processing using FKMA for CCI suppression

With a suitable initial conditionsuitable initial condition for , FKMA will converge at a super-exponential rate with for high SNR.

v

41

T1[ ] [ ] [ ] [ ] [ ]n v n e n n u n ev

T]][ , ],1[ ],0[[ Lvvv vT]][ , ],1[ ],[[][ Lnenenen e

where

Temporal processing using FKMA for ISI removal

(L: order of the temporal filter)

1[ ] [ ]je n s n1[ ] [ ]n u n Temporal

FilterSpatial Filter ][nv

][nxv

Space-time Processor

11 , ,2 ,1 ,][][ pjnsne j

the CCI, (i.e., ). So

The performance of the spatial filter (to suppress CCI) using FKMA is The performance of the spatial filter (to suppress CCI) using FKMA is

worse for smaller and worse for larger , worse for smaller and worse for larger , leading to limited performance of the temporal filter of the blind CSTR.

It can be easily shown that (Chi et al., 2003)

1 1 1( [ ]) ( [ ]) ( [ ]), j jJ s n s n J u n

where 1

1

4

10

1 22

10

[ ]0 ( [ ]) 1

[ ]

j

j

L

jm

jL

jm

h ms n

h m

42

1( [ ]),js n 1 jL

Usually, the ISI-distorted (desired) signalISI-distorted (desired) signal , has higher power than all

1 [ ]js n

implying that can be used as the initial conditioninitial condition for the spatial filter needed by the FKMA.

2 2

1 { [ ] } { [ ] }j k iE s n E s n 1k

1( )a jv

2H1 arg max { ( ) [ ] },

a xj E n (DOA estimate by delay-and-(DOA estimate by delay-and-

sum)sum)

][nv

v

TSTR[ ] [ ]n v nv v

TTSTR[ ] [ ] [ ]n n n v x

Space-Time Filter for Source Extraction (Chi et al. 2003, 2006):

])[( nJ Maximization

Optimum

Design Criterion:

T TTSTR

1 1 1 1

[ ] [ ] [ ] [ ]

[ ] [ ] [ ]j j

n n n n

s n v n u n

v x v y

T1 1 1[ ] [ ] [ ] [ ]j j je n s n n s n v x

Blind Turbo Space-Time Receiver (TSTR)

43

(noise-free case)

[ ] [ ] [ ]n n v n y x

( ) T

1

[ ] [ ]

[ ]

i

j

e n n

s n

v x

Spatial FilterSpatial Filter

FKMAFKMA ][][ )1( nvnv i

][)( nv i)(iv

)(ivv Temporal Temporal

FilterFilterFKMAFKMA

( )2 1[ ] [ ]i n u n

][][

][)(

nvn

ni

x

y

(S2)(S2)

(S1)(S1)][)(1 ni

][nx

][nxTemporal Temporal FilterFilter

Spatial FilterSpatial Filter

Signal processing procedure at the th cycle: i

Proposed Blind TSTR Using FKMA

])[( )(1 nJ i ])[( )(

2 nJ i

44

i=i+1

CSTR

CSTR

( ) ( ) ( ) ( )2 1

( )1 1 1 1 1 1

[ ] [ ] [ ] [ ] [ ]

( [ ] [ ]) [ ] [ ] [ ]

i i i ij

ij j

n e n v n s n v n

u n h n v n u n g n

Interpretations:

Why? Performance of blind TSTR is superior to blind CSTR.

1 [ ]jg n

Increasing is equivalent to increasing

])[( )(2 nJ i

1) 1) TemporaTemporal l ProcessiProcessing:ng:

( 1) ( ) T11 1 1 2 1[ ] ( [ ],..., [ ] [ ] [ ] [ ], , [ ], , [ ] )

K

i ij j k Kpn s n s n s n v n n s n s n s

][~][][][][ )1( nwnnvnn i sAxy 2) Spatial 2) Spatial

Processing:Processing:

4

1( )2 122

1

[ ]( [ ]) ( [ ])

[ ]

jmi

j

jm

g mn s n

g m

( )2 1 1( [ ]) ( [ ]) ( [ ]) and ( [ ]), 1, l i

j j kln s n s n s n k j

45

Compared with the blind CSTR, the proposed blind TSTR is the proposed blind TSTR is insensitive to insensitive to the value of the value of (i.e., robust against channel with multiple paths or severe ISI).

Remarks:

It can be proven that

for all , implying the guaranteed convergencethe guaranteed convergence of the proposed blind TSTR. Typically, the number of cyclesthe number of cycles spent by the TSTR before convergence, is equal to 2 or 32 or 3. The computational load of the blind TSTR is approximately 2 or 32 or 3 times that of the blind CSTR.

i

Because the design of and that of are coupled in a coupled in a constructiveconstructive and boosting manner and boosting manner, the proposed blind TSTR outperforms the blind CSTR for all , and meanwhile their performance difference is their performance difference is largerlarger for larger . for larger .

v

])[( ])[( ])[( ])[( 1)(

2)(

1)1(

2 nuJnJnJnJ iii

][nv

LL

1( [ ])js n

46

1 [ ]jh n

CASE I: CCI suppression by CCI suppression by

Performance of the blind TSTR:

TSTR[ ] [ ]n v nv vTTSTR[ ] [ ] [ ], n n n v x where

T1 11 1

1 1 1

[ ] ( ) [ ] [ ] [ ] residual CCI and noise

[ ]

n h n v n u n

u n

v a

CASE II: CCI suppression byCCI suppression by

1

T T1 1 1 1 1 1

1 1 1

[ ] ( ) [ ] [ ] [ ] ( ) [ ] [ ]* [ ]

residual CCI and noise [ ]

p

i i j jj i

i i

n h n v n u n h n v n u n

u n

v a v a

Multiple DOAs suppressed also byMultiple DOAs suppressed also by

1T

1 1 11

[ ] ( ) [ ] [ ] [ ] residual CCI and noise p

j jj

n h n v n u n

v a

GENERAL CASE: CCI suppression CCI suppression byby

the spatial filterthe spatial filter and the temporal filterand the temporal filter combine the signalscombine the signalsfrom all the DOAs in a constructive and boosting fashionfrom all the DOAs in a constructive and boosting fashion

v

1 1 1[ ]g u n 47

v

[ ]v n

v

v

v

: zero-mean, independent binary sequence of with equal probability

, ,

: white Gaussian noise vector SNR:

50 independent runs

01 402

Scenario of Case I

][nui

][nw

}1{

10P (array size)

}])[]([{]0[ HnnE ssRS : Diagonal matrix: Diagonal matrix

543211 z7073.0z4712.0z3089.0z7073.00.4325z 6178.0)z( H

543212 0.1622z0.1217z0.2839z0.3650z0.2839z0.4056)z( H

543213 0.1195z0.1992z0.2390z0.3586z0.3187z0.3984)z( H

603

22

2

12

2

11 1}][{

}][{

}][)({SNR

ww

nsE

nE

nsE

w

a

48

7. Simulation Results --- Part 2

Order of the temporal filter =20

SNR=20 dB

Data length =2000

L

N

402 01 603

49

Blind CSTRBlind CSTR Proposed Blind TSTRProposed Blind TSTR

50

Data length =2000

Order of the temporal filter =20

N

L

51

SNR=30 dB

Order of the temporal filter =20

L

52

SNR=30 dB

Data length =2000

N

53

: zero-mean, independent binary sequence of with equal probability

: real white Gaussian noise vector

SNR:

50 independent runs

][nui

][nw

}1{

10P (array size)

1 211(z) 0.5199 0.3639z 0.3119zH 1 2

21(z) 0.3562 0.3206z 0.1425zH

3 4 512 (z) 0.5754z 0.2466z 0.3288zH 3 4 5

22 (z) 0.3776z 0.2098z 0.2518zH

011 2012 4021 6022

54

: Diagonal matrix

}][{

}][)(][)({SNR

2

212121111

nE

nsnsE

w

aa

}])[]([{]0[ HnnE ssRS

Scenario of Case II

Order of the temporal filter =20

SNR=20 dB

Data length =2000

L

N55

011 2012 4021 6022

56

Blind CSTRBlind CSTR Proposed Blind TSTRProposed Blind TSTR

Data length =2000

Order of the temporal filter =20

N

L

57

SNR=30 dB

Order of the temporal filter =20

L

58

SNR=30 dB

Data length =2000

N

59

: zero-mean, independent binary sequence of with equal probability

: white Gaussian noise vector

SNR:

50 independent runs

Scenario of the general case

][nui

][nw

}1{

10P (array size)

011 2012 4021 6022

1 2 311(z) 1 0.7z 0.6z 0.5zH 1 2 3

21(z) 1 0.9z 0.4z 0.3zH 2 3 4 5

22 (z) z 0.9z 0.5z 0.6zH

: Block-diagonal matrix

}][{

}][)(][)({SNR 2

2

12121111

nE

nsnsE

w

aa

}])[]([{]0[ HnnE ssRS

60

543212 0.4z0.3z0.7z0.8z )z( H

Order of the temporal filter =20

SNR=30 dB

Data length =2000

011 2012 4021 6022

61

L

N

62

Blind CSTRBlind CSTR Proposed Blind TSTRProposed Blind TSTR

Data length =2000

Order of the temporal filter =20

N

L

63

SNR=30 dB

Order of the temporal filter =20

L

64

SNR=30 dB

Data length =2000

N

65

We have introduced a novel blind source extraction algorithm, TSEATSEA, which operates cyclically using the FKMAFKMA for both of the temporal processing and spatial processing. The proposed TSEATSEA outperforms the FKMA for in addition to sharing convergence speed and computational efficiency of the later at each cycle.

FKMAFKMA only involves spatial processing for extraction of one non-Gaussian (i.i.d. or colored) source from source mixtures. It performs well with super-exponential convergence rate, but its performance depends on the parameter .

8. Conclusions

0 ( [ ]) 1is n

( [ ]) 1is n

Because of performance degradation resultant from the error propagation in the MSC procedure, we further introduced two non-cancellation BSS algorithms, namely, NCMS-FKMA and NCMS-TSEA, that can extract a distinct source at each stage without error propagation.

66

The two BSS algorithms, NCMS-FKMA and NCMS-TSEA perform better than the existing MSC-FKMA and the MSC-TSEA, respectively, with moderately higher computational complexities. FKMA and TSEA are under investigation for CCI and ISI in MIMO wireless communications (e.g., OFDM and multi-rate CDMA) and other applications such as 2-D MIMO systems in biomedical signal processing (with certain constraints or partial correlation between source signals).

67

Some works of Part 1/Part 2 will be published in C.-Y. Chi and C.-H. Peng, “Turbo source extraction algorithm and non- cancellation source separation algorithms by kurtosis maximization,” IEEE Trans. Signal Processing, vol. 54, no. 8, pp. 2929-2942, Aug. 2006.

C.-H. Peng, C.-Y. Chi and C.-W. Chang, “Blind multiuser detection by kurtosis maximization for asynchronous multi-rate DS/CDMA systems,” EURASIP Journal on Applied Signal Processing, vol. 2006, Article ID 84930, 17 pages, 2006. doi:10.1155/ASP/2006/84930. (special issue: Multisenor Processing for Signal Extraction and Applications)

Thank you very Thank you very muchmuch

68

Background materials of the talk can be found in the following book: C.-Y. Chi, C.-C.Feng, C.-H. Chen and C.-Y. Chen, Blind Equalization Blind Equalization and System Identification and System Identification, London: Springer-Verlag, 2006.

[1] L. Tong, R.-W. Liu, V. C. Soon, and Y.-F. Huang, ``Indeterminacy and identifiability of blind identification,'' IEEE Trans. Circuits and Systems, vol. 38, pp. 499-509, May 1991.

[2] A. Belouchrani, K. Abed-Meraim, J. -F. Cardoso, and E. Moulines, ``A blind source separation technique using second-order statistics,'' IEEE Trans. Signal Processing, vol. 45, pp. 434-444, Feb. 1997.

[3] C.–Y. Chi, C.-J. Chen, F.-Y. Wang, C.-Y. Chen and C.-H. Peng, ``Turbo source separation algorithm using HOS based inverse filter criteria,'' Proc. IEEE International Symposium on Signal Processing and Information Technology, Darmstadt, Germany, Dec. 14-17, 2003.

[4] C.–Y. Chi and C.-Y. Chen, ``Blind beamforming and maximum ratio combining by kurtosis maximization for source separation in multipath,'' Proc. IEEE Workshop on Signal Processing Advances in Wireless Communications, Taoyuan, Taiwan, Mar. 20-23, 2001, pp. 243-246.

[5] C.-Y. Chi and C.-Y. Chen , C.-H. Chen and C.-C. Feng, ``Batch processing algorithms for blind equalization using higher-order statistics,'' IEEE Signal Processing Magazine, vol. 20, pp. 25-49, Jan. 2003.

References

69

[6] J. M. Mendel, ``Tutorial on higher-order statistic (spectra) in signal processing and system theory: theoretical results and some applications,'' Proc. IEEE, vol. 79, pp. 278-305, Mar. 1991.

[7] C.-Y. Chi, C.-H. Chen and C.-Y. Chen, ``Blind MAI and ISI suppression for DS/CDMA systems using HOS-based inverse filter criteria,'' IEEE Trans. Signal Processing, vol. 50, pp. 1368-1381, June 2002.

[8] Z. Ding and T. Nguyen, ``Stationary points of a kurtosis maximization algorithm for blind signal separation and antenna beamforming,'' IEEE Trans. Signal Processing, vol. 48, pp. 1587-1596, June 2000.

[9] C. Chang, Z. Ding, S. F. Yau, and F. H. Y. Chan, ``A matrix-pencil approach to blind separation of non-white sources in white noise,'' Proc. IEEE International Conference on Acoustics, Speech, and Signal Processing, Seattle, WA, May 12-15, 1998, pp. 2485-2488.

[10] D. J. Moelker, A. Shah and Y. Bar-Ness, ``The generalized maximum SINR array processor for personal communication systems in a multipath environment,'' Proc. IEEE International Symposium on Personal, Indoor, and Mobil Radio Communications, vol. 2, Taipei, Oct. 15-18, 1996, pp. 531-534.

[11] V. Venkataraman, R. E. Cagley and J. J. Shynk, ``Adaptive beamforming for interference rejection in an OFDM system,'' IEEE Conference Record of the Thirty-Seventh Asilomar Conference on SSC, Nov. 9-12, 2003, vol. 1, pp. 507-511.

70

[12] J.-F. Cardoso, ``Source separation using higher order moments,'' Proc. IEEE International Conference on Acoustics, Speech, and Signal Processing, Glasgow, UK, May 23-26, 1989, pp. 2109-2112.[13] A. Hyvärinen, J. Karhunen, and E. Oja, Independent Component Analysis. New York: Wiley- Interscience, 2001.

[14] A. Hyvärinen and E. Oja, ``A fixed-point algorithm for independent component analysis,'' Neural Computation, vol. 9, pp. 1482-1492, 1997.

71

[15] J. Jelitto and G. Fettweis, ``Reduced dimension space-time processing for multi-antenna wireless systems,'' IEEE Wireless Communications Mag., vol. 9, pp. 18-25, Dec. 2002.

[16] Jen-Wei Liang and A. J. Paulraj, ``Two stage CCI/ISI reduction with space-time processing in TDMA cellular networks,'' Proc. 30th Asilomar Conference on Signals, Systems, and Computers, vol. 1, Pacific Grove, CA, Nov. 3-6, 1996, pp. 607-611.