*Corresponding author. Also with Department of Mathematics, Neijiang Normal College, Sichuan, P.R. China. Tel.: 00852-2788-7756; fax: 00852-2788-7791; e-mail: yfang@ee.cityu.edu.hk

Signal Processing 76 (1999) 37}42

Linear neural network based blind equalization

Yong Fang*, Tommy W.S. Chow, K.T. Ng

Department of Electronic Engineering, City University of Hong Kong, 83 Tat Chee Avenue, Kowloon, Hong Kong

Received 31 July 1998; received in revised form 11 December 1998

Abstract

This letter considers the problem of blind equalization in digital communications by using linear neural network.Unlike most adaptive blind equalization methods which are based on matrix decomposition or the Hankel property ofmatrix, we give a stochastic approximate learning algorithm for the neural network according to the property of thecorrelation matrices of the transmitted symbols. The network outputs provide an estimation of the source symbols, whilethe weight matrix of network estimates the inverse of the channel matrix. Simulation results demonstrate the perfor-mance and validity of the proposed approach for blind equalization. ( 1999 Elsevier Science B.V. All rights reserved.

Zusammenfassung

Dieser Beitrag behandelt das problem der blinden Entzerrung in der digitalen Kommunikation mit Hilfe eines linearenneuronalen Netzes. Im Unterschied zu den meisten adaptiven blinden Entzerrungsmethoden, die auf Matrizenzerlegun-gen oder der Hankeleigenschaft von Matrizen basieren, stellen wir einen stochastischen approximativen Trainings-algorithmus entsprechend der Eigenschaft der Kovarianzmatrizen der gesendeten Symbole fuK r das neuronale Netz vor.Die Ausgangssignale des Netzes liefern eine SchaK tzung der Quellsymbole, waK hrend die Gewichtungsmatrix des Netzesdie Inverse der Kanalmatrix schaK tzt. Simulationsergebnisse demonstrieren die LeistungsfaK higkeit und GuK ltigkeit desvorgeschlagenen Ansatzes zur blinden Entzerrung. ( 1999 Elsevier Science B.V. All rights reserved.

Re2 sume2

Cet article considere le probleme de l'eH galisation aveugle de communications numeH riques par reH seaux de "ltreslineH aires. Contrairement a la plupart des meH thodes d'eH galisation aveugle adaptative, qui reposent sur la deH composition dematrices ou la proprieH teH de Hankel des matrices, nous proposons un algorithme d'apprentissage par approximationstochastique pour le reH seau de neurones selon la proprieH teH des matrices de correH lation des symboles transmis. Les sortiesdu reH seau fournissent une estimation des symboles de source, alors que la matrice des poids du reH seau est une estimationde l'inverse de la matrice du canal. Des reH sultats de simulations deHmontrent les performances et la validiteH de l'approcheproposeH e pour l'eH galisation aveugle. ( 1999 Elsevier Science B.V. All rights reserved.

Keywords: Intersymbol interference; Blind equalization; Linear neural network; Stochastic approximate learning

0165-1684/99/$ } see front matter ( 1999 Elsevier Science B.V. All rights reserved.PII: S 0 1 6 5 - 1 6 8 4 ( 9 8 ) 0 0 2 4 5 - X

1. Introduction

The problem of blind equalization is of consider-able interest in digital communications and related"elds [1}14,16}20]. Adaptive blind equalization isan important tool for eliminating intersymbol in-terference caused by linear channel distortion ormultipath. Given a received signal, the goal is torecover the transmitted input symbol sequence.Classical approaches to the problem have focusedon the exploitation of the constant modulus prop-erty, or discrimination based on higher-orderstatistics [3}5,8,11,13,18,19]. But the major short-coming of these techniques is underlined by its slowconvergence rate. Recently, blind equalization byusing only second-order cyclostationary has re-ceived an increasing interest [6,7,9,10,12,14,16,17,20]. A major progress was made by Tong et al.[17], in which they explored the cyclostationaryproperties of an oversampled communication sig-nal to estimate the channel responses and sourcesymbols. In their study, second-order satisticsbased method was used. Many types of signalsexhibit the particular cyclostationarity ([17]and the references therein). All these proposedalgorithms can be readily reinterpreted as applyingto the multiple sensor case, but most of thembelong to the category of batch algorithms asthey are implemented by matrix decomposition.The input sequence is estimated once after a blockof the samples are received. Such algorithm su!ersfrom a large amount of computation e!ort and maynot be implemented for on-line operation [2]. Inaddition, these methods mainly focused on estima-ting the channel parameters rather than the trans-mitted symbols, which must be estimatedindirectly. Recently, a single layer linear neuralnetwork is introduced as a blind equalizer [2]. Thealgorithm of the neural network is proposed inaccordance with the Hankel property of the inputvector samples. The algorithm can either estimatedirectly the transmitted symbols or be implementedon-line. The original input symbols can be approxi-mated by the estimated symbols up to a non-zeroconstant.

In this letter, a single layer linear neural networkis considered as an equalizer for digital commun-ications. The received signal is oversampled so that

the channel can be described by a full-column rankmatrix. Based on the property of the correlationmatrices of the transmitted symbols, we are able toderive a stochastic approximate learning algorithmfor the neural network. The algorithm may be im-plemented on-line, and the network outputs pro-vide a direct estimation of the source symbols.Moreover, the estimated symbols can approximatethe transmitted symbols up to a constant of whichthe norm is not restricted at one.

2. Data model

Let us consider a multipath digital communica-tion system. A complex data sequence Ms

kN,

k"2,!1,0,1,2,2, is sent over a communicationchannel with T time apart. The channel is charac-terized by a continuous function h(t). The signalsmay be corrupted by noise w(t). The receivedbaseband signal x(t) can be represented as

x(t)"`=+~=

skh(t!k¹ )#w(t). (1)

The object of blind equalization is to estimate theinput symbols Ms

kN given only the received signal

x(t). Throughout this letter, we will use the notationAT, AH, AH, EMAN and AK for the transpose,hermitian, complex conjugate, expectation andestimate of A, respectively. We also assume thefollowing throughout the sequel:H1: The symbol interval ¹ is known and is an

integer multiple of the sampling period.H2: The impulse response h(t) has "nite support.

The duration of h(t) is ¸h, i.e., h(t)"0, for

t(0 or t*¸h.

H3: MskN is zero mean stationary process, and

EMsksHlN"d(k!l), where d(t) is the discrete-

time impulse function.If x(t) is sampled in the observation interval

(t0#j¹, t

0#j¹#¸) with a sampling period D,

the vector representation of the received signal isgiven by [10]

x( j)"Hs( j)#w( j), j"0,1,2, (2)

where x( j)"[x(t0#j¹#D),2,x(t

0#j¹#

mD)]T3Rm, s( j)"[sK0`j~1

,2,sK0`j`d~2

]T3Rd,

38 Y. Fang et al. / Signal Processing 76 (1999) 37}42

H"Ah(t

0#D!K

0¹) 2 h(t

0#D!(K

0#d!1)¹)

F 2 F

h(t0#mD!K

0¹) 2 h(t

0#mD!(K

0#d!1)¹)B3Rmxd, (3)

w( j)"[w(t0#j¹#D),2,w(t

0#j¹#mD]T3Rm,

and

K0"

t0!¸

h¹

#1, d"t0!(K

0!1)¹#¸

¹

.

vxw (xxy ) stands for the smallest (greatest) integerthat is greater than or equal to (less than or equalto) x.

To assure solvability of the blind equalization,the length of the observation interval L or thesampling period D is selected such that m is not lessthan d. Hence, we have the following assumption:H4: The number of row of the channel matrix H is

greater than or equal to the number of itscolumn, and the channel matrix H is full col-umn rank.

3. Theoretical analysis

For the sake of simplicity, the e!ect of noise isignored in this study. The blind equalization prob-lem can be restated as follows: given a vector pro-cess x( j), j"0,1,2,2, obtained from a linearmodel

x( j)"Hs( j), j"0,1,2,2, (4)

to estimate s( j). In this case, it is clear that thereexits a solution s( ( j)"(HHH)~1HHx( j) for the fullcolumn rank assumption on the channel matrix H.

Assume that the input symbols MskN are driven

from a set of i.i.d. (independent identically distrib-uted) random variables, the autocorrelation func-tion of s( j) de"ned by

Rs(n)"EMs( j)sH(j#n)N (5)

is represented in the following form:

Rs(0)"I, and R

s(n)"Jn, n'0, (6)

Rs(n)"(JT)@n@, n(0, (7)

where J is a d]d shifting matrix

J"A0 0 2 0 0

1 0 2 0 0

0 1 2 0 0

F F F F F

0 0 2 1 0B .

The following theorem states a necessary andsu$cient condition for the blind equalization.

Theorem 1. Suppose that H and s( j) satisfy the linearmodel (8) and its constraints. There exists a lineartransformation= by d]m, i.e.,

y( j)"=x( j), (8)

such that for some constant a, aaH"1,

y( j)"as( j), (9)

if and only if the following conditions are satisxed:

Ry(0)"I, (10)

Ry(1)"J, (11)

where I is a d]d identity matrix.

Proof. (i) Necessity: It is straightforward.(ii) Suzciency: Substitute Eq. (4) into Eq. (8) to

obtain

y( j)"=Hs( j). (12)

According to Eqs. (10)}(12), we get

=H(=H)H"I, (13)

=HJ(=H)H"J. (14)

Let Q"=H, then QQH"I. This implies that Q isan orthogonal matrix. Denote the ith column of

Y. Fang et al. / Signal Processing 76 (1999) 37}42 39

Fig. 1. Architecture of the linear neural network.

Q as Qi. Eq. (14) gives a Jordan chain of length p@

associated with eigenvalue 0 of matrix J.

JQ1"Q

2,2, JQ

p{~1"Q

p{, JQ

p{"0. (15)

The last equation in (15) implies Qp{"[0,2,0,a]T

and consequently Q"aI, i.e., y( j)"as( j). Recallthat Q is an orthogonal matrix, so aaH"1 holds.

4. Neural network and learning algorithm

Consider a neural estimation of the expansion(4). Based on Theorem 1, this can be realized usingthe single layer linear neural network as shown inFig. 1. The m inputs of the network are the compo-nents of the vector x( j). The output of the network,y( j), is regarded as the estimation of s( j), if y( j)satis"es the conditions (10) and (11), through thelearning of the weight matrix= in Eq. (8). In orderfor y( j) to satisfy condition (10), we need to trainthe weight matrix= such that the network outputsare uncorrelated and normalized. This can be per-formed by a stochastic approximate algorithm [15]

*=( j)"!jj[y( j)yH( j)!I]=( j), (16)

where=( j) is the value of= after jth update, jjthe

learning rate used in the jth update. Similarly, inorder to make y( j) satisfy condition (11), the learn-ing of the weight matrix,=, is assured by Theorem2 and its associated algorithm (17).

Theorem 2. The following algorithm provides a con-vergent solution for the condition (11):

*=( j)"!gj[y( j)yH( j#1)!J]JT=( j). (17)

Proof. Let us consider the general form and theexpected version of the algorithm (16):

*A( j)"!jjEMy( j)yH( j)!INA( j). (18)

According to [15], the following algorithm is con-vergent because the channel matrix H is full columnrank, and R

s(0)"I,

*B( j)"!jj[B( j)BH( j)!I]B( j), (19)

where B( j)"A( j)H. Consider the following algo-rithm:

*C( j)"!gj[C( j)JCH( j)!J]JTC( j), (20)

where J is a (d#1)](d#1) shifting matrix,C( j)3R(d`1)C(d`1). It is clear that the "rst row ofthe matrix [C( j)JCH( j)!J]JT is always zero.Thus, the "rst row of C( j) is not changed when thealgorithm (20) is used. Applying an operator JT toEq. (20), we have

JT*C( j)"!gjJT[C( j)JCH( j)!J]JTC( j). (21)

The tth row of C( j) and *C( j) are permuted to(t!1)th row of JTC( j) and JT*C( j), respectively,t"2,3,2d#1. While the last row of JTC( j) andJT*C( j) are zero. Eq. (20) can be rewritten as

A*CM ( j)

2

0 B"!gj GA

CM ( j)

2

0 B (CM H( j) F 0)

!AI 0

0 0B HACM ( j)

2

0 B, (22)

where C( j)"(CT1( j) F CM ( j))T, C

1( j) is the "rst row of

C( j), and I is a d]d identity matrix. If CM ( j)"B( j),Eq. (22) is equivalent to algorithm (19). So, CM ( j) isconvergent when the algorithm (20) is used andCM ( j)JCM H( j) converges to I. Based on the abovediscussion, algorithm (20) is convergent, andC( j)JCH( j) asymptotically converges to J if the "rstrow of C( j) is initially set to [1,0,2,0]. Algorithm(20) is valid for all data (matrix) size, which makesuse of the following algorithm in training theweight matrix for the condition (11) possible:

*=( j)"!gjEMy( j)yH( j#1)!JNJT=( j), (23)

40 Y. Fang et al. / Signal Processing 76 (1999) 37}42

Fig. 2. The output of the unequalized channel.Fig. 4. NRMSISI versus the training cycles of the neural net-work with 100 Monte Carlo tests.

Fig. 3. The output of the neural network.

where gj

is the learning rate, J is a d]d shiftingmatrix. In practice, the expectation valuesEMy( j)yH( j#1)!JN are not available and they areapproximated by their instantaneous values. Thestochastic version of the algorithm is Eq. (17).

Based on the above analysis, we form a bigradi-ent algorithm from Eqs. (16) and (23) for the neuralnetwork,

=( j#1)"=( j)!jj[ y( j)yH( j)!I]=( j)

!gj[ y( j)yH( j#1)!J]JT=( j). (24)

By applying the algorithm, both conditions (10)and (11) can be approximated simultaneously. Thealgorithm operates recursively until the weightmatrix= reaches a de"ned level of accuracy. Thesample vectors x( j) can be used for a number oftimes for achieving convergence.

5. Simulation results

To demonstrate the convergence property of theneural network and the e$ciency of the proposedblind equalization method, we simulated a three-ray multipath channel as in [17]. The source sym-bols were drawn from the QPSK signal constella-tion with a uniform distribution. In theimplementation of the proposed algorithm, wechose t

0"0, and an observation interval of length

¸"3¹. The channel outputs were sampled twiceas fast as the symbol rate. Fig. 2 shows 400 outputsof unequalized channel for 205 symbols. The 200

input vectors were used 60 times sequentially intraining the neural network in Fig. 1. The learningparameter j

jand g

jwere 0.0028 in Eq. (24). The

initial weight matrix of the neural network is set as="I. The output of neural network shown in Fig.3 indicates that the channel is well equalized. Toobtain a performance measure of the symbol es-timation, the normalized root-mean square inter-symbol interference (NRMSISI) is de"ned as

NRMSISI"1

EsES1

N5

+N5

i/1Es(

(i)!a(

(i)sE2, (25)

where N5is the number of Monte Carlo trials, s(

(i)is

the estimate of the inputs from the ith trial,a((i)"(1/N)+N

k/1s( (i)k/s

kis the estimate of the scale

factor, N is the number of used symbols. Fig. 4 isa plot of the NRMSISI versus the training cycles ofthe neural network for 100 Monte Carlo trials.

Y. Fang et al. / Signal Processing 76 (1999) 37}42 41

6. Conclusions

Based on the property of the correlation matricesof the transmitted symbols, we derived a su$cientand necessary condition for blind equalization.The results enable us to design a blind equalizerusing a single layer linear neural network. In thispaper, we also proposed a stochastic approximatelearning algorithm for the neural network.The architecture of the neural network is simpleand can be implemented for on-line operation.Promising simulation results demonstrate that thee$ciency of the proposed blind equalizationmethod is e$cient.

References

[1] S.I. Amari, A. Cichocki, Adaptive blind signal processing} neural network approaches, Proc. IEEE 86 (1998)2026}2048.

[2] X.R. Cao, J. Zhu, Blind equalization with a linear feedfor-ward neural network, in: Proc. 5th European Symp. onArti"cial Neural Networks, ESANN'97, Bruges, Belgium,April 1997, pp. 267}272.

[3] B. Friedlander, B. Porat, Asymptotically optimal estima-tion of MA and ARMA parameters of non-Gaussian pro-cess from high-order statistics, IEEE Trans. Automat.Control 35 (1990) 27}36.

[4] G.B. Giannakis, J.M. Mendel, Identi"cation of nonmini-mum phase systems using higher order statistics, IEEETrans. Acoust. Speech Signal Process. 37 (1989) 360}377.

[5] G.B. Giannakis, A. Swami, On estimating non-minimumphase ARMA models of non-Gaussian processes, IEEETrans. Acoust. Speech Signal Process. 38 (1990) 478}495.

[6] Y. Hua, Fast maximum likelihood for blind identi"cationof multiple FIR channels, IEEE Trans. Signal Process. 44(1996) 661}672.

[7] M. Kristensson, B. Ottersten, Statistical analysis of a sub-space method for blind channel identi"cation, in: Proc.IEEE ICASSP, Atlanta, USA, Vol. 5, 1996, pp. 2435}2438.

[8] K.S. Lii, M. Rosenblatt, Deconvolution and estimation oftransfer function phase and coe$cient for non-Gaussianlinear processes, Ann. Statist. 10 (1982) 1195}1208.

[9] H. Liu, G. Xu, A deterministic approach to blind symbolestimation, IEEE Signal Process. Lett. 1 (1994).

[10] H. Liu, G. Xu, Closed-form blind symbol estimation indigital communications, IEEE Trans. Signal Process. 43(1995) 2714}2723.

[11] J.M. Mendel, A. Swami, ARMA parameter estimationusing only output cumulants, IEEE Trans. Acoust. SpeechSignal Process. 38 (1990) 1257}1265.

[12] E. Moulines, P. Duhamel, J.F. Cardoso, S. Mayrargue,Subspace methods for the blind identi"cation of multi-channel "r "lters, IEEE Trans. Signal Process. 43 (1995)516}525.

[13] C.L. Nikias, ARMA bispectrum approach to nonmini-mum phase system identi"cation, IEEE Trans. Acoust.Speech Signal Process. 36 (1988) 513}525.

[14] W. Qiu, Y. Hua, A GCD method for blind channel identi-"cation, Digital Signal Process. 7 (1997) 199}205.

[15] F.M. Silva, L.B. Almeida, A distributed solution for dataorthonormalization, in: Proc. ICANN, Espoo, Finland,Vol. 2, 1991, pp. 943}948.

[16] D.T.M. Slock, Blind fractional space equalization, per-fect-reconstruction "lter back and multichannel linear pre-diction, in: Proc. IEEE ICASSP'94, Adelaide, Australia,1994, pp. 585}588.

[17] L. Tong, G. Xu, T. Kailath, Blind identi"cation and equal-ization based on second-order statistics: a time domainapproach, IEEE Trans. Inform. Theory 40 (1994) 340}349.

[18] J.K. Tugnait, Identi"cation of non-minimum phase linearstochastic systems, Automatica 22 (1986) 454}464.

[19] J.K. Tugnait, Identi"cation of linear stochastic system viasecond and fourth-order cumulant matching, IEEE Trans.Inform. Theory 33 (1987) 393}407.

[20] G. Xu, H. Lui, L. Tong, T. Kailath, A least-squares ap-proach to blind channel identi"cation, IEEE Trans. SignalProcess. 43 (1995) 2982}2993.

42 Y. Fang et al. / Signal Processing 76 (1999) 37}42