adaptive bayesian multiuser detection for synchronous cdma with gaussian and impulsive noise

IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 47, NO. 7, JULY 2000 2013

Adaptive Bayesian Multiuser Detection forSynchronous CDMA with Gaussian and Impulsive

NoiseXiaodong Wang, Member, IEEE,and Rong Chen

Abstract—We consider the problem of simultaneous parameterestimation and data restoration in a synchronous CDMA systemin the presence of either additive Gaussian or additive impulsivewhite noise with unknown parameters. The impulsive noise ismodeled by a two-term Gaussian mixture distribution. Bayesianinference of all unknown quantities is made from the superim-posed and noisy received signals. The Gibbs sampler (a Markovchain Monte Carlo procedure) is employed to calculate theBayesian estimates. The basic idea is to generate ergodic randomsamples from the joint posterior distribution of all unknown andthen to average the appropriate samples to obtain the estimates ofthe unknown quantities. Adaptive Bayesian multiuser detectorsbased on the Gibbs sampler are derived for both the Gaussiannoise synchronous CDMA channel and the impulsive noisesynchronous CDMA channel. A salient feature of the proposedadaptive Bayesian multiuser detectors is that they can incorporatethe a priori symbol probabilities, and they produce as output thea posteriori symbol probabilities. (That is, they are “soft-inputsoft-output” algorithms.) Hence, these methods are well suited foriterative processing in a coded system, which allows the adaptiveBayesian multiuser detector to refine its processing based on theinformation from the decoding stage, and vice versa—a receiverstructure termed adaptive Turbo multiuser detector.

Index Terms—Adaptive multiuser detection, Bayesian inference,Gibbs sampler, impulsive noise, iterative processing.

I. INTRODUCTION

T HE THEME of this paper is to treat three related problemsin multiuser detection under a general Bayesian frame-

work. These problems are

i) optimal multiuser detection in the presence of unknownchannel parameters;

ii) optimal multiuser detection in impulsive ambient noise;iii) multiuser detection for coded CDMA systems.

We first provide a perspective on the related works in these threeareas.

Optimal Multiuser Detection with Unknown Parame-ters: The optimal multiuser detection algorithms with known

Manuscript received June 29, 1999; revised February 17, 2000. This work wassupported in part by the Interdisciplinary Research Initiatives Program, TexasA&M University. X. Wang was supported in part by the U.S. National ScienceFoundation under Grant CAREER CCR-9875314. R. Chen was supported inpart by the U.S. National Science Foundation under Grant DMS 9626113. Theassociate editor coordinatig the review of this paper and approving it for publi-cation was Dr. Rick S. Blum.

X. Wang is with the Department of Electrical Engineering, Texas A&M Uni-versity, College Station, TX 77843 USA (e-mail: [email protected]).

R. Chen is with the Department of Statistics, Texas A&M University, CollegeStation, TX 77843 USA.

Publisher Item Identifier S 1053-587X(00)04935-7.

channel parameters, that is, the multiuser maximum-likelihoodsequence detector (MLSD) and the multiuser maximumaposteriori symbol probability (MAP) detector, were firstinvestigated in [46] and [47] (cf. [49]). The analysis of thecomputational complexity and the proof that the optimal mul-tiuser detection problem is combinatorially hard appeared in[46] and [48]. When the channel parameters (e.g., the receivedamplitudes and the noise variance) are unknown, it is of interestto study the problem of joint multiuser channel parameterestimation and data detection from the received waveform. Thisproblem was first treated in [36], where a solution based on theexpectation-maximization (EM) algorithm is derived. In [42],the problem of sequential multiuser amplitude estimation in thepresence of unknown data is studied, and an approach basedon stochastic approximation is proposed. In [55], a tree-searchalgorithm is given for joint data detection and amplitudeestimation. Other works concerning multiuser detection withunknown channel parameters include [14], [23], [24], [33],[35], and [43].

Multiuser Detection with Impulsive Noise:To date, mostof the work on multiuser detection assumes that the channelambient noise is Gaussian. However, in many physical chan-nels where multiuser detection may be applied, such as urbanand indoor radio channels [8], [9], [28], [29], [31], [32] andunderwater acoustic channels [11], [30], the ambient noise isknown through experimental measurements to be decidedlynon-Gaussian, due to the impulsive nature of the man-madeelectromagnetic interference and a great deal of naturalnoise as well. The results of an early study of error rates innon-Gaussian CDMA channels are found in [1]–[3], in whichthe performance of the conventional and modified conventionaldetectors is shown to depend significantly on the shape ofthe ambient noise distribution. In [37], it is observed that theperformance gains afforded by maximum likelihood multiuserdetection in impulsive noise can be substantial when comparedwith optimum multiuser detection based on a Gaussian noiseassumption. In [52], robust multiuser detection methods forimpulsive noise CDMA channels based on the Huber robustregression technique are proposed.

Multiuser Detection for Coded CDMA:Most CDMA sys-tems employ error control coding to protect the transmitted datafrom being corrupted by the channel. Some recent work has ad-dressed multiuser detection for coded CDMA systems. In [18],the optimal decoding scheme for convolutionally coded CDMAsystem is studied, which is shown to have a prohibitive com-putational complexity. In [19], some low-complexity receivers

1053–587X/00$10.00 © 2000 IEEE

2014 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 47, NO. 7, JULY 2000

Fig. 1. Coded synchronous CDMA communication system.

that perform multiuser symbol detection and decoding eitherseparately or jointly are studied. In [34], [38], and [51], Turbomultiuser detection schemes for coded CDMA systems are pro-posed, which iterate between multiuser detection and channeldecoding to successively improve the receiver performance.

In this paper, we present novel adaptive Bayesian multiuserdetection techniques for synchronous CDMA communicationswith unknown channel parameters in both Gaussian andimpulsive ambient noise channels. The impulsive noise ismodeled by a two-term Gaussian mixture distribution. Weconsider Bayesian inference of all unknown quantities (e.g.,the received amplitudes, the data symbols, and the noiseparameters) from the received waveforms. A Markov chainMonte Carlo procedure called the Gibbs sampler is employedto calculate the Bayesian estimates. The performance of theproposed adaptive multiuser detectors is demonstrated viasimulations. The proposed Bayesian multiuser detectors cannaturally exploit the structure of the coded signals. Anothersalient feature of the proposed methods is that being soft-inputsoft-output demodulation algorithms, they can be used in con-junction with soft channel decoding algorithm to accomplishiterative joint adaptive multiuser detection and decoding—theso-calledadaptive Turbo multiuser detection.

The rest of the paper is organized as follows. In Section II,the system under study is described. In Section III, somebackground material on the Gibbs sampler is provided. Theproblems of adaptive Bayesian multiuser detection in Gaussiannoise and impulsive noise synchronous CDMA channels aretreated in Sections IV and V, respectively. In Section VI,an adaptive Turbo multiuser detection scheme is presented.Some discussions, including a decoder-assisted convergenceassessment scheme and a code-constrained Bayesian multiuserdetector, are found in Section VII. Simulation results areprovided in Section VIII. Finally, Section IX contains theconclusions.

II. SYSTEM DESCRIPTION

We consider a coded synchronous CDMA system withusers, employing normalized modulation waveforms

and signaling through a channel with additivewhite noise. The block diagram of the transmitter end of such asystem is shown in Fig. 1. The binary information bitsfor user are encoded using some channel code (e.g., blockcode, convolutional code, or Turbo code), resulting in a codebit stream . A code-bit interleaver is used to reduce

the influence of the error bursts at the input of the channeldecoder. The interleaved code bits are then mapped to BPSKsymbols, yielding symbol stream . Each data symbol isthen modulated by a spreading waveformand transmittedthrough the channel. The received signal is the superposition ofthe users’ transmitted signals plus the ambient noise, whichis given by

(1)

In (1), is the number of data symbols per user per frame;, , and denote, respectively, the amplitude, theth

symbol, and the normalized spreading waveform of theth user;and is a zero-mean whitenoise vector. The spreading waveform is of the form

(2)

where is the spreading factor. It is assumed that the receiverknows the spreading waveforms of all active users in the system.Define the followinga priori symbol probabilities:

(3)

Note that when no prior information is available, then, i.e., all symbols are equally likely.

It is further assumed that the additive ambient channelnoise vector is a sequence of zero-mean independentand identically distributed (i.i.d.) random vectors, and it isindependent of the symbol sequences . Moreover,the noise vector is assumed to consist of i.i.d. samples

. In this paper, we consider two types of noisedistributions corresponding to the additive Gaussian noise andthe additive impulsive noise, respectively. For the former case,the noise is assumed to have a Gaussian distribution, i.e.,

(4)

where is the variance of the noise. For the latter case, thenoise is assumed to have a two-term Gaussian mixturedistribution, i.e.,

(5)

WANG AND CHEN: ADAPTIVE BAYESIAN MULTIUSER DETECTION FOR SYNCHRONOUS CDMA 2015

with and . Here, the term rep-resents the nominal ambient noise, and the term rep-resents an impulsive component, withrepresenting the prob-ability that an impulse occurs. The total noise variance underdistribution (5) is given by

(6)

Denote . In Sections IVand V, we consider the problem of estimating thea posterioriprobabilities of the transmitted symbols

(7)

based on the received signals and the prior information, without knowing the channel amplitudes

and the noise parameters (i.e., for Gaussian noiseand , , and for impulsive noise). Thesea posterioriprobabilities are then used by the channel decoder to decodethe information bits shown in Fig. 1, which will bediscussed in Section VI.

III. GIBBS SAMPLER

Over the last decade or so, a large body of methods hasemerged based on iterative Monte Carlo techniques that areespecially useful in computing Bayesian solutions to estimationproblems with high parameter dimensions. These methods arebased on the theory of Markov chain limiting behavior and arecollectively known asMarkov chain Monte Carlo(MCMC)techniques [44]. Most of these methods are aimed at estimatingthe entire posterior density and not just finding the maximumaposteriori(MAP) estimates of the parameters. One of the mostpopular of these methods is known as theGibbs sampler[15],which is described next.

Let be a vector of unknown parameters,and let be the observed data. Suppose that we are interested infinding thea posteriorimarginal distribution of some parameter,say, , conditioned on the observation, i.e., ,

. Direct evaluation involves integrating out the rest of theparameters from the jointa posterioridensity, i.e.,

(8)In most cases, such a direct evaluation is computationally infea-sible, especially when the parameter dimensionis large. TheGibbs sampler is a Monte Carlo procedure for numerical evalua-tion of the above multidimensional integral. The basic idea is togenerate random samples from the joint posterior distribution

and then to estimate any marginal distribution usingthese samples. Given the initial values ,this algorithm iterates the following loop:

• Draw sample from .• Draw sample from , , , ,

....

• Draw sample from .Under regularity conditions, in the steady state, the sequence ofsample vectors is a realization ofa homogeneous Markov chain with the transition kernel fromstate to state given by

(9)

The convergence behavior of the Gibbs sampler is investigatedin [13], [15], [17], [26], [41], and [45], and general conditionsare given for the following two results:

• The distribution of converges geometrically to, as .

• , as ,

for any integrable function .The Gibbs sampler requires an initial transient period to con-verge to equilibrium. The initial period of length is knownas the “burn-in” period, and the first samples should alwaysbe discarded. Detecting convergence is usually done in someadhocway. Some methods are found in [44].

The roots of MCMC methods can be traced back to the well-known Metropolis algorithm [27], which was initially used toinvestigate the equilibrium properties of molecules in a gas. Thefirst use of the Metropolis algorithm in a statistical context isfound in [22]. The Gibbs sampler, which is a special case of theMetropolis algorithm, was so termed in the seminal paper [17]on image processing. It is brought to statistical prominence by[15], where it was observed that many Bayesian computationcould be carried out via the Gibbs sampler. For tutorials on theGibbs sampler, see [4] and [12].

IV. A DAPTIVE BAYESIAN MULTIUSERDETECTION INGAUSSIAN

NOISE

In this section, we consider the problem of computing thea posterioriprobabilities in (7) under the assumption that theambient noise distribution is Gaussian. That is, the pdf ofin (1) is given by

(10)

Denote


Then, (1) can be written as

(11)

(12)

The problem is solved under a Bayesian framework. First, theunknown quantities , , and are regarded as realizationsof random variables with some prior distributions. The Gibbssampler, which is a Monte Carlo method, is then employed tocalculate the maximuma posteriori(MAP) estimates of theseunknowns.

A. Bayesian Inference

Assume that the unknown quantities, and are indepen-dent of each other and have prior distributions , , and

, respectively. Since is a sequence of indepen-dent Gaussian vectors, using (10) and (11), the joint posteriordistribution of these unknown quantities based onthe received signal takes the form of

(13)

where is a normalization constant independent of the un-known parameters . Thea posterioriprobabilities (7)of the transmitted symbols can then be calculated from the jointposterior distribution (13) according to

(14)

Clearly, the computation in (14) involves multidimen-sional integrals, which is certainly infeasible for any practicalimplementations. To avoid the direct evaluation of the Bayesianestimate (14), we resort to the Gibbs sampler discussed in Sec-tion III. The basic idea is to generate ergodic random samples

, , : , from the poste-rior distribution (13), and then to average : ,

to obtain an approximation of thea posterioriprob-abilities in (14).

B. Prior Distributions

General Considerations:Noninformative Priors: In Bayesian analysis, prior distri-

butions are used to incorporate the prior knowledge about theunknown parameters. When such prior knowledge is limited,the prior distributions should be chosen such that they have aminimal impact on the posterior distribution. Such priors are

termednoninformative. The rationale for using noninformativeprior distributions is to “let the data speak for themselves,” sothat inferences are unaffected by information external to currentdata [10], [16].

Conjugate Priors: Another consideration in the selectionof the prior distributions is to simplify computations. To thatend,conjugate priorsare usually used to obtain simple analyt-ical forms for the resulting posterior distributions. The propertythat the posterior distribution belongs to the same distributionfamily as the prior distribution is called conjugacy. The conju-gate family of distributions is mathematically convenient in thatthe posterior distribution follows a known parametric form [10],[16]. Finally, to make the Gibbs sampler more computationallyefficient, the priors should also be chosen such that the condi-tional posterior distributions are easy to simulate.

For an introductory treatment of the Bayesian philosophy, in-cluding the selection of prior distributions, see [10], [16], and[25]. An account of criticism of the Bayesian approach to dataanalysis can be found in [5] and [40], and a defense of “TheBayesian Choice” can be found in [39].

Prior Distributions of the Unknowns:Following the generalguidelines in Bayesian analysis [10], [16], [25], we choose theconjugate prior distributions for the unknown parameters ,

and , as follows.For the unknown amplitude vector, a truncated Gaussian

prior distribution is assumed

(15)

where is an indicator that is 1 if all elements ofarepositive and it is zero otherwise. Note that large value ofcor-responds to the less-informative prior. Note also that althoughuser amplitudes are assumed to be independent, the matrixis not restricted to be diagonal. For the noise variance, an in-verse chi-square prior distribution is assumed

(16)

or

(17)

The small value of corresponds to the less informative priors(roughly the prior knowledge is worth data points). The valueof reflects the prior belief of the value of . Finally, sincethe symbols are assumed to be independent, theprior distribution can be expressed in terms of the priorsymbol probabilities defined in (3) as

(18)

where is the indicator such that if andif .


C. Conditional Posterior Distributions

The following conditional posterior distributions are requiredby the Gibbs multiuser detector in Gaussian noise. The deriva-tions are found in Appendix A.

1) The conditional distribution of the amplitude vector,given , and , is given by

(19)

with

(20)

and

(21)

where in (20) .2) The conditional distribution of the noise variance

given , and is given by

(22)

or

(23)

with

(24)

3) The conditional probabilities of , given , ,, and , can be obtained from [where denotes

the set containing all elements of except for ]

(25)

where.

D. Gibbs Multiuser Detector in Gaussian Noise

Using the above conditional posterior distributions, the Gibbssampling implementation of the adaptive Bayesian multiuserdetector in Gaussian noise proceeds iteratively as follows. Giventhe initial values of the unknown quantities , ,drawn from their prior distributions, and for , wehave the following.

1)(19).

2)(23).

3)

(25)

Note that to draw samples offrom (15) or (19), the so-calledrejection method[50] can be used. For instance, after a sample isdrawn from or , check to see if the con-straint , is satisfied; if not, the sample isrejected, and a new sample is drawn from the same distribution.The procedure continues until a sample is obtained that satisfiesthe constraint.

To ensure convergence, the above procedure is usually car-ried out for iterations, and samples from the lastiterations are used to calculate the Bayesian estimates of the un-known quantities. In particular, thea posteriorisymbol proba-bilities in (14) are approximated as

(26)

where is the indicator such that if andif . An MAP decision on the symbol

is then given by

(27)

Furthermore, if desired, the estimates of the amplitude vectorand the noise variance can also be obtained from the corre-sponding sample means

(28)

and

(29)

The posterior variances of and , which reflect the uncer-tainty in estimating these quantities on the basis of, can alsobe approximated by the sample variances, as

(30)


and

Var

(31)

Note that the above computations are exact in the limit as. However, since they involve only a finite number of samples,

we think of them as approximations but realize that in theoryany order of precision can be achieved given sufficiently largesample size .

The complexity of the above Gibbs multiuser detectorper it-eration is , i.e., it has a term that is quadraticwith respect to the number of users[due to the inversion ofthe positive definite symmetric matrix in (20)] and a term that islinear with respect to the symbol block size. The total com-plexity is then . This is a substan-tial complexity reduction compared with the direct implemen-tation of the Bayesian symbol estimate (14), whose complexityis .

V. ADAPTIVE BAYESIAN MULTIUSER DETECTION IN IMPULSIVE

NOISE

In the previous section, we assumed that the distribution ofthe ambient channel noise is Gaussian, as did in most of theprevious work on multiuser detection. However, in many re-alistic communication channels, especially wireless channels,the ambient noise is known to be decidedly impulsive, due toimpulsive phenomena [8], [9]. In this section, we develop theGibbs multiuser detector in impulsive noise. It is assumed thatthe noise samples of in (1) are independentwith a common two-term Gaussian mixture pdf, which is givenby

(32)

with and . This model serves as an approxi-mation to the more fundamental Middleton Class A noise model[30], [31], [56] and has been used extensively to model physicalnoise arising in radar, acoustic, and mobile radio channels.

A. Prior Distributions

Define the following indicator random variable to indicate thedistribution of the noise sample

if ,

if ,(33)

Denote , and

(34)

The unknown quantities in this case are, , , , , . Thejoint posterior distribution of these unknown quantities based onthe received signal takes the form of

(35)

where is the number of’s in , , , ,. [Note that .] We next specify the

conjugate prior distributions of the unknown quantities in (35).As in the case of Gaussian noise, the prior distributions

and are given, respectively, by (15) and (18). For the noisevariances , , independent inverse chi-square distribu-tions are assumed, i.e.,

with (36)

For the impulse probability, a prior of Beta distribution is as-sumed, i.e.,

Beta (37)

Note that the value reflects the prior knowledge ofthe value of . Moreover, reflects the strength of theprior belief, i.e., roughly the prior knowledge is worthdata points. Given, the conditional distribution of the indicator

is then

and (38)

(39)

with

B. Conditional Posterior Distributions

The following conditional posterior distributions are requiredby the Gibbs multiuser detector in impulsive noise. The deriva-tions are found in Appendix B.

1) The conditional distribution of the amplitude vectorgiven , , , , , and is

(40)

with

(41)

and

(42)


2) The conditional distribution of the noise variancegiven , , , , , and is given by [here if

, and if ].

(43)

with

(44)

In (44), is the indicator function such that it is1 if , and it is 0 if ; is the th row ofthe spreading waveform matrix, .

3) The conditional probability of , given , ,, , , , and can be obtained from [where

denotes the set containing all elements ofexcept for]

(45)

where.

4) The conditional distribution of , given , , , ,, , and is given by [ denotes the set containing

all elements of except for ]

(46)

5) The conditional distribution of, given , , , , ,and is given by

(47)

C. Gibbs Multiuser Detector in Impulsive Noise

Using the above conditional posterior distributions, theGibbs sampling implementation of the adaptive multiuserdetector in impulsive noise proceeds iteratively as follows.Given initial values of the unknown quantities , ,

, , , drawn from their prior distributions, andfor , we have the following.

1) , , , ,, (40).

2) , , , ,, (43);

, , , ,, (43).

3)

, ,, , , ,

(45),

4)

, , ,, , , (46),

5) , , , , ,(47).

As in the case of Gaussian noise, thea posteriorisymbol prob-abilities are computed using (26). Theaposteriorimeans and variances of the other unknown quantitiescan also be computed similar to (28)–(31).

The complexity of the above Gibbs multiuser detector isper iteration. Note that the direct

implementation of the Bayesian symbol estimate based on (35)has a computational complexity of .

VI. I TERATIVE JOINT MULTIUSER DETECTION AND

DECODING—ADAPTIVE TURBO MULTIUSER DETECTION

Since the discovery of the powerful Turbo codes [6], [7], con-siderable attention has been redrawn to iterative (“Turbo”) pro-cessing techniques. The so-called Turbo principle can be suc-cessfully applied to many detection/decoding problems such asserial concatenated decoding, equalization, coded modulation,multiuser detection, and joint source and channel decoding [21].In this section, we consider employing iterative joint multiuserdetection and decoding to improve the performance of the adap-tive Bayesian multiuser detector in a coded CDMA system. Be-cause it utilizes thea priori symbol probabilities, and it producessymbol (or bit)a posterioriprobabilities, the adaptive Bayesianmultiuser detectors developed in this paper is well suited for it-erative processing, which allows the adaptive multiuser detectorto refine its processing based on the information from the de-coding stage, and vice versa. In [51], a Turbo multiuser receiveris developed for coded CDMA systems with Gaussian noise,under the assumption that the received amplitudes and the noisevariance areknownto the receiver. In what follows, we develop


Fig. 2. Iterative processing for joint Bayesian multiuser detection and decoding—adaptive Turbo multiuser detection.

adaptive Turbo multiuser receivers for both Gaussian and im-pulsive noise CDMA channels, withunknownamplitudes andnoise parameters.

The iterative (Turbo) receiver structure is shown in Fig. 2.It consists of two stages: i) the adaptive multiuser detector de-veloped in the previous sections followed by ii) a soft-inputsoft-output channel decoder. The two stages are separated bydeinterleavers and interleavers. As discussed in the previoussections, the adaptive multiuser detector delivers thea poste-riori symbol probabilities . Basedon these, we first compute thea posteriorilog-likelihood ratiosof a transmitted “ ” symbol and a transmitted “ ” symbol

(48)

Using the Bayes’ rule, (48) can be written as

(49)

where the second term in (49), which is denoted by ,represents thea priori LLR of the code bit , which is com-puted by the channel decoder in the previous iteration, inter-leaved, and then fed back to the adaptive Bayesian multiuserdetector. (The superscriptindicates the quantity obtained fromthe previous iteration). For the first iteration, assuming equallylikely code bits, i.e., no prior information available, we thenhave , , . Thefirst term in (49), which is denoted by , represents theextrinsic information delivered by the adaptive Bayesian mul-tiuser detector, based on the received signals, the structureof the multiuser signal given by (1), and the prior informationabout all other code bits. The extrinsic information ,which is not influenced by thea priori informationprovided by the channel decoder, is then reverse interleaved andfed into the channel decoder, as thea priori information in thenext iteration.

Based on the extrinsic information of the code bitsand the structure of the channel code,

the soft-input soft-output channel decoder computes theaposterioriLLR of each code bit [51], as shown in (50), shownat the bottom of the page. It is seen from (50) that the outputof the soft-input soft-output channel decoder is the sum of theprior information , and theextrinsic information

delivered by the channel decoder. This extrinsicinformation is the information about the code bitgleaned from the prior information about the other code bits

based on the constraint structure of the code.The soft channel decoder also computes thea posterioriLLRof every information bit, which is used to make decisionon the decoded bit at the last iteration. After interleaving,the extrinsic information delivered by the channel decoder

is then used to compute thea priori

symbol distributions defined in (7) from thecorresponding LLR’s as follows. Since

after some manipulations, we have

(51)

The symbol probabilities are then fed back tothe adaptive Bayesian multiuser detector as the prior informa-tion for the next iteration. Note that at the first iteration, theextrinsic information and are statisti-cally independent. However, subsequently, since they use thesame information indirectly, they will become increasingly cor-related, and finally, the improvement through the iterations willdiminish.

code constraints

code constraints(50)


VII. D ISCUSSIONS

A. Decoder-Assisted Convergence Assessment

Detecting convergence in the Gibbs sampler is usually donein somead hocway. Some methods can be found in [44]. Oneof them is to monitor a sequence of weights that measure thediscrepancy between the sampled and the desired distribution.In the application considered here, since the adaptive multiuserdetector is followed by a bank of channel decoders, we can as-sess convergence by monitoring the number of bit correctionsmade by the channel decoders. If this number exceeds somepredetermined threshold, then we decide that convergence isnot achieved. In that case, the Gibbs multiuser detector will beapplied again to the same data block. The rationale is that ifthe Gibbs sampler has reached convergence, then the symbol(and bit) errors after multiuser detection should be relativelysmall. On the other hand, if convergence is not reached, thenthe code bits generated by the multiuser detector are virtuallyrandom and do not satisfy the constraints imposed by the codetrellises. Hence, the channel decoders will make a large numberof corrections. Note that there is no additional computationalcomplexity for such a convergence detection. We only need tocompare the signs of the code-bit log-likelihood ratios at theinput and the output of the soft channel decoder to determinethe number of corrections made.

B. Code-Constrained Gibbs Multiuser Detectors

Another approach to exploiting the coded signal structure inadaptive Bayesian multiuser detection is to make use of the codeconstraints in the Gibbs sampler. For instance, suppose that theuser information bits are encoded by someblock codeof length

and that the code bits arenot interleaved. Then, one signalframe of symbols contains code words, with theth code word given by

Let be the set of all valid code words for user. Now, inthe Gibbs sampler, instead of drawing each individual symbol

once a time according to (25) or (45), we draw a codeword of symbols from each time. Specifically, let

denote the code word with all entries being “”s (this isthe so-called all-zero code word, and it is always a valid codeword for any block code [53]). If the ambient channel noiseis Gaussian, then for any code word , the conditionalprobability of , given the values of the rest of theunknowns, can be obtained from

(52)

where denotes the set containing all elements of, ex-

cept for ; ; and ,, , , , . On the other hand, if the

ambient channel noise is impulsive, we have

(53)

The conditional distributions for sampling the other unknownsremain the same as before. The advantage of sampling a codeword instead of sampling an individual symbol is that it can sig-nificantly improve the accuracy of samples drawn by the Gibbssampler since only valid code words are drawn. This will bedemonstrated by some simulation examples in the next section.

C. Relationship Between the Gibbs Sampler and the EMAlgorithm

The expectation-maximization (EM) algorithm has also beenapplied to joint parameter estimation and multiuser detection[36]. The major advantage of the Gibbs sampling techniqueover the EM algorithm is that the Gibbs sampler is aglobaloptimization technique. The EM algorithm is alocal optimiza-tion method, and it can easily get trapped by local extrema onthe likelihood surface. The EM method performs well if theinitial estimates of the channel and symbols are close to theirtrue values. On the other hand, the Gibbs sampler is guaran-teed to converge to the global optimum with any random initial-ization. Of course, the convergence rate crucially depends onthe shape the joint posterior density surface. When the poste-rior distribution has several modes separated by very low den-sity regions (energy gap), then the Gibbs sampler that generates“random walks” according to the distribution may have diffi-culties crossing such gaps in visiting all modes. If such a gap issevere, then the random walk may get stuck within one modefor a long time before it moves to another mode. Many modifi-cations of the Gibbs sampler have been developed to combat the“large energy gap” situation. For example, see [20] and [54].

VIII. SIMULATIONS

A. Simulation Setup

In this section, we provide a number of simulation examplesto illustrate the performance of the adaptive Bayesian multiuserdetectors developed in this paper. We consider a five-user () synchronous CDMA channel with processing gain .


The user-spreading waveform matrixand the correspondingcorrelation matrix are given, respectively, by

In all the simulations described in this section, the followingnoninformative conjugateprior distributions are used in theGibbs sampler. For the case of Gaussian noise

and for the case of impulsive noise

Beta

Note that the performance of the Gibbs sampler is insensitive tothe values of the parameters in these priors, as long as the priorsare noninformative.

In adaptive Turbo multiuser detection, for the first iteration,the prior symbol probabilitiesfor all symbols; in the subsequent iterations, the prior symbolprobabilities are provided by the channel decoder, as given by(51). The data block size of each user is . For eachdata block, the Gibbs sampling is performed for 100 iterations,with the first 50 iterations as the “burn-in” period. The decoder-assisted convergence assessment is employed. Specifically, ifthe number of bit corrections made by the decoder exceeds onethird of the total number of bits (i.e., ), then it is decided thatconvergence is not reached, and the Gibbs sampler is applied tothe same data block again.

B. Convergence Behavior of the Gibbs Multiuser Detectors

Example 1: We first illustrate the convergence behavior ofthe proposed adaptive Bayesian multiuser detector in Gaussiannoise. In this example, the user amplitudes and the noise vari-ance are given by

dB dB dB

dB dB dB

In Fig. 3, we plot the first 100 samples drawn by the Gibbssampler of the parameters , , , , and . Thecorresponding true values of these quantities are also shown inthe same figure as the straight lines. Note that in this case, the

Fig. 3. Samples and histograms—Gaussian noise.A = �4 dB,A = �2dB,A = 0 dB,A = 2 dB,A = 4 dB, and� = �2 dB. The histogramsare based on 500 samples collected after the initial 50 iterations.

Fig. 4. Samples and histograms—impulsive noise.A = �4 dB,A = �2

dB,A = 0 dB,A = 2 dB,A = 4 dB, � = 0:1, � =� = 100,� = (1��)� + �� = 7 dB. The histograms are based on 500 samples collected afterthe initial 50 iterations.

number of unknown parameters is (i.e., , ,and ). Remarkably, it is seen that the Gibbs sampler reachesconvergence within about 20 iterations. The marginal posteriordistributions of the unknown parameters, , and in thesteady state can be illustrated by the corresponding histograms,which are also shown in Fig. 3. The histograms are based on 500samples collected after the initial 50 iterations.

Example 2: We next illustrate the convergence behavior ofproposed adaptive Bayesian multiuser detector in impulsivenoise. The user amplitudes are the same as in Example 1.The noise samples are generated according to the two-termGaussian model (5) with the following parameters:

dB

The first 100 samples drawn by the Gibbs sampler of the param-eters , , , , , and are shown in Fig. 4. Thecorresponding true values of these quantities are also shown in


(a)

(b)

Fig. 5. Bit error rate performance—convolutional code, Gaussian noise. Allusers have the same amplitudes.

the same figure as the straight lines. Note that in this case, thenumber of unknown parameters is (i.e., ,

, , , and ). It is seen that as in the Gaussian noise case,the Gibbs sampler converges within about 20 samples. The his-tograms of the unknown parameters, , , and are alsoshown in Fig. 4, which are based on 500 samples collected afterthe initial 50 iterations.

C. Performance of the Adaptive Turbo Multiuser Detectors

Examples 3 and 4:We now illustrate the performance ofthe adaptive Turbo multiuser detectors discussed in Section VI.The channel code for each user is a rate that is one half ofthe constraint length-5 convolutional code (with generators 23,35 in octal notation). The interleaver of each user is indepen-dently and randomly generated and fixed for all simulations.The block size of the information bits is 128. (i.e., the code bitblock size is .) The code bits are BPSK modulated,i.e., . All users have the same signal-to-noise

(a)

(b)

Fig. 6. Bit error rate performance—convolutional code, impulsive noise. Allusers have the same amplitudes.� =� = 100 and� = 0:1.

ratio ( ). The symbol posterior probabilities are computedaccording to (26) with .

Fig. 5 illustrates the bit error rate performance of the adap-tive Turbo multiuser detector for User 1 and User 3. The codebit error rate at the output of the adaptive Bayesian multiuserdetector is plotted for the first three iterations. The curve cor-responding to the first iteration is the uncoded bit error rateat the output of the adaptive Bayesian multiuser detector. Theuncoded and coded bit error rate curves in a single-user addi-tive white Gaussian noise (AWGN) channel are also shown inthe same figure (as, respectively, are the dash-dotted and thedashed lines). It is seen that by incorporating the extrinsic in-formation provided by the channel decoder as the prior symbolprobabilities, the proposed adaptive Turbo multiuser detectorapproaches the single-user performance in an AWGN channelwithin a few iterations. The bit error rate performance of theadaptive Turbo multiuser detector in impulsive noise is illus-trated in Fig. 6, where the code bit error rates at the output ofthe adaptive Bayesian multiuser detector for the first three itera-


(a)

(b)

Fig. 7. Bit error rate performance—block code, Gaussian noise. All users havethe same amplitudes.

tions are shown. The uncoded and coded bit error rate curves in asingle-user additive white impulsive noise (AWIN) channel arealso shown in the same figure (as the dash-dotted and dashedlines, respectively), where the conventional matched-filter re-ceiver is employed for demodulation. Note that at high ,the performance of user 3 after the first iteration is actually betterthan the single-user performance. This is because the matched-filter receiver is not the optimal single-user receiver in impulsivenoise. Indeed, when , the maximum likelihood detectorfor signal model (1) is given by

D. Performance of the Code-Constrained Gibbs MultiuserDetectors

Examples 5 and 6:Finally, we consider the performance ofthe code-constrained Gibbs multiuser detectors discussed in

(a)

(b)

Fig. 8. Bit error rate performance—block code, impulsive noise. All usershave the same amplitudes.� =� = 100 and� = 0:1.

Section VII. We assume that each user employs the (7, 4) cyclicblock code with eight possible codewords [53]:

The bit error rate performance of the code-constrained Gibbsmultiuser detector in Gaussian noise is shown in Fig. 7. In thiscase, the Gibbs sampler draws a codeword fromat each time,according to (52). In the same figure, the unconstrained Gibbsmultiuser detector performance before and after decoding is alsoplotted. It is seen that by exploiting the code constraints in theGibbs sampler, significant performance gain is achieved. Theperformance of the code-constrained Gibbs multiuser detector


in impulsive noise is shown in Fig. 8, and similar performancegain over the unconstrained Gibbs multiuser detector is evident.

IX. CONCLUSIONS

In this paper, we have developed a new adaptive multiuserdetection scheme that is optimal in the sense that it is basedon the Bayesianinference of all unknown quantities. Suchan adaptive Bayesian multiuser detector can be efficientlyimplemented using the Gibbs sampler, which is a Markov chainMonte Carlo procedure for computing Bayesian estimates. Wehave derived the adaptive multiuser detection algorithms forboth the Gaussian noise and the impulsive noise synchronousCDMA channel. The proposed adaptive Bayesian multiuserdetectors can incorporate thea priori symbol probabilities, andthey produce as output thea posteriori symbol probabilities.That is, they are “soft-in soft-output” algorithms. Hence, theyare very well suited for iterative processing in a coded system,which allows the adaptive Bayesian multiuser detector to refineits processing based on the information from the decodingstage, and vice versa—a receiver structure termed asadaptiveTurbo multiuser detector.Furthermore, the channel decoderfacilitates a simple way of assessing the convergence of theadaptive multiuser detector by monitoring the number of bitcorrections made. Moreover, if the user data are encoded by ashort block code, then by exploiting the constraint on the validcode words in the Gibbs sampler, significant improvement onthe performance of the adaptive Bayesian multiuser detectorcan be obtained. Finally, we notice that the CDMA signalmodel (1) treated here is a rather simple one, namely, it isreal-valued and synchronous. Future extensions to this workinclude generalizations of the techniques proposed here toasynchronous CDMA systems and to systems with multipathfading effects.

APPENDIX A

Derivation of (19):

(54)

Derivation of (23):

(55)

Derivation of (25):

(56)

(57)

[ is -dimensional vector with all-zero entries except for theth entry, which is 1.]

APPENDIX B

Derivation of (40):


(58)

Derivation of (43):

(59)

Derivation of (45):

(60)

(61)

Derivation of (46):

(62)

(63)

Derivation of (47):

(64)

ACKNOWLEDGMENT

The authors would like to thank the anonymous reviewers andthe Associate Editor for their helpful comments.

REFERENCES

[1] B. Aazhang and H. V. Poor, “Peformance of DS/SSMA communica-tions in impulsive channels—Part I: Linear correlation receivers,”IEEETrans. Commun., vol. COMM-35, pp. 1179–1187, Nov. 1987.

[2] , “Peformance of DS/SSMA communications in impulsivechannels—Part II: Hard-limiting correlation receivers,”IEEE Trans.Commun., vol. COMM-36, pp. 88–96, Jan. 1988.

[3] , “An analysis of nonlinear direct-sequence correlators,”IEEETrans. Commun., vol. COMM-37, pp. 723–731, July 1989.


[4] S. F. Arnold, “Gibbs sampling,” inHandbook of Statistics, C. R. Rao,Ed. New York: Elsevier, 1993, vol. 9, pp. 599–625.

[5] J. O. Berger,Statistical Decision Theory and Bayesian Analysis, 2nded. New York: Springer-Verlag, 1985.

[6] C. Berrou and A. Glavieux, “Near optimum error-correcting coding anddecoding: Turbo codes,”IEEE Trans. Commun., vol. COMM-44, Oct.1996.

[7] C. Berrou, A. Glavieux, and P. Thitimajshima, “Near Shannon limiterror-correction coding and decoding: Turbo codes,” inProc. 1993 Int.Conf. Commun., 1993, pp. 1064–1070.

[8] K. L. Blackard, T. S. Rappaport, and C. W. Bostian, “Measurements andmodels of radio frequency impulsive noise for indoor wireless commu-nications,”IEEE J. Select. Areas Commun., vol. 11, pp. 991–1001, Sept.1993.

[9] T. K. Blankenship, D. M. Krizman, and T. S. Rappaport, “Measure-ments and simulation of radio frequency impulsive noise in hospitalsand clinics,” inProc. IEEE Veh. Technol. Conf., 1997, pp. 1942–1946.

[10] G. E. Box and G. C. Tiao,Bayesian Inference in Statistical Anal-ysis. Reading, MA: Addison-Wesley, 1973.

[11] P. L. Brockett, M. Hinich, and G. R. Wilson, “Nonlinear andnon-Gaussian ocean noise,”J. Acoust. Soc. Amer., vol. 82, pp.1286–1399, 1987.

[12] G. Casella and E. I. George, “Explaining the Gibbs sampler,”Amer. Stat.,vol. 46, pp. 167–174, 1992.

[13] K. S. Chan, “Asymptotic behavior of the Gibbs sampler,”J. Amer. Stat.Assoc., vol. 88, pp. 320–326, 1993.

[14] U. Fawer and B. Aazhang, “A multiuser receiver for code-divisionmultiple-access communications over multipath channels,”IEEE Trans.Commun., vol. COMM-43, no. 2/3/4, pp. 1556–1565, Feb./Mar./Apr.1995.

[15] A. E. Gelfand and A. F. W. Smith, “Sampling-based approaches to cal-culating marginal densities,”J. Amer. Stat. Assoc., vol. 85, pp. 398–409,1990.

[16] A. Gelman, J. B. Carlin, H. S. Stern, and D. B. Rubin,Bayesian DataAnalysis. London, U.K.: Chapman & Hall, 1995.

[17] S. Geman and D. Geman, “Stochastic relaxation, Gibbs distribution, andthe Bayesian restoration of images,”IEEE Trans. Pattern Anal. MachineIntell., vol. PAMI-6, pp. 721–741, Nov. 1984.

[18] T. R. Giallorenzi and S. G. Wilson, “Multiuser ML sequence estimatorfor convolutional coded asynchronous DS-CDMA systems,”IEEETrans. Commun., vol. COMM-44, pp. 997–1008, Aug. 1996.

[19] , “Suboptimum multiuser receivers for convolutionally coded asyn-chrnous DS-CDMA systems,”IEEE Trans. Commun., vol. COMM-44,pp. 1183–1196, Sept. 1996.

[20] P. J. Green, “Revisible jump Markov chain Monte Carlo computationand Bayesian model determination,”Biometrika, vol. 82, pp. 711–732,1985.

[21] J. Hagenauer, “The Turbo principle: Tutorial introduction and state ofthe art,” inProc. Int. Symp. Turbo Codes Related Topics, Brest, France,Sept. 1997, pp. 1–11.

[22] W. K. Hastings, “Monte Carlo sampling methods using Markov chainsand their applications,”Biometrika, vol. 57, pp. 97–109, 1970.

[23] R. A. Iltis, “An EKF-based joint estimator for interference, multipathand code delay in a DS spread-spectrum receiver,”IEEE Trans.Commun., vol. 42, no. 2/3/4, pp. 1288–1299, Feb./Mar./Apr. 1994.

[24] R. A. Iltis and L. Mailaender, “An adaptive multiuser detector with jointamplitude and delay estimation,”IEEE J. Select. Areas Commun., vol.12, pp. 774–785, June 1994.

[25] E. L. Lehmann and G. Casella,Theory of Point Estimation, 2nded. New York: Springer-Verlag, 1998.

[26] J. S. Liu, A. Kong, and W. H. Wong, “Covariance structure of the Gibbssampler with applications to the comparisons of estimators and augmen-tation schemes,”Biometrika, vol. 81, pp. 27–40, 1994.

[27] N. Metropolis, A. W. Rosenbluth, A. H. Teller, and E. Teller, “Equationsof state calculations by fast computing machines,”J. Chem. Phys., vol.21, pp. 1087–1091, 1953.

[28] D. Middleton, “Man-made noise in urban environments and transporta-tion systems: Models and measurement,”IEEE Trans. Commun., vol.COMM-21, pp. 1232–1241, Nov. 1973.

[29] , “Statistical-physical models of electromagnetic interference,”IEEE Trans. Electromagn. Compat., vol. EMC-19, pp. 106–127, 1977.

[30] , “Channel modeling and threshold signal processing in underwateracoustics: An analytical overview,”IEEE J. Ocean. Eng., vol. OE-12,pp. 4–28, 1987.

[31] , “Non-Gaussian noise models in signal processing for telecom-munications: New methods and results for class A and class B noisemodels,” IEEE Trans. Inform. Theory, vol. 45, pp. 1122–1129, May1999.

[32] D. Middleton and A. D. Spaulding, “Elements of weak signal detectionin non-Gaussian noise,” inAdv. Stat. Signal Process. Vol. 2: Signal De-tection, H. V. Poor and J. B. Thomas, Eds. Greenwich, CT: JAI, 1993.

[33] S. Y. Miller, “Detection and estimation in multiple-access channels,”Ph.D. thesis, Dept. Elect. Eng., Princeton Univ., Princeton, NJ, 1989.

[34] M. Moher, “An iterative multiuser decoder for near-capacity commu-nications,”IEEE Trans. Commun., vol. COMM-46, pp. 870–880, July1998.

[35] T. K. Moon, Z. Xie, C. K. Rushforth, and R. T. Short, “Parameter esti-mation in a multiuser communication system,”IEEE Trans. Commun.,vol. 42, pp. 2553–2560, Aug. 1994.

[36] H. V. Poor, “On parameter estimation in DS/CDMA formats,” inAdv.Commun. Signal Process., W. A. Porter and S. C. Kak, Eds. New York:Springer-Verlag, 1989.

[37] H. V. Poor and M. Tanda, “An analysis of some multiuser detectors inimpulsive noise,” inProc. 16th GRETSI Symp. Signal Image Process.,Grenoble, France, Sept. 1997.

[38] M. C. Reed, C. B. Schlegel, P. D. Alexander, and J. A. Asenstorfer, “It-erative multiuser detection for CDMA with FEC: Near single user per-formance,”IEEE Trans. Commun., vol. 46, pp. 1693–1699, Dec. 1998.

[39] C. P. Robert,The Bayesian Choice: A Decision-Theoretic Motiva-tion. New York: Springer-Verlag, 1994.

[40] T. J. Rothenberg, “The Bayesian approach and alternatives in economet-rics,” in Studies in Bayesian Econometrics and Statistics, Vol. 1, S. Fien-berg and A. Zellners, Eds. Amsterdam, The Netherlands: North-Hol-land:, 1977, pp. 55–75.

[41] M. J. Schervish and B. P. Carlin, “On the convergence of successivesubstitution sampling,”J. Comput., vol. 1, pp. 111–127, 1992.

[42] Y. Steinberg and H. V. Poor, “Sequential amplitude estimation inmultiuser communications,”IEEE Trans. Inform. Theory, vol. 40, pp.11–20, Jan. 1994.

[43] E. Strom and S. Miller, “Asynchronous DS-CDMA systems: Lowcomplexity near–far resistant receiver and parameter estimation,” Tech.Rep., Dept. Elect. Eng., Univ. Florida, Gainesville, Jan. 1994.

[44] M. A. Tanner, Tools for Statistics Inference. New York:Springer-Verlag, 1991.

[45] M. A. Tanner and W. H. Wong, “The calculation of posterior distributionby data augmentation (with discussion),”J. Amer. Statist. Assoc., vol. 82,pp. 528–550, 1987.

[46] S. Verdú, “Optimal multiuser signal detection,” Ph.D. dissertation, Dept.Elect. Comput. Eng., Univ. Illinois, Urbana, 1984.

[47] , “Minimum probability of error for asynchronous Gaussianmultiple-access channels,”IEEE Trans. Inform. Theory, vol. IT-32, pp.85–96, Jan. 1986.

[48] , “Computational complexity of optimum multiuser detection,”Al-gorithmica, vol. 4, pp. 303–312, 1989.

[49] , Multiuser Detection. Cambridge, U.K.: Cambridge Univ. Press,1998.

[50] J. von Neumann, “Various techniques used in connection with randomdigit,” Natl. Bur. Stand. Appl. Math. Ser., vol. 12, pp. 36–38, 1951.

[51] X. Wang and H. V. Poor, “Iterative (Turbo) soft interference cancellationand decoding for coded CDMA,”IEEE Trans. Commun., vol. 47, July1999.

[52] , “Robust multiuser detection in non-Gaussian channels,”IEEETrans. Signal Processing, vol. 47, pp. 289–305, Feb. 1999.

[53] S. B. Wicker,Error Control Systems for Digital Communication andStorage. Upper Saddle River, NJ: Prentice-Hall, 1995.

[54] W. H. Wong and F. Liang, “Dynamic importance weighting inMonte Carlo and optimization,”Proc. Nat. Acad. Sci., vol. 94, pp.14 220–14 224, 1997.

[55] Z. Xie, C. K. Rushforth, R. T. Short, and T. K. Moon, “Joint signal de-tection and parameter estimation in multiuser communications,”IEEETrans. Commun., vol. 41, pp. 1208–1216, Aug. 1993.

[56] S. M. Zabin and H. V. Poor, “Efficient estimation of the class A param-eters via the EM algorithm,”IEEE Trans. Inform. Theory, vol. 37, pp.60–72, Jan. 1991.


Xiaodong Wang (M’98) received the B.S. degreein electrical engineering and applied mathematics(with the highest honor) from Shanghai Jiao TongUniversity, Shanghai, China, in 1992, the M.S.degree in electrical and computer engineering formPurdue University, West Lafayette, IN, in 1995,and the Ph.D. degree in electrical engineering fromPrinceton Univesity, Princeton, NJ, in 1998.

In July 1998, he joined the Department of Elec-trical Engineering, Texas A&M University, CollegeStation, as an Assistant Professor. His research inter-

ests fall in the general areas of computing, signal processing, and communica-tions. He has worked in the areas of digital communications, digital signal pro-cessing, parallel and distributed computing, nanoelectronics, and quantum com-puting. His current research interests include multiuser communications theoryand advanced signal processing for wireless communications. He worked at theAT&T Bell Laboratories-Research, Red Bank, NJ, during the summer of 1997.

Dr. Wang is a member of the American Association for the Advancement ofScience. He received the NSF CAREER Award in 1999.

Rong Chenreceived the B.S. degree in mathematicsfrom Peking University, Beijing, China, in 1985and the M.S. and Ph.D. degrees in statistics fromCarnegie Mellon University, Pittsburgh, PA, in 1987and 1990, respectively.

Since September 1990, he has been with TexasA&M University, College Station, where he is cur-rently as Associate Professor of statistics. His currentresearch interests include nonlinear/nonparametrictime series and Monte Carlo methods for nonlinearnon-Gaussian dynamic systems.

Dr. Chen is a member of the American Statistical Association and the Instituteof Mathematical Statistics.

adaptive bayesian multiuser detection for synchronous cdma with gaussian and impulsive noise

Documents