Joint Detection and the AoA Estimation ofNoncoherent Signals in Multi-Element Antennas

Olesya BolkhovskayaMobile Communications Systems Dept.

Lobachevsky UniversityNizhny Novgorod, Russia

obol@rf.unn.ru

Alexander Maltsev Member, IEEE,Mobile Communications Systems Dept.

Lobachevsky UniversityIntel Corporation

Nizhny Novgorod, Russia

Victor SergeevMobile Communications Systems Dept.

Lobachevsky UniversityIntel Corporation

Nizhny Novgorod, Russia

Abstract—In this paper, we investigate two schemes of ajoint detection and signal wavefront estimation for noncoherentsignals. Both the developed schemes are based on the GLR(Generalized Likelihood Ratio) method. The first scheme doesnot exploit assumptions about the shape of signal wavefront. Thatleads to the known max lambda test, which uses maximum eigen-value of a sample correlation matrix of signals in multi-elementantenna as a decision statistics. The second scheme uses a prioriinformation about the plane shape of the useful signal wavefront.A three-stage procedure of angle-of-arrival (AoA) estimationfor signals with plane wavefront is proposed. This procedureallows to avoid ambiguities in the signal phases measurementon the antenna elements, and is applicable for antenna arrayswith arbitrary known geometry. The numerical simulations havebeen done for comparative analysis of the characteristics of thedeveloped joint detection-estimation algorithms. It was shownthat the second scheme provides 1dB-2dB depending on thenumber of antenna elements) in comparison with the max lambdatest.

Index Terms—GLR, signal wavefront, sample correlation ma-trix, angle-of-arrival, multi-element antenna arrays, AoA estima-tion

I. INTRODUCTION

Fast detection and accurate estimation of the signals AoAis important for optimal signal processing in multi-elementantenna arrays (AAs), which are used in modern radio com-munications and radar systems [1]- [2]. In the last decadethe interest for this problem is increasing again in connectionwith the development of cognitive radio approach for futurewireless communications systems [3]- [6]. Different detection-estimation schemes based on ED (energy detection) and GLRT(Generalized Likelihood Ratio Test) were investigated in liter-ature for purpose of the fast and reliable detecting an unknownsignal of a primary user in a cognitive radio networks byemploying multi-element antenna arrays. Theoretical analysisand simulations made in [7]- [8] have shown that the maximumeigenvalue detection test (max lambda test) is much betterthan simple ED. Most of the proposed schemes for detectingnoncoherent signals with an a priori unknown time structurein multi-element antennas are based on the analysis of eigen-values and eigenvectors of the sample correlation matrix [3],[4]. At the same time it is usually assumed that the wavefrontof the useful signal is unknown because it is determined bythe initially unknown communication channel.

In this paper we compare two approaches to the develop-ment of detection-estimation schemes for noncoherent signalsin multi-element antennas. Both of them are based on theGLR method, but in the first approach the signal wavefrontis considered as unknown, and in the second one it is as-sumed that wavefront has the form of a plane wave withunknown AoA. The first approach leads to the construction ofa detection scheme based on the calculation of the maximumeigenvalue of the sample correlation matrix. To implement thesecond approach, it is proposed to use additional discrete andcontinuous processing to estimate the signal plane wavefrontat the first stage, and then use this estimate for detection atthe second stage. The paper presents a comparative analysisof the detection characteristics of the proposed algorithms. Itis shown that the use of additional a priori information aboutthe wavefront shape of the signal leads to significant signal-to-noise ratio (SNR) gain.

II. DETECTION-ESTIMATION SCHEME FOR NONCOHERENTSIGNAL WITH ARBITRARY WAVEFRONT

Let’s consider the problem of detection and signal wave-front estimation in M-element antenna array on the internalGaussian noise background. This problem can be formulatedas follows [1]:

H0 : x[n] = ξ[n],H1 : x[n] = s[n] + ξ[n], n = 1, ..., N.

(1)

Here, H1 and H0 are the hypotheses of a useful signal in thereceived signal realization presence or absence respectively,x[n] = (x1[n], ...,xM [n])† is the vector of the observedsignals, s[n] is vector of the useful signals received on theantenna elements from the external source, ξ[n] - vector ofinternal noise, N is a sample size.

It is well known [9] that in the simple hypotheses case,when the n-dimensional probability density functions (PDF)of the observed signals x[n] are given, the optimal solutionof the signal detection problem is based on the calculation ofthe likelihood ratio (LR) and its comparison with a certainthreshold value:

Λ(x1, ...,xN ) =W (x1, ...,xN/H1)

W (x1, ...,xN/H0)

h1

>

<

h0

Λth, (2)

where W (x1[n], ...,xM [n]/H1), W (x1[n], ...,xM [n]/H0)are the likelihood functions LF1 and LF0 under hypothesesH1 and H0 respectively, calculated for the observed signalsxn = x[n], h1 and h0 are the accepted solutions for thesignal presence or absence, and the threshold value Λthdepends on the criterion used. In the case of complex H1

and H0 hypotheses testing, likelihood functions in (2) dependon some unknown a priori parameters, and for solving thedetection problem GLR method is usually applied (see, forexample, [10]- [11]), using GLRT decision statistics [9]. Inthe GLRT approach, maximum likelihood (ML) estimates ofthe unknown parameters of signals and noise are inserted inthe LR to form a GLRT decision statistics:

Λ(x[1], ...,x[N ]) =maxp1

W (x[1], ...,x[N ]/H1)

maxp0

W (x[1], ...,x[N ]/H0), (3)

where p0 and p1 are vectors of unknown parameters underthe hypothesis H0, and the hypothesis H1 respectively.

In this paper, we consider the application of the GLRmethod to solve the problem of joint detection and wavefrontestimation of the noncoherent useful signal when the signalvector s[n] and the internal noise vector ξ[n] are independentcomplex Gaussian multidimensional signals. In this case, weassume that the correlation matrix of the internal noise vectoris identity matrix. This assumption is equivalent to the knownpower of internal noise. In this case, the likelihood functionsfor the hypotheses H0 and H1 can be written as follows:

W (x1, ...,xN/H0) = π−MNe−Sp

N∑n=1

x[n]x†[n],

(4)

W (x1, ...,xN/H1) = π−MNdet(R)−Ne−x[n]†Rx[n], (5)

where R is the correlation matrix of the vector x[n] underthe hypothesis H1. The type of correlation matrix R dependson the time structure and the wavefront of the useful signals.It is convenient to write down LF1 and LF0 using a samplecorrelation matrix:

R =1

N

N∑n=1

x[n]x†[n], (6)

W (x1, ...,xN/H0) = π−MNe−SpR, (7)

W (x1, ...,xN/H1) = π−MNdet(R)−Ne−SpR(R−1R). (8)

Next, let’s consider the narrow-band signal case, when theuseful signals received by the antenna elements can be writtenas:

s[n] = A[n]s, (9)

where s is unknown vector describing the useful signal wave-front (amplitudes and phases of the narrow-band signal onthe elements of the AA), A[n] is a complex Gaussian signalwith zero mean 〈A[n]〉 = 0 and unknown power ν = 〈A2[n]〉.Without loss of generality it will be assumed |s|2 = s†s = M .

For these assumptions, the correlation matrix of the vector x[n]at the H1 hypothesis is written in the following form:

R = R(ν, s) =⟨x[n]†x[n]

⟩= I + νss† (10)

and depends on unknown parameters: scalar ν and vector s.It is convenient to use the known expression for an inversematrix of R

R−1 = I−ψ1ψ1† + (1 + Mν)−1ψ1ψ1

†, ψ1 =s√M, (11)

where the vector ψ1 is the eigenvector of the matrix Rcorresponding to the maximum eigenvalue λ1 = (1 + Mν).For the GLR approach it is necessary to find ML estimates ofν and s. Using (11), the LF1 (8) is written as follows:

W (x1, ...,xN/H1, ν,ψ1) =

= π−MN (1 +Mν)−Ne−N(SpR− Mν1+Mν Sp(ψ1ψ1

†R)).(12)

The LF1 (12) should be maximized by two parameters: aver-age signal power ν and the normalized vector ψ1 = s/

√M .

For finding the maximum of (12) for ψ1 it should be noted thatit monotonically depends on the Sp

(ψ1ψ1

†R)

. Therefore,the LF1 reaches its maximum value (regardless of the valueof ν) when expression (13) is maximal:

Sp(ψ1ψ1

†R)

= ψ1†Rψ1. (13)

Taken into account the normalization condition ψ1†ψ1 = 1, it

is easy to show that the maximum of the quadratic form (13)is equal to the maximum eigenvalue λ1 of the matrix R

maxψ

Sp(ψ1ψ

†1R)

= λ1 (14)

with vector ψ1 = ψ1, corresponding to the maximum eigen-value λ1 of the matrix R. Note that the ML estimate of theuseful signal wavefront will be equal to

s =√Mψ1 (15)

and does not depend on ν. Substituting ψ1 in (12), we obtain

W (x/H1, ν, ψ1) = π−MNe−N(SpR−λ1)(1 +Mν)−N

e−Nλ1

1−Mν .(16)

It is easy to see that LF1 (16) can be represented asCz−Ke−B/z (where z = 1 + Mν), and B, C, K are knownconstant parameters independent of ν and the maximum ofthis function is reached at z = B/K. Therefore ML estimateof the signal power can be easily found as

ν =λ1 − 1

M. (17)

As a result, the final maximum value of the LF1 (12) for thecase of a useful Gaussian signal with unknown power andunknown wavefront will be equal to

maxψ1,ν

W (x/H1, ν, ψ1) = π−MNe−N(SpR−λ1)λ−N1 e−N . (18)

The optimal decision rule on the presence or absence of auseful signal under the aforenamed assumptions is reduced to

Fig. 1. General scheme of joint detection and the AoA estimation of the useful noncoherent signal in multi-element antenna array.

finding the maximum value of the GLR by dividing (18) by(7) and comparing it with the threshold:

Λ(x) =(λ−1

1 eλ1

)−N h1

>

<

h0

Λth. (19)

Since the left part of the expression LF1 (19) is a monotoni-cally increasing function of λ1 (at λ1 ≥ 1), the decision basedon the likelihood ratio (19) with the threshold Λth is equivalentto the comparison of the λ1 with another threshold valueλth, which may be calculated by using the known analyticalexpression for the CDF (cumulative distribution function) ofthe value λ1:

Λ(x) =

h1

>

<

h0

λth. (20)

It will be the simplest decision statistics for the considereddetection task. Thus, to detect the noncoherent Gaussian signalwith unknown wavefront and power, the optimal decision ruleis to calculate the maximum eigenvalue λ1 of the samplecorrelation matrix R and compare it with a certain thresholdλth, depending on the selected criterion.

For example, for optimal detection by the Neymann-Pearsoncriterion, the threshold values of λth can be found theoreticallyin advance, using analytical expressions for the maximumnoise eigenvalue of the sample correlation matrix CDF [12]:

Fλ1(λ1) = |γ(N −M + i− j − 1, λ1N)

Γ(N −M + i)Γ(j)|, i, j = 1...M, (21)

where Γ(α) =∞∫0

e−ttα−1dt and γ(α) =x∫0

e−ttα−1dt are the

complete andincomplete gamma functions, N ≥M .

III. DETECTION-ESTIMATION PROCEDURE FORNONCOHERENT SIGNAL WITH PLANE WAVEFRONT

In the most important practical case, when the externalsource of the useful signal is in the far zone, its wavefrontcan be considered as a plane wave with the same amplitudeon the elements of the AA. Therefore, the problem of thewavefront estimating in this case is reduced to only the signalAoA evaluation. Mathematically, finding the ML estimate ofthe AoA leads to maximization of the LF1 (12) with theconstraint on the form of the vector ψ1 = s/

√M In this

case, the restriction on the wavefront vector s for the linearequidistant AA considered below is written as:

s = s(ϕ0,∆ϕ) =[ejϕ0 ejφ0+∆ϕ . . . ejϕ0+(M−1)∆ϕ

]†, (22)

where ϕ0 is the initial signal phase on the first element,∆ϕ = 2πsinθd/l is a phase difference of the useful signalsin the neighboring antenna elements, l is a wavelength, d is adistance between the elements, θ is the signal AoA relativelyto the antenna broadside. In this paper, the following three-stage procedure is proposed to find the maximum value ofLF1 (12) with the given constraint (22).

At the first stage (see Fig.1) the sample correlation matrixR (7) and the eigenvalue decomposition are calculated. Theeigenvector ψ1 = s/

√M corresponding to the maximum

eigenvalue λ1 gives the ML estimate of the signal wavefronts (17). In this case the λ1 itself is a sufficient statistics for thedetection problem (20).

At the second stage, the rough estimation of the usefulsignal AoA by the full scanning of the sector of possiblesignal angles-of-arrival is carried out by using discrete Fourier

Fig. 2. Mean square error (MSE) values σ sinθ of the AoA sines estimates

transform (DFT). In practical tasks, to increase the accuracy ofthe estimation, instead of DFT, spatial discrete scanning witha smaller angular steps (”oversampling”) is used. Formally,such a discrete estimate of the plane wavefront of the signalis found from the following equation:

sD = arg maxs∈DFT

Re(s†ψ1). (23)

The discrete estimate sD, obtained at this stage, can also beused to solve the detection problem, taking into account apriori information about the plane wavefront of the signal.In this case, the maximum value of the LF1 is found bysubstituting the vector ψ1 = s/

√M in (12) and then varying

the parameter ν. It is easy to show (similarly (17)) that theexpression for the LF1 reaches its maximum value when

νD =Sp(sD s

†DR)− 1

M. (24)

In this case, the decision rule of the signal presence (19)with the discrete wavefront vector estimate sD will takes thefollowing form:

Λ(x) =(Sp(sD s

†DR)−1eSp(sD s†DR)

)Ne−N

h1

>

<

h0

Λth. (25)

By analogy (19) it is equivalent to the comparison with anotherthreshold of the following decision statistics:

PD(x) = Sp(sD s†DR) =

1

N

N∑n=1

|sDx[n]|2h1

>

<

h0

PDth. (26)

The obtained decision statistics PD(x) has a physical meaningof the measured signal power coming from the directiondetermined by the beamsteering vector-phasor sD.

At the third stage, the rough discrete estimate of thewavefront vector sD , obtained at the second stage, is usedto exclude ambiguity in the phases of the plane wavefront ofthe useful signal, which may occur in low SNR region. To dothis, the vector δψ1 is calculated by the formula:

δψ1 = [a1eϕ1 , . . . aMe

ϕM ] = c · diag(ψ†1sD

)· ψ1, (27)

where the scalar coefficient c =(ψ†1sD)|(ψ†1sD)| , |c| = 1 compen-

sates the average phase progression of the vector δψ1 compo-nents. It can be seen that this operation (27) leads to the factthat phases of the vector δψ1 represent the phase differences ofcomponents of vectors ψ1 and sD, and amplitudes of δψ1 areequal to amplitudes of vector ψ1 components. Refinement ofthe useful signal AoA is carried out by continuous smoothingof the vector δψ1 phases using the trial vector s = s(ϕ0,∆ϕ)in form (22), which is found by maximizing the scalar product

Re(ψ†1sD

)−→S

max . (28)

Fig. 3. Probabilities of misdetection for different detection-estimation algo-tithms

It can be shown that the solution of the problem (28) for smallphase differences of the vector components δψ1 and sD isreduced to the problem of minimization of the quadratic form

M∑m=1

am(ϕm − ϕm)2−→min, ϕm = ϕ0︸︷︷︸α

+ ∆ϕ︸︷︷︸β

(m− 1). (29)

The solution of (29) is found analytically by using a linearregression model with coefficients α and β [13], [14].

A refined continuous estimate of the vector wavefront ofthe useful signal obtained at this stage

sDC = s = s(ϕ0,∆ϕ), ϕ0 = α, ∆ϕ = β (30)

can also be used to solve the problem of a useful signaldetection with a plane wavefront (see Fig.1). Substituting (30)in the expression (26) we obtain the decision statistics in theform:

PDC(x) =1

N

N∑n=1

|sDCx[n]|2h1

>

<

h0

PDCth. (31)

The resulting decision statistics PDC(x) has a physical mean-ing of the measured signal power coming from the directiondetermined by the beam steering vector-phasor sDC . It can beshown that the described detection-estimation procedure maybe straightforwardly generalized for multi-element antennaarrays with arbitrary known geometry by introducing in (22)vector-phasors corresponding to this geometry. The accuracyof the obtained estimates of the useful signal AoAs can beinvestigated by comparison with the Cramer-Rao lower bound(CRB). The CRB of the standard deviation of the AoA sineestimate was determined by the formula [15]:

σ2sin θ ≥

1

(2π · d/λ)2· 6

M2(M2 − 1)· 1 +M · SNR

N · SNR2, (32)

SNR = ν is the signal-to-noise ratio on one antenna element.

IV. SIMULATION RESULTS

A comparative analysis of the characteristics of the con-sidered detection-estimation algorithms was carried out for alinear equidistant M-element AA for the AWGN channel. Arandom complex Gaussian sequence of independent sampleswith length N=32 was used as a useful signal amplitude A[n].The following estimates of the plane wavefront vector wereanalysed for the useful signal AoA estimation:

1. The optimal wavefront vector estimate found from afinite set of DFT vectors by maximizing the scalar product(23): sD = arg max

s∈DFTRe(s†ψ1). To compare the accuracy,

these discrete estimates in two cases were considered: withoutoversampling (the dimension of the DFT is equal to M) andwith double oversampling.

2. The optimal wavefront vector estimate found by usingthe proposed three-stage procedure, when the discrete estimatesD was continuously smoothed by linear regression procedureapplied to the vector δψ1 (28),(29): sDC = s(ϕ0,∆ϕ) =

arg maxs∈DFT

Re(s†δψ1). For the considered estimation algo-rithms the mean square error (MSE) values of the AoA sinesσsin θ were calculated.

Fig.2 shows the graphs of σsin θ depending on the SNR forantenna arrays with different number of elements M=8,16,32.For comparison, the limiting accuracies of the AoA estimates,defined by the CRB (32), are also shown. From the presentedresults it can be seen, that the MSEs for discrete estimationmethods tend to the non-zero constants due to the limitedbeamwidth of the main lobe of the AA radiation patternwhich depend on the number of antenna elements. The MSEsof the continuous estimates decrease with increasing SNRand tend to the CRB. For the considered parameters of thescheme, continuous AoA estimates achieve the CRB at thevalues SNR = −4dB,−6dB,−8dB for the number of theantenna array elements M = 8, 16, 32, respectively. At theseSNRs values, the MSE values σsin θ are respectively equal to:σsin θ = 10−2, 5 · 10−3, 2 · 10−3.

In Fig. 3 the detection characteristics for different schemesare given for the considered decision statistics (20), (26), (31).The threshold values of these statistics were determined bythe Neymann-Pearson criterion for the false alarm probabilityPfa = 0.01. For comparison, Fig.3 also shows the probabilityof misdetection curves for the detection scheme with a prioriknown angle-of-arrival of the useful signal. From presentedresults it can be seen, that usage of a priori knowledgeabout the useful signal plane wavefront in detection-estimationalgorithms (26) and (31) provides SNR gain, in comparisonwith max-λ test, by values of the order +1.0dB,+1.5dBand +2.0dB, for antenna arrays with the number of elementsM=8,16, and 32 respectively. It is interesting to note thatthe scheme with the discrete plane wavefront estimation withdouble oversampling provides almost the same levels of theprobabilities of misdetection as the scheme with continuouswavefront vector-phasor estimates.

V. SUMMARY

In this paper we considered the problem of joint detectionand wavefront estimation of noncoherent signal in multi-element antenna arrays. Based on the GLR method threealgorithms for signal detection and wavefront estimation havebeen developed. A comparative analysis of the characteristicsof these algorithms on the example of a linear equidistant M-element antenna array has been carried out. It is shown thatthe accuracy of the continuous signal AoA estimates tends tothe CRB with increasing SNR.

For antenna arrays with different number of elements thevalues of SNR have been found, when the mean squareerrors of the AoA estimates practically reach the Cramer-Raolower bounds. The performance characteristics of the detectionschemes, using different wavefront estimation algorithms, havebeen evaluated. It was shown that the usage of a priopi knowl-edge about useful signal plane wavefront provides additionalSNR gain in the detection characteristics of the proposedscheme in comparison with max-λ test. For multi-element

antenna arrays with the number of elements M = 8, 16, 32 thisSNR gain achieves +1.0dB,+1.5dB,+2.0dB, respectively.

REFERENCES

[1] Richards M.A. Fundamentals of Radar Signal Processing // McGraw-HillEducation, 2014.

[2] Sand S., Dammann A., Mensing C., Positioning in wireless communica-tions systems. // John Wiley&Sons, 2014.

[3] Wang, Pu, et al. Multiantenna-Assisted Spectrum Sensing for CognitiveRadio. Vehicular Technology, IEEE Transactions Vol. 59, N 4 (2010).

[4] R. Zhang, T. J. Lim,Y. C. Liang, Y. Zeng Multi-Antenna Based SpectrumSensing for Cognitive Radios: A GLRT Approach, IEEE Transactions onCommunications, Vol. 58, N 1 (2010).

[5] L. Jing, E. D. Carvalho, P. Popovski, and A. O. Martınez, Design andperformance analysis of noncoherent detection systems with massivereceiver arrays, IEEE Trans. Signal Process., vol. 64, no. 19, 2016.

[6] Anil Kumar, Rajender Kumar Simulation analysis of GLRT and eigen-value based methods for congnitive radio 2nd IEEE International Con-ference on Recent Trends in Electronics, Information & CommunicationTechnology (RTEICT), 2017.

[7] Y. Zeng, C. L. Koh and Y.-C. Liang Maximum Eigenvalue Detection:Theory and Application, 2008 IEEE International Conference on Com-munications.

[8] Asmaa Maali, Hayat Semlali, Najib Boumaaz,Abdallah Soulma Energydetection versus Maximum Eigenvalue based detection: A comparativestudy, 2017 14th International Multi-Conference on Systems, Signals &Devices (SSD).

[9] Harry L. Van Trees. Detection, estimation and modulation theory. P. 1.John Wiley&Sons, 2001.

[10] Bolkhovskaya O.V., Maltsev A.A. The performance of the GLRT for thespatial signals detection with a small number of observations. // IEEESignal Processing Letters, Oct. 2004.

[11] Bolkhovskaya O.V., Maltsev A.A. Decision Statistics for IncoherentSignal Detection in Multi-Element Antenna Arrays // Radiophysics andQuantum Electronics, 61(2), 2018.

[12] Ermolaev, V.T., Rodyushkin, K.V. The distribution function of themaximum eigenvalue of a sample correlation matrix of internal noiseof antenna-array elements Radiophys Quantum Electron (1999) 42: 439.https://doi.org/10.1007/BF02677625

[13] J. Johnston, Econometric Methods, 3rd Edition, McGraw Hill, NewYork, 1984.

[14] Bolkhovskaya O.V., Maltsev A.A The Wavefront Estimation and SignalDetection in MultiElement Antenna Arrays at Low SNR 2nd EuropeanConference on Electrical Engineering and Computer Science, Bern 2018.

[15] Ermolaev, V.T., Flaksman, A.G. Signal processing methods in adaptiveantenna arrays and noise compensators (in Russian) N.Novgorod, 2015.