signal reconstruction from multiple correlations: frequency- and time-domain approaches

682 J. Opt. Soc. Am. A/Vol. 6, No. 5/May 1989

Signal reconstruction from multiple correlations:frequency- and time-domain approaches

Georgios B. Giannakis

Department of Electrical Engineering, University of Virginia, Charlottesville, Virginia 22901

Received June 30, 1988; accepted January 16, 1989

One-dimensional (1-D) ultrashort laser signals cannot be recorded directly, although it is possible to detect theirmultiple correlations. The reconstruction of 1-D deterministic sampled signals from their multiple correlations isstudied. A computationally efficient, fast-Fourier-transform-based, frequency-domain algorithm is described forsimultaneously reconstructing the amplitude and the phase of a finite-duration signal. It is shown that, bymodeling the Fourier transform of a discrete sequence as a pole-zero rational function, unique (modulo time shifts)signal recovery is possible from any multiple correlation of order greater than 2. The resulting time-domainalgorithm uses all the nonredundant 1-D slices of a multiple-correlation sequence and applies to one- or two-sided,finite- or infinite-duration signals. The signal parameters are obtained in closed form by using a set of linearequations. Noise effects are studied theoretically and experimentally through simulated data. Both frequency-and time-domain algorithms are applicable to modeling and interpolation of raster-scanned images.

1. INTRODUCTION

Over the past decade there has been a growing interest inmultiple (especially triple) correlations for time-series anal-ysis, optics, and statistical signal processing. In time se-ries,1-3 multiple correlations, called cumulants, and theirFourier transforms, known as polyspectra (or higher-orderspectra), are used to analyze non-Gaussian and/or nonlinearrandom processes.

Signal reconstruction of deterministic signals by usingtriple correlations or their Fourier counterparts, called bi-spectra, has been studied mostly in the optics literature (fora tutorial treatment see Ref. 4). Applications include mea-surement of ultrashort laser pulses,5 analysis of nonlinearinteractions between ocean wave components, 6 acoustic im-aging,7 intensity interferometry,8 and astrophotography. 9-1"In astrophotography, the speckle masking method uses tri-ple autocorrelations or triple cross correlations, computedand averaged over many interferograms, to restore the mag-nitude and the phase of images distorted by photon noise, byatmospheric turbulence, and by telescope aberrations.Speckle masking has been applied successfully to real astro-nomical data, but it is limited to finite-duration sequencesand is less effective when the signals involved are bandlimited.

In digital signal processing, polyspectral analysis has beenadopted in various problems,12-6 such as phase reconstruc-tion, detection of phase coupling, retrieval of harmonics innoise, order determination, and identification of noncausaland nonminimum phase systems that are excited by an un-observable input signal that is assumed to be independentand non-Gaussian distributed. A thorough review of cumu-lants and their applications in statistical signal processingcan be found in Ref. 17.

In cumulant-based signal processing the signals are dis-crete random sequences, and the systems are described gen-erally by pole-zero transfer-function models. In Section 2discrete deterministic sequences are represented by linear

constant-coefficient difference equations.'8 The main ob-jective of this paper is to show, by using this representation,how the model-based bispectral identification approaches ofstochastic systems14"16 can be modified to reconstructuniquely (modulo time shifts) deterministic, although per-haps noisy, signals.

In the approach described in this paper an attempt ismade to narrow the gap between the optics viewpoint (usual-ly continuous and deterministic) and the statistical signal-processing viewpoint (usually discrete and stochastic).Along the same lines a stochastic solution'2 was proposed inRef. 19, after the triple correlation of a deterministic tran-sient signal was equated with the triple correlation of arandom signal-plus-noise sequence. This approach is validonly when many data records with independent noise com-ponents are available.

In Section 3 one-dimensional (1-D) frequency- and time-domain algorithms are developed that are appropriate forreconstructing even from a single data record. Potentialapplications include retrieval of ultrashort laser pulses5 andmodeling 20 of 1-D image data obtained from the output of araster scanner. Additionally, the 1-D algorithms of thispaper are useful in reducing computational complexity,when two-dimensional (2-D) image reconstruction is per-formed with triple correlations of their 1-D projections.2 'Our approaches are similar to the frequency-domain ap-proach of Ref. 22 because they reduce the noise effects byaveraging over redundant information present in the multi-ple-correlation domain. Moreover, the time-domain ap-proach applies to finite-impulse-response (FIR; see Subsec-tion 3.A) discrete signals and to one- or two-sided infinite-impulse-response (IIR; see Subsection 3.B) discrete signals.The 1-D two-sided signal models obtained in Subsection 3.Bare useful for image coding and interpolation. 2 0 23 In Ap-pendix D comments are made on the extension of the 1-Dfrequency-domain algorithm to 2-D signals.

In Section 4 the problem of model selection is addressed;extensions of the method are discussed that would be appro-

0740-3232/89/050682-16$02.00 © 1989 Optical Society of America


Vol. 6, No. 5/May 1989/J. Opt. Soc. Am. A 683

priate for signal reconstruction from one or more noisy datarecords. Although in Section 5 the algorithms are testedwith computer-simulated data, implementation of the mul-tiple-correlation-based algorithms with acousto-optic mod-ulators is possible along the lines described in Ref. 24.

2. MODELING: PRELIMINARIES

The discrete signal s(k)j that we want to reconstruct isrepresented by the following linear, constant-coefficient dif-ference equation:

P2 q2

(i)s(k - i) = E b(i)(k - i), (la)i=-Pl i=-ql

where (k) stands for the Dirac delta function.Equation (la) represents a large class of infinite-duration,

one- or two-sided sequences s(k). This class includes thesampled version of continuous, finite-duration interfero-grams 9-1 1 22 if P1 = P2 = 0 and a(0) = 1. In the noisy case wehave

x(k) = s(k) + v(k). (lb)

The following assumptions are made about the determinis-tic signal, s(k), and the random noise, v(k):

(2) The additive noise v(k) is stationary, zero-mean, in-dependent, and identically distributed (i.i.d.) with a vari-ance o,2 and a third-order moment 'Y3v = EIv3(k)} < -.

The goal of this study is to estimate the signal parameters{a(i)jj=iP and b(j)jj=oq and the orders PI, P2, and q on thebases of multiple correlations. Specifically, we shall usetriple correlations, which are defined as

s3(m, n) E s(i)s(i + m)s(i + n).,= X

(3)

Equation (3) imposes the following symmetry conditions fortriple correlations:

s3(m, n) = s3(n, m) = s3(-m, n - m) = s3(n - m, -in)

= s3(m - n, -n) = S3(-n, m - n). (4)

Symmetry restricts the nonredundant samples of the 2-Dtriple-correlation sequence, s3(m, n), to the wedge definedby 0 S n S m < . Furthermore, for finite-duration signalsof length q + 1, the triple correlation has finite support, andthe nonredundant sA(m, n) samples are contained in thetriangle T(q) defined by 0 S n S m q (see also Fig. 1).

The Z transform of s3(m, n) is given by

(1) The Z transform S(z) of s(k) exists and is given by

S(z) s(k)z-i = A(z)AWz

q2 ql+q2Z 6(i)zi (i-ql)zi_=_ql _qjp i=O (2a)

P2 P+P2

a(i)z a(i-pz-ii=-pl i=O

Furthermore, S(z) is assumed to be free of pole-zero cancel-lations and to be stable. The region of convergence of S(z) istherefore Pmaxin < Iz < minoUt where maxin (Pminout) is thepole with maximum (minimum) amplitude inside (outside)the unit circle. Moreover, stability requires that A(z)llzl=1 wd0.

With multiple correlations, it is possible to recover s(k)[and hence S(z)] within a time-shift ambiguity. 22 Thus wecan drop the term Zql-Pl from Eq. (2a) and assume with noloss of generality that

Sz =B(z) AWz

q

I b(i)z`i=O

1 + Y a(i)z-i=1

S3(Z1, Z2) = I s3(m, n)ZljmZ2 nm,n=--

= S(z1)S(z 2)S(zj 1_z 2 _1 ) = B3(l, ) (5a)A3(zl, Z2 )

where

q

B3 (zl, Z2) = B(z 1 )B(z 2)B(z- 1 z 21 ) = > b3 (i, j)zg'z 21

ij-q(5b)

and

(2b)

where

P- P1 +P2 -qq + q2

and

b(i) b(i - q), a(i) - a(i - P1 ).

.edge O n m

(2c) Fig. 1. The nonredundant region of support, T(p), of a3(i, j) is thetriangular region with vertices at (0, 0), (p, 0), and (p, p). Thenonredundant region of support, T(q), of 3 (i, j) is obtained byreplacing p with q. The wedge 0 < n < m is the nonredundant

(2d) region of support of an infinite-duration signal.


m


pA 3 (Z1 , Z2) A(z1)A(z2)A(z-1z2') = E a3(i, j)z1iz 2 -.

ij=-p(5c)

Notice that the bispectrum, S3[exp(jwl), exp(jW2)], is ob-tained from S 3 (zl, Z 2 ) if we use the substitutions z1 =

exp(jwl) and Z2 = exp(j 2).The b(i, ) and a3(i, P) terms are related to the signal

parameters by

q

b3(ij) = b(k)b(k + i)b(k + ) (5d)k=O

JB 3[exp(Aw), exp(jwl)JI = B[exp(ji)I IIB[exp(jo)]I

X IBexp[-](Wh + 11)]1 (6b)

'k3(Wk, W) = k(W) + k(Wj) - P(Wh + c&d), (6c)

where B3 [exp(jWk), exp(jwl)]I and O3(Ak, wol) denote the am-plitude and the phase of B3[exp(jwk, exp(jic)], respectively.

A fast modification of this frequency-domain algorithmresults if we choose Wk = 2rk/(q + 1) and reconstructthe signal's discrete Fourier transform (DFT) B(k) -Zq=O b(n)exp[-j27rnk/(q + 1)], using its bispectrum B3 (k, I).The bispectrum can be computed efficiently if the fast Fou-rier transform (FFT) is used in the DFT counterpart of Eq.(6a):

andp

a3(i, j) = a(k)a(k + i)a(k + i). (5e)k=O

Comparing Eqs. (5d) and (5e) with Eq. (3), we observe thatb3 (i, i)[aA(i, I)] is, in fact, the triple correlation of the dis-crete, finite-duration signal b(k) [a(k)] of length q + l(p + 1).In Fig. 1 the triangles T(q) and T(p) denote the nonredun-dant regions of support of b3(i, j) and a3(i, j), respectively.Notice that the infinite-duration triple-correlation sequencein Eq. (3) is characterized completely by the finite-durationtriple-correlation sequences in Eqs. (5d) and (5e).

B3(k, ) = B(k)B(I)B*(k + ). (7a)

In Eq. (7a) we used the fact that b(i) is real and thereforeB(-k - ) = B*(k + 1), where * denotes complex conjugation.

With the substitutions (k - k, 1 - 0) and (k - k - 1, 11), Eq. (7a) yields

B 3(k, 0) = B(k)B(O)B*(k) (7b)

and

B3(k - 1, 1) = B(k - )B()B*(k), (7c)

respectively. EliminatingB*(k) from Eqs. (7b) and (7c) andsolving with respect to B(k), we obtain

3. SIGNAL-RECONSTRUCTION ALGORITHMS

In this section it is proved theoretically that la(i)Ji=P and_b(i)}g=1q can be determined uniquely by using exact triple-

correlation samples s3(m, n), when the orders p and q areknown. In Subsection 3.A the existing frequency-domainapproaches for finite-extent sequences are reviewed andmodified. Also, a time-domain approach is developed, as isa practical algorithm to estimate the signal parameters wheninexact 93(m, n) samples are available. In Subsection 3.Bthe case of infinite-duration signals is examined. Orderdetermination and explicit inclusion of the noise in the triplecorrelation xA(m, n) of the noisy signal x(k) are discussed inSection 4.

A. Finite-Duration SignalsSignals of finite duration q + 1 are represented by Eq. (la),provided that P1 = P2 = p = 0 and a(0) = 1. In the FIR casewe have that s(i) = b(i), s3(i, j) = b3(i, j), and S3(z1, z2) =

B3(z1 , Z2). In Ref. 22 it was shown that the zeros of B(z) canbe determined uniquely from B3 (z1, Z2) by using the analyticcontinuation of bispectra. Furthermore, a practical fre-quency-domain reconstruction algorithm can be devel-oped22 if we rewrite Eq. (5b) for discrete frequencies Zk

exp(UW, k = 0,1, . ., q:

B3 [exp(jiW), exp(wl)J = B[exp(jWk)]B[exp(wl)I

X Blexp[-j(w, + WI)]). (6a)

When amplitudes and phases are equated on both sides ofEq. (6a), the amplitude IB[exp(Lk)I and the phase 4(Wk) ofB [exp(jWk)] can be recovered by using the recursive solutionof the equations 2 2

B3 (k, 0)B(O)B3(k - 1, 1) ( (7d)

Equation (7d) is a recursive form for computing B(k) fromits bispectrum B3(k, 1). To initialize it, we compute B(0) bysetting k = 1 = 0 in Eq. (7a):

B(0) = [B3(0, 0)1"3. (7e)

By substituting k = 1, = 0 into Eq. (7a) we find the ampli-tude of B(1):

IB(1)I = (B 3 (iO0)) (7f)

The phase of B(1) is initially set to zero. With (1) = 0 [i.e.,B(1) = B(1)1], we can fix = 1 and iterate Eq. (7d) to find theestimates A=2 (q+l)/2. The remaining Fourier coeffi-cients can be found by conjugation because {f(q + 1 - k) =

B3*(k)lk= I(q+2)/2.

In Ref. 22 the amplitude and the phase were reconstructedseparately by means of Eqs. (6b) and (6c), respectively. Theproposed frequency-domain modification reconstructs theDFT of the signal by using a single equation, namely, Eq.(7d).

Notice that the Bl(k) values can be computed from Eq.(7d) in more than one way; e.g., f(4) can be computed byusing Eq. (7d) with k = 4, = 1 or with k = 4,1 = 2. Thisredundancy can be used in averaging the B(k) estimatesobtained for different 's in Eq. (7d) to obtain the closedform:

fl(k) = 1 -' B3(k, 0) fl(1yP(k - 1),[k/2] o k B(0)B3(k - 1 1)

k = 2, 3,.. ,(q + 1)/2. (7g)



In Eq. (7g), [k/2] denotes the integer part of k/2 and repre-sents the number of P(k) estimates averaged for each k.Because the Fourier coefficients fP(k)1k=2 l+ )1 2, computedfrom Eq. (7d) with I = 1 or from Eq. (7g) with I = 1, . . , [/2], are obtained with 0(1) = 0, their phase must be corrected.By solving Eq. (6c) iteratively, we can show22 that p(k) in-cludes the term ko(1). Therefore fA(k) found with f(1) = 0must be multiplied by exp~jk0(1)] for the true B(k) to beobtained.

To determine /(1) we recall that b(i) is real, which impliesthat

0[(q + 1)/2] = 0[(q + 1)/2] + (q + 1)/2 0(1) = nr,

n = 0,1. (7h)

In Eq. (7h), [(q + 1)/2] denotes the phase of A[(q + 1)/2]found in Eq. (7g) under the assumption that M(1) = 0. Solv-ing Eq. (7h) for 0(1), we obtain

2 2nOM (1 - [(q + 1)/2 + + T 1.q +1 q+ 1

Because we cannot determine whether n = 0 or n = 1, weassume that n = 0, correct A(k) by the factor expi-j2/(q +1)7[(q + 1)/2]k}, and take the inverse FFT of the resultingsequence to obtain the desired time signal. If n = 1, thereconstructed signal will be a circularly time-shifted versionof the original sequence because we have omitted a factorexpfj27rk/(q + 1)] from its DFT. This result justifies thefact that unique (modulo time shifts) signal reconstructionis possible from bispectra. Several remarks are now in or-der:

(1) The averaging in the estimation of A(k) from Eq. (7g)reduces the noise effects in the reconstruction when inexacttriple correlations are available. The same idea was used inthe phase-reconstruction algorithm of Ref. 22 and can beextended to higher-than-third-order correlations. Thefourth-order case is briefly described in Appendix A.

(2) In deriving Eq. (7d) we tacitly assumed that B(0) EX=o b(k) < 0 and that B3(k - 1,1) < 0. If B(O) = O then wedrop b(q), reconstruct the sequence b(i)i=0q-1, and computeb(q) as b(q) =- k-0 b(k). If B3 (k - 1, 1) = 0 for some (k, 1),then we can either omit it from Eq. (7g) or oversample thebispectrum, as was suggested in Ref. 22.

(3) The bispectral slice = 0 was not used in Eqs. (7d)and (7g) because it corresponds to an identity.

(4) Zero padding of the signal is necessary for FFT com-putations in which q + 1 is not a power of 2 and for identify-ing the original sequence from circularly shifted versions ofit.

(5) Both the frequency-domain method of Ref. 22 and itsmodification proposed in this subsection are inappropriatefor infinite-duration signals unless the signals are periodic.For the case of a periodic signal, it is well known18 that thediscrete Fourier series over a period coincides with the DFTof one period of the signal. Therefore the previously devel-oped algorithm can be used for a periodic signal, too.

(6) If a finite-extent signal is effectively band limited,high-frequency noise components will decrease the perfor-mance of the recursive frequency-domain reconstructionmethods.

(7) The 1-D frequency-domain reconstruction algorithmcan be extended to multidimensional signals. In Appendix

D we shall discuss the 2-D extension, which is potentiallyuseful for image reconstruction and enhancement. But, aswas mentioned in Ref. 21, the triple correlation of an imageis a four-dimensional function, and in order to decrease thecomputational burden it is better to reconstruct its 1-Dprojections. In this case the 1-D reconstruction algorithmsdescribed in this paper are directly applicable. Finally,from the reconstructed projections the 2-D image can beobtained by using the inverse Radon transform.2 '

In what follows, a time-domain reconstruction method isdeveloped for finite-duration signals that is also useful forreconstructing infinite-extent sequences. For notationalconvenience we shall restrict ourselves to triple correlations,although extensions to higher-than-third-order correlationsare straightforward. In Appendix B the fourth-order case isdiscussed briefly.

From Eq. (5d) we obtain b3(q, j) = b(0)b(q)b(j); hence

b(j) = bO) b3(q j)b3(q, 0)

j = 1, 2, . .. ,q. (8a)

Substituting Eq. (8a) into Eq. (5d), we find that

b3(m, n) = b3(O) b3(q, i)b3(q, i + m)b 3(q, i + n),b 3 (q, 0)i=

(m, n) T(q). (8b)

Next, we solve Eq. (8b) for b(0) and substitute the result intoEq. (8a) to obtain

b(i) = b3"13(m, n)b3(q, j)b)= q-m _1v3

, b3(q, i)b3(q, i + m)b3(q, i + n)i=o_

j=0,,... ,q. (9)

For m = q, n = 0, Eq. (9) simplifies to

b(j) =b3(q, j)

(10)[b3(q, O)b3(q, q)] 1/3

Notice that the denominator in Eq. (10) is nonzero, be-cause b3 (q, 0) = b2(0)b(q) # 0 and b3(q, q) = b(O)b2(q) 0.Moreover, Eq. (10) shows that a finite-duration discretesignal can be determined uniquely in closed form by using anappropriate 1-D slice of its 2-D triple correlation. Thissimple but interesting result, which establishes a one-to-onecorrespondence between finite-extent sequences and theirtriple correlations, was described graphically in Ref. 4 andformulated in a stochastic framework [as in Eq. (8a) withb(0) = 1 in Ref. 14 (see also Ref. 25). Despite its theoreticalvalue, Eq. (10) requires knowledge of the exact triple-corre-lation slice b,(q, j)}j=0q. Because only one 1-D slice is usedin Eq. (10), the reconstructed signal will show high variancein the noisy case16 unless many data records are averaged tocompute reliable triple-correlation estimates (as is done inspeckle masking 0).

To improve the accuracy of the reconstructed signal in thenoisy case, we use redundant information contained in all 1-D triple-correlation slices of the T(q) triangle shown in Fig.1. Averaging over the estimates obtained by Eq. (9) for alltriple-correlation lags (m, n) e T(q), we obtain



631£3(m, n)63(q, j)

£3(q, i)63(q, i + Mi)6 3(q, i + n)

j =0,1, .. . ,q. (11)If one or more of the 6

3(q, i + m) are zero, to avoid divisionby zero in Eq. (11) the corresponding triple-correlation slicesmust be omitted, as is done in the frequency-domain ap-proach for the bispectral slices. Note that in Eq. (11) onecan average over a limited number of (m, n) lags.

An alternative approach is to find initial estimates of thesignal parameters by means of Eq. (10) and to solve itera-tively the nonlinear optimization problem of minimizing thequadratic objective function

q mZ > [b3(m, n) - 3(m, n)]2, (12)m=O n=O

where b3(m, n) is given by Eq. (5d) and 63 (m, n) denotes theinexact triple-correlation samples. The same computation-ally complex procedure was suggested in a stochastic frame-work in Ref. 3, but convergence to the global optimum is notalways achieved.

Additional iterative algorithms that use the redundancypresent in triple correlations can be used to update theinitial signal estimates obtained from the closed-form solu-tion [Eq. (10)]. One of them is described in Appendix C. Ingeneral, there is no iterative algorithm that is uniformlysuperior to any other, because the relative performance of analgorithm depends on the noise distribution, which is un-known. In some cases, using the final estimate of one algo-rithm as the initial estimate for another improves the recon-struction result.

B. Infinite-Duration SignalsTo reconstruct an infinite-duration signal s(k) by using itstriple correlation s3 (m, n), it suffices to obtain the triplecorrelations b3 (i, j) and a3 (i, j) of its parameters given in Eqs.(5d) and (5e), respectively. Once a 3 (i, j) and b3 (i, j) areavailable, the signal parameters can be determined by usingeither the frequency-domain approach or the time-domainapproach developed for finite-extent signals.

Multiplying both sides of Eq. (5a) by A 3(z1 , Z 2 ) and takinginto account Eqs. (5b) and (5c), we obtain

P qZ a3(i, j)z- z2jS 3(z1 , z2) = 2 b3(i, j)z1-iz 2 . (13)ij=-p ij=-q

Taking the inverse Z transforms of both sides of Eq. (13), weobtain

pZ a3 (i, j)s3(m - i, n -j) = 0ij=-p

(14a)

= b3(m, n) (m, n) E T(q).

(14b)

From recursion Eqs. (14a) and (14b), the basic steps of theIIR reconstruction algorithm are as follows:

Step 1. Given s3(m, n), form the system of linear equationsresulting from Eq. (14a) [see also Eq. (17c) below]. Itssolution determines ta3(m, n)Im,n=-pP.

Step 2. Given s3(m ,n) and ja3(m, n)jm,n=-pP, compute 1b3(m,n)lm,n=-qq from Eq. (14b).Step 3. Given a3(m, n)1m,n=_pP and b3(m, n)1m,n=_qq, findfa(i)J=1P and b(i)),=1q by using the FIR algorithms describedin Subsection 3.A.Step 4. Substitute Eq. (2e) into Eq. (la) and compute thereconstructed signal values.

The IIR algorithm applies to one- or two-sided infinite-duration signals. The reconstructed signal will be one sidedif the a(k) terms obtained from the a3(i, j) terms correspondto the coefficients of a polynomial A(z) whose roots lie eitherall inside or all outside the unit circle. If A(z) has roots bothinside and outside the unit circle, then the reconstructedsignal will be two sided. Moreover, the IIR algorithm is notrestricted to triple correlations. The counterparts of Eqs.(14a) and (14b) for quadruple correlations can be found inAppendix B.

Because pixels of raster-scanned images exhibit depen-dencies on both the forward and the backward directions,the two-sided models obtained by using triple correlationscan be used for image modeling and interpolation.2 0 Fur-thermore, the 1-D reconstruction algorithms of two-sidedmodels are useful for deblurring distorted images,2 3 decon-volution of astronomical data,26 and absorption spectrosco-py.2 7 The two-sided (or noncausal) signals in these applica-tions represent the distortion introduced by the sensor orthe channel. The objective is to reconstruct or deconvolvethe original signal from noisy measurements. Two-dimen-sional versions of the time-domain algorithms described inthis subsection are possible, such as those described in Ref.28, but, as we mentioned in Subsection 3.A, they are compu-tationally demanding.

In what follows, it is shown how the symmetries involvedin multiple correlations can be used to reduce computationsin step 1 of the IIR algorithm. For notational simplicity weshall confine ourselves to triple correlations. In Eq. (14a) itappears that there are (2p + 1)2 unknowns, namely, 1a3(i,j)Jij=pP. However, because a3(i, j) corresponds to the triplecorrelation of the sequence a(i) =oP, the nonredundant a3(i,j) samples are contained in the triangle T(p) defined by 0 < j< i < p (see also Fig. 1). Grouping together s3(m, n) termsthat have symmetric a3(i, j) coefficients, we can rewrite Eqs.(14a) and (14b) as

(a) a3 (0, 0)s3 (m, n)(b) + 8=2g~J= a3(i, ) [S3(m - i, n - i) + s3(M - i, n - i)

+ s3(m + i, n + i -j) + s3(m + i -j, n + i) + s3(M +j, n - i +) + s(m - i + i, n +i)]

(c) + Eg=1 a3(i, i)[s3(M-i, n-i) + s3(M + i, n) + s3(m, n+ i)]

(d) + = a3(i, O) [s3(m - i, n) + s3(m, n - i) + s3(M + i, n+ i)]

= 0 (m, n) $ T(q) (15a)

(15b)= b3(m, n) (m, n),F T(q),

In Eqs. (15a) and (15b) term (a) includes only the triplecorrelation a3(i, i) at the origin, term (b) includes the a 3(i, j)terms with arguments (i, j) inside the triangle T(p), term (c)has a3(i, j) terms with i = j [on the hypotenuse of the T(p)

6(.)= 2(q + 1)(q + 2)

q m

X E1 E1 -

M==O n=O ,


(m, n) $ T(q),


triangle], and term (d) contains a3(i, j) terms with (i, j) termson the bottom horizontal line of the T(p) triangle. Noticethat the number of a3 (i, ) unknowns in the T(p) triangle,andhenceinEqs. (15a),isl+2+ ... +p+1 = (p+l)(p+2)/2. Furthermore, from Eq. (8a) with a(0) = 1 [cf. Eq. (2b)]we obtain

.~) = 3p (15a3 (p, 0)

which expresses the a(i)-i=jP signal parameters in terms oftheir triple-correlation samples with arguments on the lastvertical slice of the T(p) triangle.

To simplify Eqs. (15a) and (15b) further, let us define thescaled coefficients

a3(i, j)d3(i j) =a

b3(i, j)&3 b" j a3 (P, 0) (15d)

Next, we divide both sides of (15a) by a3 (p, 0) = a(p) 3d 0 [cf.Eq. (5e) with a(0) = 1] and use Eqs. (15c) and (15d) to obtainp-lI a(j)[s(m - p, n - j) + s3(m - j, n - p)J=1

+ 3(m + p, n + p - j) + s3(m + p - j, n + p)

+ 3(m + j, n - p + j) + s 3(m - p + j, n + j)]+ a(p)[s 3 (m - p, n - p) + s3 (m + p, n) + 3(m, n + p)]

p-1 i-1

+ a3(0, 0)s3(m, n) + , a 3(i, j)[s3(m-i, n-j)i=2 j=1

+ s3(m -j, n - i) + s3(m + i, n + i -j)+ 3( + i-j, n + i) + 3(m +j, n - i +j)

p-1+ s3(m-i + j, n + )] + E x3(i, i)

i=l

X [s3(m-i, n - i) + s3 (m + i, n) + s3 (m, n + i)]p-1

+ 3(i, 0)[s3( - i, n) + s3 (m, n - i) + s3(m + i, n + )]i=l

=- 1[s 3(m - p, n) + s3(m, n - p) + 3 (m + p n + p)]

(m, n) p T(q), (16a)

=- [1S3(M - p n) + 3(m, n -p)+ s3 (m + p, n + p)] + 63(m, n) (m, n) e T(q). (16b)

Notice that after the normalization by a3 (p, 0) the number ofunknowns in Eq. (16a) is (p + 1)(p + 2)/2 - 1 = p(p + 3)/2.

By concatenating Eq. (16a) for m = p + 1, .. ., 2p and n =p . . . , m we can obtain the following system of p(p + 3)/2equations with p(p + 3)/2 unknowns:

S101 = s1, (17a)

where

01' = [a(1) ... a(p); 3 (0, 0); 3(2, 1) ... 3( - 1, p - 2);

a3(1, 1), ... , 3( - l1p -1); 3(1, 0) .. (p -1, 0)],(17b)

and the definitions of SI and s can be inferred from Eq.(16a). The maximum 3 (i, ) lags in Eqs. (16a) and (16b)come from the term s3(m + p, n + p) when m = n = p.Careful examination of the left-hand sides of Eqs. (16a) and(16b) for m = p + 1, .. ., 2p and n = p, . ., m shows that thetriple-correlation lags s 3 (i, ) in SI and s lie in the regionbounded by m = 1, m = n, m =3p, m = n + 2p, and n = 0.Note that the triple-correlation lag (0, 0) is not used in SIand s1.

If SI is of full rank, then Eq. (17a) has a unique solutiongiven by

01 = S1 s 1S. (17c)

* jf ' 1, -... JJ. V J1 ULL.l. UUW'1I111V UilqUtuy

the a(i) parameters. Having available the a(i) parameters,we compute 163(i, j)jj=_qq by using Eq. (16b) and obtain theb(i) parameters by using Eq. (10). Therefore, in the idealnoise-free case, the signal parameters can be obtaineduniquely by using triple correlations.

Alternatively, we can incorporate 3(m, n) into 01 andcombine the previous set of equations with the ones ob-tained by using Eq. (16b) with m = 0, 1, ... , q, and n = 0, 1,. . . , m, to obtain

S2 02 = S2 (18a)

where

02' = [01'; 3(0, 0); 63(1, 0)53(1, 1); ... ; 6 3 (q - 1, 0),

... , 3(q - 1, q - 1); &3(q, 0) .. . 53(q, q)], (18b)

If S2 is of full rank, then Eq. (18a) can be solved to obtain

02 = S 2 S2. (18c)

The first p components of 02 correspond to the a(i) parame-ters. The last q + 1 components are the normalized triplecorrelations 1b3(q, j)lj=oq, from which bA(q, j)j=oq can befound by means of Eq. (15d). b3(q, j)j=oq can be used toobtain the b(i) coefficients as in Eq. (10).

In practice, the exact triple correlations that appear in Eq.(17c) or (18c) are replaced by 93(m, n), obtained after trun-cating the infinite sum in Eq. (3):

L-I93 (m, n) = s(i)s(i + m)s(i + n), m n 0, (19a)

i=O

where L denotes the length of the available data record.To improve the accuracy of the signal parameter estimates

in the nonideal case, we use all the redundant informationcontained in solution 01 or 02 and use either the frequency-domain algorithm or the time-domain algorithm of Subsec-tion 3.A to obtain the finite-signal parameters. The time-domain approach is to use Eq. (11) and to estimate the a(j)parameters as

J = _ 2au) (p+1)(p+2)

p m a3 /3(m, n) 3(p, j)

m=O n=O a3(p, i) 3 (p, i + m)&3(p, i + n)

i=o 0

j = 01, . .. ,p. (19b)


'Ph. fl,,.+ - -- P +k- -1-4.- A-4-_-__ --- 1-


The () parameters can also be obtained by using Eq. (11),after using Eq. (15d) to find 63(m, n) from 53(m, n).

Overdetermined solutions of Eq. (17a) or (18a) can beobtained by considering Eq. (16a) with m = p + 1, .. , 2p,2p + 1, . . . and n = p, . . , m to produce a system of moreequations than unknowns:

(20a)SoOO = SO.

The overdetermined system of equations described by Eq.(20a) can be solved by using (possibly weighted) leastsquares:

0 0 WLS = (S0'WS 0)-'S 0 Ws 0 (20b)

where W is a constant matrix whose weights are adjustedbased on our prior knowledge about the noise distribution.Least-squares solutions correspond to the choice W = I.

An alternative interpretation of the least-squares solution[Eq. (20b)] results if we substitute Eq. (19) into Eq. (14a)and rewrite Eq. (14a) as

pZ 3 (i, j) 3 (m- i, n -j) = e(m, n), (m, n) T(q),41=-p

(21)

where e(m, n) can be thought of as an equation-error termaccounting for the inexact 93(m, n) samples. In Eq. (21),163 (i, D)ij=-pp can be viewed as the least-squares fit of arational model to the 2-D time-series 93(m, n). From thisviewpoint the solution in Eq. (20b) is optimal in the sensethat it minimizes the sum of the squared errors,2p+12 e2(m, n).If we know a priori that the IIR sequence is one sided, a

system of linear equations can be derived for obtaining thea(i) terms that contains fewer equations than the systemsused in Eqs. (17c) and (18c). For example, if we know thats(k) is right sided, i.e., that all the roots of A(z) lie inside theunit circle, then by multiplying both sides of Eq. (5a) byA(z1-1z2-1) we obtain

P ~~~~~B3 (Zl, Z2 )E a(i)zliZ2iS3(Zl, Z2) = A(z )A(z2) (22a)

.i=o 2-Az)~ 2

Taking the inverse Z transforms of both sides of Eq. (22a),we obtain

did not use this solution for two reasons: (1) The signalmust be assumed to be one sided. (2) In practice, when93(m, n) is used instead of s3(m, n), the a(i) terms obtainedlead to an A(z) polynomial some roots of which might lieoutside the unit circle; hence the reconstructed sequencemight be two sided (or one sided but unstable), although thetrue signal is one sided and stable.

4. MODEL SELECTION: NOISY SIGNALS

In this section we discuss methods for determining the or-ders p and q, based on triple correlations. In Section 3 weassumed that S, is of full rank, p = p(p + 3)/2, and that p isknown. If p is known, orders P2 (P1) of the right-hand (left-hand) part of the general two-sided IIR sequence are givenby the number of roots of the polynomial A(z) that lie inside(outside) the unit circle.

It was shown in Ref. 16 that if we collect the forms of Eq.(16a) with m =-p-1, .. ., -2p - 1 and n = q - p, . ., q +p, the resulting matrix So in Eq. (20a) will be of full rank, p =p(p + 3)/2, and hence its solution will be unique. To deter-mine p, rank indicators of right-hand-sided and left-hand-sided Hankel matrices formed by using triple-correlationlags are used.16 If the signal is one sided, the rank of anappropriate one-sided block Hankel matrix suffices'6 forobtaining the order p. A practical approach for determiningthe rank of the aforementioned Hankel matrices, especiallywhen sampled triple correlations are used, is the singularvalue decomposition.'5 Notice that determining p is rela-tively easy, since p = p(p + 3)/2 is not permitted to take allnatural numbers (e.g., if p = 1, 2, 3, 4, .. , then p = 2, 5, 9,14,....).

To determine the order q we search for the lag q for whichbA(q + 1, 0) is effectively zero while b3(0, 0) is effectivelynonzero. A detailed description of this method can be foundin Ref. 15.

Although these methods yield the autoregressive movingaverage orders, when p and q are estimated from raw dataone tends to overfit. For this reason the sensitivity of thepresent algorithm for modeling order mismatch is studied inthe simulations in Section 5.

In the algorithms of Section 3 the noise effects appear asdeterministic noise perturbations of the exact triple-correla-tion sequence. Thus one approach to modeling the noisycase is to consider

Pa(i)s3(m + i, n + i) = 0, m n, m > q, (22b)

i=O

where we have used the fact that the right-hand side of Eq.(22a) has a right-sided Laurent expansion. The derivationof Eq. (22b) for deterministic signals is new, but for randomsignals Eq. (22b) was derived in Ref. 2 and was used subse-quently in Refs. 12 and 13. In Ref. 19, Eq. (22b) was used fortransient analysis after the triple correlation of a determinis-tic transient signal was equated with that of the stochasticsignal-plus-noise signal.

By concatenating Eq. (22b) for m = p + 1, .. , 2p and n =

q - p, . . , q + p we can obtain a system of p linear equationsthat can be solved to obtain the 1a(i)1i=P parameters. We

§3(m, n) = s3(m, n) + v(m, n), (23)

where v(m, n) denotes the 2-D perturbation sequence.Equation (23) represents a model in which the noise is addedin the triple-correlation domain. Such a noise model isapplicable if, for example, a noise-free ultrashort laser pulse5

is triple correlated by a nonideal triple correlator. In prac-tice, the reconstruction algorithms of Section 3 use 93(m, n)and redundant information in the triple-correlation domainto average out v(m, n) and to improve the signal-to-noiseratio (SNR) in the reconstructed signal.

A more realistic noise model is that of Eq. (lb), in whichrandom noise is added to the signal instead of to its triplecorrelation. In this case it is easy to show that

Georgios B. Giainnakis


x3(m, n) = s3(m, n) + s(i)s(i + n)v(i + m)

+ s(i + m)s(i + n)v(i)

+ s(i)s(i + m)v(i + n)

+ v(i)v(i + n)s(i + m)

+ v(i)v(i + m)s(i + n) + v3(m, n), (24)

where x 3 (m, n) and v3 (m, n) are defined as in Eq. (3) with x(i)and v(i) used instead of s(i).

The noise distribution is generally unknown, and the sta-tistical properties of x3(m, n) cannot be found from a singledata record that includes a single noise realization. Forthese reasons, we assume that many data records,JxWi(k)}j= 1N, are available (as is the case in astrophotogra-phy,'0 for which N = 102 - 106) and that the noise satisfiesassumption (2). Under these assumptions,

N

1 x3P)(m, n) s3(m, n) + Elv(i)jHi=l

X , [s(i)s(i + m) + s(i)s(i + n) + s(i +m)s(i + n)]

+ [Efv(i)v(i + m)j + Ejv(i)v(i + n)} + Efv(i + m)v(i + n)j]

X s(i) + Etv(i)v(i + m)v(i + n)}. (25)

If the random noise is zero-mean and i.i.d. with a variance cr,2

and a third-order moment 7Y3v, then, for sufficiently large N,relation (25) becomes

1Nx3(m, n) - X )(m, n)

i=l

= s3(m, n) + a,2[6(m) + 6(n) + 6(m - n)]

X s(i) + 7356(m, n). (26)

Equation (26) shows that when the signal is zero mean [i.e., s= Ti s(i) = 0, only the (0, 0) triple-correlation lag will beaffected by the additive noise. Furthermore, if the noise isdistributed symmetrically [e.g., if v(i) is Gaussian], then Y3

= 0, and the reconstruction algorithms will be insensitive tonoise. If s X 0 but Etv(i) = 0, then we can estimate byusing 7i x(i) and apply the reconstruction algorithms byusing x(k) = x(k) - instead of x(k).

domain approach, we generated the 32-point-long, double-peaked signal shown as a solid curve in Fig. 2(a). Aftercomputing its triple correlation from Eq. (3), we added i.i.d.Gaussian noise, v(m, n), at different SNR levels to obtain thenoisy triple correlation 6

3(m, n) = 3 (m, n). In this experi-ment SNR is defined as SNR - 5m,n b3

2(m, n)/Ymn v32(m,

n). The noisy bispectrum P3(k, ) was computed by meansof a 32-point FFT, and the frequency-domain reconstruc-tion [Eq. (7g)] was used to obtain the other signals plotted inFig. 2(a) (dashed curve, SNR = 10; dotted curve, SNR = 1).The reconstructed signals computed by using the time-do-main solution [Eq. (11)] at SNR = 10 (dashed curve) andSNR = 1 (dotted curve) are shown in Fig. 2(b). To suppressthe noise effects further the noisy triple-correlation sampleswere averaged over the six triangles of their hexagonal regionof support, before the time-domain algorithms were applied.Similarly, as in Ref. 22, the noisy bispectral samples wereaveraged over the different octants before the frequency-domain algorithm was applied. By comparing Fig. 2(a) withFig. 2(b) we can infer that the frequency-domain algorithmoutperformed the time-domain algorithm at least in thisparticular example. Perhaps this is partially due to theadditional averaging that is included in the bispectrum com-putation performed in the frequency-domain approach.

To study the effect of the noise in the signal domain, 70data records were created from the same FIR signal, whichwas contaminated with i.i.d. Gaussian noise according to Eq.(lb). The SNR in this experiment was defined as SNR F, b2 (i)/E v(i). The reconstructed signal was computedafter the triple correlations of each record were averaged andthe time-domain iterative solution described in Appendix Cwas used.

To demonstrate the progressive SNR improvement, Figs.3(a), 3(b), and 3(c) depict the reconstructed signal obtainedafter the averaging of signal estimates obtained from the lastone, the last two, and the last three triple-correlation verticalslices, respectively, at SNR = 1. The same results for SNR= 1/10 are plotted in Figs. 3(d), 3(e), and 3(f), respectively.Notice from Figs. 3(a)-3(c) that the initial signal estimateobtained [from Eq. (10)] from the last triple-correlation ver-tical slice is quite good, gets better after the second slice isincorporated, but becomes worse when the last three triple-correlation slices are used. Apparently the third slice wasinaccurate for this noise realization. In the SNR = 1/10case, the reconstruction quality improves as more slices areadded in the iterative time-domain algorithm [see Figs.3(d)-3(f)]. A conclusion drawn from simulations with noiseadded to more than 50 signal records is that in most casesthree triple-correlation slices yield adequate SNR improve-ment in the reconstructed signal.

To test the IIR reconstruction algorithms, an infinite-duration two-sided signal with a Z transform

S(z) =1

(1 - 0.8z)(1 - 0.8z- + 0.65z-2 )

1

-0.8z( - 2.05z- + 1.65z-2 +-0.8125z- 3 )5. SIMULATIONS

To test the frequency-domain reconstruction approach forfinite-duration signals and to compare it with the time-

was chosen. The true signal is shown in Fig. 4 (solid curve).By truncation of the infinite sum in Eq. (3) from -32 to 32,30 records (65 points each) were created, which subsequent-

Georgios B. Giarmakis


ly were contaminated by zero-mean, i.i.d. Gaussian noise.Next, the noisy triple correlations of each record were aver-aged. When the order-determination approach of Ref. 16was applied, it was found that D = 3. The system of equa-tions in Eq. (17c) was solved with p = 3, and the sequence0 3 (i, j)lij=_

3' was obtained from which the a(i) terms were

computed by using Eq. (19b). The estimated signal forSNR = 10 is plotted in Fig. 4 (dashed curve). To improve

3.5

3

2.5

2

1.5

0.5 _o

the reconstruction quality, four equations were added, andthe resulting overdetermined system [Eq. (20b)] was solvedby using least squares, with SNR = 10 (see Fig. 4, dottedcurve). Because the order-selection procedure does not al-ways yield the correct orders, the solution was overfittedwith a (4, 1) model; i.e., it was assumed that b = 4 and q = 1,and the signal parameters a(i)}i=1

4 and b(1) were computedfrom the solution of Eq. (18c). The reconstructed signal at

35

time

(a)

3-

2.5 -

-

CZ

2-

1.5-

10-

0.5 -

0 10 20 30

time

(b)Fig. 2. (a) True signal (solid curve) versus signals reconstructed by using the frequency-domain method, at SNR = 10 (dashed curve) and SNR= 1 (dotted curve). (b) True signals (solid curve) versus signals reconstructed by using the time-domain method at SNR = 10 (dashed curve)and SNR = 1 (dotted curve).

5 10 15 20 25 30

4',

I

* .~ ~ ~ . .X C_'X ;X ..X ,* ...X 5


Vs

;S.


3-

2.5 -

2-

1.5 -

1-

3 -

2.5 -

2-

1.5 -

0 10 20 30

time

(a)

II

I1

.~~~~~~~~~~~~

0 10 20 30

time

(b)(Figure 3 continued)

SNR = 10 is not so good as the one obtained with the correctorders, but one can still identify the shape of the originalsignal (see Fig. 4, dashed-dotted curve).

In the third simulation example the IIR algorithm wasapplied to the right-hand-sided infinite-duration signal witha Z transform:

S(z) =1 - 1.25z-

1 - 1.5z-1 + 0.8z-2

The first 64 points of the signal are shown in Fig. 5 (solidcurve). With the correct orders p = 2 and q = 1 and 50 noisydata records (64 points each, SNR = 1), the signal depictedin Fig. 5 (dashed curve) was reconstructed by using the least-squares solution [Eq. (20b)].

6. CONCLUSIONS

Existing frequency-domain algorithms use the bispectrumto reconstruct separately the amplitude and the phase of afinite-duration signal. By using a digital-signal-processingapproach, a recursive FFT-based algorithm was developedthat reconstructs the DFT of the signal from third- or high-er-order correlations. The algorithm is developed for both1-D and 2-D signals. By averaging over redundant informa-tion present in multiple correlations, the signal-to-noise-ratio is improved in the reconstructed signal.

A time-domain method has also been proposed that ap-plies to finite- or infinite-duration, one- or two-sided signalsthat are described by a linear, constant-coefficient paramet-ric difference equation. For finite-duration (perhaps noisy)

am

p

i

tU

de

I

'/

am

p

i

tU

de

1



3-

am

p1

i

tU

de

2.5 -

2-

1.5 -

1 -

0.5 -

3.5-

am

pI

i

t

U

de

3 -

2.5 -

2-

1 .5 -

1-

0 10 20 30

time

(c)

M~~~~~

Iit%~~~~~~~~~~~~~~IA gI~~~~~~~Af-<e, ^ § t~IZ . iat0j , I 11I .,A'1\

tu'V \. g .~~~~~~

I I I I

0 10 20 30

time

(d)

signals, closed-form and iterative solutions of the signal pa-rameters were derived in terms of their nonredundant triple-correlation samples. In simulated examples, the frequency-domain method outperformed the time-domain methodwhen the available triple correlations were corrupted byi.i.d. Gaussian noise.

The case of infinite-duration parametric signals reducesto the finite-duration case after the triple correlations of thesignal parameters are obtained (by means of linear equa-tions). Overdetermined least-squares solutions appropriatefor recovering noisy signals were also described. In theory,model selection becomes a rank-determination problem,and, as verified by simulations, the correctness choice of themodel that is chosen has a large effect on the quality of the

reconstruction. When noise is present in the signal domain,averaging over many records is necessary to obtain reliabletriple-correlation estimates in low-SNR environments.

APPENDIX A: FREQUENCY-DOMAIN FINITE-IMPULSE-RESPONSE ALGORITHM WITHFOURTH-ORDER CORRELATIONS

The quadruple correlation of a real signal fb(i)),=0q is definedas

qb4(nl, n2, n)= b(i)b(i + n)b(i + n)b(i + n. (Al)

i=o

.J-d



3-am

i

tU

de

2.5 -

2-

1.5

1

3-am

p

itU

de

2.5 -

2-

1.5-

1 -

I I I I

0 10 20 30

time

(e)

III

BeA l I I % I~~~~~ %_.~~~~~~~~~~~'

0 10 20 30time

(f)Fig. 3. (a)-(c) True signal (solid curves) versus reconstructed signals (dashed curves) obtained by using the time-domain iterative method(Appendix C) with (a) one, (b) two, and (c) three triple-correlation slices computed and averaged over 70 data records corrupted by i.i.d.Gaussian noise at SNR = 1. (d), (e), (f) Same as (a), (b), and (c), respectively, but for SNR = 1/10.

In the same notation as in Eq. (7a), the fourth-order spec-trum is given by

B4(k, m, 1) = B(k)B(m)B(l)B*(k + m + 1). (A2)

The fourth-order counterpart of the recursive Eq. (7d) andits initial conditions are

Bk = B4 (k, m, O)B()B -, B(l)B(k-) (A3)

B(O) = B4(0, 0, O 4)]l, (A4)

IB(1) = [B4(1, 0, 0)11/2B(0)

(1) = . (A5)

To resolve the ambiguity of picking the positive or negativereal root in Eq. (A4), we can bias the original signal to havealways positive B(O). Note that in the noisy case we canaverage the estimates obtained from Eq. (A3) not only withrespect to 1, as is done in Eq. (7g), but also with respect to m.The remaining steps of correcting for the phase (1) andinverting the resulting sequence, by means of the inverse

I II II I it

I k I YI I I

I I l

~A I


'IV

II

III


FFT, to obtain the reconstructed signal are identical to thosefor the triple-correlation case.

APPENDIX B: TIME-DOMAIN FINITE- ANDINFINITE-IMPULSE-RESPONSE ALGORITHMSWITH FOURTH-ORDER CORRELATIONS

From Eq. (Al) we obtain b4(q, q, j) = b(O)b2(q)b(j); hence

where

s4(ml, M2, M3, M4) s(i)s(i + m)s(i + m2)s(i + M3 )

i=-X

(B5)

and

b0i) = b(O) b4(q q j) j=0,l, ... ,q. (B1)P

a4(i1, i2, i3) = a(i)a(i + i)a(i + i2)a(i + i3).i=o

By following a procedure similar to that used in the deriva-tion of Eq. (9), we obtain

b(j) b4 l/4 (n,, n2, n 3)b4 (q, q, j)(Z q-m_1/

[r b4(q, q, i)b4 (q, q, i + m)b 4(q, q, i + n)i=o

j = 0,1,.. ,q. (B2)

Using Eq. (B2), we can reconstruct FIR signals based onquadruple correlations. In the noisy case, an equation anal-ogous to Eq. (11) can be derived for quadruple correlations ifwe average Eq. (B2) over the nonredundant (n1 , n2 , n3)fourth-order correlation lags.

For the IIR case, the counterparts of Eqs. (14a) and (14b)for quadruple correlations are

Solution of the linear system of equations resulting from Eq.(B3) will yield a4(nl, n2, n3), which can be substituted intoEq. (B4) to compute b4(nl, n2, n3). The signal parameterscan be subsequently obtained either from the closed-formsolution (B2) or by an iterative algorithm similar to the onedeveloped in Appendix C based on triple correlation slices.

APPENDIX C: ITERATIVE TIME-DOMAINFINITE-IMPULSE-RESPONSE ALGORITHMWITH TRIPLE CORRELATIONS

Using Eq. (10), we can find initial estimates b(0)(j)j=q of thesignal parameters from the last vertical slice 16 3(q, j)j=oq ofthe T(q) triangle (see Fig. 1). These initial estimates can beused subsequently to obtain new estimates based on the nextvertical slice 6 3 (q - 1, j)lj=oql:

p

Z a4(i1, 2, i3)s4(ml - il, M2 - i2, m3 - 3) = 0.'1= b n)=-P

= b4(Ml, M2, Md)

2

1.5

1

0.5::

0

-0.5

-10

timeFig. 4. True signal (solid curve) versus the reconstructed, two-sided, infinite-duration signal obtained from triple correlations computed andaveraged over 30 noisy-signal records (SNR = 10). The reconstructed signals are found by using Eq. (17a) (dashed curve), the least-squares so-lution [Eq. (20b)] (dotted curve), and Eq. (18c) with model-order mismatch (dashed-dotted curve).

Imil > q for some i = 1, 2, 3

elsewhere,

(B3)

(B4)

(B6)

10 20 30 40 50 60 70

- |



0.8

0.6

0.4

0.2

0

-0.2

-0.4

-0.6

-0.8

-10 10 20 30 40 50 60 70

timeFig. 5. True signal (solid curve) versus reconstructed one-sided, infinite-duration signals (dashed curve) obtained from triple correlationscomputed and averaged over 30 noisy signal records (SNR = 1). The reconstructed signal is found by using the least-squares solution [Eq.(20b)J.

* £=3(q -1,j) - 6(0)(I)6(0)(q)6(°)I(°)(0)b()(q - 1)

j=0,1, ... q-1. (C1)At this point we can either accept 1, 6(1), ... , (q - 1),6(0)(q)} as the new signal estimate or keep 6(0)(q) found in Eq.(10) and average the estimates obtained by Eqs. (10) and(Cl) for the remaining q - 1 signal parameters to obtain

&(1)() = ) + 6(0)0) j0,1, ... ,q-1. (C2)

In general, information from the (q - )th vertical slice canbe incorporated into the iterative algorithm by using

6(i) =

6 3(q -, ) - 81, (j)6i' 1^(j + q - 1)6b(-)(i + j)i=O

b^(1-1)()b^(')(q -I) 1)*j=,, ... ,q -l, (C3)

and

-= ) + -1)(C4)

parameter b(n) is estimated (and averaged) in q - n + 1iterations. Because b(l) is averaged the most (it is estimat-ed q times), and b(q) is averaged the least [it is estimatedonly once, in Eq. (10)], it is expected that accuracy decreasestoward the decreasing edges, or tails, of the reconstructedsignal. If the signal's tail is of importance, we may flip thesignal and run the algorithm with the triple-correlation se-quence of the reversed signal.

Because of the progressive nature of the algorithm, if wehave a priori knowledge of the signal's shape we can termi-nate the iterations when the reconstructed signal is of ade-quate quality. Moreover, if, at a certain stage, good esti-mates are obtained, further improvement can be achieved ifwe adopt these estimates as initial conditions and restart theiterative algorithm.

APPENDIX D: TWO-DIMENSIONALFREQUENCY-DOMAIN ALGORITHM WITHTRIPLE CORRELATIONS

The triple correlation of a (q + 1) X (q + 1) real-valued imageb(i, j) is defined as

b3 [(ml, n); ( 2 , n 2)]

If intermediate averaging is not used in Eqs. (C2) and(C4), then the estimates on the right-hand side of Eq. (C3)must be supplied from the initial signal estimates. In thiscase, if 6(')(j) denotes the 6(j) estimate obtained from the (q- )th slice, then the final averaged estimates will be givenby

q-j6(i) = -j +1 ()), j = 0l, ... ,q.

1=0

(C5)

As we span the vertical triple-correlation slices, the signal

q q

= Z Z b(i, j)b(i + m, j + nj)b(i + M2,] + n2),.=o j=0

(D1)

and its 2-D FFT (image bispectrum) is defined as

B3[(kl, 11); (k 2, 12)] = B(kl, 1j)B(k2, 12)B*(k + k 2, 11 + 12).

(D2)

Substituting k2 = 12 = 0, k - k - k2 , and 11 - 11- 12 into Eq.(D2) yields


j=0,1, ... q-1.


B3[(k1, 11); (0, 0)] = B(kl, 11)B(0, O)B*(k1, 11), (D3)

B 3 [(kl - k2 , 11 - 12); (k2, 12)] = B(k1 - k2 1 - 12)

x B(k2,1 2)B*(k 1, 11). (D4)

By eliminating B*(ki, 11) and solving for B(kl, 1l) in Eqs.(D3) and (D4), we obtain

B(k, ) B 3[(kl, 11); (0, 0)]B(k, - k2,11 - 1 2)B(k2,12)( 1 B3[(k- k 2, 11 - 12); (k2, 12)]B(0, 0)

(D5)

Equation (D5) is the 2-D counterpart of Eq. (7d) and can beused recursively to compute the 2-D FFT B(kl, 1l) from itsbispectrumB3[(k1, 11); (h2, 12)]. To initialize Eq. (D5), we setk = 1l = 0 in Eq. (D3) to obtain, as in Eq. (7e),

B(O, 0) = B31/3 [(0, 0); (0, 0)]. (D6)

Knowledge of B(O, 0) allows us to use Eq. (D3) to find theamplitude at any point of B(i, j) as

IB(kl, l)I = {B3 [(kl, 11); (0, 0)]IB(O, 0)11/2. (D7)

The main difference between the 1-D case and the 2-D caseis that in the latter two phase points must be set initially tozero (cf. the 2-D shift ambiguity); that is, by setting ,(0, 1) =0(1, 0) = 0, we can solve Eq. (D5) iteratively to obtain theestimates (k, 1), [k = 0, (q + 1)/2, 0 < 1 < (q + 1)/2], [0 < k <(q + 1)/2, 0 S l S q]. The remaining coefficients can befound by using the conjugate symmetry. As in the 1-D case,the redundancy in B3 can be used by averaging the f(kl, l)estimates to obtain the closed form,

4. A. W. Lohman and B. Wirnitzer, "Triple correlations," Proc.IEEE 72, 889-901 (1984).

5. B. Wirnitzer, "Measurement of ultrashort laser pulses," Opt.Commun. 48, 225-228 (1983).

6. K. Hasselman, M. Munk, and G. McDonald, "Bispectra of oceanwaves," in Proceedings of the Symposium on Time Series Anal-ysis M. Rosenblatt, ed. (Wiley, New York, 1962), p. 125.

7. K. Sasaki, T. Sato, and Y. Nakamura, "Holographic passivesonar," IEEE Trans. Sonics Ultrason. SU-24, 193-200 (1977).

8. H. Gamo, "Phase determination of coherence functions by theintensity interferometer," in Proceedings of Symposium onElectromagnetic Theory and Antennas (Pergamon, London,1963), pp. 801-810.

9. G. P. Weigelt and B. Wirnitzer, "Image reconstruction by thespeckle masking method," Opt. Lett. 8, 3893 (1983).

10. A. W. Lohman, G. Weigelt, and B. Wirnitzer, "Speckle maskingin astronomy: triple correlation theory and applications,"Appl. Opt. 22, 4028-4037 (1983).

11. K.-H. Hofman and G. Weigelt, "Astronomical speckle masking:image reconstruction by cross-triple correlation," Appl. Opt. 26,2011-2015 (1987).

12. M. R. Raghuveer and C. L. Nikias, "Bispectrum estimation: aparametric approach," IEEE Trans. Acoust. Speech Signal Pro-cess. ASSP-33, 1213-1230 (1985).

13. G. B. Giannakis and J. M. Mendel, "Identification of non-mini-mum phase systems using higher-order statistics," IEEE Trans.Acoust. Speech Signal Process. 37, 360-377 (1989).

14. G. B. Giannakis, "Cumulants: a powerful tool in signal process-ing," Proc. IEEE 75, 1333-1334 (1987).

15. G. B. Giannakis and J. M. Mendel, "ARMA order determina-tion via higher-order statistics," in Mathematical Theory ofNetworks and Systems, C. Martin, ed. (North Holland, Amster-dam, 1989).

16. G.B. Giannakis and A. Swami, "On estimating non-causal non-minimum phase ARMA models of non-Gaussian processes," inProceedings of 4th ASSP Workshop on Speckle Estimation &Modeling (Institute of Electrical and Electronics Engineers,New York, 1988), pp. 187-192; Rep. UVA/192444B/EE88/107(University of Virginia, Charlottesville, Va., 1988).

f3(kl, 11) = 1 B 3 [(kl, 11); (0, 0)]P(kl - k2, 1- 2).i(k 2 , 12)[(k + 1)(11 + 1) - 2] OEk2k1.0512511 B 3 [(kl - k 2 , l - 12); (h2 ,l 2 )]B(0, 0)

(R2,12) , (0,0)

k = 0, (q + 1)/2,

ACKNOWLEDGMENTS

I thank the project group in EE576, Spring 1988, SignalProcessing class, and especially Amod Dandawate, for per-forming part of the simulation examples of this paper. Thehelpful comments made by J. Mait on the initial version ofthis paper are also acknowledged. This research was sup-ported by University of Virginia Engineering Research Initi-ation grant 6-42410 and Harry Diamond Laboratories con-tract 5-25227.

REFERENCES

1. D. R. Brillinger and M. Rosenblatt, "Computation and interpre-tation of kth-order spectra," in Spectral Analysis of Time Se-ries, B. Harris, ed. (Wiley, New York, 1967), pp. 189-232.

2. E. Parzen, "Time series analysis for models of signals plus whitenoise," in Spectral Analysis of Time Series, B. Harris, ed. (Wi-ley, New York, 1967), pp. 233-257.

3. K. S. Lii and M. Rosenblatt, "Deconvolution and estimation oftransfer function phase and coefficients for non-Gaussian linearprocesses," Ann. Stat. 10, 1195-1208 (1982).

0 < < (q + 1)/2; 0 < k < (q + 1)/2, 0 S < q. (D8)

17. C. L. Nikias and M. R. Raghuveer, "Bispectrum estimation: adigital signal processing framework," Proc. IEEE 75, 869-891(1987).

18. A. V. Oppenheim and R. W. Schafer, Digital Signal Processing(Prentice-Hall, Englewood Cliffs, N.J., 1975).

19. C. K. Papadopoulos and C. L. Nikias, "Bispectrum estimationof transient signals," in Proceedings of International Confer-ence on Acoustics, Speech, and Signal Processing (Institute ofElectrical and Electronics Engineers, New York, 1988), pp.2404-2407.

20. A. K. Jain, "Advances in mathematical models for image pro-cessing," Proc. IEEE 69, 502-528 (1981).

2l. M. J. Northcott, G. R. Ayers, and J. C. Dainty, "Algorithms forimage reconstruction from photon-limited data using the triplecorrelation," J. Opt. Soc. Am. A 5, 986-992 (1988).

22. H. Bartelt, A. W. Lohmann, and B. Wirnitzer, "Phase and am-plitude recovery from bispectra," Appl. Opt. 23, 3121-3129(1984).

23. B. R. Hunt, "Digital image processing," in Applications of Digi-tal Signal Processing, A. V. Oppenheim, ed., (Prentice-Hall,Englewood Cliffs, N.J., 1981).

24. W. T. Rhodes, "Acousto-optic signal processing: convolutionand correlation," Proc. IEEE 69, 65-79 (1981).

25. A. Swami and J. M. Mendel, "ARMA parameter estimation

- .



using only output cumulants," in Proceedings of 4th ASSPWorkshop on Speckle Estimation & Modeling (Institute ofElectrical and Electronics Engineers, New York, 1988), pp. 193-198.

26. J. D. Scargie, "Studies in astronomical time series analysis:Modeling random processes in the time domain," Astrophys. J.Suppl. Ser. 1, 1 (1981).

27. W. E. Blass and G. W. Halsey, Deconvolution of AbsorptionSpectra (Academic, New York, 1981).

28. A. Swami and G. B. Giannakis, "ARMA modeling and phasereconstruction of multidimensional processes using cumu-lants," in Proceedings of International Conference on Acous-tics, Speech, and Signal Processing (Institute of Electrical andElectronics Engineers, New York, 1988), Vol. 2, pp. 729-732.


signal reconstruction from multiple correlations: frequency- and time-domain approaches

Documents