![Page 1: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/1.jpg)
CSSIP-INSPIRE
Computable Performance Analysis of SparsityRecovery with Applications
Arye Nehorai
Preston M. Green Department of Electrical & Systems Engineering
Washington University in St. Louis, USA
European Signal Processing Conference (EUSIPCO)
August 28, 2012
Computable Bounds 1
![Page 2: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/2.jpg)
CSSIP-INSPIRE
Acknowledgements
• Based on collaborations with Gongguo Tang (Ph.D. 2011) and Satyabrata Sen(Ph.D 2010).
• Supported by the US National Science Foundation, Air Force Office of ScientificResearch, and the Office of Naval Research.
Computable Bounds 2
![Page 3: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/3.jpg)
CSSIP-INSPIRE
Figure Acknowledgements
• Figures on slide 8 and slide 16 are adapted from R. Baraniuk, J. Romberg andM. Wakin’s slides “Tutorial on Compressive Sensing.”
• Figures on slide 9 and slide 10 are modified from E. J. Candes, J. Rombergand T. Tao, “Robust uncertainty principles: exact signal reconstruction fromhighly incomplete frequency information,” IEEE Trans. Inf. Theory, vol. 52,no. 2, pp. 489-509, Feb. 2006.
• Figures on slide 11 and slide 12 are reproduced from J. Wright, A. Yang,A. Ganesh, S. Sastry, and Y. Ma, “Robust face recognition via sparserepresentation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no.2, Feb. 2009.
• Figure on slide 14 is from D. M. Maliouto, “A sparse signal reconstructionperspective for source localization with sensor arrays,” Master thesis, MIT,2003.
Computable Bounds 3
![Page 4: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/4.jpg)
CSSIP-INSPIRE
Outline
• Introduction
• Sparsity recovery
• Application examples
• Future work
Computable Bounds 4
![Page 5: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/5.jpg)
CSSIP-INSPIRE
Outline
• Introduction
• Sparsity recovery
• Application examples
• Future work
Computable Bounds 5
![Page 6: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/6.jpg)
CSSIP-INSPIRE
Introduction
Low-dimensional structures are ubiquitous in signals:
• Sparse vectors– Compressive sensing– MRI– Image processing and computer vision
• Block-sparse vectors– Radar– Sensor array processing
• Low-rank matrices– Collaborative filtering– Robust principal component analysis
• Low-dimensional manifolds– Subspace learning– Manifold learning
Exploiting low-dimensional structures enables more accurate signal recovery.
Computable Bounds 6
![Page 7: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/7.jpg)
CSSIP-INSPIRE
Sparsity Example: Compressive Sensing
• Interest in exploiting sparsity grew recently due to developments in compressivesensing.
• Traditional signal sampling acquires a signal using expensive hi-fidelity sensors,then compresses the data with a loss of fidelity.
• Compressive sensing (CS) combines the acquisition with the compression bysampling the signal in a novel way with less data.
• CS replaces samples with general linear projections, and linear reconstructionwith non-linear reconstruction, thus shifting the burden from the hi-fidelitysensing to reconstruction.
• Key assumption: Many natural signals x have sparse representations in sometransform domains Φ, i.e., x = Φs for some sparse vector s.
Computable Bounds 7
![Page 8: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/8.jpg)
CSSIP-INSPIRE
Sparsity Example: Compressive Sensing (cont.)
Figure 1: Paradigm of compressive sensing.
• Surprising fact1,2: Suffices to use m = O(k log n) � n linear, non-adaptive,random measurements y to reconstruct a sparse signal, where k = ‖s‖0 is thesparsity level of s.
• The reconstruction performance depends on the sensing matrix A.
1E. J. Candes, J. Romberg and T. Tao, “Robust uncertainty principles: exact signal reconstruction from highlyincomplete frequency information,” IEEE IT, vol. 52, no. 2, pp. 489-509, Feb. 2006.
2D. L. Donoho, “Compressed sensing,” IEEE IT, vol. 52, no. 4, pp. 1289–1306, Apr. 2006.
Computable Bounds 8
![Page 9: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/9.jpg)
CSSIP-INSPIRE
Sparsity Example: MRI
• MRI images can often be well approximated by piece-wise constant functions.
• Instead of observing the image directly, we observe its Fourier transformcoefficients sampled along, e.g., a radial trajectory in the 2D spatial frequencydomain.
(a) Logan-Shepp phantom (b) Fourier transform (c) Sampling trajectory
Computable Bounds 9
![Page 10: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/10.jpg)
CSSIP-INSPIRE
Sparsity Example: MRI (cont.)
• Reconstruction using minimum energy (or `2 norm) results in many artifacts.
• However, the total variation (TV) minimization, which enforces the piece-wiseconstant property of the image, recovers the original image exactly.
(d) Min-energy recovery (e) Magnitude of gradient (f) Min-TV recovery
Figure 2: Exploiting sparsity improves the MRI recovery.
Computable Bounds 10
![Page 11: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/11.jpg)
CSSIP-INSPIRE
Sparsity Example: Image Processing and Computer Vision
• Sparse recovery is also useful in single-image super-resolution, and facerecognition3.
• In face recognition, a given facial image is sparsely represented using adictionary database.
• The significant coefficients in the representation reveals the person’s identity.
Figure 3: Robust face recognition with occlusion.
3J. Wright, A. Y. Yang, A. Ganesh, S. S. Sastry, and Y. Ma, “Robust Face Recognition via Sparse Representation,”IEEE PAMI, vol.31, no.2, pp.210-227, Feb. 2009
Computable Bounds 11
![Page 12: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/12.jpg)
CSSIP-INSPIRE
Sparsity Example: Image Processing and Computer Vision(cont.)
• This approach yields accurate recognition.
• It is also robust to occlusions and noise corruptions.
Figure 4: Robust face recognition with corruption.
Computable Bounds 12
![Page 13: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/13.jpg)
CSSIP-INSPIRE
Sparsity Example: Sensor Arrays
• We can also use sparsity to estimate continuous parameters in nonlinearmodels4.
• Consider, for example, the estimation of directions-of-arrival (DOAs) using asensor array.
• The narrowband observation model is given by y = A(θ)x+w, where A(θ)is the array manifold matrix, and x is the signal vector.
• The unknown DOA parameter θ is continuous.
4D. Malioutov, M. Cetin, and A. S. Willsky, “A sparse signal reconstruction perspective for source localizationwith sensor arrays,” IEEE TSP, vol. 53, no. 8, pp. 3010–3022, 2005.
Computable Bounds 13
![Page 14: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/14.jpg)
CSSIP-INSPIRE
Sparsity Example: Sensor Arrays (cont.)
• We discretize the parameter space into grid points given as θ = [θ1, . . . , θN ]T
to create a sparse recovery problem.
(a) DOAs of two sources. (b) DOA discretization.
Figure 5: Sparse modeling for DOA estimation.
• The observation model then becomes linear:
y = [A(θ1) · · · A(θN)]x + w = Ax + w,
where the entries of x are nonzero if and only if there is a source direction atthe corresponding grid point, implying x is sparse.
Computable Bounds 14
![Page 15: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/15.jpg)
CSSIP-INSPIRE
Sparsity Example: Sensor Arrays (cont.)
Figure 6: Super resolution results.
• Discretization leads to super resolution when there is no basis mismatch.
• Sensing matrix depends on the sensor configuration and discretization strategy.
Computable Bounds 15
![Page 16: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/16.jpg)
CSSIP-INSPIRE
Common Theme
• The sparse signal x is observed by alinear model corrupted by noise:
y = Ax+w.
• The sensing matrix A ∈ Rm×nprojects the set of sparse vectors inRn onto a low dimensional space Rm.
• There exist convex programs thatexploit sparsity to recover the signal.
• The recovery performance highlydepends on the sensing matrix A.
• A good matrix A should preserve thestructure of the set of sparse vectors.
Our goal: Find computable bounds on recovery errors for a given A.
Computable Bounds 16
![Page 17: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/17.jpg)
CSSIP-INSPIRE
Motivations for Computable Performance Analysis
A computable performance analysis would enable us to:
• Quantify the confidence in the reconstructed signal, especially when there areno other ways to justify the correctness of the reconstructed signal.
• Optimize the system design.
Figure 6: Two MRI sampling trajectories. Left: radial, Right: spiral.
Computable Bounds 17
![Page 18: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/18.jpg)
CSSIP-INSPIRE
Our Contributions
• Introduce a family of functions that quantify the goodness of sensing matricesin sparsity recovery.
• Derive bounds on reconstruction error in terms of these goodness measures forrecovery algorithms.
• Design efficient algorithms to compute these goodness measures and bounds.
Computable Bounds 18
![Page 19: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/19.jpg)
CSSIP-INSPIRE
Outline
• Introduction
• Sparsity recovery
• Application examples
• Future work
Computable Bounds 19
![Page 20: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/20.jpg)
CSSIP-INSPIRE
Model
Consider the measurement model
y = Ax+w,
where
• the signal x ∈ Rn is sparse with`0-sparsity level ‖x‖0 = k � n,
• the matrix A ∈ Rm×n has m rowsand n columns with m� n,
• the noise vector w ∈ Rm is either– bounded ‖w‖� ≤ ε, where� = 1, 2,∞ or
– Gaussian w ∼ N (0, σ2I).
Computable Bounds 20
![Page 21: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/21.jpg)
CSSIP-INSPIRE
Sparse Signal Recovery
• In the absence of noise, the signal x can be recovered by solving
P0 : arg minx‖x‖0 subject to y = Ax.
• P0 is a non-convex optimization problem and it is NP hard to solve5.
• Convex relaxation methods replace the `0 norm with the `1 norm.
5B. K. Natarajan, “Sparse approximate solutions to linear systems,” SIAM J. on Computing, vol. 24, no. 2, pp.227− 234, 1995.
Computable Bounds 21
![Page 22: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/22.jpg)
CSSIP-INSPIRE
Existing Recovery Algorithms
• Basis Pursuit: minz∈Rn ‖z‖1 subject to ‖y −Az‖� ≤ ε
• Dantzig Selector: minz∈Rn ‖z‖1 subject to ‖AT (y −Az)‖∞ ≤ λ
• LASSO Estimator: minz∈Rn12‖y −Az‖
22 + λ‖z‖1
Figure 6: Geometry of `1 minimization in the noise-free case.
Computable Bounds 22
![Page 23: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/23.jpg)
CSSIP-INSPIRE
Previous Approaches for Performance Analysis
Restricted Isometry Constant (RIC)6:
δk(A) = maxz:z 6=0
∣∣‖Az‖22/‖z‖22 − 1∣∣ subject to ‖z‖0 ≤ k,
– error bounds: If ‖w‖2 ≤ ε, then the recovery error of the Basis Pursuit isbounded as
‖x− x‖2 ≤4√
1 + δ2k(A)
1− (1 +√
2)δ2k(A)ε,
– computational difficulty: No practical way to compute δk(A) exactly.
6E. J. Candes, T. Tao, “Near-optimal signal recovery from random projections and universal encoding strategies”,IEEE Trans. Inform. Theory, vol. 52, no. 12, pp. 5406-5425, Dec. 2006.
Computable Bounds 23
![Page 24: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/24.jpg)
CSSIP-INSPIRE
Previous Approaches for Performance Analysis (cont.)
Mutual Coherence (MC)7:
µ(A) = maxi6=j
|ATi Aj|‖Ai‖2‖Aj‖2
,
– sufficient condition: if ‖x‖0 ≤ 12
(1 + 1
µ(A)
), then we get exact recovery
(noise-free case) via Basis Pursuit,
– however, this condition is weak,
– If ‖w‖2 ≤ ε, then the recovery error of the Basis Pursuit is bounded as8
‖x− x‖2 ≤2
1− µ(A)(4k − 1)ε.
7D. L. Donoho and M. Elad, “Optimally sparse representation in general (nonorthogonal) dictionaries via `1minimization”, in Proc. Nat. Aca. Sci., Vol. 100, pp. 2197-2202, Mar. 2003.
8D.L. Donoho, M. Elad, and V. Temlyakov, “Stable recovery of sparse overcomplete representations in thepresence of noise”, IEEE Trans. On Information Theory, Vol. 52, pp. 6-18, Jan. 2006.
Computable Bounds 24
![Page 25: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/25.jpg)
CSSIP-INSPIRE
Related Work on Computable Performance Analysis
The following papers included sufficient and necessary conditions for exactrecovery of sparse vector in the noise-free case:
9A. Juditsky, A. Nemirovski, “On verifiable sufficient conditions for sparse signal recovery via `1
minimization,” Mathematical Programming Ser. B, vol. 127, pp. 57-88, 2011.
10A. Juditsky, F. Kilinc Karzan, A. Nemirovski, “Verifiable conditions of `1-recovery of sparse
signals with sign restrictions,” Mathematical Programming Ser. B, vol. 127, pp. 89-122, 2011.
11A. d’Aspremont, L. El Ghaoui, “Testing the nullspace property using semidefinite programming,”
Mathematical Programming Ser. B, vol. 127, pp. 123-144, 2011.
Our Innovations
• Verification: More efficient algorithms to verify exact recovery without noise.
• Computation: Computable performance bounds on recovery errors in noise.
Computable Bounds 25
![Page 26: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/26.jpg)
CSSIP-INSPIRE
Quality Measure ω�(Q, s)
• For s ∈ [1, n] and A ∈ Rm×n, we define
ω�(Q, s) = minz:z 6=0
‖Qz‖�‖z‖∞
subject to‖z‖1‖z‖∞
≤ s,
where Q = A or ATA, and � = 1, 2, or ∞.
• s is a measure of the sparsity of z: Smaller s implies more sparse z; also largerω�(Q, s) and better reconstruction performance.
• Without the sparsity constraint, with`∞ norm replaced by `2 and � replacedwith `2, ω�(Q, s) is the minimalsingular value of Q.
• ω�(Q, s) is a measure of theincoherence (quality) of A, and it willdetermine the performance bounds.
Figure 7: Constraint set in R3 for s = 1.4.
Computable Bounds 26
![Page 27: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/27.jpg)
CSSIP-INSPIRE
Reconstruction Error Bounds
Theorem 1. Suppose the noise w satisfies ‖w‖� ≤ ε, ‖ATw‖∞ ≤ λ, and‖ATw‖∞ ≤ κλ, κ ∈ (0, 1), for the Basis Pursuit, the Dantzig Selector, and theLASSO estimator, respectively, then we have
‖x− x‖∞ ≤2ε
ω�(A, 2k)for the Basis Pursuit,
‖x− x‖∞ ≤2λ
ω∞(ATA, 2k)for the Dantzig Selector, and
‖x− x‖∞ ≤(1 + κ)λ
ω∞(ATA, 2k/(1− κ))for the LASSO estimator.
• These error bounds are inversely proportional to ω�(Q, s).
• ω�(Q, s) > 0 implies exact recovery in the noise-free case (where ε = 0, λ = 0).
• When the sparsity level k of the signal decreases, ω�(Q, s) becomes larger,implying smaller reconstruction error.
Computable Bounds 27
![Page 28: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/28.jpg)
CSSIP-INSPIRE
Reconstruction Error Bounds (cont.)
The error bounds on the `1 and `2 norms can be expressed via
‖x− x‖1 ≤ ck‖x− x‖∞ and ‖x− x‖2 ≤√ck‖x− x‖∞.
Computable Bounds 28
![Page 29: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/29.jpg)
CSSIP-INSPIRE
Topics of Next Slides
Computable Bounds 29
![Page 30: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/30.jpg)
CSSIP-INSPIRE
Verification of ω�(Q, s) > 0: General Case
We provide a computable way to verify sufficient conditions for exact sparserecovery in the noise-free case (see also Shtok et. al.12).
Theorem 2. Define s∗ = max{s : ω�(Q, s) > 0}. Then,
k ≤ bs∗/2c =⇒ exact sparse recovery.
In addition, s∗ is the inverse of the maximum of the n optimal values of thefollowing linear programs:
maxz
zi subject to Qz = 0, ‖z‖1 ≤ 1, i = 1, . . . , n.
Thus, bs∗/2c is the maximal sparsity level below which exact recovery isguaranteed in the noise-free case.
12J. Shtok and M. Elad, “Analysis of the Basis Pursuit Via the Capacity Sets”, Jour. of Fourier Analysis andApplications, vol. 14, no. 5-6, pp. 688-711, Dec. 2008.
Computable Bounds 30
![Page 31: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/31.jpg)
CSSIP-INSPIRE
Verification of ω�(Q, s) > 0: Fourier Case
For the special yet important class of Fourier sensing matrices, the computationalcost can be greatly reduced.
Theorem 3. If H is the Fourier transform matrix on a finite abelian group,and the rows of A are sampled from the rows of H, then the optimal values of
maxz
zi subject to Qz = 0, ‖z‖1 ≤ 1
are equal for i = 1, 2, . . . , n.
• For these sensing matrices, we compute s∗ by solving a single linear program.
• Examples include the Fourier matrix and the Hadamard matrix, which arewidely used in compressive sensing
Computable Bounds 31
![Page 32: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/32.jpg)
CSSIP-INSPIRE
Numerical Examples: Maximal Sparsity Levels
Table 1: Comparison of our algorithm for sufficient conditions for exact recoverywith Juditsky and Nemirovski’s in bounding the maximal sparsity levels bs∗/2cfor Gaussian sensing matrices (n = 256).
Max Sparsity Level
mbs∗/2c CPU time
Our Algorithm JN’s Algorithm Our Algorithm JN’s Algorithm
25 1 1 6 9151 2 2 8 19176 3 3 10 856
102 4 4 13 5630128 4 5 16 5711153 6 6 20 1381179 7 7 24 3356204 10 10 25 10039230 13 14 28 8332
Observation: The two algorithms give similar maximal sparsity levels, but ours ismuch faster.
Computable Bounds 32
![Page 33: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/33.jpg)
CSSIP-INSPIRE
Numerical Examples: Maximal Sparsity Levels (cont.)
Table 2: Comparison of sufficient conditions for exact recovery based on ω andthe Mutual Coherence for Hadamard matrices.
Max Sparsity Level
mn
n = 2048 n = 4096 n = 8192
bs∗
2 c b12(1 + 1µ)c bs
∗
2 c b12(1 + 1µ)c bs
∗
2 c b12(1 + 1µ)c
.2 6 3 9 4 12 6
.3 9 4 13 5 18 7
.4 12 5 17 7 24 10
.5 15 6 21 9 30 13
.6 19 8 27 9 39 16
.7 25 9 35 14 50 17
.8 34 13 48 17 68 23
.9 53 20 74 22 105 34
Observation: Our sufficient condition for exact recovery is stronger than given bythe Mutual Coherence for Hadamard matrices.
Computable Bounds 33
![Page 34: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/34.jpg)
CSSIP-INSPIRE
Computation of ω�(Q, s): General Case
We provide a way to compute ω�(Q, s) for any given Q and s, which readilytranslates to upper bounds on recovery errors, in the noisy case.
Theorem 4. The quantity ω�(Q, s) is the minimum of the optimal values ofthe following n linear programs or quadratic programs:
minu∈Rn−1
‖qi −Q(:,−i)u‖� s.t. ‖u‖1 ≤ s− 1, i = 1, . . . , n.
• Here qi is the ith column of Q and Q(:,−i) are columns except the ith one.
• The ith optimization finds the best approximation of the ith column using asparse (measured by `1 norm) linear combination of the rest columns.
• Thus, if the columns of Q can well approximate each other using sparse linearcombinations, ω� is small and the reconstruction error is large.
• Note, the Mutual Coherence considers the approximability between twocolumns. Our ω� is more accurate because it considers all columns.
Computable Bounds 34
![Page 35: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/35.jpg)
CSSIP-INSPIRE
Computation of ω�(Q, s): Fourier Case
The computation cost can be greatly reduced for the special yet important classof Fourier sensing matrices, similar to the verification case.
Theorem 5. If H is the Fourier transform matrix on a finite abelian group,and the rows of A are sampled from the rows of H, then the optimal values of
minu∈Rn−1
‖qi −Q(:,−i)u‖� subject to ‖u‖1 ≤ s− 1
are equal for all i = 1, . . . , n.
• For these sensing matrices, we compute a single ω�(Q, s) by solving a singlelinear program or quadratic program.
• Examples include the Fourier matrix and the Hadamard matrix.
Computable Bounds 35
![Page 36: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/36.jpg)
CSSIP-INSPIRE
Numerical Examples: Performance Bounds Comparison I
Table 3: ω2(A, s) based bounds ‖x− x‖2 ≤ 2√
2kω2(A,2k)ε vs. RIC based bounds
‖x− x‖2 ≤4√
1+δ2k
1−(1+√
2)δ2kε for the Basis Pursuit with Bernoulli sensing matrices
and n = 256 with ε = 1.
Bound on Estimation Errork m 51 77 102 128 154 179 205
1ω bound 4.2 3.8 3.5 3.4 3.3 3.2 3.2RIC bound 23.7 16.1 13.2 10.6 11.9
2ω bound 31.4 12.2 9.0 7.4 6.5 6.0 5.6RIC bound 72.1 192.2
3 ω bound 252.0 30.9 16.8 12.0 10.1 8.94 ω bound 52.3 23.4 16.5 13.65 ω bound 57.0 28.6 20.16 ω bound 1256.6 53.6 30.87 ω bound 161.6 50.68 ω bound 93.19 ω bound 258.7
Computable Bounds 36
![Page 37: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/37.jpg)
CSSIP-INSPIRE
Numerical Examples: Performance Bounds Comparison II
Figure 8: ‖x− x‖2 ≤ 2√
2kω2(A,2k)ε vs. MC based bound ‖x− x‖2 ≤ 2
1−µ(A)(4k−1)ε
for the Basis Pursuit for Hadamard sensing matrices with n = 2048.
Computable Bounds 37
![Page 38: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/38.jpg)
CSSIP-INSPIRE
Numerical Examples: Observations
• The bounds using ω� are tighter than the bounds based on Mutual Coherenceor RIC.
• Bounds based on ω� still apply even when the bounds based on MutualCoherence or RIC do not apply, e.g., for small m and large k.
Computable Bounds 38
![Page 39: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/39.jpg)
CSSIP-INSPIRE
Quality Measure: l1-CMSV
• For s ∈ [1, n] and A ∈ Rm×n, we define the l1-constrained minimal singularvalue (l1-CMSV) of A by
ρs(A) = minz:z 6=0
‖Ax‖2‖x‖2
, subject to‖z‖21‖z‖22
≤ s.
• ρs(A) is approximated by solving a constrained optimization problem using aninterior point method.
• Similar to ω�(Q, s), ρs(A) is a measure of incoherence of A, and s is ameasure of the sparsity of z.
• Bounds using ρs(A) are computationally more amenable than using RIC.
• Bounds based on ρs(A) are also tighter than bounds using ω�(Q, s).
Computable Bounds 39
![Page 40: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/40.jpg)
CSSIP-INSPIRE
Reconstruction Error Bounds Using l1-CMSV
Theorem 6. Suppose the noise w satisfies ‖w‖2 ≤ ε, ‖ATw‖∞ ≤ λ, and‖ATw‖∞ ≤ κλ, κ ∈ (0, 1), for the Basis Pursuit, the Dantzig Selector, andthe LASSO estimator, respectively, then we have
‖x− x‖2 ≤2ε
ρ4kfor the Basis Pursuit,
‖x− x‖2 ≤4√k
ρ24k
λ for the Dantzig Selector, and
‖x− x‖2 ≤(1 + κ)
1− κ.
2√k
ρ24k/(1−κ)2
λ for the LASSO estimator.
Computable Bounds 40
![Page 41: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/41.jpg)
CSSIP-INSPIRE
Outline
• Introduction
• Sparsity recovery
• Application example
– Multi-objective optimization of OFDM radar waveform for target detection
• Future work
Computable Bounds 41
![Page 42: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/42.jpg)
CSSIP-INSPIRE
Problem Description
• Goal: Detect a far-field target in the presence of multipath reflections.
• Challenges:
– Complex physical phenomena: multiple reflections, fading effects, etc.
– Lack of line-of-sight (LOS) propagation path to the target.
– Unknown frequency response of the target.
Our Approach
• Employ OFDM signal to increase the frequency diversity and overcome fading.
• Exploit multipath reflections to improve the spatial diversity.
Computable Bounds 42
![Page 43: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/43.jpg)
CSSIP-INSPIRE
Problem Description
Radar
Targets
Buildings
Non-LOS region
Figure 9: Urban multipath scenario.
Computable Bounds 43
![Page 44: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/44.jpg)
CSSIP-INSPIRE
Our Approach (cont.)
• Reformulate the target detection problem as estimating the spectrum of asparse signal by exploiting the sparsity of multiple signal paths and targetvelocity.
• Employ a sparse-recovery algorithmbased on the Dantzig selector (DS)approach and analyze its performancein terms of the l1-constrained minimalsingular value of the measurementmatrix.
• Propose a constrained multi objectiveoptimization (MOO) technique todesign the spectral parameters of theOFDM waveform. Radar
Target A
Reflecting
surface
Reflecting
surface
Target B
Target C
Image of
target A
Image of
target C
Constant
range curve
v v
v
A
B
C
Computable Bounds 44
![Page 45: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/45.jpg)
CSSIP-INSPIRE
Measurement Model
Assumptions:
– Far-field, point target moving with a constant velocity.
– The target remains within a range cell over the coherent processing interval(CPI).
– Fixed, mono-static, and coherent radar.
– The radar knows the geometry of the environment and position of the rangecell.
– The information of the known range cell (τ) is incorporated into the modelby choosing t = τ + nT
PRI, n = 0, 1, . . . , N − 1.
n = 0 n = 1 n = N-1
Coherent processing interval
T
TPRI
Computable Bounds 45
![Page 46: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/46.jpg)
CSSIP-INSPIRE
Measurement Model (cont.)
• Signal model: The complex envelope of the transmitted OFDM signal is
s(t) =
L−1∑l=0
al ej2πl∆ft,
where
– L : number of subcarriers,– a = [a0, a1, . . . , aL−1]
T : complex transmitted weights ensuringaHa = 1 for constant energy transmission,
– ∆f = B/(L+ 1) = 1/T : subcarrier spacing,– B : signal bandwidth,– T : pulse duration.
• Adaptive design: We will select the coefficients al’s to maximize thetarget-detection performance.
Computable Bounds 46
![Page 47: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/47.jpg)
CSSIP-INSPIRE
Measurement Model (cont.)
Consider a target corresponding to the known range cell τ , and the radarreceives information about the target (moving with v) through the path p.Then, the complex envelope of the received signal at the l-th subchannel
yl(n) = al xlp φl(n, p,v) + el(n), for l = 0, . . . , L− 1, n = 0, . . . , N − 1, (1)
where
φl(n, p,v) , e−j2πflτ ej2πflβpnTPRI,
and
– xlp : target scattering coefficient at the l-th subchannel and p-th path,– βp = 2〈v,up〉/c : effective Doppler coefficient along the p-th path,– up : direction-of-arrival (DOA) unit-vector of the p-th path,– c : speed of propagation,– fl = fc + l∆f and fc is the carrier frequency,– el(n) : clutter, measurement noise, and co-channel interference (CCI).
Computable Bounds 47
![Page 48: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/48.jpg)
CSSIP-INSPIRE
Sparse Model
• We discretize the possible signal paths and target velocities into P and V gridpoints, respectively.
• Considering all possible combinations of (pi,vj), i = 1, 2, . . . , P, j =1, 2, . . . , V , we can rewrite (1) as
yl(n) = alφl(n)T xl + el(n),
where
– φl(n) =[φl(n, p1,v1), . . . , φl(n, p1,vV ), φl(n, p2,v1), . . . , φl(n, pP ,vV )
]T,
– xl is a PV × 1 sparse vector, having only kl non-zero entries, where
kl = |Il| : sparsity level of xl,
Il ={i ∈ [1, P ] : pi-th path carries target information
}.
Computable Bounds 48
![Page 49: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/49.jpg)
CSSIP-INSPIRE
Sparse Model (cont.)
Concatenating the measurements of all L subchannels and N time samples
y = Φx+ e,
where
– y =[y(0)T , . . . ,y(N − 1)T
]Tand y(n) = [y0(n), . . . , yL−1(n)]
T ,
– Φ =[(AΦ(0))
T · · · (AΦ(N − 1))T]T
with A = diag(a) and Φ(n) =
blkdiag(φ0(n)T , . . . ,φL−1(n)T
),
– x =[xT0 , . . . ,x
TL−1
]Tis a sparse-vector having k =
∑L−1l=0 kl non-zero
entries,
– e =[e(0)T , . . . , e(N − 1)T
]Tand e(n) = [e0(n), . . . , eL−1(n)]
T .
Thus, Φ contains the response bases corresponding to all possible paths andvelocities.
Computable Bounds 49
![Page 50: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/50.jpg)
CSSIP-INSPIRE
Statistical Model
• The vector e(n) represents the clutter, measurement noise, and co-channelinterference at the output of L subchannels.
• We assume that
– e(n) is temporally white, zero-mean complex Gaussian vector,– co-channel interference among the subchannels is characterized by a
covariance matrix Σ.
• Hence, the measurement vector is distributed as
y ∼ CNLN (Φx, IN ⊗Σ) .
• The identity matrix IN is due to the temporal white noise distribution.
Computable Bounds 50
![Page 51: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/51.jpg)
CSSIP-INSPIRE
Sparse Recovery and Performance
• To obtain an estimate of x (a k-sparse vector) from the noisy measurementsy, obtained through a linear model
y = Φx+ e,
we apply the Dantzig selector (DS).
• Recall that the DS is given by
xDS
= minz∈CLPV
‖z‖1 subject to∥∥ΦH (y −Φz)
∥∥∞ ≤ λ · σ, (2)
where λ =√
2 log(LPV ) is a control parameter and σ =√
tr(Σ)/L.
• To assess the reconstruction performance of this recovery algorithm, we usethe l1-constrained minimal singular value (l1-CMSV) of Φ.
Computable Bounds 51
![Page 52: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/52.jpg)
CSSIP-INSPIRE
Sparse Recovery - Further Simplification• We observe an additional structure in the sparse measurement model, i.e.,
y =[
Φ0 · · · ΦL−1
] x0...
xL−1
+ e,
where– each pair of block-matrices is orthogonal, i.e., ΦH
l1Φl2 = 0 for l1 6= l2,
– each xl, l = 0, 1, . . . , L− 1, is sparse with sparsity level kl.
• To obtain an estimate of x,
x =[xT0 , . . . , x
TL−1
]T,
by exploiting these properties, we employ simpler L decomposed Dantzigselectors.
xDDS,l = minzl∈CPV
‖zl‖1 subject to∥∥ΦH
l (y −Φlzl)∥∥∞ ≤ λl · σ, (3)
where λl =√
2 log(PV ).
Computable Bounds 52
![Page 53: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/53.jpg)
CSSIP-INSPIRE
Performance Analysis
Theorem 7. Consider the estimate x obtained by employing our decomposedDS. An upper bound exists on the `2-norm of the sparse-estimation error
‖xDDS − x‖2 ≤ 4
√√√√L−1∑l=0
λ2l kl σ
2
ρ44kl
(Φl),
whereas using the original DS we get
∥∥xDS− x
∥∥2≤ 4
λ√k σ
ρ24k(Φ)
.
Theorem 8. The L small Dantzig selectors in (3) perform better than theoriginal Dantzig selector in (2) in terms of a smaller upper bound on the `2-norm of the sparse-estimation error:
4
√√√√L−1∑l=0
λ2l kl σ
2
ρ44kl
(Φl)≤ 4
λ√k σ
ρ24k(Φ)
.
Computable Bounds 53
![Page 54: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/54.jpg)
CSSIP-INSPIRE
Adaptive Waveform Design
• We can adaptively design the OFDM spectral parameters, al, to minimize theupper bound on the sparse-estimation error as
a(1) = arg mina∈CL
L−1∑l=0
λ2l kl σ
2
a4l ρ
44kl
(Φl)subject to aHa = 1,
where Φl = alΦl.
• However, the computation of ρ4kl(Φl) is difficult with the complex variables.
Therefore, we use a computable lower bound on ρ4kl(Φl), defined as
ρ8kl(Ψl) ≤ ρ4kl(Φl),
whereΨTl Ψl =
[ΨT
1 Ψ1 + ΨT2 Ψ2 0
0 ΨT1 Ψ1 + ΨT
2 Ψ2
],
Ψ1 = Re Φl, and Ψ2 = Im Φl.
Computable Bounds 54
![Page 55: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/55.jpg)
CSSIP-INSPIRE
Adaptive Waveform Design (cont.)
• Hence, to minimize the upper bound on the sparse-estimation error, weformulate a single-objective optimization problem as
a(1) = arg mina∈CL
L−1∑l=0
λ2l kl σ
2
a4l ρ
48kl
(Ψl)subject to aHa = 1.
• Using the Lagrange-multiplier approach, we easily obtain the solution as
a(1)l =
√√√√ (2αl)1/3∑L−1
l=0 (2αl)1/3, for l = 0, 1, . . . , L− 1,
where αl =λ2l kl σ
2
ρ48kl
(Ψl).
• Note that the designed waveform depends solely on Φ, the measurementmatrix.
Computable Bounds 55
![Page 56: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/56.jpg)
CSSIP-INSPIRE
Adaptive Waveform Design (cont.)
• However, to achieve better detection performance it is also essential that thesignal parameters are adaptive to the target and noise parameters (x and Σ)
• To detect the presence of a target in the range cell under test, a standardprocedure is to construct the following decision problem{
H0 : y = eH1 : y = Φx+ e
,
and to find out whether the measurement y is distributed asCNLN (0, IN ⊗Σ) or CNLN (Φx, IN ⊗Σ).
• To optimize the detection performance, we maximize the squared Mahalanobis-distance between these two distributions
d2 = xH ΦH (IN ⊗Σ)−1
Φx.
Computable Bounds 56
![Page 57: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/57.jpg)
CSSIP-INSPIRE
Adaptive Waveform Design (cont.)
• Hence, in addition to minimizing the upper bound on the estimation error,we propose maximizing another single-objective function based on the squaredMahalanobis-distance (d2) as
a(2) = arg maxa∈CL
[xH ΦH (IN ⊗Σ)
−1Φx
]︸ ︷︷ ︸
, d2
, subject to aHa = 1.
• After some algebraic manipulation, we have
d2 = aH
[N−1∑n=0
(Φ(n)xxH Φ(n)H
)T �Σ−1
]a,
and therefore the solution of the optimization problem, a(2),will be the eigenvector corresponding to the largest eigenvalue of[∑N−1
n=0
(Φ(n)xxH Φ(n)H
)T �Σ−1].
Computable Bounds 57
![Page 58: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/58.jpg)
CSSIP-INSPIRE
Multi-Objective Optimization
• We adaptively design the OFDM spectral parameters, al, using a constrainedmulti-objective optimization (MOO) that simultaneously optimizes twoobjective functions:
– minimize the upper bound on the sparse-estimation error,– maximize the squared Mahalanobis-distance.
Mathematically, this is represented as
aopt =
arg mina∈CL
∑L−1l=0
λ2l kl σ
2
a4lρ4
4kl(Φl)
,
arg maxa∈CL aH[∑N−1
n=0
(Φ(n)xxH Φ(n)H
)T �Σ−1]a
,
subject to aHa = 1.
• We employ the well-known nondominated sorting genetic algorithm II (NSGA-II) to solve our MOO problem, imposing a restriction on the solutions to satisfythe constraint aHa = 1.
Computable Bounds 58
![Page 59: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/59.jpg)
CSSIP-INSPIRE
Numerical Examples
• Problem: Detect a moving target in the presence of multipath in 2D.
• Target and multipath parameters:
– The range cell that is at a distance of 3 kmfrom the radar.
– The target is 13.5 m east from the center line,moving with velocity v = (35/
√2) (i+ j) m/s
and remains within the range cell over a CPI.– There are two actual paths between the
target and radar: one direct and onereflected, subtending angles of 0.26◦ and 0.51◦,respectively, with respect to the radar.
Radar
Reflecting
surface
Reflecting
surface
TargetImage of
target Constant
range curve
v
3 km
13.5 m
40 m
– The scattering coefficients are varied to simulate three target responses:
∗ Target 1: x(1)d = [1, 1, 1]T , x
(1)r = [0.5, 0.5, 0.5]T .
∗ Target 2: x(2)d = [4, 1, 2]T , x
(2)r = [2, 0.5, 1]T .
∗ Target 3: x(3)d = [1, 10, 1]T , x
(3)r = [0.5, 5, 0.5]T .
Computable Bounds 59
![Page 60: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/60.jpg)
CSSIP-INSPIRE
Numerical Examples (cont.)
Radar parameters:
– Carrier frequency fc = 1 GHz– Available bandwidth B = 100 MHz– Number of OFDM subcarriers L = 3– Subcarrier spacing of ∆f = B/(L+ 1) = 25 MHz– Pulse width T = 1/∆f = 40 ns– Pulse repetition interval T
P= 4 ms
– Number of coherent pulses N = 20– All the transmit OFDM weights were equal; i.e., al = 1/
√L ∀ l
Computable Bounds 60
![Page 61: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/61.jpg)
CSSIP-INSPIRE
Numerical Examples (cont.)
Simulation:
– We partition the signal paths and target velocities into P = 5 and V = 3uniform grid points.
– Hence, the associated signal grid paths subtend angles of{−0.5◦,−0.25◦, 0◦, 0.25◦, 0.5◦} with respect to the radar, and target gridvelocities are {25, 35, 45} m/s.
– Note, these grid points are different from the true parameters.
– Generate the noise samples from a CN (0, 1) distribution, and then scale tosatisfy the required target to clutter-plus-noise ratio (TCNR)
TCNR =xH x
N Lσ20
.
Comment: We kept the clutter-plus-noise power to be the same in eachsubcarrier by considering Σ = σ2
0 IL. Hence, TCNR will depend only on thetarget’s RCS at each frequency.
Computable Bounds 61
![Page 62: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/62.jpg)
CSSIP-INSPIRE
Numerical Examples (cont.)
Parameters of NSGA-II:
– Population size = 500– Number of generations = 50– Crossover probability = 0.9– Mutation probability = 0.1– The constraint aHa = 1 is relaxed by ensuring that the solutions satisfy
0.999 ≤ aHa ≤ 1.001
Computable Bounds 62
![Page 63: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/63.jpg)
CSSIP-INSPIRE
Numerical Examples (cont.)
Performance of the standard and Decomposed Dantzig Selectors:Target 1
0 5 10 15 200
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
Target to clutter plus noise ratio (TCNR) (in dB)
Nor
mal
ized
roo
t mea
n sq
ured
err
or
Original DSDecomposed DS
10−1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Probability of false alarm (PFA
)
Pro
babi
lity
of d
etct
ion
(PD)
Original DSDecomposed DS
0 5 10 15 200
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Target to clutter plus noise ratio (TCNR) (in dB)
Com
puta
tion
time
(in s
ec)
Original DSDecomposed DS
Target 2
0 5 10 15 200
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
2
Target to clutter plus noise ratio (TCNR) (in dB)
Nor
mal
ized
roo
t mea
n sq
uare
d er
ror
Original DSDecomposed DS
10−2
10−1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Probability of false alarm (PFA
)
Pro
babi
lity
of d
etec
tion
(PD)
Original DSDecomposed DS
0 5 10 15 200
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Target to clutter plus noise ratio (TCNR) (in dB)
Com
puta
tion
time
(in s
ec)
Original DSDecomposed DS
Normalized RMSE. Empirical ROC. Computation time.
Computable Bounds 63
![Page 64: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/64.jpg)
CSSIP-INSPIRE
Numerical Examples (cont.)
Solutions of the single-objective optimization problems:
– Minimizing the upper bound on the sparse-estimation error we obtaineda(1) = [0.54, 0.16, 0.83]T , irrespective of the target parameters.
– Maximizing the squared Mahalanobis-distance we found
∗ a(2) = [1, 0, 0]T or [0, 1, 0]T or [0, 0, 1]T for Target 1,
∗ a(2) = [1, 0, 0]T for Target 2,
∗ a(2) = [0, 1, 0]T for Target 3.
Observation: The maximization of the squared Mahalanobis-distance providedan adaptive waveform with all the signal energy concentrated over a singlesubcarrier that had the strongest target response, thus losing the frequencydiversity of the system.
Computable Bounds 64
![Page 65: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/65.jpg)
CSSIP-INSPIRE
Numerical Examples (cont.)
Solutions of the NSGA-II (MOO problem) at the 50th generation:
Solutions
0
0.5
1
0
0.5
1
0
0.2
0.4
0.6
0.8
1
|a2||a
1|
|a3|
0
0.5
1
0
0.5
1
0
0.2
0.4
0.6
0.8
1
|a2|
|a1|
|a3|
0
0.5
1
0
0.5
1
0
0.2
0.4
0.6
0.8
1
|a2||a
1|
|a3|
Pareto-front
0 2 4 6 8 10
x 109
50
100
150
200
250
300
350
Squared upper bound on sparse−estimation error
Squ
ared
Mah
alan
obis
dis
tanc
e
0 2 4 6 8 10
x 108
140
160
180
200
220
240
260
Squared upper bound on sparse−estimation error
Squ
ared
Mah
alan
obis
dis
tanc
e
0 1 2 3 4 5
x 109
200
300
400
500
600
700
800
900
1000
1100
Squared upper bound on sparse−estimation error
Squ
ared
Mah
alan
obis
dis
tanc
e
Target 1 Target 2 Target 3
Computable Bounds 65
![Page 66: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/66.jpg)
CSSIP-INSPIRE
Numerical Examples (cont.)
Effect of target scattering coefficent on the NSGA-II solutions:
We averaged the whole population of 500 solutions and found
– aopt,avg = [0.61, 0.39, 0.68]T for Target 1,
– aopt,avg = [0.88, 0.20, 0.36]T for Target 2,
– aopt,avg = [0.13, 0.96, 0.15]T for Target 3.
Observation: The solution of the MOO distributes the energy of the optimalwaveform across different subcarriers in proportion to the distribution of thetarget energy; i.e., it puts more signal energy into that particular subcarrier inwhich the target response is stronger.
Computable Bounds 66
![Page 67: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/67.jpg)
CSSIP-INSPIRE
Numerical Examples (cont.)
Performance improvement due to adaptive waveform design:
0 2 4 6 8 10 12 14 16 18 200
0.5
1
1.5
2
Target to clutter plus noise ratio (TCNR) (in dB)
Nor
mal
ized
roo
t mea
n sq
uare
d er
ror
Fixed waveforml1−CMSV minimized adaptive waveform
NSGA−II optimized adaptive waveform
10−2
10−1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Probability of false alarm (PFA
)
Pro
babi
lity
of d
etec
tion
(PD
)
Fixed waveforml1−CMSV minimized adaptive waveform
NSGA−II optimized adaptive waveform
RMSE ROC
Computable Bounds 67
![Page 68: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/68.jpg)
CSSIP-INSPIRE
Outline
• Introduction
• Sparsity recovery
• Application example
• Future work
Computable Bounds 68
![Page 69: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/69.jpg)
CSSIP-INSPIRE
Open Problems
• Develop computable tight bounds.
• Bounds for other signal structures, such as block-sparse vectors and low-rankmatrices.
• Procedures for optimization of system design.
• Computationally efficient algorithms for sensing matrices other than Fourier orHadamard.
• Minimization of modeling errors due to discrete grid mismatch.
Computable Bounds 69
![Page 70: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/70.jpg)
CSSIP-INSPIRE
Possible Approaches
• Develop computable tighter bounds that control the average or typical systemperformance.
• Encode more signal structures into the model rather than using bounds, forexample using the continuous-parameter domain.
Computable Bounds 70
![Page 71: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/71.jpg)
CSSIP-INSPIRE
References
• G. Tang and A. Nehorai, “Verifiable and computable performance analysis ofsparsity recovery,” submitted for publication.
• G. Tang and A. Nehorai, “Performance analysis of sparse recovery based onconstrained minimal singular values,” IEEE Trans. Signal Processing, vol. 59,no. 12, pp. 5734-5745, Dec. 2011.
• G. Tang and A. Nehorai, “Fixed point theory and semidefinite programmingfor computable performance analysis of block-sparsity recovery,” submitted forpublication.
• S. Sen, G. Tang, and A. Nehorai, “Multiobjective optimization of OFDM radarwaveform for target detection,” IEEE Trans. on Signal Processing, Vol. 59,pp. 639-652, Feb. 2011.
Computable Bounds 71
![Page 72: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/72.jpg)
CSSIP-INSPIRE
Questions ?
Computable Bounds 72
![Page 73: Computable Performance Analysis of Sparsity Recovery with ...nehorai/research/sparse/Plenary_EUSIP… · CSSIP-INSPIRE Sparsity Example: Compressive Sensing (cont.) Figure 1: Paradigm](https://reader033.vdocuments.mx/reader033/viewer/2022051923/6010b369e339df7d69700d7a/html5/thumbnails/73.jpg)
CSSIP-INSPIRE
Thank You !
Computable Bounds 73