approximate message passing in coded aperture snapshot ......approximate message passing (amp) g =...
TRANSCRIPT
![Page 1: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/1.jpg)
Approximate Message Passing in Coded Aperture Snapshot Spectral Imaging
Jin Tan, Yanting Ma, Hoover Rueda,Dror Baron, and Gonzalo R. Arce
Orlando, FLDecember 15, 2015
Supported by NSF and ARO
![Page 2: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/2.jpg)
RGB image
Image slices at different wavelengths
Hyperspectral Images
3D cube
![Page 3: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/3.jpg)
Hyperspectral Images
• Obtain spectrum information of a scene
• Applications include
• Medical imaging
• Geology
• Astronomy
• Remote sensing
![Page 4: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/4.jpg)
Conventional Hyperspectral Imaging
• Acquire and store entire image in all spectrum bands
• Disadvantages• Long imaging time• Large storage
![Page 5: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/5.jpg)
Conventional Hyperspectral Imaging
• Acquire and store entire image in all spectrum bands
• Disadvantages• Long imaging time• Large storage
Better imaging system?
![Page 6: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/6.jpg)
Compressive Hyperspectral Imaging
![Page 7: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/7.jpg)
Coded Aperture Snapshot Spectral Imaging (CASSI) [Wagadarikar et al. 2008]
![Page 8: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/8.jpg)
+=
vectorized3D-cube noise
data on focal plane CASSI
g H (m × n) zf0
Compressive Sensing Formulation
g = Hf0 + z
![Page 9: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/9.jpg)
[Arguello et al. 2013]Higher Order CASSI
measurement rate ≈ 2/L, L: #spectrum bands
![Page 10: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/10.jpg)
Challenges
• Highly compressed measurements
• Structured sensing matrix
• Large signal dimension (need fast algorithm)
![Page 11: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/11.jpg)
Challenges
• Highly compressed measurements
• Structured sensing matrix
• Large signal dimension (need fast algorithm)
no parameter tuning runtime
![Page 12: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/12.jpg)
Approximate Message Passing[Donoho et al. 2009]
![Page 13: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/13.jpg)
Approximate Message Passing (AMP)
g = Hf0 + z ∈ ℝm qt = f0 + vt ∈ ℝncompressive sensing denoising
Pseudo-data(noisy data)
![Page 14: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/14.jpg)
Approximate Message Passing (AMP)
g = Hf0 + z ∈ ℝm qt = f0 + vt ∈ ℝncompressive sensing denoising
• Noise vt uncorrelated with input f0• Noise vt distributed as i.i.d. Gaussian 𝒩𝒩(0,σt2)
• Noise variance σt2 can be accurately estimated
• Convergence theoretically characterized
If sensing matrix H is i.i.d. Gaussian, asymptotically
![Page 15: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/15.jpg)
Approximate Message Passing (AMP)
g = Hf0 + z ∈ ℝm qt = f0 + vt ∈ ℝncompressive sensing denoising
• Noise vt uncorrelated with input f0• Noise vt distributed as i.i.d. Gaussian 𝒩𝒩(0,σt2)
• Noise variance σt2 can be accurately estimated
• Convergence theoretically characterized
If sensing matrix H is i.i.d. Gaussian, asymptotically
May break down for structured matrix!
![Page 16: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/16.jpg)
AMP PseudocodeInitialize f t=0 ← 𝟎𝟎At iteration 𝑡𝑡, do
Residual: rt ← g − Hf t +rt−1
m/n η′t−1(f t−1 + HTrt−1)
![Page 17: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/17.jpg)
AMP PseudocodeInitialize f t=0 ← 𝟎𝟎At iteration 𝑡𝑡, do
Residual: rt ← g − Hf t +rt−1
m/n η′t−1(f t−1 + HTrt−1)
Onsager correction
![Page 18: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/18.jpg)
AMP PseudocodeInitialize f t=0 ← 𝟎𝟎At iteration 𝑡𝑡, do
Residual:
Noisy signal:
Noise (vt) level:
Denoising:
rt ← g − Hf t +rt−1
m/n η′t−1(f t−1 + HTrt−1)
qt ← f t + HTrt
σt2 ← ||rt||22/m
f t+1 ← ηt(qt;σt2)
(= f0 + vt)
𝒩𝒩(0,σt2)
for i.i.d. Gaussian H
![Page 19: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/19.jpg)
Initialize f t=0 ← 𝟎𝟎At iteration 𝑡𝑡, do
Residual:
Noisy signal:
Noise (vt) level:
Denoising:
rt ← g − Hf t +rt−1
m/n η′t−1(f t−1 + HTrt−1)
qt ← f t + HTrt
σt2 ← ||rt||22/m
f t+1 ← ηt(qt;σt2)
AMP for Hyperspectral Image Recovery
3D image denoiser
![Page 20: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/20.jpg)
• 2D wavelet + 1D discrete cosine transform (DCT)
Denoising (3D-Wiener)
qt = f0 + vt θqt = θf0 + θvt
2D wavelet 1D DCT
![Page 21: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/21.jpg)
• 2D wavelet + 1D discrete cosine transform (DCT)
• Assume Var(θv,it ) = σt2(= ||rt||/m), i = 1, . . , n
Denoising (3D-Wiener)
qt = f0 + vt θqt = θf0 + θvt
2D wavelet 1D DCT
![Page 22: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/22.jpg)
• 2D wavelet + 1D discrete cosine transform (DCT)
• Assume Var(θv,it ) = σt2(= ||rt||/m), i = 1, . . , n
• Empirical variance σi2 and mean µi of θf0,i estimated
using θqt in wavelet subband
Denoising (3D-Wiener)
qt = f0 + vt θqt = θf0 + θvt
2D wavelet 1D DCT
![Page 23: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/23.jpg)
• 2D wavelet + 1D discrete cosine transform (DCT)
• Assume Var(θv,it ) = σt2(= ||rt||/m), i = 1, . . , n
• Empirical variance σi2 and mean µi of θf0,i estimated
using θqt in wavelet subband
• Wiener filter: θf,it+1 = σi2
σi2+σt2
⋅ θq,it − µi + µi
Denoising (3D-Wiener)
qt = f0 + vt θqt = θf0 + θvt
2D wavelet 1D DCT
![Page 24: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/24.jpg)
Divergence Problem
• Structured matrix H• Inaccurate model assumption in denoising problem
Iteration t
f10 f11 f12diverged
![Page 25: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/25.jpg)
Initialize f0 ← 𝟎𝟎At iteration t, do
Residual:
Noisy image:
Noise level:
Denoising:
Damping:
rt ← g − Hf t +rt−1
m/n η′t−1(f t−1 + HTrt−1)
qt ← f t + HTrt
σt2 ← ||rt||22/m
f t+1 ← ηt(qt;σt2)
AMP-3D-Wiener
f t+1 ← 𝛼𝛼 ⋅ f t+1 + 1 − 𝛼𝛼 ⋅ f t(0 < 𝛼𝛼 ≤ 1)
3D-Wiener
![Page 26: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/26.jpg)
Numerical Results
![Page 27: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/27.jpg)
Original Iteration 1
Numerical Results
Lego toy example
![Page 28: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/28.jpg)
Original Iteration 4
Numerical Results
Lego toy example
![Page 29: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/29.jpg)
Original Iteration 7
Numerical Results
Lego toy example
![Page 30: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/30.jpg)
Original Iteration 10
Numerical Results
Lego toy example
![Page 31: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/31.jpg)
Original Iteration 20
Numerical Results
Lego toy example
![Page 32: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/32.jpg)
Original Iteration 50
Numerical Results
Lego toy example
![Page 33: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/33.jpg)
Original Iteration 100
Numerical Results
Lego toy example
![Page 34: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/34.jpg)
Original Iteration 400
Numerical Results
Lego toy example
![Page 35: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/35.jpg)
• Lego toy example• 2 shots; complementary coded aperture; 20dB noise• No parameter tuning for AMP-3D-Wiener
TwIST [Bioucas-Dias & Figueiredo 2007, Wagadarikar 2008]GPSR [Figueiredo et al. 2007]
Numerical Results
![Page 36: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/36.jpg)
Numerical Results
• Lego toy example• 2 shots; complementary coded aperture; 20dB noise
TwIST [Bioucas-Dias & Figueiredo 2007, Wagadarikar 2008]GPSR [Figueiredo et al. 2007]
![Page 37: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/37.jpg)
Numerical Results
• Lego toy example• 2-12 shots; complementary coded aperture; 20dB noise
TwIST [Bioucas-Dias & Figueiredo 2007, Wagadarikar 2008]GPSR [Figueiredo et al. 2007]
![Page 38: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/38.jpg)
Natural scenes [personalpages.manchester.ac.uk/staff/d.h.foster/]
AlgorithmSNR
Numerical Results
AMP reconstructs better in all tested scenes.
TwIST [Bioucas-Dias & Figueiredo 2007, Wagadarikar 2008]GPSR [Figueiredo et al. 2007]
![Page 39: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/39.jpg)
Summary
![Page 40: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/40.jpg)
Summary
• Problem: Hyperspectral image reconstruction in CASSI
• Algorithm: Approximate message passing with adaptive Wiener filter in 2D wavelet + 1D DCT domain
• Challenges:• Highly compressed measurements• Structured sensing matrix
• Results: • Improved PSNR and runtime• No parameter tuning
![Page 41: Approximate Message Passing in Coded Aperture Snapshot ......Approximate Message Passing (AMP) g = Hf 0 +z ∈ℝ m q t = f 0 +v t ∈ℝ n compressive sensing denoising • Noise](https://reader036.vdocuments.mx/reader036/viewer/2022071421/611b686230e92b30493e7754/html5/thumbnails/41.jpg)
Thank you!