high-resolution hyperspectral imaging via matrix factorization
DESCRIPTION
High-resolution Hyperspectral Imaging via Matrix Factorization. Rei Kawakami 1 John Wright 2 Yu-Wing Tai 3 Yasuyuki Matsushita 2 Moshe Ben-Ezra 2 Katsushi Ikeuchi 3 1 University of Tokyo, 2 Microsoft Research Asia (MSRA), 3 Korea Advanced Institute of Science and Technology (KAIST) - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: High-resolution Hyperspectral Imaging via Matrix Factorization](https://reader034.vdocuments.mx/reader034/viewer/2022051518/56816027550346895dcf2c3b/html5/thumbnails/1.jpg)
High-resolution Hyperspectral Imaging via Matrix Factorization
Rei Kawakami1 John Wright2 Yu-Wing Tai3 Yasuyuki Matsushita2 Moshe Ben-Ezra2 Katsushi Ikeuchi3
1University of Tokyo, 2Microsoft Research Asia (MSRA),
3Korea Advanced Institute of Science and Technology (KAIST)
CVPR 11
![Page 2: High-resolution Hyperspectral Imaging via Matrix Factorization](https://reader034.vdocuments.mx/reader034/viewer/2022051518/56816027550346895dcf2c3b/html5/thumbnails/2.jpg)
Giga-pixel Camera
M. Ezra et al.Giga-pixel Camera
@ Microsoft research
Large-format lens CCD
![Page 3: High-resolution Hyperspectral Imaging via Matrix Factorization](https://reader034.vdocuments.mx/reader034/viewer/2022051518/56816027550346895dcf2c3b/html5/thumbnails/3.jpg)
Spectral cameras
LCTF filter35,000 $
Hyper-spectral camera55,000 $
Line spectral scanner25,000 $
• Expensive• Low resolution
![Page 4: High-resolution Hyperspectral Imaging via Matrix Factorization](https://reader034.vdocuments.mx/reader034/viewer/2022051518/56816027550346895dcf2c3b/html5/thumbnails/4.jpg)
Approach
Low-reshyperspectral
high-resRGB
High-reshyperspectral image
Combine
![Page 5: High-resolution Hyperspectral Imaging via Matrix Factorization](https://reader034.vdocuments.mx/reader034/viewer/2022051518/56816027550346895dcf2c3b/html5/thumbnails/5.jpg)
Problem formulation
W(Image width)
H(Image height)
S
Goal:
Given:
![Page 6: High-resolution Hyperspectral Imaging via Matrix Factorization](https://reader034.vdocuments.mx/reader034/viewer/2022051518/56816027550346895dcf2c3b/html5/thumbnails/6.jpg)
A1: Limited number of materials
•
Sparse vector
For all pixel (i,j)
Sparse matrix
W (Image width)
H (Image height)
S
= …
00.40…
0.6
![Page 7: High-resolution Hyperspectral Imaging via Matrix Factorization](https://reader034.vdocuments.mx/reader034/viewer/2022051518/56816027550346895dcf2c3b/html5/thumbnails/7.jpg)
Sampling of each camera
• Low-res camera • RGB camera
Spectrum
Wavelength
Intensity
![Page 8: High-resolution Hyperspectral Imaging via Matrix Factorization](https://reader034.vdocuments.mx/reader034/viewer/2022051518/56816027550346895dcf2c3b/html5/thumbnails/8.jpg)
Sparse signal recovery
• •
Filter SignalObservation
t
tm
n
![Page 9: High-resolution Hyperspectral Imaging via Matrix Factorization](https://reader034.vdocuments.mx/reader034/viewer/2022051518/56816027550346895dcf2c3b/html5/thumbnails/9.jpg)
Sparsity
Signal Basis Weights
Signal Basis Weights
0
S
S-sparse
![Page 10: High-resolution Hyperspectral Imaging via Matrix Factorization](https://reader034.vdocuments.mx/reader034/viewer/2022051518/56816027550346895dcf2c3b/html5/thumbnails/10.jpg)
Sparse signal recovery
Observation
Need not to know which bases are important
Sparsity and Incoherence matters
![Page 11: High-resolution Hyperspectral Imaging via Matrix Factorization](https://reader034.vdocuments.mx/reader034/viewer/2022051518/56816027550346895dcf2c3b/html5/thumbnails/11.jpg)
A2: Sparsity in high-res image
W (Image width)
H (Image height)
S
Sparse coefficients
Sparse vector
![Page 12: High-resolution Hyperspectral Imaging via Matrix Factorization](https://reader034.vdocuments.mx/reader034/viewer/2022051518/56816027550346895dcf2c3b/html5/thumbnails/12.jpg)
Simulation experiments
![Page 13: High-resolution Hyperspectral Imaging via Matrix Factorization](https://reader034.vdocuments.mx/reader034/viewer/2022051518/56816027550346895dcf2c3b/html5/thumbnails/13.jpg)
460 nm 550 nm 620 nm 460 nm 550 nm 620 nm
![Page 14: High-resolution Hyperspectral Imaging via Matrix Factorization](https://reader034.vdocuments.mx/reader034/viewer/2022051518/56816027550346895dcf2c3b/html5/thumbnails/14.jpg)
430 nm 490 nm 550 nm 610 nm 670 nm
![Page 15: High-resolution Hyperspectral Imaging via Matrix Factorization](https://reader034.vdocuments.mx/reader034/viewer/2022051518/56816027550346895dcf2c3b/html5/thumbnails/15.jpg)
Error images of Global PCA with back-projection
Error images of local window with back-projection
Error images of RGB clustering with back-projection
![Page 16: High-resolution Hyperspectral Imaging via Matrix Factorization](https://reader034.vdocuments.mx/reader034/viewer/2022051518/56816027550346895dcf2c3b/html5/thumbnails/16.jpg)
Estimated430 nm
![Page 17: High-resolution Hyperspectral Imaging via Matrix Factorization](https://reader034.vdocuments.mx/reader034/viewer/2022051518/56816027550346895dcf2c3b/html5/thumbnails/17.jpg)
Groundtruth
![Page 18: High-resolution Hyperspectral Imaging via Matrix Factorization](https://reader034.vdocuments.mx/reader034/viewer/2022051518/56816027550346895dcf2c3b/html5/thumbnails/18.jpg)
RGBimage
![Page 19: High-resolution Hyperspectral Imaging via Matrix Factorization](https://reader034.vdocuments.mx/reader034/viewer/2022051518/56816027550346895dcf2c3b/html5/thumbnails/19.jpg)
Errorimage
![Page 20: High-resolution Hyperspectral Imaging via Matrix Factorization](https://reader034.vdocuments.mx/reader034/viewer/2022051518/56816027550346895dcf2c3b/html5/thumbnails/20.jpg)
![Page 21: High-resolution Hyperspectral Imaging via Matrix Factorization](https://reader034.vdocuments.mx/reader034/viewer/2022051518/56816027550346895dcf2c3b/html5/thumbnails/21.jpg)
![Page 22: High-resolution Hyperspectral Imaging via Matrix Factorization](https://reader034.vdocuments.mx/reader034/viewer/2022051518/56816027550346895dcf2c3b/html5/thumbnails/22.jpg)
HS camera
Filter
CMOSLensAperture
Focus
Translational stage
![Page 23: High-resolution Hyperspectral Imaging via Matrix Factorization](https://reader034.vdocuments.mx/reader034/viewer/2022051518/56816027550346895dcf2c3b/html5/thumbnails/23.jpg)
Real data experiment
Input RGB Input (550nm) Input (620nm)Estimated (550nm) Estimated (620nm)
![Page 24: High-resolution Hyperspectral Imaging via Matrix Factorization](https://reader034.vdocuments.mx/reader034/viewer/2022051518/56816027550346895dcf2c3b/html5/thumbnails/24.jpg)
Summary
• Method to reconstruct high-resolution hyperspectral image from – Low-res hyperspectral camera– High-res RGB camera
• Spatial sparsity of hyperspectral input– Search for a factorization of the input into
• basis • set of maximally sparse coefficients.