spatial and temporal information fusion based on laplace ...optimization/l1/optseminar... · etm...
TRANSCRIPT
![Page 1: Spatial and Temporal Information Fusion Based on Laplace ...optimization/L1/optseminar... · ETM Aug 20 th, 2001. Original Our Method STARFM. Future work Improve prediction accuracy](https://reader033.vdocuments.mx/reader033/viewer/2022050418/5f8dee162f512e440c771d73/html5/thumbnails/1.jpg)
Spatial and Temporal Information Spatial and Temporal Information Fusion Based on Laplace Pyramid Fusion Based on Laplace Pyramid
Peng, Zhimin
Advisor: Prof. Huang, Bo (CUHK)
Dr. Meng, Deyu (XJTU)
![Page 2: Spatial and Temporal Information Fusion Based on Laplace ...optimization/L1/optseminar... · ETM Aug 20 th, 2001. Original Our Method STARFM. Future work Improve prediction accuracy](https://reader033.vdocuments.mx/reader033/viewer/2022050418/5f8dee162f512e440c771d73/html5/thumbnails/2.jpg)
Outline:Outline:
� Problem
� Existing Method
� Framework
Results� Results
� Future work
![Page 3: Spatial and Temporal Information Fusion Based on Laplace ...optimization/L1/optseminar... · ETM Aug 20 th, 2001. Original Our Method STARFM. Future work Improve prediction accuracy](https://reader033.vdocuments.mx/reader033/viewer/2022050418/5f8dee162f512e440c771d73/html5/thumbnails/3.jpg)
Viewing the entire Earth's surface every 1 to 2 days (high temporal resolution)
Obtain image with low spatial resolution
(250m)
TERRA
MODIS
(250m)
![Page 4: Spatial and Temporal Information Fusion Based on Laplace ...optimization/L1/optseminar... · ETM Aug 20 th, 2001. Original Our Method STARFM. Future work Improve prediction accuracy](https://reader033.vdocuments.mx/reader033/viewer/2022050418/5f8dee162f512e440c771d73/html5/thumbnails/4.jpg)
Viewing the entire Earth's surface every 16 days, (low temporalresolution)
Obtain image with high spatial resolution
(30m)
LANDSAT
ETM
(30m)
![Page 5: Spatial and Temporal Information Fusion Based on Laplace ...optimization/L1/optseminar... · ETM Aug 20 th, 2001. Original Our Method STARFM. Future work Improve prediction accuracy](https://reader033.vdocuments.mx/reader033/viewer/2022050418/5f8dee162f512e440c771d73/html5/thumbnails/5.jpg)
Crop
High Spatial and High Temporal Crop
Growing Desertification UrbanizationHigh Temporal
resolution remote sensing data
![Page 6: Spatial and Temporal Information Fusion Based on Laplace ...optimization/L1/optseminar... · ETM Aug 20 th, 2001. Original Our Method STARFM. Future work Improve prediction accuracy](https://reader033.vdocuments.mx/reader033/viewer/2022050418/5f8dee162f512e440c771d73/html5/thumbnails/6.jpg)
Problem:Problem:
A B
A’ B’
![Page 7: Spatial and Temporal Information Fusion Based on Laplace ...optimization/L1/optseminar... · ETM Aug 20 th, 2001. Original Our Method STARFM. Future work Improve prediction accuracy](https://reader033.vdocuments.mx/reader033/viewer/2022050418/5f8dee162f512e440c771d73/html5/thumbnails/7.jpg)
Existing MethodsExisting Methods
� STARFM (Gao, Masek, et al. 06)
![Page 8: Spatial and Temporal Information Fusion Based on Laplace ...optimization/L1/optseminar... · ETM Aug 20 th, 2001. Original Our Method STARFM. Future work Improve prediction accuracy](https://reader033.vdocuments.mx/reader033/viewer/2022050418/5f8dee162f512e440c771d73/html5/thumbnails/8.jpg)
FrameworkFramework
![Page 9: Spatial and Temporal Information Fusion Based on Laplace ...optimization/L1/optseminar... · ETM Aug 20 th, 2001. Original Our Method STARFM. Future work Improve prediction accuracy](https://reader033.vdocuments.mx/reader033/viewer/2022050418/5f8dee162f512e440c771d73/html5/thumbnails/9.jpg)
Step 1:Step 1:
Laplace Pyramid Laplace Pyramid Laplace Pyramid Laplace Pyramid DecompositionDecomposition
![Page 10: Spatial and Temporal Information Fusion Based on Laplace ...optimization/L1/optseminar... · ETM Aug 20 th, 2001. Original Our Method STARFM. Future work Improve prediction accuracy](https://reader033.vdocuments.mx/reader033/viewer/2022050418/5f8dee162f512e440c771d73/html5/thumbnails/10.jpg)
Laplace Pyramid(LP)Laplace Pyramid(LP)
� Introduced by Burt and Adelson in 1983
� Function:
Each level represents a different level band of spatial frequenciesband of spatial frequencies
� Basic idea:
Step 1: Build a Gaussian Pyramid
Step 2: Take the difference between one
Gaussian Pyramid level and the next
![Page 11: Spatial and Temporal Information Fusion Based on Laplace ...optimization/L1/optseminar... · ETM Aug 20 th, 2001. Original Our Method STARFM. Future work Improve prediction accuracy](https://reader033.vdocuments.mx/reader033/viewer/2022050418/5f8dee162f512e440c771d73/html5/thumbnails/11.jpg)
Gaussian PyramidGaussian Pyramid
![Page 12: Spatial and Temporal Information Fusion Based on Laplace ...optimization/L1/optseminar... · ETM Aug 20 th, 2001. Original Our Method STARFM. Future work Improve prediction accuracy](https://reader033.vdocuments.mx/reader033/viewer/2022050418/5f8dee162f512e440c771d73/html5/thumbnails/12.jpg)
Laplace PyramidLaplace Pyramid
![Page 13: Spatial and Temporal Information Fusion Based on Laplace ...optimization/L1/optseminar... · ETM Aug 20 th, 2001. Original Our Method STARFM. Future work Improve prediction accuracy](https://reader033.vdocuments.mx/reader033/viewer/2022050418/5f8dee162f512e440c771d73/html5/thumbnails/13.jpg)
Step 2: Step 2:
Match FunctionMatch FunctionMatch FunctionMatch Function
![Page 14: Spatial and Temporal Information Fusion Based on Laplace ...optimization/L1/optseminar... · ETM Aug 20 th, 2001. Original Our Method STARFM. Future work Improve prediction accuracy](https://reader033.vdocuments.mx/reader033/viewer/2022050418/5f8dee162f512e440c771d73/html5/thumbnails/14.jpg)
Match FunctionMatch Function
Level i
iM'
iM
Image Analogies , Aaron Hertzmann, etc, 2002
Level i
iE'
iE
),,( ''
iiii MEEionMatchFunctE =
![Page 15: Spatial and Temporal Information Fusion Based on Laplace ...optimization/L1/optseminar... · ETM Aug 20 th, 2001. Original Our Method STARFM. Future work Improve prediction accuracy](https://reader033.vdocuments.mx/reader033/viewer/2022050418/5f8dee162f512e440c771d73/html5/thumbnails/15.jpg)
Match FunctionMatch Function
� Scheme 1:
� — Block by Block Regression
� Scheme 2:� Scheme 2:
� — Markov Network
![Page 16: Spatial and Temporal Information Fusion Based on Laplace ...optimization/L1/optseminar... · ETM Aug 20 th, 2001. Original Our Method STARFM. Future work Improve prediction accuracy](https://reader033.vdocuments.mx/reader033/viewer/2022050418/5f8dee162f512e440c771d73/html5/thumbnails/16.jpg)
Block by Block RegressionBlock by Block Regression
� Assumption:
� Patches of the same location satisfy linear relationship
� i.e.
Combine the above two equations
![Page 17: Spatial and Temporal Information Fusion Based on Laplace ...optimization/L1/optseminar... · ETM Aug 20 th, 2001. Original Our Method STARFM. Future work Improve prediction accuracy](https://reader033.vdocuments.mx/reader033/viewer/2022050418/5f8dee162f512e440c771d73/html5/thumbnails/17.jpg)
Markov NetworkMarkov Network
iM
iE
'M
Dictionary Preparation:
'
iM
'
iE Candidates
![Page 18: Spatial and Temporal Information Fusion Based on Laplace ...optimization/L1/optseminar... · ETM Aug 20 th, 2001. Original Our Method STARFM. Future work Improve prediction accuracy](https://reader033.vdocuments.mx/reader033/viewer/2022050418/5f8dee162f512e440c771d73/html5/thumbnails/18.jpg)
Markov NetworkMarkov Network
Belief Propagation
![Page 19: Spatial and Temporal Information Fusion Based on Laplace ...optimization/L1/optseminar... · ETM Aug 20 th, 2001. Original Our Method STARFM. Future work Improve prediction accuracy](https://reader033.vdocuments.mx/reader033/viewer/2022050418/5f8dee162f512e440c771d73/html5/thumbnails/19.jpg)
Step 3: Step 3:
ReconstructionReconstructionReconstructionReconstruction
![Page 20: Spatial and Temporal Information Fusion Based on Laplace ...optimization/L1/optseminar... · ETM Aug 20 th, 2001. Original Our Method STARFM. Future work Improve prediction accuracy](https://reader033.vdocuments.mx/reader033/viewer/2022050418/5f8dee162f512e440c771d73/html5/thumbnails/20.jpg)
ReconstructionReconstruction
� Usual Reconstruction :
Simply added back the different frequency part
from coarser to finer
Sensitive to noiseSensitive to noise
� New Reconstruction:
Frame Theory (Minh N. Do 2003)
Robust
![Page 21: Spatial and Temporal Information Fusion Based on Laplace ...optimization/L1/optseminar... · ETM Aug 20 th, 2001. Original Our Method STARFM. Future work Improve prediction accuracy](https://reader033.vdocuments.mx/reader033/viewer/2022050418/5f8dee162f512e440c771d73/html5/thumbnails/21.jpg)
Usual ReconstructionUsual Reconstruction
� Decomposition
� Reconstruction
Noise Pyramid: y y e= +ɶ
![Page 22: Spatial and Temporal Information Fusion Based on Laplace ...optimization/L1/optseminar... · ETM Aug 20 th, 2001. Original Our Method STARFM. Future work Improve prediction accuracy](https://reader033.vdocuments.mx/reader033/viewer/2022050418/5f8dee162f512e440c771d73/html5/thumbnails/22.jpg)
New ReconstructionNew Reconstruction
2( ) ( )
TG H
GH GH
=
=
† 1( )T TS A A A A−
= =
Computational Expensive
TA A I=( ) ( )GH GH=
†ˆ ( )x A y G c Hd d= = − +
![Page 23: Spatial and Temporal Information Fusion Based on Laplace ...optimization/L1/optseminar... · ETM Aug 20 th, 2001. Original Our Method STARFM. Future work Improve prediction accuracy](https://reader033.vdocuments.mx/reader033/viewer/2022050418/5f8dee162f512e440c771d73/html5/thumbnails/23.jpg)
ComparisonComparison
![Page 24: Spatial and Temporal Information Fusion Based on Laplace ...optimization/L1/optseminar... · ETM Aug 20 th, 2001. Original Our Method STARFM. Future work Improve prediction accuracy](https://reader033.vdocuments.mx/reader033/viewer/2022050418/5f8dee162f512e440c771d73/html5/thumbnails/24.jpg)
Landscape change monitoringLandscape change monitoring
MO
DIS N
ov 1
2, 2
00
0
MO
DIS N
ov 1
2, 2
00
2
MO
DIS N
ov 1
2, 2
00
0
MO
DIS N
ov 1
2, 2
00
2
ET
M N
ov 1
2, 2
00
0
ET
M N
ov 1
2, 2
00
0
![Page 25: Spatial and Temporal Information Fusion Based on Laplace ...optimization/L1/optseminar... · ETM Aug 20 th, 2001. Original Our Method STARFM. Future work Improve prediction accuracy](https://reader033.vdocuments.mx/reader033/viewer/2022050418/5f8dee162f512e440c771d73/html5/thumbnails/25.jpg)
Real ETM STARFM
Markov Network Regression
![Page 26: Spatial and Temporal Information Fusion Based on Laplace ...optimization/L1/optseminar... · ETM Aug 20 th, 2001. Original Our Method STARFM. Future work Improve prediction accuracy](https://reader033.vdocuments.mx/reader033/viewer/2022050418/5f8dee162f512e440c771d73/html5/thumbnails/26.jpg)
Scatter graphScatter graph
Our Method STARFM
![Page 27: Spatial and Temporal Information Fusion Based on Laplace ...optimization/L1/optseminar... · ETM Aug 20 th, 2001. Original Our Method STARFM. Future work Improve prediction accuracy](https://reader033.vdocuments.mx/reader033/viewer/2022050418/5f8dee162f512e440c771d73/html5/thumbnails/27.jpg)
Seasonal ChangeSeasonal Change
MO
DIS M
ay 20
th, 20
01
MO
DIS A
ug 2
0th, 2
00
1
, 20
01
ET
M M
ay 20
th, 20
01
ET
M A
ug 2
0th, 2
00
1
![Page 28: Spatial and Temporal Information Fusion Based on Laplace ...optimization/L1/optseminar... · ETM Aug 20 th, 2001. Original Our Method STARFM. Future work Improve prediction accuracy](https://reader033.vdocuments.mx/reader033/viewer/2022050418/5f8dee162f512e440c771d73/html5/thumbnails/28.jpg)
Original Our Method STARFM
![Page 29: Spatial and Temporal Information Fusion Based on Laplace ...optimization/L1/optseminar... · ETM Aug 20 th, 2001. Original Our Method STARFM. Future work Improve prediction accuracy](https://reader033.vdocuments.mx/reader033/viewer/2022050418/5f8dee162f512e440c771d73/html5/thumbnails/29.jpg)
![Page 30: Spatial and Temporal Information Fusion Based on Laplace ...optimization/L1/optseminar... · ETM Aug 20 th, 2001. Original Our Method STARFM. Future work Improve prediction accuracy](https://reader033.vdocuments.mx/reader033/viewer/2022050418/5f8dee162f512e440c771d73/html5/thumbnails/30.jpg)
Future workFuture work
� Improve prediction accuracy
� Retain detail information
— Compressive Sensing
— Matrix Completion— Matrix Completion
� Explore information contained in other band
![Page 31: Spatial and Temporal Information Fusion Based on Laplace ...optimization/L1/optseminar... · ETM Aug 20 th, 2001. Original Our Method STARFM. Future work Improve prediction accuracy](https://reader033.vdocuments.mx/reader033/viewer/2022050418/5f8dee162f512e440c771d73/html5/thumbnails/31.jpg)
Thank you !Thank you !Thank you !Thank you !