uwphysret an ssec inversion package for high resolution ... · paolo antonelli, r. knuteson, r....
TRANSCRIPT
![Page 1: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/1.jpg)
Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor,
W. Smith, and H. Revercomb
UWPHYSRET an SSEC inversion package for high resolution infrared data based on LBLRTM.
4th Advanced High Spectral Resolution Infrared ObservationsEUMETSAT - Darmstadt, Germany
15-17 September, 2008
QuickTime™ and a decompressor
are needed to see this picture.
![Page 2: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/2.jpg)
Outline
• UWPHYSRET general considerations
• Applications to S-HIS data for JAIVEX
Examples of Retrieval Validation
Impact of PCA noise filter
• Conclusion
• Future development
![Page 3: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/3.jpg)
UWPHYSRET
• Attempt to build a reference physical retrieval package;
• Based on Clive Rodger’s methodology (Bayesian approach);
• Simultaneous retrieval of Temperature, Water Vapor, Ozone, Surface Temperature, Surface Emissivity;
• Uses LBLRTM 11.3;
• Analitical Jacobians can be updated at every iteration;
• Slow, flexible, accurate;
• Implemented in matlab
![Page 4: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/4.jpg)
Solution
• Iterative solution (Gauss-Newton):
• x is the state vector (a stands for a-priori, n is the iteration number);
• K is the Jacobian Matrix;
• Sa is the covariance matrix of the a-priori knowledge;
• Se is the covariance matrix of the Instrument noise;
• Y is the observation vector (radiances);
• F(x) is the calculated observation vector (radiances);
xn+1 = xa + (KnT •Se
-1 •Kn +Sa-1)-1 •Kn
T •Se-1 • ([Y - F(xn )] + Kn • (xn - xa ))
![Page 5: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/5.jpg)
UWPHYSRET
Uwphysret_tester uwphysret
SpectrumSelected Cannels
First GuessClimatology
Noise
Compute_F
Compute_residual
Estimate_K
Compute_convergence_test_values
Update_xhat
Solution VectorDiagnostic Vector
uwphysret_compute_apriori_mean_
covariance
![Page 6: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/6.jpg)
Uwphysret_tester: INPUT
• Observed Spectrum from matlab file
• Climatological Data Set: T,WV, Ozone, SKT (predefined)
First Guess (optional)
Climatology Covariance Matrix
• Surface Emissivity Data Set (predefined)
• Instrument Noise
Noise Covariance Matrix
• Apodizaion (variable)
• Number of Levels
Top Of the Atmosphere (variable)
Surface Elevation (variable)
• Variables to be retrieved (variable)
• Jacobian Update Frequency (variable)
• Convergence parameters (variable)
• External First Guess (variable)
Input Data Configuration Parameters
![Page 7: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/7.jpg)
UWPHYSRET: FW Model
• Compute_F and Estimate_K based on LBLRTM 11.3:
Highly accurate;
Slower than fast models;
Very flexible;
Highly configurable;
Easily Adaptable to different Instruments;
Provides Jacobians for 26 molecular species;
• Future implementations of UWPHYSRET will allow for easy substitution of FW through better definition of computational modules;
![Page 8: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/8.jpg)
Uwphysret_tester: OUTPUT
• xhat, x_dim, p, x0, xa
• R, wnR;
• F, wnF, K;
• yobs_minus_yhat, wnyobs_minus_yhat;
• chi_sq_optimal, xhat_update; norm_measurement_space_only, norm_solution_space_only ,
• Selwn, control;
• Sa, SaInv, diag(Se), Shat, contribution_function;
• numIterations, maxIterations
• xhat, x_dim, p, x0, xa;
• F, wnF;
• yobs_minus_yhat, wnyobs_minus_yhat;
• chi_sq_optimal xhat_update, norm_measurement_space_only, norm_solution_space_only;
• numIterations; maxIterations;
• Error_smoothing, error_measurements, error_total; averaging_kernel;
• Sa;
diagnosticsolution
![Page 9: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/9.jpg)
Solution: Retrieved Vertical Profiles
![Page 10: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/10.jpg)
Solution: Retrieval Errors
![Page 11: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/11.jpg)
Diagnostic: Residuals in radiance Units
Unfiltered Obs - Calc
Filtered Obs - Calc
NESR
![Page 12: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/12.jpg)
Diagnostic: Residuals in BT
UnfilteredObs - Calc
FilteredObs - Calc
NEDT @ 250 K
![Page 13: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/13.jpg)
Diagnostic:Retrieval Conversion Iteration by Iteration
Observed
![Page 14: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/14.jpg)
UWPHYSRET: Installation Requirements
• Availability of LBLRTM (10.4 and newer)
• MATLAB
• Tested on Linux and MAC OS X (Intel and Power PC)
• Software is freely distributed
to obtain it, please, contact R. Knuteson ([email protected])
![Page 15: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/15.jpg)
UWPHYSRET: Issues
• Technical:
Slow if K is updated every iteration;
Slower for satellite data than for aircraft data;
Number of levels is the same for all the atmospheric variables retrieved;
• Theoretical:
Based on Baysian approach, it assumes Gaussian distribution;
• General:
Proprietary software dependent;
![Page 16: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/16.jpg)
UWPHYSRET: current applications
• Using JAIVEx data:
Estimation of PCA noise filtering impact on retrieved profiles;
Investigation of the role of Noise Covariance Matrix on retrieval accuracy;
Investigation of the role of Climatology Covariance Matrix on retrieval accuracy;
• Surface Emissivity Retrieval;
• Estimation of Total Precipitable Water from retrieved profiles;
• Estimation of Stability Indices from retrieved profiles;
![Page 17: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/17.jpg)
Results on JAIVEx data
• JAIVEx stands for Joint Airborne IASI Validation Experiment
• is an international cal/val campaign in support of the NPOESS and MetOpseries of operational satellites.
• Focus is on the validation of radiance observations and meteorological products from the Infrared Atmospheric Sounding Interferometer, IASI.
• IASI measures radiation emission from the surface and atmosphere in the 645 -2760 cm-1 (i.e., 3.6-15.5 μm) spectral band with high spectral resolution (i.e., 8461 spectral channels with a spacing of 0.25 cm-1).
• The aircraft being employed are the NASA WB-57 and the FAAM BAe 146.
![Page 18: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/18.jpg)
SGP ARM Cart Site: 19 Apr 2007 15:32 UTC
S-HIS FOV
WB-57 Position
![Page 19: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/19.jpg)
Filtered Minus Unfiltered
![Page 20: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/20.jpg)
Retrieved Temperature Profile
![Page 21: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/21.jpg)
Retrieved Water Vapor Profile
![Page 22: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/22.jpg)
UNF
Radiance Residuals(compared to Unfiltered and Filtered Spectra)
UNFFIL
![Page 23: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/23.jpg)
UNF
BT Residuals(compared to Unfiltered and Filtered Spectra)
UNFFIL
![Page 24: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/24.jpg)
Unfiltered, Filtered, Averaged spectra
![Page 25: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/25.jpg)
Temperature Retrieval Comparison
UNFFILAVE
![Page 26: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/26.jpg)
Temperature Retrieval difference:Unfiltered-Filtered
![Page 27: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/27.jpg)
Uncertainties on Temperature
UNFFIL
![Page 28: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/28.jpg)
Relative Humidity Retrieval difference: Unfiltered - Filtered:
![Page 29: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/29.jpg)
Uncertainties on Water Vapor
UNF
FIL
![Page 30: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/30.jpg)
UNF
Residual for T retrieval only
FILAVE
Raob used as First GuessRetrieval from CO2 Band only
![Page 31: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/31.jpg)
Difference of Absolute Value of Radiance Residuals: Unfiltered - Filtered
Abs(RESu)-Abs(RESf)
![Page 32: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/32.jpg)
Difference of Absolute Value of Radiance Residuals:Averaged Spectrum
Abs(RESa)-Abs(RESu)
Abs(RESa)-Abs(RESf)
Ra-Rf
![Page 33: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/33.jpg)
Error Contributions not considered
• Off-diagonal elements of Instrument Noise (correlated noise):
Apodization artifacts (for retrieval from apodized radiances);
Correlated Noise;
• Forward Model Error
Diagonal;
Off-Diagonal;
• PCA noise filter
Correlated noise;
Uncorrelated noise.
![Page 34: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/34.jpg)
Retrieval difference for Apodized and Non-Apodized Data: Temperature
![Page 35: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/35.jpg)
LBLRTM 10.4 - LBLRTM 11.3 (Temperature)
![Page 36: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/36.jpg)
Next case …16 Apr 2007:Retrieved Vertical Profiles
![Page 37: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/37.jpg)
Conclusions
• UWPHYSRET was built as reference retrieval system and relies on lblrtm accuracy to allow research on hyperspectral data inversion
• PCA noise filtering effects obtained without redefinition of the noise Covariancematrix produces changes in the range of:
[0.5 - 1] K in temperature;
[10 - 30] % in lower atmospheric RH;
• These changes are comparable in magnitude to those induced by apodization or lblrtm updates;
• Under the approximations used in this study, PCA noise filtered data allows for:
better representation of the retrieval residual and for estimation of potential FWM biases
retrieval convergence when using more accurate representation of the noise covariance
• Redefinition of stable non singular full covariance matrix (including artifacts introduced by apodization, PCA noise filtering, correlations due to the sensor, and forward model error) is key to achievement of high acuracy;
![Page 38: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/38.jpg)
Future Work
• Application of the current system to IASI data;
• Re-definition of a stable, comprehensive, noise covariance matrix:
Better characterization of what we do not know;
• Improvement representation of a-priori information:
Better characterization of what we do know;
![Page 39: UWPHYSRET an SSEC inversion package for high resolution ... · Paolo Antonelli, R. Knuteson, R. Garcia, S. Bedka, D. Tobin, J.Taylor, W. Smith, and H. Revercomb UWPHYSRET an SSEC](https://reader034.vdocuments.mx/reader034/viewer/2022042914/5f4ce0b4936cca4fc4107ca2/html5/thumbnails/39.jpg)
Topics for Discussion
• Forward Model Error (Full Covariance);
• Climatology on CO2, O3, and other species;
• Use of other sources of information in the a-priori knowledge;