airs radiance and geophysical products: methodology and validation mitch goldberg, larry mcmillin...
TRANSCRIPT
![Page 1: AIRS Radiance and Geophysical Products: Methodology and Validation Mitch Goldberg, Larry McMillin NOAA/NESDIS Walter Wolf, Lihang Zhou, Yanni Qu and M](https://reader035.vdocuments.mx/reader035/viewer/2022062409/56649f515503460f94c73e94/html5/thumbnails/1.jpg)
AIRS Radiance and Geophysical Products:
Methodology and Validation
Mitch Goldberg , Larry McMillin
NOAA/NESDIS
Walter Wolf, Lihang Zhou, Yanni Qu and M. Divakarla
![Page 2: AIRS Radiance and Geophysical Products: Methodology and Validation Mitch Goldberg, Larry McMillin NOAA/NESDIS Walter Wolf, Lihang Zhou, Yanni Qu and M](https://reader035.vdocuments.mx/reader035/viewer/2022062409/56649f515503460f94c73e94/html5/thumbnails/2.jpg)
Science ActivitiesScience Activities
Data compression.Data compression. Validate and improve radiative transfer calculations.Validate and improve radiative transfer calculations. Cloud detection and clearing.Cloud detection and clearing. Cloud productsCloud products Channel selection (super channels).Channel selection (super channels). Validate and improve retrieval algorithms.Validate and improve retrieval algorithms. Trace gasesTrace gases Surface emissivitySurface emissivity Use MODIS to improve AIRS cloud detection and cloud Use MODIS to improve AIRS cloud detection and cloud
clearingclearing Radiance bias adjustmentsRadiance bias adjustments Forecast impact studiesForecast impact studies
![Page 3: AIRS Radiance and Geophysical Products: Methodology and Validation Mitch Goldberg, Larry McMillin NOAA/NESDIS Walter Wolf, Lihang Zhou, Yanni Qu and M](https://reader035.vdocuments.mx/reader035/viewer/2022062409/56649f515503460f94c73e94/html5/thumbnails/3.jpg)
TOPICS
Use of principal components (a.k.a. eigenvectors) for data compression.
Surface emissivity
Cloud detection
![Page 4: AIRS Radiance and Geophysical Products: Methodology and Validation Mitch Goldberg, Larry McMillin NOAA/NESDIS Walter Wolf, Lihang Zhou, Yanni Qu and M](https://reader035.vdocuments.mx/reader035/viewer/2022062409/56649f515503460f94c73e94/html5/thumbnails/4.jpg)
AIRS Geophysical Products
Microwave-only retrieval of sfc emissivity, sfc temperature, sfc type and profiles of temperature, water vapor and cloud liquid water.
AIRS retrieval of cloud amount and height, sfc emissivity, sfc temperature, and profiles of temperature, water vapor and ozone.
AIRS has two retrieval steps – very fast eigenvector regression followed by a physical retrieval algorithm.
![Page 5: AIRS Radiance and Geophysical Products: Methodology and Validation Mitch Goldberg, Larry McMillin NOAA/NESDIS Walter Wolf, Lihang Zhou, Yanni Qu and M](https://reader035.vdocuments.mx/reader035/viewer/2022062409/56649f515503460f94c73e94/html5/thumbnails/5.jpg)
Data Compression Data Compression Advanced IR sounder data are very large compared with current Advanced IR sounder data are very large compared with current
sounders (1 orbit ~ 2GB vs. 8 MB) Much larger for GIFTS.sounders (1 orbit ~ 2GB vs. 8 MB) Much larger for GIFTS.
Information is not independent. Principal component analysis Information is not independent. Principal component analysis (PCA) is often used to reduce data vectors with many (PCA) is often used to reduce data vectors with many components to a different set of data vectors with much fewer components to a different set of data vectors with much fewer components that still retains most of the variability and components that still retains most of the variability and information of the original data information of the original data
Data are rotated onto a new set of axes, such that the first few Data are rotated onto a new set of axes, such that the first few axes have the most explained variance.axes have the most explained variance.
Principal component scores are provided instead of the Principal component scores are provided instead of the individual channels.individual channels.
Individual channels can be reconstructed with minimal signal Individual channels can be reconstructed with minimal signal loss with added benefit of noise reduction.loss with added benefit of noise reduction.
![Page 6: AIRS Radiance and Geophysical Products: Methodology and Validation Mitch Goldberg, Larry McMillin NOAA/NESDIS Walter Wolf, Lihang Zhou, Yanni Qu and M](https://reader035.vdocuments.mx/reader035/viewer/2022062409/56649f515503460f94c73e94/html5/thumbnails/6.jpg)
Generating AIRS eigenvectors
Collect an ensemble of AIRS spectra (2378 channels).
The radiances are normalized by expected instrumental noise (signal to noise)
Compute the covariance matrix S
Compute the eigenvectors E and eigenvalues S = E ET
E = matrix of orthonormal eigenvectors (2378x2378) = vector of eigenvalues (explained variance)
![Page 7: AIRS Radiance and Geophysical Products: Methodology and Validation Mitch Goldberg, Larry McMillin NOAA/NESDIS Walter Wolf, Lihang Zhou, Yanni Qu and M](https://reader035.vdocuments.mx/reader035/viewer/2022062409/56649f515503460f94c73e94/html5/thumbnails/7.jpg)
Training Ensemble
Eigenvectors are generated from a spatial subset of AIRS data (200 mbytes vs 30 GB full data)
Eigenvectors are generated daily. A static set of eigenvectors is used, but the
ensemble is occasionally updated with new structures.
When the ensemble is updated a new set of eigenvectors is also updated.
![Page 8: AIRS Radiance and Geophysical Products: Methodology and Validation Mitch Goldberg, Larry McMillin NOAA/NESDIS Walter Wolf, Lihang Zhou, Yanni Qu and M](https://reader035.vdocuments.mx/reader035/viewer/2022062409/56649f515503460f94c73e94/html5/thumbnails/8.jpg)
Locations used in generating eigenvectors
![Page 9: AIRS Radiance and Geophysical Products: Methodology and Validation Mitch Goldberg, Larry McMillin NOAA/NESDIS Walter Wolf, Lihang Zhou, Yanni Qu and M](https://reader035.vdocuments.mx/reader035/viewer/2022062409/56649f515503460f94c73e94/html5/thumbnails/9.jpg)
Applying AIRS eigenvectors
On independent data – compute principal component scores.
P = ET R ; elements of R = (ri- ri ) /ni
Invert equation and compute reconstructed radiances R*.
R* = E P
Reconstructed radiances are used for quality control.
Reconstruction score = [ 1/N (R*i - Ri)2 ]1/2
i = 1 ….N channels
![Page 10: AIRS Radiance and Geophysical Products: Methodology and Validation Mitch Goldberg, Larry McMillin NOAA/NESDIS Walter Wolf, Lihang Zhou, Yanni Qu and M](https://reader035.vdocuments.mx/reader035/viewer/2022062409/56649f515503460f94c73e94/html5/thumbnails/10.jpg)
1 7497.60 2 1670.40 3 945.52 4 496.01 5 284.01 6 266.30 7 156.95 8 139.67 9 88.27 10 72.83 11 60.03 12 53.42 13 45.01 14 39.72 15 34.54 16 26.57 17 22.62 18 17.60
19 14.68 20 13.49 21 12.28 22 11.32 23 10.70 24 9.08 25 8.24 26 7.85 27 6.77 28 5.98 29 5.83 30 5.39 31 5.34 32 4.98 33 4.34 34 4.09 35 3.62 36 3.48
37 3.38 38 3.11 39 2.82 40 2.53 41 2.41 42 2.39 43 2.34 44 2.24 45 2.03 46 1.86 47 1.78 48 1.71 49 1.65 50 1.61 51 1.54 52 1.52 53 1.35 54 1.34
55 1.25 56 1.19 57 1.16 58 1.15 59 1.09 60 1.05 61 1.02 62 0.98 63 0.90 64 0.86 65 0.81 66 0.80 67 0.78 68 0.77 69 0.73 70 0.72 71 0.70 72 0.66
Square root of the eigenvalues
![Page 11: AIRS Radiance and Geophysical Products: Methodology and Validation Mitch Goldberg, Larry McMillin NOAA/NESDIS Walter Wolf, Lihang Zhou, Yanni Qu and M](https://reader035.vdocuments.mx/reader035/viewer/2022062409/56649f515503460f94c73e94/html5/thumbnails/11.jpg)
Reconstruction score = [ 1/N (R*i - Ri)2 ]1/2
i = 1 ….N channels
![Page 12: AIRS Radiance and Geophysical Products: Methodology and Validation Mitch Goldberg, Larry McMillin NOAA/NESDIS Walter Wolf, Lihang Zhou, Yanni Qu and M](https://reader035.vdocuments.mx/reader035/viewer/2022062409/56649f515503460f94c73e94/html5/thumbnails/12.jpg)
Reconstruction score = [ 1/N (R*i - Ri)2 ]1/2
i = 1 ….N channels
![Page 13: AIRS Radiance and Geophysical Products: Methodology and Validation Mitch Goldberg, Larry McMillin NOAA/NESDIS Walter Wolf, Lihang Zhou, Yanni Qu and M](https://reader035.vdocuments.mx/reader035/viewer/2022062409/56649f515503460f94c73e94/html5/thumbnails/13.jpg)
Monitoring Eigenvectors
Monitoring eigenvectors is critical
Eigenvectors may need to be updated due to new structures that were not in the original ensemble
![Page 14: AIRS Radiance and Geophysical Products: Methodology and Validation Mitch Goldberg, Larry McMillin NOAA/NESDIS Walter Wolf, Lihang Zhou, Yanni Qu and M](https://reader035.vdocuments.mx/reader035/viewer/2022062409/56649f515503460f94c73e94/html5/thumbnails/14.jpg)
12/4/00 reconstruction scores
![Page 15: AIRS Radiance and Geophysical Products: Methodology and Validation Mitch Goldberg, Larry McMillin NOAA/NESDIS Walter Wolf, Lihang Zhou, Yanni Qu and M](https://reader035.vdocuments.mx/reader035/viewer/2022062409/56649f515503460f94c73e94/html5/thumbnails/15.jpg)
Monitoring reconstruction score is important
Days
July Aug Sep Oct Nov Dec Jan Feb
![Page 16: AIRS Radiance and Geophysical Products: Methodology and Validation Mitch Goldberg, Larry McMillin NOAA/NESDIS Walter Wolf, Lihang Zhou, Yanni Qu and M](https://reader035.vdocuments.mx/reader035/viewer/2022062409/56649f515503460f94c73e94/html5/thumbnails/16.jpg)
![Page 17: AIRS Radiance and Geophysical Products: Methodology and Validation Mitch Goldberg, Larry McMillin NOAA/NESDIS Walter Wolf, Lihang Zhou, Yanni Qu and M](https://reader035.vdocuments.mx/reader035/viewer/2022062409/56649f515503460f94c73e94/html5/thumbnails/17.jpg)
Noise
Noise free 75 PCS
![Page 18: AIRS Radiance and Geophysical Products: Methodology and Validation Mitch Goldberg, Larry McMillin NOAA/NESDIS Walter Wolf, Lihang Zhou, Yanni Qu and M](https://reader035.vdocuments.mx/reader035/viewer/2022062409/56649f515503460f94c73e94/html5/thumbnails/18.jpg)
Observed vs noise-free reconstructed vs noise-free.Observed vs noise-free reconstructed vs noise-free.
Noise ReductionNoise Reduction
![Page 19: AIRS Radiance and Geophysical Products: Methodology and Validation Mitch Goldberg, Larry McMillin NOAA/NESDIS Walter Wolf, Lihang Zhou, Yanni Qu and M](https://reader035.vdocuments.mx/reader035/viewer/2022062409/56649f515503460f94c73e94/html5/thumbnails/19.jpg)
“ Observed” Reconstructed
![Page 20: AIRS Radiance and Geophysical Products: Methodology and Validation Mitch Goldberg, Larry McMillin NOAA/NESDIS Walter Wolf, Lihang Zhou, Yanni Qu and M](https://reader035.vdocuments.mx/reader035/viewer/2022062409/56649f515503460f94c73e94/html5/thumbnails/20.jpg)
Observed vs. ReconstructedObserved vs. Reconstructed
![Page 21: AIRS Radiance and Geophysical Products: Methodology and Validation Mitch Goldberg, Larry McMillin NOAA/NESDIS Walter Wolf, Lihang Zhou, Yanni Qu and M](https://reader035.vdocuments.mx/reader035/viewer/2022062409/56649f515503460f94c73e94/html5/thumbnails/21.jpg)
New Plan
Generate full spatial resolution AIRS principal component score datasets
Size ~ 5 MB instead of 150 MB per six minute granule
![Page 22: AIRS Radiance and Geophysical Products: Methodology and Validation Mitch Goldberg, Larry McMillin NOAA/NESDIS Walter Wolf, Lihang Zhou, Yanni Qu and M](https://reader035.vdocuments.mx/reader035/viewer/2022062409/56649f515503460f94c73e94/html5/thumbnails/22.jpg)
Surface emissivity
![Page 23: AIRS Radiance and Geophysical Products: Methodology and Validation Mitch Goldberg, Larry McMillin NOAA/NESDIS Walter Wolf, Lihang Zhou, Yanni Qu and M](https://reader035.vdocuments.mx/reader035/viewer/2022062409/56649f515503460f94c73e94/html5/thumbnails/23.jpg)
![Page 24: AIRS Radiance and Geophysical Products: Methodology and Validation Mitch Goldberg, Larry McMillin NOAA/NESDIS Walter Wolf, Lihang Zhou, Yanni Qu and M](https://reader035.vdocuments.mx/reader035/viewer/2022062409/56649f515503460f94c73e94/html5/thumbnails/24.jpg)
Retrieval error based on 18 channels
Background Std dev.
Retrieval error
![Page 25: AIRS Radiance and Geophysical Products: Methodology and Validation Mitch Goldberg, Larry McMillin NOAA/NESDIS Walter Wolf, Lihang Zhou, Yanni Qu and M](https://reader035.vdocuments.mx/reader035/viewer/2022062409/56649f515503460f94c73e94/html5/thumbnails/25.jpg)
Clear detection
![Page 26: AIRS Radiance and Geophysical Products: Methodology and Validation Mitch Goldberg, Larry McMillin NOAA/NESDIS Walter Wolf, Lihang Zhou, Yanni Qu and M](https://reader035.vdocuments.mx/reader035/viewer/2022062409/56649f515503460f94c73e94/html5/thumbnails/26.jpg)
BACKGROUND
NWP centers will assimilate clear radiances
Need very good cloud detection algorithm
Very important for radiance validation and to initiate the testing of the level 2 retrieval code.
![Page 27: AIRS Radiance and Geophysical Products: Methodology and Validation Mitch Goldberg, Larry McMillin NOAA/NESDIS Walter Wolf, Lihang Zhou, Yanni Qu and M](https://reader035.vdocuments.mx/reader035/viewer/2022062409/56649f515503460f94c73e94/html5/thumbnails/27.jpg)
Cloud Detection over Ocean Use VIS/NIR channels during day.
Compare SST with 2616 cm-1 at Night.
Predicting SST from 11 and 8 micron channels (works for day and night)
Predict 2616 from 8 micron channels (night)
11 micron window > 270 K
![Page 28: AIRS Radiance and Geophysical Products: Methodology and Validation Mitch Goldberg, Larry McMillin NOAA/NESDIS Walter Wolf, Lihang Zhou, Yanni Qu and M](https://reader035.vdocuments.mx/reader035/viewer/2022062409/56649f515503460f94c73e94/html5/thumbnails/28.jpg)
ONLY 0.5% residual clouds
![Page 29: AIRS Radiance and Geophysical Products: Methodology and Validation Mitch Goldberg, Larry McMillin NOAA/NESDIS Walter Wolf, Lihang Zhou, Yanni Qu and M](https://reader035.vdocuments.mx/reader035/viewer/2022062409/56649f515503460f94c73e94/html5/thumbnails/29.jpg)
Cloud detection – Non Sea
Predict AIRS channel at 2390.9 cm-1 from AMSU
FOV is labeled “mostly clear” if predicted AIRS – observed AIRS < 2
AND IF
SW LW IR window test is successful:
[ch(2558.224)-CH(900.562)] < 10 K
Variability of 2390.910 radiance within 3x3 < 0.0026
![Page 30: AIRS Radiance and Geophysical Products: Methodology and Validation Mitch Goldberg, Larry McMillin NOAA/NESDIS Walter Wolf, Lihang Zhou, Yanni Qu and M](https://reader035.vdocuments.mx/reader035/viewer/2022062409/56649f515503460f94c73e94/html5/thumbnails/30.jpg)
Clear Detected Fovs Cloud cleared casesClear Detected Fovs Cloud cleared cases
![Page 31: AIRS Radiance and Geophysical Products: Methodology and Validation Mitch Goldberg, Larry McMillin NOAA/NESDIS Walter Wolf, Lihang Zhou, Yanni Qu and M](https://reader035.vdocuments.mx/reader035/viewer/2022062409/56649f515503460f94c73e94/html5/thumbnails/31.jpg)
Future Work – Merge MODIS and AIRS
High spatial resolution will improve determination of clear AIRS fovs.
High spatial resolution will greatly improve clear estimate needed for cloud clearing.
![Page 32: AIRS Radiance and Geophysical Products: Methodology and Validation Mitch Goldberg, Larry McMillin NOAA/NESDIS Walter Wolf, Lihang Zhou, Yanni Qu and M](https://reader035.vdocuments.mx/reader035/viewer/2022062409/56649f515503460f94c73e94/html5/thumbnails/32.jpg)
MODIS Sounder Radiance Product
MODIS has HIRS-like sounder channels – but at high spatial resolution (1 km).
Find a few clear MODIS fovs in a 50 x 50 km area should provide a yield of 80% -- similar to AMSU
![Page 33: AIRS Radiance and Geophysical Products: Methodology and Validation Mitch Goldberg, Larry McMillin NOAA/NESDIS Walter Wolf, Lihang Zhou, Yanni Qu and M](https://reader035.vdocuments.mx/reader035/viewer/2022062409/56649f515503460f94c73e94/html5/thumbnails/33.jpg)
Summary Busy getting ready for real AIRS data Simulating AIRS in real-time has provided a means to develop ,
test and validate the delivery of products to NWP centers, AND created a platform to develop scientific tools to analyze the
data and test algorithms. Early releases of the data should be available 3 months after launch Final radiance products ~ 7 months Retrievals ~ 12 months First activity will be to examine biases between measured and
computed radiances and validation of the clear detection algorithm. “Day-2” Utilize MODIS