seviri-iasi inter-calibration uncertainty evaluation

22 March 2011: GSICS GRWG & GDWG Meeting Daejeon, Korea

Tim Hewison

SEVIRI-IASI Inter-calibration Uncertainty Evaluation


Introduction

• Following guidelines provided by QA4EO• Based on Guide to the Expression of Uncertainty in Measurement

(GUM)• To be read in conjunction with the Algorithm Theoretical Basis Document • Uncertainties provide Quality Indicators for the inter-calibration products• Each process of ATBD is considered:• Uncertainties evaluated for key variables due to

• Random and• Systematic effects

• Combined to produce error budget • (Type B evaluation of combined uncertainty)

• Used to make recommendations for ATBD adjustments• To produce more consistent uncertainty estimates


Method – Introduction

• ATBD uses weighted linear regression to compare collocated radiances• from monitored and reference instruments• weightings based on spatial variance of radiances + radiometric noise

• Regression propagates these variances• to estimate uncertainty on corrected radiance

• But these are only 2 processes introducing uncertainty to final product• Full dynamic error propagation of all processes could be prohibitive• Analysis reviews uncertainties based on measurement model of processes

• for case studies, which are assumed to be typical• Define IASI as inter-calibration reference = Truth

• => IASI errors should not contribute to uncertainty of products• But some are included here, to illustrate their magnitude


Method – Systematic Errors

• Collocation criteria designed to minimise systematic errors• by ensuring samples systematically distributed

• But in reality small residual differences remain• Sampling differences introduce radiance errors in each

collocation• These introduce systematic errors in end product, • according to its sensitivity to each variable,• which is estimated from statistics of cases studies

• Use actual sampling distribution• or assume uniform distribution between threshold limits


Method – Random Errors

• Similarly, different processes introduce random errors• in each collocated radiance

• Their magnitude is estimated from typical range of each variable• and the sensitivity of radiances to perturbations of each

variable

• Regression process used to generate GSICS Correction coefficients • Reduces impact of random errors on each collocation

• Repeat regression many times for randomly perturbed datasets• to estimate uncertainty on corrected radiances• A Monte Carlo-like approach


Method – Combining Errors

• Method may be refined for dominate processes.


General Methodology

• For each process:• Estimate typical differences in sampling variables, ∆x

•between monitored and reference instruments• Estimate sensitivity of radiances to perturbations in x: ∂L/∂x

•where Li is radiance of each collocation, i• Uncertainty on Li due to process, j:

• Regression of collocated radiances => GSICS Correction, g(L)• Perturb observed radiances Li by u(Li)• Recalculated regression gives modified function, g’(L)• Gives different corrected radiances,

jjiij x

LxLu

,)(

ii LuLgL ˆ


Methodology for Systematic Errors

• For processes introducing systematic errors:• each collocated radiance is perturbed by

• Recalculated regression gives modified function, g’(L)• Evaluate for range of scene radiances• Compare to unmodified function, g(L)• To estimate uncertainty on corrected radiance,

• due to systematic errors introduced by process j:

s

j

sjii

sj x

LxLu

,)(

LgLgLu jjsj ˆ


Temporal Mismatch

• Uniform distribution over ±∆tmax=300 swith n~30000 collocations gives mean time difference ∆t = 2∆tmax/√(3n) ≈ 2 s.

• But mean difference in sampling time of collocations, ∆t=30sdue to deficiencies in orbital selection

• Calculate sensitivity from mean rate of change of radiances from time series of observationsMuch worse using 09:30 overpasses!

Time series of mean rate of change of radiances calculated from Meteosat-9

observations on 2009-09-20 over (30°W-30°E)x(30°S-30°N)

[1mW/m2/st/cm-1/hr ~ 1K/hr]

s

jxL


Summary of Systematic Errors’ Perturbations and Sensitivities

Table 1 summarises the magnitude of typical perturbations, ∆xj, of processes introducing systematic errors in the collocated radiances and the sensitivity of the 8 infrared channels of SEVIRI to these perturbations, dLj/dx.

Table 1 Summary of Systematic Errors’ Perturbations and Sensitivities

∆xIR03.9 IR06.2 IR07.3 IR08.7 IR09.7 IR10.8 IR12.0 IR13.4

Systematic ErrorsTemporal Mismatch 0.00833 hr -0.00793 -0.03840 -0.05379 -0.06854 -0.08192 -0.50907 -0.44417 -0.54865Longitudinal Mismatch 1.30640 km -0.00001 0.00008 0.00002 -0.00048 -0.00026 -0.00063 -0.00052 -0.00017Latitudinal Mismatch 1.30640 km 0.00003 0.00007 0.00032 0.00141 0.00090 0.00192 0.00200 0.00129Geometric Mismatch -0.00069 0.00049 0.00864 0.03063 0.03461 0.10989 0.03737 0.04778 0.11284Spectral Mismatch 1.00000 0.00014 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000Spectral Calibration 0.50000 0.00009 0.00019 0.00025 0.00024 0.00029 0.00008 0.00009 0.00037

Sensitivity, dLj /dx [mW/m2/st/cm-1/∆x ]


Combining all Systematic Errors

All uncertainties due to systematic processes, added in quadrature:

Systematic mismatches in time and space dominate the total systematic uncertainty due to finite gradients

But, IR3.9 dominated by spectral correction to compensate for IASI’s incomplete coverage

21

j

2sj

s LuLu

/

)(


Methodology for Random Errors

• For processes introducing random errors:• each collocated radiance is perturbed by• where zi is a random number drawn from distribution

consistent with characteristic difference ∆xr

• Repeat regression nk times gives modified functions, gj,k’(L)• Each evaluation is used to calculated corrected radiances:• Standard deviation of over the Monte Carlo ensemble is

calculated to provide an estimate of the uncertainty on corrected radiances due to each random process, j :

r

ji

rji

rj x

LzxLu

)(

kjL ,ˆ

kjL ,ˆ

21n

1k

2

kk

kjn

1k kk

2kjr

j 1nnLg

1nnLg

Lu

/

,,ˆ


Temporal Variability

• Uniform distribution over ±∆tmax=300 s equivalent to r.m.s. difference, ∆t = ∆tmax/√3 ≈ 173 s.

• Temporal Variability of typical SEVIRI images evaluated as RMSD between radiances sampled over different time intervals

• Calculate sensitivity from RMSD between radiances sampled at 21:30 and 21:45 over target area

R.M.S. differences in Meteosat-8 10.8 μm brightness temperatures with time

intervals from Rapid Scanning Meteosat data (red diamonds) and

with spatial separation in North-South direction (black pluses) and West-East

direction (black stars)

r

jxL


Summary of Random Errors’ Perturbations and Sensitivities

Table 3 summarises the characteristic difference, ∆xrj, of processes

introducing random errors in the collocated radiances and the sensitivity of the 8 infrared channels of SEVIRI to these perturbations, dLj/dx.

Table 3 Summary of Random Errors’ Perturbations and Sensitivities∆x

IR03.9 IR06.2 IR07.3 IR08.7 IR09.7 IR10.8 IR12.0 IR13.4Random ErrorsTemporal Variability 5.00 min 0.00269 0.00576 0.03303 0.16644 0.11526 0.25066 0.26603 0.16842Longitudinal Variability 3.41 km 0.00411 0.00691 0.04069 0.23236 0.15727 0.34051 0.35429 0.21155Latitudinal Variability 3.38 km 0.00448 0.00808 0.04744 0.25987 0.17728 0.38528 0.40352 0.24479Geometric Variability 1.00 0.00049 0.00864 0.03063 0.03461 0.10989 0.03737 0.04778 0.11284Spectral Variability 1.00 0.00014 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000 0.00000Radiometric Noise (GEO) 1.00 0.00054 0.00172 0.00587 0.02545 0.02807 0.03265 0.04823 0.09398Radiometric Noise (LEO) 1.00 0.00076 0.00122 0.00210 0.02874 0.02906 0.02369 0.02798 0.02489

Sensitivity, dLj /dx [mW/m2/st/cm-1/∆x ]


Combining all Random Errors

All uncertainties due to random processes, added in quadrature:

Random variability in space and time dominate the total random uncertainty for all channels

* 300s matches 3km wellOther terms negligible=> Could relax geometric

collocation threshold!

21

j

2rj

r LuLu

/

)(


Compare with Quoted Uncertainty

• This total random uncertainty is 1-4x larger than quoted values=> ATBD does not include important random processes

• Time series of Standard Biases shows higher variability=> Implies there are real instrument calibration changes

Meteosat SEVIRI Channel IR03.9 IR06.2 IR07.3 IR08.7 IR09.7 IR10.8 IR12.0 IR13.4

Standard Scene Radiance 284 236 255 284 261 286 285 267 K

Typical Standard Correction 0.071 -0.130 0.204 -0.002 -0.048 0.002 0.095 -1.136 K

Total Random Uncertainty 0.009 0.004 0.009 0.011 0.012 0.013 0.011 0.006 K

Median Uncertainty Quoted 0.004 0.004 0.004 0.003 0.006 0.003 0.004 0.006 K

Rolling SD of Standard Bias 0.013 0.016 0.017 0.022 0.022 0.020 0.016 0.021 K


Combining Systematic and Random Errors

Total uncertainties due to random and systematic processes:

Random components dominate total errorin most conditions

Uncertainties increase rapidly for low Tb (fewer collocations)

Errors much lower in WV channels

212r2sc LuLuLu/

)(


Recommendations

• Geometric collocation criteria could be relaxed by a factor of 10• Would give more collocations and reduce random error

• ATBD should be revised to account for correlations when estimating uncertainty on GSICS Correction• Or inflate uncertainty from regression by a factor of ~2

• Analysis assumes published SRFs are correctly interpreted• Misinterpretation would dominate systematic errors• Need clear guidance in the application of published SRFs

• Should repeat this analysis for all GSICS Products!

seviri-iasi inter-calibration uncertainty evaluation

Documents