Enhancements of Radiance Bias Correction in NCEP’s Data Assimilation Systems

Yanqiu Zhu

Contributions from: John Derber, Andrew Collard, Dick Dee, Russ

Treadon, George Gayno, Jim Jung, Emily Liu, Min-Jeong

Part I: The enhanced radiance bias correction scheme

Part II: New emissivity predictor to handle large land-sea differences with the new CRTM (will be presented at the 6th WMO Symposium on Data Assimilation)

Part III: Radiance bias correction for all-sky radiance bias correction

Part I: Enhanced radiance bias correction scheme

The original radiance bias correction scheme

The motives to enhance the original scheme – the goals for the enhanced scheme

Methodology for the enhanced scheme

Experiment setup and results

Conclusions

Yanqiu Zhu John Derber, Andrew Collard, Dick Dee, Russ Treadon, George Gayno, Jim Jung

Why radiance bias correction – to generate unbiased analysis

• the error associated with the instrument and calibration

• the error of the radiative transfer model

• Inconsistencies among the assumptions of data usage and model as well as algorithms

Derber et al. (1991), Harris and Kelly (2001), Dee (2004), etc

4

The control variables and are estimated by minimizing (Derber et al., 1991, Derber and Wu, 1998)

),(~

),(~

2

1

)()(2

1)()(

2

1),(

1

11

xhyRxhy

BxxBxxxJ

T

bT

bbxT

b

is radiative transfer model )(xh

x

Air-mass dependent in GSI

Separate scan-angle dependent

Original radiance bias correction scheme

A two-step procedure

,),()(~ angleair bxbxhh

where )(),(1

xpxb ii

N

ii

air

is pre-specified parameter

x

5

is predictor term with predictor coeff. ip i

i

What we expect to achieve with the enhanced radiance bias correction scheme

Combine the scan angle and air-mass bias components inside the GSI variational framework. Eliminate the duplications and potential risk of opposite drifting

between the two components

Remove the pre-specified parameter for each predictor, so avoid the trial-and-error procedure for any new bias predictors

Apply a new pre-conditioning to the predictor coefficients taking into account of observation contribution, speeding up the convergence

6

What we expect to achieve with the enhanced radiance bias correction scheme (cont.)

Radiance bias correction is based on the data passing the quality control

Reasonable radiance bias correction estimate need to be manually derived prior to the use of any new radiance data or during the re-analysis process with the original scheme

Quality control is performed on the bias corrected data

7

What we expect to achieve with the enhanced radiance bias correction scheme (cont.)

Automatically detect any new/missing/recovery of radiance data and initialize new radiance data; Quickly capture any changes in the data and the system.

A new approach of specifying the background error variances for the predictor coefficients

An new initialization procedure for any new radiance dataAdd capability of bias correction for passive channels

We can use any new radiance data with initial radiance bias correction set to be zero 8

• is computed outside GSI, a running average of OMF

• The predictors of air-mass are computed inside the GSI

Enhanced scheme: a one-step procedure

Original scheme: a two-step procedureangleb

kK

kkN

angleb

1

• Variational angle bias correction, is computed inside the GSI along with other predictor terms

• satang file no longer needed

• Special attention to the initialization of bias coefficients: using quality-controlled data from previous run; or mode of all data

angleb

9

Change of the Preconditioning

• Hessian w.r.t. predictor coeff. of the cost function

aa ββ

ββ1T

β1

β

ββ2

2

βHHRHB

β

hJ

~ ,

Modified block-diagonal preconditioning is set to be the inverse of (Dee 2004)

Current preconditioner Modified preconditioner

βM

β

x

B

BB

0

0

β

x

M

BB

0

0

2

2

β J

For simplicity, only diagonal elements of are considered at each analysis cycle

βM

10

Currently scheme

Nii

,,1

,0.102

Set to be the estimate of analysis error variance of the coeff. from previous analysis cycle

New data: is initialized to be 10000.0

Missing data: is twice as much as that of the previous analysis cycle

Enhanced scheme

),,( 22

1 NdiagB

2

i

2

i

2

iAutomatically adjust variances of coeff.

More & high quality data

Smaller variance

Background variance for predictor coeff.

11

Bias correction for passive channels (passive_bc=.true.)

• Make preparation for assimilating passive channels data

• Aid quality control of other channels

),(~

),(~

2

1)()(

2

1)( 11 a

T

abT

b xhyRxhyBF

At the end of analysis, minimizing the functional

angleai

N

iiaa bxpxhxh

)()( ),(~

1

where

)()( ),(~

1ai

KN

iiaa xpxhxh

or If adp_anglebc=.t.12

Experiment configuration

• High Resolution: T574L64 3DVAR and T254L64 EnKF with 80 ensemble members

• 0.25 for static B and 0.75 for ensemble• Two-way hybrid coupling• EnKF relies on the GSI to perform the radiance bias

correction

CTL: prda141b, current radiance bias correction

RBC : prda141br, enhanced radiance bias correction

Experiment period: 00Z July 5, 2012 to 00Z Aug. 16, 2012

Spin-up period: 00ZJuly 5, 2012 to 18Z July 13, 2012 13

Convergence comparison

00Z July 15, 2012 14

Variance vs. Observation number

A big drop of obs number prompts a jump of variance

AIRS CH22 15

Global offset and total bias terms: AMSUA-NOAA18

16

OMF with and without bias correction

CTL RBC

Without BC

With BC

AMSUA-N18, CH4 00Z July 10, 2012 17

Mean Temperature Analysis & Difference 700hpa

18

HGT 500hpa NH Anomaly Correlation

19

HGT 500hpa SH Anomaly Correlation

20

RMSE Wind Vector 200hpa Tropics

21

24h & 48h Forecasts Fits to Obs

22

Conclusions

• One-step bias correction procedure

• Improved convergence rate

• Made it easier to add new bias predictors

• Added the capability to automatically detect any new/missing/recovery of radiance data

• New radiance data can be used without providing prior bias correction estimate

• Neutral forecast skills in NH and Tropics, better in SH

• The scheme was also tested on NDAS/NAM

• Currently is used in the wind energy project (Jacob’s talk at 6th WMO Symposium on Data Assimilation)

23

Part III: Radiance bias correction with all-sky radiance assimilation

Yanqiu Zhu, Emily Liu, Min-Jeong Kim, Andrew Collard, John Derber

Two other talks on all-sky radiance assimilation at NCEP: Emily Liu’s and Min-Jeong Kim’s talks

Bias correction for Operational clear radiance

• Angle bias correction

• Air-mass bias correction with five predictor terms: global offset, a zenith angle term, cloud liquid water, lapse rate, and the square of the lapse rate.

All-sky radiance assimilation

• Control variable: total cw/normalized cw/normalized RH

• State variables: Hydrometeors ql, qi, (qr, qs, qg, qh)

• The cloud liquid water predictor is removed

• Observation error based on averaged CLW

• Quality control: cloud filtering is removed

• Linearized moist physics

Issues of radiance bias correction with all-sky radiance assimilation

• Different error characteristics between clear-sky and cloudy radiance

• Different cloud information between radiance observation and first guess

1. O:clear vs. F:clear

2. O:clear vs. F:cloudy eliminate cloud

3. O:cloudy vs. F:clear add cloud

4. O:cloudy vs. F:cloudy

Ideally, Gaussian distributed (O:clr,F:clr) & (O:cld,F:cld), with one hump ((O:clr,F:cld), (O:cld,F:clr)) on each side

Issues of radiance bias correction with all-sky radiance assimilation (cont.)

OmF before BC OmF after BC

Lots of (O:cld, F:cld), too fewer other categories

Over bias corrected (O:cld,F:cld) and (O:clr,F:clr)

Issues of radiance bias correction with all-sky radiance assimilation (cont.)

OmF before BC OmF after BC

METOPA AMSUA Ch. 15

Issues of radiance bias correction with all-sky radiance assimilation (cont.)

METOPA AMSUA CH. 15

OmF before BC OmF after BC

(O:cld, F:cld)

• What may have hurt the radiance bias correction

• Experiments are underway, and results will be presented later at the GSI meeting

1. Bias correction coeff. are retrieved using only (O:clr,F:clr) and (O:cld,F:cld); Fixed previous coeff. are used to calculate the bias corrected OmF for (O:clr,F:cld) and (O:cld,F:clr).

2. Variational quality control

1. Inclusion of the (O:clr,F:cld) and (O:cld,F:clr) in bias correction

2. Radiance data with large OmF but small obs error

Experiment strategies: