Maturation of Data Assimilation Over the Last Two Decades

John C. Derber

Environmental Modeling Center

NCEP/NWS/NOAA

Roger Daley (1996) ECMWF seminar on data assimilation

• “Fifteen years ago, data assimilation was a minor and often neglected sub-discipline of numerical weather prediction. The situation is very different today. Data assimilation in now felt to be important for all climate/environmental monitoring and estimating the ocean state. There has been great advances in both modelling and instrumentation for a variety of atmospheric phenomena and variables, and data assimilation provides the bridge between them.”

Optimal Interpolation (1980s)

• With the advent of optimal interpolation, analysis schemes transitioned from empirical techniques to theoretically based techniques.

• With these techniques, one could begin to use information on:– Observations and observation errors– Short term forecasts and forecast errors

• However, for most applications of optimal interpolation, many approximations had to be made.

Variational assimilation (1990s and 2000s)

J = Jb + Jo + Jc

J = (x-xb)TB-1(x-xb) + (H(x)-y0)T(E+F)-1(H(x)-y0) + JC

J = Fit to background + Fit to observations + constraints

x = Analysisxb = BackgroundB = Background error covarianceH = Forward modely0 = ObservationsE+F = R = Instrument error + Representativeness errorJC = Constraint term

Variational assimilation

• Inclusion of observation operator (H), transforming the analysis variable into the form of the observation operator, which in turn allowed;

– Inclusion of radiances and other indirect observations– Definition of analysis variables different than the

model variables or observations– Inclusion of forecast model in interpolation operator

(4DVAR)• Use of all observations at once, eliminating

many approximations/complex codes which were prone to failure

• Allowed inclusion of additional constraint terms

Variational Assimilation

• Background error covariances– Background error variance now

approximately ½ radiosonde error variance (ECMWF)

– Fully non-separable covariance matrices– Inclusion of constraints within background

error– Ongoing research in situation dependent

background errors

Isotropic Error Correlation in Valley

Plotted Over Utah Topography obs influence extends into mountains indiscriminately

Anisotropic Error Correlation in ValleyPlotted Over Utah Topography

obs influence restricted to areas of similar elevation

Anisotropic Error Correlation on Slope Plotted Over Utah Topography

obs influence restricted to areas of similar elevation

Anisotropic Error Correlation on Mt Top Plotted Over Utah Topography

obs influence restricted to areas of similar elevation

Observations

• No large scale data voids.• Number of observations used in assimilation

increasing rapidly (but not as rapidly as number of observations).

• Both operational and non-operational satellite data being used operationally.

• Increased use of data assimilation systems in calibration/validation activities for satellites.

• Expected data impact from data producers always overoptimistic.

Observations

• Adequate fast forward models for observations still major problem, e.g.,– Precipitation (satellite/radar)– Clouds (IR and microwave)– Lightning

• Biases in forward models/observations greatly impact the usefulness of data

Variational Assimilation Surprises

• Computational cost for 3DVAR similar to OI (even with all observations used together).

• Much of need for Nonlinear Normal Mode Initialization came from data selection.

• Direct use of radiances produces significant impacts in both hemispheres.

– In Southern Hemisphere, significantly stronger circulations were produced using radiances.

– Southern Hemisphere forecast skill has become similar to Northern Hemisphere skill. Since NH much better observed, is SH easier?

– Microwave instruments dominate impact.

Future?

• Extension of data assimilation techniques to: – Additional analysis variables (including cloud water/ice,

etc.)– Smaller scales– Tropical disturbances– Land/ice/ocean surfaces

• Inclusion of improved bias correction for background and all types of observations

• Inclusion of observation specific observational/representativeness errors

Future?

• Use of situation dependent background errors• Trying to catch up with volume of data from new

observing platforms• Improved models and physics within analysis

systems• Ensembles?• New systems judged on performance. With data

assimilation you must do everything right!