introduction to data assimilation: lecture 2 saroja polavarapu meteorological research division...
TRANSCRIPT
![Page 1: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/1.jpg)
Introduction to Data Assimilation: Lecture 2
Saroja Polavarapu
Meteorological Research Division Environment Canada
PIMS Summer School, Victoria. July 14-18, 2008
![Page 2: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/2.jpg)
Outline of lecture 2
1. Covariance modelling – Part 1
2. Initialization (Filtering of analyses)
3. Basic estimation theory
4. 3D-Variational Assimilation (3Dvar)
![Page 3: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/3.jpg)
Background error covariance matrix filters analysis increments
)(1TT bba H xzRHBHBHxx
Bvxx ba
Analysis increments (xa – xb) are a linear combination of columns of B
Properties of B determine filtering properties of assimilation scheme!
![Page 4: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/4.jpg)
K
A simple demonstration of filtering properties of B matrix
![Page 5: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/5.jpg)
![Page 6: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/6.jpg)
2
1exp
2
0.2
d
d
x yx y
L
L
cos(x)
cos(2x)
Choose a correlation function and obs increment shape then compute analysis increments
obs/b = 0.5
![Page 7: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/7.jpg)
cos(3x)
cos(4x)
cos(5x)
cos(6x)
![Page 8: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/8.jpg)
cos(7x)
cos(8x)
cos(9x)
cos(10x)
![Page 9: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/9.jpg)
1. Covariance Modelling
1. Innovations method2.2. NMC-methodNMC-method3.3. Ensemble methodEnsemble method
![Page 10: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/10.jpg)
Background error covariance matrix
Ttbtbb xxxxP
•If x is 108, Pb is 108 x 108. •With 106 obs, cannot estimate Pb.•Need to model Pb.•The fewer the parameters in the model, •the easier to estimate them, but•less likely the model is to be valid
![Page 11: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/11.jpg)
• Historically used for Optimal Interpolation • (e.g. Hollingsworth and Lonnberg 1986,• Lonnberg and Hollingsworth 1986, Mitchell et al. 1990)• Typical assumptions:
•separability of horizontal and vertical correlations
•Homogeneity
•Isotropy
1. Innovations method
),(),,,( jibxjjii
bH ryxyx CC
),(),,,(),,,,,( jibVjjii
bHjjjiii
b zzyxyxzyxzyx CCC
)(),,,( ibxjjii
bH ryxyx CC
r
i
j
r
l
m
lr
i
j
m
r
![Page 12: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/12.jpg)
Instrument+representativeness
Background error
Choose obs s.t. these terms =0
Assume homogeneous, isotropiccorrelation model. Choose acontinuous function (r) which hasonly a few parameters such as L, correlation length scale. Plot allinnovations as a function of distanceonly and fit the function to the data.
Dec. 15/87-Mar. 15/88radiosonde data. Model: CMC T59L20
Mitchell et al. (1990)
![Page 13: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/13.jpg)
Mitchell et al. (1990)Mitchell et al. (1990)
Obs and Forecast error variances
![Page 14: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/14.jpg)
Vertical correlationsof forecast error
Height
Non-divergent wind
Lonnberg and Hollingsworth (1986)
Hollingsworth and Lonnberg (1986)
![Page 15: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/15.jpg)
![Page 16: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/16.jpg)
![Page 17: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/17.jpg)
Multivariate correlations
Mitchell et al. (1990)
Bouttier and Courtierwww.ecmwf.int 2002
![Page 18: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/18.jpg)
If covariances are homogeneous,variances are independent of space
If correlations are homogeneous,correlation lengths are independentof location
Covariances are not homogeneous
Correlations are not homogeneous
![Page 19: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/19.jpg)
Gustafsson (1981)
Daley (1991)
Correlations are not isotropic
![Page 20: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/20.jpg)
Are correlations separable?
If so, correlation length should beIndependent of height.
Mitchell et al. (1990)Lonnberg and Hollingsworth (1986)
![Page 21: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/21.jpg)
Covariance modelling assumptions:1. No correlations between background and
obs errors
2. No horizontal correlation of obs errors
3. Homogeneous, isotropic horizontal background error correlations
4. Separability of vertical and horizontal background error correlations
None of our assumptions are really correct. Therefore Optimal Interpolation is not optimal so it is often called Statistical Interpolation.
![Page 22: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/22.jpg)
2. Initialization
1. Nonlinear Normal Mode (NNMI)2. Digital Filter Initialization (DFI)3. Filtering of analysis increments
![Page 23: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/23.jpg)
Daley 1991
Balance in data assimilation
![Page 24: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/24.jpg)
The “initialization” step
• Integrating a model from an analysis leads to motion on fast scales
• Mostly evident in surface pressure tendency, divergence and can affect precipitation forecasts
• 6-h forecasts are used to quality check obs, so if noisy could lead to rejection of good obs or acceptance of bad obs
• Historically, after the analysis step, a separate “initialization” step was done to remove fast motions
• In the 1980’s a sophisticated “initialization” scheme based on Normal modes of the model equations was developed and used operationally with OI.
![Page 25: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/25.jpg)
),(
0),(
,
|
)(
T1
T
TT
T
RRGGGG
RRGGGGG
RRGG
GR
Ni
Nidt
d
Nidt
d
cEcEEc
cEcEEcc
xEcxEc
EEE
EEA
xAxx
Nonlinear Normal Mode Initialization (NNMI)
Consider model
Determine modes
Separate R and G
Project onto G
Define balance
Solution
![Page 26: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/26.jpg)
G
R
SA
N
L
The slow manifold
0dt
d Gc
![Page 27: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/27.jpg)
Daley 1991
![Page 28: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/28.jpg)
Digital Filter Initialization (DFI)
N
Nk
ukk
I xhx0
Lynch and Huang (1992)
N=12, t=30 min
Tc=8 hTc=6 h
Fillion et al. (1995)
![Page 29: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/29.jpg)
Combining Analysis and Initialization steps
• Doing an analysis brings you closer to the data.• Doing an initialization moves you farther from the data.
Daley (1986)
N
Gravity modes
Rossby modes
![Page 30: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/30.jpg)
Variational Normal model initializationDaley (1978), Tribbia (1982), Fillion and Temperton (1989), etc.
hgdSwwvvwuuIS AIVAIVAI
~~,)()(
~)(
~ 222
Minimize I such that uI, vI, I stays on M.
Daley (1986)
![Page 31: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/31.jpg)
Some signals in the forecast e.g. tides should NOT be destroyed by NNMI!
So filter analysis increments only
Seaman et al. (1995)
Semi-diurnal mode has amplitude seen in free model run, if anl increments are filtered
![Page 32: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/32.jpg)
3. A bit of Estimation theory(will lead us to 3D-Var)
![Page 33: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/33.jpg)
a posteriori p.d.f.
![Page 34: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/34.jpg)
![Page 35: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/35.jpg)
![Page 36: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/36.jpg)
![Page 37: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/37.jpg)
![Page 38: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/38.jpg)
![Page 39: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/39.jpg)
Data Selection
Bouttier and Courtier (2002)
From: ECMWF training course available at www.ecmwf.int
![Page 40: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/40.jpg)
The effect of data selection
Cohn et al. (1998)
OI
PSAS
![Page 41: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/41.jpg)
The effect of data selection
Cohn et al. (1998)
![Page 42: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/42.jpg)
Advantages of 3D-var
1. Obs and model variables can be nonlinearly related.
• H(X), H, HT need to be calculated for each obs type
• No separate inversion of data needed – can directly assimilate radiances
• Flexible choice of model variables, e.g. spectral coefficents
2. No data selection is needed.
))(())(()()()( 1 xzRxzxxBxxx b1b HHJ TT
![Page 43: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/43.jpg)
3D-Var Preconditioning
• Hessian of cost function is B-1 + HTR-1H
• To avoid computing B-1 in (1), change control variable to x=Lso first term in (1) becomes ½ and we minimize w.r.t. . Herex=x-xb
• After change of variable, Hessian is I + term• If no obs, preconditioner is great, but with
more obs, or more accurate obs, it loses its advantage
(1)))(())(()()()( 1 xzRxzxxBxxx b1b HHJ TT
![Page 44: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/44.jpg)
With covariances in spectral space,longer correlation lengths scales arepermitted in the stratosphere
With flexibility of choice of obs,can assimilate many new typesof obs such as scatterometer
Andersson et al. (1998) Andersson et al. (1998)
![Page 45: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/45.jpg)
Normalized AMSU weighting functions
1413121110 9 8 7 6 5
To assimilate radiances directly, H includes an instrument-specific radiative transfer model
))(())(()()()( 1 xzRxzxxBxxx b1b HHJ TT
![Page 46: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/46.jpg)
Kalnay et al. (1998)
Impact of Direct Assimilation of RadiancesAnomaly = difference between forecast and climatolgyAnomaly correlation = pattern correlation between forecast anomalies and
verifying analyses
1974 – improvedNESDIS VTPRRetrievals1978 – TOVSretrievals
![Page 47: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/47.jpg)
Center Region Operational Ref.
NCEP U.S.A. June 1991 Parrish& Derber (1991)
ECMWF* Europe Jan. 1996 Courtier et al. (1997)
CMC* Canada June 1997 Gauthier et al. (1998)
Met Office* U.K. Mar. 1999 Lorenc et al. (2000)
DAO NASA 1997 Cohn et al. (1997)
NRL US Navy 2000? Daley& Barker (2001)
JMA* Japan Sept. 2001 Takeuchi et al. (2004) SPIE proceedings,5234, 505-516
Operational weather centers used 3D-Var from1990’s *Later replaced by 4D-Var
![Page 48: Introduction to Data Assimilation: Lecture 2 Saroja Polavarapu Meteorological Research Division Environment Canada PIMS Summer School, Victoria. July 14-18,](https://reader038.vdocuments.mx/reader038/viewer/2022102707/56649e545503460f94b4b48c/html5/thumbnails/48.jpg)
Summary (Lecture 2)• Estimation theory provides mathematical basis
for DA. Optimality principles presume knowledge of error statistics.
• For Gaussian errors, 3D-var and OI are equivalent in theory, but different in practice
• 3D-var allows easy extension for nonlinearly related obs and model variables. Also allows more flexibility in choice of analysis variables.
• 3D-var does not require data selection so analyses are in better balance.
• Improvement of 3D-var over OI is not statistically significant for same obs. Systematic improvement of 3DVAR over OI in stratosphere and S. Hemisphere. Scores continue to improve as more obs types are added.