introduction to ensemble kalman filter - welcome to the ... · introduction to ensemble kalman...
TRANSCRIPT
Introduction to EnsembleKalman Filter
E. Kalnay with Hong Li and T. MiyoshiUniversity of MarylandChaos-Weather Group
B. Hunt, J. Yorke, E.OttK. Ide, T. Miyoshi, E. Kalnay,
I. Szunyogh (TAMU), E. Kostelich (ASU) and 16 currentstudents and Ph.D.s
JCSDA Summer Colloquium 2009
Recall the basic formulation of OIRecall the basic formulation of OI
Ta= T
b+ w(T
o! T
b) 0 " w " 1
Optimal weight to minimize the analysis errors: w =!b
2
!b
2+!
o
2
OI for a scalar (analysis):
Recall the basic formulation of OIRecall the basic formulation of OI
OI for a scalar (analysis):Ta= T
b+ w(T
o! T
b) 0 " w " 1
Optimal weight to minimize the analysis errors: w =!b
2
!b
2+!
o
2
OI for a vector: xib= Mx
i!1a
xi
a= x
i
b+K(y
i
o ! Hxi
b)
K = PbHT HPbHT+ R"# $%
!1
Pa = I !KH[ ]Pb
Pb= !x
b•!x
b
T!x
b= x
b" x
t
In OI is statistically pre-estimated and constant in time.Is this a good approximation?
Pb
OI and Kalman Filter for a scalarOI and Kalman Filter for a scalar
OI for a scalar: Tb(ti) = MT
a(ti!1); "
b
2= (1+ a)"
a
2=
1
1! w"
a
2= const
w ="b
2
"b
2+"
o
2; "
a
2= (1! w)"
b
2
Ta(ti) = T
b(ti) + w(T
o(ti) ! T
b(ti)) 0 # w # 1
OI and Kalman Filter for a scalarOI and Kalman Filter for a scalar
OI for a scalar: Tb(ti) = MT
a(ti!1); "
b
2= (1+ a)"
a
2=
1
1! w"
a
2= const
w ="b
2
"b
2+"
o
2; "
a
2= (1! w)"
b
2
Ta(ti) = T
b(ti) + w(T
o(ti) ! T
b(ti)) 0 # w # 1
Kalman Filterfor a scalar:
Now the background error variance is forecasted using thelinear tangent model and its “adjoint”.
Tb(ti) = MT
a(ti!1); "
b
2= (L"
a)(L"
a)T; L = dM / dT
w ="b
2
"b
2+"
o
2; "
a
2= (1! w)"
b
2
Ta(ti) = T
b(ti) + w(T
o(ti) ! T
b(ti)) 0 # w # 1
Ensemble Kalman Filter (EnKF)
Pib=
1
m !1(xk
f! x
t)i (xk
f! x
t)iT
k=1
m
" , m#$
We use the ensemble to estimate *
Pib!
1
K "1(xk
f" x
f)i (xk
f" x
f)iT
k=1
m
#
=1
K "1Xi
bXi
bT
Problem left: how to estimate theanalysis error covariance and theanalysis perturbations?
Pi
a=
1
K !1X
i
aX
i
aT
Local Ensemble Transform Kalman Filter(Ott et al, 2004, Hunt et al, 2004, 2007)
• Model independent(black box)• Obs. assimilatedsimultaneously at eachgrid point• 100% parallel: fast• No adjoint needed• 4D LETKF extension
(Start with initial ensemble)
LETKFObservationoperator
Model
ensemble analyses
ensemble forecasts
ensemble“observations”
Observations
Perform data assimilation in a local volume, choosing observations
The state estimate is updated at thecentral grid red dot
Localization based on observations
Perform data assimilation in a local volume, choosing observations
The state estimate is updated at thecentral grid red dot
All observations (purple diamonds)within the local region are assimilated
Localization based on observations
The LETKF algorithm can be described in a single slide
Local Ensemble Transform Kalman Filter (Local Ensemble Transform Kalman Filter (LETKFLETKF))
Forecast step:Analysis step: construct
Locally: Choose for each grid point the observations to be used, andcompute the local analysis error covariance and perturbations inensemble space:
Analysis mean in ensemble space:and add to to get the analysis ensemble in ensemble space
The new ensemble analyses in model space are the columns of . Gathering the grid point analyses forms the new
global analyses. Note that the the output of the LETKF are analysisweights and perturbation analysis matrices of weights . Theseweights multiply the ensemble forecasts.
x
n,k
b= M
nx
n!1,k
a( )Xb
= x1
b ! xb | ... | xK
b ! xb"# $%;
yi
b= H (x
i
b); Y
n
b= y
1
b ! yb | ... | yK
b ! yb"# $%
!Pa= K !1( )I + YbT
R!1Y
b"# $%!1;W
a= [(K !1) !Pa ]1/2
Xn
a= X
n
bW
a+ x
b
wa= !P
aY
bTR
!1(yo! y
b )
Wa
Globally:
wa
Wa
Repeat: Repeat: LETKFLETKF, globally, with notes (4D-LETKF), globally, with notes (4D-LETKF)
Forecast step:
Analysis step: construct
x
n,k
b= M
nx
n!1,k
a( )
Xb= x
1
b ! xb | ... | xK
b ! xb"# $%;
yi
b= H (x
i
b); Y
n
b= y
1
b ! yb | ... | yK
b ! yb"# $%
Globally:
Notes: Pb= X
bX
bT
Yb! HX
b , but it is computed nonlinearly
4D-LETKF is done by simply concatenating in the vertical thevectors and matrix valid at different times in theassimilation window. The rest is the same.
yo, y
b
Yb
Repeat: LETKFRepeat: LETKF, locally, with notes, locally, with notesLocally: Choose for each grid point the observations to be used, andcompute the local analysis error covariance and perturbations inensemble space:
Analysis mean in ensemble space:and add to to get the analysis ensemble in ensemble space
The new ensemble analyses in model space are the columns of . Gathering the grid point analyses forms the new
global analyses. Note that the the output of the LETKF are analysisweights and perturbation analysis matrices of weights . Theseweights multiply the ensemble forecasts.
!Pa= K !1( )I + YbT
R!1Y
b"# $%!1; W
a= [(K !1) !Pa ]1/2
Xn
a= X
n
bW
a+ x
b
wa= !P
aY
bTR
!1(yo! y
b )
Wa
wa
Wa
!Pa= K !1( )I + YbT
R!1Y
b"# $%!1
Notes: is Pa!1
= Pb!1
+HTR
!1H"# $%
in ensemble space, where is the unit matrix/(K-1)! !Pb!1
= I / (K !1)
Note that here, the columns of are the full analysisensemble members (not the perturbations).
Xa
The 4D-LETKF produces an analysis in terms ofweights of the ensemble forecast members at theanalysis time tn, giving the trajectory that best fits allthe observations in the assimilation window.
analysis time weightsImplication of getting analysis weights
The 4D-LETKF produces an analysis in terms ofweights of the ensemble forecast members at theanalysis time tn, giving the trajectory that best fits allthe observations in the assimilation window.
analysis time weightsA linear comb. of trajectories is ~a trajectory. If it is closeto the truth at the end it should be close to the truththroughout the trajectory (neglecting model errors).
No-cost LETKF smoother ( ): apply at tn-1 the sameweights found optimal at tn. It works for 3D- or 4D-LETKF
The no-cost smoother makes possible: Outer loop (like in 4D-Var) “Running in place” (faster spin-up) Use of future data in reanalysis Ability to use longer windows
No-cost LETKF smoothertested on a QG model: It works!
“Smoother” reanalysis
LETKF Analysisxna= xn
f+ Xn
fwn
aLETKF analysis
at time n
Smoother analysis at time n-1
!xn!1a
= xn!1f
+ Xn!1
fwn
a
This very simple smoother allows us to go backand forth in time within an assimilation window:it allows assimilation of future data in reanalysis
Example: forecast sensitivity to observationsLiu, Hong and Kalnay, in preparation
The only difference between and is the assimilation of observations at 00hr.
Observation impact on the reduction of forecast error:
(Adapted from Langlandand Baker, 2004)
et |0 = xt |0f! xt
a
et |0
et |!6
!e2= (e
t |0
Tet |0" e
t |"6
Tet |"6)
analysis t
et |!6
et |0
-6hr 00hr
OBS.
Example: forecast sensitivity to observations
!e2= (et |0
Tet |0 " et |"6
Tet |"6
) = (et |0T " et |"6
T)(et |0 + et |"6
)
= (xt |0f " xt |"6
f)T
(et |0 + et |"6)
= M(x0
a " x0|"6
b)#$ %&
T
(et |0 + et |"6), so that
!e2= MK(y " H (x
0|"6
b)#$ %&
T
(et |0 + et |"6)
Langland and Baker (2004) solve this with the adjoint:!e2 = MK(y " H (x
0|"6b)#$ %&
T
(et |0+ e
t |"6 )
= (y " H (x0|"6b)#$ %&
T
KTM
T(e
t |0+ e
t |"6 )
This requires the adjoint of the model and of the dataassimilation system (Langland and Baker, 2004)
Example: forecast sensitivity to observations
Langland and Baker (2004):
!e2 = MK(y " H (x0|"6b)#$ %&
T
(et |0+ e
t |"6 )
= (y " H (x0|"6b)#$ %&
T
KTM
T(e
t |0+ e
t |"6 )
With EnKF we can use the original equation without “adjointing”:
!e2 = MK(y " H (x0|"6b)#$ %&
T
(et |0 + et |"6 )
= Xt |0
f Y0
aTR"1(y " H (x
0|"6b)#$ %&
T
(et |0 + et |"6 )
K = PaH
TR
!1= X
aX
aTH
TR
!1 so that we can write
MK =MXa(X
aTH
T)R
!1= Xt |0
fY
aTR
!1
This is a simple product using the available nonlinear forecastensemble and
so thatRecall that
Xt |0
fY0
aT= (HP
a)T
Test ability to detect the poor quality observation onthe Lorenz 40 variable model
Like adjoint method, ensemble sensitivity method can detect the observationpoor quality (11th observation location)
The ensemble sensitivity method has a stronger signal when the observation hasnegative impact on the forecast.
Observation impact from LB (red) and from ensemble sensitivity method (green)
Larger random error Biased observation case
Test ability to detect poor quality observation fordifferent forecast lengths
After 2-days theadjoint has the wrongsensitivity sign!
The ensemblesensitivity method hasa strong signal evenafter forecast error hassaturated!
Larger random error Biased observation case
2 days
5 days
20 days
How can we possibly detect bad observations evenafter all skill is lost???
After 20-days there is noforecast skill but theensemble sensitivity stilldetects the wrongobservation.
The ensemble sensitivityis based on the assumptionthat the analysis weightscan be used in theforecasts. This is accurateeven after forecast errorhas saturated (triangles).
As a result we canidentify a bad observationeven after forecast skill islost.
20 days
Mean Square Error of the -6hr weighted forecasts (diamonds),MSE of the 0hr ensemble mean (circles) and MS Differencebetween ensemble mean and weighted forecasts (triangles).
Error made by using the -6hrweights in the forecasts
Summary
• EnKF is simple, does not require linear tangentmodel or adjoint of either the model M or the forwardoperator H
• There are two types of EnKF: perturbed observations(e.g., Evensen, Houtekamer), and Square Root Filter(e.g., Whitaker and Hamill, Anderson, Bishop). Theseassimilate one obs at a time.
• The LETKF (Hunt et al.) is a SQRF done locally inspace: the analysis and analysis perturbations areweighted averages of the ensemble forecasts.
• This makes possible a “no cost smoother”.• EnKF calculates exactly Pa (the Hessian!), and the
Kalman gain matrix K. This makes possible tocompute longer forecasts sensitivity to observations.