ensemble kalman filtering with one-step-ahead smoothing ... · hydraulic head and the conductivity...

40
King Abdullah University of Science and Technology Earth Science & Engineering International workshop on coupled data assimilation Ensemble Kalman Filtering with One-Step-Ahead Smoothing for Efficient Data Assimilation into One-Way Coupled Models Naila Raboudi, B. Ait-El-Fquih, A. Subramanian, I. Hoteit October, 2016

Upload: others

Post on 30-Oct-2019

7 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

King Abdullah University of Science and TechnologyEarth Science & Engineering

International workshop on coupled data assimilation

Ensemble Kalman Filtering with One-Step-Ahead Smoothing forEfficient Data Assimilation into One-Way Coupled Models

Naila Raboudi, B. Ait-El-Fquih, A. Subramanian, I. Hoteit

October, 2016

Page 2: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

Context

Consider a discrete-time state-parameter dynamical system{xn = Mn−1 (xn−1, θ) + ηn−1; ηn−1 ∼ N (0, Qn−1)yn = Hnxn + εn; εn ∼ N (0, Rn)

,

Standard solutions

– Joint EnKF: Apply EnKF on the augmented state zn =[xTn θT

]TSubject to instability and intractability (e.g. Moradkhani et al., 2005)May lead to inconsistency between state and parameters estimates (Wen andChen, 2006)

– Dual-EnKF: Operates as a succession of two separate EnKFs updatingfirst the parameters and then the state.

The separation of the update steps was shown to provide more consistentestimatesHeursitic algorithm, not fully Bayesian consistent

2 International workshop on coupled data assimilation

Page 3: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

Context

Consider a discrete-time state-parameter dynamical system{xn = Mn−1 (xn−1, θ) + ηn−1; ηn−1 ∼ N (0, Qn−1)yn = Hnxn + εn; εn ∼ N (0, Rn)

,

Standard solutions

– Joint EnKF: Apply EnKF on the augmented state zn =[xTn θT

]TSubject to instability and intractability (e.g. Moradkhani et al., 2005)May lead to inconsistency between state and parameters estimates (Wen andChen, 2006)

– Dual-EnKF: Operates as a succession of two separate EnKFs updatingfirst the parameters and then the state.

The separation of the update steps was shown to provide more consistentestimatesHeursitic algorithm, not fully Bayesian consistent

2 International workshop on coupled data assimilation

Page 4: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

Context

Consider a discrete-time state-parameter dynamical system{xn = Mn−1 (xn−1, θ) + ηn−1; ηn−1 ∼ N (0, Qn−1)yn = Hnxn + εn; εn ∼ N (0, Rn)

,

Standard solutions

– Joint EnKF: Apply EnKF on the augmented state zn =[xTn θT

]TSubject to instability and intractability (e.g. Moradkhani et al., 2005)May lead to inconsistency between state and parameters estimates (Wen andChen, 2006)

– Dual-EnKF: Operates as a succession of two separate EnKFs updatingfirst the parameters and then the state.

The separation of the update steps was shown to provide more consistentestimatesHeursitic algorithm, not fully Bayesian consistent

2 International workshop on coupled data assimilation

Page 5: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

Context

{xa,(i)n−1(θ(i)|n−1)}

Ne

i=1{xf1,(i)n }

Ne

i=1

{xa,(i)n−1(θ(i)|n )}

Ne

i=1{xf2,(i)n }

Ne

i=1

{xa,(i)n }Ne

i=1

Mn−1

Mn−1

Analysis x (yn)

Smoothing θ

(yn)

The classical Dual-EnKF for state-parameter estimation

EnKF θEnKF x

Heuristic, not fully Bayesian consistent.

Recently, Ait-El-Fquih et al., (2016) proposed a new Bayesian consistentdual-like filtering scheme, which introduced a new smoothing step between twosuccessive analysis steps.

3 International workshop on coupled data assimilation

Page 6: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

Context

{xa,(i)n−1(θ(i)|n−1)}

Ne

i=1{xf1,(i)n }

Ne

i=1

{xa,(i)n−1(θ(i)|n )}

Ne

i=1{xf2,(i)n }

Ne

i=1

{xa,(i)n }Ne

i=1

Mn−1

Mn−1

Analysis x (yn)

Smoothing θ

(yn)

The classical Dual-EnKF for state-parameter estimation

EnKF θEnKF x

Heuristic, not fully Bayesian consistent.

Recently, Ait-El-Fquih et al., (2016) proposed a new Bayesian consistentdual-like filtering scheme, which introduced a new smoothing step between twosuccessive analysis steps.

3 International workshop on coupled data assimilation

Page 7: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

Context

{xa,(i)n−1(θ(i)|n−1)}

Ne

i=1{xf1,(i)n }

Ne

i=1

{xa,(i)n−1(θ(i)|n )}

Ne

i=1{xf2,(i)n }

Ne

i=1

{xa,(i)n }Ne

i=1

Mn−1

Mn−1

Analysis x (yn)

Smoothing θ

(yn)

The classical Dual-EnKF for state-parameter estimation

EnKF θEnKF x

Heuristic, not fully Bayesian consistent.

Recently, Ait-El-Fquih et al., (2016) proposed a new Bayesian consistentdual-like filtering scheme, which introduced a new smoothing step between twosuccessive analysis steps.

3 International workshop on coupled data assimilation

Page 8: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

Context

{xa,(i)n−1(θ(i)|n−1)}

Ne

i=1{xf1,(i)n }

Ne

i=1

{xa,(i)n−1(θ(i)|n )}

Ne

i=1{xf2,(i)n }

Ne

i=1

{xa,(i)n }Ne

i=1

Mn−1

Mn−1

Analysis x (yn)

Smoothing θ

(yn)

The classical Dual-EnKF for state-parameter estimation

EnKF θEnKF x

Heuristic, not fully Bayesian consistent.

Recently, Ait-El-Fquih et al., (2016) proposed a new Bayesian consistentdual-like filtering scheme, which introduced a new smoothing step between twosuccessive analysis steps.

3 International workshop on coupled data assimilation

Page 9: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

Context

{xa,(i)n−1(θ(i)|n−1)}

Ne

i=1{xf1,(i)n }

Ne

i=1

{xa,(i)n−1(θ(i)|n )}

Ne

i=1{xf2,(i)n }

Ne

i=1

{xa,(i)n }Ne

i=1

Mn−1

Mn−1

Analysis x (yn)

Smoothing θ

(yn)

The classical Dual-EnKF for state-parameter estimation

EnKF θEnKF x

Heuristic, not fully Bayesian consistent.

Recently, Ait-El-Fquih et al., (2016) proposed a new Bayesian consistentdual-like filtering scheme, which introduced a new smoothing step between twosuccessive analysis steps.

3 International workshop on coupled data assimilation

Page 10: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

Context

{xa,(i)n−1(θ(i)|n−1)}

Ne

i=1{xf1,(i)n }

Ne

i=1

{xa,(i)n−1(θ(i)|n )}

Ne

i=1{xf2,(i)n }

Ne

i=1

{xa,(i)n }Ne

i=1

Mn−1

Mn−1

Analysis x (yn)

Smoothing θ

(yn)

The classical Dual-EnKF for state-parameter estimation

EnKF θEnKF x

Heuristic, not fully Bayesian consistent.

Recently, Ait-El-Fquih et al., (2016) proposed a new Bayesian consistentdual-like filtering scheme, which introduced a new smoothing step between twosuccessive analysis steps.

3 International workshop on coupled data assimilation

Page 11: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

Context

{xa,(i)n−1(θ(i)|n−1)}

Ne

i=1{xf1,(i)n }

Ne

i=1

{xa,(i)n−1(θ(i)|n )}

Ne

i=1{xf2,(i)n }

Ne

i=1

{xa,(i)n }Ne

i=1

Mn−1

Mn−1

Analysis x (yn)

Smoothing θ

(yn)

The classical Dual-EnKF for state-parameter estimation

EnKF θEnKF x

Heuristic, not fully Bayesian consistent.

Recently, Ait-El-Fquih et al., (2016) proposed a new Bayesian consistentdual-like filtering scheme, which introduced a new smoothing step between twosuccessive analysis steps.

3 International workshop on coupled data assimilation

Page 12: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

Context

{xa,(i)n−1(θ(i)|n−1)}

Ne

i=1{xf1,(i)n }

Ne

i=1

{xa,(i)n−1(θ(i)|n )}

Ne

i=1{xf2,(i)n }

Ne

i=1

{xa,(i)n }Ne

i=1

Mn−1

Mn−1

Analysis x (yn)

Smoothing θ

(yn)

The classical Dual-EnKF for state-parameter estimation

EnKF θEnKF x

Heuristic, not fully Bayesian consistent.

Recently, Ait-El-Fquih et al., (2016) proposed a new Bayesian consistentdual-like filtering scheme, which introduced a new smoothing step between twosuccessive analysis steps.

3 International workshop on coupled data assimilation

Page 13: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

Context

{xa,(i)n−1(θ(i)|n−1)}

Ne

i=1{xf1,(i)n }

Ne

i=1

{xs,(i)n−1(θ(i)|n )}

Ne

i=1{xf2,(i)n }

Ne

i=1

{xa,(i)n }Ne

i=1

Mn−1

Mn−1

Analysis x (yn)

Smoothing θ

and x (yn)

A Dual-EnKF with OSA smoothing for state-parameter estimation

EnKF θEnKF x

Fully Bayesian consistent

Recently, Ait-El-Fquih et al., (2016) proposed a new Bayesian consistentdual-like filtering scheme based on a one-step-ahead (OSA) formulation, whichintroduces a new smoothing step between two successive analysis steps.

3 International workshop on coupled data assimilation

Page 14: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

context

Successfully tested with a groundwater flow model to estimate thehydraulic head and the conductivity field at KAUST, and with a marineecosystem model at NERSC (Gharamti et al, 2015; 2016)

One-way coupled filtering problems could be considered as ageneralization of the state-parameter estimation problem where theparameter θ evolves according to a dynamical model and is observed.

ObjectiveBuild an efficient Dual-like EnKF scheme for data assimilation intoone-way coupled systems: Separate the updates to tackle inconsistencies,and inject back the ”lost” information through a ”re-forecast” step with thenonlinear model

4 International workshop on coupled data assimilation

Page 15: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

context

Successfully tested with a groundwater flow model to estimate thehydraulic head and the conductivity field at KAUST, and with a marineecosystem model at NERSC (Gharamti et al, 2015; 2016)

One-way coupled filtering problems could be considered as ageneralization of the state-parameter estimation problem where theparameter θ evolves according to a dynamical model and is observed.

ObjectiveBuild an efficient Dual-like EnKF scheme for data assimilation intoone-way coupled systems: Separate the updates to tackle inconsistencies,and inject back the ”lost” information through a ”re-forecast” step with thenonlinear model

4 International workshop on coupled data assimilation

Page 16: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

context

Successfully tested with a groundwater flow model to estimate thehydraulic head and the conductivity field at KAUST, and with a marineecosystem model at NERSC (Gharamti et al, 2015; 2016)

One-way coupled filtering problems could be considered as ageneralization of the state-parameter estimation problem where theparameter θ evolves according to a dynamical model and is observed.

ObjectiveBuild an efficient Dual-like EnKF scheme for data assimilation intoone-way coupled systems: Separate the updates to tackle inconsistencies,and inject back the ”lost” information through a ”re-forecast” step with thenonlinear model

4 International workshop on coupled data assimilation

Page 17: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

Outline

1 Standard EnKF schemes for coupled systems

2 The concept of filtering with OSA smoothing

3 A joint EnKF-OSA for one-way coupled systems

4 Numerical experiments with Lorenz-96

5 Summary

5 International workshop on coupled data assimilation

Page 18: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

Standard solutions for coupled systems

One-way coupled systems{xn = Fn−1(xn−1)+ηxn−1

yxn = Hxnxn+ε

xn

x free variable

{wn = Gn−1(wn−1,xn−1)+ηwn−1

ywn = Hwn wn+εwn

w forced variable

Standard solutions

Standard joint (strong) EnKF

Standard EnKF

yzn =

(yxnywn

)zfn =

(xfnwf

n

)zan =

(xanwa

n

)

Cross-correlations between coupled variables are considered in the update

Cross-correlations may not be well estimated with small ensembles and notvery representative in presence of strong nonlinearities

6 International workshop on coupled data assimilation

Page 19: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

Standard solutions for coupled systems

One-way coupled systems{xn = Fn−1(xn−1)+ηxn−1

yxn = Hxnxn+ε

xn

x free variable

{wn = Gn−1(wn−1,xn−1)+ηwn−1

ywn = Hwn wn+εwn

w forced variable

Standard solutions

Standard weak EnKF

Standard EnKF

yxn (or ywn )

xfn (or wfn) xan (or wa

n)

Separate updates are more practical in real applications

Lost of information from neglecting cross-correlations Lost of informationfrom neglecting cross-correlations

6 International workshop on coupled data assimilation

Page 20: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

The concept of filtering with OSA smoothing

Standard path OSA smoothing based path

Smoothing step:p(xn−1|y0:n) is first computed using the likelihood p(yn|xn−1, y0:n−1)

p(xn−1|y0:n) ∝ p(yn|xn−1, y0:n−1)p(xn−1|y0:n−1)

Analysis step:p(xn|y0:n) is computed using the a posteriori transition p(xn|xn−1, y0:n)

p(xn|y0:n) =∫p(xn|xn−1, y0:n)p(xn−1|y0:n)dxn−1,

7 International workshop on coupled data assimilation

Page 21: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

The concept of filtering with OSA smoothing

Standard path OSA smoothing based path

Smoothing step:p(xn−1|y0:n) is first computed using the likelihood p(yn|xn−1, y0:n−1)

p(xn−1|y0:n) ∝ p(yn|xn−1, y0:n−1)p(xn−1|y0:n−1)

Analysis step:p(xn|y0:n) is computed using the a posteriori transition p(xn|xn−1, y0:n)

p(xn|y0:n) =∫p(xn|xn−1, y0:n)p(xn−1|y0:n)dxn−1,

7 International workshop on coupled data assimilation

Page 22: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

The concept of filtering with OSA smoothing

Two classical stochastic sampling properties are used to derive the jointEnKF-OSA algorithm

Property 1 (Hierarchical sampling)Assuming that one can sample from p(x1) and p(x2|x1), then a sample, x∗2, fromp(x2) can be drawn as follows:

1 x∗1 ; p(x1);

2 x∗2 ; p(x2|x∗1).

Property 2 (Conditional sampling)Consider a Gaussian pdf, p(x,y), with Pxy and Py denoting the cross-covariance ofx and y and the covariance of y, respectively. Then a sample, x∗, from p(x|y), canbe drawn as follows:

1 (x, y) ; p(x,y);

2 x∗ = x+PxyP−1y [y − y].

8 International workshop on coupled data assimilation

Page 23: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

A joint EnKF-OSA for one-way coupled systems

{za,(i)n−1}Ne

i=1{zf1,(i)n }

Ne

i=1

{zs,(i)n−1}Ne

i=1{zf2,(i)n }

Ne

i=1

{za,(i)n }Ne

i=1

Fn−1,Gn−1

Fn−1,Gn−1

Analysis z (yzn)

Smoothing z

(yzn

)

OSA adds one smoothing step and another re-forecast step

Data are exploited twice in a fully Bayesian framework

9 International workshop on coupled data assimilation

Page 24: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

A joint EnKF-OSA for one-way coupled systems

{za,(i)n−1}Ne

i=1{zf1,(i)n }

Ne

i=1

{zs,(i)n−1}Ne

i=1{zf2,(i)n }

Ne

i=1

{za,(i)n }Ne

i=1

Fn−1,Gn−1

Fn−1,Gn−1

Analysis z (yzn)

Smoothing z

(yzn

)

OSA adds one smoothing step and another re-forecast step

Data are exploited twice in a fully Bayesian framework

9 International workshop on coupled data assimilation

Page 25: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

A joint EnKF-OSA for one-way coupled systems

{za,(i)n−1}Ne

i=1{zf1,(i)n }

Ne

i=1

{zs,(i)n−1}Ne

i=1{zf2,(i)n }

Ne

i=1

{za,(i)n }Ne

i=1

Fn−1,Gn−1

Fn−1,Gn−1

Analysis z (yzn)

Smoothing z

(yzn

)

OSA adds one smoothing step and another re-forecast step

Data are exploited twice in a fully Bayesian framework

9 International workshop on coupled data assimilation

Page 26: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

A joint EnKF-OSA for one-way coupled systems

{za,(i)n−1}Ne

i=1{zf1,(i)n }

Ne

i=1

{zs,(i)n−1}Ne

i=1{zf2,(i)n }

Ne

i=1

{za,(i)n }Ne

i=1

Fn−1,Gn−1

Fn−1,Gn−1

Analysis z (yzn)

Smoothing z

(yzn

)

OSA adds one smoothing step and another re-forecast step

Data are exploited twice in a fully Bayesian framework

9 International workshop on coupled data assimilation

Page 27: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

A joint EnKF-OSA for one-way coupled systems

{za,(i)n−1}Ne

i=1{zf1,(i)n }

Ne

i=1

{zs,(i)n−1}Ne

i=1{zf2,(i)n }

Ne

i=1

{za,(i)n }Ne

i=1

Fn−1,Gn−1

Fn−1,Gn−1

Analysis z (yzn)

Smoothing z

(yzn

)

OSA adds one smoothing step and another re-forecast step

Data are exploited twice in a fully Bayesian framework

9 International workshop on coupled data assimilation

Page 28: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

A joint EnKF-OSA for one-way coupled systems

{za,(i)n−1}Ne

i=1{zf1,(i)n }

Ne

i=1

{zs,(i)n−1}Ne

i=1{zf2,(i)n }

Ne

i=1

{za,(i)n }Ne

i=1

Fn−1,Gn−1

Fn−1,Gn−1

Analysis z (yzn)

Smoothing z

(yzn

)

OSA adds one smoothing step and another re-forecast step

Data are exploited twice in a fully Bayesian framework

9 International workshop on coupled data assimilation

Page 29: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

An EnKF-OSA with separate updates for one-way coupled systems

Assumption to separate the update steps

Given the past observations of both x and w, wn needs to be independent of the current andfuture observations of the variable x.

EnKF free variable EnKF forced variable

{xa,(i)n−1}Ne

i=1{xf1,(i)n }

Ne

i=1 {wa,(i)n−1 }

Ne

i=1{wf1,(i)

n }Ne

i=1

{xs,(i)n−1}Ne

i=1{xf2,(i)n }

Ne

i=1 {ws,(i)n−1}

Ne

i=1{wf2,(i)

n }Ne

i=1

{xa,(i)n }Ne

i=1 {wa,(i)n }

Ne

i=1

Fn−1

Fn−1

A of x (yxn)

S of x(yxn)

Gn−1

Gn−1

A of w (ywn )

S of w(ywn

)S of x (ywn )

A of x (ywn)

The free variable x is updated by both yxn and ywn while the forced variable w is updatedby ywn only.

10 International workshop on coupled data assimilation

Page 30: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

An EnKF-OSA with separate updates for one-way coupled systems

Assumption to separate the update steps

Given the past observations of both x and w, wn needs to be independent of the current andfuture observations of the variable x.

EnKF free variable EnKF forced variable

{xa,(i)n−1}Ne

i=1{xf1,(i)n }

Ne

i=1 {wa,(i)n−1 }

Ne

i=1{wf1,(i)

n }Ne

i=1

{xs,(i)n−1}Ne

i=1{xf2,(i)n }

Ne

i=1 {ws,(i)n−1}

Ne

i=1{wf2,(i)

n }Ne

i=1

{xa,(i)n }Ne

i=1 {wa,(i)n }

Ne

i=1

Fn−1

Fn−1

A of x (yxn)

S of x(yxn)

Gn−1

Gn−1

A of w (ywn )

S of w(ywn

)S of x (ywn )

A of x (ywn)

The free variable x is updated by both yxn and ywn while the forced variable w is updatedby ywn only.

10 International workshop on coupled data assimilation

Page 31: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

A weak version for one-way coupled systems

Assumption to separate the update steps

Given the past observations of both x and w, wn needs to be independent of the current andfuture observations of the variable x.

EnKF free variable EnKF forced variable

{xa,(i)n−1}Ne

i=1{xf1,(i)n }

Ne

i=1 {wa,(i)n−1 }

Ne

i=1{wf1,(i)

n }Ne

i=1

{xs,(i)n−1}Ne

i=1{xf2,(i)n }

Ne

i=1 {ws,(i)n−1}

Ne

i=1{wf2,(i)

n }Ne

i=1

{xa,(i)n }Ne

i=1 {wa,(i)n }

Ne

i=1

Fn−1

Fn−1

A of x (yxn)

S of x(yxn)

Gn−1

Gn−1

A of w (ywn )

S of w(ywn

)

The weak version is derived by simply neglecting the cross-correlations and updatingeach variable by its own observations using an EnKF-OSA on each model

11 International workshop on coupled data assimilation

Page 32: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

EnKF Vs EnKF-OSA

EnKF OSA filtering uses an alternative path to better exploit the data (useful innon-optimal ensemble implementation)

Constraining the state with future observations in the smoothing step shouldprovide improved background for the next analysis

=⇒ Mitigate some of the background limitations of EnKF-like methods

EnKF-OSA conditions the ensemble resampling with future data

May help mitigating for the loss of information in a weak formulation, by betterexploiting of the data and re-forecasting with the nonlinear model.

OSA algorithm is computationally (almost twice) more expensive.

12 International workshop on coupled data assimilation

Page 33: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

Numerical experiments with Lorenz-96

Governing equations

dxidt

= (xi+1 − xi−2)xi−1 − xi + F, i = 1, · · · , Nx

dwj,i

dt= (wj−1,i − wj+2,i) cbwj+1,i − cwj,i +

hc

bxi, j = 1, · · · , Nw

Nx = 40; Nw = 1;

Experimental setupTwin experiments5-years simulation periodTime step: 0.005

Assimilation scenarios3 different observation scenarios: all (40), half (20), and quarter (10)model state variables are observed3 different ensemble sizes: 10, 20 and 40Observations are assimilated every 1 day

13 International workshop on coupled data assimilation

Page 34: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

Numerical experiments with Lorenz-96

Reference states for both models

0 5000 10000 15000

-10

-5

0

5

10

15

Sta

te v

ari

ab

les

x10

w1,10

0 5000 10000 15000

-10

-5

0

5

10

15

x20

w1,20

0 5000 10000 15000

Time steps

-10

-5

0

5

10

15

Sta

te v

ari

ab

le x30

w1,30

0 5000 10000 15000

Time steps

-10

-5

0

5

10

15

x40

w1,40

We only analyze the free variable x: assimilation results are consistentfor both variables, but improvements are more pronounced with x

14 International workshop on coupled data assimilation

Page 35: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

40 members and assimilation every 24h

EnKF-Strong EnKF-Strong-OSA EnKF-Weak EnKF-Weak-OSA

10 20 30 40

1

1.1

1.2

1.3

1.4

In

fla

tio

n

EnKF-Strong, All Obs

Min = 0.39

0.35

0.4

0.6

1

10 20 30 40

1

1.1

1.2

1.3

1.4

EnKF-OSA-Strong, All Obs

Min = 0.36

0.35

0.4

0.6

1

10 20 30 40

1

1.1

1.2

1.3

1.4

In

fla

tio

n

EnKF-Strong, Half Obs

Min = 0.71

0.6

1

2

10 20 30 40

1

1.1

1.2

1.3

1.4

EnKF-OSA-Strong, Half Obs

Min = 0.63

0.6

1

2

10 20 30 40

Localization

1

1.1

1.2

1.3

1.4

In

fla

tio

n

EnKF-Strong, Quarter Obs

Min = 1.08

0.85

1

3

10 20 30 40

Localization

1

1.1

1.2

1.3

1.4

EnKF-OSA-Strong, Quarter Obs

Min = 0.88

0.85

1

3

10 20 30 40

1

1.1

1.2

1.3

1.4

In

fla

tio

n

EnKF-Weak, All Obs

Min = 0.39

0.35

0.4

0.6

1

10 20 30 40

1

1.1

1.2

1.3

1.4

EnKF-OSA-Weak, All Obs

Min = 0.36

0.35

0.4

0.6

1

10 20 30 40

1

1.1

1.2

1.3

1.4

In

fla

tio

n

EnKF-Weak, Half Obs

Min = 0.71

0.6

1

2

10 20 30 40

1

1.1

1.2

1.3

1.4

EnKF-OSA-Weak, Half Obs

Min = 0.63

0.6

1

2

10 20 30 40

Localization

1

1.1

1.2

1.3

1.4

In

fla

tio

n

EnKF-Weak, Quarter Obs

Min = 1.09

0.85

1

3

10 20 30 40

Localization

1

1.1

1.2

1.3

1.4

EnKF-OSA-Weak, Quarter Obs

Min = 0.94

0.85

1

3

EnKF-Strong (OSA/Reg) EnKF-Weak (OSA/Reg)Ne = 40 Ne = 40

all 8.2 % 8.1 %half 10.5 % 10.4 %

quarter 19.2 % 14.1 %

15 International workshop on coupled data assimilation

Page 36: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

20 members and assimilation every 24h

EnKF-Strong EnKF-Strong-OSA EnKF-Weak EnKF-Weak-OSA

10 20 30 40

1

1.1

1.2

1.3

1.4 In

flati

on

EnKF−Strong, All Obs

Min = 0.43

0.35

0.5

1

2

10 20 30 40

1

1.1

1.2

1.3

1.4

EnKF−OSA pseudo, All Obs

Min = 0.39

0.35

0.5

1

2

10 20 30 40

1

1.1

1.2

1.3

1.4

In

flati

on

EnKF−Strong, Half Obs

Min = 0.79

0.7

1

1.5

2

3

10 20 30 40

1

1.1

1.2

1.3

1.4

EnKF−OSA pseudo, Half Obs

Min = 0.72

0.7

1

1.5

2

3

10 20 30 40

1

1.1

1.2

1.3

1.4

Localization

In

flati

on

EnKF−Strong, Quarter Obs

Min = 1.24

1.05

1.5

2

2.5

3

10 20 30 40

1

1.1

1.2

1.3

1.4

Localization

EnKF−OSA pseudo, Quarter Obs

Min = 1.07

1.05

1.5

2

2.5

3

10 20 30 40

1

1.1

1.2

1.3

1.4

In

flati

on

EnKF−Weak, All Obs

Min = 0.43

0.35

0.5

1

2

10 20 30 40

1

1.1

1.2

1.3

1.4

EnKF−OSA−Weak, All Obs

Min = 0.38

0.35

0.5

1

2

10 20 30 40

1

1.1

1.2

1.3

1.4

In

flati

on

EnKF−Weak, Half Obs

Min = 0.79

0.7

1

1.5

2

3

10 20 30 40

1

1.1

1.2

1.3

1.4

EnKF−OSA−Weak, Half Obs

Min = 0.71

0.7

1

1.5

2

3

10 20 30 40

1

1.1

1.2

1.3

1.4

Localization

In

flati

on

EnKF−Weak, Quarter Obs

Min = 1.23

1.05

1.5

2

2.5

3

10 20 30 40

1

1.1

1.2

1.3

1.4

Localization

EnKF−OSA−Weak, Quarter Obs

Min = 1.07

1.05

1.5

2

2.5

3

EnKF-Strong (OSA/Reg) EnKF-Weak (OSA/Reg)Ne = 20 Ne = 20

all 9 % 11.2 %half 10.2 % 10.8 %

quarter 13.5 % 13.6 %

16 International workshop on coupled data assimilation

Page 37: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

10 members and assimilation every 24h

EnKF-Strong EnKF-Strong-OSA EnKF-Weak EnKF-Weak-OSA

10 20 30 40

1

1.1

1.2

1.3

1.4

In

fla

tio

n

EnKF-Strong, All Obs

Min = 0.51

0.4

1

3

10 20 30 40

1

1.1

1.2

1.3

1.4

EnKF-OSA-Strong, All Obs

Min = 0.47

0.4

1

3

10 20 30 40

1

1.1

1.2

1.3

1.4

In

fla

tio

n

EnKF-Strong, Half Obs

Min = 0.99

0.85

1

3

10 20 30 40

1

1.1

1.2

1.3

1.4

EnKF-OSA-Strong, Half Obs

Min = 0.91

0.85

1

3

10 20 30 40

Localization

1

1.1

1.2

1.3

1.4

In

fla

tio

n

EnKF-Strong, Quarter Obs

Min = 1.61

1.3

2

4

10 20 30 40

Localization

1

1.1

1.2

1.3

1.4

EnKF-OSA-Strong, Quarter Obs

Min = 1.44

1.3

2

4

10 20 30 40

1

1.1

1.2

1.3

1.4

In

fla

tio

n

EnKF-Weak, All Obs

Min = 0.50

0.4

1

3

10 20 30 40

1

1.1

1.2

1.3

1.4

EnKF-OSA-Weak, All Obs

Min = 0.45

0.4

1

3

10 20 30 40

1

1.1

1.2

1.3

1.4

In

fla

tio

n

EnKF-Weak, Half Obs

Min = 0.98

0.85

1

3

10 20 30 40

1

1.1

1.2

1.3

1.4

EnKF-OSA-Weak, Half Obs

Min = 0.89

0.85

1

3

10 20 30 40

Localization

1

1.1

1.2

1.3

1.4

In

fla

tio

n

EnKF-Weak, Quarter Obs

Min = 1.61

1.3

2

4

10 20 30 40

Localization

1

1.1

1.2

1.3

1.4

EnKF-OSA-Weak, Quarter Obs

Min = 1.34

1.3

2

4

EnKF-Strong (OSA/Reg) EnKF-Weak (OSA/Reg)Ne = 10 Ne = 10

all 6.3 % 9.6 %half 8 % 9.8 %

quarter 10.5 % 17 %

17 International workshop on coupled data assimilation

Page 38: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

Sensitivity to ensemble size

Quarter of the observations are assimilated

Ne = 10 Ne = 20 Ne = 400.8

1

1.2

1.4

1.6

1.8

2Min RMSE

EnKF−StrongEnKF−OSA−Strong

EnKF−weakEnKF−OSA−weak

Slight improvement with same computational cost; need to examinelarger ensembles

18 International workshop on coupled data assimilation

Page 39: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

Summary

EnKF-OSA (weak and strong formulations) is found beneficial for one-waycoupled DA compared to the standard EnKF

EnKF-OSA produces better estimates when the number of observations issmaller

The weak formulation is more beneficial when cross-correlations are not wellestimated, otherwise, better use a strong formulation

The strong OSA version was shown to be more beneficial than the weak OSAwhen the ensemble is enough representative

At the same computational cost and large enough ensemble, EnKF-OSA seemsto outperform the standard EnKF, but, more tests are needed

19 International workshop on coupled data assimilation

Page 40: Ensemble Kalman Filtering with One-Step-Ahead Smoothing ... · hydraulic head and the conductivity field at KAUST, and with a marine ecosystem model at NERSC (Gharamti et al, 2015;

Thank you for your attention

20 International workshop on coupled data assimilation