an earth system model based on the ecmwf integrated forecasting system ifs-model what is the ifs?...

An Earth System Model based on the ECMWF Integrated Forecasting System

IFS-model

•What is the IFS?•Governing equations•Dynamics and physics•Numerical implementation•Overview of the code

What is the IFS?• Total software package at ECMWF:

• Data-assimilation to get initial condition• Make weather forecasts• Make ensemble forecasts• Monthly forecasts• Seasonal forecasts• Atmosphere, ocean, land, wave models• Has a long history that is present in the

code ….

Numerical Weather Prediction• The behaviour of the atmosphere is governed by a set of physical

laws

• Equations cannot be solved analytically, numerical methods are needed

• Additionally, knowledge of initial conditions of system necessary

• Incomplete picture from observations can be completed by data assimilation

• Interactions between atmosphere and land/ocean important

ECMWF’s operational analysis and forecasting systemThe comprehensive earth-system model developed at ECMWF forms the basis for all the data assimilation and forecasting activities. All the main applications required are available through one integrated computer software system (a set of computer programs written in Fortran) called the

Integrated Forecast System or IFS

• Numerical scheme: TL799L91 (799 waves around a great circle on the globe, 91 levels 0-80 km) semi-Lagrangian formulation 1,630,000,000,000,000 computations required for each 10-day forecast

• Time step: 12 minutes

• Prognostic variables: wind, temperature, humidity, cloud fraction and water/ice content, pressure at surface grid-points, ozone

• Grid: Gaussian grid for physical processes, ~25 km, 76,757,590 grid points

A history

• Resolution increases of the deterministic 10-day medium-range Integrated Forecast System (IFS) over ~25 years at ECMWF:

– 1987: T 106 (~125km)

– 1991: T 213 (~63km)

– 1998: TL319 (~63km)

– 2000: TL511 (~39km)

– 2006: TL799 (~25km)

– 2010: TL1279 (~16km)

– 2015?: TL2047 (~10km)

– 2020-???: (~1-10km) Non-hydrostatic, cloud-permitting, substan-tially different cloud-microphysics and turbulence parametrization, substantially different dynamics-physics interaction ?

Ultra-high resolution global IFS simulations

• TL0799 (~ 25km) >> 843,490 points per field/level• TL1279 (~ 16km) >> 2,140,702 points per field/level• TL2047 (~ 10km) >> 5,447,118 points per field/level• TL3999 (~ 5km) >> 20,696,844 points per field/level

(world record for spectral model ?!)

Orography – T1279Max global altitude = 6503m

Alps

Orography - T3999

Alps

Max global altitude = 7185m

Deterministic model grid (T799)

EPS model grid (T399)

The wave model

• Coupled ocean wave model (WAM cycle4)

2 versions: global and regional (European Shelf & Mediterranean)

numerical scheme: irregular lat/lon grid, 40 km spacing; spectrum with 30 frequencies and 24 directions

coupling: wind forcing of waves every 15 minutes, two way interaction of winds and waves, sea state dep. drag coefficient

extreme sea state forecasts: freak waves

wave model forecast results can be used as a tool to diagnose problems in the atmospheric model

Numerical Methods and Adiabatic Formulation of Models

30 March - 3 April 2009

Physical aspects, included in IFS• Orography (terrain height and sub-grid-scale characteristics) • Four surface and sub-surface levels (allowing for vegetation cover, gravitational drainage, capillarity exchange, surface / sub-surface runoff)• Stratiform and convective precipitation• Carbon dioxide (345 ppmv fixed), aerosol, ozone• Solar angle• Diffusion • Ground & sea roughness • Ground and sea-surface temperature • Ground humidity• Snow-fall, snow-cover and snow melt • Radiation (incoming short-wave and out-going long-wave)• Friction (at surface and in free atmosphere)• Sub-grid-scale orographic drag • Gravity waves and blocking effects • Evaporation, sensible and latent heat flux

Parameterization of Diabatic Processes

11 – 21 May 2009

Starting a forecast: The initial conditions

Data Assimilation• Observations measure the current state, but provide an incomplete picture

Observations made at irregularly spaced points, often with large gaps

Observations made at various times, not all at ‘analysis time’

Observations have errors

Many observations not directly of model variables

• The forecast model can be used to process the observations and produce a more complete picture (data assimilation)

start with previous analysis

use model to make short-range forecast for current analysis time

correct this ‘background’ state using the new observations

Observations

“True” state of the atmosphere

Model vari

able

s, e

.g.

tem

pera

ture

00 UTC 5 May

Analysis

Background Analysis

12 UTC 5 May

00 UTC 6 May

12 UTC 6 May

12-h

our fo

reca

st

Data Assimilation

Every 12 hours ~ 60 million observations are processed

to correct the 8 million numbers that define the model’s virtual atmosphere

Data Assimilation• Observations measure the current state, but provide an incomplete picture

Observations made at irregularly spaced points, often with large gaps

Observations made at various times, not all at ‘analysis time’

Observations have errors

Many observations not directly of model variables

• The forecast model can be used to process the observations and produce a more complete picture (data assimilation)

start with previous analysis

use model to make short-range forecast for current analysis time

correct this ‘background’ state using the new observations

• The forecast model is very sensitive to small differences in initial conditions

accurate analysis crucial for accurate forecast

EPS used to represent the remaining analysis uncertainty

What is an ensemble forecast?

Forecast time

Tem

pera

ture

Complete description of weather prediction in terms of aProbability Density Function (PDF)

Initial condition Forecast

Flow dependence of forecast errors

If the forecasts are coherent (small spread) the atmosphere is in a more predictable state than if the forecasts diverge (large spread)

aa

34

30

28

26

24

22

20

18

16

14

12

10

0 1 2 3 4 5 6 7 8 9 10Forecast day

UK

Control Analysis Ensemble

ECMWF ensemble forecast - Air temperatureDate: 26/06/1994 London Lat: 51.5 Long: 0

30

28

26

24

22

20

18

16

14

12

10

8

0 1 2 3 4 5 6 7 8 9 10Forecast day

UK

Control Analysis Ensemble

ECMWF ensemble forecast - Air temperatureDate: 26/06/1995 London Lat: 51.5 Long: 0

26th June 1995 26th June 1994

ECMWF’s Ensemble Prediction Systems

• Account for initial uncertainties by running ensemble of forecasts from slightly different initial conditions singular vector approach to sample perturbations

• Model uncertainties are represented by “stochastic physics”

• Medium-range VarEPS (15-day lead) runs twice daily (00 and 12 UTC) day 0-10: TL399L62 (0.45°, ~50km), 50+1 members

day 9-15: TL255L62 (0.7°, ~80km), 50+1 members

• Extended time-range EPS systems: monthly and seasonal forecasts coupled atmosphere-ocean model (IFS & HOPE) monthly forecast (4 weeks lead) runs once a week seasonal forecast (6 months lead) runs once a month

Principal Goal

Forecast errorsTwo principal sources of forecast error:

Uncertainties in the initial conditions (“observational error”)

Model error

Two kinds of forecast error

Random error (model+initial error)

Systematic error (model error*)

Systematic Error Growth

How do systematic errors grow throughout the forecast?

Systematic Z500 Error Growth from D+1 to D+10

(a) Z500 Difference D+1 FC-AN (DJF 1990-2001 ERA Interim)

-1-0.8-0.6-0.4-0.3-0.2-0.1

0.10.20.30.40.60.81

(b) Z500 Difference D+3 FC-AN (DJF 1990-2001 ERA Interim)

-3-2.4-1.8-1.2-0.9-0.6-0.3

0.30.60.91.21.82.43

(c) Z500 Difference D+5 FC-AN (DJF 1990-2001 ERA Interim)

-5-4-3-2-1.5-1-0.5

0.511.52345

(d) Z500 Difference D+10 FC-AN (DJF 1990-2001 ERA Interim )

-10-8-6-4-3-2-1

12346810

Systematic Z500 Errors: Medium-Range and Beyond

-2

Z500 Difference 32R1-er40 (12-2 1963-2006)

-14

-12

-10

-8

-6

-4

-22

4

6

8

10

12

14

(a) Z500 Difference D+1 FC-AN (DJF 1990-2001 ERA Interim)

-1-0.8-0.6-0.4-0.3-0.2-0.1

0.10.20.30.40.60.81

(b) Z500 Difference D+3 FC-AN (DJF 1990-2001 ERA Interim)

-3-2.4-1.8-1.2-0.9-0.6-0.3

0.30.60.91.21.82.43

(c) Z500 Difference D+5 FC-AN (DJF 1990-2001 ERA Interim)

-5-4-3-2-1.5-1-0.5

0.511.52345

(d) Z500 Difference D+10 FC-AN (DJF 1990-2001 ERA Interim )

-10-8-6-4-3-2-1

12346810

D+10 ERA-Interim Asymptotic: 31R2

Evolution of Systematic Error

How did systematic errors evolve throughout the years?

Evolution of D+3 Systematic Z500 Errors

1983-1987 1993-1997 2003-2007

5440

5600

Mean Z500 Error DJF 1983-1987

-50

-35

-25

-20

-15

-10

-55

10

15

20

25

35

50

5440

5440

5600


-50

-35

-25

-20

-15

-10

-55

10

15

20

25

35

50

5600

5760


-50

-35

-25

-20

-15

-10

-55

10

15

20

25

35

50

Evolution of Systematic Z500 Errors: Model Climate

-2

Z500 Difference 35R1-er40 (12-2 1963-2006)

-14

-12

-10

-8

-6

-4

-22

4

6

8

10

12

14

Z500 Difference 33R1-er40 (12-2 1963-2006)

-14

-12

-10

-8

-6

-4

-22

4

6

8

10

12

14

2

Z500 Difference 32R3-er40 (12-2 1963-2006)

-14

-12

-10

-8

-6

-4

-22

4

6

8

10

12

14

2

6

Z500 Difference 32R2-er40 (12-2 1963-2006)

-14

-12

-10

-8

-6

-4

-22

4

6

8

10

12

14

-2

Z500 Difference 32R1-er40 (12-2 1963-2006)

-14

-12

-10

-8

-6

-4

-22

4

6

8

10

12

14

-6

Z500 Difference 31R1-er40 (12-2 1963-2006)

-14

-12

-10

-8

-6

-4

-22

4

6

8

10

12

14

-2

6

Z500 Difference 30R1-er40 (12-2 1963-2006)

-14

-12

-10

-8

-6

-4

-22

4

6

8

10

12

14

-2

Z500 Difference 29R2-er40 (12-2 1963-2006)

-14

-12

-10

-8

-6

-4

-22

4

6

8

10

12

14

35R1 33R1 32R3 32R2

32R1 31R1 30R1 29R2

Systematic Z500 Errors: Impact of Recent Changes

Z500 Difference f127-er40 (12-2 1990-2005)

-14

-12

-10

-8

-6

-4

-22

4

6

8

10

12

14

-2

6

Z500 Difference f3wt-er40 (12-2 1990-2005)

-14

-12

-10

-8

-6

-4

-22

4

6

8

10

12

14

2

Z500 Difference f251-er40 (12-2 1990-2005)

-14

-12

-10

-8

-6

-4

-22

4

6

8

10

12

14

Z500 Difference f3y0-er40 (12-2 1990-2005)

-14

-12

-10

-8

-6

-4

-22

4

6

8

10

12

14


-14

-12

-10

-8

-6

-4

-22

4

6

8

10

12

14

-2


-14

-12

-10

-8

-6

-4

-22

4

6

8

10

12

14

Control Old Convection Old TOFD

Old Vertical Diff Old Radiation Old Soil Hydrology

Recent Model Changes

29R2

28 Jun 2005

Modifications to convection

30R1

1 Feb 2006

Increased resolution (L60 to L91)

31R1

12 Sep 2006

Revised cloud scheme (ice supersaturation + numerics); implicit computation of convective transports; introduction of orographic form drag scheme; revised GWD scheme

32R1

not operational

New short-wave radiation scheme; introduction of McICA cloud-radiation interaction; MODIS aerosol; revised GWD scheme; retuned ice particle size

32R2

5 Jun 2007 Minor changes to forecast model

32R3

6 Nov 2007

New formulation of convective entrainment and relaxation time scale; reduced vertical diffusion in the free atmosphere; modification to GWD scheme (top of the model); new soil hydrology scheme

33R1

3 Jun 2008 Slightly increased vertical diffusion; increased orographic form drag; retuned entrainment in convection scheme; bugfix scaling of freezing term in convection scheme; changes to the surface model


IFS-model


Primitive (hydrostatic) equations in IFS

forMomentum equations

Sub-grid model :“physics”

Numerical diffusion

IFS hydrostatic equations

Thermodynamic equation

Moisture equation

Note: virtual temperature Tv instead of T from the equation of state.

IFS hydrostatic equations

Continuity equation

dp

pp

t hs

s

1

0.

1)ln( v

Vertical integration of the continuity equation in hybrid coordinates

One word about water species….• Phase changes are treated inside the “physics” (P terms)

• But the prognostic water species have a weight. They are included in the full density of the moist air and in the definition of the “specific” variables. It does some “tricky” changes in the equations. For ex. :

• Prognostic water species should be advected. They are then also treated by the dynamics.

k

ilvdk mmmmmVm scondensate pronostic

... with /

)1(1or )1(1with

or or

v

avv

v

avd

vdvdd

M

MqTT

M

MqRR

TRpRTpRvTTRp Perfect gas equation

mmq kk /


IFS-model


Resolution problem

• So far : We derived a set of evolution equations based on 3 basic conservation principles valid at the scale of the continuum : continuity equation, momentum equation and thermodynamic equation.

• What do we want to (re-)solve in models based on these equations?

grid

sca

lere

solv

ed s

cale

(?)

The scale of the grid is much bigger than the

scale of the continuum

“Averaged” equations : from the scale of the continuum to the mean grid size scale

• The equations as used in an operational NWP model represent the evolution of a space-time average of the true solution.

• The equations become empirical once averaged, we cannot claim we are solving the fundamental equations.

• Possibly we do not have to use the full form of the exact equations to represent an averaged flow, e.g. hydrostatic approximation OK for large enough averaging scales in the horizontal.

Different scales involved

NH-effects visible

“Averaged” equations

• The sub-grid model represents the effect of the unresolved scales on the averaged flow expressed in terms of the input data which represents an averaged state.

• The mean effects of the subgrid scales has to be parametrised.

• The average of the exact solution may *not* look like what we expect, e.g. since vertical motions over land may contain averages of very large local values.

• The averaging scale does not correspond to a subset of observed phenomena, e.g. gravity waves are partly included at TL799, but will not be properly represented.

Physics – Dynamics coupling

• ‘Physics’, parametrization: “the mathematical procedure describing the statistical effect of subgrid-scale processes on the mean flow expressed in terms of large scale parameters”, processes are typically: vertical diffusion, orography, cloud processes, convection, radiation

• ‘Dynamics’: “computation of all the other terms of the Navier-Stokes equations (eg. in IFS: semi-Lagrangian advection)”

• The ‘Physics’ in IFS is currently formulated inherently hydrostatic, because the parametrizations are formulated as independent vertical columns on given pressure levels and pressure is NOT changed directly as a result of sub-gridscale interactions !

• The boundaries between ‘Physics’ and ‘Dynamics’ are “a moving target” …


IFS-model


Fundamentals of series-expansion methodsFundamentals of series-expansion methods

0)(

fHt

f

)(),( 00 xftxf

(1)

(1a)

We demonstrate the fundamentals of series-expansion methods on the followingproblem for which we seek solutions:

Partial differential equation:(with an operator H involvingonly derivatives in space.)

Initial condition:

Boundary conditions: Solution f has to fulfil some specified conditions on the boundary of the domain S.

To be solved on the spatial domain S subject to specified initial and boundary conditions.

(3)

Fundamentals of series-expansion methods (cont)Fundamentals of series-expansion methods (cont)

1

)()(),(i

ii xtatxf

The basic idea of all series-expansion methods is to write the spatial dependenceof f as a linear combination of known expansion functions )(xi

1)( ii x should span the L2 space, i.e. a HilbertThe set of expansion function

space with the inner (or scalar) product of two functions defined as

dxxfxgfgS

)()(, * (4)

The expansion functions should all satisfy the required boundary conditions.


)(),...,(1 tata N

The task of solving (1) has been transformed into the problem of finding the unknown coefficients

in a way that minimises the error in the approximate solution.

N

iii xtatxf

1

)()(),(ˆ

Numerically we can’t handle infinite sums. Limit the sum to a finite numberof expansion terms N

(3a)

f̂ is only an approximation to the true solution f of the equation (1).

=>

Transform equation (1) into series-expansion form:

NjallforfRfHft jjj ,...,10)ˆ(,)ˆ(,ˆ,

=0 (Galerkin approximation)

=> NjallfordxaHdt

dadx iS

N

iij

N

i

i

S ij ,...,10)(1

*

1

*

(9)


)ˆ()ˆ(ˆ

fRfHt

f

(5)Start from equation (5) (equivalent to (1))

Take the scalar product of this equation with all the expansion functions and apply the Galerkin approximation:

The Spectral Method on the SphereThe Spectral Method on the Sphere

Spherical Harmonics Expansion

Spherical geometry: Use spherical coordinates: longitude

latitude

, sin

),(),( ,, nmm mn

nm Yaf

Any horizontally varying 2d scalar field can be efficiently represented in spherical geometry by a series of spherical harmonic functions Ym,n:

imnmnm ePY )(),( ,,

associated Legendre functions Fourier functions

(40)

(41)

The Spectral Method on the SphereThe Spectral Method on the Sphere

Definition of the Spherical Harmonics

imnmnm ePY )(),( ,,Spherical harmonics

0),()1()!(

)!(

2

)12()( 2/2

2/1

,

mPd

d

mn

mnnP nm

mm

nm

The associated Legendre functions Pm,n are generated from the Legendre Polynomials via the expression

nn

n

nn d

d

nP )1(

!2

1)( 2

Where Pn is the “normal” Legendre

polynomial of order n defined by

This definition is only valid for ! nm

(41)

(42)

(43)

nmm

nm PP ,, )1(

The Spectral Method on the SphereThe Spectral Method on the SphereSome Spherical Harmonics for n=5

Schematic representation of the spectral transform method in the ECMWF model

Grid-point space -semi-Lagrangian advection -physical parametrizations -products of terms

Fourier space

Spectral space -horizontal gradients -semi-implicit calculations -horizontal diffusion

FFT

LT

Inverse FFT

Inverse LT

Fourier space

FFT: Fast Fourier Transform, LT: Legendre Transform

Cubic B-splines as basis elements

Basis elementsfor the represen-tation of thefunction tobe integrated(integrand)

f

Basis elementsfor the representationof the integral

F

• High order accuracy (8th order for cubic elements)• Very accurate computation of the pressure-gradient

term in conjunction with the spectral computation of horizontal derivatives

• More accurate vertical velocity for the semi-Lagrangian trajectory computation

• Reduced vertical noise in the model• No staggering of variables required in the vertical: good for

semi-Lagrangian scheme because winds and advected variables are represented on the same vertical levels.

Benefits from using this finite-element scheme in the vertical in the ECMWFmodel:

The Finite-Element Scheme in the ECMWF modelThe Finite-Element Scheme in the ECMWF model

Advection: The semi-Lagrangian technique

dt

d

xu

tviewofpointEul

.

material time derivativeor time evolution along a trajectory thus avoiding quadratic terms;

x

x

x

x

x

x

x

x

x

xx

x

xx

x

xxx

From a regular array of pointswe end up after Δt with a non-regular distribution

Semi-Lagrangian: (usually) tracking back

Solution of the one-dimensional advection equation:

1*0 n n

j

u u du

uu

t xu

dt

t

uu

0

origin point interpolation

udt

dx

computing the origin pointvia trajectory calculation

disadvantage: not flux-form!

Example in one dimensionLinear advection equation without r.h.s.

00

xU

tdt

d

jx

p

αOrigin of parcel at j: X* =Xj-U0Δt“multiply-upstream”

xptU )(0 p: integer

Linear interpolation npj

npj

n1)1(

npj

npj

nj 1

1 )1(

α is not the CFL number except when p=0, then=> upwind

Von Neumann: xikjnnj e 0 xipkxik ee )1(1

)]cos(1)[1(212

xk |λ|≤1 if 0 ≤α ≤1(interpolation fromtwo nearest points)

Damping!

e.g. McDonald (1987)

Single timestep in two-time-level-scheme


IFS-model


IFS repository : a versioned object base or VOB

IFS source code management

Naming of IFS code versions

q2 select_client to get IFS code

List of IFS directory

List of ifs/adiab

List of ifs/phys_ec

“French” cptend.F90

List of lapinea.F90

Flow chart IFS code

MasterCnt0 (evaluates NCONF to decide which type of action)Cnt1 (pertubs observations)Cnt2Cnt3 (does initialization)Cnt4: calls STEP0 in time loop,

spch: spectral space computations scan2H->scan2MDM (interfaces to gridpoint

computationsGP_model: gridpoint computationsEC_phys: callpar.F90

Callpar.F90

Why is an adjoint model useful?

Suppose we are dealing with a nonlinear model M of the form:

y=M(x)

and a differentiable scalar J defined for model output fields y :

J=J(y)=J(M(x))

Dependence of J on y is often straightforward,

but determining seems impossible for high-dimensional models.

It would require perturbed model runs for every (~108 ) entry of x.

xJ /

Example 0: Sensitivity calculation (method to improve a forecast retrospectively)

y=M(x)

x=analysis1 analysis2

perturbed analysis1

J?

J(x)=[y-analysis2,y-analysis2], with [.,.] a suitable inner product

See: Rabier et al. (1996), Klinker et al. (1998), …… ,…, Isaksen et al. (2005), Caron et al. (2006),…

(J=0)

How to minimize J ?

j

k

k

M

kj x

y

y

J

x

J

1

JJ yT

x M

Application of the chain rule learns that

Assume that a small perturbation yj of yj is associated to a smallperturbation xkof xk through:

and consequently

jdefkk k

jj x

x

yy )x( M

How to determine MT ?

Assume that the linear model describing the evolution of initial time perturbations has the form

(1) ddt = L

with propagator M: (t2)=M(t1,t2)(t1)

Define the adjoint model by

(2) d/dt = -LTwith [La,b] = [a,L

Tb],

with propagator S and where [.,.] is a suitable inner product.

N.B. Adjoint model depends on chosen inner product [.,.].

Solutions a(t) and b(t) of (1) and (2) respectively satisfy the property:

d/dt [a(t),b(t)]=[La(t),b(t)]+[a(t),-LTb(t)]=0

and consequently

[M(t1,t2)a(t1),b(t2)]=[a(t1),S(t2,t1)b(t2)]

M(t1,t2)T= S(t2,t1)

YES!

a(t1) M(t1,t2)a(t1)

S(t2,t1)b(t2) b(t2)time

How to determine MT ? (2)

Gradient J can be determined efficiently by running the adjoint model (2) backwards in time!

an earth system model based on the ecmwf integrated forecasting system ifs-model what is the ifs?...

Documents

fieldlevel t

grid points

code slide

earth system model

eps model grid t399

integrated forecast

spectral model

ifs numerical scheme