V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014

Inherently mass-conservative semi-Lagrangian transport scheme and global hydrostatic

atmospheric model

V. Shashkin1,2(vvshashkin@gmail.com), M. Tolstykh1,2 , R. Fadeev2

August 21, 2014

1 - Hydrometeorological centre of Russia

2 - Institute of Numerical Mathematics, Russian Academy of Sciences

V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014

Outline

• Motivation (Why inherently mass-conservative SL?)

• Transport scheme design (Cell integrated SL scheme)

• Advection tests (DCMIP advection tests result)

• Inherently mass-conservative (IMC) SL model design

• IMC SL model tests (Held&Suarez, Baroclinic instability, Medium-range

NWP)

V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014

Motivation

More invariants = more trust in model (and hopely more credible results) in long-

period simulations

Why mass-conservation?

Why SL?

• Operationaly verified model at Hydrometcentre of Russia (SL-AV, Tolstykh 2010)

• Multi-tracer efficient – benefit for using with atmospheric chemistry block

• Large CFL permitting – better computational performance at lower number of

cores

Why no global mass-correction?

Mass-correction – aggravates locality problems, how to deal with chemical

species?

V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014

( )

0V t

dq dV

dt

Integral formulation of transport equation:

Air densityLagrangian air volume

Tracer mass conservation provided no physical sources/sinks

*

1

*

1

( )

( )

( ) ( )ijk

nijk

nijk

n nijkijk

V

V t V

V t V

q V q dV

- Arrival volume = Grid cell

- Departure volume

Prognostic variable: Tracer density

Time discretization:

Tracer specific concentration

Transport scheme design – Cell Integrated Semi-Lagrangian type

V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014

Transport scheme design – Departure volume integration

*

1 1( ) ( )

ijk

n nijk

ijk V

q q dVV

- Integral over departure volume

• Approximation of departure volume geometry O(Δx2)• Tracer density approximation O(Δx3) (PPM, Colella & Woodward, 1984 & monotonic options)

3D integral

3 x 1D integrals (remappings)(using cascade approach - CCS)

(2D – Nair et al, MWR, 2002,3D – Shashkin, HMC Proc, 2012)

Works with reduced lat-lon grid

V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014

Transport scheme design – Parallel computation & Scalability

Parallel computations:• 1D MPI decomposition (in latitude)• OpenMP longitude cycles• innermost vertical cycle

90

-900 360

MPI-0

MPI-1

MPI-2

MPI-3

OMP-0 OMP-1 OMP-2 OMP-3

Scalability:• 1600x756 (points lon x lat) x 60 levs x 5 tracers:1024 cores(256 MPI x 4 OMP) eff. 62% • 1600x800 (points lon x lat) x 60 levs x 5 tracers:800 cores(200 MPI x 4 OMP) eff. 78%• 1600x800 (points lon x lat) x 60 levs x 5 tracers:1600 cores(200 MPI x 8 OMP) eff. 53%• Probably more cores but no supercomputer to test it• Rather moderate scalabity. Do we really need more?

V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014

Advection tests – DCMIP 1-1 (Kent et al, QJRMS 2014) @ 10x10x60 levs, time-step 1800s

Unlimited

Monotonic

Exact=Initial

T=6 days

T=6 days

T=12 days

T=12 days

T=0 and12 days

l1 l2 l∞ max

Unlim .159 .132 .275 .078

Monot .150 .140 .285 -0.015

MCore .177 .155 .263

CAM-FV .121 .0998 .192

V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014

Inherently mass-conservative (IMC) SL hydrostatic atmospheric model

Basic model: SL-AV (Semi-Lagrangian Absolute Vorticity):• Operational at Hydrometeorological center of Russia (medium-range and seasonal forecasts)• Dyn.Core of own development (Institute of Numerical Mathematics & HMC)• Physics – ALADIN-LACE• Global, Hydrostatic• Non mass-conservative (a posteriori mass correction) => Development of Inherently mass-conservative version for the longer term forecasts

Model equations & discretization (Tolstykh, JCP, 2002 & Tolstykh and Shashkin, JCP, 2012):• Semi-implicit semi-Lagrangian, SETTLS, pseudo-second order decentering• Regular (and reduced) lat-lon grid, sigma-coordinate in vertical• Vorticity-Divergence formulation (Z-grid) in horizontal• Fourier representation in longitude

V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014

Inherently mass-conservative (IMC) SL hydrostatic atmospheric model

Basic version continuity equation:

1ln s

s s

dp D u

dt R T R T

ddt

s - orography geopotential

T - ref. temperature D - 2d divergence

u

- horizontal wind

IMC version continuity equation (Shashkin and Tolstykh, GMD, 2014):

( ) ( ) ( )

0 ref refs s

V t V t V t

d dp dV p p dV p u dV

dt dt

1013,25 expref sp hPaRT

- reference pressure ( )V t - Lagrangian volume

Basic version cont. eq. also kept for discretization of divergence equation

IMC version tracer transport equation:( ) ( )

s s q

V t V t

dp qdV p F dV

dt

q - specific concentration

qF - phys. source/sink

Thermodynamic equation:1

ln ss s T

p p

dT RT dp u F

dt c dt R T c

Consistent with Basic or IMC cont. eq.

V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014

IMC-SLAV tests

Standard dynamics tests:

• Baroclinic instability (Jablonowski and Williamson, QJRMS, 2006)

• Held&Suarez (Held and Suarez, 1994)

Test with physics:

• Medium range weather forecast

V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014

Baroclinic instability test

Baroclinic wave

evolution day 6-9

IMC-SLAV

0.90x0.720x28 levsJablonowski and

Williamson, QJRMS,

2006 initial conditions

V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014

Baroclinic instability test

Day 9 ps (left) and T 850 hPa (right) convergence. Resolution (lon x lat)

from top to bottom: 0.9x0.72, 0.5625x0.45, 0.45x0.36, 0.3x.24, 28 levs all

V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014

Baroclinic instability test

Day 9 relative vorticity (RV)

Good agreement with reference results (Jablonowski and Williamson, NCAR TN,

2006) despite some minor details. RV amplitude is easily tuned by diffusion.

0.9x0.72 0.5625x0.45

0.45x0.36 0.3x.24

V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014

Held & Suarez testcase

Held and Suarez, 1994:• Initially atmosphere at rest, small perturbations• Temperature forcing (idealized solar radiation) and Railey wind damping• 200 days for spin-up and 1000 days – control period

1000 days average zonal mean Temperature (left) and zonal wind speed (right)

V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014

Held & Suarez testcase

Held and Suarez, 1994:• Initially atmosphere at rest, small perturbations• Temperature forcing (idealized solar radiation) and Releygh wind damping• 200 days for spin-up and 1000 days – control period

1000 days average zonal mean Temperature (left) and zonal wind speed (right)

32 m/s

-12,5 m/sStrongly affected by diffusion

V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014

Held & Suarez testcase

1000 days average zonal mean Eddy Kinetic energy (1)Eddy heat flux (2)Eddy momentum flux (3)Temperature second moment (4)

Good agreement with IFS, GM, GME and other models

V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014

Medium range weather forecast

• Real physics, real orography• Real flow configuration• Computationally cheap

Ideal for preliminary testing

Model configuration:• Regular latitude-longitude grid 0.720x0.90, 28 levels• Initial data – HMCR operational analysis Jan 1- Jan 31 2012, Jul 1 – Jul 31 2012• Setup (physics, diffusion etc) identical to basic SLAV operational version, except IMC dynamical core

Results:

• Reasonable verification scores (BIAS, RMS, Absolute err against analysis) essentialy

similar to basic SLAV results. Note no tuning was performed

• One apparent flaw – spurious orographic resonance stronger (as compared to basic

version) – gradient error verification score suffers

• Precipitation maxima generally weaker (as compared to basic version)

V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014

Medium range weather forecast

Examples of spurious orographic resonance. Left IMC-SLAV, right basic SLAV.72 hours forecast of 500 hPa geopotential height over and downstream Himalayas.

IMC-SLAV basic-SLAV

V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014

Medium range weather forecast

Examples of spurious orographic resonance. Left IMC-SLAV, right basic SLAV.72 hours forecast of 500 hPa geopotential height over and downstream Himalayas.

IMC-SLAV basic-SLAV

V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014

Medium range weather forecast

Example of 24 hours forecast of 6 hours accumulated precipitation over Europe.Typical case: IMC and basic SLAV give generally similar results, but IMC-SLAV precipitation pattern

is slightly broader and less intensive.

V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014

Summary and outlook

Summary:• Locally mass-conservative CISL transport scheme performs well (as compared to DCMIP advection test cases reference results)• Inherently mass conservative SLAV atmospheric model gives reasonable results in both idealized test cases and real flows• It seems reasonable to continue with CISL-based IMC SL technique

Future development:• To do something with spurious orographic resonance (any ideas?)• Seasonal forecast and longer simulations testing• Consistent tracer transport• Improved scalability

V. Shashkin et al. Mass-conservative SL, WWOSC-2104, P&P August 21, 2014

Thank you for attention!