experience in numerical forecast verification in the hydrometeorological centre of russia
DESCRIPTION
COSMO 7th General Meeting. Experience in numerical forecast verification in the Hydrometeorological Centre of Russia. N. P. Shakina, E. N. Skriptunova, A. R. Ivanova Z ürich 2005. The models: - PowerPoint PPT PresentationTRANSCRIPT
Experience Experience in numerical forecast verification in numerical forecast verification
in the Hydrometeorological in the Hydrometeorological Centre of RussiaCentre of Russia
N. P. Shakina, E. N. Skriptunova, A. R. Ivanova
Zürich 2005
COSMO 7th General Meeting
The models:The models:• SM HMC, global spectral model (T85L31) of Hydrometeorological Centre of Russia
• SLM HMC, global semi-Lagrangian model, resolution 0.9x0.720, 28 σ-levels to 6 hPa
• UKMO global model
The objective analyses:The objective analyses:• OA UKMO, 2.5x2.5o
• OA HMC, operative version, 2.5x2.5o and 1.25x1.25o
• OA DAS (Data Assimilation System) of SLM HMC
WMO/ICAO standards for high levelsWMO/ICAO standards for high levels::
Wind speed, error above 250 hPa 10 m/s in 90 % of points
Wind speed, error below 250 hPa 7 m/s in 90 % of points
Wind direction error 30 deg in 90 % of points
Max wind level height, error 600 m in 70 % of points
Tropopause height error 600 m in 70 % of points
Convective cloud top error 600 m in 70 % of points
RMSE Abs. error Bias Projec tion SM
HMC SLM HMC
SM HMC
SLM HMC
SM HMC
SLM HMC
24 7.60 7.05 4.28 3.87 -1.55 -1.21 48 10.18 9.70 5.60 5.06 -2.07 -1.49 72 12.77 12.24 6.80 6.20 -2.31 -1.76
Standard estimates of 250 hPa wind speed forecasts (Laboratory for Forecast Testing), August 2005, 00 UTC, Northern Hemisphere
Maximum wind and jet streamsMaximum wind and jet streams(standard approach)(standard approach)
From the cubic spline approximation of wind component profiles, maximum wind speed, MW, and level, H(MW), are determined in the gridpoints
100150
200
250
300
400
500
700
850
925
1000
P , m b
Wm ax
W(p)
SLM forecasts SM
forecasts UKMO forecasts
ОА DAS SLM Date Time
ОА HMC
ОА UKMO 2.5х2.50 0.72х0.900
OA HMC ОА
UKMO
1 00 86.9 80.3 92.5 91.6 72.4 93.8 1 12 84.3 78.9 96.0 96.1 79.5 95.0 2 00 87.5 85.7 97.9 96.4 75.5 97.0 2 12 84.2 74.3 92.7 94.3 76.3 97.6 … … … … … … … … 8 12 91.4 81.7 90.1 91.2 85.1 97.1 9 00 85.1 80.0 90.0 91.3 80.7 97.1 9 12 92.0 80.1 95.3 94.5 82.1 98.4
10 00 90.7 86.6 95.0 94.6 83.0 98.5 10 12 92.4 86.4 97.3 97.0 85.7 98.4
00 87.6 82.3 93.4 93.3 79.7 96.2 12 88.7 79.7 93.6 94.2 82.5 96.9
Ave rage Total 88.1 81.0 93.5 93.7 81.1 96.5
Percentage of ICAO standards fulfilled for maximum wind speed by the SLM, SM and UKMO models,
with different OA taken as “fact”:1-10 Oct 2004, 24-h projection
maximum wind speeds, m/s Model
<30 30 – 40
40 – 50
50 – 60
>60 >30 Total
SLM HMC -0.8 -2.0 -3.1 -3.7 -4.5 -2.8 -1.5 SM
HMC -1.6 -3.1 -4.9 -6.3 -8.9 -4.6 -2.7
UKMO 0.7 1.2 1.3 1.1 0.7 1.2 0.9
Distribution of maximum wind speed 24-h forecast biases for SLM, SM HMC and UKMO models
along the maximum wind speed spectrum, 1-10 Oct 2004
temperature
potential vorticity
0 2 4 6 8 10
200-20-40100
1000
500
700
850
300
T, C
PV, pvu
p, hPa
pf
y
u
x
vgpPV
)()(
Typical profiles of temperature and PV in the troposphere and lower stratosphere
Comparison of thermal and dynamic tropopause 24-h forecasting bias, hPa (UKMO model), with respect to the
corresponding UKMO objective analysis, 1-10 Oct 2004
P, hPa >400 400-350
350-300
300-250
250-200
200-150
150-100
<100 Total
Thermal tropopause
-47.2 -24.0 1.3 5.5 2.2 1.9 9.9 - 3.9
Number of cases
310 365 2412 8826 8414 2430 6143 0 28900
Dynamic tropopause
-16.9 -1.6 0.1 2.3 3.9 3.0 0.9 2.3 2.1
Number of cases
185 1004 3812 8345 6511 5927 2616 500 28900
Dynamic tropopause height, hPa
400-350 350-300 300-250 250-200
200-150
Average
24-h projection July, SM HMC -31.4 -14.8 -5.9 3.5 10.4 -1.4 July, UKMO -0.6 1.6 0.0 2.5 2.6 1.3 Oct, SM HMC -42.8 -19.7 -6.8 0.4 5.2 -5.0 Oct, UKMO -17.0 -0.7 0.6 2.7 3.7 1.1
36-h projection July, SM HMC -39.0 -18.3 -6.6 5.5 12.7 -1.2 July, UKMO -7.5 0.2 -0.1 3.7 4.1 2.3 Oct, SM HMC -51.6 -23.9 -7.7 1.4 6.2 -5.6 Oct, UKMO -16.8 -0.1 2.4 5.0 5.0 2.8
Distribution of the tropopause height bias as dependent on the tropopause height; 1-10 July and 1-10 October 2004
Percentage of ICAO standard fulfilled for the tropopause height predicted by SM HMC and UKMO model
24 h 36 h Model
00 12 Total 00 12 Total July 74.7 75.3 75.0 67.5 66.5 67.0
SM HMC Oct 78.1 78.2 78.2 71.5 72.4 71.9 July 90.4 89.8 90.1 78.6 78.3 78.5
UKMO Oct 89.9 90.2 90.0 82.4 83.1 82.8
Note: the UKMO thermal tropopause 24-h forecasts meet the ICAO standard in 77% of the gridpoints (100W-0-1800E-1600W, 35-750N)
Level of neutral buoyancy, LNB, km
Convective available potential energy, CAPE, J
Convective inhibition energy, CIN, J
Free convection level, FC, km
Downdraft origination level, DOL, km
Characteristics Characteristics of of convective instabilityconvective instability
Comparison of LNB>2km 24-h forecasts against different objective analyses taken as ‘facts”, 1-10 October 2004
SLM Time UKMO SM HMC
0.72х0.900 2.5х2.50 00 503 396 3511 472 Number of gridpoints
with LNB >2 km 12 480 419 3736 349 Total number of gridpoints
1445 1445 13104 1445
Fact: OA UKMO 00 83.6 61.5 - 69.0 Correct forecasts
LNB>2 km occurrence, %
12 84.3 63.2 - 66.0
00 88.5 84.4 - 77.7 Correct forecasts LNB >2 km non- occurrence, %
12 89.5 84.4 - 79.5
00 75 232 - 117 False alarms, number of gridpoints 12 68 197 - 136
00 112 132 - 235 Aim missings, number of gridpoints 12 104 142 - 212
SLM Time UKMO SM HMC
0.72х0.900 2.5х2.50 00 503 396 3511 472 Number of gridpoints
with LNB >2 km 12 480 419 3736 349 Total number of gridpoints
1445 1445 13104 1445
12 104 142 - 212 Fact: OA HMC
00 - 45.7 - 53.7 Correct forecasts LNB >2 km occurrence, %
12 - 50.6 - 53.3
00 - 85.7 - 82.1 Correct forecasts LNB >2 km non- occurrence, %
12 - 83.8 - 80.4
00 - 327 - 178 False alarms, number of gridpoints 12 - 263 - 188
00 - 121 - 189 12 - 147 - 203
Aim missings, number of gridpoints
12 - - 765 86
Comparison of LNB>2km 24-h forecasts against different objective analysis taken as ‘facts”, 1-10 October 2004 (cont.)
SLM Time UKMO SM HMC
0.72х0.900 2.5х2.50 00 503 396 3511 472 Number of gridpoints
with LNB >2 km 12 480 419 3736 349 Total number of gridpoints
1445 1445 13104 1445
Fact: OA DAS SLM
00 - - 78.2 77.3 Correct forecasts LNB >2 km occurrence, %
12 - - 65.0 65.0
00 - - 83.2 83.6 Correct forecasts LNB >2 km non- occurrence, %
12 - - 91.8 91.7
00 - - 768 87 False alarms, number of gridpoints 12 - - 1312 142
00 - - 1611 175 Aim missings, number of gridpoints 12 - - 765 86
Comparison of LNB>2km 24-h forecasts against different objective analyses taken as ‘facts”, 1-10 October 2004 (cont.)
CONCLUSIONS:
1. Apart from standard criteria of numerical forecast accuracy, we estimate regularly (10 days in central month of seasons) accuracy of predicted quantities obtained from the model output data by means of post-processing.
2. The quantities under consideration represent cumulative characteristics of
wind profile (maximum wind),
wind and temperature profiles and horizontal distributions (potential vorticity),
temperature and humidity profiles (convective instability).
CONCLUSIONS (cont.):
3. Accuracies of maximum wind, tropopause, convection (and others) are estimated for the operatively used global models (SM HMC, SLM HMC, UKMO model) and for the corresponding objective analyses.
4. The results allow us revealing and quantitatively estimating features of models and their DAS which are not clearly seen from the standard accuracy criteria, for example:
spectral dependence of wind errors;
smoothing of tropopause funnels and domes by the models;
“diurnal cycle” of convective instability
CONCLUSIONS (cont.):
5. Analysis of these effects, in a close co-operation with the model developers and with numerical forecast users (aviation forecasters) leads to better interpretation of the numerical forecasts and to improving of the models.
Thank you!