deterministic vs probabilistic forecast j.p. céron – météo-france
TRANSCRIPT
Deterministic vsProbabilistic Forecast
J.P. Céron – Météo-France
The Predictability
« a Thunderstorm will be observed next Sunday over the Toulouse « Météopole » between 15h and 16h » Irrealistic, the confidence that one can
have in this forecast is very low
« a rainy system will cross the Toulouse region Sunday afternoon »
realistic, one can be quite confident in this forecast
The Predictability
The predictability depends on : The scale of the forecasted phenomenum
(Thunderstorm, Easterly Wave, Blocking situation, ENSO, …)
The Range of the forecast (NowCasting, Short , Medium , Seasonnal , Climatic)
Deterministic formulation error or lost of informations
Probabilistic formulation more possibilities in the forecast but interpretation problems
The different Forecasts How do you play to Horse races?
Informations about the form of horses, trainers, jockey, race conditions (type of soil, weather, …), predictions, …
Synthesis then making a bet on horses (in a more or less subjective choice)
Use of seasonnal forecasts. Informations about the states of atmosphere, continental
surfaces, oceanic system and its evolutions, the different forecasts, …
Synthesis and decision/action (i.e. make a bet on the real solution in a more or less objective way)
The different Forecasts The horse n°5 will win the race.
It will rain 650 mm at Niamey during the next rainy season.
Consequences : gain or lost depending of the success or not of the forecast.
The different ForecastsDeterministic forecast
Deterministic forecast
The different Forecasts
The different ForecastsDeterministic forecast
Limits of Numerical Forecasting
The forecast errors can come from different part of the forexast system :
ANALYSIS- errors (or uncertainty) from measurements, assimilation. - lack of observations over some regions
But also ...
Network density of surface observationsOver the whole globe
Limits of Numerical Forecasting
MODELS LIMITS- mesh, number of vertical levels (resolution of the model) - Equations, parameterisations - small scale phenomena (under the mesh or badly know)
But also ...
- L 'é q u a tio n v e c to r ie lle d u m o u v e m e n t :
S u r l'h o r iz o n ta le :
d V h
d t
1
h p - 2 V h + F h=
V
S u r la v e rtic a le : é q u a tio n d e l'h y d ro s ta tiq u e d 'a p rè s le s a p p ro x im a tio n s
0 =1
p
Zg
- L 'é q u a tio n d e la T h e rm o d y n a m iq u e :
d t
d (c pT )=
d t
d pR T
p+ Q
etc...
Equations are generally “simplifyed”and one calibrate “parametrisations”in the model, that is to say that one
use data computed in an approximateform (even sometime as “constant”).
ANALYSIS- errors (or uncertainty) from measurements, assimilation. - lack of observations over some regions
Limits of Numerical Forecasting
INTERPRETATION- Models generally provide “raw data” andconsequently the interpretation is often difficult)
And finally ...
MODELS LIMITS- mesh, number of vertical levels (resolution of the model) - Equations, parameterisations - small scale phenomena (under the mesh or badly know)
ANALYSIS- errors (or uncertainty) from measurements, assimilation. - lack of observations over some regions
Limits of Numerical Forecasting
COMMUNICATION- misfitted vocabulary - misreading of users’ needs - dissemination without feedbacks from the user
MODELS LIMITS- mesh, number of vertical levels (resolution of the model) - Equations, parameterisations - small scale phenomena (under the mesh or badly know)
ANALYSIS- errors (or uncertainty) from measurements, assimilation. - lack of observations over some regions
INTERPRETATION- Models generally provide “raw data” andconsequently the interpretation is often difficult)
Limits of Numerical Forecasting Natural variability of the Ocean/Atmosphere system and
the Atmospheric respons to external forcing
Limits of Numerical Forecasting Uncertainties of the initial state of the Climatic system Modelisation Error (both Oceanic and Atmosphéric) Natural varibility of Atmosphere and its respons to
external forcing Interpretation of the forecast
To sample the initial state uncertainty disturbances of the analysis
To sample the model uncertainty distubances of the model
To sample in the forecast all the possible solutions of the Ocean/Atmosphere system
Limits of Numerical Forecasting
The numerical forecast using "ARPEGE« model at short range is a « déterministic » forecast. It uses the initial state description of the climate
system and, using model’s equation, allow to perform the time evolution of the atmospheric state.
Major errors of this type of forecasts mainly come from initial state errors intoduced inside the model. The uncertainty spread increase with the range of
the forecast.
error at time t0
(initial uncertainty)
error at time t0 + range
Limits of Numerical ForecastingTo take into account the probability of the deterministic evolution, one
perform several forecasts starting from the same initial time but using slightly modified values of the parameters of the simulation
(namely inside the probable range of errors introduced at the initial time).
Deterministic forecast
At this range:Strong probability
region
Forecast range
valu
e o
f th
e p
aram
eter
Initial time
Limits of Numerical ForecastingBecause of computer ressources, one use a larger mesh model (to limit the computation time) and one perform, typically, around thirty
forecast using different conditions for the model.
850 hPa Temperature plumes (Toulouse)base 21 september 1999 at 12h UTC
125 - 50%
6 - 25%
0 - 6%
50 - 75%
75 - 100% Deterministic model
verification
Limits of Numerical ForecastingOne can look at the parameter’s dispersion as a function of the time
intergration and describe the encountered value distribution rath rather than the values themselves. One can also give a confidence
indice with a more or less high value (the larger the dispersion of the forecasts, the lower the indice that is to say the lower the confidence).
Temperature dispersion plumes as a function of the time integration
850 hPa Temperature plumes (Toulouse)base 21 september 1999 at 12h UTC
125 - 50%
6 - 25%
0 - 6%
50 - 75%
75 - 100% Deterministic model verification
This method is named “Ensemble forecast”
Limits of Numerical Forecasting
The ensemble forecast (Resume) To sample the initial state uncertainty disturbances
of the analysis To sample the model uncertainty distubances of the
model To sample in the forecast all the possible solutions of
the Ocean/Atmosphere system
Several forecasts – trying to get the distribution of the possible solutions instead of a single value
Several models – How can we merge the informations coming from many different models – empirical, AGCM, COAGCM ? Multimodel approach.
Limites of Statistical Forecasting Uncertainty already included inside the statistical tools
The different Forecasts The horse n°5 will win the race. It will rain 650 mm at Niamey during the next
rainy season. The horse n°5 has good chances to be in the
firsts in this race. The situation of the Ocean/Atmosphere
system and its probable evolutions indicate that the next rainy season in Niger has a good probability to be « above the Normal ».
Consequences : gain or lost depending of the success or not of the forecast
The different ForecastsProbabilistic Forecast
The different ForecastsProbabilistic Forecast
The different ForecastsProbabilistic Forecast
The different ForecastsProbabilistic Forecast
The different Forecasts Analogue technics
Probabilistic Forecast : formulation 1 model et n members p models et n members Gaussian : mean + standard deviation Analogues Statistical Methods (Discriminant Analysis,
Multiple Regression, Probabilistic Regression, …)
Catégories ForecastProbabilistic Forecast
Catégories Forecast How to define the categories?
Number Categories Limits Needs of user?
How to evaluate the forecasted probabilities for each category? Frequency/Probability , Climatological Probabilities,
Conditionnal probabilities, Confidence Indice Statistical Models Numerical Models
How to transform the forecast in « readable and comprehensive » form for the user?
Quadratic Scores (Brier, RPS, …) Relative Operating Characteristic (FA vs ND) Cost/Lost ratio approach
Deux categories: + dry / + wet et Ratio c/L=0.5
C1= averaged cost using a climatological forecastC2 = averaged cost using a perfect forecast C3= averaged cost using a the model forecast
Probabilistic Forecast: verification
obs non obs
prev c c
non prev
L 0
obs non obs
prev n11 n12
non prev
n21 n22
C2C1C3C1100V
Brier Scores :
BrSc = 1/N (pi – oi)2
pi probabilité prévue pour l’événement oi variable indicatrice de l’observation de l’événement 1
BrSc= o(p) 1-p2 + 1-o(p) p2 g(p) dp 0
g(p) Fréquence relative avec laquelle l’événement est prévu avec une probabilité comprise entre p et p+dp 1 1 1
BrSc= f 1- f + p1-o(p) 2- f-o(p) 2 g(p) dp 0 0 0
Uncertainty Reliability Resolution
Pour un système qui aurait toujours prévu la probabilité climatique d’occurrence ( pflt ) LCBrSc= f 1- f + f 1- flt
[L/S]CBrSkSc = 1 – BrSC / [L/S]CBrSc Brier Skill Score
BSSREL = 1 – BSREL // [L/S]CBrSc Brier Reliability SC
BSSRSL = BSRSL / Uncertainty Brier Resolution SC
Probabilistic Forecast: verification Reliability Diagrams
Probabilistic Forecast
Cost/Lost ratio approach
Probabilistic Forecast Differents users
Probabilistic Forecast Comparison between Deterministic and
Probabilistic formulation
Probabiliste
Déterministe
Probabilistic Forecast
Resume