assessing forecast uncertainty in the ndfd
DESCRIPTION
Assessing Forecast Uncertainty in the NDFD. Report to MDL Matt Peroutka, Greg Zylstra, John Wagner July 1, 2005. Overview. Overall Process Methods Data Sources Transformation to Percentiles Diagnostic Data Results Transformation to Percentiles Modeling the Joint Distribution - PowerPoint PPT PresentationTRANSCRIPT
Assessing Forecast Assessing Forecast Uncertainty in the Uncertainty in the NDFDNDFD
Report to MDLReport to MDL
Matt Peroutka, Greg Zylstra, John Matt Peroutka, Greg Zylstra, John WagnerWagner
July 1, 2005July 1, 2005
OverviewOverview
Overall ProcessOverall Process MethodsMethods
– Data SourcesData Sources– Transformation to PercentilesTransformation to Percentiles– Diagnostic DataDiagnostic Data
ResultsResults– Transformation to PercentilesTransformation to Percentiles– Modeling the Joint DistributionModeling the Joint Distribution– Diagnostic DataDiagnostic Data
PlansPlans
Overall ProcessOverall Process
NUNCA
NDFD Forecast
NDFD Performance
Related Guidance
Expected Distribution of Observations
What’s in a name?What’s in a name?
NDFD UNCertainty Assessment NDFD UNCertainty Assessment (NUNCA)(NUNCA)
Numerical Uncertainty Numerical Uncertainty Assessment of NDFD via Assessment of NDFD via Climatology and Ensembles Climatology and Ensembles (NUANCE) (NUANCE)
Development PhaseDevelopment Phase
Observations (x)
Forecasts (f)
Diagnostic Data (d)
MergeJoint
Distribution Model p(f,x,d)
Transform to
Percentiles
Implementation PhaseImplementation Phase
Forecasts (f)
Diagnostic Data (d)
Joint Distribution
Model p(f,x,d)
Extract
Inferred Conditional Distribution
p(x | f,d)
Transform from
Percentiles
Transform to
Percentiles
Data SourcesData Sources
Ideally, NDFD grids and Analysis of RecordIdeally, NDFD grids and Analysis of Record Prototype with NDFD points and METAR Prototype with NDFD points and METAR
observationsobservations– October 2004 to April 2005, inclusiveOctober 2004 to April 2005, inclusive
US Historical Climatological Network (USHCN)US Historical Climatological Network (USHCN) Ensemble MOS (ENSMOS) archivesEnsemble MOS (ENSMOS) archives
– One bulletin from control runOne bulletin from control run– Five bulletins created from runs with positive Five bulletins created from runs with positive
perturbationsperturbations– Five bulletins created from runs with negative Five bulletins created from runs with negative
perturbationsperturbations
Transformation to Transformation to PercentilesPercentiles Addresses lack of cases in Addresses lack of cases in
development data with extreme development data with extreme values of values of ff or or xx..
Encourages combining of data.Encourages combining of data. NDFD has a short history.NDFD has a short history. NDFD includes a variety of NDFD includes a variety of
forecasting techniques.forecasting techniques.
Diagnostic DataDiagnostic Data
Standard Deviation (SD) of 11 Standard Deviation (SD) of 11 ENSMOS forecasts.ENSMOS forecasts.
““Ensemble Deviation” (ED)Ensemble Deviation” (ED)– Difference each perturbed forecast Difference each perturbed forecast
with control forecast;with control forecast;– Compute r. m. s.Compute r. m. s.
Results: Results: Transformation to Transformation to PercentilesPercentiles Obtained daily MaxT observations for 168 Obtained daily MaxT observations for 168
stations from USHCN.stations from USHCN. Percentile function computed at 5-day Percentile function computed at 5-day
intervals throughout the year.intervals throughout the year. Standard probability distributions used to Standard probability distributions used to
model distribution.model distribution.– Percentile function fitted to observations.Percentile function fitted to observations.– Fit parameters expressed as cosine series over day Fit parameters expressed as cosine series over day
of the year.of the year.– Quality of fit judged subjectively.Quality of fit judged subjectively.– Additional terms added to cosine series, if needed.Additional terms added to cosine series, if needed.
Statistical Statistical DistributionsDistributionsDistributioDistribution n
Variable Variable CommentComment
Normal Normal MaxT MaxT Poor fit “in the tails” Poor fit “in the tails”
Normal Normal ln (MaxT) ln (MaxT) Improved fit “in the tails” Improved fit “in the tails”
Binormal Binormal MaxT MaxT Skewness improved fit for some Skewness improved fit for some stations. Poor fit “in the tails.”stations. Poor fit “in the tails.”
Logistic Logistic MaxT MaxT Better fit than either version of Better fit than either version of Normal Normal
Laplace Laplace MaxT MaxT Worst fitWorst fit
Gumbel Gumbel MaxT MaxT Skewness improved fit for some Skewness improved fit for some stations. stations.
GumbelGumbel -(MaxT) -(MaxT) Skewness improved fit for some Skewness improved fit for some stations. stations.
Generalized Generalized Lambda Lambda
MaxTMaxT Best overall fitBest overall fit
Results: Modeling Results: Modeling Joint DistributionJoint Distribution Modeling Modeling p(f,x,d)p(f,x,d) straightforward straightforward
for prototype.for prototype.– Small number of stations.Small number of stations.– All values can be retained in All values can be retained in
memory.memory.– Will be re-designed for grids.Will be re-designed for grids.
Results: Diagnostic Results: Diagnostic DataData Stratified scatter diagrams for Stratified scatter diagrams for
day 7 day 7 – ED < 6 ED < 6 °F°F– ED ED ≥ ≥ 6 6 °F°F
Results: SD vs. EDResults: SD vs. ED
NDFD percentile errors binned NDFD percentile errors binned (interval 0.1 (interval 0.1 °F) °F) by associated by associated ED/SD value.ED/SD value.
Plot NDFD percentile error vs. Plot NDFD percentile error vs. ED/SD.ED/SD.
Future PlansFuture Plans
Quantitatively assess uncertainty.Quantitatively assess uncertainty.– Variations by forecast projection.Variations by forecast projection.
Expand to include MinT.Expand to include MinT. Prototype products.Prototype products.
– 50% confidence interval around NDFD50% confidence interval around NDFD– Probability Density FunctionProbability Density Function– Exceedence probabilities for key valuesExceedence probabilities for key values