post processing

47
Post Processing

Upload: allie

Post on 06-Jan-2016

85 views

Category:

Documents


0 download

DESCRIPTION

Post Processing. Model Output Can Usually Be Improved with Post Processing. Can remove systematic bias Can produce probabilistic information from deterministic information - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Post Processing

Post Processing

Page 2: Post Processing

Model Output Can Usually Be Improved with Post Processing

• Can remove systematic bias

• Can produce probabilistic information from deterministic information

• Can provide forecasts for parameters that the model incapable of modeling successfully due to resolution or physics issues (e.g., shallow fog)

Page 3: Post Processing

Post Processing

• Model Output Statistics was the first post-processing method used by the NWS (1969)

• Based on multiple linear regression.

• Essentially unchanged in 40 years.

• Does not consider non-linear relationships between predictors and predictands.

• Does take out much of systematic bias.

Page 4: Post Processing

MOS Developed by and Run at the NWS Meteorological Development Lab (MDL)

• Full range of products available at:

http://www.nws.noaa.gov/mdl/synop/index.php

Page 5: Post Processing
Page 6: Post Processing

Global Ensemble MOS• Ensemble MOS forecasts are based on the 0000 UTC

run of the GFS Global model ensemble system. These runs include the operational GFS, a control version of the GFS (run at lower resolution), and 20 additional runs.

• Older operational GFS MOS prediction equations are applied to the output from each of the ensemble runs to produce 21 separate sets of alphanumeric bulletins in the same format as the operational MEX message.

Page 7: Post Processing

Gridded MOS•The NWS needs MOS on a grid for many reasons, including for use in their IFPS analysis/forecasting system.

•The problem is that MOS is only available at station locations.

•To deal with this, NWS created Gridded MOS.

•Takes MOS at individual stations and spreads it out based on proximity and height differences. Also does a topogaphic correction dependent on reasonable lapse rate.

Page 8: Post Processing
Page 9: Post Processing
Page 10: Post Processing

Current “Operational” Gridded MOS

Page 11: Post Processing

Localized Aviation MOS Program(LAMP)

• Hourly updated statistical product

• Like MOS but combines:– MOS guidance– the most recent surface observations– simple local models run hourly– GFS output

Page 12: Post Processing
Page 13: Post Processing
Page 14: Post Processing

MOS Performance

• MOS significantly improves on the skill of model output.

• National Weather Service verification statistics have shown a narrowing gap between human and MOS forecasts.

Page 15: Post Processing

Cool Season Mi. Temp – 12 UTC Cycle

Cool Season'66-67 '71-72 '76-77 '81-82 '86-87 '91-92 '96-97 '01-02

2

3

4

5

6

7

24-h GUID. 24-h LCL

48-h GUID. 48-h LCL

Average Over 80 US stations

Page 16: Post Processing

Prob. Of Precip.– Cool Season(0000/1200 UTC Cycles Combined)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

1966 1969 1972 1975 1978 1981 1984 1987 1990 1993 1996 1999 2002

Year

Brier Score Improvement over Climate

Guid POPS 24 hr Local POPS 24 hr

Guid POPS 48 hr Local POPS 48 hr

Page 17: Post Processing

MOS Won the DepartmentForecast Contest in 2003For the First Time!

Page 18: Post Processing
Page 19: Post Processing
Page 20: Post Processing
Page 21: Post Processing

Average or Composite MOS

• There has been some evidence that an average or consensus MOS is even more skillful than individual MOS output.

• Vislocky and Fritsch (1997), using 1990-1992 data, found that an average of two or more MOS’s (CMOS) outperformed individual MOS’s and many human forecasters in a forecasting competition.

Page 22: Post Processing

Some Questions• How does the current MOS performance…driven by

far superior models… compare with NWS forecasters around the country.

• How skillful is a composite MOS, particularly if one weights the members by past performance?

• How does relative human/MOS performance vary by forecast projection, region, large one-day variation, or when conditions vary greatly from climatology?

• Considering the results, what should be the role of human forecasters?

Page 23: Post Processing

This Study

• August 1 2003 – August 1 2004 (12 months).

• 29 stations, all at major NWS Weather Forecast Office (WFO) sites.

• Evaluated MOS predictions of maximum and minimum temperature, and probability of precipitation (POP).

Page 24: Post Processing

National Weather Service locations used in the study.

Page 25: Post Processing

Forecasts Evaluated• NWS Forecast by real, live humans• EMOS: Eta MOS• NMOS: NGM MOS• GMOS: GFS MOS• CMOS: Average of the above three MOSs• WMOS: Weighted MOS, each member is weighted

by its performance during a previous training period (ranging from 10-30 days, depending on each station).

• CMOS-GE: A simple average of the two best MOS forecasts: GMOS and EMOS

Page 26: Post Processing

The Approach: Give the NWS the Advantage!

• 08-10Z-issued forecast from NWS matched against previous 00Z forecast from models/MOS.– NWS has 00Z model data available, and has added

advantage of watching conditions develop since 00Z.– Models of course can’t look at NWS, but NWS looks at

models.

• NWS Forecasts going out 48 (model out 60) hours, so in the analysis there are:– Two maximum temperatures (MAX-T),– Two minimum temperatures (MIN-T), and – Four 12-hr POP forecasts.

Page 27: Post Processing

Temperature Comparisons

Page 28: Post Processing

MAE (F) for the seven forecast types for all stations, all time periods, 1 August 2003 – 1 August 2004.

Temperature

Page 29: Post Processing

MAE for each forecast type during periods of large temperature change (10F over 24-hr), 1 August 2003 – 1 August 2004. Includes data for all stations.

Large one-day temp changes

Page 30: Post Processing

MAE for each forecast type during periods of large departure (20F) from daily climatological values, 1 August 2003 – 1 August 2004.

Page 31: Post Processing

Number of days each forecast is the most accurate, all stations.

In (a), tie situations are counted only when the most accurate temperatures are exactly equivalent. In (b), tie situations are cases when the most accurate temperatures are within 2F of each other.

Looser Tie Definition

Page 32: Post Processing

Number of days each forecast is the least accurate, all stations.

In (a), tie situations are counted only when the least accurate temperatures are exactly equivalent. In (b), tie situations are cases when the least accurate temperatures are within 2F of each other.

Looser Tie Definition

Page 33: Post Processing

Time series of MAE of MAX-T for period one for all stations, 1 August 2003 – 1 August 2004. The mean temperature over all stations is shown with a dotted line. 3-day smoothing is performed on the data.

Highly correlated time series

Page 34: Post Processing

Time series of bias in MAX-T for period one for all stations, 1 August 2003 – 1 August 2004. Mean temperature over all stations is shown with a dotted line. 3-day smoothing is performed on the data.

Cold spell

Page 35: Post Processing

MAE for all stations, 1 August 2003 – 1 August 2004, sorted by geographic region. MOS Seems to have the most problems at high elevation stations.

Page 36: Post Processing

Bias for all stations, 1 August 2003 – 1 August 2004, sorted by geographic region.

Page 37: Post Processing

Precipitation Comparisons

Page 38: Post Processing

Brier Scores for Precipitation for all stations for the entire study period.

Page 39: Post Processing

Brier Score for all stations, 1 August 2003 – 1 August 2004. 3-day smoothing is performed on the data.

Page 40: Post Processing

Brier Score for all stations, 1 August 2003 – 1 August 2004, sorted by geographic region.

Precipitation

Page 41: Post Processing

Reliability diagrams for period 1 (a), period 2 (b), period 3 (c) and period 4 (d).

Page 42: Post Processing

There are many other post-processing approaches

• Neural netsAttempts to duplicate the complex interactions between neurons in the human brain.

Page 43: Post Processing
Page 44: Post Processing

Dynamic MOS• MOS equations are updated frequently, not

static like the NWS.

• Example: DiCast used by the Weather Channel

Page 45: Post Processing

They don’t MOS!

Page 46: Post Processing

UW Bias Correction of WRF

Page 47: Post Processing

And many others…