garrett wedam lynn mcmurdie, cliff mass
DESCRIPTION
East Coast vs. West Coast: A Documentation of Model Forecast Failures for Eta, NAM, GFS, GEM, and ECMWF. Garrett Wedam Lynn McMurdie, Cliff Mass. 72-hour NAM Forecast: Ridging. 1024 mb contour. 24. 72-hr NAM forecast with Obs verification Valid Dec 24, 2006. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Garrett Wedam Lynn McMurdie, Cliff Mass](https://reader035.vdocuments.mx/reader035/viewer/2022070421/56815fff550346895dcefc4f/html5/thumbnails/1.jpg)
East Coast vs. West Coast: A Documentation of Model Forecast
Failures for Eta, NAM, GFS, GEM, and ECMWF
Garrett WedamLynn McMurdie, Cliff Mass
![Page 2: Garrett Wedam Lynn McMurdie, Cliff Mass](https://reader035.vdocuments.mx/reader035/viewer/2022070421/56815fff550346895dcefc4f/html5/thumbnails/2.jpg)
72-hour NAM Forecast: Ridging 1024 mb contour
72-hr NAM forecast with Obs verification Valid Dec 24, 2006
24
![Page 3: Garrett Wedam Lynn McMurdie, Cliff Mass](https://reader035.vdocuments.mx/reader035/viewer/2022070421/56815fff550346895dcefc4f/html5/thumbnails/3.jpg)
72-hour Reality: Low Center
00-hr NAM analysis and obs verification Dec 24, 2006
1000 mb contour
![Page 4: Garrett Wedam Lynn McMurdie, Cliff Mass](https://reader035.vdocuments.mx/reader035/viewer/2022070421/56815fff550346895dcefc4f/html5/thumbnails/4.jpg)
• 72-hour forecast errors– Eta/NAM: 24.1 mb– CMC-GEM: 14.4 mb– ECMWF: 10.0 mb– GFS: 3.2 mb
• 48-hour forecast errors– Eta/NAM: 13.7 mb– CMC-GEM: 9.1 mb– ECMWF: 7.6 mb– GFS: 1.5 mb
• 24-hour forecast errors– Eta/NAM: 4.7 mb– CMC-GEM: 2.4 mb– ECMWF: 2.7 mb– GFS: -0.2 mb
GFS 72-hour forecast verifying 12-26
Verifying buoy
![Page 5: Garrett Wedam Lynn McMurdie, Cliff Mass](https://reader035.vdocuments.mx/reader035/viewer/2022070421/56815fff550346895dcefc4f/html5/thumbnails/5.jpg)
72-hour Forecast: Low Center
72-hr NAM forecast with Obs verification Valid Dec 24, 2006
992 mb contour
24
-24
-20
-27
-31
27
25
21
-8
![Page 6: Garrett Wedam Lynn McMurdie, Cliff Mass](https://reader035.vdocuments.mx/reader035/viewer/2022070421/56815fff550346895dcefc4f/html5/thumbnails/6.jpg)
72-hour Reality: Ridging
00-hr NAM Analysis and Obs verification Dec 24, 2006
1020 mb contour
![Page 7: Garrett Wedam Lynn McMurdie, Cliff Mass](https://reader035.vdocuments.mx/reader035/viewer/2022070421/56815fff550346895dcefc4f/html5/thumbnails/7.jpg)
Method
• Directly compare observations to interpolated model forecasts
• Emphasize East and West Coasts– Buoys eliminate terrain effects
• Emphasize Sea Level Pressure comparisons– SLP is good indicator of model performance (i.e., storm
strength and location can be defined and compared using SLP)
– Insufficient off-coast upper-level observations
![Page 8: Garrett Wedam Lynn McMurdie, Cliff Mass](https://reader035.vdocuments.mx/reader035/viewer/2022070421/56815fff550346895dcefc4f/html5/thumbnails/8.jpg)
Method
• Useful Tools for Comparing Forecast Skill
– Mean Absolute Error (MAE) – Mean Error, or Bias (ME)– Frequency of Errors which exceed an arbitrary “Large
Error” criterion (different for 24, 48, 72-hr forecasts)– Timelines of said “Large Errors” to cross-reference
failure dates between models– Error Histograms to compare distributions and sizes of
largest errors
![Page 9: Garrett Wedam Lynn McMurdie, Cliff Mass](https://reader035.vdocuments.mx/reader035/viewer/2022070421/56815fff550346895dcefc4f/html5/thumbnails/9.jpg)
•Matching Variance in Sea Level Pressure:
![Page 10: Garrett Wedam Lynn McMurdie, Cliff Mass](https://reader035.vdocuments.mx/reader035/viewer/2022070421/56815fff550346895dcefc4f/html5/thumbnails/10.jpg)
Eta to WRF-NAM operational switchover
> 7 mb
> 5 mb
> 3 mb
West Coast: Number of Large Errors by model and month
![Page 11: Garrett Wedam Lynn McMurdie, Cliff Mass](https://reader035.vdocuments.mx/reader035/viewer/2022070421/56815fff550346895dcefc4f/html5/thumbnails/11.jpg)
East Coast: Number of Large Errors by model and month
CMC - GEM major model update. Included: increase in vertical and horizontal resolution, new physics scheme, decreased time step, data assimilation changes
![Page 12: Garrett Wedam Lynn McMurdie, Cliff Mass](https://reader035.vdocuments.mx/reader035/viewer/2022070421/56815fff550346895dcefc4f/html5/thumbnails/12.jpg)
Number of Large Errors: West Coast *minus* East Coast
![Page 13: Garrett Wedam Lynn McMurdie, Cliff Mass](https://reader035.vdocuments.mx/reader035/viewer/2022070421/56815fff550346895dcefc4f/html5/thumbnails/13.jpg)
Mean Absolute Error: West Coast *minus* East Coast
For reference, typical MAE values: GFS West Coast average for winter 2006/07:
24-hr: 1mb; 48-hr: 1.4 mb; 72-hr: 2.0 mb
![Page 14: Garrett Wedam Lynn McMurdie, Cliff Mass](https://reader035.vdocuments.mx/reader035/viewer/2022070421/56815fff550346895dcefc4f/html5/thumbnails/14.jpg)
Comparing models: East Coast Mean Absolute Error *Standardized* to ECMWF
CMC – GEM update
![Page 15: Garrett Wedam Lynn McMurdie, Cliff Mass](https://reader035.vdocuments.mx/reader035/viewer/2022070421/56815fff550346895dcefc4f/html5/thumbnails/15.jpg)
Comparing models: West Coast Mean Absolute Error *Standardized* to ECMWF
![Page 16: Garrett Wedam Lynn McMurdie, Cliff Mass](https://reader035.vdocuments.mx/reader035/viewer/2022070421/56815fff550346895dcefc4f/html5/thumbnails/16.jpg)
Results• Comparing models: ECMWF generally outperforms
and NAM underperforms others. There are indications that a CMC-GEM model update significantly improved some forecasts, and that GFS has not performed as well this season as the past season.
• More “forecast bust” events occur on the West than East Coast for 24, 48, and 72 hour forecasts
• The East Coast is better forecast in terms of MAE in sea level pressure
![Page 17: Garrett Wedam Lynn McMurdie, Cliff Mass](https://reader035.vdocuments.mx/reader035/viewer/2022070421/56815fff550346895dcefc4f/html5/thumbnails/17.jpg)
Dates of Large Forecast Errors (by model)Errors greater than 5 mb
On the Horizon
![Page 18: Garrett Wedam Lynn McMurdie, Cliff Mass](https://reader035.vdocuments.mx/reader035/viewer/2022070421/56815fff550346895dcefc4f/html5/thumbnails/18.jpg)
On the Horizon
•Forecast Failures can be a result of initialization errors, insufficient “realism” of the model, and other model inadequacies. Knowing the most common causes of failure for high-impact forecast failures can help determine policy: do we invest more in improving models or in a new observation buoy?
•One cause of failure is High Model Sensitivity…
![Page 19: Garrett Wedam Lynn McMurdie, Cliff Mass](https://reader035.vdocuments.mx/reader035/viewer/2022070421/56815fff550346895dcefc4f/html5/thumbnails/19.jpg)
On the Horizon
Courtesy of Ryan Torn
![Page 20: Garrett Wedam Lynn McMurdie, Cliff Mass](https://reader035.vdocuments.mx/reader035/viewer/2022070421/56815fff550346895dcefc4f/html5/thumbnails/20.jpg)
On the Horizon
![Page 21: Garrett Wedam Lynn McMurdie, Cliff Mass](https://reader035.vdocuments.mx/reader035/viewer/2022070421/56815fff550346895dcefc4f/html5/thumbnails/21.jpg)
On the Horizon
![Page 22: Garrett Wedam Lynn McMurdie, Cliff Mass](https://reader035.vdocuments.mx/reader035/viewer/2022070421/56815fff550346895dcefc4f/html5/thumbnails/22.jpg)
On the Horizon
![Page 23: Garrett Wedam Lynn McMurdie, Cliff Mass](https://reader035.vdocuments.mx/reader035/viewer/2022070421/56815fff550346895dcefc4f/html5/thumbnails/23.jpg)
On the Horizon
![Page 24: Garrett Wedam Lynn McMurdie, Cliff Mass](https://reader035.vdocuments.mx/reader035/viewer/2022070421/56815fff550346895dcefc4f/html5/thumbnails/24.jpg)
On the Horizon
•Thirty-four of 420 forecasts showed the highest sensitivity
•Nineteen of the 34 “sensitive” cases (55%) resulted in multiple models meeting “large error criterion”
•Compare to: 10-20% of all forecasts result in multiple models meeting “large error criterion”
motivates a more formal investigation!
![Page 25: Garrett Wedam Lynn McMurdie, Cliff Mass](https://reader035.vdocuments.mx/reader035/viewer/2022070421/56815fff550346895dcefc4f/html5/thumbnails/25.jpg)
On the Horizon
Picture from fijaciones.org
Are Two Winters Representative? Currently adding data from 2004/05 and remainder of 2006/07 to further examine year-to-year variability in forecast skill
What About The Rest of the Country? And Beyond? Will examine open-ocean buoys, and land sites; over land will use 850 mb observations instead of SLP
What Are the Features of “Busts”: Forecast failures with high impacts on people still occur. What are their features? And why do some models fail when others don’t?
![Page 26: Garrett Wedam Lynn McMurdie, Cliff Mass](https://reader035.vdocuments.mx/reader035/viewer/2022070421/56815fff550346895dcefc4f/html5/thumbnails/26.jpg)
East Coast vs. West Coast: A Documentation of Model Forecast
Failures for Eta, NAM, GFS, GEM, and ECMWF
Garrett WedamLynn McMurdie, Cliff Mass
Thank You!