evaluation of a challenging warm season qpf month at hpc: june 2009 brendon rubin-oster richard otto...

Evaluation of a Challenging Warm Season QPF month at HPC:

June 2009

Brendon Rubin-Oster Richard Otto

(with contributions from Mike Bodner, Keith Brill, David Novak and HPC management)

June 3, 2010

1

Motivation For This ReviewHPC 1” Threat Score was for June 2009

• Lowest since 1999 (0.119)

• FY 2009 goal is 0.290

• NAM threat score of 0.111 is lowest since at least 1998

• GFS threat score of 0.092 tied its lowest in 10 years (1999)

• HPC % Improvement over models: 18% NAM, 43% GFS, 16% ECMWF

0.131

2

(2009)

3

Seek To Answer the Following

• Was there a common theme associated with the busted forecasts? (MCS activity, scattered convection, synoptic scale forcing, mesoscale boundaries, etc.)

• What kinds of errors were observed?

(timing/duration, placement, magnitude, etc.)

• How did the models perform?

• What (if anything) was so different about June 2009?

4

• What were the causes of these errors?

Data & Methodology

Selected model data archived at HPC for later review deterministic (NAM, GFS, ECMWF, CMC, UKMET)ensembles (SREF, CMCE, ECENS)

• 2 km base reflectivity radar composite archive (NMAPRAD)• VIS/IR satellite & analysis data from internet

(SPC archives, UCAR case studies)• NCEP/NCAR reanalysis for monthly means

Limitations on available data• Surface: winds, moisture convergence, T, Td, θe, moisture transport, 1000 mb frontogenesis, mixing ratio• Thickness: 1000-850 mb & 850-700 mb• Instability: CAPE, lifted indices• Miscellaneous: Q-vectors, Corfidi vectors• And others…

5

6

What (if anything) was so different

about June 2009?

7

Source: http://www.esrl.noaa.gov/psd/data/composites/day/

500mb Geopotential Height means (m)

8

250 mb mean zonal winds (m/s): June 1-30

Source: http://www.esrl.noaa.gov/psd/data/composites/day/

9

Observed Precipitation (in) (June 1-30)

Source: http://water.weather.gov/10

Percent of Normal Precipitation (June 1-30)

Source: http://water.weather.gov/11

Was there a common theme associated with the busted forecasts?

Defining Modes of Convection

● Mesoscale Convective System (MCS):

organized cluster of thunderstorms

resides significant distance from synoptic boundaries

persists for at least 3 to 6 hours

Defining Modes of Convection

● Convection along a synoptic scale boundary:

thunderstorms developing within 150 km of a synoptic

boundary

Defining Modes of Convection

● Convection along a mesoscale boundary:

thunderstorms developing within 150 km of a mesoscale

surface boundary

consists of: sea breezes, surface trofs, outflow boundaries

Defining Modes of Convection

● Scattered convection:

thunderstorms not appearing to be associated with

synoptic/mesoscale boundaries

does not meet MCS guidelines

Was there a common theme associated with the busted forecasts?

Busted forecasts (14 cases)

Convection along synopticboundaries (7)

MCS activity (3)

Convection alongmesoscale boundaries (2)

Scattered convection (1)

Stratiform precip fromsynoptic system (1)

50%

22%

7%

14%

7%

What kinds of errors were observed?

Error types (14 cases)

Magnitude (6)

Placement & Magnitude(4)

Placement (3)

Timing/Duration &Magnitude (1)

29%

43%

7%

21%

What were the causes of these errors?

Reasons for busts (14 cases)

Unforecast mesoscaleboundary (3)

Mishandling of MCV(s)(3)

Erroneous convectivefeedback H5 vort (2)

Incorrect placement ofsynoptic boundary (2)

Uncertain MCSdevelopment (2)

Unknown (2)

14%

22%

14%14%

22%

14%

How did the models perform

during these 14 cases?

20

Rank Performer Threat Score

1 NAM 0.042

2 HPC 0.033

3 GFS 0.027

4 ECMWF 0.026

Rank Performer Bias

1 NAM 1.353

2 ECMWF 1.462

3 GFS 1.932

4 HPC 2.810

Source: http://www2.hpc.ncep.noaa.gov/npvu/

45mm / 1.78” PW

June 26, 2009 case study

OBSERVED NAM f036 AUTO f036

HPC f036 GFS f036 EC f036

24-hr radar loop {June 25 (1200Z)- June 26 (1200Z)}

Satellite loop June 25 (1415Z-2315Z)

Final Notes / Future Work?

• Explore how atypical the June 2009 observed phenomena and forecast performance were relative to other June cases

• HPC is currently applying MODE (method for object-based diagnostic evaluation) in real-time

• Use object oriented verification to objectively quantify errors

• Compare performance of high resolution mesoscale models