review of ncep gfs forecast skills and major upgrades fanglin yang imsg - environmental modeling...

Review of NCEP GFS Forecast Skills and Major Upgrades

Fanglin Yang

IMSG - Environmental Modeling Center

National Centers for Environmental Prediction

Camp Springs, Maryland, USA

24th Conference on Weather and Forecasting & 20th Conference on Numerical Weather Prediction

91th AMS Annual Meeting, 23–27 January 2011, Seattle, WA

2

GFS 500hPa Height Anomaly Correlation, 00Z Cycle Day-5 Forecast

• GFS forecast skill has been steadily improving in both the Northern and Southern Hemisphere.

• GFS performs better in NH than in SH, especially in early years.

• NH score was improved significantly in 2010.

3

Day-5 Northern-Hemisphere 500-hPa Height Anomaly Correlation

GFS: NCEP Global Forecast System;    FNO: Navy Fleet Numerical Meteorology and Oceanography Center;    ECM: European Center for Medium-Range Weather Forecasts;    UKM: The United Kingdom Met Office;    CMC: The Canadian Meteorological Center;    CDAS: T62 1995 version of GFS used for the NCEP/NCAR Reanalysis.

• The forecast skills of all NWP models have been steadily improving.

• GFS lags behind ECMWF, and is comparable to UKM.

• The difference between GFS and CDAS is an indicator of improvement in GFS physics, dynamics and data assimilation system.

4

Day-5 Southern-Hemisphere 500-hPa Height Anomaly Correlation

• GFS lags behind ECMWF;

• UKM surpassed GFS since 2005;

• CMC improved in the past couple of years.

GFS: NCEP Global Forecast System;    FNO: Navy Fleet Numerical Meteorology and Oceanography Center;    ECM: European Center for Medium-Range Weather Forecasts;    UKM: The United Kingdom Met Office;    CMC: The Canadian Meteorological Center;    CDAS: T62 1995 version of GFS used for the NCEP/NCAR Reanalysis.

5

Tropical Wind RMSE850 hPa 200 hPa

• GFS 850hPa wind has been significantly improved over the years, and is closing to ECMWF forecast. Improvement of 200hPa wind is modest.

• It is worth noting that RMSE can be misleading if the forecast model is heavily damped.

6

NH

SH

500-hPa HGT RMSE, Dec 2010 500-hPa HGT, Fcst v.s. Analysis

ECMWF develops cold bias in the stratosphere

• Each NWP model has its own strength and weakness. • For instance, GFS has smaller forecast error than ECMWF in the stratosphere.

Forecast Hour 0-240

7

GFS Annual Mean Day-5 NH 500-hPa Height Anomaly Correlation

•Forecasts with AC >= 0.6 is usually regarded as useful.

•GFS useful forecasts have improved from 6.4 days in 2001 to 8 days in 2010.

•ECMWF is still about 0.5 day ahead of GFS.

Day-5 AC has improved by about 0.1 in the past 10 years

8

GFS Annual Mean Day-5 SH 500-hPa Height Anomaly Correlation

•Forecasts with AC >= 0.6 is usually regarded as useful.

•GFS SH useful forecasts have improved from 6.1 days in 2001 to 7.4 days in 2010.

•ECMWF is about 0.8 day ahead of GFS.

SH Day-5 AC has improved by about 0.12 in the past 10 years

9

GFS Precip Skill Scores Over CONUS, fh60-84 (day-3)

ETS increased and BIAS decreased in recent years

10

Precip Skill Scores

• contingency table: – Hits (a): occasions/counts where both forecast and observation are greater

than or equal to a threshold over, say, CONUS; – False alarms (b): occasions where forecast is above a threshold whereas

observation is under the same threshold; – Misses (c): occasions where the observation is above a threshold and

forecast is under the same threshold; – No forecasts (d): occasions where both forecast and observation are

under the threshold.

• Bias Score: BS=(a + b)/(a + c) measures over-forecasts (BS>1) or under-forecasts (BS<1) precipitation frequency over an area for a selected threshold.

• Equitable Threat Score: EQ_TS=(a - ar)/(a + b + c - ar) where ar is the expected number of correct forecasts above the threshold in a random forecast where forecast occurrence/non-occurrence is independent from observation/non-observation, ar=(a + b)*(a + c)/(a + b + c + d). EQ_TS=1 means a perfect forecast. EQ_TS <=0 means the forecast is useless.

Obs Yes Obs NO

Fcst YES a b

Fcst NO c d

11

Reduce unrealistic excessive heavy precipitation (so called grid-scale storm or bull’s eye precipitation)

New (SAS, PBL and Shallow Convection)

24 h accumulated precipitation ending at 12 UTC, July 24, 2008 from (a) observation and 12-36 h forecasts with (b) control GFS and (c) revised model

OBS CTL

Han & Pan, 2010

12

GFS Hurricane Track and Intensity Forecasts, Atlantic Basin

• Intensity forecast has been significantly improved, mainly due to increases in model resolution.

• The current GFS intensity forecast is catching up with HWRF. Forecasters have started to take GFS into account for intensity forecast.

• Track forecast within 3 days has been steadily improving, although the pace is slow.

• Beyond day 3, the forecast still varies from year to year.

13

GFS Hurricane Track and Intensity Forecasts, Eastern Pacific

• Intensity forecast not improved in the Eastern Pacific Basin. Why ??

• Track forecast within 3 days has been steadily improving, although the pace is slow.

14

What kind of model changes have contributed to the improvement of GFS forecast skills?

Does very model upgrade always lead to better forecast skill?

15

GFS Changes

• 3/1999– AMSU-A and HIRS-3 data

• 2/2000– Resolution change: T126L28 T170L42 (100 km 70 km)– Next changes

• 7/2000 (hurricane relocation)• 8/2000 (data cutoff for 06 and 18 UTC)• 10/2000 – package of minor changes• 2/2001 – radiance and moisture analysis changes

• 5/2001– Major physics upgrade (prognostic cloud water, cumulus momentum transport)– Improved QC for AMSU radiances– Next changes

• 6/2001 – vegetation fraction• 7/2001 – SST satellite data• 8/200 – sea ice mask, gravity wave drag adjustment, random cloud tops, land surface

evaporation, cloud microphysics…)• 10/ 2001 – snow depth from model background• 1/2002 – Quikscat included

The “GFS Changes” slides were first scripted by Stephen Lord

16

GFS Changes (cont)

• 11/2002– Resolution change: T170L42 T254L64 (70 km 55 km)– Recomputed background error– Divergence tendency constraint in tropics turned off– Next changes

• 3/2003 – NOAA-17 radiances, NOAA-16 AMSU restored, Quikscat 0.5 degree data• 8/2003 – RRTM longwave and trace gases• 10/2003 – NOAA-17 AMSU-A turned off• 11/2003 – Minor analysis changes• 2/2004 – mountain blocking added• 5/2004 – NOAA-16 HIRS turned off

• 5/2005– Resolution change: T254L64 T382L64 ( 55 km 38 km )– 2-L OSU LSM 4-L NOHA LSM– Reduce background vertical diffusion– Retune mountain blocking– Next changes

• 6/2005 – Increase vegetation canopy resistance

• 7/2005 – Correct temperature error near top of model

17

GFS Changes (cont)•8/2006

– Revised orography and land-sea mask– NRL ozone physics– Upgrade snow analysis

•5/2007– SSI (Spectral Statistical Interpolation) GSI ( Gridpoint Statistical Interpolation).

– Vertical coordinate changed from sigma to hybrid sigma-pressure– New observations (COSMIC, full resolution AIRS, METOP HIRS, AMSU-A and MHS)

•12/2007– JMA high resolution winds and SBUV-8 ozone observations added

•2/2009– Flow-dependent weighting of background error variances– Variational Quality Control– METOP IASI observations added– Updated Community Radiative Transfer Model coefficients

•7/2010– Resolution Change: T382L64 T574L64 ( 38 km 23 km )– Major radiation package upgrade (RRTM2 , aerosol, surface albedo etc)

– New mass flux shallow convection scheme; revised deep convection and PBL scheme

– Positive-definite tracer transport scheme to remove negative water vapor

18

500-hPa Height AC Frequency distribution, GFS 00Z Cycle Day-5 Forecast

Twenty bins were used to count for the frequency distribution, with the 1st bin centered at 0.025 and the last been centered at 0.975. The width of each bin is 0.05.

Look at the history of extremes in the distribution

– Poor Forecasts (AC < 0.7 )

– Excellent forecasts ( AC > 0.9 )

19

Resolution:

1. 3/1991: T80L18 T126L28 (100km)

2. 2/2000: T126L28 T170L42 (70km)

3. 11/2002: T170L42 T254L64 (55km)

4. 6/2005: T254L64 T382L64 (38km)

5. 7/2010: T382L64 T574L64 (23km)

Percent of Poor Forecasts (AC <0.7) v.s. Model Changes

Physics and Data Assimilation:

A. 3/1999: AMSU-A & HIRS-3 data

B. 5/2001: prognostic cloud water, cumulus momentum transport

C. 6/2005: OSU 2-L LSM to 4-L NOHA LSM

D. 5/2007: SSI to GSI; Hybrid sigma-p; New observations

E. 2/2009: flow-dependent error covariance; Variational QC

F. 7/2010: New shallow convection; updated SAS and PBL; positive-definite tracer transport.

A

2

B

3

4, C

5, F

year

NH

20

Resolution:

1. 3/1991: T80L18 T126L28 (100km)

2. 2/2000: T126L28 T170L42 (70km)

3. 11/2002: T170L42 T254L64 (55km)

4. 6/2005: T254L64 T382L64 (38km)

5. 7/2010: T382L64 T574L64 (23km)

Percent of Poor Forecasts (AC <0.7) v.s. Model Changes

A

2

B

3 4, C 5, F

year

SH

D

Physics and Data Assimilation:

A. 3/1999: AMSU-A & HIRS-3 data

B. 5/2001: prognostic cloud water, cumulus momentum transport

C. 6/2005: OSU 2-L LSM to 4-L NOHA LSM

D. 5/2007: SSI to GSI; Hybrid sigma-p; New observations

E. 2/2009: flow-dependent error covariance; Variational QC

F. 7/2010: New shallow convection; updated SAS and PBL; positive-definite tracer transport.

E

21

Resolution:

1. 3/1991: T80L18 T126L28 (100km)

2. 2/2000: T126L28 T170L42 (70km)

3. 11/2002: T170L42 T254L64 (55km)

4. 6/2005: T254L64 T382L64 (38km)

5. 7/2010: T382L64 T574L64 (23km)

Percent of Excellent Forecasts (AC >0.9) v.s. Model Changes

4, C

5, F

year

NH

Physics and Data Assimilation:

A. 3/1999: AMSU-A & HIRS-3 data

B. 5/2001: prognostic cloud water, cumulus momentum transport

C. 6/2005: OSU 2-L LSM to 4-L NOHA LSM

D. 5/2007: SSI to GSI; Hybrid sigma-p; New observations

E. 2/2009: flow-dependent error covariance; Variational QC

F. 7/2010: New shallow convection; updated SAS and PBL; positive-definite tracer transport.

E

3

22

Resolution:

1. 3/1991: T80L18 T126L28 (100km)

2. 2/2000: T126L28 T170L42 (70km)

3. 11/2002: T170L42 T254L64 (55km)

4. 6/2005: T254L64 T382L64 (38km)

5. 7/2010: T382L64 T574L64 (23km)

Percent of Excellent Forecasts (AC >0.9) v.s. Model Changes

3

D

year

SH

Physics and Data Assimilation:

A. 3/1999: AMSU-A & HIRS-3 data

B. 5/2001: prognostic cloud water, cumulus momentum transport

C. 6/2005: OSU 2-L LSM to 4-L NOHA LSM

D. 5/2007: SSI to GSI; Hybrid sigma-p; New observations

E. 2/2009: flow-dependent error covariance; Variational QC

F. 7/2010: New shallow convection; updated SAS and PBL; positive-definite tracer transport.

E

?

23

Comments

• Most implementations include both major and minor changes. They all contribute to improving the system. Accumulated impact of many small changes is significant but not measurable.

• Predictability may change from year to year.

24

Most Recent GFS Upgrade 28-July-2010 Implementation

T382L64 (38 km) T574L64 (23 km)& Major physics upgrade

• Major changes• Testing and evaluation• Benefits and remaining issues

25

Major Changes

• Resolution and ESMF– T382L64 to T574L64 ( ~38 km -> ~27 km) for fcst1 (0-192hr) & T190L64 for

fcst2 (192-384 hr) .– fcst2 step with digital filter turned on– ESMF 3.1.0rp2

• Radiation and cloud– Changing SW routine from ncep0 to RRTM2– Changing longwave computation frequency from three hours to one hour– Adding stratospheric aerosol SW and LW and tropospheric aerosol LW– Changing aerosol SW single scattering albedo from 0.90 in the operation to

0.99– Changing SW aerosol asymmetry factor. Using new aerosol climatology.– Changing SW cloud overlap from random to maximum-random overlap– Using time varying global mean CO2 instead of constant CO2 in the

operation– Using the Yang et al. (2008) scheme to treat the dependence of direct-beam

surface albedo on solar zenith angle over snow-free land surface

26

Example: Improving GFS Surface Albedo Using ARM-SURFRAD Observations

Dependencies of direct-beam albedo, normalized by the diffuse albedo, on SZA. The ten colored long-dashed lines represent the empirical fits derived from observations at the three ARM and seven SURFRAD stations for the entire-day cases. The blue line with filled circles is based on the observations at all stations except the Desert Rock station (the line with crosses). The black lines with open circles and squares are governed by the NCEP GFS parameterization with the constant being set to 0.4 and 0.1, respectively.

Fits using data at ARM and SURFRAD stations

32

1 cos34.2cos92.4cos02.427.2,

mndiff

mndirf

322 cos02.2cos13.4cos34.389.1

,60

,

omndir

mndirf

Fanglin Yang, Kenneth Mitchell, Yu-Tai Hou, Yongjiu Dai, Xubin Zeng, Zhou Wang, and Xin-Zhong Liang, 2008: Dependence of land surface albedo on solar zenith angle: observations and model parameterizations. Journal of Applied Meteorology and Climatology. No.11, Vol 47, 2963-2982.

27

• Gravity-Wave Drag Parameterization – Using a modified GWD routine to automatically scale mountain block

and GWD stress with resolution.– Compared to the T382L64 GFS, the T574L64 GFS uses four times

stronger mountain block and one half the strength of GWD.

• Removal of negative water vapor– Using a positive-definite tracer transport scheme in the vertical to

replace the operational central-differencing scheme to eliminate computationally-induced negative tracers.

– Changing GSI factqmin and factqmax parameters to reduce negative water vapor and supersaturation points from analysis step.

– Modifying cloud physics to limit the borrowing of water vapor that is used to fill negative cloud water to the maximum amount of available water vapor so as to prevent the model from producing negative water vapor.

– Changing the minimum value of water vapor mass mixing ratio in radiation from 1.0e-5 in the operation to 1.0e-20. Otherwise, the model artificially injects water vapor in the upper atmosphere where water vapor mixing ratio is often below 1.0e-5.

Major Changes

28

Vertical Advection of Tracers: Current GFS Scheme

pq

p

q

p

q

t

q Flux form conserves mass

2

1

2

1

2

1

2

1

2

1

2

1

11kk

kk

kkkkk

k qp

qqp

A

2

1

2

1

kk

k ppp

Current GFS uses central differencing in space and leap-frog in time.

The scheme is not positive definite and may produce negative tracers.

kkk

qqq

1

2

1 2

1

1

2

11

2

12

1kk

kkk

kk

k qqqqp

A

nk

nk

nk Atqq 211

kq

1kq

1kq

21kq

21kq21k

21k

29

Example: Removal of Negative Water Vapor

Fanglin Yang et al., 2009: On the Negative Water Vapor in the NCEP GFS: Sources and Solution. 23rd Conference on Weather Analysis and Forecasting/19th Conference on Numerical Weather Prediction, 1-5 June 2009, Omaha, NE

Sources of Negative Water Vapor• Vertical advection• Data assimilation• Spectral transform• Borrowing by cloud water• SAS Convection

Ops GFS

_

Positive-definite

Data Assimilation

A: vertical advection, computed in finite-difference form with flux-limited positive-definite scheme in space

Flux-Limited Vertically-Filtered Scheme, central in time

1*

2

1 nk

nk

nk AAA New

nk

nkhh AB

p

qqV

t

q

*11 2 nk

nk

nk

nk AtBtqq

B: horizontal advection, computed in spectral form with central differencing in space

Data Assimilation

30

Vertical Advection of Tracers: Flux-Limited Scheme

1211121 k

Hkkkk qqqq Thuburn (1993)0 if 21 k

121 2

1 kk

Hk qqq

1

11

11 k

kk

kr

rr

1

2

1

121

k

k

kk

kkk q

q

qq

qqr

Van Leer (1974) Limiter, anti-diffusive term

Lq

21Lq

21Lq021 L

21L

kq

1kq

1kq

21kq

21kq21k

21k

1q

2q

21q

211q2

11

021

0 since 0 1for 212121 qk

Special boundary conditions

1231123 qqqq H

1

11

11 r

rr

1

0

21

101 q

q

qq

qqr

0 if 2 ,0min

0 if 2 ,0max

121

1210

qqq

qqqq

2for k

0q

31

Vertical Advection of Tracers: Flux-Limited Scheme

kHkkkk qqqq

2121

Thuburn (1993)0 if 21 k

121 2

1 kk

Hk qqq

k

kk

kr

rr

1

11

1

k

k

kk

kkk q

q

qq

qqr

Van Leer (1974) Limiter, anti-diffusive term

Lq

21Lq

21Lq021 L

21L

kq

1kq

1kq

21kq

21kq21k

21k

1q

2q

21q

211q2

11

021

Lfor kSpecial boundary condition

LHLLLL qqqq

2121

L

LL

Lr

rr

1 11

1

L

L

LL

LLL q

q

qq

qqr

0 if 2 ,0min

0 if 2 ,0max

1

11

LLL

LLLL qqq

qqqq

1Lq

32

Vertical Advection of Tracers: Idealized Case Study

wind

Upwind (diffusive)

Flux-Limited

GFS Central-in-Space

Initial condition

33

Summary: Negative Water Vapor in the GFS

Causes Importance Solutions

Vertical Advection 1. Semi-Lagrangian2. Flux-Limited Positive-Definite Scheme for current Eulerian GFS

GSI Analysis Tuning factqmin and factqmax

Spectral Transform 1. Semi-Lagrangian GFS: running tracers on grid, no spectral transform2. Eulerian GFS: no solution yet.

Cloud Water Borrowing Limiting the borrowing to available amount of water vapor

SAS Mass-Flux Remains to be resolved

34

• New mass flux shallow convection scheme (Han & Pan 2010)– Use a bulk mass-flux parameterization same as deep convection scheme– Separation of deep and shallow convection is determined by cloud depth (currently 150

mb)– Entrainment rate is given to be inversely proportional to height (which is based on the

LES studies) and much smaller than that in the deep convection scheme– Mass flux at cloud base is given as a function of the surface buoyancy flux (Grant,

2001), which contrasts to the deep convection scheme using a quasi-equilibrium closure of Arakawa and Shubert (1974) where the destabilization of an air column by the large-scale atmosphere is nearly balanced by the stabilization due to the cumulus

• Revised deep convection scheme (Han & Pan 2010)– Random cloud top selection in the current operational scheme is replaced by an

entrainment rate parameterization with the rate dependent upon environmental moisture– Include the effect of convection-induced pressure gradient force to reduce convective

momentum transport (reduced about half)– Trigger condition is modified to produce more convection in large-scale convergent

regions but less convection in large-scale subsidence regions– A convective overshooting is parameterized in terms of the convective available

potential energy (CAPE)

Major Changes

35

• Revised Boundary Layer Scheme (Han & Pan 2010)– Include stratocumulus-top driven turbulence mixing based on Lock et

al.’s (2000) study– Enhance stratocumulus top driven diffusion when the condition for cloud

top entrainment instability is met– Use local diffusion for the nighttime stable PBL rather than a surface

layer stability based diffusion profile– Background diffusivity for momentum has been substantially increased

to 3.0 m2s-1 everywhere, which helped reduce the wind forecast errors significantly

• Hurricane relocation– Running hurricane relocation at the 1760x880 forecast grid instead of

the 1152x576 analysis grid– Posting GDAS pgb files first on Guassian grid (1760x880), then convert

to 0.5-deg for hurricane relocation.

Major Changes

36

Operational shallow convection scheme (Diffusion scheme, Tiedke, 1983)

New shallow convection scheme (Mass flux scheme)

Mass flux analogy (de Roode et al., 2000) :

Au (updraft area)=0.5

Ad (downdraft area)=0.5

Au~0.0; Ad~1.0

Environment is dominated by subsidence resulting in environmental warming and drying.

Example: New Mass-Flux Based Shallow ConvectionBy Jongil Han and Hua-lu Pan

37

Ops GFS New shallow convection scheme

Heating by Shallow Convection

38

ISCCP

Ops GFS New Shallow

Low cloud cover (%)

Marine Stratus

39

No stratocumulus top driven diffusion

With stratocumulus top driven diffusion

Low cloud cover (%)

40

Reduce unrealistic excessive heavy precipitation (so called grid-scale storm or bull’s eye precipitation)

New

24 h accumulated precipitation ending at 12 UTC, July 24, 2008 from (a) observation and 12-36 h forecasts with (b) control GFS and (c) revised model

OBS CTL

41

Parallel Test & Evaluation

– 2008 Hurricane Season (June 20 – November 10)

– 2009 Hurricane Season (June 20 – November 10)

– 2009/2010 Winter and 2010 Spring (December 1 – present)

42

500hPa Height AC

NH 2008 NH 2009

SH 2009SH 2008

Significant improvement in Anomaly correlations for week-one Fcst

43

Tropical Wind RMSE

2008

2009

850 hPa

850 hPa

Significant reduction in tropical wind RMSE

44

Precipitation Skill Scores over CONUS

2008 2009

Significantly improved EQ scores, reduced biases for heavy precip events

45

Hurricane Track and Intensity: 2008

Atlantic Track

Reduced track errors in both basins, significantly improved intensity forecast

Atlantic Intensity

East Pacific Track

East Pacific Intensity

T574

T382

46

Hurricane Track and Intensity: 2009

Atlantic Track

Reduced track error in East Pacific, significantly improved intensity forecast in both basins.

Atlantic Intensity

East Pacific Track

East Pacific Intensity

47

Hurricane Intensity Tendency Forecast: 2008

Better tendency forecast

Atlantic East Pacific

T382 Control T574 Parallel

48

Summary

The upcoming T574L64 implementation in July 2010 was a major improvement upon the last operational T382L64 GFS in terms of height AC, wind RMSE, precipitation skill score, and hurricane track and intensity.

However, there are still a few remaining issues.

• T382 GFS is closer to ECMWF than the T574 GFS does.• T574 GFS has weaker easterly than T382 GFS in 2009 and 2010. • This is caused by overly too strong vertical diffusion of momentum in the

stratosphere. The spring 2011 GFS implementation will has this problem corrected.

T574

ECMWF

Ops T382

QBO transition from westerly phase to easterly phase

50

• T878 L64 or L91 Semi- Lagrangian GFS• NEMS (a unified national environment modeling

system)

Planed Upgrades

51

Annual Mean AC

GFSECMWF

52

GFS v.s. ECMWF (AC < 0.7)

GFS ECMWF

GFS lags ECMWF more in the SH than in the NH, especially in recent years.

53

GFS v.s. ECMWF (AC >0.9)

GFS ECMWF

54

GFS 4-Cycle Comparison, 500-hPa Height day-5 AC

55

GFS 4-Cycle Comparison, Tropical Wind RMSE

Larger height RMS in the lower stratosphere

likely caused by to small a minimum value of water vapor mixing ratio (1.0E-20)