Dictionary Comparison and Dropout Team Update

Dr. Bradley BallishNCEP/NCO/PMB

and co-chair JAG/ODAA

September 2009 CSAB Meeting

“Where America’s Climate and Weather Services Begin”

2

Overview

• Upper air dictionary comparison• Impact tests with NCEP Upper air

dictionary • Further work is needed with surface

dictionaries• COPC AI 2009-1.5• COPC AI 2008-1.5• Data impact tests• Big Dropout Case• Future work• Summary

3

ID Source Lat Lon Elev Elev-H

17062 AFWA 4054N 02909E 33 18.2 17062 CANADA 4054N 02909E 33 18.2 17062 FNMOC 4058N 02905E 33 18.2 17062 NCDC 4058N 02905E 40 18.2 17062 NCEP 4058N 02905E 40 18.2 17062 UKMET 4054N 02909E 33 18.2 17062 WMO 4054N 2909E 18 18.2

Only WMO Elv looks correct ID Source Lat Lon Elev Elev-H 17095 CANADA 3954N 04117E 1869 1869.5 17095 FNMOC 3954N 04117E 1869 1869.5 17095 NCEP 3854N 04117E 1869 1869.5 17095 WMO 3954N 4117E 1869 1869.5

NCEP Lat has typo, no AFWA, NCDC or UKMET entry

Station dictionaries from different centers were converted toa common format and compared. Below are two examples.Hydrostatic elevation estimates under Elev-H were very useful

4

Dictionary Experiments: Cntrl vs. NewprepdaCase: 2009070500

0.5

0.55

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1

Day3-NH AC1000mb

Day5-NH AC1000mb

Day3-NH AC500mb

Day5-NH AC500mb

Day3-SH AC1000mb

Day5-SH AC1000mb

Day3-SH AC500mb

Day5-SH AC500mb

An

om

aly

Co

rrela

tio

nNewprep-2009070500

Cntrl-2009070500

• Overall, results show that the dictionary updates have little impact on our 3-day and 5-day AC scores.

• However, there is one particular case (2009070500) where the SH 5-day AC score is improved for the New prepda (0.68) compared to the control (0.60)

5

SYNOP METAR Lat Lon Elv Num Pdifa 72327 KBNA 36.13 273.32 180 64 -3.63 72494 KSFO 37.62 237.62 5 32 -2.60 72407 KACY 39.45 285.43 20 32 -1.90 72405 KDCA 38.85 282.97 4 63 -1.75 72295 KLAX 33.93 241.60 32 32 -1.74 72785 KGEG 47.63 242.47 721 64 -1.62 72550 KOMA 41.30 264.10 299 32 -1.58 ...72565 KDEN 39.87 255.33 1656 64 1.61 72520 KPIT 40.50 279.78 373 63 1.83 72464 KPUB 38.28 255.48 1439 30 1.97 72476 KGJT 39.12 251.47 1475 32 2.38 72570 KCAG 40.50 252.47 1915 64 2.78 72386 KLAS 36.08 244.83 664 32 3.13 72402 KWAL 37.93 284.52 41 30 3.20

Work on surface dictionaries is waiting for information from NWS on US synoptic and METAR elevations and locations as well as how the GATEWAY makes fake synoptic reports from the METARS

This is important as we find systematic differences in station pressure comparing synoptic and METAR sites at exactly the same time and elevations

The above is from US sites, but this is a world wide problem

6

COPC AI 2009-1.5The dropout team will evaluate the affect of satellite wind, satellite radiance, and aircraft data on model dropouts. They will also invite JCSDA and ECMWF to coordinate in their efforts – Action Plan:

• We will run GFS/GSI experiments for 10 dropout cases (5 in the SH and 5 in the NH) with and without the above major classes of observational data (5 point if test determines if further analysis is needed)

• We will run experiments with QC limits for satellite winds more like that of the ECMWF

• We will assess the impact of removing aircraft temperature data, with their warm biases, on the resulting change in model temperature bias and skill

• We will ask the JCSDA if someone can participate in our dropout meetings or become even more involved

• We will inform the ECMWF of our work and ask if they can share similar reports

7

COPC AI 2009-1.5The dropout team will evaluate the affect of satellite wind, satellite radiance, and aircraft data on model dropouts. They will also invite JCSDA and ECMWF to coordinate in their efforts –Progress:

• We have run a number of GFS/GSI experiments with and without satellite radiance or wind data (slides will follow)

• We will have run experiments with QC limits for satellite winds more like that of the ECMWF (slides follow)

• JCSDA’s Lidia Curcurull will assess the impact of removing aircraft temperature data, on the resulting change in model temperature bias and skill as part of her GPSRO/COSMIC data tests

• Yang-Grong Ling of the JCSDA will participate in our dropout meetings

• We have gotten a copy of the ECMWF reject-list and their data monitoring website. Lars Peter Riishojgaard will help with further questions

8

COPC AI 2008-1.5Develop a monitoring system to analyze differences between the NCEP and FNMOC global models and the ECMWF global model in real-time and make this real-time system available to OPCs as a daily tool – Action Plan:

• We will alert NCEP 24x7 staff for cases of abnormal forecast correlations at day 5 for the GFS and FNMOC forecasts versus the ECMWF

• A restricted website will show graphics of forecast height differences versus ECMWF each 12 hours to day 5 at 4 pressure levels (850, 500, 250 and 100 hPa)

• A text analysis of extreme analysis differences will be made

9

COPC AI 2008-1.5Develop a monitoring system to analyze differences between the NCEP and FNMOC global models and the ECMWF global model in real-time and make this real-time system available to OPCs as a daily tool – Progress:

• A plan of action was made based on input from the NCEP dropout team, NCEP forecaster managers and select Navy scientists (Langland, Vermeulen and Skupniewicz)

• A draft project management charter is being worked at NCEP

10

SOUTHERN HEMISPHERE 5-DAY 500 MB ANOMALY CORRELATION SCORES

0.4

0.45

0.5

0.55

0.6

0.65

0.7

0.75

0.8

0.85

INITIAL CONDITION DATE (YYYYMMDDHH)

AN

OM

AL

Y C

OR

RE

LA

TIO

N MINDATA

PREPB

AMSUA

AMSUB

MHS

GPSRO

AIRS

HIRS

CNTRL

Figure 10. Impact of satellite radiance data on Southern Hemisphere dropouts. 5-day anomaly correlation scores shown for 10 cases. MINDATA is the GFS/GSI run with only TRMM and SSMI data present, PREPB is the GFS/GSI run with only conventional observations (i.e. RAOB, satellite winds, profiler, etc..), AMSUA is the GFS/GSI run with conventional observations and amsua satellite radiance data, AMSUB is the GFS/GSI run with conventional observations and amsub satellite radiance data, MHS is the GFS/GSI run with conventional observations and mhs satellite radiance data, GPSRO is the GFS/GSI run with conventional observations and gpsro satellite radiance data, AIRS is the GFS/GSI run with conventional observations and airs satellite radiance data, HIRS is the GFS/GSI run with conventional observations and hirs satellite radiance data, and CNTRL is the GFS/GSI run with conventional observations and all satellite radiance data present with a 6-hr data ingest window.

11

NORTHERN HEMISPHERE 5-DAY 500 MB ANOMALY CORRELATION SCORES

0.4

0.5

0.6

0.7

0.8

0.9

1

INITIAL CONDITON DATE

ANOM

ALY

CORR

ELAT

ION

NODATA

PREPB

AMSUA

AMSUB

MHS

GPSRO

AIRS

HIRS

CNTRL

12

Impact of Conventional and Satellite Radiance Obs on 3-d and 5-d GFS Forecasts

10 Case Composite Satellite Radiance Impact on NH and SH 500 mb AC

0.5

0.6

0.7

0.8

0.9

1

3-D NH MEAN 3-D SH MEAN 5-D NH MEAN 5-D SH MEAN

An

om

aly

Co

rre

lati

on

NODATA

PREPB

AMSUA

AMSUB

MHS

GPSRO

AIRS

HIRS

CNTRL

• Satellite radiance data shows positive impact on NH 3-D and 5-D forecasts

• In the SH, conventional and satellite observations (PREPB alone) show a large negative impact (8 points) in 5-D forecasts! -- another puzzle

• Addition of satellite radiance data to conventional and satellite observations have a positive impact on 5-D forecasts

• AMSUA along with PREPB conventional observations (yellowyellow) show the largest positive impact in the NH and SH experiments and typically are correlated with the results of the CNTRL

•This is confirmed by other studies (ECMWF and Joint Center) and shows how each case has individual characteristics requiring incisive diagnostics.

13ECMWF analysis has large difference to NCEP’s due to bad sonde

WindDifferencesIn Knots

14Analysis Difference 1 day after QC error

15Analysis Difference 4 days after QC error

16

Satellite Wind QC Experiments

EXPcon=control

EXPsat=Su’s Satwnd QC

EXPsat2=Carlis’ Satwnd QC

Su’s (Xiujuan Su) experiment reduces the weighting of the observation, while Carlis’ experiment is more heavy handed removing observations that are less than the background guess.

17

Big GFSDropout

ECMWF had a very high score

18

Comparison of GFS vs. ECMWF for the 2009081200 Dropout

19

Regional Analysis of 2009081200 Dropout Case

At 925mb near 54S 109E, a dipole (phase and amplitude difference) exists with -20 – 40 m height and 55 knot vector wind differences between the GFS and ECMWF

After 12-hrs, the height error grows to -70 m and maintain vector wind differences near 55 knots

20

Prepbufr ExperimentsDropout Case: 2009081200

0.4

0.5

0.6

0.7

0.8

0.9

GDAS C

NTRLECM

GDAS C

NTRL NOPREPB

NO D

ATA

GDAS C

NTRL NOADPUPA

GDAS C

NTRL NOAIR

CFT

GDAS C

NTRL NOAIR

CAR

GDAS C

NTRL NOSATW

ND

GDAS C

NTRL NOG

PSIPW

GDAS C

NTRL NOSPSSM

I

GDAS C

NTRL NOW

DSATR

GDAS C

NTRL NOASCATW

GDAS C

NTRL NOQ

KSWND

GDAS C

NTRL NOADPSFC

GDAS C

NTRL NOSFCSHP

GDAS C

NTRL NO4DATA

Experiment Name

5-d

ay A

no

mal

y C

orr

elat

ion

Sco

re

5-day AC NH

5-day AC SH

SH 5-day AC scores improve by removing SATWND, SFCSHP QKSWND, and SPSSMI

SH Droppout Case in Orange

21

Future Work

• Surface dictionary comparison• More data impact tests for AI 2009-1.5• Get work started on AI 2008-1.5• Will contact ECMWF as requested in

COPC AI 2009-1.5• Work on thinned pseudo obs for ECM runs• Using high resolution ECMWF analysis• Aircraft temperature bias correction work• Miscellaneous work

22

Conclusions

• Upper air dictionary comparison was useful

• Already have some data impact tests for AI 2009-1.5 – more is needed

• Getting work started on AI 2008-1.5

• Will contact ECMWF as requested in COPC AI 2009-1.5

• Already have JCSDA involvement