Update on Dropout Team Work andRelated COPC Action Items

Presented byDr. Bradley Ballish

Co-Chair JAG/ODAA andDropout Team* Member

16 November 2010COPC Meeting

AFWA

*Including EMC’s J. Alpert, D. Carlis, Y. Ling, X. SuAnd NCO’s K. Kumar and J. Carr

November 16, 2010 COPC Dropout Update 2

Outline

• Recent AC scores with dropouts indicated• COPC Action Item (AI) 2008-1.5 (real-

time monitoring)• COPC AI 2010-1.6 (data impact tests)• NCEP activity summary• Dropout related projects• NCEP GFS implementation summary• Summary

November 16, 2010 COPC Dropout Update 3

00Z 5-day AC Scores in September/October 2010

No GFS dropouts and GFS isnumber 2 in scores!

FNMOC dropout

Multiple dropouts for FNMOCand GFS in SHEM

ECM runScored .73

November 16, 2010 COPC Dropout Update 4

AI 2008-1.5COPC Action Item 2008-1.5: Develop

a monitoring system to analyze differences between the NCEP and FNMOC global models and the ECMWF global model in real-time and make this real-time system available to OPCs as a daily tool.

Status• Project management charter has

been signed and work has started• Initial operating capability at NCEP

planned for February 2011 • Recommend AI remain open

November 16, 2010 COPC Dropout Update 5

AI 2010-1.6• COPC Action Item 2010-1.6:The dropout team will provide a

final summary report on the impact of observational data types suspected of causing model dropouts and make specific recommendations on data quality control and usage for operational implementation

Status• Team has conducted thorough analysis for many dropout

occurrences• More work and analysis will be needed well beyond the time

frame of typical COPC action items• Impact tests with satellite winds and radiances have been run

(see the next four slides)– Satellite radiances generally have good impact– Satellite winds have shown some negative impact in dropout cases

with GFS– Speed dependent QC and ECMWF filtering of satellite winds

showed positive impact– FNMOC super obs of satellite winds show big positive impact

November 16, 2010 COPC Dropout Update 6

Impact of Conventional and Satellite Radiance Obs on 3-d and 5-d GFS Forecasts

10 Case Composite Satellite Radiance Impact on NH and SH 500 mb AC

0.5

0.6

0.7

0.8

0.9

1

3-D NH MEAN 3-D SH MEAN 5-D NH MEAN 5-D SH MEAN

An

om

aly

Co

rre

lati

on

NODATA

PREPB

AMSUA

AMSUB

MHS

GPSRO

AIRS

HIRS

CNTRL

• Retaining satellite radiance data shows small positive impact on NH 3-D and 5-D forecasts

• In the SH, conventional and non-radiance satellite observations (PREPB alone) show a large negative impact (8 points) in 5-D forecasts! -- another puzzle

• Addition of satellite radiance data to conventional and satellite observations has a positive impact on 5-D forecasts even for dropout cases

• AMSUA along with PREPB conventional observations (yellowyellow) show the largest positive impact in the NH and SH experiments and typically are correlated with the results of the CNTRL

puzzle

November 16, 2010 COPC Dropout Update 7

Satellite Wind QC Experiments

Satellite winds (SATWIND) are a data type of concern at EMC due to case studies that have shown negative impact by using SATWIND’s in the GDAS system.

Su’s SATWND QC lowers the gross check asymmetrically, is planned for operations rather than Carlis’ QC, and its implementation is TBD

Carlis’ Satwnd QC removes about 50% of the observations by deleting wind speeds that are less than the background guess

CONTROL=0.804

SU’S QC=0.817

CARLIS’ QC=0.815

CONTROL=0.818

SU’S QC=0.824

CARLIS’ QC=0.823

November 16, 2010 COPC Dropout Update 8

By Riishojgaard, Jung and Velden

Plan for operational SATWIND QC upgrades TBD

November 16, 2010 COPC Dropout Update 9

Notice big impact fromSatellite winds

FNMOC has big positive impact from satellite winds possibly from super obs

November 16, 2010 COPC Dropout Update 10

AI 2010-1.6 (continued) Recommend closing this AI and continue working on the

data usage improvements listed below:• Implement improved satellite wind QC and filtering (TBD)• Aircraft data

– Implement NRL track-checking code (Q2 2011)* and then improve – Aircraft temperature bias corrections– Improve waypoint dictionaries for AIREP data (Q1 2011)*– Test and implement aircraft moisture data

• Implement surface pressure bias corrections• Use adjoint codes to analyze dropout cases and tune reject-

list criterion• Improve timeliness of adding bad data platforms to reject-list

(Q1 2011)* • Develop and implement use-list and improve methods of

adding bad data platforms to reject-list * Indicates concrete implementation date

November 16, 2010 COPC Dropout Update 11

NCEP Activity Summary

• Dropouts– Data QC and data usage– Dropout analysis

• GFS implementation summary– GFS QC, data assimilation and model

changes

November 16, 2010 COPC Dropout Update 12

Dropout Related ProjectsProject Status

Using Quikscat QC Flags Implemented 2009

Dictionary Corrections Upper air changes in operations 2009 North American surface changes implemented in June 2010

Multicenter waypoint updates

(Q1 2011)*

Observational Data Impact Tests Continuous testing at NCEP plus Langland has started tests

Adjoint Estimates of Observation

Impact

Langland has been running tests and NCEP has adjoint of GSI analysis running with help from NASA/Goddard

Surface Pressure Bias Corrections Preliminary testing

Data QC and Data Usage

November 16, 2010 COPC Dropout Update 13

Dropout Related Projects

Project Status

Aircraft Temperature Bias Corrections Waiting for NRL aircraft QC upgrade - ECMWF to implement in 2010

Satellite Wind QC Upgrades Both speed dependent QC and ECMWF filtering rules showed positive impact - NCEP implementation (TBD)

Profile QC Preliminary investigation

Improved Reject-list Preliminary investigation

Use-list for Conventional Obs Preliminary testing

Data QC and Data Usage

November 16, 2010 COPC Dropout Update 14

Dropout Related Projects (cont)

Establish Dropout Investigation Procedures

Is done and performed for dropout cases

Dropout Relation to Data Counts RTDMS extended to 30 days, currently is in NCO testing for December 2010 implementation

Dropout Climatology Studies Common origin areas found – Langland has shown preferred synoptic patterns

ECM Runs Journal publication in progress – High resolution ECMWF input being tested

Improved Diagnostics for Analysis Differences for Dropouts

Preliminary investigation at both NCEP and FNMOC

Dropout Analysis

November 16, 2010 COPC Dropout Update 15

NCEP GFS Implementation Summary

GFS T574 Model Bundle Implemented on 27 July 2010

Aircraft Track-Checking Implementation of NRL code

(Q2 2011)*

GSI Analysis Upgrades Operational GSI updates (Q2 2011)* implementation

QC, Analysis and Model Changes

November 16, 2010 COPC Dropout Update 16

Summary• Observational data usage continues to be explored as

a possible cause of dropouts– GFS model upgrade helped in NH, but less so in SH– GFS performance likely depends on the quality of each

observation, not simply a particular class of data– Continue to work on improving and testing QC rules and

metadata on a case by case basis• Recommend AI 2010-1.6 to be closed• Operational implementations

– FY10• GFS model July 2010• Surface dictionary upgrades June 2010

– FY11• GSI upgrade (Q2 2011)*• NRL Aircraft QC (Q2 2011)*

– TBD• Improved satellite wind QC and filtering implementations

November 16, 2010 COPC Dropout Update 17

Background Slides

November 16, 2010 COPC Dropout Update 18

Interagency Participants• NCEP/EMC

– Jordan Alpert– Yangrong Ling – DaNa Carlis

• NCEP/NCO– Bradley Ballish– Krishna Kumar– Joe Carr

• NCEP/HPC– James Cisco

• NRL– Rolf Langland

• FNMOC– Chuck Skupniewicz

• NESDIS – Kathy Kelly– Tom Renkevens

• The NCEP dropout team meets weekly with the EMC director, John Derber and sometimes with the JCSDA director and reports quarterly to the NCEP director

November 16, 2010 COPC Dropout Update 19

Latest Dropout Team Findings (continued)

• For more information on dropout research, see the reports presented at the AMS meetings:

http://ams.confex.com/ams/89annual/techprogram/paper_142644.htm http://ams.confex.com/ams/89annual/techprogram/paper_142649.htmhttp://ams.confex.com/ams/23WAF19NWP/techprogram/paper_154268.htmhttp://ams.confex.com/ams/23WAF19NWP/techprogram/paper_154282.htm

November 16, 2010 COPC Dropout Update 20

• Based on the code from the NASA/Goddard/GMAO

GSI, the dropout team has the GSI analysis adjoint code running

• Further work is needed to develop an adjoint of the NCEP global model that with the analysis adjoint will produce estimates of how every observation helps or hurts the forecast skill

• We will determine if smart statistics from these promising new tools will give us more precision on how to better use observations, such as:– Satellite radiance data impact estimates by type, channel,

surface conditions (water, ice, etc.) and more– Possibly better rules on filtering of satellite winds– Improved reject-list criterion– Possibly better analysis of dropout cases

Adjoint Codes

November 16, 2010 COPC Dropout Update 21

AI 2008-1.5 Official Action Plan (Revised)

• NCEP will generate warnings on a restricted website when the GFS global analysis has extreme localized differences from ECMWF analyses on a 1x1 degree grid

• NCEP will develop a real-time warning system to alert us when the GFS global model 5-day forecast height correlations with the ECMWF exceed normal limits, which will give an early warning of a possible dropout

• NCEP website will have graphics to allow staff to study the divergence in forecasts in real time and show likely analysis problem areas