Observing System Evaluation Task Team Report

Peter Oke and Gilles LarnicolCSIRO and CLS

Talk Outline

2013 GOV review – implications for the OSEval-TT OSEval-TT membership Main achievements Focus areas for annual TT activities/projects Main priorities Expected outcomes Expected investment and return for investment Annual projects Synergies between GOV task teams

2013 GOV review

Review held in Nov 2013 OSEval–TT main achievements presented participation to

the discussion

Findings from the panel work to be of high quality work closely with GOOS/OOPC and Clivar/GSOP responsiveness were particularly noteworthy

2013 GOV review

Recommendations:1. further articulation of the OSEval work plan within the context

of the overall GOV strategy with: more explicit connection to partners; greater emphasis on system-wide evaluation.

2. to develop this TT as an objective and authoritative source of advice and evidence on the relevance of the observing system taking account of model and assimilation changes and improvements link with GOOS/OOPC, JCOMM; link with Observational bodies as CEOS, GHRSST, OSTST,

ARGO, …

OSEval-TT membership

Core Members: Magdalena Balmaseda (ECMWF) Anthony Weaver (CERFACS) Laurent Bertino (NERSC) Pavel Sakov (NERSC) Gary Brassington (BoM) Jim Cummings (NRL) Pat Hogan (NRL) Yosuke Fujii (JMA/MRI) Villy Kourafalou (Univ. Miami) Daniel Lea (UKMet) Matthew Martin (UKMet) Avichal Mehra (NOAA) Elisabeth Remy (Mercator-Océan) Greg Smith (EC) Clementa Tanajura (Remo) Zhang Tianyu (NMEFC)

Associate members:Mike Bell (UKMet)Eric Dombrowsky (Mercator-Océan)Fabrice Hernandez (Mercator-Océan)Eric Lindstrom (NASA)Andreas Schiller (CSIRO)Hans Bonekamp (EUMETSAT)

Co-Chairs: Peter Oke (CSIRO) Gilles Larnicol (CLS)

OSEval-TT membership

Core Members: Magdalena Balmaseda (ECMWF) Laurent Bertino (NERSC) Jim Cummings (NRL) Yosuke Fujii (JMA/MRI) Villy Kourafalou (Univ. Miami) Daniel Lea (UKMet) Elisabeth Remy (Mercator-Océan) Greg Smith (EC) Clementa Tanajura (Remo) Zhang Tianyu (NMEFC)

Associate members:Andreas Schiller (CSIRO)Hans Bonekamp (EUMETSAT)Paul DiGiacomo

Co-Chairs: Peter Oke (CSIRO) Gilles Larnicol (CLS)

Achievements: three community papers

GOV OSEval workshop December 10-11th 2014 (Toulouse) :

Jointly organised with GSOP-CLIVAR group & E-AIMS FP7 European Project

More than 25 abstracts submittedShould be focused on the results analysis and work plan

refinement- To be decided now

Achievements: OSEval-TT next workshop

Priorities

Community engagement - both within GOV, and with the observational agencies. Community activities for us means applying multiple systems to equivalent experiments - and is a way of establishing system-independent, robust results.

Workshopse.g., OSEval-GSOP

Quarterly newslettere.g., GOV newsletter, OSEval newsletter,

submission to Mercator newsletter

Special sessionse.g., AGU, EGU

Official OSEval-TT presentation

Participation at Obs meetingse.g., Argo Steering team, XBT steering

team, OSTST, …

Outcomes

Guidance for observational agencies to optimise the GOOS, and convince funders of the value of ocean observations to ocean forecasting and those beneficiaries of ocean forecasting

Defence Search & rescue

Environmental monitoring

Oil spill response

Oil and gas planning, assessment & approval

Fisheries management

Ship routing

Leisure (yatch racing, non-commercial fishing, idiots trying

to paddle small craft between continents)

Investment and benefit

It’s hard to motivate people to work for free … The efficacy of our community activities would be

substantially increased if we had a dedicated effort … dedicated PostDoctoral researcher

Technically coordinate community activities

Analyse community results

Disseminate resultsCost: $100K-

AU$180KPublish in peer-reviewed literature

Linkages with other TTs

OSEval IV DA

Annual project 1

Annual project 2

Annual project 3

Annual project 4

Annual project 1

Annual project 2

Annual project 3

Annual project 4

Annual project 1

Annual project 2

Annual project 3

Annual project 4

Linkages with other TTs

OSEval IV DA

Annual project 2

Annual project 3

OSEval-IV-DA Annual project 1

Annual project 2

Annual project 3

OSEval-IV Annual project 4

Annual project 2

Annual project 3

Annual project 4

Annual projects

NRT OSEs Routine monitoringRoutine demonstration of observation impact by disseminating information on what observations are assimilated into GOV systems. This is simply regularly (e.g., quarterly) reporting how many observations of each type (e.g., altimetry, Argo, XBT, TAO, AVHRR SST etc) are assimilated by each forecast system.

Information to be provided by each participating forecast systems for every assimilation cycle.

Forecast centers are encouraged to routinely compute affordable obs-impact metrics – and analyse results for contribution to the Official OSEval-TT presentation

Altimeter Argo XBT AVHRR … …

Bluelink 3x105 300 (2x104) 35 (4x103) 8x105

FOAM 3.5x105 200 (2x104) 50 (5x103) 9x105

Mercator 3.2x105 250 (2x104) 25 (3.5x103) 7x105

GIOPS 4x105 300 (2x104) 10 (1x103) 7.5x105

:

Sample only

Annual projects

Routine monitoring Annual community OSEs - COSEs

3 OSEs per year 6 month period Daily-mean “best-estimate” fields produced each week (i.e., 26 fields

provided) along with class 4 fields for the following set of OSEs:ALL: OSE that assimilates “all” observations;NONE: OSE that assimilates no observations; andCOSE: OSE that with-holds an agreed set of observations.

Analysed by a central agency

Annual projects

Routine monitoring Annual community OSEs - COSEs

ORAS4

FOAM REMO

Mercator Bluelink

Annual projects

Routine monitoring Annual community OSEs - COSEs

Fujii et al. (2014) – TPOS community paper

Annual projects

Routine monitoring Annual community OSEs - COSEs

Initial time End time Data with-held Due date7/2013 12/2013 Degraded Argo array (-20%) 4/2015

7/2014 12/2014 X altimeter(s) of the constellation (TBD) 4/2016

7/2015 12/2015 Tropical moorings 4/20177/2016 12/2016 X% of Argo (TBD) 4/2018

7/2017 12/2017 X altimeter(s) of the constellation (TBD) 4/2019

Annual projects

Routine monitoring Annual community OSEs Inter-comparison of single-obs impacts;

Inter-comparison of single observation impacts: using an array of “idealised background innovations” (representing an

difference between a set of hypothetical observations and each system’s background field) participating agencies will compute

increments derived from their forecast system.

Obs X variable Y by Z% …

Annual projects

Routine monitoring Annual community OSEs Inter-comparison of single-obs impacts;

System 1(EnKF)

System 1(EnOI)

Sandery et al. (2014)

Covariance shown – not increments

Summary

Annual Projects

1. Routine monitoring

2. Annual community OSEs

3. Inter-comparison of single-obs impacts

Ongoing Initiatives Official OSEval-TT presentation; Encourage forecast centers to routinely compute

affordable obs-impact metrics (e.g., DFS, SRF); Compare conclusions drawn from OSEs and affordable

obs-impact metrics; and Community OSSE guidance.

Strategic Plan document update