1

FIRST ANNUAL NUOPC WORKSHOP Preliminary Report

FIRST ANNUAL NUOPC WORKSHOP Preliminary Report

Scott SandgatheNUOPC Technical Lead

19 Aug 2010

Scott SandgatheNUOPC Technical Lead

19 Aug 2010

OutlineOutline

• NUOPC Vision and Goals• NUOPC Progress Report• NUOPC R&D Workshop

o Quick look!

• NUOPC Vision and Goals• NUOPC Progress Report• NUOPC R&D Workshop

o Quick look!

2

3

NUOPC STRATEGIC VISIONNUOPC STRATEGIC VISION

•A National System with a Tri-Agency commitment to address common requirements

o Technical architectureo Committing sufficient funding, facilities and human capital

resourceso Focusing research on coordinated national objectives

•Multi-component system with interoperable components built upon common standards and a framework such as the ESMF

•A National System with a Tri-Agency commitment to address common requirements

o Technical architectureo Committing sufficient funding, facilities and human capital

resourceso Focusing research on coordinated national objectives

•Multi-component system with interoperable components built upon common standards and a framework such as the ESMF

4

•Managed ensemble diversity •Significantly improve forecast accuracy•Quantify, bound, and reduce forecast uncertainty

•Joint ensemble •Produce most probable forecast, e.g. high impact weather•Provide mission specific ensemble products•Drive high-resolution regional/local predictions•Drive other down stream models.

•Establish a national global NWP research agenda to accelerate development and transition

•Managed ensemble diversity •Significantly improve forecast accuracy•Quantify, bound, and reduce forecast uncertainty

•Joint ensemble •Produce most probable forecast, e.g. high impact weather•Provide mission specific ensemble products•Drive high-resolution regional/local predictions•Drive other down stream models.

•Establish a national global NWP research agenda to accelerate development and transition

NUOPC STRATEGIC VISION (2)NUOPC STRATEGIC VISION (2)

5

GOALSGOALS• Accelerate improvement of our National capability by:

o Implementing a global atmospheric ensemble system designed to enhance predictive capability

o Clearly articulating operational requirements and a corresponding National research agenda, with initial emphasis on hurricane track/intensity forecasts, joint wind and seas forecasts, and ceiling/visibility forecasts

o Sharing the predictive burden among the operational agencieso Promoting collaboration on development among government

agencieso Accelerating the transition of new technology into the operational

centerso Implementing ways to enhance broad community participation in

addressing the National research agenda

• Accelerate improvement of our National capability by:o Implementing a global atmospheric ensemble system designed to

enhance predictive capabilityo Clearly articulating operational requirements and a corresponding

National research agenda, with initial emphasis on hurricane track/intensity forecasts, joint wind and seas forecasts, and ceiling/visibility forecasts

o Sharing the predictive burden among the operational agencieso Promoting collaboration on development among government

agencieso Accelerating the transition of new technology into the operational

centerso Implementing ways to enhance broad community participation in

addressing the National research agenda

6

NUOPC PROGRESS REPORTNUOPC PROGRESS REPORT• Implementation of an operational multi-agency global

atmosphere ensembleo Initial capability based on NAEFS scheduled for 16 Dec 2010o Currently successfully passing fields from FNMOC to NCEP for testing in

NAEFS ensemble• Agreement on how to manage a multi-agency ensemble in an

operational environmento Being addressed by Committee for Operational Processing Centers

(COPC)o Annex to current Data Acquisition, Processing, and Exchange (DAPE)

agreement• Shared post processing agreement

o Working group addressing who, where, when, how mucho Development of common post processing toolbox

• Implementation of an operational multi-agency global atmosphere ensembleo Initial capability based on NAEFS scheduled for 16 Dec 2010o Currently successfully passing fields from FNMOC to NCEP for testing in

NAEFS ensemble• Agreement on how to manage a multi-agency ensemble in an

operational environmento Being addressed by Committee for Operational Processing Centers

(COPC)o Annex to current Data Acquisition, Processing, and Exchange (DAPE)

agreement• Shared post processing agreement

o Working group addressing who, where, when, how mucho Development of common post processing toolbox

77

COMMITTEE PROGRESS REPORT COMMITTEE PROGRESS REPORT • Committees currently addressing details of greater

collaboration and coordinationo Common Model Architecture Committee addressing overall

architecture and future standards (presentation later)o Content Standards Committee working with Earth System Modeling

Framework (ESMF) committees to address standardized implementation of the Earth System Modeling Framework and development of a NUOPC layer.

o Technology Transition Processes Committee addressing common operational needs, developing a research and development agenda and outreach including hosting this workshop.

o Metrics Subcommittee working on agreement for standard evaluation of model and ensemble performance, standard climatology, standard test cases, metrics for operational impact

• Committees currently addressing details of greater collaboration and coordinationo Common Model Architecture Committee addressing overall

architecture and future standards (presentation later)o Content Standards Committee working with Earth System Modeling

Framework (ESMF) committees to address standardized implementation of the Earth System Modeling Framework and development of a NUOPC layer.

o Technology Transition Processes Committee addressing common operational needs, developing a research and development agenda and outreach including hosting this workshop.

o Metrics Subcommittee working on agreement for standard evaluation of model and ensemble performance, standard climatology, standard test cases, metrics for operational impact

8

WORKSHOP GOALSWORKSHOP GOALS

Bring the operational community together with the research and development community and the funding agencies to arrive at a requirements-driven, prioritized research agenda for global atmospheric ensemble prediction and post processing.

Bring the operational community together with the research and development community and the funding agencies to arrive at a requirements-driven, prioritized research agenda for global atmospheric ensemble prediction and post processing.

9

OBJECTIVESOBJECTIVES

• Review NUOPC progress• Review Agency operational needs and

identify common needs• Review ongoing ensemble research• Propose prioritized research and

development agenda to meet common needs

• Review NUOPC progress• Review Agency operational needs and

identify common needs• Review ongoing ensemble research• Propose prioritized research and

development agenda to meet common needs

10

Session 3: OPS Needs vs EnsemblesConsolidated Report??

Session 3: OPS Needs vs EnsemblesConsolidated Report??

Given the operational needs presented:• What is common?• Which ones are appropriate for ensembles to

address? • Where do ensembles have to be in 10 years to

meet these needs?• What infrastructure is required?

Common needsCommon needs• Better support for decision making

o Mission-related (protection of assets, safety of personnel – Navy and AF)

o Emergency management (life and property – NOAA)• Common interests

o Aviation (turbulence, icing, ceiling, visibility)o Tropical cyclones track and intensityo Maritime safety and routingo Air quality and visibilityo Surface weather variableso High impact events

Common needs (2)Common needs (2)• Improved Global Forecast Performance• Support Coupling (mesoscale, ocean, etc)• Sharp and reliable probability based products/tools to

support improved decision making• Improved deterministic sensible weather forecasts.• Tools for interpretation of ensemble output for

improved decision making (More?)• Training of forecasters and other end users in the use

of ensemble products, including social scientists. (More?)

Common needs (3)Common needs (3)• Improved means for gathering user requirements and

understanding decision processes.• Improved verification metrics and scorecards, including

extension into end user decision making processes and the production and archiving of an “analysis of record. Post Processing Toolkit (Calibration)

• Communication/Visualization Toolso Global and Mesoscale

• Training, Education, and Marketing• Metrics• Research to Ops Support• Common Standards and Architecture• Research support tools (data, archive, etc)

Infrastructure requiredInfrastructure required• The number one mentioned infrastructure requirement by all groups was

computational resources adequate to handle high resolution EnDA, ensembles, and post processing including reforecasting, to support long term research.

• All groups felt adequate communications were required to pass, access rapidly, and distribute complex ensemble fields and products.

• All groups mentioned tools and software support for efficient data access to enhance R2O.

• Two groups mentioned data storage adequate to handle very large historical ensemble data sets and to support long term research.

• Two groups mentioned training or training infrastructure. • Two groups mentioned outreach or marketing with the assistance of social

scientists – infrastructure to support customer feedback.• One group mentioned verification.• One group mentioned adequate observations – infrastructure to assess data

needed to meet goals.

Big Picture - NeedsBig Picture - Needs

• Computer Resources, infrastructureo For operationso For transition to opso For research on operational systems

• Marketing infrastructureo Assess user needso Assess user valueo Present/adapt new products

• Computer Resources, infrastructureo For operationso For transition to opso For research on operational systems

• Marketing infrastructureo Assess user needso Assess user valueo Present/adapt new products

15

16

Needs driven research agendaNeeds driven research agenda

Given the common needs for ensembles:

• What are realistic goals?• What are the critical research areas?• What are the gaps? • Can we prioritize? • What specific additional resources are

required?

Given the common needs for ensembles:

• What are realistic goals?• What are the critical research areas?• What are the gaps? • Can we prioritize? • What specific additional resources are

required?

17

Realistic GoalsRealistic Goals

• Two day forecast improvement/decade in global scale

• Improve the prediction of position and timing of mesoscale phenomena to match nowcasting. Reduce sensible prediction errors from 6 hr to 1 hr in below 20 km scale.

• Nonhydrostatic, new dynamic core global forecast system.

• Fully coupled earth prediction system, air, ocean, sea ice, land, and wave models

• Coupled data assimilation

• Two day forecast improvement/decade in global scale

• Improve the prediction of position and timing of mesoscale phenomena to match nowcasting. Reduce sensible prediction errors from 6 hr to 1 hr in below 20 km scale.

• Nonhydrostatic, new dynamic core global forecast system.

• Fully coupled earth prediction system, air, ocean, sea ice, land, and wave models

• Coupled data assimilation

18

Realistic GoalsRealistic Goals

• Research to inform what makes sense on how structure operations.

• Improved estimation and quantification of weather related socio-economic impacts

• Continue to drive model predictability (and forecastability) towards true predictability

• Close the Valley of Death (testbeds, application research, tools/applications)

• Research to inform what makes sense on how structure operations.

• Improved estimation and quantification of weather related socio-economic impacts

• Continue to drive model predictability (and forecastability) towards true predictability

• Close the Valley of Death (testbeds, application research, tools/applications)

19

Critical Research AreasCritical Research Areas

• Trade off between number of ensemble and resolutions.

• Initial condition uncertaintyo Critical for small scales and multi-scale

phenomena. o How to perturb the initial conditions. o Obs uncertainty vs background uncertainty.

Improve the initial condition of the deterministic model.

• Model uncertainty. o Dynamics and physics

• Trade off between number of ensemble and resolutions.

• Initial condition uncertaintyo Critical for small scales and multi-scale

phenomena. o How to perturb the initial conditions. o Obs uncertainty vs background uncertainty.

Improve the initial condition of the deterministic model.

• Model uncertainty. o Dynamics and physics

20

Critical Research AreasCritical Research Areas

• Multi-scale error growth.• Verification and calibration for ensemble forecast,

o Post processing,o Tools for customerso Re-forecasts

• Improved understanding of atmospheric processes and phenomena

• The science of predictiono Reducing and accounting for model inadequacy o Data Assimilation o Observing systemso Intrinsic midlatitude predictability o The error dynamics of ensemble prediction

systems

• Multi-scale error growth.• Verification and calibration for ensemble forecast,

o Post processing,o Tools for customerso Re-forecasts

• Improved understanding of atmospheric processes and phenomena

• The science of predictiono Reducing and accounting for model inadequacy o Data Assimilation o Observing systemso Intrinsic midlatitude predictability o The error dynamics of ensemble prediction

systems

21

Critical Research AreasCritical Research Areas

• Decision and Behavior research, o Ties with the funding opportunity. o Economic value/benefit. o Operational Risk management tool. o Decision based on cost savings. Best

communication tools.

• Decision science research and applicationso Use of forecast information in decision making o Communicating forecast uncertainty o User-relevant verificationo Quantifying the value of forecasts o Developing decision support tools and systems

• Decision and Behavior research, o Ties with the funding opportunity. o Economic value/benefit. o Operational Risk management tool. o Decision based on cost savings. Best

communication tools.

• Decision science research and applicationso Use of forecast information in decision making o Communicating forecast uncertainty o User-relevant verificationo Quantifying the value of forecasts o Developing decision support tools and systems

22

Critical Research AreasCritical Research Areas

• Analysis related uncertainties:o Best way to exploit ensemble covariance info

• 2-way couplingo Dealing w/ non-Gaussian error distributions

• Particularly for fine scale• Moisture

o Non-linearity and non-locality in observation operators

o Coupled systemo Data preprocessing

• QC• Thinning

o Tools for assessing observing systems

• Analysis related uncertainties:o Best way to exploit ensemble covariance info

• 2-way couplingo Dealing w/ non-Gaussian error distributions

• Particularly for fine scale• Moisture

o Non-linearity and non-locality in observation operators

o Coupled systemo Data preprocessing

• QC• Thinning

o Tools for assessing observing systems

23

Critical Research AreasCritical Research Areas

• Uncertainty exploitation, customer dependent.

• Model related uncertainties:– Model physics paradigm shift

• Cloud resolving model• Probabilistic model physics

parameterization

– Multi-model vs. one model w/ stochastic physics/parameters/etc.

• Configurations for multi-model• Acquisition lifecycle cost considerations

• Uncertainty exploitation, customer dependent.

• Model related uncertainties:– Model physics paradigm shift

• Cloud resolving model• Probabilistic model physics

parameterization

– Multi-model vs. one model w/ stochastic physics/parameters/etc.

• Configurations for multi-model• Acquisition lifecycle cost considerations

24

Critical Research AreasCritical Research Areas

– Calibration• Best technique(s) (w.r.t. applications, ops)• Min training datasets (variables, scales,

etc.)• Techniques for high-dim problems (e.g.

aircraft route)• Handling derived variables• Train on observations or analyses• Modular (e.g., bias cor. first) vs. holistic

approach• Re-forecasts

– Calibration• Best technique(s) (w.r.t. applications, ops)• Min training datasets (variables, scales,

etc.)• Techniques for high-dim problems (e.g.

aircraft route)• Handling derived variables• Train on observations or analyses• Modular (e.g., bias cor. first) vs. holistic

approach• Re-forecasts

– Applications, Forecast System Configuration • Multiscale modeling (global, regional, or

mixed), value to customer, and costs in ops• Best way to interact with users – feedback loop

into products• Ways to effectively educate• Propagating uncertainty into expensive

downstream applications (storm surge, dispersion, etc.)

Critical Research AreasCritical Research Areas

• Verification:o Probabilistic Metrics for high-dimensional,

multi-variate applicationso MORE data, and data mining tools

• HPC designo Software engineeringo GPU?o Exploit new architectureso Fault tolerance

• Verification:o Probabilistic Metrics for high-dimensional,

multi-variate applicationso MORE data, and data mining tools

• HPC designo Software engineeringo GPU?o Exploit new architectureso Fault tolerance

Critical Research AreasCritical Research Areas

Big Picture – R&DBig Picture – R&D

• Ask me tomorrow??• Ask me tomorrow??

27