the evolution of a cdr: proposed systematic program

The Evolution of a CDR:Proposed Systematic Program

Jeffrey L. Privette, NOAA

On behalf ofThe Joint Agency Study on the Climate Impacts of

the Nunn-McCurdy NPOESS Certification

22Proposed Systematic CDR Program

NRC Workshop on Recovery of Climate Measurements from NPOESS

CDR Status Quo

NASA, NOAA, USGS and other organizations have developed some CDRs as ad hoc research products (e.g., SST, Ozone, earth radiation budget)

Current investments (Approx. # of grants)

– NASA: ROSES’06 (A.15; Starting Fall 2007): 20– NOAA: CCDD (Stratified starts since 2004): 4– NOAA: SDS (Started Summer 2007): 7– Other activities indirectly supported through mission and

Research & Application Program investments



Future CDR Support Outlook

NASA, NOAA and USGS are studying the potential of NRC Decadal Survey missions to provide and/or advance CDRs– Post-mission funding of EOS-based products

not clear

CDR production, refinement, reprocessing is not in any operational agency budget

CDR storage (CLASS) and distribution (Data Centers) is not in NOAA budget



Joint-Agency CDR Program Goals

Incorporate NRC input Devise and cost an end-to-end program

– CDRs and Climate Information Records (CIRs)– Systematic and Coherent– Comprehensive (GCOS, IPCC, NRC, CCSP, etc.)– Sustainable – Stakeholder and Science Need-driven

Build on current investments and expertise– Interagency and International

Need systematic program for evolving CDRs



EVOLUTION OF A CLIMATE DATA RECORD (CDR)

Develop Joint Model of CDR Process

Develop a "Generic" Budget

Develop a list of Candidate CDRs focusing on the NPOESS

C1 Mission

Assess the Maturity and Cost Complexity of each Candidate

CDR

Priortize the Candidate CDRs for order of implementation

Develop a schedule of implementation with a gradual

start

Develop the accompanying budget

Planning Development Path



Program Architecture

CDRs develop through overlapping Research (RM) and Operational Missions (OM)– Joint agency cooperation on all science & applications of each

mission

Research-to-Operations occurs as function of CDR maturity– Developed 6-Level Maturity Matrix to define path– Not all sensor products will become Level 6 CDRs– Research programs continue developing algorithm alternatives and

advancements to challenge “released” Level 6 CDRs

3 community groups coordinate and manage CDR evolution– Working Group: Science team plans and executes development of

CDR (1 per CDR subset)– CDR Science Advisory Board: Senior climate scientists review and

prioritize CDR planning and development– Steering Committee: Government senior scientists and managers

coordinate budgetary matters and research-to-operations transitions



Core Activities in CDR Evolution

Sensor calibration and characterization Algorithm development and refinement

– Continuing incubation of algorithm alternatives that may eventual replace previous “standard”

Product (Re-)Processing– Research & Operational agencies co-generate

Maturity Level 3-4 products as part of transition– Assures transition readiness

Product validation and use-driven evaluation Archive, Distribution, Documentation



Notional Evolution of a CDR



Research and Operational Mission Cost Profiles

-15

-13

-11-9-7-5-3-11357911131517

Years Before OM Launch Years After OM Launch

Co

stResearch Mission

Operational Mission

Research and Operational Mission Cost Profiles

Launch

Peaks reflect higher intensity periods associated with a new product version,typically leading to a step-up in maturity level.



Cost Estimation Approach

1. Notional CDR lifecycle provides schedule and activity breakdown– NASA historical cost data estimates cost/activity– Separate Research and Operational cost profiles

2. Algorithm maturity* determines relative year in notional CDR lifecycle

3. Production complexity* determines multiplier of notional cost profile

4. CDR “ramp-up” rate treated as independent variable– Required CDR prioritization strawman

*Maturity and complexity estimates from joint agency sensor expert teams (names provided in April Panel brief)

Maturity Sensor Use Algorithm stability

Metadata & QA Documentation Validation Public

ReleaseScience &

Applications

1 Research MissionSignificant

changes likelyIncomplete Draft ATBD Minimal

Limited data availability to

develop familiarity

Little or none

2 Research MissionSome changes

expectedResearch grade

(extensive)ATBD Version 1+

Uncertainty estimated for select locations/times

Data available but of unknown

accuracy; caveats required for use.

Limited or ongoing

3 Research MissionsMinimal changes expected

Research grade (extensive);

Meets international

standards

Public ATBD; Peer-reviewed algorithm and product

descriptions

Uncertainty estimated over widely distribute

times/location by multiple investigators; Differences

understood.



Provisionally used in applications and

assessments demonstrating positive value.

4 Operational MissionMinimal changes expected

Stable, Allows provenance tracking and

reproducibility; Meets

international standards

Public ATBD; Draft Operational Algorithm

Description (OAD); Peer-reviewed algorithm and

product descriptions

Uncertainty estimated over widely distribute

times/location by multiple investigators; Differences

understood.



Provisionally used in applications and

assessments demonstrating positive value.

5

All relevant research and operational missions; unified and coherent record demonstrated

across different sensors

Stable and reproducible


reproducibility; Meeting


Public ATBD, Operational Algorithm Description

(OAD) and Validation Plan; Peer-reviewed algorithm, product and validation

articles

Consistent uncertainties estimated over most

environmental conditions by multiple investigators

Multi-mission record is publicly

available with associated uncertainty

estimate

Used in various published

applications and assessments by

different investigators

6

All relevant research and operational missions; unified and coherent record over complete

series; record is considered scientifically

irrefutable following extensive scrutiny

Stable and reproducible; homogeneous and published error budget


reproducibility; Meeting


Product, algorithm, validation, processing and

metadata described in peer-reviewed literature

Observation strategy designed to reveal

systematic errors through independent cross-checks,

open inspection, and continuous interrogation

Multi-mission record is publicly

available from Long-Term

archive

Used in various published

applications and assessments by

different investigators

CDR Maturity Matrix:Determines Start-Up Phase



Complexity Serves As Multiplier of Notional Cost Profile

Factors:

1) Number, quality & diversity of input streams

2) Resolutions (vertical, horizontal, temporal, spectral)

3) Algorithm complexity4) Algorithm outputs (#CDRs)5) Cal/Val complexity and cost



CDR Prioritization Strawman

Current cost scoping required prioritization strawman– CDR list from joint agency sensor expert teams

Name, Complexity

– Ranking followed sequential sorting: Maturity: In development? Significance to Global Change First launch date

– Caveats FCDRs deliberately ranked above TCDRs per sensor Forcing and State variables funded at 1:1 rate

In practice, CDR Science Advisory Board develops prioritization– 5-7 senior climatologists from community



Atmosphere TEMPERTURE PROFILES CrIMMS H X F 2009 1

Terrestrial REFLECTANCE PRODUCTS VIIRS H X S 2009 1

Atmosphere WATER VAPOR PROFILES CrIMMS H X F 2009 2

Terrestrial LAND/LAKE SURFACE TEMPERATURE VIIRS H X S 2009 2

Atmosphere RADIATION BUDGET ERBS H X F 2009 3

Terrestrial GLACIERS/ICE CAPS VIIRS H S 2009 3

Atmosphere OZONE OMPS H X F 2009 4

Ocean SEA PROPERTIES VIIRS M X S 2009 4

Significance: Global Change

In Develop-

ment

Rank (S)tate or (F)orcing

First Launch

Primary SensorDiscipline Area CDR Bundle

TCDR Prioritization Sample

etc. (28 CDR bundles in total)

CDR bundle is costing convenience to group CDRs typically produced from the same or similar algorithm(s).

Primary sensor is a costing convenience to associate each CDR with one and only one NPOESS era sensor. It is recognized that data from multiple satellite sensors, and in situ data, are often used to determine a CDR.

Significance to Global Change follows from a review of IPCC Fourth Assessment Report (2007).

In development: Based on NASA ROSES ’06 A.15 and NOAA SDS 2007 selections.

First launch considers only NPP/NPOESS era launches per the mandate of the NPOESS Climate Recovery activity.

State and Forcing variable bundles, as defined by CCSP Strategic Plan Chapter 12, are prioritized at a 1:1 ratio.



Different Ramp-Up Rates Costed

Cost

CDR Coverage (Full Maturity By 2026)

■ Slow Rate: Most reach Stage 6 Maturity

■ Proof-of-Concept: Some low complexity CDRs reach Stage 6 Maturity

■ Fast Rate: All reach Stage 6 Maturity

17 TCDRs by 2026 75 TCDRs by 2026

Program Ramp-Ups: Bounds

Proof-of-Concept Cost Profile

1993 1997 2001 2005 2009 2013 2017 2021 2025

Year

$M

RM Total (Inflated)

OM Total (Inflated)

Research FCDR Costs

Operational FCDR Costs

Research TCDR Costs

Operational TCDR Costs

Research CIR Costs

Operational CIR Costs

Fast Ramp Cost Profile

1993 1997 2001 2005 2009 2013 2017 2021 2025

Year

$M

RM Total (Inflated)

OM Total (Inflated)

Research FCDR Costs

Operational FCDR Costs

Research TCDR Costs

Operational TCDR Costs

Research CIR Costs

Operational CIR Costs

TCDR and CIR Starts: Fast Ramp

0

10

20

30

40

50

60

70

1993 1997 2001 2005 2009 2013 2017 2021 2025

Year

Co

un

t

New TCDRs in Production

TCDR Running Total

CIR Running Total

TCDR and CIR Starts: Proof-of-Concept Ramp

0

5

10

15

20

1993 1997 2001 2005 2009 2013 2017 2021 2025

Year

Co

un

t

New TCDRs in Production

TCDR Running Total

CIR Running Total



Cost Subsidies Under Study

Cost model estimates total cost of program

Core-funded agency activities can reduce new funding needs– NASA EOS and Research & Application

programs– Integrated Program Office activities



Notional Lead Roles and Responsibilities

Activity NASA NOAA/USGS

CDRDevelopment

Research Mission(s) / Theoretical

Development &Advancements

Multimission / Operational Refinement

CDR Production Stages 1-3 Stages 4-6

Archiving Single Mission Storage and Distribution

Long-term Archive and Distribution

Applications of CDRs

Partner Partner



Status and Next Steps

OSTP/OMP briefed in September Subsidization analysis now underway Will seek NRC Review in Fall Writing Phase 2 revision of the OSTP

Assessment (a.k.a. White Paper)



Thank You

Joint Team Members

NASA – Bryant Cramer, Jack Kaye, Dave Young, Charles Taylor, Thomas Jasin

NOAA – Chet Koblinsky, Mike Tanner, Jeff Privette, Tom Karl, John Bates, Mike Bonadonna, Kandis Boyd, Jim Oneal, Gary Davis, Brent Smith

USGS - DeWayne Cecil



Working Definitions

Working Definitions of Climate Products for NASA-NOAA Discussion1) Climate Data Record (CDR): A Climate Data Record is a time series of measurements

of sufficient length, consistency, and continuity to determine climate variability and change [NRC, 2005]. CDRs typically use data from different satellites and sensors extending from present back to the beginning of the relevant satellite observation period. For the NPOESS Climate Capability Restoration objectives, CDRs satisfy three additional characteristics:

– CDRs are approved and prioritized by the CDR Science Advisory Team (TBD) composed of climate science leaders representing government, academia and industry; Group reviews of the CDR every 3-5 years to ensure it meets objectives and remains a priority or should be adjusted or sunsetted.

– CDRs’ geophysical retrieval algorithms, underlying theoretical bases, and heritage products are mature, validated and proven useful in downstream research and applications.

– A CDR wholly or partially satisfies provision of a CCSP Earth Climate System Observation or a GCOS Essential Climate Variable, acknowledging that these consensus lists will change with time.

2) A Climate Information Product is a time series derived from CDRs and related long-term measurements to provide specific information about an environmental phenomena of particular importance to science and society. CIRs are often designed to convey key aspects of complex environmental phenomena in a manner useful to a variety of applications of particular interest to certain stakeholder communities.

– Examples of NOAA CIRs include: El Nino Occurrence/Persistence/Magnitude, Antarctic Ozone Hole Area and Magnitude, Drought Indices and Occurrence/Persistence/Magnitude, Hurricane Intensity and Tracks, Residential Energy Demand Temperature Index, Various Drought Indices



CDR Production Occurs Within An End-to-End Program



Sensor DataRecords (SDRs)

Data (Direct & Remotely Sensed)

Fundamental Climate Data

Records (FCDRs)

Thematic Climate Data Records (TCDRs)

Homogenization and Calibration

Time-tagged Geo-Referenced

Converted to Bio-Geophysical

Variables

EnvironmentalData Records

(EDRs)

Converted to Bio-Geophysical

Variables

CDR Development Requires Different Expertise, Data Paths and

ResourcesClimate Data Records

Climate Information Records (CIRs)

Climate Data Records or Homogenized Time Series



Example CDR Prioritization(Sort By IPCC, Algorithm Investment,

Launch)

CDR Bundle Candidate Climate Data RecordsTEMPERTURE

PROFILESLower Stratospheric Temperature Profile, Lower Tropospheric Temperature Profile, Upper

Tropospheric Temperature Profile, Surface Air TemperatureREFLECTANCE

PRODUCTSSurface Reflectance (BRF; NBAR), Blue Sky Albedo (instantaneous), Black/White Sky

Albedo,Vegetation IndexWATER VAPOR

PROFILESLower Stratospheric Water Vapor Profile, Lower Tropospheric Water Vapor Profile, Upper

Tropospheric Water Vapor Profile, Total Column Water Vapor, Surface Water VaporLAND/LAKE SURFACE

TEMPERATURE Land Surface Temperature, Land Surface Emissivity (TIR), Lake Surface Temperature

RADIATION BUDGETTop-of-Atmosphere Outgoing Long Wave & Short Wave Radiation, Surface Up/Down Long

Wave & Short Wave RadiationGLACIERS/ICE CAPS Glacier Extent, Ice Sheet Extent

OZONEOzone Stratospheric Profile , Tropospheric Ozone , Total Column Ozone , Coarse Ozone

Profile SEA PROPERTIES Sea Surface Temperature

CLOUD PROPERTIESCloud Column Water, Cloud Optical Thickness,Particle Phase, Particle Size, Cloud Top

Pressure, Cloud Top Temperature, Cloud Top Emissivity, Cloud Column Water VaporSNOW Snow Areal Extent

AEROSOLS Aerosol Particle Size, Aerosol Optical Thickness , Aerosol Refractive IndexSEA ICE

(TEMPERATURE) Ice Surface TemperatureCLOUD PROPERTIES Cloud Cover, Cloud Type

FIRES (ENERGY)Active Fire Area, Active Fire Occurrence, Active Fire Temperature, Fire Radiative Power

(FRP)

TRACE GASESTropospheric Column of CO, Tropospheric Column of CH4, Stratospheric CH4, Stratospheric

CO2, Tropospheric Column of CO2, Lower Tropospheric CO2, Tropospheric N2OLAND COVER

(THEMATIC) Land Cover Map (Thematic)

IRRADIANCEDownwelling Total Solar Radiation at Top-of-Atmosphere, Downwelling Spectral Solar

Radiation at Top-of-AtmosphereFIRES (BURNED AREA) Burned Area

PRECIPITATION Rain RateBIOPHYSICAL

PRODUCTS Fraction of Absorbed Photosynthetically Active Radiation (fAPAR), Leaf Area IndexOCEAN WINDS Global Ocean Surface Vector Winds (OSVW), Coastal Ocean Surface Vector Winds (OSVW)OCEAN COLOR

PRODUCTS Calcite Concentration, Fluorescence, Chlorophyll A Concentration, Total Suspended MatterLAKES Lake Area Extent

LAND COVER (Cont. Fields) Fractional Tree/Grass Cover

SEA LEVEL Basic Sea Level Geophysical Data Record (GDR), Significant Wave HeightSNOW DEPTH Snow Water Equivalent, Snow Depth



Candidate Climate Information Records from Mitigated NPOESS

Discipline Area

CIRs (list names)Primary Sensor Dependencies

Climate Forecast System VIIRS, CrIMMS, MISClimate Data Assimilation System (CDAS) / Reanalysis VIIRS, CrIMMS, MISOzone Hole Area OMPS, CrIMMSEarth Radiation Budget ERBSStratospheric Warming CrIMMSExtratropical Storm Tracks CrIMMSTropical Storm tracks VIIRS, CrIMMS, MISTeleconnection Indices CrIMMSMulti-Decadal Signal VIIRS, CrIMMS, MISTropical Atlantic Circulation Indices VIIRS, CrIMMS, MISAccumulated Cyclone Energy Index VIIRS, CrIMMS, MISNorth Atlantic Oscillation Index CrIMMSPrecipitation MISOzone Hole Size OMPS, CrIMMSPolarward Heat Flux CrIMMSPolar Vortex Area CrIMMSPolar Stratospheric Cloud Area CrIMMSCross Tropopause Mass Flux OMPS, CrIMMSStratosphere Temperature Monitoring CrIMMSTotal and Profile Ozone Monitoring OMPS, CrIMMSSW/LW Radiation (evaporation) CrIMMS

ENSO Indices VIIRS, CrIMMS, MISOcean Data Assimilation System VIIRS, CrIMMS, MISSurface Wind MISSurface Heat Flux VIIRS, CrIMMS, MISSalinity MISSea Level Height Altimetry

Drought coverage/intensity VIIRS, CrIMMS, MISFlood monitoring VIIRS, CrIMMS, MISSnow Cover Area VIIRSSoil Moisture MIS

Atmosphere

Terrestrial

Ocean

Climate Information Record (CIR)

A Climate Information Product is a time series derived from CDRs and related long-term measurements to provide specific information about an environmental phenomena of particular importance to science and society. CIRs are often designed to convey key aspects of complex environmental phenomena in a manner useful to a variety of applications of particular interest to certain stakeholder communities.

Year Primary CDR Program Events Key NASA Activities Key NOAA/USGS Activities

2007 Prelaunch sensor characterizationGround system development & testingCDR algorithm development

Sensor characterization and calibrationPrototype CDR Development & TestsPEATE Critical Design Review

CLASS protocolsMetadata /QA schemePrototype CDR Development & TestsParticipate in NASA PEATE activities

2008 Prelaunch sensor characterizationGround system development & testingCDR algorithm developmentIDPS-PEATES-CLASS-Data Centers Ground Readiness Review

Prototype CDR Development & TestsComplete VIIRS VIS/NIR FCDR AlgorithmGround Readiness Testing for FCDR productionTCDR alg focus: Glaciers/Ice Caps

Continue 2007 ActivitiesCDR system engineering & procurement (“Climate PEATE”)

2009 Start 1 CDR, 1 CIRNPP Launch (October)

Complete VIIRS TIR FCDR AlgorithmTest VIIRS FCDR AlgorithmsGround Readiness Testing for TCDR productionTCDR alg focus: Sea PropertiesTCDR tests: Glaciers/Ice Caps

Continue “Climate PEATE” developmentCLASS CDR Ground Readiness TestsTest NASA-NOAA CDR data protocolsImplement and Test CIR_1 code

2010 Start 2 CDRs, 2 CIRs Complete OMPS-N & –L FCDR AlgorithmsProduce VIIRS FCDRsTCDR alg focus: SnowTCDR tests: Sea PropertiesTCDR Production: Glaciers/Ice Caps

Archive/Distribution VIIRS FCDRsTransition VIIRS FCDR codesImplement and Test CIR_2 code

2011 Start 3 CDRs, 3 CIRs Produce VIIRS & OMPS FCDRsComplete CrIS & ATMS FCDR AlgorithmsTCDR alg focus: Sea Ice (Temp)TCDR tests: SnowTCDR Production: Sea Properties+2010

Co-produce: VIIRS Level 3&4 FCDRsTransition: OMPS –N & -L FCDR codesTransition: Glaciers/Ice Caps TCDRImplement and Test CIR_3 code

2012 Start 3 CDRs, 3 CIRs Complete ERBS & Jason FCDR AlgorithmsProduce OMPS, CrIMMS FCDRsTCDR alg focus: CloudsTCDR tests: Sea Ice (Temp)TCDR Production: Snow +2011

Co-produce OMPS Level 3&4 FCDRsCo-produce Glaciers/Ice Caps TCDRTransition CrIS & ATMS FCDR codesTransition: Sea Properties TCDRRefine/Produce VIIRS Level 5 FCDRsImplement and Test CIR_4 code

2013 Start 2 CDRs, 2 CIRsNPOESS C1 Launch

Test ERBS & Jason FCDR AlgorithmsProduce CrIMMS FCDRsTCDR alg focus: Land Cover MapTCDR tests: CloudsTCDR Production: Sea Ice (Temp)+2012

Co-produce CrIMMS Level 3&4 FCDRsCo-produce Sea Properties TCDRTransition ERBS & Jason FCDR codesTransition Snow TCDRRefine/Produce OMPS Level 5 FCDRRefine/Produce VIIRS Level 6 FCDRs

Draft Implementation In NPP Era

the evolution of a cdr: proposed systematic program

Documents