system quality assurance document atmospheric ... -...

ECMWF COPERNICUS REPORT

Copernicus Climate Change Service

System Quality Assurance Document Atmospheric Composition ECVs Annex C: Greenhouse Gases (CO2 & CH4)

Version 1.3

Issued by: Michael Buchwitz, Germany, University of Bremen,

Institute of Environmental Physics (IUP)

Date: 30/09/2019

Ref: C3S_D312b_Lot2.2.0-v2_SQAD_201909_Annex_C_GHG_v1.3

Official reference number service contract: 2018/C3S_312b_Lot2_DLR/SC1

This document has been produced in the context of the Copernicus Climate Change Service (C3S). The activities leading to these results have been contracted by the European Centre for Medium-Range Weather Forecasts, operator of C3S on behalf of the European Union (Delegation Agreement signed on 11/11/2014). All information in this document is provided "as is" and no guarantee or warranty is given that the information is fit for any particular purpose. The user thereof uses the information at its sole risk and liability. For the avoidance of all doubts, the European Commission and the European Centre for Medium-Range Weather Forecasts has no liability in respect of this document, which is merely representing the authors view.


C3S_312b_Lot2_DLR SQAD-GHG of 37

Contributors

INSTITUTE OF ENVIRONMENTAL PHYSICS (IUP), UNIVERSITY OF BREMEN, BREMEN, GERMANY (IUP) M. Buchwitz M. Reuter O. Schneising-Weigel

SRON NETHERLANDS INSTITUTE FOR SPACE RESEARCH, UTRECHT, THE NETHERLANDS (SRON) I. Aben L. Wu O. P. Hasekamp

UNIVERSITY OF LEICESTER, LEICESTER, UK (UoL) H. Boesch A. Di Noia

CENTRE NATIONAL DE LA RECHERCHE SCIENTIFIQUE (CNRS), LABORATOIRE DE METEOROLOGIE DYNAMIQUE (LMD), PALAISEAU, FRANCE (LMD/CNRS) C. Crevoisier R. Armante



History of modifications

Version Date Description of modification Chapters / Sections

v1.0 19-September-2017 New document All

v1.1 5-October-2017

One sentence corrected in Sect. 1.2.1.4

Paragraph added addressing the switch from ERA-Interim to ERA5

Sect. 5 (User Support) updated

Sect. 1.2.1.4

Sect. 4

Sect. 5

v1.2 5-March-2019

Minor adjustments primarily for contract change (312a_Lot6 -> 312b_Lot2) plus several minor updates and corrections

All

V1.3 21-May-2019 Minor corrections as suggested by ASSIMILA

30-September-2019 Re-issued without any change



Related documents

Reference ID

Document

D1

GCOS-154: Global Climate Observing System (GCOS), SYSTEMATIC OBSERVATION REQUIREMENTS FOR SATELLITE-BASED PRODUCTS FOR CLIMATE, Supplemental details to the satellite-based component of the “Implementation Plan for the Global Observing System for Climate in Support of the UNFCCC (2010 update)”, Prepared by World Meteorological Organization (WMO), Intergovernmental Oceanographic Commission, United Nations Environment Programme (UNEP), International Council for Science, Doc.: GCOS 154, link: http://cci.esa.int/sites/default/files/gcos-154.pdf, 2011.

D2

GCOS-200: The Global Observing System for Climate: Implementation Needs, GCOS 2016 Implementation Plan, World Meteorological Organization (WMO), GCOS-200 (GOOS-214), pp. 325, link: https://library.wmo.int/opac/doc_num.php?explnum_id=3417, 2016.

D3 ESA-CCI-GHG-URDv2.1: Chevallier, F., et al., User Requirements Document (URD), ESA Climate Change Initiative (CCI) GHG-CCI project, Version 2.1, 19 Oct 2016, link: http://www.esa-ghg-cci.org/?q=webfm_send/344, 2016.

D4

TRD GHG, 2017: Buchwitz, M., Aben, I., Anand, J., Armante, R., Boesch, H., Crevoisier, C., Detmers, R. G., Hasekamp, O. P., Reuter, M., Schneising-Weigel, O., Target Requirement Document, Copernicus Climate Change Service (C3S) project on satellite-derived Essential Climate Variable (ECV) Greenhouse Gases (CO2 and CH4) data products (project C3S_312a_Lot6), Version 1.2, 21-August-2017, pp. 53, 2017.



Acronyms

Acronym Definition

AIRS Atmospheric Infrared Sounder

AMSU Advanced Microwave Sounding Unit

ATBD Algorithm Theoretical Basis Document

BESD Bremen optimal EStimation DOAS

CAR Climate Assessment Report

C3S Copernicus Climate Change Service

CCDAS Carbon Cycle Data Assimilation System

CCI Climate Change Initiative

CDR Climate Data Record

CDS (Copernicus) Climate Data Store

CMUG Climate Modelling User Group (of ESA’s CCI)

CRG Climate Research Group

D/B Data base

DOAS Differential Optical Absorption Spectroscopy

EC European Commission

ECMWF European Centre for Medium Range Weather Forecasting

ECV Essential Climate Variable

EMMA Ensemble Median Algorithm

ENVISAT Environmental Satellite (of ESA)

EO Earth Observation

ESA European Space Agency

EU European Union

EUMETSAT European Organisation for the Exploitation of Meteorological Satellites

FCDR Fundamental Climate Data Record

FP Full Physics retrieval method

FTIR Fourier Transform InfraRed

FTS Fourier Transform Spectrometer



GCOS Global Climate Observing System

GEO Group on Earth Observation

GEOSS Global Earth Observation System of Systems

GHG GreenHouse Gas

GOME Global Ozone Monitoring Experiment

GMES Global Monitoring for Environment and Security

GOSAT Greenhouse Gases Observing Satellite

GDAS GOSAT Data Archive Service

IASI Infrared Atmospheric Sounding Interferometer

IMAP-DOAS (or IMAP) Iterative Maximum A posteriori DOAS

IPCC International Panel in Climate Change

IUP Institute of Environmental Physics (IUP) of the University of Bremen, Germany

JAXA Japan Aerospace Exploration Agency

JCGM Joint Committee for Guides in Metrology

L1 Level 1

L2 Level 2

L3 Level 3

L4 Level 4

LMD Laboratoire de Météorologie Dynamique

MACC Monitoring Atmospheric Composition and Climate, EU GMES project

NA Not applicable

NASA National Aeronautics and Space Administration

NetCDF Network Common Data Format

NDACC Network for the Detection of Atmospheric Composition Change

NIES National Institute for Environmental Studies

NIR Near Infra Red

NLIS LMD/CNRS neuronal network mid/upper tropospheric CO2 and CH4 retrieval algorithm

NOAA National Oceanic and Atmospheric Administration

Obs4MIPs Observations for Climate Model Intercomparisons

OCO Orbiting Carbon Observatory

OE Optimal Estimation



PBL Planetary Boundary Layer

ppb Parts per billion

ppm Parts per million

PR (light path) PRoxy retrieval method

PVIR Product Validation and Intercomparison Report

QA Quality Assurance

QC Quality Control

REQ Requirement

RMS Root-Mean-Square

RTM Radiative transfer model

SCIAMACHY SCanning Imaging Absorption spectroMeter for Atmospheric ChartographY

SCIATRAN SCIAMACHY radiative transfer model

SRON SRON Netherlands Institute for Space Research

SWIR Short Wava Infra Red

TANSO Thermal And Near infrared Sensor for carbon Observation

TANSO-FTS Fourier Transform Spectrometer on GOSAT

TBC To be confirmed

TBD To be defined / to be determined

TCCON Total Carbon Column Observing Network

TIR Thermal Infra Red

TR Target Requirements

TRD Target Requirements Document

WFM-DOAS (or WFMD) Weighting Function Modified DOAS

UoL University of Leicester, United Kingdom

URD User Requirements Document

WMO World Meteorological Organization

Y2Y Year-to-year (bias variability)



General definitions

Table 1 lists some general definitions relevant for this document.

Table 1: General definitions.

XCO2 Column-averaged dry-air mixing ratio (mole fraction) of CO2

XCH4 Column-averaged dry-air mixing ratio (mole fraction) of CH4

L1 Level 1 satellite data product: geolocated radiance (spectra)

L2 Level 2 satellite-derived data product: Here: CO2 and CH4 information for each ground-pixel

L3 Level 3 satellite-derived data product: Here: Gridded CO2 and CH4 information, e.g., 5 deg times 5 deg, monthly

L4 Level 4 satellite-derived data product: Here: Surface fluxes (emission and/or uptake) of CO2 and CH4



Table of Contents

History of modifications 4

Related documents 5

Acronyms 6

General definitions 9

Scope of the document 11

Executive summary 12

1. System overview 13

1.1 System elements and interfaces 13 1.1.1 General overview about the processing chain 13

1.2 Hardware, supercomputers and cloud computing 16 1.2.1 SRON sub-system 16 1.2.2 UoL sub-system 21 1.2.3 CNRS-LMD sub-system 26 1.2.4 IUP-UB sub-system 29

2. Upgrade cycle implementation procedure 32

3. Procedures for reprocessing CDR’s 32

4. System maintenance and system failures 33

5. User support 34

References 35



Scope of the document

This document is the System Quality Assurance Document (SQAD) for the Copernicus Climate Change Service (C3S, https://climate.copernicus.eu/) component as covered by the greenhouse gas (GHG) aspects of project C3S_312b_Lot2 led by DLR, Oberpfaffenhofen, Germany.

Within this project satellite-derived atmospheric carbon dioxide (CO2) and methane (CH4) Essential Climate Variable (ECV) data products are generated and made available for ECMWF for inclusion into the Copernicus Climate Data Store (CDS, https://cds.climate.copernicus.eu/), from where users can access these data products and the corresponding documentation.

These satellite-derived data products are:

Column-average dry-air mixing ratios (mole fractions) of CO2 and CH4, denoted XCO2 (in parts per million, ppm) and XCH4 (in parts per billion, ppb), respectively.

Mid/upper tropospheric mixing ratios of CO2 (in ppm) and CH4 (in ppb).



Executive summary

In this System Quality Assurance Document (SQAD) of the Copernicus Climate Change Service (C3S) project C3S_312b_Lot2 the system is described, which is used to generate satellite-derived atmospheric carbon dioxide (CO2) and methane (CH4) Essential Climate Variable (ECV) data products.

These satellite-derived data products are:

Column-average dry-air mixing ratios (mole fractions) of CO2 and CH4, denoted XCO2 (in parts per million, ppm) and XCH4 (in parts per billion, ppb), respectively.

Mid/upper tropospheric mixing ratios of CO2 (in ppm) and CH4 (in ppb).

Described are the system elements and its interfaces including the presentation of a general overview.

The system is a distributed system consisting of several sub-systems, each generating a sub-set of the data products. Each sub-system has been implemented at a different institution, which has the appropriate expertise to generate the corresponding data products.

These data products are primarily individual sensor Level 2 products, i.e., “swath or footprint products” containing information for each individual satellite ground pixel. These individual sensor Level 2 products are collected by University of Bremen, responsible for overall quality assurance. At University of Bremen these products are merged to generate merged Level 2 (mL2) and merged Level 3 (mL3) products.

Also described in this document are the procedures implemented to upgrade the processing cycle, system maintenance aspects and how to deal with system failures. Finally, the planned user support is described.



1. System overview

1.1 System elements and interfaces

In this section all elements that contribute to the processing chain are described, including how the interfaces to C3S as well as to ancillary data providers are set-up.

1.1.1 General overview about the processing chain

Figure 1 presents and overview about the C3S_312b_Lot2 project greenhouse gas (GHG) component of C3S. On the left hand side, the input data are shown. This includes:

the satellite Level 1 data products (radiance spectra) in particular of TANSO-FTS/GOSAT (Kuze et al., 2009) and IASI on Metop-A and Metop-B

o the TANSO-FTS/GOSAT Level 1 data are obtained from the ESA Third Party Mission archive or directly from NIES, Japan, via the GOSAT Data Archive Service (GDAS)

o the IASI Level 1 data are obtained from EUMETSAT through the EUMETCast system.

and additional data sets needed for the generation of the greenhouse gas Level 2 data products:

meteorological data (primarily ERA Interim (Dee et al., 2011))

model data (primarily CarbonTracker (Peters et al., 2007) CO2 for the methane proxy (PR) algorithm products)

and other data sets (e.g., spectroscopic line parameters and surface topography))

Research and Development (R&D) is covered by external projects, in particular via the planned GHG-CCI follow-on project GHG-CCI+ (http://www.esa-ghg-cci.org/) of ESA’s Climate Change Initiative (CCI, Hollmann et al., 2007).

In the middle part of Fig. 1, the main components as carried out by C3S_312b_Lot2/GHG are shown. This comprises the sub-systems to generate individual sensor Level 2 (L2) products from the external input data using retrieval algorithms, which generate the CO2 and CH4 atmospheric data products (e.g., Buchwitz et al., 2015, 2016, 2017; Butz et al., 2011; Crevoisier et al., 2009, 2013; Detmers et al., 2015; Parker et al., 2011; Reuter et al., 2013, 2016).

These data products are collected and archived and used for quality control and to generate the merged Level 2 (mL2) and merged Level 3 (mL3) data products. These products along with their documentation will be made available to users via the Copernicus Climate Data Store (CDS) (https://climate.copernicus.eu/). C3S / C3S_312b_Lot2 also provides user support, e.g., via answering questions from the users.



Figure 1 - Overview C3S_312b_Lot2/GHG component of C3S.

The input data gathering, processing and data provision system is decentralized and consists of four sub-systems (see also Figure 1) implemented at the following institutions:

IUP-UB: Institute of Environmental Physics (IUP), University of Bremen (UB), Germany o Responsibilities:

Collection of all individual sensor Level 2 data products from project partners and external institutions via ftp server

Comprehensive quality control and additional validation and Key Performance Indicator (KPI) assessment of all XCO2 and XCH4 Level 2 and Level 3 products; final quality check of the IASI mid-tropospheric CO2 and CH4 products

Generation of the merged XCO2 and XCH4 Level 2 and Level 3 products Archiving of data products and documentation Dashboard Delivery of the data products to DLR via ftp Main point of contact for user support

SRON: SRON Netherlands Institute for Space Research, Utrecht, The Netherlands o Responsibilities:

Collection of all input data needed for generation of the SRON GOSAT Level 2 XCO2 and XCH4 data products

Generation of GOSAT Level 2 XCO2 and XCH4 data products including quality control and validation and corresponding documentation

Transfer of data products to the IUP-UB ftp server



UoL: University of Leicester, Leicester, UK o Responsibilities:

Collection of all input data needed for generation of the UoL GOSAT Level 2 XCO2 and XCH4 data products

Generation of GOSAT Level 2 XCO2 and XCH4 data products including quality control and validation and corresponding documentation

Transfer of data products to the IUP-UB ftp server

CNRS-LMD: Centre National de la Recherche Scientifique-Laboratoire de Météorologie Dynamique, Palaiseau, France

o Responsibilities: Collection of all input data needed for generation of the IASI Level 2 mid

tropospheric CO2 and CH4 data products Generation of IASI Level 2 CO2 and CH4 data products including quality control

and validation and corresponding documentation Transfer of data products to the IUP-UB ftp server



1.2 Hardware, supercomputers and cloud computing

In this section, an overview about the computer hardware as used in the processing chain(s) are described.

1.2.1 SRON sub-system

1.2.1.1 Overview of Processing Sub-System

Figure 2 provides a schematic overview of the RemoteC GHG-CCI processing sub system at SRON (Butz et al., 2011, Schepers et al., 2012). The first step is to download the required data from the respective data servers to SRON (GOSAT data and ECMWF data are dynamic datasets that are continuously updated, SRTM topography is a static dataset). In the next step a pre-processing program is combining all relevant information per GOSAT ground pixel. This includes interpolation of ECMWF data in space and time to the coordinates of the GOSAT ground pixel and calculating the average height of a GOSAT ground pixel and its standard deviation from the topography database.

Figure 2 - Schematic overview of SRON’s RemoteC algorithm processing sub-system.



The pre-processor produces for each GOSAT L1B file an auxiliary input file, hereafter referred to as the ‘retrieval ini ‘ file, that contains this information. In the next step columns of CO2, CH4, H2O, and O2 are retrieved under the assumption of an atmosphere without aerosol/cirrus/cloud scattering ('RemoTeC non scattering mode' in Figure 2). The outcome of these retrievals (in an intermediate ASCII output file) is used to create the RemoTeC XCH4 proxy product, but also for cloud filtering to select cloud free scenes to be processed by the RemoTeC Full Physics algorithm. The Full Physics retrieval produces intermediate (ASCII) output files which go into an a posterior filtering procedure, quality check (based on non-convergence, parameter boundary hits, retrieved aerosol parameters), and bias correction and finally a NetCDF output file is created.

Figure 3 gives an schematic overview of the core RemoTeC retrieval algorithm (same for non-scattering and Full Physics). Here, multi-threading capability is implemented using openMP, where different ground pixels are divided over multiple threads. Figure 4 shows the processing per ground pixel (i.e. for a single thread) in more detail.

Figure 3 - Schematic overview of the RemoTeC retrieval procedure including multi-threading.

The static input that is required is a lookup table with the relevant absorption cross sections (read into memory at beginning of processing), a lookup table with aerosol optical properties, and a file indicating the retrieval settings (i.e. fit parameters, spectral range, etc.). Further, the auxiliary retrieval input files that are produced by the pre-processor are needed for each GOSAT ground pixel to be processed, together with the GOSAT-FTS level 1 data. The retrieval per pixel is then run



(iterative scheme with forward model and inversion module) and after convergence an intermediate (ASCII) output file is created that is used in the a posteriori filtering and quality check (see Figure 4) and the processing of the next ground pixel starts.

Figure 4 - Overview of RemoTeC processing per ground pixel.

1.2.1.2 Validation and Bias Correction

The output is validated and bias corrected using the TCCON GGG2014 data before being transformed into the C3S specific NetCDF format. The C3S specific NetCDF data are then once again validated with TCCON as an additional check that no errors during the final format change occurred.

1.2.1.3 Description of System Hardware and Software

Hardware:

The SRON GOSAT Linux cluster used for processing the RemoTeC algorithm consists of one HP Proliant DL360-G7 node (X5650 - 28 times dual core - 2.67 GHz - 128 GB RAM), which means 48 processes can run in parallel (The nodes use a 10Gbit network for communication and to access nearly 100 TB of storage (SAS disks in RAID-5 configuration).



Software:

The SRFP/RemoteC core algorithm is implemented in a FORTRAN90 computer program. The pre- and post-processing steps and the cloud filtering are performed using python scripts. As a compiler the Intel FORTRAN compiler (ifort) is being used for the FORTRAN code.

1.2.1.4 Input Data

Dynamic input data:

ECMWF data: The ECMWF data are used to provide information on the vertical profiles on pressure, temperature, and humidity. The ERA-interim re-analysis data are downloaded from http://data-portal.ecmwf.int/data/d/interim_full_daily/. The data archive at the ECMWF server is updated every month and the data are available with a delay up to 3 months w.r.t. near-real-time. Once the archive at the ECMWF server is updated, the new data are downloaded (manually) to the SRON archive. The total data volume is about 120 GB per year.

GOSAT data: The GOSAT-FTS data containing the NIR and SWIR spectra are downloaded from the ESA ftp server (eoa-dp.eo.esa.int). At SRON a script is running that automatically checks for new data and downloads the data. The volume of GOSAT-FTS data is about 2 TB per year.

Prior CO2 (CarbonTracker2013 and LMDZ inversions) and CH4 (TM4) model data for use as input for the retrievals. These data are not automatically updated, which means these have to be manually updated when needed (and if possible). Due to the strict processing deadlines for C3S (and other) projects, not the most up-to-date data can be used in most cases (as for TM4 for instance).

Static input data:

SRTM topographic information. The SRTM digital elevation map at 90 meter horizontal resolution is downloaded from http://www.cgiar-csi.org/. In principle this is a one-time download as the database is not updated. Total size of the SRTM database is ~18 GB.

Absorption cross sections: For the retrieval lookup-tables are used with pre-calculated absorption cross sections of the species of interest (O2, CO2, CH4, H2O) as a function of wavenumber, temperature, and pressure. One lookup-table per species and per spectral band is being used. At the start of processing at a given CPU (Fig. 3) the cross-section lookup table is read into memory. The total size of the cross-section lookup table is 5.5 GB.

Aerosol optical properties: A lookup table is being used with pre-calculated aerosol optical properties (Mie and t-Matrix theory) as a function of size parameter and refractive index. These tables, total size being 500 Mb, are stored into RAM

Retrieval settings: A file is read in with retrieval settings such as fit parameters, spectral range, etc. The file has negligible size.



1.2.1.5 Output Data

The output data are stored in one NetCDF file per day. The file size varies between 3 and 5 Mb. Both retrieved XCH4 and XCO2 are given in the same output file. A separate output file is being generated for the XCH4 proxy product. These NetCDF files are initially in a RemoTeC specific format. After validation and bias correction determination, the data are transformed into the C3S specific format, which has daily NetCDF files per product type (CH4_GOS_SRPR,CH4_GOS_SRFP,CO2_GOS_SRFP).

1.2.1.6 Processing Demands

For the processing system described here (i.e. with 28 CPUs) it takes about 12 days to process one year of GOSAT data, i.e. a factor 30 faster than real-time. Here both the non-scattering retrievals (to create the proxy product and cloud filtering) and full physics retrievals take about 6 days per year of data.



1.2.2 UoL sub-system


Figure 5 shows a schematic of the University of Leicester Full Physics (UoL-OCFP) and Proxy (OCPR) processing subsystem. Firstly, the relevant input datasets (GOSAT L1B, ECMWF, and MACC/CAMS) are downloaded from the external sources and stored on the University of Leicester server. This data along with other ancillary data such as the DEM surface topography dataset are input into the preprocessing module.

The preprocessing module extracts the spectra and sounding information (time, latitude, longitude, etc.) from the GOSAT L1B data files on a per sounding basis before writing this information to the input file used by the retrieval algorithm. At the same time, the ECMWF and model data are interpolated to construct the a priori atmosphere information at the sounding time/location, writing them to the relevant input file.

Soundings from the L1B dataset are screened to remove soundings affected by issues such as low SNR or high solar zenith angle.

Next, an initial apparent surface pressure retrieval is run for the O2-A band which acts as a first-order cloud screening. Any retrievals within ±30 hPa of the ECMWF a priori surface pressure are considered to be sufficiently cloud-free and progress along the processing chain. Retrievals that fall outside of this threshold are considered to be cloudy and are not processed further. A fast retrieval of Solar Induced Fluorescence (SIF) is also performed to provide an a priori estimate of SIF for the OCFP retrieval. Finally, another fast retrieval for the ratio of the apparent CO2 columns from the weak and strong CO2 band (strong/weak CO2 ratio) is performed to provide input for the OCFP a posteriori quality filtering.

Input data for every GOSAT measurement being retrieved for a given month is produced and collated into a single HDF5 file. This file is transferred to the STFC JASMIN supercomputing cluster, where the OCFP retrievals are performed. The OCPR retrievals are conducted on University of Leicester system (ALICE).



Figure 5 - Schematic overview of the University of Leicester Full-Physics (OCFP) and Proxy (OCPR) algorithm processing sub-system.

OCFP retrievals of XCO2 and XCH4 and OCPR retrievals are then performed on soundings that pass the cloud screening. In the case of OCPR, the surface pressure derived from the O2-A band cloud screening retrieval is used as the a priori surface pressure. These retrievals are distributed as batches of retrievals across multiple CPUs, with the retrievals themselves running as serial processes. This stage is outlined in Figure 6 for the OCFP retrieval.



Figure 6 - Schematic overview of the University of Leicester Full-Physics (UoL-OCFP) algorithm at each CPU.


Hardware:

The preprocessing stage outlined in Figure 6 is performed on the University of Leicester High Performance Computing Cluster (ALICE - http://www2.le.ac.uk/offices/ithelp/services/hpc/alice). Although the processing is a serial process this allows several jobs to be run at once (typically each month is preprocessed separately).

Once preprocessed, the input data is transferred to a high performance supercomputing cluster to perform the retrieval and post-processing. The OCFP retrievals is performed on the STFC JASMIN supercomputing cluster (http://www.jasmin.ac.uk), which has over 6000 CPUs and over 30 TB of guaranteed storage.

The OCPR retrievals and OCFP post-processing are carried out on ALICE, where each standard compute node has a pair of 14-core 2.6GHz Intel Xeon Skylake CPUs and 128GB of RAM. In total there are 4760 CPU cores available for running jobs for University-wide use, with these GOSAT retrievals typically having somewhere between 100 and 500 CPUs available.



Software:

The University of Leicester Full-Physics (UOL-FP) retrieval algorithm is based on the original OCO Full-Physics retrieval algorithm and is implemented in FORTRAN. Primarily the code is FORTRAN 90/95 but some modules are FORTRAN77. The code is compiled with the Intel 64-bit FORTRAN compiler (ifort).

The processing module and post-filtering utilizes the IDL and Python programming languages.

1.2.2.3 Input Data

Dynamic Input Data:

ERA Interim High Resolution ECMWF data is obtained from http://apps.ecmwf.int/datasets/data/interim-full-daily/levtype=sfc/. The data archive at the ECMWF server is available with a delay of up to 3 months. The data is downloaded in NetCDF format and only the surface pressure, temperature, geopotential, specific humidity and u/v wind vectors are stored. These files are typically 1.5GB/month of data for all vertically resolved data, with the geopotential and surface pressure for the entire time series (2009-2017) being stored in single files of approximately 1GB each. In total the data volume is approximately 75GB/year.

Aerosol mass mixing ratio vertical profiles are sourced from a climatology based on the CAMS near-real time (http://apps.ecmwf.int/datasets/data/cams-nrealtime/levtype=ml/) model ensemble dataset. The climatology is based on the data modelled between 2014-2017. This data is downloaded in NetCDF format, which contains the volume mixing ratio and total AOD at 550 nm. These files are typically 600 MB/month of data for each aerosol species. In total the data volume is approximately 87 GB/year.

The CO2 volume mixing ratio vertical profiles are taken from the LMDZ CAMS v18r1 CO2 model (http://apps.ecmwf.int/datasets/data/macc-ghg-inversions/), while the CH4 volume mixing ratio vertical profiles are taken from the MACC-II Reanalysis (S1-NOAA) dataset (http://apps.ecmwf.int/datasets/data/macc-ghg-inversions/) merged with a TOMCAT full chemistry run model. Both datasets are downloaded as monthly NetCDF files, and are typically 360 MB/month. In total the data volume is approximately 8.6 GB/year.

The GOSAT L1B data is obtained from the GOSAT Data Archive Service (GDAS). The total data volume of the uncompressed data is approximately 2TB/year.

Static Input Data:

Shuttle Radar Topography Mission (SRTM) Digital Elevation model. This data is static and is ~3.5GB in size.

Aerosol microphysical properties from the CAMS model ensemble dataset. In total, these are ~10 MB in size.

Absorption cross-section: Pre-calculated absorption cross-sections for each species of interest from the ABSCO (JPL and TCCON) dataset are used by the retrieval algorithm. The total size of all cross-section files is approximately 3GB.



1.2.2.4 Output Data

The retrieval output data are initially stored in ASCII text files before being collated into a single HDF5 file. This file is then used to produce the post-retrieval filtering and bias correction, before being used to produce daily netCDF files containing the final dataset (size about 0.5Mb/day).


The 3-band CO2 and CH4 OCFP retrievals take typically around 10 minutes per sounding depending on the number of iterations required. A typical month will contain ~ 30,000 cloud free soundings over land, so the retrievals take approximately 5,000 CPU hours per month to complete. The OCPR CH4 retrievals take typically around 2 minutes per sounding, and so a typical month of data would take approximately 1,000 CPU hours to process.



1.2.3 CNRS-LMD sub-system


Figure 7 shows a schematic of CNRS-LMD processing scheme for generating Level2 products CH4_IASA_NLIS, CH4_IASB_NLIS, CO2_IASA_NLIS and CO2_IASB_NLIS. First, IASI Level 1c and AMSU Level 1b spectra are downloaded every day from the near-real-time EUMETCast system, and stored on local storage facilities. Each IASI/AMSU observations are then co-located and only channels relevant for the Level2 retrieval are stored to be used in the CNRS-LMD retrieval schemes. Retrievals are then performed at the FOV level and stored sequentially day-by-day in the final output daily NetCDF file.

Figure 7: Schematic overview of CNRS-LMD algorithm processing sub-system.



Figure 8 illustrates the retrieval performed for one given FOV. First, based on geolocation and observation data provided in the Level 1 files, the “type” of the observation is characterized by night/day/sea/land/air-mass type/angle of observation. For the corresponding type: (i) a cloud and aerosol detection is applied to keep “clear-sky” situations only; (ii) stored systematic radiative biases existing between observations and radiative simulations are added to the input IASI/AMSU channels; (iii) using the stored neural network parameters, the retrieval is performed; (iv) the retrieval output is written.

Figure 8: Schematic overview of CNRS-LMD algorithm at FOV resolution.




Hardware:

The CNRS-LMD IASI processing is performed on the ESPRI cluster (based at Ecole Polytechnique) where main standard compute nodes have a pair of quad-pro AMD Opteron 6378 16-core 2,4 GHz, 270 Go (Linux based cluster). In total, there are 1050 CPU cores available for running jobs for IPSL-wide use, with these IASI retrievals typically use around 16 CPUs. The nodes use a 100Gbit network for communication and to access nearly 150 TB of storage (SAS disks in RAID-5 configuration).

Software:

The CNRS-LMD non-linear inference scheme used to generate mid-tropospheric averaged mixing ratio of GHG from infrared sounders is implemented in FORTRAN 90. The code is compiled with the Intel 64-bit FORTRAN compiler (ifort).

1.2.3.3 Input Data

Dynamic input data:

IASI Level 1c and AMSU Level 1b data are obtained from EUMETSAT through EUMETCast in Near-Real-Time. They are stored on the AERIS/Ether national center located at Sorbonne University-Paris, from which the full archive for all Metop satellites are available to French national labs. For one year, and for one Metop satellite, the data volume is approximately 480 Gb for IASI and 10 Gb for AMSU.

Static input data:

Neural network parameters: pre-computed parameters for the various neural networks (neural weights, biases and architecture). The total size for all files, for one Metop, is 15 Mb.

Systematic radiative biases: files containing the systematic radiative biases to be applied to IASI/AMSU input data in order to compensate for unknown radiative transfer biases that need to be corrected once and for all. These biases have been computed once and for all using the SPARTE chain (Armante et al., 2016, see C3S ATBD document). The total size for all files, for one Metop, is about 8 Mb.

NASA digital elevation model: the file size is 20 Mb.

1.2.3.4 Output Data

The retrieval output data ate initially stored in ASCII text files, following the format decided during the MACC projects, and then converted into NetCDF format, with one file per day, containing about 10,000 retrievals for an average size of about 11Mb each. 1.2.3.5 Processing Demands

The inference scheme can process a full day of Metop in ~20mn real time (quad-pro AMD Opteron 6378 2.4 GHz, 256 Gb).



1.2.4 IUP-UB sub-system

The SCIAMACHY Level 2 data products (e.g., Buchwitz et al., 2015, 2016; Reuter et al., 2010, 2011; Schneising et al., 2011) are taken “as is” from the GHG-CCI project (http://www.esa-ghg-cci.org/) and the generation of these products is not part of C3S and therefore not described here.

For a description of the IUP-UB system to generated the SCIAMACHY Level 2 products please see also the GHG-CCI project System Specification Document (SSD):

Hasekamp et al., ESA Climate Change Initiative (CCI) System Specification Document (SSD) for the Essential Climate Variable (ECV) Greenhouse Gases (GHG), version 1, 30-Sept. 2014, link: http://www.esa-ghg-cci.org/?q=webfm_send/193, 2014.

The following description is related to the generation of the merged Level 2 (e.g., Reuter et al., 2013) and merged Level 3 (e.g., Buchwitz et al., 2017; Reuter et al., 2016) products.


As shown in Figure 7 and Figure 8, the primary input data for the EMMA L2 and Obs4MIPS data generation are L2 data products of the following algorithms: ACOS (NASA), BESD (IUP), NIES, RemoTeC (SRON), and UoL-FP (UoL) for XCO2 and WFMD (IUP), RemoTeC-FP, RemoTeC-PR, UoL-FP, UoL-PR, and NIES for XCH4.

The initial step of the processing is to bring all L2 input datasets of the contributing algorithms to a common file format and to adjust the individual a priori information to a common a priori (SC4C for CH4 and SECM for CO2 (Reuter et al., 2012)).

The next step is the generation of the EMMA L2 database. This step includes a validation with TCCON data in order to scale the individual reported errors to match the single sounding precision obtained from the validation, a potential L2 data thinning to prevent a single algorithm to outweigh the ensemble, a global offset correction of the individual data products to match on average the common a priori in overlapping grid boxes, and finally the median selection and exporting of the EMMA L2 data product.

Following this, the EMMA L2 product is gridded and filtered for grid boxes with too large standard errors of the mean. The outcome of this final step is the Obs4MIPS L3 data product.



Figure 7: Schematic overview of the EMMA L2 database generation and Obs4MIPS L3 database generation method for XCO2.

Figure 8: Schematic overview of the EMMA L2 database generation and Obs4MIPS L3 database generation method for XCH4.




Hardware:

Linux server with 16 Intel Xeon E5-2687W CPUs running at 3.10GHz.

17TB Raid5 HDD storage

Software:

IDL8.6

1.2.4.3 Input Data

Satellite XCO2 L2 data sets: ACOS (NASA), BESD (IUP), NIES, RemoTeC (SRON), UoL-FP (UoL)

Satellite XCH4 L2 data sets: WFMD (IUP), RemoTeC-FP, RemoTeC-PR, UoL-FP, UoL-PR, NIES

Validation data set: TCCON

1.2.4.4 Output Data

EMMA XCO2 and XCH4 L2 data

Obs4MIPS XCO2 and XCH4 L3 data


Days (only reproduction of existing L2 conversions, EMMA L2 and Obs4MIPS L3 data products; no analyses for new data sets, algorithm modifications, quality analyses, interpretations, etc.)



2. Upgrade cycle implementation procedure

In this section the procedure to implement new data cycles are described, including notice period to users.

The baseline is to extend each year the existing data sets by one additional year. Note that this not only includes the generation of the data products but also the generation (or update) of all related documentation including detailed assessments related to data quality.

3. Procedures for reprocessing CDR’s

The baseline is to extend existing data sets such that high-quality consistent (e.g., no “jumps”) long-term data sets are available for the users, which are regularly extended in time. This is achieved by applying the same version of the retrieval algorithm to the entire data set, which has the same version number as the algorithm used to generate it.

If an improved retrieval algorithm is available (e.g., arising from research and development activities in the framework of ESA’s CCI or for other reasons) and if the needed resources are available for reprocessing, then the entire time series of a certain data product will be regenerated via reprocessing of all available Level 1 data. In this case, the corresponding data product will get a new version number.



4. System maintenance and system failures

System maintenance activities are mostly “business as usual” and it is very unlikely that this leads to any issues such as serious delay or system failure as time critical near-real-time processing is not foreseen.

So far only one aspect has been identified which cannot be classified as “business as usual”: The planned switch of meteorological dynamic input data from ERA-Interim to ERA5:

The XCO2 and XCH4 L2 products are currently generated using ERA-Interim as "dynamic input". In the (near) future ERA-Interim will not be available any more. Instead ERA5 will be available (covering past and future time periods). It is planned to switch to ERA5 as soon as possible. As the impact of meteorological data on the characteristics of the retrieved XCO2 and XCH4 is "typically small" (this will be confirmed and quantified for the switch from ERA-Interim to ERA5), it is assumed that the existing time series can be extended each year by one additional year as currently planned for data processing and delivery, i.e., that the current baseline does not have to be changed. Should the impact of the switch of the meteorological data be larger than expected, than it will be investigated (and discussed with ECMWF) how to deal with this. In the worst case reprocessing of entire (multi-year) time series will be needed, which may have a negative impact on the current planning for data delivery (i.e., potentially a significant delay can be expected). In any case the first step will be to process (e.g., at least one month of) data with ERA-Interim and with ERA5 as input and to investigate what the impact is and if it is possible or not to continue the existing time series using ERA5 instead of ERA-Interim.

The data product generation, quality control and data provision schedule contains sufficient margin to deal with potential issues such as failure of computer hardware.

However, some risk remains, in particular in case of long-term unavailability of key personnel, e.g., due to illness.



5. User support

All questions and comments related to Copernicus data products

(including the ones described in this document)

shall be submitted via this website:

http://copernicus-support.ecmwf.int



References

Buchwitz et al., 2015: Buchwitz, M., Reuter, M., Schneising, O., Boesch, H., Guerlet, S., Dils, B., Aben, I., Armante, R., Bergamaschi, P., Blumenstock, T., Bovensmann, H., Brunner, D., Buchmann, B., Burrows, J.P., Butz, A., Chédin, A., Chevallier, F., Crevoisier, C.D., Deutscher, N.M., Frankenberg, C., Hase, F., Hasekamp, O.P., Heymann, J., Kaminski, T., Laeng, A., Lichtenberg, G., De Mazière, M., Noël, S., Notholt, J., Orphal, J., Popp, C., Parker, R., Scholze, M., Sussmann, R., Stiller, G.P., Warneke, T., Zehner, C., Bril, A., Crisp, D., Griffith, D.W.T., Kuze, A., O'Dell, C., Oshchepkov, S., Sherlock, V., Suto, H., Wennberg, P., Wunch, D., Yokota, T., Yoshida, Y., The Greenhouse Gas Climate Change Initiative (GHG-CCI): comparison and quality assessment of near-surface-sensitive satellite-derived CO2 and CH4 global data sets. Remote Sens. Environ. 162:344–362, http://dx.doi.org/10.1016/j.rse.2013.04.024, 2015.

Buchwitz et al., 2016: Buchwitz, M., Reuter, M., Schneising, O., Hewson, W., Detmers, R. G., Boesch, H., Hasekamp, O. P., Aben, I., Bovensmann, H., Burrows, J. P., Butz, A., Chevallier, F., Dils, B., Frankenberg, C., Heymann, J., Lichtenberg, G., De Mazière, M., Notholt, J., Parker, R., Warneke, T., Zehner, C., Griffith, D. W. T., Deutscher, N. M., Kuze, A., Suto, H., and Wunch, D.:, Global satellite observations of column-averaged carbon dioxide and methane: The GHG-CCI XCO2 and XCH4 CRDP3 data, Remote Sensing of Environment (in press), Special Issue on Essential Climate Variables, DOI: 10.1016/j.rse.2016.12.027, http://dx.doi.org/10.1016/j.rse.2016.12.027, 2016.

Buchwitz et al., 2017: Buchwitz, M.; Reuter, M.; Aben, I.; Boesch, H.; Butz, A.; Detmers, R.G.; Frankenberg, C.; Hasekamp, O.P.; Parker, R.; Schneising, O.; Somkuti, P., ESA Greenhouse Gases Climate Change Initiative (GHG-CCI): Merged SCIAMACHY and GOSAT Level 3 gridded atmospheric column-average methane (XCH4) product in Obs4MIPs format version 2 (CRDP#4), Technical Note, link: http://www.esa-ghg-cci.org/?q=webfm_send/349, pp. 11, 1 February 2017, 2017.

Butz et al., 2011: Butz, A., Guerlet, S., Hasekamp, O., et al., Toward accurate CO2 and CH4 observations from GOSAT, Geophys. Res. Lett., doi:10.1029/2011GL047888, 2011.

Crevoisier et al., 2009: Crevoisier, C., Chédin, A., Matsueda, H., et al., First year of upper tropospheric integrated content of CO2 from IASI hyperspectral infrared observations, Atmos. Chem. Phys., 9, 4797-4810, 2009.

Crevoisier et al., 2013: Crevoisier, C., Nobileau, D., Armante, R., et al., The 2007–2011 evolution of tropical methane in the mid-troposphere as seen from space by MetOp-A/IASI, Atmos. Chem. Phys., 13, 4279-4289, 2013.

Dee et al., 2011: Dee, D. P., et al.: The ERA-Interim reanalysis: Configuration and performance of the data assimilation system, Q. J. R. Meteorol. Soc., 137(656), 553–597, doi:10.1002/qj.828, 2011.

Detmers et al., 2015: Detmers, R. G., O. Hasekamp, I. Aben, S. Houweling, T. T. van Leeuwen, A. Butz, J. Landgraf, P. Koehler, L. Guanter, and B. Poulter, Anomalous carbon uptake in Australia as seen by GOSAT, Geophys. Res. Lett., 42, doi:10.1002/2015GL065161, 2015.

Hollmann et al., 2013: Hollmann, R., C.J. Merchant, R. Saunders, C. Downy, M. Buchwitz, A. Cazenave, E. Chuvieco, P. Defourny, G. de Leeuw, R. Forsberg, T. Holzer-Popp, F. Paul, S. Sandven, S.



Sathyendranath, M. van Roozendael, and W. Wagner, The ESA Climate Change Initiative: satellite data records for essential climate variables, Bulletin of the American Meteorological Society (BAMS), 0.1175/BAMS-D-11-00254.1, pp. 12., 2013.

Kuze et al., 2009: Kuze, A., H. Suto, M. Nakajima, and T. Hamazaki, Thermal and near infrared sensor for carbon observation Fourier-transform spectrometer on the Greenhouse gases Observing Satellite for greenhouse gases monitoring, Appl. Opt., 48, 6716–6733, 2009.

Parker et al., 2011: Parker, R., Boesch, H., Cogan, A., et al., Methane Observations from the Greenhouse gases Observing SATellite: Comparison to ground-based TCCON data and Model Calculations, Geophys. Res. Lett., doi:10.1029/2011GL047871, 2011.

Peters et al., 2007: Peters, W., A. R. Jacobson, C. Sweeney, A. E. Andrews, T. J. Conway, K. Masarie, J. B. Miller, L. M. P. Bruhwiler, G. Pétron, A. I. Hirsch, D. E. J. Worthy, G. R. van derWerf, J. T. Randerson, P. O. Wennberg, M. C. Krol, and P. P. Tans (2007), An atmospheric perspective on North American carbon dioxide exchange: CarbonTracker, Proceedings of the National Academy of Sciences (PNAS) of the United States of America, 27 November 2007, 104(48), 18 925–18 930, 2007.

Reuter et al., 2010: Reuter, M., Buchwitz, M., Schneising, O., Heymann, J., Bovensmann, H., and Burrows, J. P., A method for improved SCIAMACHY CO2 retrieval in the presence of optically thin clouds, Atmos. Meas. Tech., 3, 209-232, 2010.

Reuter et al., 2013: Reuter, M. H. Bösch, H. Bovensmann, A. Bril, M. Buchwitz, A. Butz, J. P. Burrows, C. W. O’Dell, S. Guerlet, O. Hasekamp, J. Heymann, N. Kikuchi, S. Oshchepkov, R. Parker, S. Pfeifer, O. Schneising, T. Yokota, and Y. Yoshida, A joint effort to deliver satellite retrieved atmospheric CO2 concentrations for surface flux inversions: The ensemble median algorithm EMMA, Atmos. Chem. Phys., 13, 1771-1780, 2013.

Reuter et al., 2016: Reuter, M.; Buchwitz, M.; Aben, I.; Boesch, H.; Butz, A.; Detmers, R.G.; Hasekamp, O.P.; Heymann, J.; Parker, R.; Schneising, O.; Somkuti, P., ESA Greenhouse Gases Climate Change Initiative (GHG_cci): Merged SCIAMACHY and GOSAT Level 3 gridded atmospheric column-average carbon dioxide (XCO2) product in Obs4MIPs format. Centre for Environmental Data Analysis, 10 October 2016, doi:10.5285/3FAE8371-0CBB-4B21-9EA6-7A1FC293C4A2, link: http://www.esa-ghg-cci.org/?q=webfm_send/330, pp. 11, 2016.

Schepers et al., 2012: Schepers, D., Guerlet, S., Butz, A., Landgraf, J., Frankenberg, C., Hasekamp, O., Blavier, J.-F., Deutscher, N. M., Griffith, D. W. T., Hase, F., Kyro, E., Morino, I., Sherlock, V., Sussmann, R., Aben, I. (2012), Methane retrievals from Greenhouse Gases Observing Satellite (GOSAT) shortwave infrared measurements: Performance comparison of proxy and physics retrieval algorithms, J. Geophys. Res., 117, D10307, doi:10.1029/2012JD017549, 2012.

Schneising et al., 2011: Schneising, O., Buchwitz, M., Reuter, M., et al., Long-term analysis of carbon dioxide and methane column-averaged mole fractions retrieved from SCIAMACHY, Atmos. Chem. Phys., 11, 2881-2892, 2011.

TRD GHG, 2017: Buchwitz, M., Aben, I., Anand, J., Armante, R., Boesch, H., Crevoisier, C., Detmers, R. G., Hasekamp, O. P., Reuter, M., Schneising-Weigel, O., Target Requirement Document, Copernicus Climate Change Service (C3S) project on satellite-derived Essential Climate Variable (ECV) Greenhouse Gases (CO2 and CH4) data products (project C3S_312a_Lot6), Version 1.2, 21-August-2017, pp. 53, 2017.


climate.copernicus.eu copernicus.eu ecmwf.int

ECMWF - Shinfield Park, Reading RG2 9AX, UK Contact: [email protected]

system quality assurance document atmospheric ... -...

Documents