cosmic data analysis and archive center (cdaac) current

Oct 27‐29, 2009 Boulder, CO Fourth FORMOSAT‐3/COSMIC Data Users Workshop

COSMIC Data Analysis and Archive Center (CDAAC) Current Status and Future Plans

Bill Schreiner B. Kuo, C. Rocken, S. Sokolovskiy, D. Hunt, B. Ho, X. Yue, T.‐K. Wee,

K. Hudnut, M. Sleziak‐Sallee, T. VanHove

UCAR/COSMIC Program Office


Outline

•  COSMIC System Overview •  CDAAC Data Product Status and Highlights

– Neutral Atmosphere

–  Ionosphere •  Future Plans

– Planned CDAAC 3.0 Enhancements

– Development and Processing Schedule

•  Summary


CDAAC

NESDIS

GTS

NCEP

ECMWF

CWB

UKMO

Canada Met.

JMA

BUFR Files WMO standard 1 file / sounding

Getting COSMIC Results to Weather Centers

JCSDA

Meteo France

COSMIC OperaConal Processing

Science & Archive

TACC

AFWA

Input Data -  COSMIC data -  GPS ground data -  GPS Bits -  GFS Forecast -  IGS/IGU ORB/CLK -  Bernese Config files

Research Community

SFTP

Unidata’s LDM

WGET

RTS


CDAAC Processing Flow

LEO data Level 0--level 1

POD (Orbits and clocks)

Excess Phase Full Spectrum Abel Inversion

1-D Var Moisture Correction

Fiducial data

Atmospheric processing

Excess Phase Processing Abel Inversion

Combination with other data

Ionospheric processing

Profiles

Real time Task Scheduling Software

atmPhs atmPrf

ionPrf ionPhs

bfrPrf

podTec scnLv1 tipLv1

wetPrf


COSMIC System Summary

•  COSMIC spacecraU (power and aWtude anomalies) and payloads (low SNRs) funcZoning with some degradaZon

•  COSMIC RTS Download staZons ‐ Nominal –  NOAA: Fairbanks, Tromso, Wallops (> 70 passes/day)

–  NSF/NASA: McMurdo (10 passes/day)

•  ~20% missed dumps to due SpacecraU/Payload/RTS problems •  COSMIC 6‐staZon Bit Grabber (GPS Data Bits) network – Nominal

•  CDAAC (2.0) –  producing 1500–2500 profiles per day –  > 2.0M/2.1M Neutral‐Atm/Ionospheric profiles to date

–  Data Latency •  > 90% observaZons available within 3 hr •  Average BUFR file data latency ~90 min, single dump ~ 75 min

–  CDAAC Reliability: > 99.5% –  Providing data to 1,198 registered users from 54 countries

•  OperaZons funded through April 2011


COSMIC OccultaCon Count

> 2 Million Profiles in Real Time Launch to present


Neutral Atmospheric Highlights


Data Product Summary

•  MulZ‐mission profiles > 2.5M (atmPrf: version 2007.3200) -  COSMIC (4/06‐3/09: ~2M), CHAMP (5/01‐10/08: 393,000), GRACE (11/06‐3/09: 90,000), GPSMET

(5/95‐2/97: ~5,000: ~4,000 AS)

•  COSMIC penetraZon to 1 km of surface -  85% high laZtudes, negligible N bias

-  65% low laZtudes, negaZve N bias wrt ECMWF (~2%) and Radiosondes (~1%)

•  COSMIC Rising/SeWng mean N differences (~0.2‐0.3%) in stratosphere (> 30 km) wrt ECMWF

•  COSMIC Quality Control: ~35% marked bad -  ~25% due to poor L2 signal quality

•  Data Processing Status -  Implemented/evaluated L2C processing

-  Implemented Phase Matching processing, sZll using FSI

-  InvesZgated/improved filtering methods

-  InvesZgated/modified truncaZon of RO signals

-  SZll using dynamic transiZon height between GO/FSI

•  DisseminaZng New Version of Data Products -  Worked with weather centers to update BUFR files

-  Real‐Zme: since Oct 12, 2009

-  Post‐Processing (version 2009.2650): since Apr 2009


•  ECMWF Experiments: Old/New COSMIC data Nov ‐ Dec 08 and Jan 09

•  The COSMIC processing changes have improved the GRAS departures considerably, indicaCng much beIer consistency between the data.

Noise normalised GRAS Departures

Red = with oper. COSMIC

Black = with new COSMIC

Mean (o-b)/ SD (o-b)/

Courtesy: Sean Healy ECMWF

Improvements to CDAAC Inversion SoKware: > 20 km


Improvements to CDAAC Inversion SoKware: < 6km

StaCsCcal comparison of COSMIC and ECMWF (high resoluCon model) fracConal refracCvity differences versus alCtude (km) for 1 July 2007. The leK panel is for COSMIC inversions that use the sliding median filter and the right panel is for inversions that don’t use the sliding median filter. The red lines are the mean differences, the green lines are the standard deviaCons of the differences, and the blue lines are the number of samples.

OLD Inversion Software (With sliding median filter)

NEW Inversion Software (Without sliding median filter)

COSMIC - ECMWF COSMIC - ECMWF


QuanCfy the Reproducibility of RO data: EsCmates of Structural Uncertainty of

RO data

(Ho et al. JGR, 2009)

Shu-peng Ben Ho, UCAR/COSMIC http://www.cosmic.ucar.edu/~spho/

Comparison of CHAMP RO data processed by GFZ, JPL, UCAR, and WegC FracZonal RefracZvity Anomalies

Independent of processing procedures : the trend uncertainty from GPS RO data processed by different centers < 0.02%/5yrs


Ionospheric Highlights


Electron Density Profiles •  Electron Density Profiles, EDPs (level2:ionPrf)

-  Abel Inversion: Potentially significant inversion errors below F2 layer, less so above F2 layer -  Number: COSMIC (4/06-3/09: 2.1M), CHAMP (5/01-10/08: 300,000) -  NOTE: Fixed COSMIC s/w bug, corrected noisy profiles after 2008.001 -  Quality Control still needs improvement

Lei et al., 2006JA012240

110 km

Unit: 1×105/cm3

ED at 100km from COSMIC

Simulation

Yue et al., GRL (submitted)


Absolute TEC Data •  Absolute TEC (level1b:podTec)

-  Number: COSMIC (4/06‐3/09: 2.5M), CHAMP (5/01‐10/08: 200,000)

-  NOTE: Fixed COSMIC s/w bug and reprocessed 2008.001 onwards (700,00 more arcs)

-  Absolute accuracy ~ 1‐3 TECU

-  RelaZve accuracy ~ 0.0024 TECU at 1‐Hz

-  Quality Control is good

Ionospheric response to sudden stratospheric warming event (Jan 09) seen in topside ionosphere - vTEC in the topside increase in the morning -  vTEC decreases in the afternoon -  This results agrees with study of ground-based GPS [Goncharenko et al. 2009,CEDAR meeting]

-  NEW: Topside TEC is good evidence that the SSW affects the ionosphere by changing the tides and then the electric field in ionosphere (only EXB can reach high altitude)

Deviation (TECU) vTEC(SSW month) - vTEC(preSSW month)


ScinCllaCon Data

•  ScinZllaZon S4 (level2:atmPrf) -  Computed from 50‐Hz OccultaZon L1 SNR observaZons (60‐80 km alt)

-  Number: COSMIC (4/06‐3/09: ~2M), CHAMP (5/01‐10/08: 393,000), GRACE (11/06‐3/09: 90,000), GPSMET (5/95‐2/97: ~5,000: ~4,000 AS)

•  COSMIC ScinZllaZon S4 (level1b:scnLv1) -  Downloaded 1‐Hz values from all line of sights (based on 50‐Hz data)

-  Number: > 4.5M LEO‐GPS arcs

Scintillation from level1b:scnLv1 (9-sec averages)

E layer F layer


Tiny Ionospheric Photometer (TIP)

•  COSMIC Calibrated TIP radiances (level1b:tipLv1) -  TIP payload off for FM2/3 -  > 1400 hours of quality controlled data

COSMIC TIP: 2006-2009, ~1400 hours


Future Plans – CDAAC Enhancements

•  CDAAC 3.0 System –  Improve data processing, storage capability, reliability

–  Improve soUware and configuraZon management for mulZ‐mission processing (GRAS)

–  Improve data access (FTP site, ..), update website, DB interface

–  Improve orbit, clock and excess phase processing

•  Neutral Atmospheric Products –  ConZnue to invesZgate structural uncertainty in LT

–  Explore new methods of BA opZmizaZon

–  InvesZgate ionospheric correcZon (large‐scale higher‐order residuals, use of L5)

–  Restructure Inversion Process (for mulZ‐missions, efficiency and tailored products)

–  Add new BAD flag bitmask

–  New 1DVAR procedure

–  Provide climate products

•  Ionospheric Products –  Improve EDPs by using informaZon on horizontal gradients

–  Improve Quality Control: add BAD flag, error esZmates

–  Provide climate products


Future Plans – cont

•  Data Product ValidaZon / CalibraZon -  Develop comparison tools -  RO‐RO -  NWP/Climate Models -  Radiosondes -  Ground‐based GPS – PWV -  AMSU, AIRS, SSMI, Saber

•  Development and Data Processing Schedule -  ConZnue development of CDAAC 3.0

-  Re‐process CHAMP (2009.2650) 1‐2 months -  Re‐process COSMIC (2009.2650) 3‐6 months -  Re‐process GRACE (2009.2650) 1 month -  Re‐process GPSMET (2009.2650) 1 month -  Process TerraSAR‐X (2009.2650) 1 month -  Process SAC‐C (2009.2650) 1‐2 months

-  Process Metop/GRAS ? -  Complete CDAAC 3.0 9 months -  Re‐Process


Summary

•  Some level of degradaZon of COSMIC spacecraU and payloads

•  ~90% COSMIC observaZons (1500‐2000) available within 3 hr (~90 min average latency)

•  CDAAC Reliability: > 99.5% •  COSMIC data having posiZve impact on NWP, ionosphere and

climate studies, instrument validaZon/calibraZon

•  DisseminaZng new version of data products

•  ConZnuing development of CDAAC 3.0 •  Re‐processing with consistent soUware underway


Acknowledgments

•  NSF •  Taiwan’s NSPO •  NASA/JPL, NOAA, USAF, ONR, NRL •  Broad Reach Engineering

cosmic data analysis and archive center (cdaac) current

Documents