A Brief History A Brief History of the 50-Year-Old Space of the 50-Year-Old Space

Physics Data EnvironmentPhysics Data Environment

Joseph H. King Joseph H. King

Perot Systems at NASA/Goddard Space Flight Perot Systems at NASA/Goddard Space Flight Center, Greenbelt, MDCenter, Greenbelt, MD

A Framework for the TalkA Framework for the TalkHow do selected, publicly accessible, space physics systems and interfaces, How do selected, publicly accessible, space physics systems and interfaces,

created at various times, satisfy various functional and search created at various times, satisfy various functional and search requirements?requirements?

Some repository-created systems provide value-added access to data Some repository-created systems provide value-added access to data held at those repositoriesheld at those repositories

Other systems are specifically oriented towards a distributed data Other systems are specifically oriented towards a distributed data environment environment

Focus is not on underlying technologies, although their evolution has Focus is not on underlying technologies, although their evolution has enabled the great progress in satisfying functional requirements.enabled the great progress in satisfying functional requirements.

We suppress programmatic history (e.g., NASA’s Rice U. and River Bend We suppress programmatic history (e.g., NASA’s Rice U. and River Bend meetings).meetings).

We underemphasize format and documentation issues.We underemphasize format and documentation issues.

We do not address data preservation/stewardship issues nor creation of We do not address data preservation/stewardship issues nor creation of value-added data products like “OMNI”.value-added data products like “OMNI”.

Functional RequirementsFunctional RequirementsData Finding

Data product levelData granule level

Data RetrievalFile levelSubset by parameter

Data DisplayListsPlots

Data Processing to higher levelsData Analysis tools

Coordinate transformationsCross correlationsEtc.

Data Usability Supporting material, including error bars or equivalent, enables long-term correct and independent usability

Search RequirementsSearch RequirementsFree text

Physical ObservablesTime span

Time resolutionSpectral range

Spectral resolutionRegion sensed

In situRemote sensing

Observatory and InstrumentPerson Repository

Rare event occurrenceParameter inside or outside specified rangeConcurrence of data in multiple favorable locations

Some repository-specific systems and interfaces Some repository-specific systems and interfaces

(many dates are approximate)(many dates are approximate) Pre-network1978 CDAW at NSSDC, 10’s of data sources

Pre-Web1981 SDDAS @ SWRI for Cluster, DE, Tether, Astrid…1983 OMNI captive account at NSSDCUCLA magnetic field data from Polar, ISEE 1,2 3, IMP8…1992 SDAC @ GSFC for SMM, Yohkoh, SOHO,

Web era1994 OMNIWeb @ NSSDC1994 COHOWeb @ NSSDC for Voyager,Ulysses, P10&11...1994 CDAWeb @ SPDF for ISTP s/c and many moreSPIDR @ NGDC for GOES, etc. DARTS @ JAXA, Japan, for Geotail, Reimei, AkebonoCDPP (Toulouse), for Cluster, Wind, Ulysses, ARCAD3…ACE Science Center (Caltech) for ACE and SAMPEX

Etc.

Coordinated Data Analysis WorkshopCoordinated Data Analysis Workshop Developed at NSSDC in 1977-8 to support International Magneto-

spheric Study analysis

Major effort to find good events (spacecraft well located, interesting geophysical events, good data coverage)

Event selection supported by Satellite Situation Center with satellite-satellite and satellite-ground orbits and conjunctions

Participants submitted space and ground data in advance, then came to NSSDC for workshop

Eight CDAW’s hosted, 1978-1990. Clusters of JGR papers from CDAW’s 2, 6, 8. (10’s of data sets per CDAW, e.g., 65 data sets used for CDAW 6)

Last 2 CDAW’s hosted on VAX’s, based on CDF and network-accessible

Paper data catalogs used by participants, plots and lists made on the fly at CDAW. New parameters creatable on the fly.

The term “CDAW” was taken over by the solar community in about 1999

Southwest (SWRI) Data Display and Southwest (SWRI) Data Display and Analysis System (SDDAS)Analysis System (SDDAS)

First created at SWRI in 1981 to support Dynamics Explorer data analysis.

Using calibration and related data and information, worked with telemetry-level data and produced plots of geophysical parameters for analysis.

Required Instrument Data File Set (IDFS), involving a header file, a data file, a “virtual instrument” data file (VIDF), and an optional “plot information and description file” (PIDF).

Can output data files in several formats (CDF, netCDF, ASCII).

Now operates in client-server mode, with IDFS-packaged data at sites in addition to SWRI (e.g., MSSL, IRF) – significant usage in Cluster community.

Spacecraft supported: TSS, UARS, AMPTE, ARIA, DE, DMSP, ISEE, Spacerad, Interball, IMP, Astrid, Cluster, MARS, Prognoz, Viking, Freja, ISIS, NOAA-12, etc.

““NSSDC Account” for OMNI Data AccessNSSDC Account” for OMNI Data Access

Captive account created by NSSDC in ~1983 to provide access to OMNI data and then some other data.

Allowed users to select from list of several tens of physical parameters per OMNI record and to receive output listing or file of just the selected parameters for the time span specified.

Retired when OMNIWeb became available in ~1994.

_________

OMNI: A multi-spacecraft compilation of hourly resolution solar wind magnetic field and plasma data, energetic particle data and geomagnetic and solar activity indices spanning 1963 to near current.

HRO (High Resolution OMNI: 1-min and 5-min) spanning 1995 to near current recently made available; data are as shifted to Earth’s bow shock nose.

COHOWebCOHOWebSystem created at NSSDC to provide access to coordinate-system-

uniformized and content-uniformized hourly solar wind magnetic field and plasma data from several deep-space spacecraft.

Provides subset, plot and list capabilities.

Provides additional analysis capabilities:

Scatter plots and linear regression fits

Cross correlation coefficients

Distribution functions, means and medians

Filtering for any of these by any physical parameters

Spacecraft supported:

Voyager 1 and 2

Pioneer 10, 11 and PVO

Ulysses

Helios 1 and 2

Main Main COHOWeb COHOWeb Web PageWeb Page

COHOWEB scatter plot COHOWEB scatter plot interfaceinterface

Solar Wind flow speed Solar Wind flow speed vs. heliolatitude for first vs. heliolatitude for first

half orbit leg,half orbit leg,1992.0-1995.21992.0-1995.2

Solar Wind flow speed Solar Wind flow speed vs. heliolatitude for vs. heliolatitude for

second half orbit leg,second half orbit leg,1995.2-1998.01995.2-1998.0

Solar Wind flow speed Solar Wind flow speed vs. heliolatitude for third vs. heliolatitude for third

half orbit leg,half orbit leg,1998.0-2001.41998.0-2001.4

Solar Wind flow speed vs. Solar Wind flow speed vs. heliolatitude for forth half heliolatitude for forth half

orbit leg,orbit leg,2001.4-2004.22001.4-2004.2

Solar Wind flow speed vs. Solar Wind flow speed vs. heliolatitude for fifth half heliolatitude for fifth half

orbit leg,orbit leg,2004.4-2007.32004.4-2007.3

Systems for supporting distributed dataSystems for supporting distributed data

NSSDC’s NASA Master Directory

SPDS’s Subdiscipline-specific web pages

Virtual Solar Observatory

Virtual Space Physics Observatory

Virtual Heliospheric Observatory

Virtual Solar Terrestrial Observatory

Others in development

NASA Master DirectoryNASA Master Directory

Started by NSSDC in ~1985 to enable data product finding.

Discipline horizon spanned space and Earth science data.

Became space science specific after Global Change MD separated in early 1990’s.

Query functionality:Free textParameters measuredTime spanRegion sensed, locationObservatory and instrumentPerson

Network connections via SPAN/DECnet to underlying systems holding data or further information (about 20 such systems in 1991).

Abandoned in mid-1990’s due to limited usage and difficulties maintaining information current.

SPDS Information SystemsSPDS Information Systems

A series of information systems about distributed data created in ~1996-7; now largely abandoned.

Intended to assist in the data product finding effort.

MOLD – Magnetospheric OnLine Data directory, Jeff Reeves, LANL. Hosted by GSFC/SPDF in later years.

Cosmic & Heliospheric – NSSDC

ITM – SWRI(?)

MOLD and C&H reached a reasonable population level and had modest-level usage.

Virtual Space Physics ObservatoryVirtual Space Physics Observatory

Started at GSFC (A. Roberts) in about 2003 to aid in data finding, retrieval and display.

Discipline horizon is space and solar physics.

~ 400 space- and ground-based product descriptions as of today

Developed in parallel with SPASE, therefore highly SPASE compatible.

Query functionality: Free text Parameters measured

Time span and resolution, Spectral range Region or remote object sensed.

Observatory and Instrument Person, Repository

Provides parameter-subsetted data retrieval and display functionalities for some data by linking to underlying interface (e.g., CDAWeb) via Web services or by emulating underlying interface (e.g., SPDF’s FTPBrowser)

VirtualVirtual Heliospheric ObservatoryHeliospheric Observatory

In development at GSFC (Szabo et al) with a In development at GSFC (Szabo et al) with a distributed teamdistributed team..

Provides current access to ACE, Wind, Genesis, Provides current access to ACE, Wind, Genesis, Helios, IMP 8, SOHO and MGS field and plasma Helios, IMP 8, SOHO and MGS field and plasma data.data.

Search by time, location (ranges in X,Y,Z gse), Search by time, location (ranges in X,Y,Z gse), physical observable (field or plasma), spacecraft, physical observable (field or plasma), spacecraft, instrumentinstrument

Clickable file-level URL’s returned.Clickable file-level URL’s returned.

Virtual Solar Terrestrial ObservatoryVirtual Solar Terrestrial Observatory

In development at NCAR (P. Fox)In development at NCAR (P. Fox)

Provides current access to data at CEDAR and MLSOProvides current access to data at CEDAR and MLSO (Coupling, Energetics and Dynamics of Atmospheric Regions)(Coupling, Energetics and Dynamics of Atmospheric Regions) (Mauna Loa Solar Observatory)(Mauna Loa Solar Observatory)

Significant emphasis on ontology (developed independently of SPASE)Significant emphasis on ontology (developed independently of SPASE)

Search by time, instrument or physical parameter, with “filtering” by Search by time, instrument or physical parameter, with “filtering” by domain (CEDAR vs. MLSO) or by parameter type.domain (CEDAR vs. MLSO) or by parameter type.

Deals with registration requirement for downloads of CEDAR digital Deals with registration requirement for downloads of CEDAR digital data.data.

Get URL’s of files satisfying requirements, with option to choose format Get URL’s of files satisfying requirements, with option to choose format (FITS, ASCII, JPEG, etc.)(FITS, ASCII, JPEG, etc.)

Thoughts for current VO activitiesThoughts for current VO activitiesData modeling and ontology work is very valuable for data finding, but to

ensure data understanding and correct and independent usability, more data-descriptive documentation is needed than that required to pass SPASE (or equivalent) validation tests.

Whole data file delivery is desired even when a (desirable) subsetting option is available, especially if the files are immediately usable (I have a bias for ASCII files!).

Data environment should facilitate users’ access to either repositories’ data access paths or to VOs’ access paths. (e.g., VSPO does this for VSO-accessible data)

Data environment should support data access by either people or by computer programs.

In multi-level multi-nodal data environment, new resource-acknowledgement practices will be important to each node’s desire for

continuation.