planetary science archive (psa) and pds introduction vex / mex data workshop 27 th june to 1 st july...

Planetary Science Archive (PSA) and PDS Introduction

VEX / MEX Data Workshop

27th June to 1st July 2011

ESAC

Dave Heather ([email protected])

PSA PDS Overview |27/06/2011 | Slide 2

Planetary Science Archive

• Available since March 2004:• http://www.rssd.esa.int/PSA

• Active development, PSA 3.0 to come in December

• PSA Data Handling Team @ ESA-ESAC (Spain)• David Heather lead• Archive Scientist, setting up the requirements• Interface with Instrument Teams • Validating Data Sets

• PSA Development Team @ ESA-ESAC (Spain)• Christophe Arviset lead• PSA systems software development


PSA Definition and Purpose

The Planetary Science Archive (PSA) is the initiative, the setup, the process and the implementation to preserve data from ESA’s spacecraft to planetary bodies, as well as supplementary information acquired in laboratories or ground-based observatories.

The main objectives of the PSA are: to support the experimenter teams in the preparation for the spacecraft and ground-based long-term archives

to enable and ensure the long-term preservation of these archives

distribution of scientific useful data to the world wide scientific community

provision of supplementary data services aiming to maximize the usage of planetary mission data and ease the scientific data analysis.


PSA : one archive, several missions

Mars Express

Rosetta

Smart-1

GiottoHuygens

VenusExpress

ALL IN PDS FORMAT


Current Status, Available Datasets

GIOTTO data from comet Halley and Gripp-Skellerup

Comet Halley ground-based observations (Halley-Watch)

Comet Wirtanen ground-based observations

Mars Express instrument and auxiliary data (ongoing)

Venus Express instrument and auxiliary data (ongoing)

Smart-1 instrument and auxiliary data (pending)

Rosetta instrument and auxiliary data (ongoing)

Huygens (complete)

BepiColombo data handling and archive support – in preparation

ESA supported instruments on Chandrayaan 1 – pipeline development and support from PSA


The PDS Standards

• All PSA data are compliant with NASA’s Planetary Data System (PDS) Standards

• This is the first step in ensuring that the data stored in the PSA are as widely usable as possible and will remain so for many years to come

• Widely accepted and understood within the planetary science community

• Standards available online and downloadable as a ‘Standards Reference’ pdf file.

Within the PDS Standards, data are organized into volumes and data sets, collecting together observations of similar type, processing level, and/or from a specific mission phase or observation campaign.


The PDS Standards

All PDS compatible data formats conform to a unified standard format incorporating documentation, calibration, and raw and processed data files.

Meta-data Files At the intermediate level

are files which describe the circumstances and parameters of the data collection (for example: the instrument used, the observatory site, the type of data collected, etc.). These files are called "catalog objects" or just "catalog" files.

Volume/Dataset Description Files These files are included on any

distribution medium intended to be an entire, self-contained archive - for example, a set of DVDs or a ZIP file. The volume description files detail the organization of the data in the archive (in terms of directories, e.g.) and any additional documentation and software which might be included.

The format distinguishes:


The PDS Standards

The lowest level is that of the data themselves.

• Each file will have a PDS label either appended to the beginning of the file or, more often, in a separate but proximate file. "One file, one label" is the general rule.

The LBL file must contain a full description of the data object(s)

• Allows for all PDS products to be read by existing standard software

• Allows for long-term data usage with clear description of data to allow users to read files with their own software


PDS Resources

http://pds.jpl.nasa.gov

More information on the PDS and complete versions of the Standards Reference and Data Dictionary

The Data Dictionary can be found here

Supporting Software Tools can be downloaded here


A Few PDS Tools

http://pds.jpl.nasa.gov

Go to the Tools menu

NASA View

A useful tool for visualising labels and simple small data products

PDS Software Tools

Some general tools reading tables and manipulating PDS files.


NASA View Example

NASAView can open simple PDS compliant data

Click on File -> Open Object and select the product you want to see

You can also click on Label -> Full Label to view the complete PDS label with full information on the data product being viewed


ReadPDS

ReadPDS is an IDL package that is maintained and distributed by PDS's Small Bodies Node (SBN). It allows most PDS compliant products to be read in to the IDL software.

To retrieve the latest fully tested release or previous releases, check out the page:http://pdssbn.astro.umd.edu/nodehtml/software.shtml

Within your IDLDE, select the File->Preferences and check that your path contains the ReadPDS.


ReadPDS 2

To run the ReadPDS you also need to have the astrolib library installed. If not already done, you can retrieve the astrolib library at

http://idlastro.gsfc.nasa.gov/ . Read the aareadme.txt file distributed with the ReadPDS package.

There is no further way to test your installation than to call a procedure within IDL. So, on your IDLDE command input, enter e.g.:

If the routine READPDS is compiled successfully, your installation is ok.


PSA Standard User Interface

• User friendly web access : Java Applet • http://www.rssd.esa.int/PSA

• Powerful queries, organized by panel• General query panels• Mission / instrument specific panels

• Hierarchical data presentation• 1 Dataset -> several data products

• Public data accessible to all, proprietary data accessible only to privileged users

• Images preview (icons, full image)

• Quick download (1 click) at various processing levels, shopping basket retrieval


PSA Mars Map Based Interface

• Interface for Mars Express image data• http://www.rssd.esa.int/PSA• Java Applet• Currently HRSC and OMEGA

• No need to be an expert• Easy for general public

• Area selection by mouse• Display images footprints

• Image download by 1 mouse click• Some are big (be patient! )

• Go back and forth to the standard interface to refine search parameters


PSA UI – The Map Interface

Select Base Map Type (MOLA / VIKING…)

Zoom function

Select Instrument / Detector

Long / Lat information

Search Button

Access to ‘standard’ UI

4. Download directly or view label / docs etc.6. Return to the map

browser, you can finish your search as required.

1. Click and drag to select your area of interest

2. Press ‘Search’ and wait for footprints

3. Click on desired footprint

5. Transfer at any time to the standard interface for advanced searches. Your search parameters will be remembered.


PSA Data Set Browser Interface

Opening ScreenClick on the instrument

that interests youSelect the data set you

wish to look at

Browse through the directories to locate the files you want

Right click the product you want to save etc. to bring up the menu

Left click to view directly (where possible)


Automatic Download Scripts / Mirror Scripts

You can use any automatic download script (mirror, etc) to recursively download the data on the PSA.

Before installing a regular download scheme on your side (mirror, etc), please inform us by sending a short note to [email protected].

Be aware that the files that you see in your www-browser or your ftp-client are not the physical files, but a link into the PSA database system. When downloading data, the PSA server resolves the link on the fly and delivers the data back to you.

When building mirror scripts, you can compare your data items against size or time with the ones on the PSA server.

Be careful when using automatic downloads, as the size of the datasets can be huge (several 100GBytes or larger)! Please contact the PSA Helpdesk beforehand.


Notification Management:Registration I

1. Goto Login Page

1. Login with your user id / password

1. Login and stay here

1. Select ‘Change Notification Setting'


Registration II

Select Add

View Return


Notification via Email

Once you have registered, you will receive e-mails with announcements of the new data in the archive that you are interested in.


Interoperability aspects in Planetary

• IPDA contains representatives from all national agencies active in planetary data collection

• Standards being defined in IPDA

• Closer link required between IPDA and EuroPlanet

• IDIS and IPDA ?

• Closer link required between Planetary and Astronomy

• Registry

• Access protocols

• But very different Data Model and data generation process


Interoperability ESA PSA – NASA PDS

• Prototype in development In the IPDA context

• From Mars Map Browser, Select region of interest

• Contact PSA and PDS using the PDAP (Planetary Data Access Protocol)

• Display NASA PDS and ESA PSA images


VEX I/O via PDS Atmos


Comments and Suggestions

Please send us your remarks: [email protected]

In case of problems, please visit the FAQ page before contacting us

In case you can not find the data you are looking for, please read the Mission Specific pages on the PSA home page

(http://www.rssd.esa.int/psa)

planetary science archive (psa) and pds introduction vex / mex data workshop 27 th june to 1 st july...

Documents

scientific data

type of data

data collection

pds standardsall psa

processed data files

metadata files

auxiliary data ongoingsmart

available datasetsgiotto