astronomy data in the dataverse by august muench

8
Astronomy Data in the Dataverse In Scope : taking advantage of standardized file type to enhance discoverability and curation of Astronomy data; Out of scope: a. Outreach to Astronomy Community; b. Connections to Literature and Article search.

Upload: datascienceiqss

Post on 15-Aug-2015

140 views

Category:

Education


0 download

TRANSCRIPT

Page 1: Astronomy Data in the Dataverse by August Muench

Astronomy Data in the Dataverse● In Scope: taking advantage of standardized file type to

enhance discoverability and curation of Astronomy data;● Out of scope:

a. Outreach to Astronomy Community;b. Connections to Literature and Article search.

Page 2: Astronomy Data in the Dataverse by August Muench

array dataheaderkey : valueFITS File array dataheader

key : valueFITS File array dataheaderkey : value

FITS Files● Flexible Image Transport System

a. 1981, developedb. 2008, Version 3

● Our catch all data containera. Tables, Sparse Matrices, Images, Cubes, Photon listsb. once FITS, always FITS(*)

Page 3: Astronomy Data in the Dataverse by August Muench

array dataheaderkey : valueFITS File array dataheader

key : valueFITS File array dataheaderkey : value

DataverseDatasetCitation Metadata(manual authoring)

Astronomy & Astrophysics Metadata(automated extraction + curation)

field0 = [unique(key0:value, key1:value, )]field1 = [unique(key2:value, null, … )]field2 = array.type(key3:value, key4:value,...)field3 = array.columns...fieldN = there is no value validation

Page 4: Astronomy Data in the Dataverse by August Muench

array dataheaderkey : valueFITS File array dataheader

key : valueFITS File array dataheaderkey : value

Suggested directions:

1. Retain File Level metadata (no aggregation);2. Validate File Level metadata by completeness & content;3. Expose full unparsed header via API;4. Provide transmutable data objects (CSV/TSV tables); 5. Compressed image previews.

File Metadata

key0 : valuekey1 : value…keyN :

validate completeness

Page 5: Astronomy Data in the Dataverse by August Muench

array dataheaderkey : valueFITS File array dataheader

key : valueFITS File array dataheaderkey : value

Suggested directions:

1. Retain File Level metadata (no aggregation);2. Validate File Level metadata by completeness & content;3. Expose full unparsed header via API;4. Provide transmutable data objects (CSV/TSV tables); 5. Compressed image previews and other data widgets.

File Metadata

key0 : valuekey1 : value…keyN :

validate completeness

Page 6: Astronomy Data in the Dataverse by August Muench

array dataheaderkey : valueFITS File array dataheader

key : valueFITS File array dataheaderkey : value

Suggested directions:

1. Retain File Level metadata (no aggregation);2. Validate File Level metadata by completeness & content;3. Expose full unparsed header via API;4. Provide transmutable data objects (CSV/TSV tables); 5. Compressed image previews and other data widgets.

/api/access/datafile/$id/metadata/header

Page 7: Astronomy Data in the Dataverse by August Muench

array dataheaderkey : valueFITS File array dataheader

key : valueFITS File array dataheaderkey : value

Suggested directions:

1. Retain File Level metadata (no aggregation);2. Validate File Level metadata by completeness & content;3. Expose full unparsed header via API;4. Provide transmutable data objects (CSV/TSV tables); 5. Compressed image previews and other data widgets.

array.tsvarray.csvarray.json

Page 8: Astronomy Data in the Dataverse by August Muench

array dataheaderkey : valueFITS File array dataheader

key : valueFITS File array dataheaderkey : value

Suggested directions:

1. Retain File Level metadata (no aggregation);2. Validate File Level metadata by completeness & content;3. Expose full unparsed header via API;4. Provide transmutable data objects (CSV/TSV tables); 5. Compressed image previews and other data widgets.