space telescope science institute operated for nasa by aura cos pipeline pdr file structure overview...

SPACETELESCOPESCIENCEINSTITUTE

Operated for NASA by AURA

COS Pipeline PDR

File Structure Overview

Charles D. (Tony) Keyes

7 December 2000

COS PIPELINE PDRTony Keyes

December 7, 2000 COS File Structure Overview

Space Telescope Science Institute

2 of 13

COS Data Format 2-d data: FITS files containing imsets consisting of science and 2-d data: FITS files containing imsets consisting of science and

data quality images data quality images (unlike STIS, no error array)(unlike STIS, no error array) 1-d data and TIME-TAG lists: FITS tables containing fluxes, 1-d data and TIME-TAG lists: FITS tables containing fluxes,

wavelengths, errors, quality, positions, and other related wavelengths, errors, quality, positions, and other related quantitiesquantities

May explore compression for reduced dataset volumeMay explore compression for reduced dataset volume Data from each FUV detector segment stored as separate imset Data from each FUV detector segment stored as separate imset

series within a single fileseries within a single file– All associated data (e.g., wavecals, FP-SPLIT components, REPEATOBS All associated data (e.g., wavecals, FP-SPLIT components, REPEATOBS

components, STIMs) use separate files components, STIMs) use separate files (unlike STIS)(unlike STIS)




3 of 13

Data File Naming Conventions Product Suffix Type Contents --------------------------------------------------------------- Uncalibrated _raw image Raw counts image _rtg table Raw TIME-TAG event list _asn table Association file

_trl table Trailer file (input)

_lrc image Local rate check image [TBD]

_lsp text LRC support file (header) [TBD] _jit table _jif image _pha table Pulse-height (FUV ACCUM only) _rs1 image Stim-pulse 1 (FUV ACCUM only) _rs2 image Stim-pulse 2 (FUV ACCUM only)




4 of 13

Data File Naming Conventions (continued) Product Suffix Type Contents ------------------------------------------------------------------------------------------------------------------------------ Calibrated _ctg table Calibrated TIME-TAG event list

_flt image Flat-fielded, corrected counts _fls image Summed flat-fielded corrected-counts _x1d table 1-D extracted spectra _x1s table Summed (combined) 1-D extracted spectra)

_cs1 image Corrected stim-pulse 1 (FUV ACCUM only) _cs2 image Corrected stim-pulse 2 (FUV ACCUM only) _trl table Trailer file (output) _spt image Support file (header) ------------------------------------------------------------------------------------------------------------------------------ Wavecal exposures produce same calibrated files as science, except there Wavecal exposures produce same calibrated files as science, except there

may be no flux calibration applied to the _x1* files (TBD). Dark and flat may be no flux calibration applied to the _x1* files (TBD). Dark and flat exposures produce no _x1* files. Acquisition exposures produce no calibrated exposures produce no _x1* files. Acquisition exposures produce no calibrated products. products.




5 of 13

COS Observational Quantity Definitions x = raw detector x-position (I*2) y = raw detector y-position (I*2) t = time of photon detection (R*4) ph = pulse-height (I*1) xcorr = corrected detector x-position (R*4) ycorr = corrected detector y-position (R*4) EPS = photon list sensitivity-parameter (R*4) obsC = observed counts (R*4) C = corrected counts (R*4) q = data quality (I*1) lambda = wavelength (R*4) F = calibrated flux (F-lambda) (R*4) G = gross extracted counts (R*4) N = net extracted counts (R*4) B = background extracted counts (R*4) S = sensitivity function (R*4)




6 of 13

File Contents: Observational Quantities _rtg: contains x,y,t,ph (no ph for NUV)

– FUV: 9 bytes/event; NUV: 8 bytes/event _ctg: contains xcorr,ycorr,t,ph,EPS,q,lambda (no ph for NUV)

– FUV: 22 bytes/event; NUV: 21 bytes/event _raw: contains obsC

– FUV: 32 Mbyte/file; NUV: 4 Mbyte/file _flt: contains C,q (in each imset)

– TIME-TAG: FUV: 160 Mbyte/file; NUV: 5 Mbyte/file– ACCUM: FUV: 40 Mbyte/file; NUV: 5 Mbyte/file

_fls: same as _flt “_obs”: contains obsC (in each imset) [size same as _raw] _x1d: contains F,lambda,G,N,B,S,q (in each imset)

– FUV: ~1 Mbyte/file; NUV: ~0.08 Mbyte/file _x1s: same as _x1d _rs1: contains obsC (same for _rs2) _cs1: contains C (same for _cs2)




7 of 13

Exposure type rt

g ra

w

ctg

pha

“obs

” fl

t fl

s x1

d x1

s rs

1 rs

2 as

n tr

l sp

t

TIME-TAG No FP FUV NUV

X X

X X

X X

X X

X X

X X

X X

X X

TIME-TAG FP FUV NUV

X X

X X

X X

X X

X X

X X

X X

X X

ACCUM No FP FUV NUV

X X

X X X

X X

X X

X X

X X

X X

X X

X X

ACCUM rptobs FUV NUV

X X

X X X

X X

X X

X X

X X

X X

X X

X X

X X

X X

ACCUM FP FUV NUV

X X

X X X

X X

X X

X X

X X

X X

X X

X X

X X

X X

IMAGE T-TAG X X X X X X

IMAGE ACCUM X X X X X

IMAGE ACCUM rpt X X X X X X

ACQ and PEAKUPs X WAVECAL X X X X X

FLAT and DARK X X X X




8 of 13

Time-Tag Data File Structure

ext 1 - b inary tab le


ext 2 - b inary tab le detector B

Time-tag /FPsplit = no

NUV FUV


PHUext 1 - b inary tab le detector A

PHU


PHU

Time-tag / FPsplit = yes

Lpppssoot_RTG.FITS

PHU

PHU


PHUexposure 4 oo = 04

exposure 3 oo = 03

exposure 2 oo = 02

exposure 1 oo = 01

ext 2 - b inary tab le detector Bext 1 - b inary tab le detector A

PHU


PHU


PHU


PHU




9 of 13

ACCUM / FPsplit = no Data File Structure

ext 4 - detector Bdata quality array

ext 3 - detector Bscience data array

im set 2

ext 2 - data qualityarrayACCUM /

FPsplit = no

NUV FUV

ext 1 - sciencedata array

PHU

Lpppssoot_RAW .FITSLpppssoot_RS1.FITSLpppssoot_RS2.FITS

im set 1

ext 2 - detector Adata quality array

ext 1 - detector Ascience data array

PHU

im set 1




10 of 13

ACCUM / FPsplit = yes or REPEATOBS=NData File Structure

NUV FUV

Lpppssoot_RAW .FITSLpppssoot_RS1.FITSLpppssoot_RS2.FITS

ACCUM /FPsplit = yes ext 2 - data quality array

ext 1 - science data arrayPHU

ext 4 - detector B DQ arrayext 3 - detector B sci array im set 2

ext 2 - detector A DQ arrayext 1 - detector A sci array

PHUim set 1

exposure 4 oo = 04

exposure 3 oo = 03

exposure 2 oo = 02

exposure 1 oo = 01

ext 2 - data quality array








PHUim set 1



PHUim set 1



PHUim set 1




11 of 13

File Naming TIME-TAG “FP-SPLIT=NO" exposure:

Wavecal exposure science exposure (401s) (402s) association file: 4010_asn

wavecal exposure files: 401s_rtg 401s_ctg 401s_flt 401s_x1d

science exposure files: 402s_rtg 402s_ctg 402s_flt

association science product file: 4010_x1d




12 of 13

File Naming TIME-TAG FP-SPLIT=YES exposure:

Wavecal exposure science exposure (401s) (402s) science exposure science exposure (403s) (404s) science exposure (405s) association file: 4010_asn

wavecal exposure files: 401s_rtg401s_ctg

401s_flt 401s_x1d

science exposure 1 files: 402s_rtg402s_ctg

402s_flt 402s_x1d

science exposure 2 files: 403s_rtg403s_ctg403s_flt

403s_x1d science exposure 3 files: 404s_rtg

404s_ctg404s_flt 404s_x1d

science exposure 4 files: 405s_rtg405s_ctg405s_flt 405s_x1d

association science product file: 4010_x1d




13 of 13

COS Typical File Sizesexposure POD

(Mb) rtg raw

(Mb)

ctg (Mb)

flt (Mb)

“counts” (Mb)

OPUS (Mb)

into HDA (Mb)

TIME-TAG 18 45 110 160 160 493 63

ACCUM FUV

16 32 --- 40 40 128 48

ACCUM NUV

2 4 --- 5 5 16 6

WAVECAL

<18 <45 <110 160 160 320-376

<63

for 3000 sec FUV exposure: TIME-TAG produces 18 Mbyte full buffer of data for F-lambda=3e-14; TIME-TAG=ACCUM OPUS file size crossover at 1.e-14: for fainter TIME-TAG OPUS fileset smaller; onboard memory usage equivalent at 2.8 e-14; 6 Mb of TIME-TAG photons produces OPUS volume = ACCUM std volume

space telescope science institute operated for nasa by aura cos pipeline pdr file structure overview...

Documents