VO Data Access Layer

IVOACambridge, UK 12 May 2003

Doug Tody, NRAO

Data access/analysis portal

• Link between client (user) data analysis software and the VO

• Enables distributed multiwavelength data access and analysis

• Key longer-term goals:– Integrate everyday data analysis with VO– Achieve a scalable VO architecture (same software

used in multiple contexts)

DAL Scope: Software

• Service data access protocols– Catalog, image, spectrum, etc.

• Service frameworks– Virtual data generation and manipulation, grid computing

• Client side integration– API, Web Services client, components, etc.

• Test cases and demonstrations

Denotes first year priority

DAL Scope: Types of data

• Source catalog• Image• Spectrum, SED• Time series• Event and visibility data

• Key challenges– DAL depends upon registry, query, UCD, DM, representation, etc.– Keep it simple; lower the bar for service implementors

Goals for this meeting

• Agree on DAL scope, WG scope– this will be our kick-off meeting

• Strawman data service architecture– what services are required? (e.g., extend image model, add spectra)

• Define roadmap for next year– drivers, priorities, goals of the partners– define Y1 standards development efforts– test cases and demos

• Example– SIA V1.1+V2, SSA V1, better RG/DM/UCD/QL integration, web

services, test cases

DAL AgendaTime Activity Comments10:00 Intro/Goals Review agenda

10:15 Scope/Priorities/Architecture What standards are needed; common elements, protocols

11:00 BREAK11:30 Spectral Data Access Use cases, priorities; image model vs 1D

spectrum, FITS/XML

13:00 LUNCH14:00 SIA Enhancement Update on Aladin/CDS integration;

priorities

15:00 Roadmap Drivers, priorities

15:30 BREAK16:00 Roadmap Standards efforts, test cases, demos;

standards process

17:00 CLOSE

Goals for this meeting

• Agree on DAL scope, WG scope– this will be our kick-off meeting

• Strawman data service architecture– what services are required? (e.g., extend image model, add spectra)

• Define roadmap for next year– drivers, priorities, goals of the partners– define Y1 standards development efforts– test cases and demos

• Example– SIA V1.1+V2, SSA V1, better RG/DM/UCD/QL integration, web

services, test cases

Existing DAL Prototypes

• Cone search, SIA (NVO)• Jodrell Bank – imaging online from Merlin data

(Anita Richards)• CVO – WFPC assoc, ROSAT, 2QZ (spectroscopic

survey) – generic DA service for all data (query is separate); different service instances, obey same interface

• AUS/VO – HIPASS, ATCA archive, MACHO, SUMSS, 2DF GRSS – Web query i/f

Existing DAL Prototypes

• AVO– SIA, IDHA (all images)

• AstroGrid– HDX (Starlink) – mainly data model, not much data access yet– Also solar data, space physics data

DAL Portal Concept

• Link between client (user) data analysis software and the VO

• Enables distributed multiwavelength data access and analysis

• Key longer-term goals:– Integrate everyday data analysis with VO– Achieve a scalable VO architecture (same software

used in multiple contexts)

DAL Scope: Software

• Service data access protocols– Catalog, image, spectrum, etc.

• Service frameworks– Virtual data generation and manipulation, grid computing

• Client side integration– API, Web Services client, components, etc.

• Test cases and demonstrations

Denotes first year priority

DAL Scope: Types of data

• Dataset (add later)– Query for any type of data in a given ROI– Use other data access services to drill-down to actual data

• Source catalog– May need simplified catalog query in DAL portal– More sophisticated catalog operations elsewhere in VO

• Image– 2D sky images, spectral data cubes, long slit spectra– Access to raw data via VO framework– Data model issues, representation (FITS, Graphic, XML)– Sparsely sampled images including IFU data

• Spectrum, SED• Time series (later?)• Event and visibility data (later?)

• Key challenges– DAL depends upon registry, query, UCD, DM, representation, etc.– Keep it simple; lower the bar for service implementors

DAL Architecture

• Data Access Services– Multiple services

• One service for each view of the data• At a high level may not need to differentiate data

– Integration of common elements of services– Can sometimes view the same data via different services

• E.g., and event list or visibility dataset viewed as an image

– Simple services for most common cases

Application

DAL ClientAdapter

DALService

SimpleServer Grid

ServiceAnalysis

Component

Data MediatorComponent

Data Access Framework

Client Portal

Service Protocols

VO Data Access Layer Architecture

DAL Services

• Key common elements– Registry (caches resource and service data)– Query syntax and capabilities– Metadata– Data models– Data representation (e.g., VOTable)

First cut at priorities for Y1

• SIA V1.1 – Same as now with minor enhancements

• Simple spectral access– Emphasize 1D and SED

• SIA V2.0– General image model (spectral data cubes etc.)– Explore ways to structure metadata (table vs hierarchical)

• Better integration with VO standards– Query standards, registries, MD(UCDs), DM, data representations

• First steps for event, visibility data– E.g., metadata standards to publish raw data

Spectral Data Access

• Goals / Use Cases– discover, retrieve, display spectra for some object– discover, retrieve, display SED for some object– 1D science spectrum for analysis, e.g., classification– multi-wavelength spectral combination– Generate spectra on the fly from grism/radio/HE data– ??

Spectral Data Access

• Priorities– 1D science spectra– SEDs – spectral image cubes (2 spatial + 1 energy)– 2D science spectra (slit spectra)– ??

Spectral Data Access

• Key Issues– general 3/4D image model vs simple 1D/SED– two services versus 1– FITS vs XML (VOTable)

• both should be available for 1D spectra, SEDs (also graphic)• Would need a new FITS standard for 1D spectra to use FITS here

– ??

SIA Evolution

• Interface concepts– roadmap: V1.0, 1.1; V2.0– separation of query/discovery and access– types of services: atlas, pointed, cutout, mosaic– image generation parameters: query ROI supplies default– full specification of IGP possible for mosaic services

SIA Evolution

• Feedback on SIA V1.0– Aladin/CDS integration (Francois)

SIA – Proposed Enhancements

• Registry integration• Image Attributes

– Image provenance and identification - 1– Spectral bandpass (already present) - 1– Spatial bandpass - 3– Resolution - 1– Limiting flux - 2– Image type (future- v2)– Time of observation (1)– Default for case where there are multiple versions of same dataset

• Image Query– Use of image attributes to refine query (e.g., band)– POS, SIZE generalization? Coordinate frameworks

SIA – Proposed Enhancements

• Others– Support image compression in access protocol

Second cut at priorities for Y1

• SIA V1.1 – Same as now with minor enhancements

• Simple spectral access– Emphasize 1D and SED

• SIA V2.0– General image model (spectral data cubes etc.)– Explore ways to structure metadata (table vs hierarchical)

• Better integration with VO standards– Query standards, registries, MD(UCDs), DM, data representations

• First steps for event, visibility data– E.g., metadata standards to publish raw data

• Web services versions of DAL services

Drivers/Priorities

• IAU demo July 03– Add registry support– Add Aladin, other SIA services– Integrate SIA registry into clients (Aladin, DIS, etc.)

• AAS, AVO demos January 04– Web services versions of services?