vo data access layer
Post on 10-Feb-2016
61 Views
Preview:
DESCRIPTION
TRANSCRIPT
VO Data Access Layer
IVOACambridge, UK 12 May 2003
Doug Tody, NRAO
Data access/analysis portal
• Link between client (user) data analysis software and the VO
• Enables distributed multiwavelength data access and analysis
• Key longer-term goals:– Integrate everyday data analysis with VO– Achieve a scalable VO architecture (same software
used in multiple contexts)
DAL Scope: Software
• Service data access protocols– Catalog, image, spectrum, etc.
• Service frameworks– Virtual data generation and manipulation, grid computing
• Client side integration– API, Web Services client, components, etc.
• Test cases and demonstrations
Denotes first year priority
DAL Scope: Types of data
• Source catalog• Image• Spectrum, SED• Time series• Event and visibility data
• Key challenges– DAL depends upon registry, query, UCD, DM, representation, etc.– Keep it simple; lower the bar for service implementors
Goals for this meeting
• Agree on DAL scope, WG scope– this will be our kick-off meeting
• Strawman data service architecture– what services are required? (e.g., extend image model, add spectra)
• Define roadmap for next year– drivers, priorities, goals of the partners– define Y1 standards development efforts– test cases and demos
• Example– SIA V1.1+V2, SSA V1, better RG/DM/UCD/QL integration, web
services, test cases
DAL AgendaTime Activity Comments10:00 Intro/Goals Review agenda
10:15 Scope/Priorities/Architecture What standards are needed; common elements, protocols
11:00 BREAK11:30 Spectral Data Access Use cases, priorities; image model vs 1D
spectrum, FITS/XML
13:00 LUNCH14:00 SIA Enhancement Update on Aladin/CDS integration;
priorities
15:00 Roadmap Drivers, priorities
15:30 BREAK16:00 Roadmap Standards efforts, test cases, demos;
standards process
17:00 CLOSE
Goals for this meeting
• Agree on DAL scope, WG scope– this will be our kick-off meeting
• Strawman data service architecture– what services are required? (e.g., extend image model, add spectra)
• Define roadmap for next year– drivers, priorities, goals of the partners– define Y1 standards development efforts– test cases and demos
• Example– SIA V1.1+V2, SSA V1, better RG/DM/UCD/QL integration, web
services, test cases
Existing DAL Prototypes
• Cone search, SIA (NVO)• Jodrell Bank – imaging online from Merlin data
(Anita Richards)• CVO – WFPC assoc, ROSAT, 2QZ (spectroscopic
survey) – generic DA service for all data (query is separate); different service instances, obey same interface
• AUS/VO – HIPASS, ATCA archive, MACHO, SUMSS, 2DF GRSS – Web query i/f
Existing DAL Prototypes
• AVO– SIA, IDHA (all images)
• AstroGrid– HDX (Starlink) – mainly data model, not much data access yet– Also solar data, space physics data
DAL Portal Concept
• Link between client (user) data analysis software and the VO
• Enables distributed multiwavelength data access and analysis
• Key longer-term goals:– Integrate everyday data analysis with VO– Achieve a scalable VO architecture (same software
used in multiple contexts)
DAL Scope: Software
• Service data access protocols– Catalog, image, spectrum, etc.
• Service frameworks– Virtual data generation and manipulation, grid computing
• Client side integration– API, Web Services client, components, etc.
• Test cases and demonstrations
Denotes first year priority
DAL Scope: Types of data
• Dataset (add later)– Query for any type of data in a given ROI– Use other data access services to drill-down to actual data
• Source catalog– May need simplified catalog query in DAL portal– More sophisticated catalog operations elsewhere in VO
• Image– 2D sky images, spectral data cubes, long slit spectra– Access to raw data via VO framework– Data model issues, representation (FITS, Graphic, XML)– Sparsely sampled images including IFU data
• Spectrum, SED• Time series (later?)• Event and visibility data (later?)
• Key challenges– DAL depends upon registry, query, UCD, DM, representation, etc.– Keep it simple; lower the bar for service implementors
DAL Architecture
• Data Access Services– Multiple services
• One service for each view of the data• At a high level may not need to differentiate data
– Integration of common elements of services– Can sometimes view the same data via different services
• E.g., and event list or visibility dataset viewed as an image
– Simple services for most common cases
Application
DAL ClientAdapter
DALService
SimpleServer Grid
ServiceAnalysis
Component
Data MediatorComponent
Data Access Framework
Client Portal
Service Protocols
VO Data Access Layer Architecture
DAL Services
• Key common elements– Registry (caches resource and service data)– Query syntax and capabilities– Metadata– Data models– Data representation (e.g., VOTable)
First cut at priorities for Y1
• SIA V1.1 – Same as now with minor enhancements
• Simple spectral access– Emphasize 1D and SED
• SIA V2.0– General image model (spectral data cubes etc.)– Explore ways to structure metadata (table vs hierarchical)
• Better integration with VO standards– Query standards, registries, MD(UCDs), DM, data representations
• First steps for event, visibility data– E.g., metadata standards to publish raw data
Spectral Data Access
• Goals / Use Cases– discover, retrieve, display spectra for some object– discover, retrieve, display SED for some object– 1D science spectrum for analysis, e.g., classification– multi-wavelength spectral combination– Generate spectra on the fly from grism/radio/HE data– ??
Spectral Data Access
• Priorities– 1D science spectra– SEDs – spectral image cubes (2 spatial + 1 energy)– 2D science spectra (slit spectra)– ??
Spectral Data Access
• Key Issues– general 3/4D image model vs simple 1D/SED– two services versus 1– FITS vs XML (VOTable)
• both should be available for 1D spectra, SEDs (also graphic)• Would need a new FITS standard for 1D spectra to use FITS here
– ??
SIA Evolution
• Interface concepts– roadmap: V1.0, 1.1; V2.0– separation of query/discovery and access– types of services: atlas, pointed, cutout, mosaic– image generation parameters: query ROI supplies default– full specification of IGP possible for mosaic services
SIA Evolution
• Feedback on SIA V1.0– Aladin/CDS integration (Francois)
SIA – Proposed Enhancements
• Registry integration• Image Attributes
– Image provenance and identification - 1– Spectral bandpass (already present) - 1– Spatial bandpass - 3– Resolution - 1– Limiting flux - 2– Image type (future- v2)– Time of observation (1)– Default for case where there are multiple versions of same dataset
• Image Query– Use of image attributes to refine query (e.g., band)– POS, SIZE generalization? Coordinate frameworks
SIA – Proposed Enhancements
• Others– Support image compression in access protocol
Second cut at priorities for Y1
• SIA V1.1 – Same as now with minor enhancements
• Simple spectral access– Emphasize 1D and SED
• SIA V2.0– General image model (spectral data cubes etc.)– Explore ways to structure metadata (table vs hierarchical)
• Better integration with VO standards– Query standards, registries, MD(UCDs), DM, data representations
• First steps for event, visibility data– E.g., metadata standards to publish raw data
• Web services versions of DAL services
Drivers/Priorities
• IAU demo July 03– Add registry support– Add Aladin, other SIA services– Integrate SIA registry into clients (Aladin, DIS, etc.)
• AAS, AVO demos January 04– Web services versions of services?
top related