Slide 1

1 8-Jun-2004Andy Lawrence : PharmaGrid talk, Diessenhofen The Virtual Observatory The VObs concept The structure of the VObs Standards, Standards, Standards Glue : the AstroGrid project Slide 2 The VObs concept Slide 3 3 8-Jun-2004Andy Lawrence : PharmaGrid talk, Diessenhofen VObs : Generic Science Drivers data growth : volume and richness desire to work on-line multi-archive science large database science empowerment Slide 4 4 8-Jun-2004Andy Lawrence : PharmaGrid talk, Diessenhofen Data re-use : a market fact HST : more retrieval than ingest Slide 5 multi- views of a Supernova Remnant Shocks seen in the X- ray Heavy elements seen in the optical Dust seen in the IR Relativistic electrons seen in the radio Slide 6 needles in a haystack Hambly et al 2001 - faint moving object is a cool white dwarf - may be solution to the dark matter problem - but hard to find : one in a million - even harder across multiple archives Slide 7 7 8-Jun-2004Andy Lawrence : PharmaGrid talk, Diessenhofen VObs : Technical Drivers data rate, storage and flops : x 1000 /decade but device bw x 10/decade search engines next to the data backbone network bw ~Gbps but end-end bw ~10Mbps analyse in situ : shift results not data Slide 8 8 8-Jun-2004Andy Lawrence : PharmaGrid talk, Diessenhofen network development higher level protocols ==> transparency TCP/IP message exchange HTTPdoc sharing (web) grid suiteCPU sharing XML/SOAP data exchange Slide 9 9 8-Jun-2004Andy Lawrence : PharmaGrid talk, Diessenhofen the VObs concept web all docs in the world inside your PC VObs all databases in the world inside your PC Slide 10 The structure of the VObs Slide 11 11 8-Jun-2004Andy Lawrence : PharmaGrid talk, Diessenhofen what its not not a monolith not a warehouse Slide 12 12 8-Jun-2004Andy Lawrence : PharmaGrid talk, Diessenhofen VObs framework standards inter-operable data inter-operable software modules no central VO-command Slide 13 13 8-Jun-2004Andy Lawrence : PharmaGrid talk, Diessenhofen data centre primacy data centres take a central role they become active service centres they need to follow common standards keyword is interoperability Slide 14 14 8-Jun-2004Andy Lawrence : PharmaGrid talk, Diessenhofen VObs geometry not a warehouse not a hierarchy not a peer-to-peer system small set of service centres and large population of end users Slide 15 yesterday browser front end CGI request html web page DB engine SQL data Slide 16 today application web service SOAP/XML request SOAP/XML data DB engine SQL native data anything standard formats Slide 17 tomorrow application web service job results anything web service web service web service web service web service Registry Workflow GLUE Certification VO Space standard semantics publish WSDL grid connected Slide 18 18 8-Jun-2004Andy Lawrence : PharmaGrid talk, Diessenhofen publishing metaphor experiments/facilities are authors data centres are publishers VObs portals are shops end-users are readers VObs infrastructure is distribution system. Slide 19 Standards, Standards, Standards Slide 20 20 8-Jun-2004Andy Lawrence : PharmaGrid talk, Diessenhofen International VO alliance (IVOA) http://www.ivoa.net Slide 21 21 8-Jun-2004Andy Lawrence : PharmaGrid talk, Diessenhofen IVOA activities development of standards technical working groups bi-annual interoperability workshops exchange of best practice formal process modelled on W3C Slide 22 22 8-Jun-2004Andy Lawrence : PharmaGrid talk, Diessenhofen IVOA standards process Slide 23 23 8-Jun-2004Andy Lawrence : PharmaGrid talk, Diessenhofen IVOA standards key standards so far protocols for tables, images, spectra resource and service metadata definitions standardised query language provisional semantic dictionary standardised grid and web service interfaces coming up generic data model, identification, semantics workflow and distributed storage Slide 24 Building the Glue : AstroGrid Slide 25 25 8-Jun-2004Andy Lawrence : PharmaGrid talk, Diessenhofen glue accomplishments three big projects US-NVO, UK-AstroGrid, EU-AVO prelim versions of all key components Registry, Workflow, Sign-on, MySpace several crafted-demos some real science done first open user systems early 2005 Slide 26 26 8-Jun-2004Andy Lawrence : PharmaGrid talk, Diessenhofen The AstroGrid project PPARC/EU funded project 2002-2007 7 institutions, 7M, 20 FTEs Main aim : build the VObs glue + contribute to standards programme + working implementation Jan 2005 + full system late 2007 e-project single distributed team co-ord by web-tools : News/Forum/Wiki/Maven http://www.astrogrid.org Slide 27 AVO/AstroGrid Demo Jan 2003 - remote analysis Slide 28 AstroGrid/Aus-VO Demo June 2003 - trans-globe visualisation Slide 29 AVO/AstroGrid Demo Jan 2004 - data federation and first science Slide 30 30 8-Jun-2004Andy Lawrence : PharmaGrid talk, Diessenhofen AstroGrid glue Writing modular components mix-and-match plug-and-play Data Centres and VObs projects worldwide can deploy as they wish But we also deploy through our own portal Slide 31 Current Deployment build flow insert tool in step fill-in tool params end-user needs only browser server s/w installed by data centres Slide 32 Current Deployment : MySpace save workflow, edit and re-use save outputs make available to colleagues Slide 33 33 8-Jun-2004Andy Lawrence : PharmaGrid talk, Diessenhofen next three years intelligent glue ontology, agents analysis services cluster analysis, multi-D visualisation, etc theory services simulated data, models on demand embedding facilities VO ready facilities links to data creation Slide 34 FIN Slide 35 Slide 36 deleted scenes Slide 37 37 8-Jun-2004Andy Lawrence : PharmaGrid talk, Diessenhofen Data Growth : one of many archives Slide 38 38 8-Jun-2004Andy Lawrence : PharmaGrid talk, Diessenhofen solar-terrestrial links Coronal mass ejection imaged by space- based solar observatory Effect detected hours later by satellites and ground radar Slide 39 39 8-Jun-2004Andy Lawrence : PharmaGrid talk, Diessenhofen Workflow Sequence Sequential steps Flow Steps executed in parallel Step Application/Tool, Data query Configured via XForm specification or Query form Workflow definition Subset of BPEL Slide 40 40 8-Jun-2004Andy Lawrence : PharmaGrid talk, Diessenhofen AstroGrid use : workflow Slide 41 41 8-Jun-2004Andy Lawrence : PharmaGrid talk, Diessenhofen Create Workflow Slide 42 42 8-Jun-2004Andy Lawrence : PharmaGrid talk, Diessenhofen Add tools to steps Slide 43 43 8-Jun-2004Andy Lawrence : PharmaGrid talk, Diessenhofen Configure step: data query Slide 44 44 8-Jun-2004Andy Lawrence : PharmaGrid talk, Diessenhofen Configure step: tool Slide 45 45 8-Jun-2004Andy Lawrence : PharmaGrid talk, Diessenhofen Submit workflow, Check status Slide 46 46 8-Jun-2004Andy Lawrence : PharmaGrid talk, Diessenhofen Data query step Insert valid ADQL Assisted query builder coming soon Slide 47 47 8-Jun-2004Andy Lawrence : PharmaGrid talk, Diessenhofen MySpace: virtual storage Operations: Copy, Delete, Move Upload from file, URL End-point Multiple servers per manager Usually hidden from user Slide 48 48 8-Jun-2004Andy Lawrence : PharmaGrid talk, Diessenhofen eg Copy file, Upload file