pandata photon and neutron data infrastructure 3 november 2011 juan bicarregui
Post on 01-Apr-2015
222 Views
Preview:
TRANSCRIPT
PaNdata Photon and Neutron Data Infrastructure
3 November 2011
Juan Bicarregui
Safety Information for use at RAL
Fire bells... Please leave the building by the closest route
and go to the fire assembly point.
Klaxon... Please stay inside the building and close any
doors or windows having external access.
Emergency Number for FIRE, AMBULANCE or FIRST AID.
short code from any internal phone 2222
or from a mobile phone 01235 778888
Additionally... Please take a few moments to check that any equipment that may be plugged into the mains supply has undamaged leads, no exposed cables, a secure plug and that you have not created a trip hazard especially if you are on an escape route!
Agenda - 3rd Nov 2011 12:00 Arrivals and Lunch14:00 Start of meeting
14:00 Introduction, Introductions and review agenda (Juan Bicarregui )14:05-15:30 Overview of projects (6 projects, 10 minutes each)“Vertical” projects• PaNdata (Juan Bicarregui and others)• CRISP (Jean Francois Perrin)• HDRI (Rainer Gehrke)“Horizontal” Projects• EUDAT (David Corney)• OpenAirePlus (Natalia Manola)• ORCID (Cameron Neylon)Any other related projects or activities (Round Table)15:30 Break16:00-17:30 Discussions on technical areas. Identification of areas for cooperation. Eg • Metadata catalogues and cross searching;• Ids for Data and/or publications. • Unique identification and disambiguation of people; 17:30 End (Taxis to MHH at 17:30, Dinner at MHH at 20:00)
Agenda – 4th Nov 2011 Morning (ISIS TS2 CR16-17, Taxis from MHH at 8:30)
9:00 Introduction (Juan Bicarregui)• Review agenda, Overview, Introductions (JCNS, MaxLab), etc.• Review of any actions from the day before9:15 - 10:45 PaNdata ODI Service Activities • WP3 User AAA Service - Heinz Joseph Weyer (20 mins)• WP4 Data Catalogue Service - George Kourousias (20 mins)
– Common Data Model Access project - Alain Buteau (15 mins)• WP5 Virtual Laboratories - Frank Schluenzen (15 mins)• Discussion (20 mins)10:45 Break11:00 PaNdata ODI Joint Research Activities (3 x 20 minutes + 15)• WP6 Provenance - Brian Matthews (20 mins)• WP7 Preservation - Jean Francoise Perrin (20 mins)• WP8 Scalability - Bill Pulford (20 mins)• Discussion (15 mins)12:15 Lunch
Agenda – 4th Nov 2011 Afternoon (ISIS TS2 CR16-17)
13:15 Dissemination and Engagement• Dissemination (both projects) - Frank Schluenzen (15 mins)• IUCr Working group on Data Deposition - Heinz Joseph Weyer and Alun Ashton (15 mins)13:45 Finalising PaNdata Europe• Review of actions from Period 1 review - Simon Lambert (5 mins)• Remaining deliverables - status and plans • D2.4 Integrated Policy (Due Sept) - Rudolf Dimper (5 mins)• D3.4 Dissemination (Due Nov) - Frank Schluenzen (5 mins - covered above?)• D6.3 and 6.4 Software (Due Sept and Nov) - Mark Johnson (10 mins)• D7.3 and 7.4 Integration (Due Sept and Nov) - Brian Matthews (10 mins) • D1.4 Final Management Report (Due Nov) - Simon (5 mins) • Finances - Juan (5 mins)• Initial planning for the final review – Juan (5 mins) • Collaboration with the US (INFRA-2012-3.2 and 3.3) - Juan (10 mins)14:45 Break15:00 Starting PaNdata ODI.• Management and Administration (Juan Simon, Denise) (30 mins)• Contractual Agreements (GA and CA)• Prepayments• Procedures• Reporting• Review of Early Deliverables and short term plans - Juan (15 mins)• Action review – Juan (15 mins)16:00 Close
Agenda - 3rd Nov 2011 12:00 Arrivals and Lunch14:00 Start of meeting
14:00 Introduction and review agenda (Juan Bicarregui )14:05-15:30 Overview of projects (6 projects, 10 minutes each)“Vertical” projects• PaNdata (Juan Bicarregui and others)• CRISP (Jean Francois Perrin)• HDRI (Rainer Gehrke)“Horizontal” Projects• EUDAT (David Corney)• OpenAirePlus (Natalia Manola)• ORCID (Cameron Neylon)Any other related projects or activities (Round Table)15:30 Break16:00-17:30 Discussions on technical areas. Identification of areas for cooperation. Eg • Metadata catalogues and cross searching;• Ids for Data and/or publications. • Unique identification and disambiguation of people; 17:30 End
Introduction and review agenda
• Thank you!• Introductions (round table)• Structure of EU e-Infrastructure Programme• Reason for invitation
– Technical and organisation benefits• Aims for the afternoon
– Schedule for the afternoon
••• 8
OpenAIREplus
M€ 95
2.3.5. PRACE 3.4 SA – 3.5 NCPs 1.2.2 Data 1.2.1 e-Science env.
Bdg
M€18 M€ 5 M€ 45 M€ 27
EarthServer
BioVeL DRIHM
SCI-BUS
VERCE N4U
GLORIA
SCIDIP-ES ESPAS
transPLANT
PanDataODI ENGAGE
diXa iMarine
agINFRA
EUDAT
PRACE-2IP Discover the COSMOS
FISCAL
ELLA
Virtual Campus Hub
GLOBAL excursion
M€ 18 M€ 5 M€ 45 M€ 27
EuroRIs-Net+
ORIENTplus
FP7-Infrastructures Call 9 Projects Overview
In addition: Exa-scale HPC Call – 3 projects - M€ 25
Tools for virtual research environments
Tools for virtual research environments
Generic services, storage and computation
OA participatory infrastructure
Agricultur
e
Environment
Physics, Engineering
Biolo
gy
Medici
n
e
Atmosphere/Space Physics
Social SciencesScientific Data
(Discipline Specific)
Other Data
Researcher 1
Non Scientific World
Scientific WorldResearcher 2
Aggregated Data Sets(Temporary or Permanent)
Workflows
Aggregation Path
transPLANT
EUDAT
AgINFRA
iMarine
OPENAire Plus
diXa
SCIDIP-ES
ESPAS
ENGAGE
PanDataODI
Scientific Data Landscape of Initiatives – results from call9
VREs
VREs
Introduction and review agenda
• Thank you!• Introductions (round table)• Structure of EU e-Infrastructure Programme• Reason for invitation
– Technical and organisation benefits• Aims for the afternoon
– Schedule for the afternoon
Agenda - 3rd Nov 2011 12:00 Arrivals and Lunch14:00 Start of meeting
14:00 Introduction and review agenda (Juan Bicarregui )14:05-15:30 Overview of projects (6 projects, 10 minutes each)“Vertical” projects• PaNdata (Juan Bicarregui and others)• CRISP (Jean Francois Perrin)• HDRI (Rainer Gehrke)“Horizontal” Projects• EUDAT (David Corney)• OpenAirePlus (Natalia Manola)• ORCID (Cameron Neylon)Any other related projects or activities (Round Table)15:30 Break16:00-17:30 Discussions on technical areas. Identification of areas for cooperation. Eg • Metadata catalogues and cross searching;• Ids for Data and/or publications. • Unique identification and disambiguation of people; 17:30 End
Overview
The PaNdata Collaboration
The Vision
The PaNdata Europe Project
The PaNdata Open Data Infrastructure Project
Looking Forwards
The PaNdata Collaboration• Established 2007 with 4 partners• Expanded since to 11 (now 13) organisations
(see next slide)
• Aims: – “...to construct and operate a shared data
infrastructure for Neutron and Photon laboratories...”
2007 2008 2009 2010 2011 2012 2013 2014 EDNS (4) EDNP (10) PaNdataEurope(11) Pandata ODI(11)
PaN-data bring together 11 major European Research Infrastructures
PaN-data is coordinated by the e-Science Department at the Rutherford Appleton Laboratory, UK
ISIS is the world’s leading pulsed spallation neutron source
ILL operates the most intense slow neutron source in the world
PSI operates the Swiss Light Source, SLS, and Neutron Spallation Source, SINQ, and is developing the SwissFEL Free Electron Laser
HZB operates the BER II research reactor the BESSY II synchrotron
CEA/LLB operates neutron scattering spectrometers from the Orphée fission reactor
ESRF is a third generation synchrotron light source jointly funded by 19 European countries
Diamond is new 3rd generation synchrotron funded by the UK and the Wellcome Trust
DESY operates two synchrotrons, Doris III and Petra III, and the FLASH free electron laser
Soleil is a 2.75 GeV synchrotron radiation facility in operation since 2007
ELETTRA operates a 2-2.4 GeV synchrotron and is building the FERMI Free Electron Laser
ALBA is a new 3 GeV synchrotron facility due to become operational in 2010
PaN-data Partners
JCNS Juelich Centre for Neutron Science MaxLab, Max IV Synchrotron
PaN-data Applications
The partners operate hundreds of instruments used by over 30,000 scientists each year
These instruments support scientific fields as varied as:• Physics, Chemistry, Biology, Material sciences, Energy technology,
Environmental science, Medical technology and Cultural heritage
Applications include:
• crystallography that reveals the structures of viruses and proteins important
for the development of new drugs
• neutron scattering that identifies stresses within engineering components
such as turbine blades
• tomography that can image microscopic details of the 3D-structure of the
brain
Industrial applications include pharmaceuticals, petrochemicals and microelectronics
PaN-data Europe – building a sustainable data infrastructure for Neutron and Photon laboratories
Overview
The PaNdata Collaboration
The Vision
The PaNdata Europe Project
The PaNdata Open Data Infrastructure Project
Looking Forwards
Science driver – Data IntegrationNeutron diffraction X-ray diffraction
}NMR
High-qualitystructure refinement
}
What is e-Infrastructure?
DataCreation
Archival
Access
Storage ComputeNetwork
Services
Curation
the researcher actsthrough ingest and access
Virtual Research Environment
the researcher shouldn’t have to worry about the information infrastructure
Information Infrastructure
EDNS - European Data Infrastructure for Neutron and Synchrotron Sources
PaNdata Vision
Single Infrastructure Single User Experience
CapacityStorage
Publications Repositories
Data Repositories
Software Repositories
Raw Data
Data Analysis
Analysed Data
Publication Data
Publications
Facility 1
Raw Data
Data Analysis
Analysed Data
Publication Data
Publications
Facility 2
Raw Data
Data Analysis
Analysed Data
Publication Data
Publications
Facility 3
Different Infrastructures Different User ExperiencesRaw Data Catalogue
Data Analysis
Analysed Data Catalogue
Publication Data Catalogue
Publications Catalogue
In words:PANdata will provide our user communities with data repositories and data management tools to: • deal with large sets and large data rates from the experiments, • enable easy and standardised annotation of data, • allow transparent and secure remote access to data, • establish sustainable and compatible data catalogues, allow long-term preservation of data, and • provide compatible open source data analysis software.
This will have a major impact on our scientific user community because it will offer: • cross facility and cross discipline data analysis, • secure access to large data sets over the network instead of using portable media, • maintaining the records of science by having properly annotated data, • linking publications to data, • allowing efficient software developments, and
• efficient scientific collaborations across Europe by providing compatible data formats and analysis software.
Metadata and Digital Curation
Proposal
Approval
SchedulingExperiment
Data cleansing
Record Publication
Scientist submits application for
beamtime
Facility committee approves application
Facility registers, trains, and schedules
scientist’s visit
Scientists visits, facility run’s experiment
Subsequent publication registered
with facility
Raw data filtered and cleansed
Data analysis
Tools for processing made available
Overview
The PaNdata Collaboration
The Vision
The PaNdata Europe Project
The PaNdata Open Data Infrastructure Project
Looking Forwards
PaN-data Standardisation
PaN-data Europe is undertaking 5 standardisation activities:
1. Development of a common data policy framework
2. Agreement on protocols for shared user information exchange
3. Definition of standards for common scientific data formats
4. Strategy for the interoperation of data analysis software enabling the most appropriate software to be used independently of where the data is collected
5. Integration and cross-linking of research outputs completing the lifecycle of research, linking all information underpinning publications, and supporting the long-term preservation of the research outputs
PaN-data Europe – building a sustainable data infrastructure for Neutron and Photon laboratories
PaN-data Europe TimelinePaN-data Europe runs from June 2010 until December 2011 with workshops in Spring and Autumn 2011.
PaN-data Europe – building a sustainable data infrastructure for Neutron and Photon laboratories
Workpackage (abbreviated title) Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov
Milestones M1 M2 W1 M3 M4 W2
WP1 Management D D D D WP1 Management
WP2 Common data policy framework D D D D WP2 Common data policy framework
WP3 Knowledge exchange/dissemination D D D D WP3 Knowledge exchange/dissemination
WP4 Common user information exchange D D D WP4 Common user information exchange
WP5 Scientific data D D D WP5 Scientific data
WP6 Data analysis software infrastructure D D D D WP6 Data analysis software infrastructure
WP7 Integration and cross-linking D D D WP7 Integration and cross-linking
Key
D - Deliverable
M - MilestoneW - Workshop
Workpackage (abbreviated title)
Workshops
Data Policy
Development and delivery
of the comm
on data policy
User and Data Standards
Delivery of draft standards
for data and user information
Baseline for integration
Delivery of policy on user inform
ation, first report on
publications and integration
Integration proposalDelivery of policy and
first proposal on integration and on analysis soft
ware
Final Workshop
Final reports on standards
M1
M2
M3
M4
2.1 Data Policy
2.2Software
Policy
2.3UserPolicy
2.4Integrated
Policy
4.1User
Proposal
4.2User
Workshop
4.3User
Revision
5.1Data
Proposal
5.2Data
Workshop
5.3Data
Revision
6.1SoftwareReview
6.2Software
Workshop
6.3SoftwareProposal
6.4Software Revision
7.1Integration
Report
7.2Integration Proposal
7.3Integration Revision
3.4
Final
Workshop
Project Management, Knowledge Exchange and Dissemination Activities
Dependencies between the major project tasks
Dependencies
Overview
The PaNdata Collaboration
The Vision
The PaNdata Europe Project
The PaNdata Open Data Infrastructure Project
Looking Forwards
ERA Open Access Sharing Initiatives (examples, etc)
ERA Infrastructure Platform Initiatives (EGI, etc)
PaNdata Support Action
(Ends 30 Nov 11)
Policies and Standards
PaNdataODI
(begins end2011)
JRAs
Users
Data
Software
Integration
Provenance
Preservation
Scalability
PaNdataODI
(begins end2011)
ServicesUsers
Data
PaNdataODI
Virtual Labs
Policies Powder Diff
SAXS & SANS
Tomography
ObjectivesObjective 2 – UsersTo deploy, operate and evaluate a system for pan-European user identification across the participating facilities and
implement common processes for the joint maintenance of that system.
Objective 3 – DataTo deploy, operate and evaluate a generic catalogue of scientific data across the participating facilities and promote
its integration with other catalogues beyond the project.
Objective 4 – Provenance To research and develop a conceptual framework, defined as a metadata model, which can record the analysis
process, and to provide a software infrastructure which implements that model to record analysis steps hence enabling the tracing of the derivation of analysed data outputs.
Objective 5 – PreservationTo add to the PaNdata infrastructure extra capabilities oriented towards long-term preservation and to integrate
these within selected virtual laboratories of the project to demonstrate benefits. These capabilities should, as for the developments in the provenance JRA, be integrated into the normal scientific lifecycle as far as possible. The conceptual foundations will be the OAIS standard and the NeXus file format.
Objective 6 – Scalability To develop a scalable data processing framework, combining parallel filesystems with a parallelized standard data
formats (pNexus pHDF5) to permit applications to make most efficient use of dedicated multi-core environments and to permit simultaneous ingest of data from various sources, while maintaining the possibility for real-time data processing.
Objective 7 – DemonstrationTo deploy and operate the services and technology developed in the project in virtual laboratories for three specific
techniques providing a set of integrated end-to-end data services.
PaNdata ODI Joint Research Activities
PaNdata ODI Service Activities
PaNdata ODI Service ReleasesStandards from
PaNdataSupport Action
uCat
dCat
vLabs
Prov
Pres
Scale
Rel 1 Rel 2 Rel 3 Rel 4
users
data
s/w
Integ
Jun 2014Jun 2013 Dec 2013Dec 2012
Overview
The PaNdata Collaboration
The Vision
The PaNdata Europe Project
The PaNdata Open Data Infrastructure Project
Looking Forwards
Data
The Research Lifecycle
the researcher actsthrough ingest and access
Research Environment
Creation
Archival
Access
Storage ComputeNetwork
Data
Services
the researcher shouldn’t have to worry about the information infrastructure
Information Infrastructure
ICAT
TopCAT
EGIGEANT
Local resources
User Info feedDAQ feed
Data Analysis feed Provenanced Data
OECD Principles and Guidelines for Access to Research Data from Public Funding
13 principles
A – Openness • Openness means access on equal terms for the international research community at
the lowest possible cost, ....
B – Flexibility, C – Transparency, D – Legal conformity, E – Protection of intellectual property, F – Formal responsibility, G – Professionalism
H – Interoperability• Technological and semantic interoperability is a key consideration in enabling and
promoting international and interdisciplinary access to and use of research data. ...
I – Quality, J – Security, K – Efficiency, L – Accountability
M – Sustainability• ... taking administrative responsibility for the measures to guarantee permanent access
to data that have been determined to require long-term retention.
[http://www.oecd.org/dataoecd/9/61/38500813.pdf]
The 7 C’s
Creation Collection
Capacity
Computation
Curation
Collaboration Communication
PaNdataEurope SA
PaNdata ODI
PaNdata VRE
DataCreation
Archival
Access
Storage ComputeNetworkServices
Curation
Overview
The PaNdata Collaboration
The Vision
The PaNdata Europe Project
The PaNdata Open Data Infrastructure Project
Looking Forwards
www.pan-data.eu
Thank You
Agenda - 3rd Nov 2011 12:00 Arrivals and Lunch14:00 Start of meeting
14:00 Introduction and review agenda (Juan Bicarregui )14:05-15:30 Overview of projects (6 projects, 10 minutes each)“Vertical” projects• PaNdata (Juan Bicarregui and others)• CRISP (Jean Francois Perrin)• HDRI (Rainer Gehrke)“Horizontal” Projects• EUDAT (David Corney)• OpenAirePlus (Natalia Manola)• ORCID (Cameron Neylon)Any other related projects or activities (Round Table)15:30 Break16:00-17:30 Discussions on technical areas. Identification of areas for cooperation. Eg • Metadata catalogues and cross searching• Ids for Data and/or publications• Unique identification and disambiguation of people17:30 End
Agenda – 4th Nov 2011 Morning (ISIS TS2 CR16-17)
• 9:00 Introduction (Juan Bicarregui)• Review agenda, Overview, Introductions (JCNS, MaxLab), etc.• Review of any actions from the day before• 9:15 - 10:45 PaNdata ODI Service Activities • WP3 User AAA Service - Heinz Joseph Weyer (20 mins)• WP4 Data Catalogue Service - George Kourousias (20 mins)
– Common Data Model Access project - Alain Buteau (15 mins)• WP5 Virtual Laboratories - Frank Schluenzen (15 mins)• Discussion (20 mins)• 10:45 Break• 11:00 PaNdata ODI Joint Research Activities (3 x 20 minutes + 15)• WP6 Provenance - Brian Matthews (20 mins)• WP7 Preservation - Jean Francoise Perrin (20 mins)• WP8 Scalability - Bill Pulford (20 mins)• Discussion (15 mins)• 12:15 Lunch
Agenda – 4th Nov 2011 Afternoon (ISIS TS2 CR16-17)
13:15 Dissemination and EngagementDissemination (both projects) - Frank Schluenzen (15 mins)IUCr Working group on Data Deposition - Heinz Joseph Weyer and Alun Ashton (15 mins)13:45 Finalising PaNdata EuropeReview of actions from Period 1 review - Simon Lambert (5 mins)Remaining deliverables - status and plans D2.4 Integrated Policy (Due Sept) - Rudolf Dimper (5 mins)D3.4 Dissemination (Due Nov) - Frank Schluenzen (5 mins - covered above?)D6.3 and 6.4 Software (Due Sept and Nov) - Mark Johnson (10 mins)D7.3 and 7.4 Integration (Due Sept and Nov) - Brian Matthews (10 mins) D1.4 Final Management Report (Due Nov) - Simon (5 mins) Finances - Juan (5 mins)Initial planning for the final review – Juan (5 mins) Collaboration with the US (INFRA-2012-3.2 and 3.3) - Juan (10 mins)14:45 Break15:00 Starting PaNdata ODI.Management and Administration (Juan Simon, Denise) (30 mins)Contractual Agreements (GA and CA)PrepaymentsProceduresReportingReview of Early Deliverables and short term plans - Juan (15 mins)Action review – Juan (15 mins)16:00 Close
WP2 DisseminationObjectives Engagement with other initiatives and dissemination of project results, in particular to other
research infrastructures.Task 2.1. Establish an external web site as an extension to the existing website for the PaNdata
collaboration (www.pandata.eu). Task 2.2. Establish an interest group for project news items via community channels, informing
them of project progress. Task 2.3. Presentations to relevant international audiences at conferences, symposia, other
project meetings etc. Task 2.4. Provision of the open source software and appropriate documentation to potential
partner bodies. Task 2.5. Workshops to present the integrated systems to user and facility communities.
D2.1 : Project Website (M1) – November 2011D2.2 : Dissemination plan (M3)D2.3 : First Open Workshop (M15) – January 2013D2.4 : Open Source software distribution procedure (M21)D2.5 : Second Open Workshop (M27) - January 2014
WP3 User Catalogue and AAA ServiceObjectives To deploy, operate and evaluate a protocol for pan-European user identification across the participating facilities and implement common processes for the joint maintenance of that system.Task1: Consultation on existing software components recommendations for technologies to be implemented.Task 2: Set up team includes representatives from the user office and/or IT staff of the partners.Task 3: Specify an architecture which ... builds on the IRUVX "umbrella" concept. Task 4: Implement ... the necessary local modifications (including trust management). Task 5: Implement a standard affiliation database which is accessible for update and use by the participating facilities ...
Introduce a central affiliation database according to the PaNdata de-facto standard.Provide an interface of the local WUO systems to this standard. Organise and support the migration of the local WUOs to this new affiliation database.
Task 6: Deploy the user management system at all participating facilities. A major factor will be the integration with the facility's bespoke user administration systems. The deployment will include setting up of an administration authority for the system.
Task 7: Evaluate the system within a subset of the collaborating facilities. Task 8: Operate and report on the AAA trust system for the remainder of the project. Task 9: Maintain communication with other user authentication systems (through Workpackage 2) ...
D3.1 : Specification of AAA infrastructure (M6) Apr 2012D3.2 : Pilot deployment of initial AAA service infrastructure (M12) Nov 2012D3.3 : Production deployment of AAA service infrastructure (M18) Apr 2013D3.4 : Evaluation of initial AAA service infrastructure (M24) Nov 2014
WP4 Data CatalogueObjectives To deploy, operate and evaluate a generic catalogue of scientific data across the participating facilities and promote its integration with other catalogues beyond the project• develop generic software infrastructure to support interoperation of facility data catalogues• deploy this software to establish a federated catalogue of data across the partners, • provide data services based upon this generic framework which will enable users to deposit, search, visualise, and analyse data across the partners’ data repositories, • evaluate this service from the perspective of facility users, • manage jointly the evolution of this software and the services based upon it, • promote the take up of this technology and the services based upon it beyond the projectTask 4.1. Survey the features of existing implementations of metadata catalogues ...Task 4.2. ... deploy the chosen metadata catalogue in the legacy context of the facilities. Task 4.3. Provide remote API access to the individual catalogues and integrate to provide a single search capability across the collaborating facilities. Task 4.4. Evaluate the performance of searching the metadata catalogue and retrieving data.
D4.1. Requirements analysis for common data catalogue (M9) D4.2. Populated metadata catalogue with data from the virtual laboratories (M15) D4.3. Deployment of cross-facility metadata searching (M21) D4.4. Benchmark of performance of the metadata catalogue (M27)
WP5 Virtual Laboratories (Service)Objectives To deploy a set of integrated end-to-end user and data services supporting three specific
techniques: • Structural 'joint refinement' against X-ray & neutron powder diffraction data • Simultaneous analysis of SAXS and SANS data for large scale structures • Access to tomography data exemplified through paleontological samples
D5.1: Specific requirements for the virtual laboratories (M6) Apr 2012
D5.2: Deployment of Specification of the three virtual laboratories (incorporating any specific requirements software to support them) (M18) Apr 2013
D5.3: Report on the implementation of the three virtual laboratories (M30) Apr 2014
WP6 Provenance (JRA)Objectives To develop a conceptual framework, which can record and recall the data continuum, and
especially the analysis process, and to provide a software infrastructure which implements that model to record analysis steps hence enabling the tracing of the derivation of analysed data outputs
Task 1: Requirements for Provenance Task 2: Modelling the data continuum Task 3: Ontologies for specific instruments/techniques Task 4: Tool Support for the Data Continuum Task 5: Tracing the Data Continuum Task 6: Evaluation
D6.1: Model of the data continuum in Photon and Neutron Facilities (M12) Nov 2012D6.2: Common ontology definition and definition of tools to support the use of provenance
for Photon and Neutron Facilities (M18) Apr 2012D6.3: Tools for building research objects in Photon and Neutron Facilities (M24) Nov 2013D6.5: Evaluation report on provenance management in Photon and Neutron Facilities (M30)
Apr 2014
WP7 Preservation (JRA)Objectives To incorporate models and tools oriented towards long-term data preservation into the
PaNdata infrastructure, focussing on several aspects considered of benefit: an OAIS-based infrastructure; persistent identifiers; and certification of authenticity and integrity
Task 1. Baseline and OAIS application Task 2. Persistent identifiers (for datasets)Task 3. Representation information and archiving
RI for datasets, and AIPs (Archival Information Packages)This will include software as a kind of representation information, and the need to preserve the software itself.
Task 4. Integrity of datasets Mechanisms for maintaining and checking integrity of datasets. (for individual datasets (as preservation actions are
performed) and for data holdings as a whole.
Task 5. Evaluation and reporting
D7.1 Implementation of persistent identifiers for PaNdata datasets (M15) Jan 2013D7.2 Mechanisms and tools for representation information and archiving (M21) July 2013D7.3 Mechanisms and tools for integrity of datasets(M27) Jan 2014D7.4 Report on evaluation of preservation mechanisms (M30) Apr 2014
WP8 Scalability (JRA)Objectives To develop a scalable data processing framework combining parallel filesystems with a
parallelized standard data format (pNexus pHDF5) to permit applications to make most efficient use of dedicated multi-core environments and to permit simultaneous ingest of data from various sources, while maintaining the possibility for real-time data processing.
Task 1: pNexus API (Develop a pHDF5 compliant Nexus API.)Task 2: Investigate parallel file systems. Task 3: Investigate implementations on specific file systems
MPI-I/O implementations and pHDF5/pNexus on an even smaller number of preselected file systems.
Task 4: Coupling of advanced (pre-)processing engines.– Test the capability of the system to cope with multiple parallel data streams. This will contain for example
explicit tests feeding a pHDF5-file consisting of a large number of individual images into a multi-core analysis engine.
Task 5: Demonstration.D8.1: Definition of pHDF5 capable Nexus implementation (M9) - Software D8.2: Evaluation of Parallel filesystems and MPI I/O implementations (M9) - Report D8.3: Implementation of pNexus and MPI I/O on parallel filesystems (M21) - Prototype D8.5: Examination of Distributed parallel filesystem (M21) - Report D8.6: Demonstrate capabilities on selected applications (M21) - Demonstrator D8.7: Evaluation of coupling of prototype to multi-core architectures (M30) - Report
PaN-data Europe: actions fromPeriod 1 review
• Review report received 11 October• All deliverables accepted except D1.1.3 “Second
(annual) management report”– Revise financial statements; clarification of Table 3.5
(PSI) required– For Recommendation 4 see page 18, 2nd sentence:
“organized transition” to PaN-data ODI• But is D1.1.3 the correct place for this?
PaN-data Europe: actions fromPeriod 1 review
D2.4
D3.4
D2.4?
???
PaN-data Europe: actions fromPeriod 1 review
• We still need to resubmit the cost claims for Period 1
• To be submitted along with the revised management report in the NEF session
PaN-data Europe deliverable D1.4• D1.1.4 = Final Management Report• Due Month 18 = end of November• Also cost claim submission for last six months
PaN-data Europe deliverable D1.4• Standard template for final report
– Final publishable summary report• Executive summary (1 page)• Summary description of project context and objectives (≤ 4
pages)• Description of main S&T results/foregrounds (≤ 25 pages)• Potential impact (including the socio-economic impact and
the wider societal implications of the project so far) and the main dissemination activities and exploitation of results (≤ 10 pages)
• Address of the project public website and relevant contact details
PaN-data Europe deliverable D1.4• Standard template for final report
– Use and dissemination of foreground• Section A: List of scientific papers and dissemination activities• Section B: Specifies the exploitable foreground and provides
the plans for exploitation
– Report on societal implications• Ethics, workforce statistics, gender aspects, synergies with
science education, interdisciplinarity, engaging with civil society and policy makers, use and dissemination, media and communication to the general public
WP1 ManagementObjectives To establish an effective and efficient collaboration between the partners... To ensure that the project achieves its objectives ... To report to the Commission as required...
Task 1.1: Set up mechanisms to run the project through the rest of its duration (M1–M2).Task 1.2: Monitor progress of project activities and put in place appropriate corrective actionsTask 1.3: Organise general meetings of the project (kick-off and bi-annually thereafter).Task 1.4: Report to EC on the technical and financial progress of the project (annually and at
the end of the project).
D1.1: Project management structures, reporting, risk and quality ... procedures (M3)D1.2: First annual management report (M12)D1.3: Second annual management report (M24)D1.4: Final management report (M30)
Project procedures• Start up• Governance• Communications and meetings• Deliverables• Reporting and cost statements
Start-up• Grant Preparation Forms (GPFs)
– All done!• Grant Agreement
– All done!• Consortium Agreement
– DESCA v2 agreed in principle– Draft distributed and feedback received– Almost ready
• Pre-financing– Need to check
GovernanceProject Management Board (PMB).
Responsible for : Budget, consortium, activities, performance of the contractors, arbitrating on any conflict, IPR, risks, approve all new contractors etc
From Proposal:“The PMB will be chaired by a senior representative from the coordinating partner and include the Project Manager and one voting representative from each of the partners. Dr. Robert McGreevy will be the initial chair of the PMB his possible replacement could be undertaken by a majority vote of the PMB. A meeting of the PMB will be held at the Project Kick Off for validating the activities, the structural methods, the planning and the budget, and then at least 4 times a year.”
From CA: For the purposes of this Consortium Agreement references to the General Assembly shall mean the Project Board.The General Assembly shall consist of one representative of each Party (hereinafter referred to as “Member”).Each Member shall be deemed to be duly authorised to deliberate, negotiate and decide on all matters listed in Article 6.3.6 of this Consortium Agreement.The Coordinator shall chair all meetings of the General Assembly, unless decided otherwise by the General Assembly.
Project Manager (PM) ...interface between the Consortium and the European Commission.
“The PM is in charge of all administrative and financial matters, included in WP1, ...The PM is responsible for the follow up of the deliverables and milestones with help from WP WPLs. ... chairs the monthly project meetings via teleconference,Dr. Juan Bicarregui from the e-Science Centre, STFC will be appointed project manager for the duration of the project. His possible replacement is the responsibility of the Project Management Board.”
GovernanceWork Package Leader (WPL).
“the WPL will be responsible for scheduling work tasks, allocating resources available, and coordinating the production of deliverables to time and budget. The WPL will report on progress to the PM ...The PM and WPLs will consult regularly, with monthly teleconferences. ...Tolerances will be agreed between PM and WPL on each of the workpackages, ....
Management STFC Juan BicarreguiDissemination DESY Rainer GehrkeUsers PSI Heinz Joseph WeyerData ELETTRA George Kourousias ?Virtual Laboratories DESY Thorsten KrachtProvenance STFC Brian MatthewsPreservation ILL Jean-François Perrin ?Scalability Diamond Bill Pulford
Decision-making Process• The ultimate decision making entity of the project is the PMB. However, day to day decisions will be made by
either the PM or the WPLs as required. Decisions within the PMB are reached by consensus. In the event that no consensus is reached, decisions will be made by simple majority vote of the project partners. ...
Management of Knowledge and IPR• ... scientific publications and presentations at conferences or exhibitions. • ...Software and standards arising from the project will be available on an open-source basis and will be
disseminated to other large-scale scientific facilities. These activities will be under the co-ordination of the WP3 Leader.
• The Consortium Agreement will lay down rules for the ownership and protection of knowledge as well as for access rights. ...
• the WP3 leader will be in charge of collecting and proposing matters referring to the results for dissemination. ...Open Access• In accordance with the European Commission‘s Open Access Pilot (see for example
ftp://ftp.cordis.europa.eu/pub/fp7/docs/open-access-pilot_en.pdf), the project team will deposit peer-reviewed articles arising from the project into suitable institutional or subject-based repositories, using best efforts to ensure open access to the articles within six months. An example of such a repository already well established within the consortium is STFC‘s ePubs (http://epubs.stfc.ac.uk). (Or the Wiki)
Risk Management and Mitigation Plan• ...Section 1.3.5 gives a summary of the initial high level risks and a prevention and remedy strategy for each.• The project management, coordinated by the PM, shall identify and monitor risks that may have an impact on the
project schedule and outcomes and shall take appropriate measures to limit and/or mitigate their effects. ...• Risk management will be a standing agenda item of all PMB meetings.Quality Management • ...The project will establish a quality assurance system, under the responsibility of the PM, and devolved to WPLs
for each work package. Each deliverable will be subject to internal review for completeness, accuracy and consistency.
Governance
Communications and meetings• Monthly telecon• Project Management Board – four times per year
– Voting procedure etc. is in the CA• Aim for three face to face meetings per year
Communications and meetings• We have 3 or 4 official names per partner
– signatories, admin contact, scientific contact• We have the PANDATA mailing list (54 names)
– PANDATA smaller list?– PANDATA WP leaders (and friends)
Communications and meetingsfreddie.akeroyd@STFC.AC.UK Freddie AkeroydDebbie.Greenfield@STFC.AC.UK Debbie Greenfieldtom.griffin@STFC.AC.UK Tom Griffinneil.geddes@STFC.AC.UK Neil Geddesjuan.bicarregui@STFC.AC.UK Juan Bicarreguisimon.lambert@STFC.AC.UK Simon Lambertbrian.matthews@STFC.AC.UK Brian Matthewsrobert.mcgreevy@STFC.AC.UK Robert McGreevyk.shankland@READING.AC.UK Kenneth Shanklanddenise.small@STFC.AC.UK Denise Smallalun.ashton@DIAMOND.AC.UK Alun Ashtonbill.pulford@DIAMOND.AC.UK Bill Pulfordfulvio.bille@ELETTRA.TRIESTE.IT Fulvio BilleMirjam.vanDaalen@PSI.CH Mirjam van Daalendimper@ESRF.FR Rudolf Dimperstephan.egli@PSI.CH Stephan Egliderek.feichtinger@PSI.CH Derek Feichtingerdfernandez@CELLS.ES David Fernandezpicca@SYNCHROTRON-SOLEIL.FR picca frédéric-emmanuelbrigitte.gagey@SYNCHROTRON-SOLEIL.FR Brigitte GAGEYcgirbau@CELLS.ES Conchi Girbaugoetz@ESRF.FR Andy Goetzandy.gotz@ESRF.FR Andy Gotzvolker.guelzow@DESY.DE Volker Guelzowdietmar.herrendoerfer@HELMHOLTZ-BERLIN.DE Dietmar Herrendoerferdeborah.iorio@SYNCHROTRON-SOLEIL.FR Deborah IORIOjohnson@ILL.FR Mark Johnson
The PaN-data email list: stephane.longeville@CEA.FR Stephane LONGEVILLEphilippe.martinez@SYNCHROTRON-SOLEIL.FR Philippe Martinezchristian.jung@HELMHOLTZ-BERLIN.DE Christian Jungklora@CELLS.ES Jorg KloraMark.koennecke@PSI.CH Mark Koenneckkegeorge.kourousias@GMAIL.COM George Kourousias(2)thorsten.kracht@DESY.DE Thorsten Krachtute.krell@DESY.DE Ute Krellmetge@CELLS.ES Joachim Metgemutti@ILL.EU Paolo Muttijean.pearce@STFC.AC.UK Jean Pearceperrin@ILL.EU Jean-François Perrinstephane.poirier@SYNCHROTRON-SOLEIL.FR Stephane Poirierporte@ESRF.FR Dominique Portemilan.prica@ELETTRA.TRIESTE.IT Milan Pricapascale.prigent@SYNCHROTRON-SOLEIL.FR Pascale Prigentpugliese@ELETTRA.TRIESTE.IT Roberto Pugliesermpugliese@GMAIL.COM Roberto Pugliese(2)isabelle.rauschenbach@DESY.DE Isabelle RauschenbachTobias.Richter@DIAMOND.AC.UK Tobias Richterdsalvat@CELLS.ES Daniel Salvatfrank.schluenzen@DESY.DE Frank Schluenzenschwarzkopf@BESSY.DE Olaf Schwarzkopfsole@ESRF.FR Armando Soleheinz-josef.weyer@PSI.CH Heinz-Joseph Weyermichael.wilson@STFC.AC.UK Michael Wilsonalain.buteau@SYNCHROTRON-SOLEIL.FR BUTEAU Alain
* Total number of users subscribed to the list: 54
Deliverables• No period of grace in FP7 – the due date is the due
date!• Internal review procedure
Reporting and cost statements• Quarterly activity progress reports with effort
estimates• Six-monthly effort reports with progress• Formal report annually
– Including costs– STFC can input into NEF
• Prepayment plus annual payments and final payment
Website and Dissemination• Website• Open workshops Month 15, 27
Actions• User: Harmonisation mtg Dec 8 Hamburg• 2012 - 1 Feb - friendly user phase start• Ask US facility if they want to be involved in
counting users• End Spring 2012, review firnedly user phase• Data: ISIS – Elettra meeting mid Nov.• Comparison of alternatives: input to ICAT
roadmap.• Vlabs: identify software, definitin of Wflows
Action• Work with vlabs on user cases• IUCr liaison (HJW)• Scaleability – list of whos involved and VC.• Brian and Heinz to liasie with ORCid• Communicate with OpenAire (Brian)• Send list of review reply plans (Simon) • Next Mtg – February (before M6 deliverable)• Plan review – suggest Feb • Vlabs reqs mtg/ ICAT mtg (all 3 at DESY or ESRF)
• Propose dates for PMB mtgs. ( Denise) • Propose dates for monthly telecons.• Check single PMB member per organisation
(Denise)• Check WP leaders
top related