safe replication and data staging core services of the eudat cdi johannes reetz, rzg 2nd eudat...

Safe Replication and Data StagingCore Services of the EUDAT CDI

Johannes Reetz, RZG

2nd EUDAT conference

Rome, 29 Oct 2013

2

domain of registered data

individual value

preservationacquisitiongenerationdescription

enrichmentprocessingreductionanalysis

Data Stage-Out

publication

Safe Replication

community valueDat

a Sta

ge-In

society value

IdentifierService

EPIC Handle PID

4

Replication from repositories to data storages in different administrative domains

• (long-term) archiving and preservation

• optimize access for users from different regions

• bring data closer to powerful computers for data analytics

Typical policies triggered by Community Data Managers:• Replicate collection X from my repository to data centres A and B• Store the replica safely for N years• Check the integrity of the replica every M years

5

Transferring data from EUDAT storages to compute facilities

• reliable, efficient, easy-to-use tools to manage data transfers

• ingest data into the EUDAT domain of registered data

Each data centre (administrative domain) has a dedicated Handle namespace (prefix)

replica storages

repositories

6

EPICHandleService

http://cmon.eudat.eu

EPICHandleService

enabled EUDAT sites

replica storages

repositories

7

http://cmon.eudat.eu

EPICHandleService

GridFTP

8

CLARIN (MPI-PL/TLA)production

9

CLARIN (EKUT)production

10

CLARIN (CUNI/LINDAT)production

ENES (DKRZ/CMIP5)

11

production

12

EPOS (INGV)production

13

VPH (UCL)production

14

EPOS (ORPHEUS/ KNMI)work in progress

15

INCF (KTH)work in progress

16

DiXa (EBI)work in progress

Track 1 – Agenda – Session 1 17

14:00: B2SAFE and B2STAGE: Core Services of the EUDAT CDI, Johannes Reetz

14:10: B2SAFE adoption in the EPOS community, Claudio Cacciari

14:25: B2SAFE for Replicating Linguistic Resources, Willem Elbers

14:40: B2SAFE Setup with a Dspace Repository, Pavel Stranak

14:55: VPH and the Biomedical Scientific Case for EUDAT, Peter Coveney

15:10: Community integrated B2STAGE tools, Stefan Zasada/Giuseppe Fiameni

15:35: B2SAFE Practical Policies, Willem Elbers

15:45: B2SAFE Data Policy Manager, Maria Francesca Iozzi

Track 1 – Agenda – Session 1

use cases in production

Track 1 – Agenda – Session 1 18

14:00: B2SAFE and B2STAGE: Core Services of the EUDAT CDI, Johannes Reetz

14:10: B2SAFE adoption in the EPOS community, Claudio Cacciari

14:25: B2SAFE for Replicating Linguistic Resources, Willem Elbers

14:40: B2SAFE Setup with a Dspace Repository, Pavel Stranak

14:55: VPH and the Biomedical Scientific Case for EUDAT, Peter Coveney

15:10: Community integrated B2STAGE tools, Stefan Zasada/Giuseppe Fiameni

15:35: B2SAFE Practical Policies, Willem Elbers

15:45: B2SAFE Data Policy Manager, Maria Francesca Iozzi

use case in production

Track 1 – Agenda – Session 1 19

14:00: B2SAFE and B2STAGE: Core Services of the EUDAT CDI, Johannes Reetz

14:10: B2SAFE adoption in the EPOS community, Claudio Cacciari

14:25: B2SAFE for Replicating Linguistic Resources, Willem Elbers

14:40: B2SAFE Setup with a Dspace Repository, Pavel Stranak

14:55: VPH and the Biomedical Scientific Case for EUDAT, Peter Coveney

15:10: Community integrated B2STAGE tools, Stefan Zasada/Giuseppe Fiameni

15:35: B2SAFE Practical Policies, Willem Elbers

15:45: B2SAFE Data Policy Manager, Maria Francesca Iozzi

Track 1 – Agenda – Session 1

practical policies and policy management

Johannes Reetz RZG / MPG

Claudio Cacciari CINECA

Willem ElbersMax-Planck-Institute for Psycholinguistics

Pavel StranakCharles University

Peter CoveneyUniversity College London

Stefan Zasada University College London

Giuseppe FiameniCINECA

Maria Francesca Iozzi SIGMA/University of Oslo

SAFE REPLICATION AND DATA STAGING - Speakers

20