OpenUp!

Natural History Heritage Information for Europeana

Gerda KochAIT-Angewandte Informationstechnik

Forschungs-GmbH, Graz/Austriahttp://www.ait.co.at kochg@ait.co.at

OpenUP! > Overall Objective

• Mobilising content from natural history museums, botanical gardens etc.– project aim: 1.1 Mio. Records until Feb. 2014

• Provide infrastructure• Quality control • Access new user communities

– Mapping between Community and EUROPEANA data standards

– Enrichment of metadata towards compliance with EUROPEANA standards and Incorporation of multilingual metadata, in particular common names of organisms

– A single access point to distributed natural history multimedia content for EUROPEANA

– Build upon existing networks in the Natural History domain:• CETAF (Consortium of European Taxonomic Facilities)• GBIF (Global Biodiversity Information Facility)• BioCASe (Biological Collection Access Services)

OpenUP! > Technical and Metadata Objectives

Current Content Delivery Status

Data provision December 2014:1.5+ Mio. records

Images & Sounds & VideosBotany, Zoology, Mineralogy, Anthropology …

Current Metadata Status 2011-2012:OpenUp! has delivered the data in the first two project years via an OAI PMH Provider that delivers ESE data.

2013-2014:A metadata mapping to EDM has been established at the end of 2012. In 2013 a second OAI PMH Provider has been set up that delivers OpenUp! data in EDM format.Test harvests from the EDM provider have been initiated in autumn 2013. From December 2013 onwards OpenUp! is forwarding data in EDM format to Europeana

Metadata Transformation Process

Raw data 1

2

3

4

6

5

Metadata Transformation ProcessThe heterogenous databases of the various providers are imported into BioCASe providers. BioCASe is a transnational network of biological collections.The BioCASe providers use the ABCD(EFG) metadata schema:Access to Biological Collection Databases Extended For Geosciences The ABCD schema has about 1.200 elements and can be used for a wide range of collections/databases:

- data specification for biological collection units, including living and preserved specimens, along with field observations - Used in recording both specimen-specific and collection-specific data

Output: ABCD Record (snippet)

Metadata Transformation ProcessThe data is harvested from the BioCASe providers with the GBIF Harvesting and Indexing Toolkit (HIT).The GBIF Harvesting and Indexing Toolkit (HIT) is a software platform developed by the Global Biodiversity Information Facility (http://www.gbif.org/) to manage biodiversity data harvesting and quickly build indexes of the harvested data.

Metadata Transformation ProcessThe HIT Harvester stores bulks of ABCD–Records into a file system.The Mapping Tool (Pentaho Kettle – Job) picks up the data from this file system.

For data transformation the Open source Business Intelligence Tool Pentaho is used. Pentaho Data Integration delivers the needed Extraction, Transformation and Loading (ETL) capabilities.

Metadata Transformation ProcessThe Mapping Tool (Pentaho Kettle – Transformation) receives ABCD records and processes them:–The data is mapped to EDM –Enriched with the bibliographic information from BHL (relation)–Enriched with the geonames information if coordinates available–Enriched via OpenUp! Vocabulary webservices (common names) –The transformed records are stored into a data base (or file system)

Metadata Transformation ProcessPentaho Kettle – Transformation (Excerpt)

Metadata Transformation Process

Finally the EDM valid data is imported into the OAI PMH Provider for Europeana

During the import the links contained in the data are checked.

Sample OAI

Record

part 1Description of

the Object

Sample OAI Recordpart 2

Description of Images & Website

Sample OAI Recordpart 3

Vocabulary information

Sample OAI Recordpart 4

Aggregation information

Carousel of images

Within the carousel the information related to the web resource will be displayed

(Still work in progress for Europeana)

This geonames info is added by Europeana.

Sample Europeana Record

This geonames info is added by OpenUp!

Sample Europeana Record

Sample Europeana Record

Sample Europeana Record

Metadata Mapping

The OpenUp! ABCD(EFG) to EDM/ESE mapping is documented in the Deliverable D24 and online at: OpenUp! to ESE/EDM documentation http://open-up.eu/node/1238

Finally..

What OpenUp! did in respect to the mapping process…

• Use standard networks and open source tools for data harvesting and data transformation

• Use EDM as it is (no refinements and extensions so far).

Finally..Specific Metadata Issues we had to face…• Copyright Information

– The metadata is not only data about a CHO – BUT the metadata is also the „CHO“ (> research work)

– The metadata can provide very sensitive information (> geolocation of endangered species)

Solution: Restricted and unrestricted ESE/EDM metadata

mappings

– Community Vocabularies must be referenced properly• skos:note:

– rights information for this common name– Geographical information for the common name– Time reference for the usage of the common name– The value “Common Name” (inserted as information to the user)

• skos:editorialnote: – Webpage with the above information > mapped as final dc:subject field

Europeana will not display skos:note in the near future therefore in January 2014 the following workaround was implemented:

OpenUp! WP3AIT Angewandte

Informationstechnik Forschungsgesellschaft mbH

8010 Graz, AustriaGerda Koch, kochg@ait.co.at