locloud micro services and the digitisation workflow

XIIth Annual International Conference for Professionals in Cultural HeritageLoCloud Micro Services and the Digitisation Workflow

Walter Koch ([email protected])AIT-Angewandte Informationstechnik

Forschungs GmbH, Graz - Austria

LoCloud is funded by the European Commission's ICT Policy Support Programme

2015-11-08 Walter Koch (AIT) 2

Content

• LoCloud Project (relevant activities)• Vocabulary Microservice• Microservice used in the OpenUp! Project• Digitisation Workflow (Books)• Digital Curation (Long Term Preservation)

LoCloud Project(relevant activities)



locoud.eu

About:– LoCloud is a Best Practice Network co-funded under the

CIP ICT-PSP programme of the European Commission.– LoCloud is supporting small and medium-sized

institutions in making their content and metadata available to Europeana.

– A cloud-based technology infrastructure will enable the aggregation of local content, and

– a number of micro-services will help to reduce technical, semantic and skills barriers and to render the content more discoverable and interoperable.


Microservices

Six development teams: AIT, Athena RC, AVINET, IPCHS, UPV/EHU, VUKFC

All the services are implemented on virtual machines in a cloud testlab (using the OpenNebula cloud computing platform)

API documentation is available for each serviceEnd-user interfaces are provided for three microservicesEach service has been integrated in MORe


LoCloud Microservices

http://support.locloud.eu/

Cloud-based micro-services for metadata enrichment and capture: •Geo-location API•Vocabulary service•Historic Place Names service•Geo-coding application

• Vocabulary matching service • Background linking service• Wikimedia application


LoGeo API• Uses NER-Name Entity Recognition• Recognizes and returns one or more place names

candidates accompanied by geographical coordinates.

GPLIPCHS


Vocabulary service• Based on “TemaTres”• Incorporates 29 standard

thesauri • allows users to add and

create vocabularies, and add terms or translations

• add thesauri terms to all items of a metadata package

• Integrated in MORE• Exploited by the LoCloud

vocabulary matching microservice via its API

GPLAIT


Historic Place Names service

• A semi-automatic historical geo-information management function and web service.

• Enables local cultural institutions to collaborate in developing the Thesaurus of HPN and enriching ingested metadata.

• Enables export of selected LoCloud HPN Thesaurus data sets.

GPLVUKF


Geo-coding application• End-user oriented application• allows institutions to add geographical locations to existing content in

a controlled crowd-sourcing environment and then • load the augmented data back into their original production

databases.

GPLAVINET


Vocabulary matching service

• Builds on work in the PATHS project (FP7)• Automatically assigns relevant concepts and terms to item

records • Retrieves terms from the Vocabulary serviceUPV/EHU Apache License 2.0


Background linking service• Based on work in PATHS• Follows the Named Entity

Disambiguation approach• Enriches items with links to

external contextual information in DBpedia and Wikipedia

Apache License 2.0UPV/EHU


Wikimedia application• A web service that uses a REST interface to communicate with

Wikimedia commons • The main functionalities are harvesting content form

Wikimedia, parsing the harvested content, and identifying useful entities that can be mapped to the ESE or EDM metadata schemas.

• The mapped ESE / EDM records are then sent to MORe to be delivered to Europeana.

Wikimedia Commons LicensingAthena RC


The LoCloudAggregation Structure

EnrichmentServices

Euro

pean

a Cl

oud

Testlab


About: Microservice

Wikipedia: https://en.wikipedia.org/wiki/Microservices (2015-09-15)

[...] microservices is a • software architecture style in which • complex applications are composed

of small, independent processes • communicating with each other using language-

agnostic APIs.• These services are small, highly decoupled and

focus on doing a small task,facilitating a modular approach to system-building.


Evolution of Architecturespwc: Technology Forecast: Rethinking Integration, Issue 1, 2014, page 3

Vocabulary Microservice



http://vocabulary.locloud.eu/• Vocabulary Microservices • This is the entry page for the

LoCloud vocabulary application. • LoCloud uses a variety of standard vocabularies

during the metadata enrichment process.These vocabularies have been established with the TemaTres tool and are available in the LoCloud cloud testlab.

• In addition, it is possible to feed the tool with your own vocabularies, establish new vocabularies or add languages to existing vocabularies.


Vocabulary WebService20+ Functions, API

Parameters

Example


Example: SearchUNESCO-Thesaurus

http://test113.ait.co.at/tematres/unesco/services.php?task=search&arg=peace


SPARQL-ExampleDISMARC-Genres

• Query: "Find all Broader Term for 'Acid Jazz‘“• SPARQL End Point: http://test113.ait.co.at/tematres/vocab/sparql.php

QueryResult

ThesaurusEntry


Service Call („fetchTopTerms“)embedded in a test application

Micro- Service

Vocabulary Web Service


Test Application DevelopmentDesigner View

Development of a Drop Down List

Development of a Drop Down List


The Test Application:Part of Data Entry

Path to the Thesaurus Item

„Candidate Term“ Crowdsourcing activity

Next Slide(Vocabulary Management)

Microservice used in the OpenUp! Project (open-up.eu)

„Common Name Service“



OpenUp! - Natural HistoryAggregator for Europeana

Micro Service:Common Name

Service

Micro Service:Common Name

Service


OpenUp!Transformation Workflow

Invoke Micro Service:

Common Name Service

Next Slide


OpenUp!Transformation Parameters

Micro Service:Common Name Service

Service End Point (Cache)


Result: OpenUp!Enriched Meta Data Set

Latin Name

Common Names

Digitisation Workflow(Books)



Bibliotheca Alexandrinahttp://www.bibalex.org

Points for MicroService Integration


Digital Curation(Long Term Preservation)


Digital CurationState-of-the-Art Examples (2010)

• Archivematica– Linux VM based toolset allows producers and archivists

to implement a small scale workflow

• UC3 / CDL Merritt– Micro-Services specifications – implement building blocks for the Digital Curation

Lifecycle


ArchivmaticaMicro Services

Linux

Toolsuite

Micro-

Services

Categories

of Micro-

Services


California Digital LibraryUC3 Merritt

• California Digital Library UC Curation Center• Independant, Interoperable Micro-Services• Complex curation function through combination of

atomistic services

UC3 Model

Knowledge Enhancement

Phases

Micro-

Services

Annotation


Micro-Services(Merritt vs. Archivematica)

CDL/UC3-MERRITT ARCHIVEMATICA

Micro-Services

for Pre-Ingest


High-Level Workflow

• OAIS activities providea sound basis

• Archivematica Use Cases

• Merritt specifications• Micro-Services are the

building blocks

Producer

OAIS Activity

Archivist

Micro-Services


CDLCuration Micro Services

1) „An Emergent Micro-Services Approach to Digital Curation Infrastructure“, IPRES 2009https://escholarship.org/uc/item/5313h6k9

2) UC Curation Center / California Digital Library, merritt: An Emergent Micro-services Approach to Digital Curation Infrastructure; Rev. 0.6, 2010-03-25

1) Annotation service 2)

The Annotation service provides a mechanism by which the consumers of managed digital content can enrich that content through additional description, association, or related content. The service supports the following methods:

• Annotate

• Submit.


Contact

Walter Koch, [email protected]

http://www.ait.co.at

locloud micro services and the digitisation workflow

Internet