jcdl 2011: semantically augmented annotations in digitized map collections
DESCRIPTION
Historic maps are valuable scholarly resources that record information often retained by no other written source. With the YUMA Map Annotation Tool we want to facilitate collaborative annotation for scholars studying historic maps, and allow for semantic augmentation of annotations with structured, contextually relevant information retrieved from Linked Open Data sources. We believe that the integration of Web resource linkage into the scholarly annotation process is not only relevant for collaborative research, but can also be exploited to improve search and retrieval. In this paper, we introduce the COMPASS Experiment, an ongoing crowdsourcing eff ort in which we are collecting data that can serve as a basis for evaluating our assumption. We discuss the scope and setup of the experiment framework and report on lessons learned from the data collected so far.TRANSCRIPT
Semantically Augmented Annotations in Digitized Map CollectionsRainer SimonAustrian Institute of Technology
Bernhard HaslhoferCornell University
Werner Robitza, Elaheh MomeniUniversity of Vienna
ACM/IEEE Joint Conference on Digital Libraries | June 13-17, 2011, Ottawa, Canada
Part of EuropeanaConnect EU ProjectCore Components for the European Digital Library
www.europeanaconnect.eu
2EuropeanaConnect is coordinated by the Austrian National Library
Co-funded by the European Community Programme eContentplus
EuropeanaConnect | Annotation & Scholarship
3EuropeanaConnect is coordinated by the Austrian National Library
Co-funded by the European Community Programme eContentplus
“Annotation” is a fundamental scholarly practice common across disciplines
Scholars share and exchange knowledge through annotations
Scholars manage, interpret and coordinate sources throughannotations
by Romana Klee CC BY-SA 2.0
4
HistoryArt
Technology
Geography
YUMA Map | YUMA Universal Media Annotator
5
Viewing of high-resolution map scans as zoomable images Free-text annotation Reply functionality & RSS feeds (on items, annotations, users)
to support collaboration Map geo-referencing
Data overlay – modern-day country borders, coastlines, etc. Annotation export to KML – viewing in Google Earth Place search
Semantic tagging – linking free text with contextual Linked Data resources! Named entity recognition & link discovery via DBpedia Spotlight1
Geographical features in the annotated area via Geonames Semi-automatic approach: human-verified through the Context Tag Cloud
EuropeanaConnect is coordinated by the Austrian National Library
Co-funded by the European Community Programme eContentplus
1 http://dbpedia.org/spotlight, http://wiki.dbpedia.org/spotlight/knownuses
YUMA Map | Annotation & Semantic Tagging (Screencast)
6EuropeanaConnect is coordinated by the Austrian National Library
Co-funded by the European Community Programme eContentplus
http://vimeo.com/21798530
The Assumption...
7
“A map retrieval system that indexes annotations with user-verified links to Linked Data resources can be more effective with regard to retrieval than systems that index only metadata or purely textual annotations.”
EuropeanaConnect is coordinated by the Austrian National Library
Co-funded by the European Community Programme eContentplus
The COMPASS1 Crowdsourcing Experiment
8EuropeanaConnect is coordinated by the Austrian National Library
Co-funded by the European Community Programme eContentplus
1 Collection Of MaPs, Annotations & Semantic LinkS
Test data kindly provided by the Library of Congress 130,935 user search queries collected over two years 6,306 high-resolution digitized map images Descriptive metadata Pooling to reduce amount of queries for Step 1
Step 1: Users are shown a map/query pairRelevant or Not Relevant?
Step 2: Users are invited to annotate maps with YUMA Map
Step 3: precision and recall analysis for different retrieval approachesMetadata only vs. Annotated (plain text) vs. Annotated (Linked Data context)
COMPASS | Screenshot
9EuropeanaConnect is coordinated by the Austrian National Library
Co-funded by the European Community Programme eContentplus
Work in Progress... | Preliminary Results
10EuropeanaConnect is coordinated by the Austrian National Library
Co-funded by the European Community Programme eContentplus
90+ Users from at least 12 countries
~60% of all judgments produced by the top 10 contributors
~25% “Lurkers” – submitted no judgement at all
1/3 of active participants declared themselves experts: General library domain (~15%) Map library domain (~15%) Fields related to the scope of COMPASS, e.g. GIS, cartography (~4%)
Level of involvement higher among experts: 6 experts among top ten contributors ~59% of all submitted judgments
Work in Progress... | Next Steps
11EuropeanaConnect is coordinated by the Austrian National Library
Co-funded by the European Community Programme eContentplus
Finish COMPASS Step 1 45% of required amount for a 400-query “ground truth” reached
Start COMPASS Step 2 – Integration COMPASS/YUMA Development of a “COMPASS Portal” currently in progress Lower the entry barrier! Simplify tasks, provide “guided tour” mode Raise motivation: social interactions, rewards (game mechanics), rating &
reputation scores (gaining authority in the community) Enable users to “do something” with the content: e.g. embedding of
annotated maps
In general: encourage feedback & investigate users’ motivations
Dissemination!
Thank you for your attentionQuestions!
Map SourcesLibrary of Congress – Discovery and Exploration Maps
Wikimedia Commons
http://dme.ait.ac.at/annotation (take the tour)
http://github.com/yuma-annotation (get the code)
http://compass.cs.univie.ac.at (help us evaluate!)
12EuropeanaConnect is coordinated by the Austrian National Library
Co-funded by the European Community Programme eContentplus