APragmaticApproachtoUnderstandingandUtilizingEvents

inCulturalHeritage

LoraAroyo,Chiel vandenAkker,Marnix vanBerchum,LodewijkPetram,GerardKuys,Tommaso Caselli,Jacco vanOssenbruggen,

VictordeBoer,SabrinaSauer,BerberHagedoorn

• LoraAroyo• Marnix vanBerchum• VictordeBoer• WillemRobertvanHage• LizzyJongma• GerardKuys

• SusanLegene• AnneliesvanNispen• Jacco vanOssenbruggen

• Lodewijk Petram• Piek Vossen

Inthispanel…

Challenges relatedto– Eventasanobjectofstudy– Eventasutilityforcontextandinterpretation

Fromdifferentperspectives:– History/DigitalHumanities– ComputationalLinguistics– CulturalHeritageorganizations– ComputerScience

onlineaccesstomuseumcollections

theinterpretationofobjectsinonlinemuseumcollectionsissupportedbyenrichingthemuseumcollectionmeta-datawithastructurednotionofhistoricalevents

WhereistheEventintheMuseum?

§ biographyofartifacts(provenance)§ artifactswereusedin eventsortheydepictevents

§ toaskforthehistoricalmeaningofanartifactistoaskaquestionwhichcanbeansweredonlyinthecontextofastory

Twoconceptionsofdigitalhumanities

Slide: Guus Schreiber

DHBenelux2017,Utrecht5July2017

LodewijkPetram,MarnixvanBerchum

Historicalevents:twocases

2

1) personal attributes or events?

• what information is inherent to a person, what can be modelled differently?

Inherent data elements Events

Names Occupation

Date and place of birth Education

Date and place of death Honorific and academic titles

Gender

3

• why did people move?

• explanatory data:

• micro (life events)• meso (city or region)• macro (national)

2) 18th century migration in the maritime sector

Micro-level events

Micro-level events

• Pieter Dy: vader overlijdt in 1816 (op Texel). (In hieropvolgende periode verhuist Dy naar Den Helder)

• Foppe Fopkes Baas: 1785: eerste huwelijk op Ameland1792: kind geboren in Den Helder1800: trouwt opnieuw (is weduwnaar) op Ameland1813: dochter overlijdt in Den Helder1818: zoon Dirk trouwt1829: op 18 maart overlijdt vrouw1829: op 21 maart overlijdt Foppe zelf

Meso-level events

Nieuwe Nederlandsche Jaerboeken

Macro-level events

Clio-Infra

An Events Model for Archives

Should we model all of the world all over

time?

Gerard Kuys, July, 5, 2017

2 Gerard Kuys, July, 5, 2017

Peeling the onion: from source to interpretation

Archief-stukArchief-

stukArchief-stukArchival

document

Access on archival content

reference

reference

reference

reference

Archives’ context

Archive

anExternalDatatypeanEntity

Basic structure

3

Interpretation 1

Interpretation 2

Interpretation 3

Interpretation 4

Interpretation 5

Digital representationof source(presumably 1:1 to source content)

Real-worldsource

authority

Gerard Kuys, July, 5, 2017

Set of events may differ depending on interpretation

4

Interpretation 1

Interpretation 2

Interpretation 3

Interpretation 4

Interpretation 5

Digital representationof source(presumably 1:1 to source content)

Real-worldsource

authority

Gerard Kuys, July, 5, 2017

What kind of subjects in archives?

5

* Commissions and assignments, personnel registries

* Civil and Military Courts, captivity in times of war

* Domanial rights (requests), permits

* Taxes and Excises, payments

* Scope statements in building projects

* Motion picture content rating

* Meeting minutes

* ……

Gerard Kuys, July, 5, 2017

What kind of events with such subjects?

6

* Career event

* Prosecution event, being made prisoner in times of war

* Getting a license

* Getting paid, getting fined

* Completion events in building activities

* Movies getting admitted, getting prohibited

* Decisions made, options rejected

* ……

Gerard Kuys, July, 5, 2017

Events: What kind of data?

• Commissieboeken in Raad van State archive (1581 – 1795)

• Central Commission Film rating archieve(1928-1960)

• Cabinet ministers (cabinets 1848 – 2010), archival guide

• Japanese Internment cards in Dutch East Indies (1941-1945)

7 Gerard Kuys, July, 5, 2017

8

Gebeurtenis

Scope Frequentie algemeen

Frequentie in life cycle

Micro Meso Macro

Benoeming door RvS

Personen 1 - n Enkele malen

Afhankelijk van functie

Vervanging door RvS

Personen Bij overlijden of vertrek voorganger

Enkele malen

Afhankelijk van functie

Aanvraag keuring

Films, bedrijven

Eenmalig Eenmalig X Afh. van belang film

-

Film gekeurd

Films, bedrijven

In principe eenmalig

In principe eenmalig

X Afh. van belang film

-

Film herkeurd

Films, openbaarheid

Eenmalig Eenmalig X Afh. van belang film

-

Kabinet treedt aan

Parlement Max. 1x in de vier jaar

Eenmalig - X X

Kabinet treedt af

Parlement Max. 1x in de vier jaar

Eenmalig - X X

Bewindspers. treedt aan

Parlement, kabinet

0 - n 0 - n - X X

Bewindsper Parlement, 0 - n 0 - n - X X

Gerard Kuys, July, 5, 2017

9

• Events on the level of personsPersonal life (birth, marriage, death)Professional lifeLivelihood (dearth, famine, acquisition of wealth)Prosecution and captivity

• Events in the level of things getting done or undone

• Societal events• Natural disasters

Events: Provisional Classification

Gerard Kuys, July, 5, 2017

Look For The StoryTommaso Caselli

t.caselli@{gmail.com;vu.nl}

Utrecht, July 05th 2017

• “Things” that happen or hold as true (cit. every definition)

• Eventualities (Bach, 1986 - Algebra of Events)

• Ontological primitives

• Events = dynamic

• States = static

Set the Ground

Look Around

• Eventualities are hubs of information:

• limited, if considered in isolation

• rich (and joyful), when connected together

• Relations between/among events essential to make sense of the world and interpret the context

Start Asking: “Why? ”• Events in context give rise to stories

• different granularities

• New task: StoryLine Extraction

• semantic: general patterns of events (A Revolution)

• episodic: specific patterns of events (The French Revolution)

EventPanelDHBenelux2017JaccovanOssenbruggen,CWI

Worldof(overly)complexhumanitiesresearchquestions:• Nomodels,everythingisopentointerpretation,...

Worldof(overly)simplifieddigitaltools:• Modelsaretrivialrepresentationsofobjects(“bagofwords”),...

Worldof(imperfect)compromises,80/20rule• Eventsasimperfectbutgoodstartingpoint

↓Simplifiedwhowhatwherewhenquestions

↑Place/personNER,temporalexpressiondetection,etc

PERSON

Eventsareperspectives…Example:

PLACE TIME

“PrinceBernhardwasinWageningenonMay5th,1945”

Event:END OF WWII

Worldof“subjective”perspectives,explicitpersonalframeofthescholar

Worldof“objective”glorifiedwordcounting

Worldofformallymodelledeventsthatlookdeceptivelyobjective• Butarealwaysfromaspecificperspective(who’s?)• Nowoftenimplicitbutshouldbeexplicitlymodelled

Takehomemessage• Eventshavethepotentialtobecomethe20%ofthemetadataweneedtoaddress80%ofcurrentinteroperabilityproblems

• Theymaylook likelow-level“technical”metadata…• …butarealwaysencodingsomeperspective• Event-detectionandenrichmenttoolsneedtomaketheseperspectivesexplicit

DIVE+ INTO THE EVENT-ENRICHED LINKED OPEN CULTURAL HERITAGE

Support Event-centric browsing for Media Scholars

AccesstoIntegratedOnlineMultimedia collectionsusingLinkedOpenData tointegratemetadataofvariousheritagecollections

InteractiveExploration&DiscoveryinContextlinkingobjectstoevents andentitiesbuildingautomaticstorylines (narratives)

DIVE+

OPENIMAGES.EU3,220newsbroadcastsNetherlandsInstituteforSound&VisionGTAAthesaurus

DELPHER.NL197,199Scansof Radiobulletins1937– 1984

AMSTERDAMMUSEUM 73,447culturalheritageobjectsAMThesaurus

TROPENMUSEUM78,270culturalheritageobjectsSVNCthesaurus

CollectionsandVocabularies

sem:Event

sem:Actordive:MediaObjectdive:depictedBy

rdfs:labeldive:sourcedive:placeholderdc:identifierdc:descriptionetc.

oa:Annotation

oa:hasBodyoa:hasTarget

sem:Place

sem:Time

skos:Concept

sem:hasActor, sem:hasPlacesem:hasTimedive:isRelatedTo

skos:broader, skos:narrower etc.

dive:isRelatedTo

DIVE+datamodel

OriginalMetadata

Interpretationofcontent

NamedEntityRecognition

Humancomputation

Hybridpipeline

Where do we get events from

- LIDO,CIDOC,EDM

- creationDateStart- - Interpretationofobject

- FROG,otherpipelines

- - Crowdsourcing- -Nichesourcing,

OriginalMetadata

am:Belgische opstandam:besnijdenisam:Beurs de Keyseram:bevrijdingam:bezoekerscentrumam:bibliothekenam:Bijlmerrampam:Boulevard of Broken Dreamsam:brandam:brand van het oude stadhuis op de Damam:burgeroorlogam:capitulatieam:christendom geboorte van Christusam:christendom kruisigingam:christendom opstanding van Christusam:christus aan het kruisam:Christus schrijft op de grondam:concert "Fayence bord”

CrowdsourcingforEvents inTexts & Videos

CrowdTruth.org

Description Event"FotoisgenomentijdensdeTweedeZuidNieuw-GuineaExpeditie"

Tweede ZuidNieuw-GuineaExpeditie

FotoisgenomentijdensdeEersteZuidNieuw-GuineaExpeditie

Eerste ZuidNieuw-GuineaExpeditie

"FotoisgenomentijdensdeEerste- ofdeTweedeZuidNieuw-GuineaExpeditie"

Tweede ZuidNieuw-GuineaExpeditie

"Maskergedragentijdensoogstfeesten.HetfeestinkwestieishetSokari speldateenmaalperjaarwordtopgevoerdgedurendezevenopeenvolgendenachtennaNieuwjaar,medioapril.…” Nieuwjaar

FROGNEREventextraction

Victor Kramer

Radionewsbulletins:Everyobject1event

DIVE+Enrichments

Enrichment method

Media Objects Actors Places Events Other Alignments

OI hybrid pipeline 3,204 1,249 1,412 1,916 185,846 623

NB Interpreted + NER 197,200 194,890 54,571 197,200 6,736 6,353

AM original thesaurus 73,447 66,966 5,973 148 28,047 6,865

TMoriginal thesaurus + FROG NER 78,226 27,829 3,896 23* 13,269 -

Total 352,077 290,934 65,852 199,264 233,898 -

*) more to come

DIVE:MediaObjectNieuws uit Indonesië:opheffing van het KNIL

dive:depictedBy

sem:hasTimestamp

sem:EventANP:1950-08-11:50

dive:isRelatedTodive:relatedPlacesem:hasPlace

dive:isRelatedTodive:relatedActorsem:hasActor

dive:isRelatedTodive:relatedPlacesem:hasPlace

sem:Time25 Juli 1950

dive:depictedBy

sem:hasTimestamp

DIVE:MediaObjectMannen bij het huis van Paul Spies

aan de Parapattan 42, Djakarta

dive:depictedBy

dive:depictedBy

dive:depictedBy

DIVE:MediaObjectANP:1950-08-11:50

DIVE:MediaObjectSchaal

sem:Time11 Augustus 1950

sem:Eventontbindingsceremonie

sem:PlaceDjakarta

sem:PlaceIndonesië

sem:ActorMohammad Hatta

Linkingviacommonactors,places,objects

(Common)Eventthesaurus?

Februaristaking

WOII

Februaristaking

“De oproep 'Staakt!' voor deelname aan de februaristaking te Amsterdam op 25 en 26 februari 1941. “

stakingen

Eduard Hellendoorn

"Joseph Eijl Eduard HellendoornHermanus Coenradi 13 maart 1941 gefusilleerd Waalsdorpervlakte"

Waalsdorpervlakte

Jessie Both & Didi de hooge

DIVE+ UI:INFINITYOFEXPLORATION

/ Support exploration and serendipity // Visual inspection of media objects and entities /

/ Lets user build, save and share Proto-Narratives/

explore event

event related entities

/ Events as backbone for for deep exploration of interconnected media collections by DH experts

/ Various data enrichment strategies to construct explorable knowledge graphs

DIVE+

/ http://diveproject.beeldengeluid.nl / http://diveplus.beeldengeluid.nl

/ v.de.boer@vu.nl

DIVE+

DIVE+

GETTING THE BIGGER PICTURE: AN EVALUATION OF MEDIA EXPLORATORY SEARCH AND NARRATIVE CREATION

PANEL: PRAGMATIC APPROACH TO UNDERSTANDING AND UTILISING EVENTS IN CULTURALHERITAGE

BERBER HAGEDOORN & SABRINA SAUER (RUG)

¡ Historians are storytellers

¡ Study of media texts/representations i.r.t. media makers (industrial actors)

¡ But: media scholars are storytellers too

'DISRUPTIVE' MEDIA EVENT NARRATIVES

¡ Media (news) event: an event with a specific narrative that gives the event its meaning, and is in contemporary societies increasingly recognized as non-planned or 'disruptive' as well as cross-medial.

¡ Focus on the construction of narratives and power of narrativization in different (news) media, e.g. television, radio, newspapers…

¡ See e.g. Jiménez-Martínez 2016; Rothenbuhler 2010; Katz & Liebes 2007; Dayan & Katz 1992

¡ For instance stories of empathy, fear and change in relation to international media events

EXAMPLE: 1968 & THE NETHERLANDS IN THE1960S

'The times they are a changing'

DIVE EVENT-BASED BROWSER OF LINKED HISTORICAL MEDIA

Research Phases1.Select disruptive media events about conflict & breakingnews2. Construct event narratives using new DIVE+ features3. Organize focus groups with media researchers4. Realize incremental innovations to DIVE+ features5. Test implemented changes with same focus groups MEDIA RESEARCHERS

USER CENTERED APPROACH: CO-CREATION

Mapping search and storytelling practices

¡ Focus groups & interviews;

¡ Tasks & talk aloud protocols , surveys/questionnaires;

¡ Research diaries

Iterative approach

DATA LEVEL• Enrichment of

annotation, links, entities, narratives about events

INTERFACE LEVEL• Recommendations

for automatic suggestions of starting points for search & exploration

USER LEVEL• Relation between

exploratory search and & narrative creation

• In-depth insights into search process of media researchers

INSIGHTS of NARRATIVIZING DISRUPTION

KEY QUESTIONS

1. How can we interpret 'disruptive' media events? How are 'disruptive' media eventsconstructed as narratives across media and instilled with specific cultural or politicalmeanings?

¡ NB: from a cross-media perspective; and from the perspective of media researchers as storytellers

2. DH tools can generate new research questions: how can new digital humanities toolssupport such AV research? (e.g. exploratory search via event-centric linked data digital collectionbrowser DIVE+)

3. How can we assess the purpose and usefulness of narratives for scholarly research(specifically, search and learning), and why are narratives important for media scholars intheir research process, from beginning to end?

¡ CLARIAH Research Pilot Narrativizing Disruption: How exploratory search can support media researchers to interpret ‘disruptive’ media events as lucid narratives

¡ https://www.clariah.nl/en/projects/research-pilots/granted-pilot-research-projects/nardis/610-nardis (May 2017 - May 2018);