Do Real Archivists Use OAI?

Mid-Atlantic Regional Archives ConferenceGettysburg, PA

October 31, 2003

Chris Prom

Assistant University Archivist

University of Illinois at Urbana-Champaign

http://web.library.uiuc.edu/ahx/workpap/marac03.ppt

Goals

• Examine rationale for archivists to use OAI

• Describe my involvement in the University of Illinois Mellon OAI project

• Speculate on OAI’s present and future

• Suggest how archivists may wish to get involved with OAI

Should Archivists Use OAI?• OAI use growing, well-supported, (relatively)

simple• NSDL base format, NCSTRL project• Allows metasearches across projects and data

types (not content specific)• DL projects for archives, manuscripts, photos,

artifacts, objects use a variety of metadata standards

• Provides wider exposure for collections• University of Michigan OAIster service illustrates

both promise and problems

Why archivists should use OAI. . .

???

NOT a link to NAACP visual materials finding aid

Found! (. . . 20 mins later)5,000+ items! (11 online)

UIUC Mellon Project Goals• Test feasibility of harvesting, searching cultural

heritage with OAI• Develop data provider tools that produce usable OAI

records from disparate sources (MARC, EAD, databases, etc)

• Build open-source service provider tools• My role: EAD/OAI

– assess structural problems in mapping to OAI

– develop an effective crossmapping

– allow basic searching in an OAI environment

– test effectiveness of the search

– provide proof of concept

Our approach

• “Dumbing down” metadata has benefits

• Examined EAD encoding standards

• Generated multiple OAI records for one EAD– top-level record drawn from <eadheader> and <archdesc>

– mini records from <dsc>, with relation to top level

• Preserve context for “hits” by linking user to finding aid in the search/retrieval mechanism

• relations between levels in the finding aid rendered implicitly by our service provider, not within the OAI record

Our Data and Service Providers• OAI Repository (Data Provider)

– Snapshot of ~5,500 EAD finding aids mirrored on UIUC server

– responds to HTTP get and post with ASP scripts and XSLT stylesheet to provide requested data

– Records produced from EAD include XPointer– When EADs are split into their subordinate components,

Xpointers identify the individual parts• Service Provider

– Server-side scripts use the XPointers in DC identifier tag for rendering and linking

– Xpointer used to move user deep in finding aid, preserve context

XPointer

• WC3 Recommendation, March 2003: http://www.w3.org/XML/Linking

• Can identify XML fragments using a superset of the XPath syntax, e.g. xyz.xml#xpointer(//dsc[1]/c01[2]/c02[3]/c03[10])

• When EADs are split into their subordinate components, Xpointers identify the individual parts

• To make it work, both data providers and service providers would need to share a set of assumptions about HOW it should work

<rdf:RDF> <rdf:Description> <dc:identifier> http://…/…/test.xml#xpointer(//dsc[1]/c01[8]/c02[5]/c03[244]) </dc:identifier> <dc:title>Toensing, Richard</dc:title> <dc:type>text</dc:type> <dc:type>archives or manuscripts</dc:type> <dc:type>file</dc:type> <dcterms:isPartOf> <rdf:Description> <dc:identifier> http://…/…/test.xml#xpointer(//dsc[1]/c01[8]/c02[5]) </dc:identifier> <dc:title>Various Composers</dc:title> </rdf:Description> </dcterms:isPartOf> </rdf:Description></rdf:RDF>

Simple Searchhttp://oai.grainger.uiuc.edu/oai/search

Search Results

Full Record

Hit in Context of Finding Aid

What I learned• Using OAI can exposure your collections to new

audiences• Not good enough to harvest from image databases,

nor just give just top-level link to FA• But many users confused when stumble on a

finding aid; “where’s the beef”• An “archives friendly” service provider would

– . . . Serve as basis for cross-repository searches– . . . Reenginner the notion of what is a finding aid by

allowing access from the back end– . . . Undo some of the damage inflicted when archival

materials are treated as items– . . . need to reconstruct context using local mirrors of

the ead document

The Future

• OAI Currently better suited to representing digital objects than descriptive data

• Service providers currently limited in number, 16 listed on openarchives.org

• Will OAI take off?

• IMLS Digital Collections and Content Project: (http://imlsdcc.grainger.uiuc.edu/)

Data Provider Options

• If you want to provide dynamic (full) OAI metadata– Metadata items in Relational database

• Map fields or pages to DC record• Use webscripting language to respond to harvester requests

– XML, e.g. Finding aid (as a digital object)• map from <eadheader> and <archdesc>• use encodinganalog recommendations from

RLG Best Practices or pass entire EAD file with XSD schema– metadata for digital objects linked from EAD???

• use XSLT and XPointer to provide context and allow service providers to replicate hierarchical structure of EAD.

• Depends on SP’s ability to handle xpointer

Data Provider Options, cont.

• Static OAI metadata– Suitable for small collections which are not

growing– All metadata should be contained in a single

xml file on a webserver– Projects which will harvest metadata (e.g.

IMLS Digital Collections and Content) may be able to provide assistance in conversion.

Resources• OAI website:

– http://www.openarchives.org/

• UIUC Provider Tools– http://uilib-oai.sourceforge.net/– http://sourceforge.net/projects/uilib-ead/

• Library Hi Tech, Vol 21, No. 2, special issue on OAI

• Further implementation guidelines– http://dli.grainger.uiuc.edu/publications/twcole/jcdl-

oai/JCDL_2003_OAI_Intro.ppt

• This powerpoint:– http://web.library.uiuc.edu/ahx/workpap/marac03.ppt

Do Real Archivists Use OAI?

Mid-Atlantic Regional Archives ConferenceGettysburg, PA

October 31, 2003

Chris Prom

Assistant University Archivist

University of Illinois at Urbana-Champaign

http://web.library.uiuc.edu/ahx/workpap/marac03.ppt