the quick & the dirty harker -kizhakkethil
DESCRIPTION
Given the decreasing budgets for collections at many libraries, librarians are looking at abstracting and indexing (A&I) resources and full-text databases with a more skeptic eye. In addition to traditional evaluation measures, such as costs, usage and faculty input, we looked at the overlap of indexing and/or full-text coverage. Those who have conducted such overlap studies have approached it at either the journal or article level. Article-level overlap studies demonstrate coverage of selected articles in the databases under study. Conversely, journal-level studies examine the extent of indexing of journals among the selected databases. Both methods are very time-consuming and require extensive resources. A simplification of the journal-level method is to compare lists of journals indexed. Two tools, Cufts and JISC's Academic Database Assessment Tool (ADAT) are very useful for this purpose, but do not include all databases. Many databases need to be manually collected. This presentation will describe the background to such a project, the specific tools and procedures used, how the results were used to address budget reductions, and the limitations of the results. Members of the audience will be able to consider using this method for evaluating their own abstract and index databases for budgetary purposes. Presenter: Karen Harker Collection Assessment Librarian, University of North TexasTRANSCRIPT
The Quick & the Dirty
The Good, the Bad & the Ugly of Database
Overlap at the Journal Title Level
ByKaren R. Harker
& Priya K.,University of North
Texas Libraries
Engineerin
gDB
Physics DB
Science DB
Overlap in Coverage is Natural
Web-scale discovery systems are
changing the information ecosystem Duplication of content is no longer a necessary evil
Materials budgets being squeezed Key Question:
What could we drop, if we had to?
What we wanted to know
Done the hard way Limited to Abstract & Indexes
Only
First Attempt, 2012
• Download Lists
• Import into Database
Coverage
• Paired Comparisons
• On ISSN
Match • Overlap Rate• Unique Titles
Report
Outcome of First Attempt
The Good
Confirm suspicions Some databases cut
Updated information Some additional info
coverage years scope of coverage
Provided foundation database
The Bad & the Ugly
Took a lot of time Some lists not easily
transportable Some lists without
ISSNs Limited to one-on-
one comparisons
Another round of budget cuts Looking at full-text aggregator
databases Gathering more data about resources Less time to devote Graduate Student Assistant help
Second Attempt, 2014
JISC Academic Database
Assessment Tool (ADAT) CUFTS Serials Solutions Overlap Analysis
Reinventing the Wheel?
Academic Database Assessment Tool (ADAT) Originally from JISC (UK) Now administered by Center for Research
Libraries (CRL) Lost some functionality
Selected resources Paired comparisons only No differentiation between indexing or full-text
JISC ADAT
What ADAT used to provide
Jacso, P. (2010). The JISC academic database assessment tool -- virtues and vices. Online Information Review, 34(5), 806-814.
What you get from ADAT now
Open Source Serials Management Simon Fraser University, BC,
Canada Public tools: Resource Comparison Broad selection of resources Differentiates between indexing
and full-text
CUFTS
Like CUFTS Specific to UNT Libraries’ holdings Compare one to any number of
resources Full-text only – no information on
indexing alone
Serials Solutions Overlap Analysis
Trying other sources
CUFTS JISC ADAT Serials
Solutions Overlap Analysis
Compare results with manual version
Good enough?
Wrangling the Data
Priya KizhakkethilGraduate Library Assistant
CUFTS Maintenance Tool – Resource
Comparison Serial Solutions – Overlap Analysis Center for Academic Resources, JISC
Collection – Academic Database Assessment Tool
Tools Used
CUFTS
Serials Solutions Overlap Analysis
JISC ADAT
Number of…
Unique Titles Titles Overlapped in Target
Database Unique Full-Text Titles Full-Text Titles Overlapped in Target
Database
Kind of data collected
Data derived
Indexed
Total number of titles indexed
% of Unique titles % of titles
Overlapped
Full-Text
Total number of full-text titles
% of Unique full text titles
% of full-text titles Overlapped
Using CUFTS
Results from CUFTS
# Duplicates from CUFTS
Unique Titles via CUFTS
# Unique from CUFTS
Recording Results
CUFTS for Full-Text
Recording Results
Results from JISC ADAT
Recording Results from JISC ADAT
Conclusions on Data Gathering
Quick
Easy to run Recently updated JISC summary table Serials Solutions
summary
Dirty
Data didn’t match Data changed over
time CUFTS
Sloooow Summary difficult to
find Data requires
tweaking
Reinventing the Wheel?
Is this better than doing it manually?
The Good, the Bad & the Ugly
JISC ADAT The Good: Simple table The Bad:
Limited number of databases 15 Bibliographic 9 Full-Text
Only paired-comparison No pretty pictures
The Ugly: Unable to download or export results
CUFTS The Good:
Extensive list of databases and versions.
Can download results. Can compare indexing & full-text
separately Can compare up to 4 databases
at once The Bad: Some resources not
available The Ugly: Had not been
updated at time of first attempt.
The Good, the Bad & the Ugly
Serials Solutions Overlap Analysis
The Good: Easy to run Nice summaries Can download results Compare any number
of resources
Serials Solutions Overlap Analysis
The Bad: Limited to full-text coverage only
The Pretty: Is BEST!
Comparison of Sources
The Good
Mostly consistent Within 10%
Most databases & aggregators covered About 15% not
available Some coverage
details Year Scope
The Bad
Duplication of titles Missing ISSNs Lack of coverage
details Year Scope
Most Data Fairly Consistent
Coyote Ugly
Wildly different denominators!
Different kinds of content.
The Good: Much faster The Bad: Not all databases
available The Ugly: Some data unreliable
or inconsistent.
Outcomes
Need All Four Wheels
ADAT CUFTS
SerSol
Manual
Hart, Alfred. Improvement in Wagon-Brakes., Patent, December 4, 1877; digital images, (http://texashistory.unt.edu/ark:/67531/metapth166725/ : accessed April 24, 2014), University of North Texas Libraries, The Portal to Texas History, http://texashistory.unt.edu ; crediting UNT Libraries Government Documents Department, Denton, Texas.