how much do availability studies increase full text success?

How Much do Availability Studies Increase Full Text Success?Sanjeet MannArts and Electronic Resources Librarian, University of RedlandsSCELC Research Day, March 4, 2014http://www.slideshare.net/sanjeetmann

Availability Studies

Defined as a research method replicating the steps that library users take from a citation to full text (or an error)

Electronic Resource Infrastructure

Proxy Sourc

eKB

Resolver

Target ILLIAD

InnovativeWAM

RILM (Proquest)

Proquest 360 Core

Proquest 360 Link

Oxford Journals

OCLC ILLIAD

Example

Questions You Can Answer Can users get to the full text? How often do they get errors? What kind of

errors? Should you be satisfied with your

knowledge base (KB) vendor? Do you have enough full text in your

collection? How often do your users need ILL? Are you teaching users what they need to

know to successfully access e-resources?

Availability Studies

4 searches x 10 disciplines x 10 results = 400 item sample

(estimates overall population with 98% confidence, +/- 5% error)

Datasets

2012: http://goo.gl/606us

2013: http://goo.gl/O5XK9A

Results

2012 20130%

5%

10%

15%

20%

25%

30%

35%

40%

45%41% 41%

38%

13%

Local AvailError

Z test results

Value % before % after Z score Result

Error rate 37.5 13.5 7.79 SIGNIFICANT!

Local availability

40.8 40.5 0.09 Not significant

Troubleshooting Proxy Errors

2012 2013 Z score Result

1- Proxy errors

1.3% 0% 2.25 Significant

Proxy

Source

KB

Resolver

Target

ILLIAD

Troubleshooting Source Errors

2012 2013 Z score Result

2 – Source errors

8.5% 8.3% 0.09 Not significant

Proxy

Source

KB

Resolver

Target

ILLIAD

Troubleshooting KB errors

2012 2013 Z score Result

3 – kb errors 6.3% 4.0% 1.44 Not significant

Proxy

Source

KB

Resolver

Target

ILLIAD

Troubleshooting Resolver Errors

2012 2013 Z score Result

4 – Resolver errors

1.8% 0.0% 2.70 Significant

Proxy

Source

KB

Resolver

Target

ILLIAD

Troubleshooting Target Errors

2012 2013 Z score Result

5 – Target errors

3.8% 1.3% 2.27 Significant

Proxy

Source

KB

Resolver

Target

ILLIAD

Troubleshooting ILLIAD Errors

2012 2013 Z score Result

6 – ILLIAD errors

16.0% 0.0% 8.34 SIGNIFICANT!

Proxy

Source

KB

Resolver

Target

ILLIAD

Item Types in the Samples

articles 75%

books 6%

chapters 8%

dissertations, 9%proceedings, 2%

Error Rate by Discipline

MUS ENG PHIL PSYC HIST ECON SOC BIO CDISMATH0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

20122013

Only 2 of 5 Items at the Library

2012 20130%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Online 36% Online 37%

Print 5% Print 4%

ILL 59% ILL 60%

% o

f to

tal sam

ple

Discussion

Availability studies can help you make a statistically significant reduction in errors

Supports evidence based librarianship Advocate w/ vendors to improve metadata

chain More full text resources? More ILL requests? Should first years avoid A&I databases? Do discovery services help or hurt?

Next Steps

Conduct availability studies at other libraries

Availability study with Redlands students

What are your e-resource experiences?

Further Reading

Kanji, Gopal K. 2006. 100 Statistical Tests. 3rd ed. Thousand Oaks: Sage Publications.

Mann, Sanjeet. 2013. “Measuring Electronic Resource Availability” presented at the SCELC Research Day, March 5, Loyola Marymount University. http://www.slideshare.net/sanjeetmann.

Nisonger, Thomas E. 2007. “A Review and Analysis of Library Availability Studies.” Library Resources & Technical Services 51 (1): 30–49.

Trainor, Cindi, and Jason Price. 2010. Rethinking Library Linking: Breathing New Life into OpenURL. Vol. 46. Library Technology Reports 7. Chicago: American Library Association.

Appendix – Error examples

Proxy error (domain not in forward table)

Appendix – Error examples

Source error (both rft.jtitle and rft.title)

Appendix – Error examples KB error (“Get Article” link is missing

because SerSol collection doesn’t support article level linking)

Appendix – Error examples

Resolver error (match on wrong journal title)

Appendix – Error examples

Target error (Page unavailable)

Appendix – Error examples

ILLIAD error (book chapter treated as an article; rft.atitle used for two fields)