Measuring Value:A Survey for Assessing Our Impact

Victoria H. Goode, MLISWelch Medical LibraryClinical Informationist

vgoode1@jhmi.edu

About the Survey

• Purpose: – To better understand the effect of information and

informationist services on clinical and research decisions.– To understand the value of selected library services.

• Population:– Faculty, fellows, housestaff & residents in the Johns

Hopkins schools of medicine, nursing and public health.

• Dates: – First two weeks in November 2011

About the Survey• Used the Critical Incident Technique• Consisted of 22 questions and a rating scale of 12

items. • Distributed via email and an intranet site.• Coincided with other campus wide

events/campaigns, and this may have limited participation.

• Partners: Johns Hopkins Marketing and Communications and the Johns Hopkins Biostatistics Center.

Response Rate• Response rate: 10.8%

– (809 completed surveys out of a possible 7,490)

72%

27%

1%

School

SOM SPH SON

58%29%

13%

Status

FacultyFellowsHousestaff/residents

Information Needs

75%

13%

10%

2%

N=765

ResearchEducationPatient CareManagement & Other

Benefits to Research

Textbook

Conference output

Citation impact

Grant proposals

Publication output

11%

17%

33%

36%

63%

Top Reported BenefitsN=552

Patient Care

Handled the situation differently

Choice of drugs

Choice of test

Choice of other treatments

Diagnosis

Advice given to patient or family

11%

30%

31%

39%

42%

47%

Top Reported Positive ChangesN=74

Patient Care

Adverse drug reaction or interaction

Medication error

Additional tests or procedures

Misdiagnosis

17%

30%

32%

45%

Top Reported Avoided Events N=47

The Value of Library Services

Top Valued Services Scale

Making a wide range of journal articles, books and databases available at your computer

9.3

Bringing together all needed information sources to one online location 8.7

Analyzing the results of a search and selecting most relevant articles 7.9

Identify an applicable information source to find what you need 7.7

Training to improve your searches 7.4

Library services were rated on a ten-point scale, with "1" meaning not valuable and "10" meaning very valuable

The Value of Informationist Services

Reducing workload burden

Providing added thoroughness

Helping to find additional information

Saving time

Providing expertise in available databases

Providing expertise in finding information

40%

59%

60%

70%

70%

73%

Top Reported BenefitsN=405

55% used the services of an informationist either recently or in the past. 91% of people who used informationist services would recommend it to others

Comments• Many of the improvements we made following

this survey came from the comments we received in open-ended questions.

• A question regarding barriers to finding information provided important examples of access problems.

• The informationist program received almost all positive results, but many people also commented on not knowing it existed.

What Improvements Were Made as a Result of This

Survey?

Access

• A large number of patrons access our resources from off-campus, so improving remote access to resources is vital.

• A more streamlined access issue reporting structure was put in place so that problems can be resolved faster and tracked over time to see trends.

Library Service Development

• Based on comments, there is a high need for information on data/statistics. – Review and further develop library services in

support of data. • We will continue to seek user input on new

library services and resources.• Additionally, we will conduct periodic user

reviews of key services.

Marketing and Communication

• We need to improve communication broadly with the community.

• New methods of communication include manning information tables throughout the campus and a monthly podcast.

• New and under-used resources are highlighted on the Welch homepage.

New Informationist Collaborations• Evidence-based Medicine Rounds were

implemented in August 2012 with the Junior Assistant Residents (JARs) in Internal Medicine.

• We improved our information resource pages and added new information portals, like the Bioinformatics portal.

• ICU pilot project to determine how providers answer questions that arise on an Intensive Care Unit.

Models for the Survey • Grefsheim, S. F., Whitmore, S. C., Rapp, B. A., Rankin, J. A., Robison, R. R., & Canto, C.

C. (2010). The informationist: Building evidence for an emerging health profession. Journal of the Medical Library Association, 98(2), 147-156.

• Grefsheim, S. F., & Rankin, J. A. (2007). Information needs and information seeking in a biomedical research setting: A study of scientists and science administrators. Journal of the Medical Library Association, 95(4), 426-434.

• Marshall, JG (Primary Investigator). Value of Library and Information Services in Patient Care Study. A partnership of the National Network of Libraries of Medicine, Middle Atlantic Region and the University of North Carolina at Chapel Hill. http://nnlm.gov/mar/about/value.html

• Association of College and Research Libraries. Value of Academic Libraries: A Comprehensive Research Review and Report. Researched by Megan Oakleaf. Chicago: Association of College and Research Libraries, 2010 www.acrl.org/value

Thank you

Welch Library Value Survey Team Contact Information:Jaime BlanckChangxin (Jack) ChenVictoria GoodeNancy RodererLori RosmanStella SealSue WoodsonRob Wright

Victoria GoodeWelch Medical Library

vgoode1@jhmi.edu