How do students really do research? Revelations from the Research Confession Booth

Odile Harter, Harvard UniversityLisa Junghahn, University of California, IrvineEmily Singley, Harvard University

How do students really do research?findings from the

Research Confession Booth

Odile Harter, Harvard UniversityLisa Junghahn, UC Irvine

Emily Singley, Harvard University

Harvard Library Search & Discovery Initiative Education Subgroup, 2013-present

1. Background1.1. Provide the scientific background, rationale for the study, and importance in

adding to existing knowledge.

The initial goal of the study is to obtain a better understanding of how patrons use library resources and what they most value about the resources we do have. This information will be used to inform the design and implementation of the Library’s new discovery platform. As the Library keeps pace with a constantly changing research environment, the study will continue to investigate how researchers interact with and feel about library products and services.

Harvard Library has access to several measures of patron behavior, including user surveys, focus groups, and usability studies, but none of these offer sufficiently granular or contextualized information for us to understand the whole picture of what our users want. This study will collect information that is granular, qualitative, and not tied to a particular website or library vendor. It will also provide a recognizable site for library patrons to quickly and easily give very specific feedback about their research experiences.

Study protocol

Ingredients:

1-3 well-traveled campus locations with ambient noise

10-20 candy and granola bars 1 poster 20 info sheets 1 laptop 1 wireless mouse

1 cardboard box 1 table & chair 1 Quicktime screen recording 1 Qualtrics survey 1 task USB drives 1 IRB approval

Study protocol

TASK 1

“Please show us a feature you really like in a resource you use to do research.”

Study protocol

TASK 2“Please walk us through a situation in which you were trying to find information to use in your coursework and encountered some kind of obstacle or snag that interfered with your ability to find, obtain, or use the information you were seeking.”

Study protocol

Status

Undergraduate 15

Graduate 3

Faculty 1

Experience (self-identified)

Novice 7

Intermediate 11

Expert 1

Study protocol

Participants:

Pros and cons:

Flexible Nimble Qualitative Semi-controlled/slightly artificial Small dataset

Study protocol

What we’re learning:

Global picture – which systems are actually used

Navigation behaviors Contextualized usage - how and why

systems are used

Findings

Category Resource Use

Harvard library

HOLLIS catalog 10

Harvard Library website

1

Databases A-Z list 1

Fulfillment servicesScan & Deliver 1

Borrow Direct 1

Interlibrary Loan 1

Open web Google Scholar 5

Google 5

Wikipedia 1

Databases PubMed 4

Academic Search Premier 1

JSTOR 1

Findings

Navigation:

Typing in memorized URL Googling site name Navigating via Harvard Library

homepage or intranet

Findings

Features identified as important:

Confidence & comfort Filtering Getting relevant results:

o topicalo level of detailo academic results

Full-text fulfillment Citation management options

“This button right here is the best:”

Findings

Library as difficult, confusing, annoying:

Findings

“I sometimes feel like I don't know how to use the right ‘ands’ and ‘ors’ and quotation marks."

"I really don’t like "getting it" from interlibrary loan - I feel like that's really annoying. So I actually usually just try to find it on my own.”

“The resource I use most would probably be JSTOR, because I’ve known that since high school (…) I’m not the best at research.”

“See, this is the point where I usually give up and just go ask the research librarian for help, you know, because I don’t know how to do this stuff.”

Preference for open web:

Quick fulfillment Familiar interface Trust in how it’s interpreting your query Greater tolerance for uncertainty Ability to match effort invested to information need

Findings

User interface problems:

Invisible or incomprehensible button labels Complicated layouts

Findings

Example of user unable to find PubMed full-text link

Silo-to-silo confusion:

What resource am I in?

Findings

Other tasks we might try with this study Further analysis of the findings Companion studies Effects on our daily work

Future research

Thank you!Questions?