Usage & UsabilityDenise A. Troll

Distinguished Fellow, Digital Library Federation

Associate University Librarian, Carnegie Mellon

June 16, 2001 – LRRT, ALA

Digital Library Federation

Initiative on usage, usability, & user support http://www.clir.org/diglib/use/useframe.htm

Distinguished FellowshipConvene library directors to outline research into how & why libraries are changing

Conduct a usage & usability survey to discover DLF library issues & practices

Library Trends

Traditional

Inputs & Outputs

Digital

Inputs & Outputs

What other changes are occurring in libraries?

What do we know about the context?

What’s really happening & why?

How & Why Are Libraries Changing?

Motivational white paper http://www.clir.org/diglib/use/whitepaper.htm

Library Director Meeting Read & comment on the white paper

Submit the 5 most important reasons for tracking trends, the audiences, & the indicators

Expected consensus on 5-6 new measures

Outsell to Survey Academics

Purpose of surveyStudy use of internal & external information

Examine relevant environmental factors

Identify gaps

Provide trend data

Random sample > 3000

Conduct survey fall 2001

Raise $$

Usage & Usability Survey

Contacted Participated Rate

Institutions 26 24 92%

Individuals 83 71 86%

What data do DLF sites gather?

What do they do with the data?

What works well? What doesn’t work well?

Creating a culture of assessmentCreating a culture of assessment

Data Gathered

Traditional & digital INPUTS

Traditional & digital OUTPUTS

Traditional & digital OUTCOMESUser satisfaction ratings

Service quality gaps

Usability

Referral URLs

What about

cost-effectiveness &

learning & research

outcomes?

Data Struggles

Gathering the right data

Gathering comparable data

Interpreting the data

Compiling & managing the data

Figuring out how to use the data

How do we do this

cost-effectively?

Reference Ref Quick Ref Real Ref Direct Tech Refer

Ref Desk In person

Ref Desk Telephone

Librarian Desk Telephone

Email

Chat

Other

What data to gather?

For what purpose?

How often?

At what cost?

Compare with what?

What about

user demographics?

What about

user privacy?

Can chat transcripts, email, & FAQ

be used to create a reference database?

What about

turn-around time?

Should we

operate both print

& e-reserves?

How do you define cost-effectiveness?

E-Reserves

Costs increaseTechnology

Staff

Staff training

Disk space

Revenues decreasePhotocopying

Fines

Implications of Assessments

Reorganizing, recruiting, & creating positions Usability & Interface Specialist

Director of Organizational Effectiveness

Distance Education Department

Training staff to gather, use, & present data effectively

Stretching the budget

Creative Use of Web Statistics

Based on # page hitsPut links to high-use pages high in the hierarchy

Keep high-use pages complete & up-to-date

Put links to low-use pages low in the hierarchy

Publicize or reduce maintenance of low-use pages

Frequent queriesPut links to queried pages high in the hierarchy

Revise site vocabulary

Creative Use of Web Statistics

Use by domain, location, or user group Facilitates collection & service development

Indicates user satisfaction

Helps target publicity

Referral URLs indicate satisfaction & impact

Do publicity or redesign increase use?

Low use times are for system maintenance

Survey Questionnaires

Why conduct? – To assess usage patterns; shifts in attitude; user needs, expectations, & satisfaction; service quality; market niche; & collection strengths, weaknesses, & relevance to the curriculum

How apply the results? – Inform budget requests & investments; justify expenditures; reallocate resources; set priorities; target training & instruction

Focus Groups

Why conduct? – To understand what users do or want to do; assess the use, effectiveness, & usefulness of collections & services; verify the results of previous research; inform new research

How apply the results? – Confirm hypotheses, assumptions, & user needs; plan library renovations, training, instruction, publicity; inform collection development & access decisions; inform or validate interface design plans; reorganize the library

Think-Aloud Protocols

Why conduct? – To identify problems in interface design, functionality, navigation, & vocabulary

How apply the results? – Inform redesign of the library web site & digital collections; inform customization of the OPAC interface; verify improvements in service quality & interface design; develop library instruction; revise metadata schemes

Card sorting exercises – To group web site links, & label links & groups

Heuristic evaluations – To identify problems in interface design, functionality, & navigation

Paper prototypes & scenarios – (like protocols)

Discount Usability Research

Web Site Vocabulary

Serious Issues

Lack of standards, guidelines, benchmarks, comparable data, & articulated proficiencies

Recruiting representative human subjects

Bias & bureaucracy

Redundancy

Interpretation

Application