primo usability evaluation cody hanson, technology librarian university of minnesota libraries...

Primo Usability Primo Usability EvaluationEvaluationCody Hanson, Technology LibrarianCody Hanson, Technology LibrarianUniversity of Minnesota LibrariesUniversity of Minnesota [email protected]@umn.edu

Primo Usability Primo Usability EvaluationEvaluation

PrimoPrimo

Usability testingUsability testing

ScenariosScenarios

IssuesIssues

OutcomesOutcomes

PrimoPrimo

• PrimoPrimo• Usability testingUsability testing• ScenariosScenarios• IssuesIssues• OutcomesOutcomes

Development Development PartnershipPartnership

Aleph/Primo pipeAleph/Primo pipe

Cost of ownership assessmentCost of ownership assessment

Usability testingUsability testing

OIT Usability LabOIT Usability Lab

Eye trackingEye tracking

Staff rolesStaff roles

Usability TestingUsability Testing

RolesRoles

NarratorNarrator

LoggerLogger

HelpdeskHelpdesk

Eye-tracker observerEye-tracker observer

Usability TestingUsability TestingScenarios and tasksScenarios and tasks

DebriefingDebriefing

QualitativeQualitative

Key deliverables:Key deliverables:

Issues listIssues list

Eye-tracking dataEye-tracking data

Key deliverableKey deliverablehttp://www.flickr.com/photos/stofiska/3021577770/

Key deliverableKey deliverablehttp://www.flickr.com/photos/setr/3330196387/

Key deliverableKey deliverablehttp://www.flickr.com/photos/joelogon/2098848889/

GoalsGoalsUsers can find a variety of library Users can find a variety of library materials on their topicmaterials on their topic

Users can find a particular item, its Users can find a particular item, its availability, or its call numberavailability, or its call number

Users understand the process of Users understand the process of broadening and narrowing broadening and narrowing searchessearches

GoalsGoals

Determine whether current MNCAT Determine whether current MNCAT users see Primo as an users see Primo as an improvementimprovement

Determine if Amazon.com users Determine if Amazon.com users can easily transfer their knowledge can easily transfer their knowledge to use Primoto use Primo

ScenariosScenarios

You are taking an intro You are taking an intro course in marketing.course in marketing.

Find some articles on marketing Find some articles on marketing and advertisingand advertising

Determine whether there are more Determine whether there are more results for your query and how you results for your query and how you could see themcould see them

Find some books about Find some books about Minneapolis.Minneapolis.

Save two or three of the books you Save two or three of the books you find for future reference.find for future reference.

Make sure these items are saved Make sure these items are saved so you can come back to them so you can come back to them next week.next week.

You’d like to read You’d like to read Edmund Hillary’s Edmund Hillary’s High High AdventureAdventure..

Find the book.Find the book.

Add a tag to the record for this Add a tag to the record for this book that would help describe it.book that would help describe it.

Your classmate wrote a Your classmate wrote a review for [insert book review for [insert book here].here].

Find the book.Find the book.

Find the review and say out loud what Find the review and say out loud what rating your classmate gave the book.rating your classmate gave the book.

Add your own review and rating.Add your own review and rating.

Find this article using Find this article using this citation.this citation.

Find the online version of the Find the online version of the article and read the first article and read the first paragraph.paragraph.

IssuesIssues

Search box tabs not Search box tabs not used/seenused/seen

““Several evaluators did not use or Several evaluators did not use or see the... tabs above the search box. see the... tabs above the search box. When the tabs were not used, it lead When the tabs were not used, it lead to searching that was less likely to to searching that was less likely to find the desired results.”find the desired results.”


““Several evaluators did not use or Several evaluators did not use or see the... tabs above the search box. see the... tabs above the search box. When the tabs were not used, it lead When the tabs were not used, it lead to searching that was less likely to to searching that was less likely to find the desired results.”find the desired results.”

Advanced searches Advanced searches unsuccessfulunsuccessful““Use of the Advanced Search was Use of the Advanced Search was unsuccessful for some of the unsuccessful for some of the evaluators... One evaluator evaluators... One evaluator expressed confusion about why the expressed confusion about why the narrow search wouldn’t find the narrow search wouldn’t find the result, but the basic search would.”result, but the basic search would.”

Unclear when newly Unclear when newly tagged items will tagged items will appearappear

““Evaluators expected that their Evaluators expected that their tagged items would appear in the list tagged items would appear in the list of ‘all tagged results’ immediately...”of ‘all tagged results’ immediately...”

Article search progress Article search progress bar caused bar caused confusion/frustrationconfusion/frustration““This wait caused some evaluators This wait caused some evaluators to consider stopping the search or to consider stopping the search or changing their search terms to get a changing their search terms to get a search to complete faster.”search to complete faster.”

Key deliverableKey deliverablehttp://www.flickr.com/photos/28445735@N00/2397240784/

Positive commentsPositive comments

““Several evaluators said that they Several evaluators said that they liked the tabs and new design.”liked the tabs and new design.”

““Some of the evaluators mentioned Some of the evaluators mentioned that they thought the tabs that they thought the tabs ‘presented everything you need.’”‘presented everything you need.’”

““Some evaluators said that Primo is Some evaluators said that Primo is ‘easy to use.’”‘easy to use.’”

DesirabilityDesirability


90% positive90% positive

6 of 8 evaluators chose 6 of 8 evaluators chose “Organized”“Organized”

5 of 8 chose “Clean”5 of 8 chose “Clean”

5 of 8 chose “Straightforward”5 of 8 chose “Straightforward”

http://www.flickr.com/photos/cutsie/3614949532/

Notes from our users...

“I hate it...

I need to

look things

up by call

#...” Staff

memberhttp://www.flickr.com/photos/cutsie/3614949532/

“sucks”

Visitorhttp://www.flickr.com/photos/cutsie/3614949532/

“I think the

format is terrible

and the google-

esque linking ads

[sic] nothing... I

call it MNCat

double minus.”

Staff memberhttp://www.flickr.com/photos/cutsie/3614949532/

“Good”

Staff

memberhttp://www.flickr.com/photos/cutsie/3614949532/

“BOO-

OOOOOO!

!!!!!!” Undergrad

uatehttp://www.flickr.com/photos/cutsie/3614949532/

“It is pathetic...

Is the library

computer staff

run by

MORONS?

I expect a

prompt reply.” Professor

_______http://www.flickr.com/photos/cutsie/3614949532/


CleanClean

OrganizedOrganized

StraightforwardStraightforward

CleanClean

CleanClean

CleanClean

CleanClean

OrganizedOrganized

OrganizedOrganized

OrganizedOrganized

OrganizedOrganized

OrganizedOrganized





EfficientEfficient

EfficientEfficient

ImprovedImproved

ImprovedImproved

ModernModern

TrustworthyTrustworthyEasyEasy

EasyEasy FastFast

FastFast

FastFast

AttractiveAttractive

FamiliarFamiliar

FamiliarFamiliar

PowerfulPowerful

ConsistentConsistent

HelpfulHelpful

HelpfulHelpful SophisticatedSophisticated

GoalsGoalsUsers can find a variety of library Users can find a variety of library materials on their topicmaterials on their topic

Users can find a particular item, its Users can find a particular item, its availability, or its call numberavailability, or its call number

Users understand the process of Users understand the process of broadening and narrowing broadening and narrowing searchessearches

GoalsGoals

Determine whether current MNCAT Determine whether current MNCAT users see Primo as an users see Primo as an improvementimprovement

Determine if Amazon.com users Determine if Amazon.com users can easily transfer their knowledge can easily transfer their knowledge to use Primoto use Primo

OutcomesOutcomes

ContextContext

ConfidenceConfidence

Clear action items for vendorClear action items for vendor

Better relationship with vendorBetter relationship with vendor

Primo Usability Primo Usability EvaluationEvaluationCody Hanson, Technology LibrarianCody Hanson, Technology LibrarianUniversity of Minnesota LibrariesUniversity of Minnesota [email protected]@umn.edu

primo usability evaluation cody hanson, technology librarian university of minnesota libraries...

Documents

usedseensearch box tabs

issuessearch box tabs

basic search

narrow search wouldnt

desired results

usedseenseveral evaluators

tagged items

current mncat users