usability and human factors unit 5a usability evaluation methods

Usability and Human Factors

Unit 5aUsability Evaluation Methods

Overview

• Usability

• Evaluation methods

• Interviews

• Surveys / questionnaires

• Usability inspection methods– Heuristic evaluation– Cognitive walkthrough

• Usability testing

Component 15/Unit 5aHealth IT Workforce Curriculum

Version 1.0/Fall 20102

Usability



Why Usability Matters: The Clinical Story



Why Usability Matters: The Story from Patients and Consumers



Usability Evaluation Methods



Usability Evaluation



Interviews:Four Main Types



Interviews:Four Main Types (cont.)



Phases of the Interview



Usability Interview Example



Usability Interview # 15 (Upstate 4)I: So I am going to ask you a few questions, if I may. I

may start by asking you your age?P: 68I: Have you lived in this area all your life?P: YesI: The greater Pulaski area, Syracuse region?P: I have been here 15 years, cross the road 30

years, and was born about 4 miles country side.I: Pretty close. What do you do?P: I worked as a toolmaker; I am retired now.I: What was your highest level of education?P: I got some college; I went to night school for a

while,I: Was this your first computer?P: YesI: Do you monitor your blood sugar daily?P: Yes.I: Prior to receiving this computer, did you have the

home monitoring kit?P: Yes. I have had monitor ever since I have had it.I: So you pretty well been monitoring it?P: Yes, for the last 8 years.

Profile: 68 years of age Male English Speaker Originally from Pulaski area Retired toolmaker Works part-time at saw sharpening shop Some college education System was his first computer, later

bought another one Lives out on the route, greater Pulaski

area Lives with wife Nice, comfortable house Has exercise machine in house Avid hunter Computer in bedroom-easy access, but

no chair. Keeps devices disconnected, keeps

them in a different room. Keeps diabetes diary of results and

values

Focus Groups• Commonly used method in marketing and social

science research• Typically 3 to 10 participants representative of the

sample population– For example: attending physicians, residents, nurses,

pharmacists & administrators• Discussion led by trained facilitator• Preset agenda to guide discussion• Sufficient flexibility to pursue emergent issues• Facilitator guides and prompts discussion

– Encourages less talkative participants to speak up– Discourages more verbose participants from dominating



Questionnaire / Surveys

• Common method for collecting user demographic data and soliciting opinions

• Can reach a rather large group of people

• Questions are typically closed, though open comments may be solicited

• Some use a mix of negative and positive questions– Require respondent to be more attentive



Rating Scales: Likert

• Mostly commonly used• Measure opinions, attitudes and beliefs• Widely used for evaluating user satisfaction• Statement representing a range of opinions

– I found the system to be very easy to use

• Rate response on a continuum from one extreme (e.g., highest or strongest agreement) to another (lowest or strongest disagreement)



Sample Usability Likert Questions



Rating Scales: Semantic Differential



Online Surveys:Two Types

Component 15/Unit 5a 17Health IT Workforce Curriculum


Survey Monkey• http://www.surveymonkey

.com• Illustration of online

survey• Focuses on user

experience• Different forms of

interaction– Check boxes for multiple

selection– Radio buttons for choosing

among alternatives or rating items on Likert scale

• Configurable and flexible



http://www.surveymonkey.com/

http://www.surveymonkey.com/

Usability Inspection Methods:Expert Reviews



Heuristic Evaluation• Developed by Jakob Nielsen and Rolf Molich• Method for structuring the critique of a system using

simple & general heuristics• Several analysts critique a system to identify &

diagnose usability problems• Display based• Each evaluator assesses the system & notes

violations of usability principles• In a second pass, analyst rates severity of problems• Can be used in all phases of design including

storyboards, prototypes and fully functioning systems



Usability Principles (Nielsen, 1993)

• Visibility of system status• Match between system and the real world• User control and freedom• Consistency and standards• Help users recognize, diagnose, and recover

from errors• Minimize memory load

– Emphasize recognition rather than recall• Flexibility and efficiency• Motivation and engagement



More Specific Heuristics (Gerhardt-Powals, 1996)

• Automate unwanted workload: – Free cognitive resources for high-level tasks – Eliminate mental calculations, estimations,

comparisons, & thinking

• Reduce uncertainty; display data in a clear and obvious manner

• Reduce cognitive load by bringing together lower level data into a higher-level summation

• Present new information with meaningful aids to interpretation: – Use a familiar framework, everyday terms, metaphors



More Specific Heuristics (cont.)

• Use names that are conceptually related to function – Context-dependent– Attempt to improve recall and recognition

• Group data consistently meaningful ways to decrease search time

• Limit data-driven tasks: – Reduce time assimilating raw data, appropriate

use of color & graphics• Include only information needed by the user

at a given time



Heuristic EvaluationProblem Description, Recommended Solution & Commentary Heuristics Violated

P. Description is clear, key points should be more visually prominent.S. Use bullets or bolded Text

Consistency Standards

P. Picture is extraneous and eats up valuable space. S. Summary positioned above the picture.

Minimalism

P. An excess of textS. Convey points more succinctly, use bullets and eliminate sentences

Minimalism

P. User is not fully aware of scope of session and perhaps programS. Perhaps an advance organizer and a timeline would be helpful

Visibility

P. Can’t return to session from Discussion by clicking back arrow. Control/Flexibility

P. Change Program is a misleading label for editing preferences. Match

P. Help is not sufficiently informative and context sensitive.S. Useful if system provides some context-sensitive guidance.

Help

P. Can’t return to English after switching to Spanish Reversible action/Control



Problem Severity



usability and human factors unit 5a usability evaluation methods

Documents