1

CS 544 Discount Usability Engineering

Heuristic Evaluation

Acknowledgement: Some of the material in these lectures is based on material prepared for similar courses by Saul Greenberg (University of Calgary), Ravin Balakrishnan (University of Toronto), James Landay (University of California at Berkeley), monica schraefel (University of Toronto), and Colin Ware (University of New Hampshire). Used with the permission of the respective original authors.

2

Usability Inspection is the generic name for a set of methods based on

having evaluators inspect or examine usability-related aspects of a user interface

usability inspectors can be usability specialists, software development consultants with special expertise, end users with task/content knowledge, or other professionals

4 ways to evaluate an interface:– automatically (compute usability measures using evaluation

software)– empirically (usability testing with real users)– formally (using models to calculate usability measures)– informally (rule of thumb and general knowledge of evaluators)

usability inspection is an informal evaluation

3

Discount Usability Engineering

Developed by Nielson, based on using four techniques:

User and task observation Scenarios Simplified thinking aloud Heuristic evaluation

4

Articulate:•who users are•their key tasks

User and task descriptions

Goals:

Methods:

Products:

Brainstorm designs

Task centered system design

Participatory design

User-centered design

Evaluate

Psychology of everyday things

User involvement

Representation & metaphors

low fidelity prototyping methods

Throw-away paper prototypes

Participatory interaction

Task scenario walk-through

Refined designs

Graphical screen design

Interface guidelines

Style guides

high fidelity prototyping methods

Testable prototypes

Usability testing

Heuristic evaluation

Completed designs

Alpha/beta systems or complete specification

Field testing

Interface Design and Usability Engineering

5

Discount usability engineering

Cheap– no special labs or equipment needed– the more careful you are, the better it gets

Fast– on order of 1 day to apply– standard usability testing may take a week

Easy to use– can be taught in 2-4 hours

6

Heuristic Evaluation Developed by Jakob Nielsen Principles used to “evaluate” a system for usability problems Becoming very popular

user involvement not required catches many design flaws

Small set (3-5) of evaluators examine UI– independently check for compliance with usability principles

(“heuristics”)– different evaluators will find different problems– evaluators only communicate afterwards

findings are then aggregated Can perform on working UI or on sketches

7

Why multiple evaluators?

Every evaluator doesn’t find every problem

Good evaluators find both easy & hard ones

8

Heuristic Evaluation Process

Evaluators go through UI several times– inspect various dialogue elements– compare with list of usability principles– consider other principles/results that come to mind

Usability principles– Nielsen’s “heuristics”– supplementary list of category-specific heuristics

competitive analysis & user testing of existing products

Use violations to redesign/fix problems

9

Neilson’s Heuristics (original)

H1-1: Simple & natural dialog H1-2: Speak the users’ language H1-3: Minimize users’ memory load H1-4: Consistency H1-5: Feedback H1-6: Clearly marked exits H1-7: Shortcuts H1-8: Precise & constructive error messages H1-9: Prevent errors H1-10: Help and documentation

10

H2-1: Visibility of system status

keep users informed about what is going on

– example: pay attention to response time 0.1 sec: no special indicators needed, why? 1.0 sec: user tends to lose track of data 10 sec: max. duration if user to stay focused on action for longer delays, use percent-done progress bars

searching database for matches

11

H2-1: Visibility of system status

keep users informed about what is going on– Appropriate visible feedback

What did I select?

What mode am I in now?

How is the system

interpreting my actions?

12

H2-2: Match between system & real world

speak the users’ language follow real world conventions

(bad) example: Mac desktop– Dragging disk to trash

should delete it, not eject it!

13

H2-2: Match between system & real world

speak the users’ language follow real world conventions

e.g. withdrawing money from a bank machine

14

H2-3: User control & freedom “exits” for mistaken choices, undo, redo don’t force down fixed paths

How do I get

out of this?

15

H2-3: User control & freedom “exits” for mistaken choices, undo, redo don’t force down fixed paths

Strategies:– Cancel button (for dialogs waiting for user input)– Universal Undo (can get back to previous state)– Interrupt (especially for lengthy operations)– Quit (for leaving the program at any time) – Defaults (for restoring a property sheet)

16

H2-4: Consistency & standards Consistency of effects

– same words, commands, actions will always have the same effect in equivalent situations

predictability

Consistency of language and graphics– same info/controls in same location on all screens/dialog boxes

– forms follow boiler plate– same visual appearance across the system (e.g. widgets)

e.g. different scroll bars in a single window system!

Consistency of input– consistent syntax across complete system

17

H2-4: Consistency & standards

These are labels with a raised appearance.

Is it any surprise that people try and click on them?

18

H2-5: Error prevention try to make errors impossible modern widgets: only “legal commands” selected, or

“legal data” entered

Provide reasonableness checks on input data– on entering order for office supplies

5000 pencils is an unusually large order. Do you really want to order that many?

19

H2-6: Recognition rather than recall Computers good at remembering things, people aren’t! Promote recognition over recall

– menus, icons, choice dialog boxes vs command lines, field formats– relies on visibility of objects to the user (but less is more!)

20

H2-7: Flexibility and efficiency of use Experienced users should be able to perform

frequently used operations quickly Strategies:

– keyboard and mouse accelerators abbreviations command completion menu shortcuts function keys double clicking vs menu selection

– type-ahead (entering input before the system is ready for it) – navigation jumps

e.g., going to window/location directly, and avoiding intermediate nodes

– history systems WWW: ~60% of pages are revisits

21

H2-7: Flexibility and efficiency of useKeyboard

accelerators for menus

Customizable toolbars andpalettes for

frequent actions

Split menu, with recently used fonts on top

Scrolling controls for page-sized increments

Double-click raises object-specific menu

Double-click raises toolbar

dialog box

22

H2-8: Aesthetic and minimalist design

No irrelevant information in dialogues Bad example:

23

H2-9: Help users recognize, diagnose, and recover from errors

error messages in plain language precisely indicate the problem constructively suggest a solution

24

Errors we make

Mistakes– arise from conscious deliberations that lead to an

error instead of the correct solution

Slips– unconscious behaviour that gets misdirected en

route to satisfying goal e.g. drive to store, end up in the office

– shows up frequently in skilled behaviour usually due to inattention

– often arises from similarities of actions

25

Types of slips

Capture error– frequently done activity takes charge instead of one intended

occurs when common and rarer actions have same initial sequence– change clothes for dinner and find oneself in bed (William James, 1890)– confirm saving of a file when you don’t want to delete it

I can’t believe I pressed Yes...

26

Types of slips

Description error– intended action has much in common with others that are possible

usually occurs when right and wrong objects physically near each other– pour juice into bowl instead of glass – go jogging, come home, throw sweaty shirt in toilet instead of laundry basket– move file to trash instead of to folder

Loss of activation– forgetting what the goal is while undergoing the sequence of actions

start going to room and forget why you are going there navigating menus/dialogs and can’t remember what you are looking for but continue action to remember (or go back to beginning)!

Mode errors– people do actions in one mode thinking they are in another

refer to file that’s in a different directory look for commands / menu options that are not relevant

27

Designing for slips

General rules– Prevent slips before they occur– Detect and correct slips when they do occur– User correction through feedback and undo

Examples– mode errors

have as few modes as possible (preferably none) make modes highly visible

– capture errors instead of confirmation, make actions undoable allows reconsideration of action by user

– e.g. Mac trash can can be opened and “deleted” file taken back out– loss of activation

if system knows goal, make it explicit if not, allow person to see path taken

– description errors in icon-based interfaces, make sure icons are not too similar, check for reasonable input, etc.

28

Generic system responses for errors

General idea: Forcing functions– prevent / mitigate continuation of wrongful action

Gag– deals with errors by preventing the user from continuing

eg cannot get past login screen until correct password entered

Warn– warn people that an unusual situation is occurring– when overused, becomes an irritant

e.g., – audible bell – alert box

Do nothing– illegal action just doesn’t do anything– user must infer what happened

enter letter into a numeric-only field (key clicks ignored) put a file icon on top of another file icon (returns it to original position)

29

Generic system responses for errors

Self-correct– system guesses legal action and does it instead– but leads to a problem of trust

spelling corrector Lets talk about it

– system initiates dialog with user to come up with solution to the problem compile error brings up offending line in source code

Teach me– system asks user what the action was supposed to have meant– action then becomes a legal one

30

H2-10: Help and documentation

Help is not a replacement for bad design!

Simple systems:– walk up and use; minimal instructions

Most other systems:– feature rich– some users will want to become “experts” rather than “casual” users– intermediate users need reminding, plus a learning path

Many users do not read manuals– prefer to spend their time pursuing their task

Usually used when users are in some kind of panic, need immediate help– indicates need for online documentation, good search/lookup tools– online help can be specific to current context– paper manuals unavailable in many businesses!

e.g. single copy locked away in system administrator’s office Sometimes used for quick reference

– syntax of actions, possibilities...– list of shortcuts ...

31

Types of help

Tutorial and/or getting started manuals– short guides that people are likely to read when first obtaining their

systems encourages exploration and getting to know the system tries to get conceptual material across and essential syntax

– on-line “tours”, exercises, and demos demonstrates very basic principles through working examples

Reference manuals– used mostly for detailed lookup by experts

rarely introduces concepts thematically arranged

– on-line hypertext search / find table of contents index cross-index

32

Types of help (cont.)

Reminders– short reference cards

expert user who just wants to check facts novice who wants to get overview of system’s capabilities

– keyboard templates shortcuts/syntactic meanings of keys; recognition vs.

recall; capabilities

– tooltips text over graphical items indicates their meaning or

purpose

33

Types of help (cont.)

Context-sensitive help– system provides help on the interface component

the user is currently working with Macintosh “balloon help” Microsoft “What’s this” help

– brief help explaining whatever the user is pointing at on the screen

Wizards– walks user through typical tasks– but dangerous if user gets stuck– Remember the MS “paperclip”?

34

How to perform evaluation At least two passes for each evaluator

– first to get feel for flow and scope of system– second to focus on specific elements

If system is walk-up-and-use or evaluators are domain experts, no assistance needed

– otherwise might supply evaluators with scenarios

Each evaluator produces list of problems– explain why with reference to heuristic or other information– be specific and list each problem separately

35

How to perform evaluation (cont.)

Why separate listings for each violation?– risk of repeating problematic aspect– may not be possible to fix all problems

Where problems may be found– single location in UI– two or more locations that need to be compared– problem with overall structure of UI– something that is missing

hard w/ paper prototypes so work extra hard on those note: sometimes features are implied by design docs and

just haven’t been “implemented” – relax on those

36

Severity ratings

Used to allocate resources to fix problems Estimates of need for more usability efforts Combination of

– frequency– impact– persistence (one time or repeating)

Should be calculated after all evals. are in Should be done independently by all judges

37

Severity ratings (cont).

0 - don’t agree that this is a usability problem

1 - cosmetic problem

2 - minor usability problem

3 - major usability problem; important to fix

4 - usability catastrophe; imperative to fix

38

Summary Advantages

– the “minimalist” approach a few general guidelines can correct for the majority of usability problems easily remembered, easily applied with modest effort

– discount usability engineering cheap and fast way to inspect a system can be done by usability experts, double experts, and end users

Problems:– principles are more or less at the motherhood level

can’t be treated as a simple checklist subtleties involved in their use

– doesn’t necessarily predict user/customers overall satisfaction

– may not have the same “credibility” as user test data (one solution to this is to include the design team and developers in the usability evaluation, as in a “pluralistic walkthrough”)

39

Summary

Research result:– between 4-5 evaluators appear to be sufficient in most cases to

identify 80% of total usability problems– user testing and usability inspection have a large degree of non-

overlap in the usability problems they find (i.e., it pays to use both methods)

Cost-benefit:– usability engineering activities are often difficult to justify and

carry out in a timely way, but many activities can be done quickly and cheaply, and produce useful results

– usability inspection turns less on what is “correct” than on what can be done within development constraints

– ultimate trade-off may be between doing no usability assessment and doing some kind

40

You now know 10 Heuristics for evaluating designs:

– H2-1: Visibility of system status – H2-2: Match between system & real world– H2-3: User control & freedom – H2-4: Consistency & standards– H2-5: Error prevention– H2-6: Recognition rather than recall– H2-7: Flexibility and efficiency of use– H2-8: Aesthetic and minimalist design– H2-9: Help users recognize, diagnose, and recover from errors – H2-10: Help and documentation

Principles can be used to systematically inspect the interface for usability problems

41

Readings and References BGBG 80-86 (Evaluating Systems and Their User Interfaces)

BGBG 170-181 (Usability Inspection Methods)

Optional: Neilsen, J. (1993). Usability Engineering. Read Chapter 5:

Usability Heuristics, 115 – 163.