copyright 1999 all rights reserved evaluating paper prototypes n how do you do this? –what are...

Copyright 1999 all rights reserved

Evaluating Paper PrototypesEvaluating Paper Prototypes

How do you do this?How do you do this?– What are the considerations?What are the considerations?– What are the steps?What are the steps?

?


Evaluating Paper PrototypesEvaluating Paper Prototypes

BeforeBefore– Preparing for the evaluationPreparing for the evaluation– Selecting tasks Selecting tasks

DuringDuring– Teaching the userTeaching the user– Recording what happensRecording what happens

AfterAfter– Using the resultsUsing the results


Preparing for the EvaluationPreparing for the Evaluation Create a presentable design Create a presentable design

representation that is legible and neatrepresentation that is legible and neat Select a representative userSelect a representative user Think through what you will do with the Think through what you will do with the

useruser– Define tasks for the userDefine tasks for the user– Decide on explanations & instructions Decide on explanations & instructions – Decide how you will observe & record what Decide how you will observe & record what

happenshappens


Selecting TasksSelecting Tasks

Select enough tasks to cover the Select enough tasks to cover the range of expected userange of expected use– Use the requirements to help Use the requirements to help

determine the rangedetermine the range– Go for variety first, Go for variety first,

comprehensiveness secondcomprehensiveness second

““Use the faucet to wash your hands.”Use the faucet to wash your hands.”


Selecting TasksSelecting Tasks

Make tasks specific but do not tell Make tasks specific but do not tell the user how to operate the the user how to operate the interfaceinterface– Describe the result not the steps to Describe the result not the steps to

get thereget thereRight:Right: “Assume the sink is full of “Assume the sink is full of bits of food from having rinsed the dishes. bits of food from having rinsed the dishes. Wash these bits down the drain.”Wash these bits down the drain.”

Wrong:Wrong: “Move the faucet to the left, right, “Move the faucet to the left, right, front and back while it is on power wash.”front and back while it is on power wash.”


Teaching the UserTeaching the User

Try to avoid teaching the userTry to avoid teaching the user– Better interfaces require less instructionBetter interfaces require less instruction

Reflect real use of the systemReflect real use of the system– Will the user have instructions?Will the user have instructions?– Will the user get training?Will the user get training?

Teaching changes the evaluationTeaching changes the evaluation– Bad teaching can ruin a good designBad teaching can ruin a good design– Good teaching can save a bad designGood teaching can save a bad design


Do NOT Teach the User if...Do NOT Teach the User if...

There will not be instructions for There will not be instructions for the user under real conditionsthe user under real conditions

The task is perceived as so easy The task is perceived as so easy that the user expects to do it that the user expects to do it without instructionwithout instruction

The user will not be using the The user will not be using the interface on any regular basisinterface on any regular basis


Do Teach the User if...Do Teach the User if...

There will be instructions for the There will be instructions for the user under real conditionsuser under real conditions

The interface provides new The interface provides new capabilities unfamiliar to the usercapabilities unfamiliar to the user

The interface has many different The interface has many different functions and capabilities functions and capabilities

Warning! If you use instructions, Warning! If you use instructions, write them beforehandwrite them beforehand


Observing and RecordingObserving and Recording

Know what you want to observeKnow what you want to observe– And be alert for the unexpectedAnd be alert for the unexpected

Do not rely on memoryDo not rely on memory– Be prepared to record what happensBe prepared to record what happens


TimeTime

Record the time to perform various Record the time to perform various parts of the taskparts of the task

MaterialsMaterials– Stopwatch Stopwatch – Chart to record the start time for each Chart to record the start time for each

subtasksubtask


ErrorsErrors

Categorize each error the user makesCategorize each error the user makes– Typing errorsTyping errors– Path errorPath error - additional steps or steps in - additional steps or steps in

an unexpected orderan unexpected order– Command errorCommand error - an incorrect action for - an incorrect action for

the task the user is trying to dothe task the user is trying to do– Motor errorMotor error - user selects wrong object - user selects wrong object

accidentallyaccidentally


Thinking Aloud about the TaskThinking Aloud about the Task Asked the user to talk while doing the taskAsked the user to talk while doing the task

– Have them tell what they are doing as they Have them tell what they are doing as they perform the taskperform the task

Have them read a set of instructions aloud Have them read a set of instructions aloud before they begin the taskbefore they begin the task– Helps to get them goingHelps to get them going– Can use the task description for thisCan use the task description for this

Remind them to talk if they stopRemind them to talk if they stop– But be low-key about itBut be low-key about it


User UnderstandingUser Understanding

Check user understand after she or Check user understand after she or he has done the taskshe has done the tasks– Give the user a short test on the Give the user a short test on the

interfaceinterface• Multiple choice test Multiple choice test

– Have the user describe to you how to Have the user describe to you how to do various tasks with the interfacedo various tasks with the interface


User’s PerceptionUser’s Perception

Don’t ask how they liked it overallDon’t ask how they liked it overall Ask specific questions about the Ask specific questions about the

interfaceinterfaceRight:Right: “ “Do you think the temperature is Do you think the temperature is easy to adjust?”easy to adjust?”

Wrong:Wrong: “Rate on a scale from one to five, “Rate on a scale from one to five, how easy you found the interface to use.”how easy you found the interface to use.”


Watch out for Response Bias!Watch out for Response Bias!

User’s want you to feel good about User’s want you to feel good about your designyour design– If you ask them if they like it, they will say If you ask them if they like it, they will say

YESYES– Ask them instead what parts they like & Ask them instead what parts they like &

dislikedislike Users want to be successfulUsers want to be successful

– Norman on tendency of people to see Norman on tendency of people to see problems as their fault problems as their fault


Using the ResultsUsing the Results

There is no value unless you apply There is no value unless you apply the result of the evaluationthe result of the evaluation

Use each type of observation for Use each type of observation for insight into the designinsight into the design

Listen to the user, but do not Listen to the user, but do not completely abandon your own completely abandon your own common sensecommon sense


TimeTime

Does it take too long to do the Does it take too long to do the task?task?

Reduce the number of stepsReduce the number of steps– Favor frequent tasksFavor frequent tasks– Simple tasks should be simpleSimple tasks should be simpleCombine temperature change andCombine temperature change and

turning the water “on”turning the water “on”

Build a one-step key for hand Build a one-step key for hand washing button and cold drinkswashing button and cold drinks


ErrorsErrors

Does your user make a lot of Does your user make a lot of errors?errors?

Avoid designs that require precise Avoid designs that require precise motor performancemotor performance– Avoid very fine scales on slidersAvoid very fine scales on sliders– Avoid multiple level cascading menusAvoid multiple level cascading menus– Avoid moving from one end of the screen to Avoid moving from one end of the screen to

the other and back, multiple timesthe other and back, multiple times


ConfusionConfusion

Is the user confused?Is the user confused?– Long pauses on user talking aloud?Long pauses on user talking aloud?– Subtask take longer than others?Subtask take longer than others?– Moving from one selection to another?Moving from one selection to another?– Vague explanation of the interface?Vague explanation of the interface?


ConfusionConfusion

Cures for confusionCures for confusion– Put sequential steps near each otherPut sequential steps near each other

• Put together selections that are likely to occur Put together selections that are likely to occur together together

– Change labels - make the mapping betterChange labels - make the mapping better • Add a faucet icon showing what shower looks likeAdd a faucet icon showing what shower looks like

– Modify the conceptual modelModify the conceptual model• Draw a picture of a sink for specifying the faucet Draw a picture of a sink for specifying the faucet

directionsdirections


Total ConfusionTotal Confusion

Is the user lost?Is the user lost?– User cannot complete the taskUser cannot complete the task– User succeeds only with great difficult User succeeds only with great difficult

or a lot of helpor a lot of help


Total ConfusionTotal Confusion

Select a new conceptual modelSelect a new conceptual model– Ask the user what she or he thinks is Ask the user what she or he thinks is

the model for the interfacethe model for the interface– Ask the user to ignore the interface Ask the user to ignore the interface

you have designed and describe how you have designed and describe how they might perform the tasks you they might perform the tasks you have specifiedhave specified

copyright 1999 all rights reserved evaluating paper prototypes n how do you do this? –what are...

Documents