part 1-intro; part 2- req; part 3- design chapter 20 why evaluate the usability of user interface...
TRANSCRIPT
Part 1-Intro; Part 2- Req; Part 3- Design Chapter 20
Why evaluate the usability ofuser interface designs?
Chapter 21Deciding on what you need toevaluate: the strategy
Chapter 22 Planning who, what, when, and where
Part 4 of your book – Evaluation
2
Why Evaluate the Usability of User Interface Designs?
Jakob Nielsen's set of heuristics
Does the Interface Meet the Usability Requirements?◦ Effective
◦ Efficient
◦ Engaging
◦ Error tolerant
◦ Easy to learn
Exploring Other Concerns in Evaluations◦ Why users are unable to
complete tasks easily.
◦ Is the UI developed for all levels of users?
◦ Are all design features acceptable to users?
Visibility of system status Match between system and
the real world User control and freedom Consistency and standards Error prevention Recognition rather than
recall Flexibility and efficiency of
use Aesthetic and minimalist
design Help users recognize,
diagnose, and recover from errors
Help and documentation
3
Expert Review Cognitive Walkthrough Usability evaluation
◦ Participant◦ Observer (Logger)◦ Facilitator (Briefer/debriefer)◦ Evaluator◦ Evaluation Data
Raw Data Qualified Data Quantified Data
Terminology
4
◦Our Running Example: Global Warming The Users for S103
Users’ Tasks and the Global Warming UI CD-ROM based
The Domain for the Global Warming UI Paper-based and CD-ROM materials
The Environment for the Global Warming UI Home-study, or work
◦Description of the Global Warming User Interface
Users, Tasks, Environment, and Domain
5
6
Ex. 20.2 Consider this screen for a Usability Evaluation. How do you start?
Usability concerns◦ evidence
of visibility, affordance, and feedback
True Users Task
Scenarios with setting
7
8
Consider the use of metaphor below. Are they appropriate for target users? Explain.
9
◦The Process of Usability Evaluation Is Iterative
◦Techniques for Usability Evaluations
User Observations
Inspections of the User Interface
Conform to usability standards?
Other Evaluation Techniques
Variations of user observation or inspection
The Activities of Usability Evaluations
10
11
◦ Welcome participant, ask a couple of questions to determine participant’s attitude and current half filled/empty mood
◦ Explain purpose the system has problems that we have not been able to uncover.
◦ Make participant comfortable. Assure the participant they can stop at any time
◦ Rehearse the “think out loud” with the participant
◦ Give participant the task scenarios with setting to complete tasks
you observe and record in another location – you must give the participant privacy & room
◦ Following completion of tasks, ask for participant’s views & to complete post-test questionnaire
◦ Thank participant.
What Happens in a User Observation Evaluation Session?
12
◦ Deciding What to Test
◦ Do You Have to Work within Any Constraints?
◦ Writing Up Your Evaluation Strategy for the Global Warming Evaluation
The Global Warming UI: The Evaluation Strategy
Chapter 21
Deciding on what you need toevaluate: the strategy
14
◦ What Is the Purpose of This Evaluation? Does system meet usability requirements/concerns
Qualitative Usability Requirements Desired features
“The users on an e-shopping site should be able to order an item easily and without assistance.”
“Railway clerks work in extremely noisy environments, so any warning messages to them should be visually distinct and highlighted on the screens.”
Creating an Evaluation Strategy (1)
15
Creating an Evaluation Strategy (2)
Quantitative Usability Requirements/Usability Metrics
Explicit measures used: percentages, timings, or numbers are used.
“It should be possible for the users to load understand any page of a web site in 10 seconds using a 56K modem.”
“It should take no more than two minutes for an experienced user (one who has domain knowledge and has undergone the prescribed level of training when the new system is introduced) to enter know how to enter a customer’s details in the hotel’s database & do so with no more than 2 sec hesitation”
16
Creating an Evaluation Strategy
Prioritizing Usability Requirements and Concerns
The usability requirements most important to the success of the system are given priority.
Assign values to the five dimensions of usability, the Five Es.
17
General Museum Site
Online Museum Exhibition
18
Creating an Evaluation Strategy
◦ What Type of Data Do I Want to Collect?
Quantitative data
Numeric content
Qualitative data
Non-numeric content
19
◦ What Am I Evaluating?
Never a finish product without opportunities to improve
◦ What Constraints Do I Have?
Money
Timescales
Availability of usability equipment
Availability of participants and the costs of recruiting them
Availability of evaluators
◦ Documenting the Evaluation Strategy
Evaluation Constraints
20
21
◦ Deciding What to Test
◦ Do You Have to Work within Any Constraints?
◦ Writing Up Your Evaluation Strategy for the Global Warming Evaluation
The Global Warming UI: The Evaluation Strategy
CHAPTER 22
P L A N N I N G W H O , W H AT , W H E N , A N D W H E R E
• Who Is a True User?• Users who reflect the different skills, domain knowledge, system
experience
• Key True User Questions – what characteristics must be present to insure a true user
• Number of Participants to have valid results N=6
• Create an agreement• Before evaluation (protects all; frees user)
• Ulab Team
WHO – USERS & ULAB TEAM
23
WHAT – USABILITY EVALUATION
1. Conduct a planning meeting involving the ULAB team. Definition of the usability goals and concerns for the evaluationEstablishment of the parts of the evaluationCreation of a user profileDevelopment of screening questionnaireCreation of task scenarios (task scripts)Determination of the quantitative and qualitative measures (evaluation metrics)Assignment of team roles for the evaluationEstablishment of the method of analysis of data and the baseline criteriaEstablishment of equipment list needed for the evaluation and configuration of the evaluator room 2. Complete independent tasks to be performed by team members including participant selection criteria, development of test materials, etc.
24
WHAT – USABILITY EVALUATION (CONTINUED)
3. Conduct a lab walk-through for the planned evaluation with all team members. Do a rehearsal with one of the team members as the user. This data is not valid. . Edit all materials accordingly 4. Conduct a pilot evaluation. Edit all materials accordingly. 5. Conduct the evaluation with 6 true users. Analyze the results, prepare findings and recommendations. Prepare a final project summary report and deliver the report, all test materials, and raw capture data. Prepare a PowerPoint presentation of key findings.25
• Establishment of the project time line
• Decide the Duration of the Evaluation Sessions
• Create an Evaluation Timetable – sessions, evaluation, reporting
• Decide the metrics to capture & the baseline criteria• Look at Table 6.1 Possible Measurement Criteria
• Preparing Task Descriptions – the tasks the participant will perform while interacting with the prototype during the evaluation
• Task Cards
• Example - Task Descriptions for Global Warming
WHEN - CREATING A TIMETABLE
26
• Where Will You Do the Evaluation?
• Field Studies – user’s own environment
• Controlled Studies – other than user’s environment
• The Setting for the Global Warming Evaluation
• Arranging Usability Evaluation Sessions
• The Arrangements for the Global Warming Evaluation
WHERE
27