1

CS 430: Information Retrieval

Lecture 13

Usability 1Guest Lecture: Gilly Leshed

2

Course Administration

Midterm Examination

When: Wednesday, October 13, 7:30 to 9:00 p.m.

Where: Upson B17

About: All material covered before the midterm break

For a sample paper and further information see the Examinations page on the web site.

Note the instructions about laptop computers.

3

Course Administration

Assignment 2

To help understanding of how to calculate the similarity between a query and a document, Lecture 11, Slide 24 has been expanded to three slides.

The paper describes how to calculate the inner product between two vectors. Lecture 11, Slides 24-26 now describe how to scale the inner product to calculate the cosine of the angle between two vectors.

With the test data, the documents are all similar in length, so that a ranking based on inner product will be close to a ranking based on cosine.

4

Lecture Outline

• N-Tier Model

• User-Centered Approach

• Requirements

• Design

• Implementation

• Evaluation

• Summary

5

User InterfaceUser Interface

Business LogicBusiness Logic

DataData

N-Tier Model

Web pageWeb page

Search engineSearch engine

Index file, data filesIndex file, data files

Focus of this lecture

N-Tier model

6

User-Centered Approach

• Requirements

– Who are the users?

– What do they want?

– What do they need?

• Designing the user interface

• Implementing the user interface

• Evaluating the user interface

User-Centered Approach

Requirements

Design

Evaluation

Implementation

7

RequirementsWho are the users?

• Understanding the users via ethnographic research• Descriptions of users:

– Demographic characteristics– Computer usage background– Job description and work environment– Disabilities:

• Color blindness• Language issues• Typing issues

• Personas – archetypes of users, describing behavior patterns, goals, skills, attitudes, and environment (Cooper, 2003)

Requirements

The study of people in their natural settings; a descriptive account of social life and culture, based on qualitative methods (e.g. detailed observations, unstructured interviews, analysis of documents)

The study of people in their natural settings; a descriptive account of social life and culture, based on qualitative methods (e.g. detailed observations, unstructured interviews, analysis of documents)

8

RequirementsWhat do Users Need?

• Defining user interface requirements

• Based on task analysis:

– Task definition

– Context definition

• Several ways to accomplish task analysis:

– Ethnographic research

– Scenario-based analysis

– Discussion with users and subject-matter experts (SME)

Requirements

9

RequirementsWhat do Users Want?

• When asking users, they often:– Provide their attitudes, not their needs– Bend the truth to be closer to what they think you want

to hear– Rationalize their behavior

“I would have seen the button if it had been bigger”

• Instead of asking users what they want:– Watch what they actually do– Do not believe what they say they do– Definitely don't believe what they predict they may do

in the futureRequirements

10

Non-functional Requirements

Performance, Reliability, Scalability, Security…

• Example: Response time (Nielsen, 1994)0.1 sec – the user feels that the system is reacting instantaneously1 sec – the user will notice the delay, but his/her flow of thought stays uninterrupted10 sec – the limit for keeping the user's attention focused on the dialogue

Requirements

11

DesignMental Models

Also called conceptual model

• What a person thinks is true about a system, not necessarily what is actually true

• Similar in structure to the system that is represented

• Simpler than the represented system. A mental model includes only enough information to allow accurate predictions (i.e. no data structures)

• Allows a person to predict the results of his actions

Design

12

System Model of an ArticleSystem Model of an ArticleUser Model of an ArticleUser Model of an Article System Model of an ArticleSystem Model of an ArticleUser Model of an ArticleUser Model of an Article

DesignMental Models – Example 1

The article’s body and meta-data are conceived as a whole.

An article’s meta-data is available in one database and its data is available in a separate database.

Design

13

DesignMental Models – Example 2

System Model of a search engine

System Model of a search engine

User Model of a search engine

User Model of a search engine

The search engine retrieves return hits directly from their source

The search action does not involve accessing the documents’ source

Design

14

DesignUser Interface Design Guidelines

• Consistency – Appearance, controls, and function– Both within the system and to similar systems

• Feedback • User in control• Recognition rather than recall• Easy reversal of actions

– Error handling• Consider different expertise:

– Novice, intermediate and expert users

Design

15

Designing the Search PageMaking Decisions

• Overall organization:

– Spacious or cramped

– Division of functionality to different pages

– Positioning components in the interface

– Emphasizing parts of the interface

• Query insertion: insert text string or fill in text boxes

• Interactivity of search results

– Retrieve the information from the results

– Narrow the search

• Performance requirements

Design

16

Google Spacious organization

Spacious organization

17

Yahoo! cramped organization

cramped organization

18

AltaVista

Division of functionality to different pages

Division of functionality to different pages

19

ACM Digital Library

Emphasized components

Emphasized components

20

ACM Digital Library advance searchDifferent query insertion ways

Different query insertion ways

21

DesignReturn Hits

• A snippet is a short record that a search system returns to describe and link to a hit.

• Example: Web search “Nielsen evaluation heuristics”

Heuristic Evaluation

... Jakob Nielsen's Online Writings on Heuristic Evaluation. How to conduct a heuristic evaluation; A list of ten recommended heuristics for usable interface design ... www.useit.com/papers/heuristic/ - 5k - Cached - Similar pages

Design

22

Designing the Return HitsMaking Decisions

• Dynamic (generated from query+ document) or precomputed (from document only)

• Content only or with related information

• Highlighting of search terms

• Length vs. number on page

User must understand why the hit was returned

Design

23

Dynamic Return Hits Dynamic snippets

Dynamic snippets

24

Precomputed Return Hits Precomputed snippets

Precomputed snippets

25

Implementation

• Use HTML, Java, Visual Basic, WinForms…

• Make sure the user interface is loosely-coupled with the search engine underneath

User InterfaceUser Interface

Business LogicBusiness Logic

DataData

Implementation

26

Evaluation

• The process of determining the worth of, or assigning a value to, the user interface on the basis of careful examination and judgment.

• Make sure your system is usable before launching it!

• Two categories of evaluation methods:

– Empirical evaluation: with users

– Analytical evaluation: without users

Evaluation

27

Evaluation

But what is usability?From ISO 9241-11, Usability comprises the following aspects:• Effectiveness – the accuracy and completeness with which users

achieve certain goals– Measures: quality of solution, error rates

• Efficiency – the relation between the effectiveness and the resources expended in achieving them– Measures: task completion time, learning time, clicks number

• Satisfaction – the users’ comfort with and positive attitudes towards the use of the system– Measures: attitude rating scales

Evaluation

28

Evaluation with Users

Testing the system, not the users!

Stages of evaluation with users:

Preparation

Sessions conduct

Analysis of results

Evaluation

29

Evaluation with UsersPreparation

• Determine goals of the usability testing“The user can find the required information in no more than 2 minutes”

• Write the user tasks“Answer the question: how hot is the sun?”

• Get participants

– Use the descriptions of users from the requirements phase to detect potential users

Evaluation

30

Evaluation with UsersSessions Conduct

• Conduct the session– Usability Lab– Simulated working

environment

• Observe the user– Human observer(s)– Video camera– Audio recording

• Inquire satisfaction dataEvaluation

31

Evaluation with UsersResults Analysis

• If possible, use statistical summaries

• Pay close attention to areas where users

– were frustrated

– took a long time

– couldn't complete tasks

• Respect the data and users' responses, don't make excuses for designs that failed

• Note designs that worked and make sure they're incorporated in the final product.

Evaluation

32

Evaluation without Users

• Assessing systems using established theories and methods

Evaluation techniques• Heuristic Evaluation (Nielsen, 1994)

– Evaluate the design using “rules of the thumb”• Cognitive Walkthrough (Wharton et al, 1994)

– A formalized way of imagining people’s thoughts and actions when they use the interface for the first time

• Claims Analysis – based on scenario-based analysis– Generating positive and negative claims about the

effects of features on the user

Evaluation

33

Summary

User InterfaceUser Interface

Business LogicBusiness Logic

DataData

Summary

Requirements

Design

Implementation

Evaluation