measuring usability

Re Structuring and Measuring our Usability Testing

A more Objective Usability Testing Approach

OR?

Hunch based Usability Testing

Presented by: Indira Pai | Lead SQA Clarice Technologies | www.claricetechnologies.com

2Clarice Technologies

Usability

• Usability : The Ease of use of any system. • The users make or break the success story of

any product.


Usability Testing based on Hunch…

• Hunch• Subjective• Judgment

based?


Common Usability Testing

Challenges



Challenges

• Products are becoming multi dimensional:

• Multi user • Multi platform

• Examples (Web, Mobile, any other H/W):

• Vendor-user, • Coach-player, • Doctor-patient



Challenges

• At any given time, how do we measure the complete system’s

• net usability health status at any given point of time?

• aspect wise usability?• User-Set wise Severity

for each aspect?



Challenges

• How can we re use our findings to help revamp the product in a better way?

• How do we know our Usability testing coverage is complete?

• A big question lies in do we really address all our Usability testing challenges or not, to do justice to

• the product’s ecosystem, • to the product’s users, and • to us, as the owners of the

product quality.


Usability Testing

approaches

• Rule Based:• To follow a certain set of

rules defined at the start of the design based on the overall direction decided by the product management.

• Examples:• Apple UI Guidelines for

App acceptance in app store

• Website look and feel standards


Rule Based Usability Testing Alignment, placement

and No. of StepsConsistency in Accessibility

and Design

Context Sensitive, Accessibility Help and Documentation


Usability Testing

approaches

• Judgment Based: Based on User Experience

• Examples: • Look and Feel of the

product• Ease of re use of the

product


Judgment Based Usability

Testing

• Usability profiling:• Identifying personas of users • Studying Case Studies of similar apps for

those users to build a statistics based analysis

• Exploratory• A/B Testing• Bug Fest: Hallway Testing• Usability Expert Guidance• Automated expert guidance


Rule Based: Usability principles

• Efficiency • Amount of time and steps required for the

user to complete basic tasks.

• Accuracy• How many mistakes do users make when

trying to perform these tasks and how fatal are the mistakes?

• Sometimes, with the right information, the mistake is recoverable.

• Examples:• A Clinical Thermometer• A Financial portfolio Site• A BP Monitoring Unit


Judgment Based:

Usability principles

• Recall • After a period of non-use, how much does a

person remember about the interface and the browsing process?

• If user has to re learn how to use, how easy/difficult is it?

• Emotional response• How does the user feel about the tasks he

had to complete?

• Example:• Flight/Trains tickets Booking

System• Income Tax payment system• Games


Rule Based Usability: Nielsen's heuristics

• Visibility of System status • Anti virus scan: Progress bar and statistics

• Match between system and the real world

• ‘+’ indicates Addition and not Subtraction

• Consistency and standards• OK and Cancel buttons should have the same

text throughout all pop ups

• Error prevention• Alert that leaving a page before triggered

action like save has completed may result in Loss of data

• Help users recognize, diagnose, and recover from errors

• Data could not be saved due to poor N/W.


Judgment Based

Usability: Nielsen's heuristics

• User control and freedom• Ability to cancel any operation like

upload.

• Recognition rather than recall

• Labelling and navigation made easy

• Flexibility and efficiency of use

• Save password in browser

• Aesthetic and minimalist design

• Google search webpage

• Help and documentation• About and FAQs


Case Studies

Case Study 1: Clinical Thermometer

1. takes a lot of time to report Temperature

2. gives a deviation 2 degrees in temperature measured.

3. Has proper instruction manual and is simple and easy to handle for all age groups above 10.


Case Studies

Case Study 2: A web and mobile bill payment web app which is

1. used once in month 2. has a very confusing UI. • Understanding and learning it is a bit

difficult. • Every month one has to explore the

app to understand what is where.

3. The help documentation is not updated and hence not useful.

4. There is no error in bill payment transaction.


Case Studies

Case Study 3: An eco system of a vendor and consumer where Hardware such as WiFi connected entertainment box can be controlled by portal by vendor. Vendor login has extra tabs for all devices super user control, device inventory. The device is operated by a RC.1. Multi user and multi platform case2. The RC shows “Red” when device is on and

“Green” when device is OFF.3. The device has stock Android OS , which is

like a tablet view in television.4. The default apps are ported as is. The UI

looks unaligned when resolution for HD TVs is selected.


Solutions…?

Solution 1: All Usability issues under one hood- Optimal Usability , not Optimal Usability

Pros : Easier to maintain

Cons: Doesn’t objectify each usability

Are the Usability Testing Challenges Addressed?


Solutions…?

Solution 2: Categorizing Usability issues Conclusively based on the following acceptance levels for each usability aspect:

A. Excellent, all needs met in this categoryB. Good, but has scope for bettermentC. OK, can be much more betterD. Not Acceptable


Usability Grade Report: Case 1

Aspect Grade

Remarks if applicable

Visibility of System status A Doesn’t need any change, current user experience is refined in this aspect

Efficiency D These aspects need to be re worked on, their usability is not in acceptable threshold and hence affects user experience adversely.

Accuracy DMatch between system and the real world

D

Consistency and standards DError prevention DHelp users recognize, diagnose, and recover from errors

D

Usability Test Report for Rule Based Aspects



Aspect Grade


Recall A Doesn’t need any change, current user experience is refined in this aspect

User control and freedom C These aspects have a great scope of improvement in terms of usability. Current usability is OK, but can be made much more better.

Recognition rather than recall

C

Flexibility and efficiency of use

C

Aesthetic and minimalist design

C

Emotional Response D These aspects need to be re worked on, their usability is not in acceptable threshold and hence affects user experience adversely.

Help and documentation D

Usability Test Report for Judgment Based Aspects



Aspect Grade Remarks if applicable

Accuracy A Doesn’t need any change, current user experience is refined in this aspect

Match between system and the real world

B Current Usability is Good, but this aspect has scope for betterment

Efficiency C These aspects have a great scope of improvement in terms of usability. Current usability is OK, but can be made much more better.

Consistency and standards C

Error prevention D These aspects need to be re worked on, their usability is not in acceptable threshold and hence affects user experience adversely.

Help users recognize, diagnose, and recover from errors

D

Visibility of System status D

Usability Test Report for Rule Based Aspects





Recognition rather than recall C

Flexibility and efficiency of use C


C



Recall D

Usability Test Report for Judgment Based Aspects



Aspect Grade






Consistency and standards

C



D


Usability Test Report for Rule Based Aspects: User Role - Consumer





Recognition rather than recall C

Flexibility and efficiency of use C


C



Recall D

Usability Test Report for Judgment Based Aspects: User Role - Consumer



Aspect Grade






Consistency and standards C



D


Usability Test Report for Rule Based Aspects: User Role - Vendor



Aspect Grade


User control and freedom

C These aspects have a great scope of improvement in terms of usability. Current usability is OK, but can be made much more better.Recognition rather than

recallC

Flexibility and efficiency of use

C


D These aspects need to be re worked on, their usability is not in acceptable threshold and hence affects user experience adversely.Emotional Response D

Help and documentation

D

Recall D

Usability Test Report for Judgment Based Aspects: User Role - Vendor


Analysis

• Depending on the Grades, we can analyze the strong and weak usability aspects of the system.

• Instead of one general Usability health statement we are able to provide a more objective report.

• Better picture of usability testing coverage and usability health of the system.

• All Usability testing challenges were better addressed by this.


Conclusion

• Measuring a system’s Usability is a subjective undertaking..

• But Usability issues and decisions can be addressed better by moving to a more objective approach.

Questions?For More Information : www.claricetechnologies.com

Pune | Bangalore | USA

Thank You

Speaker Email address: [email protected] More Information : www.claricetechnologies.com

Pune | Bangalore | USA

measuring usability

Documents