wh2014 session: a pilot study of an inspection framework for automated usability guideline
DESCRIPTION
Wireless Health 2014 Conference Technical Session 5 featuring speaker Jing Xu.TRANSCRIPT
![Page 1: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/1.jpg)
WLSACONVERGENCE SUMMIT
A PILOT STUDY OF AN INSPECTION FRAMEWORK FOR AUTOMATED USABILITY GUIDELINE REVIEWS OF MOBILE HEALTH APPLICATIONSJING XU, DEPARTMENT OF
COMPUTER SCIENCE UNIVERSITY OF MASSACHUSETTS LOWELL
![Page 2: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/2.jpg)
A pilot study of an Inspection Framework for Automated Usability Guideline Reviews of Mobile Health
ApplicationsJing Xu, Xiang Ding, Ke Huang, Guanling Chen
Department of Computer ScienceUniversity of Massachusetts Lowell
![Page 3: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/3.jpg)
Motivation: importance of usability
The importance of usability
from http://www.userlab.com/evaluations.html
![Page 4: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/4.jpg)
4
![Page 5: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/5.jpg)
Motivation: fast delivery mode
http://mhealthwatch.com
![Page 6: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/6.jpg)
Motivation: negligible gap in the future
Smartphone penetration in US
from http://www.edisonresearch.com/2014-smartphone-ownership-demographics/
![Page 7: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/7.jpg)
Motivation: stricter usability requirements for mHealth Apps
Human Aging and Web Use
from http://www.nngroup.com/articles/usability-for-senior-citizens/
![Page 8: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/8.jpg)
Definition: According to Jakob Nielsen
…usability is:
Learnability
Efficiency
Memorability
Satisfaction
Error prevention
http://www.nngroup.com/articles/usability-101-introduction-to-usability/
![Page 9: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/9.jpg)
1. Empirical (users are involved)
Three types of Usability testing:
![Page 10: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/10.jpg)
2. Formal (Analytical Approaches)
Three types of Usability testing:
Usability Experts Forms, Models
![Page 11: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/11.jpg)
Three types of Usability testing:
3. Inspection (Heuristic Evaluation)
Usability Experts Heuristics & Guidelines
![Page 12: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/12.jpg)
from: https://blog.sensortower.com/
Top 25 ios App update frequency as of Apr 2014
Challenges I: fast iteration VS cost
![Page 13: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/13.jpg)
Challenges II: diversity of hardware VS scalability
Android Device Fragmentationfrom: http://opensignal.com
![Page 14: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/14.jpg)
Proposed Solution:
Automated guideline reviewsHeuristic set for the unique domain (mobile health apps)Translate heuristics into quantitative metricsAn automated app crawler
![Page 15: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/15.jpg)
Talk Outline
• Specialized heuristics for mobile health apps
• The translation
• Methods
• Conclusion
![Page 16: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/16.jpg)
Specialized Heuristics:
• Web applications
• Assistive robotics
• Collaborative systems
• Mobile games
• Game playability
• …Nielsen’sHeuristics
SpecializedHeuristics
Problems found
![Page 17: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/17.jpg)
Specialized Heuristics for Mobile Health Apps (HIMSS):• Simplicity
• Naturalness
• Consistency
• Forgiveness and feedback
• Effective use of language
• Efficient interactions
• Efficient information presentation
• Preservation of context
• Minimum cognitive overload
![Page 18: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/18.jpg)
Talk Outline
• Specialized heuristics for mobile health apps
• The translation
• Methods
• Conclusion and Future Works
![Page 19: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/19.jpg)
Translation:
Heuristic 1Heuristic 2
.
.
.
.Heuristic 9
Heuristic 1Heuristic 2
.
.
.
.Heuristic 9
Task Level
Screen Level
Widget Level
Three Levels of MetricsHIMSS Heuristics for Mobile Health Apps
![Page 20: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/20.jpg)
Task Level Metrics
• Number of steps
• Number of data inputs
• Pattern of user sequences
• Average operation distance
• Operations that require longer time
• Operations that consume more resources
![Page 21: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/21.jpg)
Screen Level Metrics
• Information content
• Layout content
• Visual noises
• Number of Medical terms
• Number of ListViewItem count
• Number of MenuItem count
• Number of scrollable groups
![Page 22: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/22.jpg)
Individual Widget Level Metrics
• Widgets leading to significant UI changes
• Default value and one-click delete
• Color, size and contrast ratio of text
• No inappropriate usage of red color
![Page 23: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/23.jpg)
Talk Outline
• Specialized heuristics for mobile health apps
• The translation process
• Methods
• Conclusion
![Page 24: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/24.jpg)
Methods:
![Page 25: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/25.jpg)
CGGM of Mobile Apps
A state machine model of UI transitions
A state is a description of the structure of all equivalent screen views
S iS i
A transition is an UI interaction
![Page 26: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/26.jpg)
CGGM of Mobile Apps
The description of the structure of a screen view
11 22
33 44
55 66
77
88
99
1010
1111
1212
1313
1414
![Page 27: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/27.jpg)
CGGM of Mobile AppsCore of CGGM: use structure equivalence to
identify a unique state
Avoid state explosion Capture core functions Concise yet comprehensive
![Page 28: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/28.jpg)
Now, we have the automated app crawler.
? What’s next
![Page 29: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/29.jpg)
Raw metrics:
• State (UI) transitions• Response time of a transition• Location, size, type of widgets• Color, size and count of text• Words• Count of ListView Items• Count of Menu Items• Scrollable views
![Page 30: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/30.jpg)
Raw Metrics to Metrics at Three Different Levels:
State Transitions Task Level
State Screen Level
Group Widget Level
The translation is based on the state machine.
![Page 31: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/31.jpg)
Raw Metrics to Metrics at Three Different Levels:
Task Level
Definition of “task” based on the state
machine
Operation, pattern, data input
measurements based on tasks
![Page 32: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/32.jpg)
Raw Metrics to Metrics at Three Different Levels:
Screen (State) Level
Space Usage
Space Complexity
Space Efficiency
Consistency
![Page 33: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/33.jpg)
Inefficiency:
79 options9 options / screen
Diabetes Plus
![Page 34: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/34.jpg)
Complexity:
ColorsColor contrastFontSpace usagetext countNumber of function groups
Diabetes Diary
![Page 35: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/35.jpg)
Raw Metrics to Metrics at Three Different Levels:
Individual Widget Level
Properties: color, font, size
Adapt to different devices
Interaction efficiency
![Page 36: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/36.jpg)
Efficiency:
Widget in group of size 1
Widgets associated with interactions that cause significant UI changes
![Page 37: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/37.jpg)
Talk Outline
• Specialized heuristics for mobile health apps
• The translation process
• Methods
• Conclusion
![Page 38: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/38.jpg)
To Conclude:
An automated inspection system cannot capture all usability problems
An assistive toolkit to help reduce the workload of human experts
Enforce a minimum bar for the usability of mobile health applications
Further adapted to different health domains and populations
![Page 39: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/39.jpg)
Thanks!
![Page 40: WH2014 Session: A pilot study of an inspection framework for automated usability guideline](https://reader036.vdocuments.mx/reader036/viewer/2022062313/55842d79d8b42a0b6d8b4dd8/html5/thumbnails/40.jpg)
WLSACONVERGENCE SUMMIT
www.wirelesshealth2014.org