making data usable: 2014 presentation at datapalooza
DESCRIPTION
Reviews the science of data display in the context of personal and public health data.TRANSCRIPT
The Science of User-Centered Data Tools
June 2, 2014
Bradford W. Hesse, PhDChief, Health Communication and InformaticsNational Cancer InstituteMonday, June 2, 14
What behavioral intervention was credited with saving 58,000 injuries
per year, & saving $655,000,000 per year?
Question: American Psychological Association (2004)
Monday, June 2, 14
Question: American Psychological Association (2004)
Monday, June 2, 14
Why?
Human Factors research focused squarely on user’s perspective: Cognitive & perceptual demands
Intervention based on human-system integration
Monday, June 2, 14
4.3% in rear end collisions
92,000 crashes / year
58,000 injuries / year
$655,000,000 in property damage
Return on Investment(ROI)
Monday, June 2, 14
“Travel agents selected flight on first line
more than half the time”
American Airlines Sabre System
Data design effects
November Line of Sale Analysis, memo to R. E. Murray from S. D. Nason, American Airlines, Dec. 3, 1981.
Monday, June 2, 14
Website designs reshape travel & other industries
Sabre system becomes Travelocity
Data design effects
November Line of Sale Analysis, memo to R. E. Murray from S. D. Nason, American Airlines, Dec. 3, 1981.
CONSUMER ACCESS SHIFTS MARKET
Monday, June 2, 14
Interface wars part of new business
Microsoft DOS
Graphical User Interface
Monday, June 2, 14
Targets in Medicine?
Monday, June 2, 14
Nudging Best Practice:HITECH & Behavior
Source: Hesse, Bradford W., Ahern, David K., & Woods, Susan S. (2011). Nudging best practice: the HITECH act and behavioral medicine Translational Behavioral Medicine, 1(1), 175-181.
• Incentives• Understand Mental Maps• Defaults• Give feedback• Expect error• Structure decisions
Monday, June 2, 14
Stakes are high
Monday, June 2, 14
Stakes are high
Monday, June 2, 14
Data smog
• Decisional paralysis
• Confusion
• Risky behaviors
• Frequent errors
• Avoidance
Source: Shenk, David. (1997). Data smog : surviving the information glut (1st ed.). San Francisco, Calif.: Harper Edge.
Consequences
Monday, June 2, 14
The case of Hugo Campos
Source: Dave deBronkart, through ONC (http://www.healthit.gov/)Monday, June 2, 14
The case of Hugo Campos
Source: Dave deBronkart, through ONC (http://www.healthit.gov/)Monday, June 2, 14
The case of Hugo Campos
Source: Dave deBronkart, through ONC (http://www.healthit.gov/)Monday, June 2, 14
ROI will go to best design
for each user niche
Interfaces across ecosystem
Source: Hesse BW, Hansen D, Finholt T, Munson S, Kellogg W, Thomas JC. Social Participation in Health 2.0. IEEE Computer. 2010;43(11):45-52.Monday, June 2, 14
Human System Integration
Monday, June 2, 14
History: Bad Design Led to Catastrophic Error
Monday, June 2, 14
Mechanistic World View• Actors: Engineers, biological scientists• Question: How can we create new
technologies?• Focus: Physical Object
Humanistic World View• Actors: Social scientists, physicians• Question: How can we create new
people?• Focus: Person
Competing World Views
Monday, June 2, 14
Source: Vicente, Kim J. (2003). The human factor : revolutionizing the way people live with technology (1st ed.). New York: Taylor and Francis Books.
Human System Integration
Monday, June 2, 14
Knowledge in the Head*
Knowledge in The World*
Task Relevant Schemata
General model
Monday, June 2, 14
Computer Human Interaction
Don Norman, 1988
Jakob Nilsen: 1993, 1999
Shneiderman & Plaisant: 2010 edition
DHHS: 2004
Monday, June 2, 14
• Strive for consistency
Eight Golden Rules
Shneiderman & Plaisant: 2010 edition
See: www.usability.gov
Consistency within application
Consistency across product line
Predictable Controls
Monday, June 2, 14
• Strive for consistency
• Cater to universal usability*
Eight Golden Rules
Shneiderman & Plaisant: 2010 edition
Sir Jonathan Ive
*universal across experience, literacy, physical ability, profession
Monday, June 2, 14
• Strive for consistency
• Cater to universal usability
• Offer informative feedback
Eight Golden Rules
Shneiderman & Plaisant: 2010 editionSeriously?!
Monday, June 2, 14
• Strive for consistency
• Cater to universal usability
• Offer informative feedback
Eight Golden Rules
Shneiderman & Plaisant: 2010 edition
Good design: sequence, progress, action all indicated
Monday, June 2, 14
• Strive for consistency
• Cater to universal usability
• Offer informative feedback
• Design dialogs to yield closure
Eight Golden Rules
Shneiderman & Plaisant: 2010 edition
user goal
Deep Support
Monday, June 2, 14
• Strive for consistency
• Cater to universal usability
• Offer informative feedback
• Design dialogs to yield closure
• Prevent errors
Eight Golden Rules
Shneiderman & Plaisant: 2010 edition
Monday, June 2, 14
• Strive for consistency
• Cater to universal usability
• Offer informative feedback
• Design dialogs to yield closure
• Prevent errors
• Permit easy reversal of actions
Eight Golden Rules
Shneiderman & Plaisant: 2010 edition
Monday, June 2, 14
• Strive for consistency
• Cater to universal usability
• Offer informative feedback
• Design dialogs to yield closure
• Prevent errors
• Permit easy reversal of actions
• Support internal locus of control
Eight Golden Rules
Shneiderman & Plaisant: 2010 edition
Monday, June 2, 14
Wrong Question:
X What can the computer do?
X How do we automate cognition?
X What is the transactional gain?
X How do we get users to conform?
Better Questions:
✓ What can humans do?
✓ How do we augment cognition?
✓ What is the relational gain?
✓ How do we optimizesociotechnical balance?
Source: Hesse BW, Shneiderman B. eHealth research from the user's perspective. Am J Prev Med 2007;32(5 Suppl):S97-103.
Asking the Right Question in H.I.T.
Monday, June 2, 14
STOP asking the wrong question
Wrong Question:
X What can the computer do?
X How do we automate cognition?
X What is the transactional gain?
X How do we get users to conform?
Better Questions:
✓ What can humans do?
✓ How do we augment cognition?
✓ What is the relational gain?
✓ How do we optimizesociotechnical balance?
“Expert systems in medicine turned out to be brittle, impracticable, and nontransparent. In short, they turned out to be bad medicine.”
Source: Hesse BW, Shneiderman B. eHealth research from the user's perspective. Am J Prev Med 2007;32(5 Suppl):S97-103.
Monday, June 2, 14
START Answering Right Question
Wrong Question:
X What can the computer do?
X How do we automate cognition?
X What is the transactional gain?
X How do we get users to conform?
Better Questions:
✓ What can humans do?
✓ How do we augment cognition?
✓ What is the relational gain?
✓ How do we optimizesociotechnical balance?
David Brailer, First Na4onal Coordinator for Health IT
“Everyone thought Health I.T. was about computers, but we’ve refined that to say that IT is
about healthcare — it’s about the experience we really have.”
Source: 1. Brailer D. Ac/on through collabora/on: a conversa/on with David Brailer. The na/onal coordinator of HIT believes that facilita/on, not mandates, are the way to move the agenda forward. Interview by Robert Cunningham. Health Aff (Millwood) 2005;24(5):1150-‐7.
Monday, June 2, 14
STOP asking the wrong question
Wrong Question:
X What can the computer do?
X How do we automate cognition?
X What is the transactional gain?
X How do we get users to conform?
Better Questions:
✓ What can humans do?
✓ How do we augment cognition?
✓ What is the relational gain?
✓ How do we optimizesociotechnical balance?
“Intelligent Agent” misfires
Root Cause: Over-‐Reliance on “Autopilot”
Monday, June 2, 14
Wrong Question:
X What can the computer do?
X How do we automate cognition?
X What is the transactional gain?
X How do we get users to conform?
Better Questions:
✓ What can humans do?
✓ How do we augment cognition?
✓ What is the relational gain?
✓ How do we optimizesociotechnical balance?
START Answering Right Question
Reminder System
Source: Hesse BW. Enhancing Consumer Involvement in Health Care. In: Parker JC, Thornson E, editors. Health Communica/on in the New Media Landscape. New York, NY: Springer Publishing Company; 2008. p. 119-‐149.
Monday, June 2, 14
STOP asking the wrong question
Wrong Question:
X What can the computer do?
X How do we automate cognition?
X What is the transactional gain?
X How do we get users to conform?
Better Questions:
✓ What can humans do?
✓ How do we augment cognition?
✓ What is the relational gain?
✓ How do we optimizesociotechnical balance?
*Zuboff S, Maxmin J. The support economy: why corporations are failing individuals and the next episode of capitalism. New York: Viking; 2002.
Monday, June 2, 14
Wrong Question:
X What can the computer do?
X How do we automate cognition?
X What is the transactional gain?
X How do we get users to conform?
Better Questions:
✓ What can humans do?
✓ How do we augment cognition?
✓ What is the relational gain?
✓ How do we optimizesociotechnical balance?
START Answering Right Question
Source: Hesse BW. Harnessing the power of an intelligent health environment in cancer control. Stud Health Technol Inform 2005;118:159-‐76..Monday, June 2, 14
Wrong Question:
X What can the computer do?
X How do we automate cognition?
X What is the transactional gain?
X How do we get users to conform?
Better Questions:
✓ What can humans do?
✓ How do we augment cognition?
✓ What is the relational gain?
✓ How do we optimizesociotechnical balance?
START Answering Right Question
*Zuboff S, Maxmin J. The support economy: why corporations are failing individuals and the next episode of capitalism. New York: Viking; 2002.
Monday, June 2, 14
STOP asking the wrong question
Wrong Question:
X What can the computer do?
X How do we automate cognition?
X What is the transactional gain?
X How do we get users to conform?
Better Questions:
✓ What can humans do?
✓ How do we augment cognition?
✓ What is the relational gain?
✓ How do we optimizesociotechnical balance?
Charlie Chaplin in “Modern
Times” (1936)
Monday, June 2, 14
Wrong Question:
X What can the computer do?
X How do we automate cognition?
X What is the transactional gain?
X How do we get users to conform?
Better Questions:
✓ What can humans do?
✓ How do we augment cognition?
✓ What is the relational gain?
✓ How do we optimizesociotechnical balance?
START Answering Right Question
HITECH Switches Emphasis to “Meaningful Use”
Monday, June 2, 14
Wrong Question:
X What can the computer do?
X How do we automate cognition?
X What is the transactional gain?
X How do we get users to conform?
Better Questions:
✓ What can humans do?
✓ How do we augment cognition?
✓ What is the relational gain?
✓ How do we optimizesociotechnical balance?
START Answering Right Question
Source: Blumenthal, D. (2010). Guiding the health information technology agenda. Interviewed by David J. Brailer. Health Aff (Millwood), 29(4), 586-595.
David Blumenthal
Monday, June 2, 14
Research on Communicating Data
Monday, June 2, 14
Making Data Talk
Inform Support Decisions
Educate Persuade
Adapting to a World of Ubiquitous Data Systems
Monday, June 2, 14
Making Data Talk
Inform Support Decisions
Educate Persuade
Adapting to a World of Ubiquitous Data Systems
Monday, June 2, 14
Making Data Talk
Inform Support Decisions
Educate Persuade
Adapting to a World of Ubiquitous Data Systems
Monday, June 2, 14
Making Data Talk
Inform Support Decisions
Educate Persuade
Adapting to a World of Ubiquitous Data Systems
Monday, June 2, 14
Chapter 4: Visual Displays
Monday, June 2, 14
SOURCE: http://alleydog.com/topics/sensation_and_perception.php
Perceptual Basics
Monday, June 2, 14
source: Carpenter PA, Shah P. A model of the perceptual and conceptual processes in graph comprehension. J Educ Psychol. 1999, 91(4): 690-702.
• Constructive process
• Gaze goes to center for pattern
• Contiguous labels for meaning
• Left to right tendency in western culture
• Perceptual rules guide meaning
Cognitive / Perceptual Research
Monday, June 2, 14
source: Carpenter PA, Shah P. A model of the perceptual and conceptual processes in graph comprehension. J Educ Psychol. 1999, 91(4): 690-702.
• Constructive process
• Gaze goes to center for pattern
• Contiguous labels for meaning
• Left to right tendency in western culture
• Perceptual rules guide meaning
Visualizing Long Term Change
Monday, June 2, 14
• Constructive process
• Gaze goes to center for pattern
• Contiguous labels for meaning
• Left to right tendency in western culture
• Perceptual rules guide meaning
Hans Rosling, BBC
Visualizing Change Dynamically
Monday, June 2, 14
Monitoring for Change in EHR Systems
Aging In Place, IntelRule of Thumb* for “Big Data” Systems
•Overview
•Zoom / filter
•Details on demand
*Ben Shneiderman
Monday, June 2, 14
Biases & Heuristics
Monday, June 2, 14
Overcome “small numbers” bias
Monday, June 2, 14
Exceptional Case
Fallacy of small numbers;Tversky & Kahneman, 1971
Illnesses322,000,000
Hospitalizations21,000,000
Prevented
Deaths732,000
Monday, June 2, 14
Improving Decision MakingProblem: Conditional (Bayesian) probabilities are counter-intuitive, arcane for practice.
source: Gigerenzer, Gerd, & Hoffrage, Ulrich. (1995). How to improve Bayesian Reasoning without Instruction: Frequency Formats. Psychological Review, 102(4), 684-704.
For example:
Monday, June 2, 14
95 out of 100 physicians estimated 70-80% instead
of a correct 7.8%
Improving Decision MakingProblem: Conditional (Bayesian) probabilities are counter-intuitive, arcane for practice.
source: Gigerenzer, Gerd, & Hoffrage, Ulrich. (1995). How to improve Bayesian Reasoning without Instruction: Frequency Formats. Psychological Review, 102(4), 684-704.
For example:
Monday, June 2, 14
Natural frequencies work better
source: Gigerenzer, Gerd, & Hoffrage, Ulrich. (1995). How to improve Bayesian Reasoning without Instruction: Frequency Formats. Psychological Review, 102(4), 684-704.
Monday, June 2, 14
See: Fagerlin, A., Ubel, P. A., Smith, D. M., & Zikmund-Fisher, B. J. (2007). Making numbers matter: present and future research in risk communication. Am J Health Behav, 31 Suppl 1, S47-56.
Icon arrays convey natural frequencies more effectively
Monday, June 2, 14
Portraying trends to policy makersChoropleth Maps: CDC Obesity Trends, BRFSS 1985
Monday, June 2, 14
Nonsegmented geographic data
Isopleth “Weather Maps,” HINTS
Monday, June 2, 14
Juxtaposing geographic distributions
Mortality Maps (SEER):Lung Cancer Mortality
For Example:Knowledge Maps (HINTS):Does Smoking Cause Cancer?
Monday, June 2, 14
Added User Controls14 datasets spanning 6 years
NSF, NIH Collaboration
Disolving Barriers Between Clinical and Community Health
Monday, June 2, 14
Added User Controls14 datasets spanning 6 years
NSF, NIH Collaboration
Disolving Barriers Between Clinical and Community Health
Monday, June 2, 14
Disparities Frame
Framing Effects
Monday, June 2, 14
Disparities Frame
Impact Frame
Framing Effects
Monday, June 2, 14
Disparities Frame
Impact Frame
Progress Frame
Framing Effects
Monday, June 2, 14
Disparities Frame
Impact Frame
Progress Frame
Framing Effects
3.25
3.5
3.75
4
4.25
4.5
4.75
5
Progress Impact Disparity
Low MistrustHigh Mistrust
3.25
3.5
3.75
4
4.25
4.5
4.75
5
Progress Impact Disparity
Low MistrustHigh Mistrust
I want to be screened for colon cancer? Framing X Medical Mistrust Questionnaire
Best influence on behavior
Monday, June 2, 14
Bottom Line
Monday, June 2, 14
Data: New “Intel Inside*”
Source: O'Reilly, Tim. (2005). What Is Web 2.0? Design Patterns and Business Models for the Next Generation of Software.
Tim O’Reilly
Monday, June 2, 14
Data = Power(by itself)
Monday, June 2, 14
Data = PowerUser Centered*
*i.e., made understandable, actionable, accessible Monday, June 2, 14
Research methods to address gap
See: Brinck, T., Gergle, D., & Wood, S. D. (2002). Designing Web sites that work : usability for the Web (1st ed.). San Francisco: Morgan Kaufmann Publishers.
Monday, June 2, 14
www.slideshare.net/BradfordHesse
http://cancercontrol.cancer.gov/brp/hcirb/
usability.gov
Guidelines
Grants
This slide deck
Thank you!
Monday, June 2, 14