analyzing user interactions for data and user modeling
DESCRIPTION
Analyzing User Interactions for Data and User Modeling. Remco Chang Assistant Professor Tufts University. Human + Computer. Human vs. Artificial Intelligence Garry Kasparov vs. Deep Blue (1997) Computer takes a “brute force” approach without analysis - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/1.jpg)
1/45 Remco Chang – Sandia 14
Analyzing User Interactions forData and User Modeling
Remco Chang
Assistant ProfessorTufts University
![Page 2: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/2.jpg)
2/45 Remco Chang – Sandia 14
Human + Computer
• Human vs. Artificial IntelligenceGarry Kasparov vs. Deep Blue (1997)– Computer takes a “brute force” approach
without analysis– “As for how many moves ahead a grandmaster
sees,” Kasparov concludes: “Just one, the best one”
• Artificial vs. Augmented IntelligenceHydra vs. Cyborgs (2005)– Grandmaster + 1 chess program > Hydra (equiv.
of Deep Blue)– Amateur + 3 chess programs > Grandmaster + 1
chess program1
1. http://www.collisiondetection.net/mt/archives/2010/02/why_cyborgs_are.php
![Page 3: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/3.jpg)
3/45 Remco Chang – Sandia 14
“The computer is incredibly fast, accurate, and stupid. Man is unbelievably slow, inaccurate, and
brilliant. The marriage of the two is a force beyond calculation.”
-Leo Cherne, 1977 (often attributed to Albert Einstein)
![Page 4: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/4.jpg)
4/45 Remco Chang – Sandia 14
Which Marriage?
![Page 5: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/5.jpg)
5/45 Remco Chang – Sandia 14
Which Marriage?
![Page 6: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/6.jpg)
6/45 Remco Chang – Sandia 14
(Modified) Van Wijk’s Model of Visualization
Data
Data
Visualization
Vis
Params
User
Perceive
Explore
Discovery
Image
Interaction
![Page 7: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/7.jpg)
7/45 Remco Chang – Sandia 14
When the Analyst is Successful….
Data
Data
Visualization
Vis
Params
User
Perceive
Explore
Discovery
Image
Interaction
Data + Vis + Interaction + User = Discovery
![Page 8: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/8.jpg)
8/45 Remco Chang – Sandia 14
Remco’s Research Goal
“Reverse engineer” the human cognitive black box (by analyzing user interactions)
A. Data Modeling– Interactive Metric Learning
B. User Modeling– Predict Analysis Behavior
C. Perception and Cognition– Perception Modeling – Cognitive Priming
D. Mixed Initiative Systems– Adaptive Visualization and Computation
R. Chang et al., Science of Interaction, Information Visualization, 2009.
![Page 9: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/9.jpg)
9/45 Remco Chang – Sandia 14
Data Modeling
1. Interactive Metric LearningQuantifying a User’s Knowledge about Data
![Page 10: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/10.jpg)
10/45 Remco Chang – Sandia 14
1. Richard Heuer. Psychology of Intelligence Analysis, 1999. (pp 53-57)
![Page 11: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/11.jpg)
11/45 Remco Chang – Sandia 14
Exploring High-Dimensional Space: iPCA
Jeong et al., iPCA: An Interactive System for PCA-based Visual Analytics. Eurovis 2009.
![Page 12: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/12.jpg)
12/45 Remco Chang – Sandia 14
Metric Learning
• Finding the weights to a linear distance function
• Instead of a user manually give the weights, can we learn them implicitly through their interactions?
![Page 13: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/13.jpg)
13/45 Remco Chang – Sandia 14
Metric Learning
• In a projection space (e.g., MDS), the user directly moves points on the 2D plane that don’t “look right”…
• Until the expert is happy (or the visualization can not be improved further)
• The system learns the weights (importance) of each of the original k dimensions
• Short Video (play)
![Page 14: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/14.jpg)
14/45 Remco Chang – Sandia 14
Dis-Function
Brown et al., Find Distance Function, Hide Model Inference. IEEE VAST Poster 2011Brown et al., Dis-function: Learning Distance Functions Interactively. IEEE VAST 2012.
Optimization:
![Page 15: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/15.jpg)
15/45 Remco Chang – Sandia 14
Results
• Used the “Wine” dataset (13 dimensions, 3 clusters)
• Added 10 extra dimensions, and filled them with random values
• Blue: original data dimension• Red: randomly added
dimensions• X-axis: dimension number• Y-axis: final weights of the
distance function
![Page 16: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/16.jpg)
16/45 Remco Chang – Sandia 14
User Modeling
2. Learning about a User in Real-TimeWho is the user,
and what is she doing?
![Page 17: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/17.jpg)
17/45 Remco Chang – Sandia 14
One Question at a Time
Data
Data
Visualization
Vis
Params
User
Perceive
Explore
Discovery
Image
Interaction
Data + Vis + Interaction + User = Discovery
Novice or Expert?
Introvert or
Extrovert?
Fast or
Slow?
![Page 18: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/18.jpg)
18/45 Remco Chang – Sandia 14
Experiment: Finding Waldo
• Google-Maps style interface– Left, Right, Up, Down, Zoom In, Zoom Out, Found
![Page 19: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/19.jpg)
19/45 Remco Chang – Sandia 14
Fast completion time
Pilot Visualization – Completion Time
Slow completion time
Eli Brown et al., Where’s Waldo. IEEE VAST 2014, Conditionally Accepted.
![Page 20: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/20.jpg)
20/45 Remco Chang – Sandia 14
Post-hoc Analysis ResultsMean Split (50% Fast, 50% Slow)
Data Representation Classification Accuracy Method
State Space 72% SVM
Edge Space 63% SVM
Action Sequence 77% Decision Tree
Mouse Event 62% SVM
Fast vs. Slow Split (Mean+0.5σ=Fast, Mean-0.5σ=Slow)
Data Representation Classification Accuracy Method
State Space 96% SVM
Edge Space 83% SVM
Action Sequence 79% Decision Tree
Mouse Event 79% SVM
![Page 21: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/21.jpg)
21/45 Remco Chang – Sandia 14
“Real-Time” Prediction (Limited Time Observation)
State-Based
Linear SVM
Accuracy: ~70%
Interaction Sequences
N-Gram + Decision Tree
Accuracy: ~80%
![Page 22: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/22.jpg)
22/45 Remco Chang – Sandia 14
Predicting a User’s Personality
External Locus of Control Internal Locus of Control
Ottley et al., How locus of control influences compatibility with visualization style. IEEE VAST , 2011.Ottley et al., Understanding visualization by understanding individual users. IEEE CG&A, 2012.
![Page 23: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/23.jpg)
23/45 Remco Chang – Sandia 14
Predicting Users’ Personality Traits
• Noisy data, but can detect the users’ individual traits “Extraversion”, “Neuroticism”, and “Locus of Control” at ~60% accuracy by analyzing the user’s interactions alone.
Predicting user’s “Extraversion”
Linear SVM
Accuracy: ~60%
![Page 24: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/24.jpg)
24/45 Remco Chang – Sandia 14
Perception and Cognition
3. What are the Factors that Correlate with a User’s Performance?
![Page 25: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/25.jpg)
25/45 Remco Chang – Sandia 14
Individual Differences and Interaction Pattern
• Existing research shows that all the following factors affect how someone uses a visualization:
Peck et al., ICD3: Towards a 3-Dimensional Model of Individual Cognitive Differences. BELIV 2012Peck et al., Using fNIRS Brain Sensing To Evaluate Information Visualization Interfaces. CHI 2013
– Spatial Ability– Experience (novice vs. expert)– Emotional State– Personality– Cognitive Workload/Mental
Demand– Perception– … and more
![Page 26: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/26.jpg)
26/45 Remco Chang – Sandia 14
Cognitive Load
Functional Near-Infrared Spectroscopy • fNIRS• a lightweight brain sensing technique • measures mental demand (working
memory)
Evan Peck et al., Using fNIRS Brain Sensing to Evaluate Information Visualization Interfaces. CHI 2013.
![Page 27: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/27.jpg)
27/45 Remco Chang – Sandia 14
Cognitive Priming
![Page 28: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/28.jpg)
28/45 Remco Chang – Sandia 14
Emotion and Visual Judgment
Harrison et al., Influencing Visual Judgment Through Affective Priming, CHI 2013
![Page 29: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/29.jpg)
29/45 Remco Chang – Sandia 14
Modeling User Perception with Weber’s Law
![Page 30: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/30.jpg)
30/45 Remco Chang – Sandia 14
Weber’s Law & Just Noticeable Difference (JND)
Objective Stimulus
Perc
eive
d Sti
mul
us
Objective Stimulus
Just
Noti
ceab
le D
iffer
ence
Ideal
Perception
Ideal
Perception
![Page 31: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/31.jpg)
31/45 Remco Chang – Sandia 14
Perception of Correlation and Weber’s
Rensink and Baldridge, The Perception of Correlation in Scatterplots. EuroVis 2010.
![Page 32: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/32.jpg)
32/45 Remco Chang – Sandia 14
Perception of Correlation and Weber’s
![Page 33: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/33.jpg)
33/45 Remco Chang – Sandia 14
Ranking Visualizations
Harrison et al., Ranking Visualization of Correlation with Weber’s Law. InfoVis 2014 (Conditional)
![Page 34: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/34.jpg)
34/45 Remco Chang – Sandia 14
Ranking Visualizations of Correlation
![Page 35: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/35.jpg)
35/45 Remco Chang – Sandia 14
Mixed Initiative (Adaptive) Systems
4. What Can a System DoIf It Knows Everything About Its User?
![Page 36: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/36.jpg)
36/45 Remco Chang – Sandia 14
(Human+Computer) Visual Analytics
User
Intent(Model)
InteractionData
(Model)
Visualization
Discovery
Wal
do
Dis-Function
Adaptive Visualization
Adaptive Computation
![Page 37: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/37.jpg)
37/45 Remco Chang – Sandia 14
Adaptive Visualization
• Color-Blindness, Cultural Differences, Personality, etc.• Cognitive Workload
Afergan et al., Dynamic Difficulty Using Brain Metrics of Workload. CHI 2014
![Page 38: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/38.jpg)
38/45 Remco Chang – Sandia 14
Adaptive Computation
• A new approach for Big Data visualization
• Observation: Data is so large that… – There are more data items than there are pixels– Each computation (across all data items) takes tremendous
amount of time, space, and energy
• Solution: User-Driven Computation– Conserve these precious resources by computing “partial”
information based on User and Data Models
![Page 39: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/39.jpg)
39/45 Remco Chang – Sandia 14
Example Problem: Big Data Exploration
Visualization on aCommodity Hardware
Large Data in aData Warehouse
![Page 40: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/40.jpg)
40/45 Remco Chang – Sandia 14
Example 1: JND + Streaming Data
• Streaming visualization (Fisher et al., CHI 2012)
• JND-based streaming data and visualization– Stop the computation and
streaming at JND– Similar to audio (mp3), image
(jpg2000), graphics (progressive meshing)
– Differ in that the JND will be based on semantic information (e.g. correlation)
t = 1 second t = 5 minute
![Page 41: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/41.jpg)
41/45 Remco Chang – Sandia 14
Example 2: Predictive Pre-Computation and Pre-Fetching
• In collaboration with MIT and Brown• Using an “ensemble” approach for prediction
– Large number of prediction algorithms – Each prediction algorithm is given more computational resources based on past
performance• Evaluated system with domain scientists using the NASA MODIS dataset (multi-
sensory satellite imagery)• Remote analysis on commodity hardware shows (near) real-time interactive
analysis
![Page 42: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/42.jpg)
42/45 Remco Chang – Sandia 14
Summary
![Page 43: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/43.jpg)
43/45 Remco Chang – Sandia 14
Summary• “Interaction is the analysis”1
• A user’s interactions in a visual analytics system encodes a large amount of data
• Successful analysis can lead to a better understanding of the user
• The future of visual analytics lies in better human-computer collaboration
• That future starts by enabling the computer to better understand the user
1. R. Chang et al., Science of Interaction, Information Visualization, 2009.
![Page 44: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/44.jpg)
44/45 Remco Chang – Sandia 14
Summary
“Reverse engineer” the human cognitive black box (by analyzing user interactions)
A. Data Modeling– Interactive Metric Learning
B. User Modeling– Predict Analysis Behavior
C. Perception and Cognition– Perception Modeling – Cognitive Priming
D. Mixed Initiative Systems– Adaptive Visualization – Adaptive Computation
![Page 46: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/46.jpg)
46/45 Remco Chang – Sandia 14
Backup
![Page 47: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/47.jpg)
47/45 Remco Chang – Sandia 14
Priming Inferential Judgment
• The personality factor, Locus of Control* (LOC), is a predictor for how a user interacts with the following visualizations:
Ottley et al., How locus of control influences compatibility with visualization style. IEEE VAST , 2011.
![Page 48: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/48.jpg)
48/45 Remco Chang – Sandia 14
Locus of Control vs. Visualization Type
• When with list view compared to containment view, internal LOC users are:– faster (by 70%)– more accurate (by 34%)
• Only for complex (inferential) tasks• The speed improvement is about 2 minutes (116 seconds)
![Page 49: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/49.jpg)
49/45 Remco Chang – Sandia 14
Priming LOC - Stimulus
• Borrowed from Psychology research: reduce locus of control (to make someone have a more external LOC)
“We know that one of the things that influence how well you can do everyday tasks is the number of obstacles you face on a daily basis. If you are having a particularly bad day today, you may not do as well as you might on a day when everything goes as planned. Variability is a normal part of life and you might think you can’t do much about that aspect. In the space provided below, give 3 examples of times when you have felt out of control and unable to achieve something you set out to do. Each example must be at least 100 words long.”
![Page 50: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/50.jpg)
50/45 Remco Chang – Sandia 14
Results: Averages Primed More Internal*
Visual Form
List-View Containment
Performance
Poor
Good
Internal LOC
External LOC
Average ->Internal
Average LOC
Ottley et al., Manipulating and Controlling for Personality Effects on Visualization Tasks, Information Visualization, 2013
![Page 51: Analyzing User Interactions for Data and User Modeling](https://reader036.vdocuments.mx/reader036/viewer/2022062521/568156d9550346895dc4791b/html5/thumbnails/51.jpg)
51/45 Remco Chang – Sandia 14
Results