vass2012 fisher
Post on 26-May-2015
114 Views
Preview:
TRANSCRIPT
Leave BlankVisually-enabled Distribution of
Cognition
Brian Fisher
SFU Interactive Arts & Technology / Cognitive ScienceUBC Media & Graphics Interdisciplinary Centre Calit2 Visiting Scholar, University of California 2012
1
Leave Blank
2
1965 Visual Analytics
Individuals who operate effectively in our culture have already been considerably "augmented." Basic human capabilities for sensing stimuli, performing numerous mental operations, and for communicating with the outside world, are put to work in our society within a system--an H-LAM/T system--the individual augmented by the language, artifacts, and methodology in which he is trained. Furthermore, we suspect that improving the effectiveness of the individual as he operates in our society should be approached as a system-engineering problem--that is, the H-LAM/T system should be studied as an interacting whole from a synthesis-oriented approach.
Doug Engelbart2
Leave Blank
On the Death of Visualization (2004)
Can It Survive Without Customers?
• Visualization, alone, is not a solution.
• Visualization is a critical part of many applications.
• Visualization, the Community, lacks application domain knowledge.
• Visualization has become a commodity.
• Visualization is not having an impact in applications.
Text
Bill Lorensen
3
Leave Blank
“This science must be built on integrated perceptual and cognitive theories that embrace the dynamic interaction between cognition, perception, and action. It must provide insight on fundamental cognitive concepts such as attention and memory. It must build basic knowledge about the psychological foundations of concepts such as ‘meaning,’ ‘flow,’ ‘confidence,’ and ‘abstraction.’ “
“Illuminating the Path” (IEEE Press)
“The science of analytical reasoning facilitated by interactive visual interfaces”
Visual Analytics
4
Leave Blank
Cybernetics
• 1948 Hixon Symposium, Macy Conferences
• Use control theory, information theory to explain cognitive and social phenomena
• Gurus: Weiner, McColloch & Pitts, Ashby, Shannon, von Neumann, Turing
• Application people: Bateson, Mead
On the Origins of Cognitive Science: The Mechanization of the Mind by Jean-Pierre Dupuy
5
Leave Blank
The study of thought, learning, and mental organization, which draws on aspects of psychology, linguistics, philosophy, and computer modelling. (OED)
Cognitive Science Society founded in 1979
Cognitive Science
6
Leave Blank
• Daniel Bobrow - AI• Eugene Charniak - AI• Allan Collins - Psychology• Edward Feigenbaum - AI• Charles Fillmore - Linguistics• Jerry Fodor - Philosophy• Walter Kintsch - Psychology• Donald Norman - Psychology• Zenon Pylyshyn - Psychology• Raj Reddy - AI• Eleanor Rosch - Psychology• Roger Schank - AI
Founders
7
Leave BlankKey figures not at
Dallas
AI Linguistics Neurosci Philosophy Psychology
Chomsky
Miller
Minsky
Newell
Simon
✓ ✓✓ ✓ ✓ ✓✓ ✓ ✓✓ ✓✓ ✓ ✓
8
Leave Blank
9
Leave Blank
• A division of labour?• Simon’s“nearly decomposable
problems”
• Conceptual & methodological “trading zone”?• Galison: Image and Logic: a Material
Culture of Microphysics• Thagard: Trading Zones in Cognitive
Science
What Happens in the Maze?
10
Leave Blank
• Unified Theories of Cognition (Newell)• Models = theories• Models should explain a range of
behaviours • “You can’t play 20 questions with
nature and expect to win”
• Fodor “Modularity of Mind” 1980• Egon Brunswick “Conceptual
Framework of Psychology” 1952
Cognitive Architecture
11
Leave Blank
Time scale (sec)
Psychologicaldomain
• Unit task• Operations• Attention
107
106
105
104
103
102
101
100
10-1
• SOCIAL
• RATIONAL(Adaptive)
• COGNITIVE(Proximal Mechanisms)
• Task
(Newell)Time scales
12
Leave BlankNew experiences,old
brains
• Transistor density doubles every 24 months
• Disk density doubles every 12 months
• Brain volume doubles every 3 x107 months
Intuitions about architectural processing are inaccurate = “metacognitive gap”
13
Leave Blank
Controller/display systems in air traffic
control • NextGen ATC
“fishtank” projection • Change camera
position for better view
• How will global motion affect tracking? Liu, G. Austen, E. L., Booth, K.S. Fisher, B., Argue, R. Rempel, M.I., & Enns, J. (2005) Multiple Object Tracking Is Based On Scene, Not Retinal, Coordinates. Journal of Experimental Psychology: Human Perception and Performance. 31(2), Apr 2005, 235-247.
http://www.youtube.com/watch?v=tKJVB4id_TY
14
Leave BlankFINST theory of spatial indexing
15
Leave BlankMultiple object
tracking (Pylyshyn)
16
Leave Blank3-D Projected
display
17
Leave BlankTest Performance at
Different Speeds
18
Leave Blank
Fit human tracking function (Lui)
19
Leave Blank... Then add display
motion
20
Leave BlankTracking vs object
speed
21
Leave Blank
Tracking in warped space
22
Leave BlankTracking in warped
space
23
Leave BlankConclusion: We track in allocentric space
• Retinal speed of targets does not determine performance
• Motion of targets relative to each other does
• But only if motion preserves good metric characteristics of space
• Explanation is at the level of a human - display cognitive system
24
Leave Blank
Optimizing analytic systems
• Personal Equation of Interaction• Test users• Model data• Describe individual differences in model parameters
• Customize display for PEI• Attentive systems adapt PEI
• Within a session: fatigue, attentiveness• Between sessions: aging changes
• Personal Equation of Interaction can support training and selection as well as tuning interaction
25
Leave BlankLimits of
Cyberpsychology
• We show how we can lower the bar for adoption• Reduce attentional demands• Customization for the individual• Adaptation to conditions of use• Support perceptual, cognitive, interactive
expertise
• ... but what if the users are bears?
26
Leave BlankD-Cog origins
• Cognition in the Wild 1995 • Navy ships and canoes
• Cognition distributed among personnel on the ship
• Cognition in the head in canoe navigation
• System of measurement, communication, confirmation, protocol navigates ship
• Challenge to cogsci: Mind-as-computer misses cognition
27
Leave BlankAnalytics in the Wild
• Emergency Management (NSERC, DHS)• Mobile analytics / sensor analytics• “Virtual EOC” visual analytic environment
• Aircraft Safety, Reliability (Boeing/MITACS)• “Pair analytics” of complex quant and text data
• Economics and finance (MITACS, NSF)• Behavioural economics (portfolios) • Systemic risk analysis
• Healthcare Monitoring & Management (DHS)• Complex data in health research (CFRI)• Public health management (BC Injury Research & Prevention)
28
Leave Blank
"Il n'existe pas une catégorie de sciences auxquelles on puisse donner le nom de sciences appliquées. Il y a la science et les applications de la science, liées entre elles comme le fruit à l'arbre qui l'a porté"
Louis Pasteur
Pure Basic Research(Bohr)
Use-inspired Basic
Research (Pasteur)
Sampling,Description,Taxonomy(Audubon)
Pure Applied Research (Edison)
Quest for Fundamental Understanding?
No
Yes
Consideration of Use ?
No
Yes(1822–95)
Pasteur’s Quadrant
(Stokes)
29
Leave Blank
How can D-Cog help design?
• Perspective for designers• Interpret theories from cognitive sciences for use
in design• Activity Theory-> Michael Cole, Yrjö Engeström• D-Cog -> Yvonne Rogers, Jim Hollan
• Develop new, more targeted theories & methods• Translation from “clinic” to lab and back again
30
Leave BlankD-Cog at Cogsci• Pluralistic nature of
Cogsci makes it difficult to exclude new approaches
• Also makes it difficult to come to a conclusion
• CogSci 2013 at Humboldt University in Berlin Jul 31 - Aug 3.
31
Leave Blank
• A division of labour?
• Conceptual & methodological “trading zone”?• Galison & Thagard
• A “Mangle of Practice”?• Pickering (Exeter)
What Happens in the Maze?
32
Leave Blank
Pickering’s Mangle of Practice
• Mangle 2 |ˈmøŋgəәl| |ˈmaŋg(əә)l| noun
• A large machine for ironing sheets or other fabrics, usually when they are damp, using heated rollers.
• chiefly Brit. a machine having two or more cylinders turned by a handle, between which wet laundry is squeezed (to remove excess moisture) and pressed
33
Leave Blank
CognitionPerceptual
ScienceMethods
SocialScienceMethods
Computation, Visualization
Methods
Graphic & Interaction
DesignMethods
Cognitionin the wild
Mangle of Visual Analy0cs Research?
34
34
Leave Blank
“For five decades I have been driven by an intuitive certainty that computer supported argumentation could increase humankind's collective problem-solving capabilities to a degree that was (is) greatly unappreciated, and that its explicit pursuit should become one of society's high-priority, "grand challenges".
Douglas Engelbart, 2003
35
Leave Blank
Putting theory into practice (and practice into theory)
36
36
Leave Blank
CognitionPerceptual
ScienceMethods
SocialScienceMethods
Computation, Visualization
Methods
Graphic & Interaction
DesignMethods
Cognitionin the wild
Mangle of Visual Analy0cs Research?
37
37
Leave BlankD-Cog VA lab
• Dr. Richard Arias-Hernández
• Dr. Linda Kaastra• Dr. Nathalie Prevost• Samar Al-Hajj• Nadya Calderón • Tera Marie Green• Ali Khalili• Hon Cheong Lam
• Amanda Pype• Aaron Smith• Numerous SFU &
UBC student analysts
38
Leave BlankLab Mascot
39
Leave BlankStudent analysts
• Best debrief VAST 2007
• Discovery Exhibition Best paper 2010 (Andrew Wade) & 2011 (Samar Al-Hajj)
• Analytics are extracted and communicated to tech developers as methods & prototypes
40
Leave Blank
• Applied Visual Analyst (2013)• Perceptually rich human-information discourse• Tech-mediated social cognition & collaboration• Learning model, personal equation
• Analytic Designer• Develop VA technologies• Customize display & interaction for user & task• Consult on organizational roles, communication
• Analytic Researcher• Advance the field
Educational Programs
41
Leave BlankAndrew Wade VA
Challenge• Aircraft Birdstrike Incident Data• Aircraft Pitot Tube Icing Data• Immunology - Flow Cytometry Data• Healthcare Records Analysis Data• Domestic Violence Data• Risk Assessment – Financial Data• Aircraft Hazardous Bills of Materials Data• Corporate PC Performance Data• UBC On-line Course Performance Data• Justice System Prisoner Management
42
Leave Blank
• “Pair analytics” sessions• Student visual analyst &
trained domain expert collaborate on analytic task
• Student “drives”, expert “navigates”
• Video session & capture screen
Setting the Stage
Arias-Hernandez, R, Kaastra, L.T., and Fisher, B. (2011) Joint Action Theory and Pair Analytics: In-vivo Studies of Cognition and Social Interaction in Collaborative Visual Analytics. In L. Carlson, C. Hoelscher, and T. Shipley (Eds.), Proceedings of the 33rd Annual Conference of the Cognitive Science Society (pp. 3244-3249). Austin TX: Cognitive Science Society.
43
Leave BlankBird Strikes
44
Leave Blank
45
Leave Blank
46
Leave Blank
47
Leave Blank
Tableau Bird Strike Pair Analysis
48
Leave Blank
IN-SPIRE Bird Strike Pair Analysis
49
Leave BlankWade Internship
• Video recorded and screen captured over 10 Paired Analysis sessions using both Tableau and IN-SPIRE
• Influenced design decisions on:• 777• P8-A• 787• 747-8
• Changes to pilot training manual
50
Leave BlankWade Internship• Presented work to:
• 787 Engineers• Aviation Safety Community of Practice• Aerodynamics, Performance, Stability and Control
flight data recorder analysis group• Advanced Analytics group• UW Aeronautics and Astronautics students • Boeing Educational Network webcast (400+)
• 500+ people exposed to Visual Analytics, Paired Analysis for Aviation Safety
51
Leave Blank
• Build on social science (GT, JAT) approaches to understand organizations and cognitive work practices.
• The innovation here is in the extension of social science to bridge to the perceptual and cognitive science theories that apply to the use of Cognitive Science in analytical tasks.
Understand technological
distribution of cognition
SocialSciences
52
Leave BlankJoint Activity Theory
(Clark)
• Language is an essentially collaborative activity, like playing duet or paddling canoe
• We work to build common ground so as to communicate effectively and efficiently
• Clark’s theory:• Defines kinds of common ground• Formalizes the notion of activity as a “joint action”• Describes the processes by which common
ground is developed through joint action
53
Leave Blank
54
Grounding
• Common ground: set of things mutually believed by both speaker and hearer
• Hearer must ground or acknowledge understanding of speakers utterance.
• Clark (1996):• Principle of closure. Agents performing an action require
evidence, sufficient for current purposes, that they have succeeded in performing it
• Need to know whether an action succeeded or failed
54
Leave Blank
Metacognition in communication
• Grounding mechanisms to establish common referents• Inflection, gesture, expression• Hesitation, pausing phrases (e.g. “um”)• Misc. visual and auditory cues• Explicit signals
• Speakers change their methods based upon unconscious assessments of these cues
55
Leave BlankGeneralizing Clark
• Clark’s theory draws upon joint activity metaphors (paddling, music) but focuses on spoken language
• Kaastra applied that framework to musical performance analysis
• Will it work for analytic duet?
56
Leave BlankJAT Analysis of Pair Session
• Structuring and navigation markers: • “Vertical markers” are verbal gestures, such
as “okay,” and “all right,” that signal transitions between different analytical tasks.
• “Horizontal markers,” such as “yeah” and “mhmm,” are used to signal continuation within a singular analytical task
57
Leave Blank
• Management of joint attention: • Gaze, finger-pointing and mouse-point are used by a
speaker to direct joint attention, and by a listener to confirm that joint attention is in place.
• Use of “self-talk” with on-screen gesturing to inform about progress on the execution of a cognitively demanding task: Pauses in analysis were accompanied by “self-talk” of participants on-task:
• We are currently studying and categorizing “self-talk” occurrences in PA as indicators of cognitive workload.
JAT Analysis of Pair Session
58
Leave Blank
Emergency Management
• Coordinated technological, methodological, organizational & training support
• Many technologies w/o rich visualization-- small form factor devices, sensors, data input.
• Example: VA for Emergency Management NSERC SPP (+ 2 SPP companion proposals)• Population: cell phones• First responders: blackberries • Data fusion centres: geotagged sensor networks, big data
processing• Command centres: interactive tabletops, walls, etc.
“Your users are bears”
59
Leave Blank
Visual analytics in the Wild
• Emergency management for mixed-bear systems• RAH to JIBC, train at Richmond EOC• CREATE-ERE cross-border earthquake
“experiment”• Richmond, Vancouver, Provincial EOC• Field reports, interviews• Build Advanced Interaction EOC• Virtual EOC to come
60
Leave BlankVA is more than
technology!• VA proposes not just new technologies but
new analytic methods • These new methods will change structures
of communication, selection, & training• New analytic systems need new analysts (Boeing) • Analysts take on new roles, organizations change
• VA must address human, organization, and technology (this includes universities!)
61
Leave BlankTranslational Science of
Analytical Reasoning
• VA must develop scientific methods that help us to augment analytical reasoning
• VA includes human, organization, & tech• Human: perceptual/cog/motor “cyberpsychology”• Community: social science of information flow• Technology: software design & engineering 1&2
• Methods to bridge these levels of analysis
Fisher, B., Green, T.M., Arias-Hernández, R. (2011) "Visual analytics as a translational cognitive science," Topics in Cognitive Science 3,3 609–625.
62
Leave BlankInjury Prevention & Response Group
Analysis• BC Injury Research & Prevention Unit (Dr Ian Pike)• Goals:
• Enhance stakeholders’ understanding of injury indicators.
• Support multi-party analysis.• Support multi-party decision-making
• Visual Analysis Expert (Samar Al-Hajj) will:• Manage and guide the group analysis • Offer stakeholders different approaches to
solving analytical problem.63
Leave Blank
Child Injury Indicators
Indicators Spanning Across All Domains
1. Mortality Rate2. Potential Years of Life Lost3. Hospital Separations Rate
Overall Health Service Implications Indicators
4. Diagnosis-Specific Hospital Separations5. Hospital Admission - Injury Severity 16. Hospital Admission - Injury Severity 27. Length of Stay in Hospital
Motor Vehicle Injury Indicators
8. Cost of Motor Vehicle Injuries9. Crash Rate10. Intersection Crash Rate11. Rural Roadways12. Drunk Driving13. Speed14. Young Drivers15. Graduated Driver Licensing16. Child Restraints17. Unrestrained Injuries18. Child Restraint Laws
Sport, Recreation and Leisure Injury Indicators
19. Bicycle Helmet Laws20. Cost of Sport, Recreation and Leisure Injuries21. Percentage of Sport Specific Injuries (Participation Rates)22. Requirements that Playgrounds Meet CSA Standards23. Legislations Requiring Pool Fencing
Other Policy Indicators 24. Window Guard By-law25. Provincial Standards for Hot Water Tap Temperature
Violence Indicators26. Violent Crime Rate27. Abusive Head Trauma Rate28. Suicide Prevention29. Anti-Violence/Anti-Bullying Policies
Trauma Care, Quality and Outcomes
Indicators
30. Access to Pediatric Trauma Centre (PTC)31. Appropriate Use of Pediatric Trauma Centre (PTC)32. Quality of Trauma System33. Pre-hospital Transport Time34. Presence of a Coordinated Pediatric Trauma System
64
Leave Blank
65
Leave Blank
66
Leave BlankVisual Rhetoric
• Child injury prevention users are creating and interpreting policies
• Can we build a visual rhetoric for them?• Concept mapping & related methods• Diagrammatic reasoning, visual communication• Visualizing argumentation• Modal logics (deontic, temporal,doxastic)
• Can we bridge the analyses?
67
Leave Blank
• We certainly do not advocate that business schools, in revising MBA curricula, abandon science. Rather, they should encourage and reward research that illuminates the mysteries and ambiguities of today's business practices. Oddly, despite B schools’ scientific emphasis, they do little in the areas of contemporary science that probably hold the greatest promise for business education: cognitive science and neuroscience.
“How Business Schools Lost Their
Way”
Text
Bennis & O’Toole, HBR, 2005
68
Leave BlankNSF Science of
Interaction
• Mixed-initiative data manipulation and discovery (thinking about data)
• Collaborative analytics • Multi-modal sensemaking • Fluid interaction • Visual discourse (rhetoric)
69
Leave Blank
Decision Support & Operational
Management Analytics mini-track
70
top related