Information Visualization

CSCI 6174: Open Problems in CS

Fall 2011

Richard Fowler

Ya gotta visualize …

• I see what you mean …

– so, visualization can be considered not just a visual process, but a cognitive (thought) process as well

• And a very large part of human brain taken up with visual system

– and that part of the brain is still useful beyond “simply” getting an image of the world

– … which is in fact pretty complicated

Visualization is …

• Visualize:– “To form a mental image or vision of …”– “To imagine or remember as if actually seeing …”– Firmly embedded in language, if you see what I mean

• (Computer-based) Visualization:– “The use of computer-supported, interactive, visual

representations of data to amplify cognition”• Cognition is the acquisition or use of knowledge• Card, Mackinlay Shneiderman ’98

– Scientific Visualization: physical

– Information Visualization: abstract

Visualization is not New

• Cave guys, prehistory, hunting

• Directions and maps

• Science and graphs– e.g, Boyle: p = vt

• … but, computer based visualization is new– … and the systematic delineation of the design

space of (especially information) visualization systems is growing nonlinearly

Visualization and Insight

• “Computing is about insight, not numbers”– Richard Hamming, 1969– And a lot of people knew that already

• Likewise, purpose of visualization is insight, not pictures– “An information visualization is a visual user

interface to information with the goal of providing insight.”, (Spence, in North)

• Goals of insight– Discovery– Explanation– Decision making

“Computing is about insight, not numbers”

• Numbers – states, %college, income:State % college degree income State % college degree income

“Computing is about insight, not numbers”

• Insights:– What state has highest income?, What is relation between education and income?, Any outliers?

State % college degree income State % college degree income

“Computing is about insight, not numbers”

• Insights:– What state has highest income?, What is relation between education and income?, Any

outliers?

Not about Useless Visual Stuff - Clutter

• “3d” adds nothing– (at best)

Detrimental useless stuff

• USA Today

An Example, Challenger Shuttle

• Presented to decision makers– To launch or not– Temp in 30’s

• “Chart junk”

• Finding form of visual representation is important– cf. “Many Eyes”

An Example, Challenger Shuttle

• With right visualization, insight (pattern) is obvious– Plot o-ring damage vs. temperature

Insight …

• Some examples ….

A Classic Static Graphics Example

• Napolean’s Russian campaign– N soldiers, distance, temperature – from Tufte

For what it’s worth …

• x

Visualization Pipeline:Mapping Data to Visual Form

• Visualizations: – “adjustable mappings from data to visual form to human perceiver”

• Series of data transformations– Multiple chained transformations– Human adjust the transformation

• Entire pipeline comprises an information visualization

RawInformation

VisualFormDataset Views

User - Task

DataTransformations

VisualMappings

ViewTransformations

F F -1

Interaction

VisualPerception

Visualization Stages

• Data transformations:– Map raw data (idiosynchratic form) into data tables (relational descriptions

including metatags)

• Visual Mappings:– Transform data tables into visual structures that combine spatial substrates,

marks, and graphical properties

• View Transformations:– Create views of the Visual Structures by specifying graphical parameters

such as position, scaling, and clipping

RawInformation

VisualFormDataset Views

User - Task

DataTransformations

VisualMappings

ViewTransformations

F F -1

Interaction

VisualPerception

Information Structure

• Visual mapping is starting point for visualization design

• Includes identifying underlying structure in data, and for display– Tabular structure– Spatial and temporal structure– Trees, networks, and graphs– Text and document collection structure– Combining multiple strategies

• Impacts how user thinks about problem - Mental model

RawInformation

VisualFormDataset Views

User - Task

DataTransformations

VisualMappings

ViewTransformations

F F -1

Interaction

VisualPerception

A “Taxonomy” of Visualization

SpacePhysical Data1D, 2D, 3DMultiple Dimensions, >3TreesNetworks

InteractionDynamic QueriesInteractive AnalysisOverview + Detail

Focus + ContextFisheye ViewsBifocal LensDistorted ViewsAlternate Geometry

Data Mapping: TextText in 1DText in 2DText in 3DText in 3D + Time

Higher-Level VisualizationInfoSphereWorkspacesVisual Objects

1D Linear Data

1D Linear Data

2D Map Data

3D World Data

Multiple Dimensions > 3

• “Straightforward” 1, 2, 3 dimensional representations– E.g., time and

concrete

• Can extend to more challenging n-dimensional representations– Which is at core of

visualization challenges

• E.g., Feiner et al., “worlds within worlds”

Temporal Data

Trees, Networks, and Graphs

• Connections between /among individual entities

• Most generally, a graph is a set edges connected by a set of vertices– G = V(e)– “Most general” data

structure

• Graph layout and display an area of iv

• Trees, as data structure, occur … a lot– E.g., Cone trees

Tree/Hierarchical Data• Workspaces

– The Information Visualizer: An Information Workspace by G. R. Robertson, S. K. Card, J. M. Mackinlay, 1991 CACM

Networks

• E.g., network traffic data

• Visualization of NSFNET

• Cox, D. & Patterson, R., NCSA, 1992

• Routes of the Internet, 1/15/05

• The opte project

• Earlier snapshot in permanent collection of NY Museum of Modern Art

• 3-d hyperbolic tree of web sites using Prefuse

Abstract – Non-physical

• Concept map– Graph of

“conceptual” information

• From Berners-Lee’s proposal to CERN for what is now called www, March 1989

• Manual “graph drawing”

http://www.nic.funet.fi/index/FUNET/history/internet/w3c/proposal.html

FYI - Demo

• http://thejit.org/

Text and Document Collection Structure

• Derivation of relationships upon which display is to be based a challenge

• E.g., Wise et al

Text and Document Collection Structure, e.g., Galaxy of News

• x

Overview Strategies

• Typically useful, or critical, to have “feel” for all data– Then, allows closer inspection in “context” of all data– Overview + detail, focus + context

• Known from the outset of visualization– Bifocal Lens

• Database navigation: An Office Environment for the Professional by R. Spence and M. Apperley

• Shneiderman mantra– “overview first, zoom and filter, details on demand”

Focus+Context: Fisheye Views, 1

• Detail + Overview – Keep focus, while remaining aware

of context

• Fisheye views– Physical, of course, also ..– A distance function. (based on

relevance)– Given a target item (focus)– Less relevant other items are

dropped from the display– Classic cover

• New Yorker’s idea of the world

Focus+Context: Fisheye Views, 2• Detail + Overview

– Keep focus while remaining aware of context

• Fisheye views– Physical, of course, also ..– A distance function. (based on relevance)– Given a target item (focus)– Less relevant other items are dropped from

the display – Or, are just physically smaller – distortion

Focus + Context – Spatial Distortion

• Selectively reduce complexity as f(user’s viewpoint)

• Spatial distortion– Project network

on distorted space

• Viewing “lens”

Focus + Context – Spatial Distortion

• Selectively reduce complexity as f(user’s viewpoint)

• Spatial distortion– Project network

on distorted space

• Viewing “lens”

• Seamless transition

Focus + Context – Hyperbolic View

• Again, selectively reduce complexity as f(user’s viewpt.)• Smooth change during interaction

Focus + Context – Hyperbolic View

• Also, in 3 space

• Demo

• 3-d hyperbolic tree of web sites using Prefuse

Tools

IBM’s Many Eyes

• Multiple visualizations

IBM’s Many Eyes

• Visualization types

IBM’s Many Eyes

• Life expectancy vs. health care costs

• http://manyeyes.alphaworks.ibm.com/manyeyes/visualizations/life-expectancy-vs-per-capita-annu

Visualization Pipeline:Mapping Data to Visual Form

• Visualizations: – “adjustable mappings from data to visual form to human perceiver”

• Series of data transformations– Multiple chained transformations– Human adjust the transformation

• Entire pipeline comprises an information visualization

RawInformation

VisualFormDataset Views

User - Task

DataTransformations

VisualMappings

ViewTransformations

F F -1

Interaction

VisualPerception

Visualization Stages

• Data transformations:– Map raw data (idiosynchratic form) into data tables (relational descriptions

including metatags)

• Visual Mappings:– Transform data tables into visual structures that combine spatial substrates,

marks, and graphical properties

• View Transformations:– Create views of the Visual Structures by specifying graphical parameters

such as position, scaling, and clipping

RawInformation

VisualFormDataset Views

User - Task

DataTransformations

VisualMappings

ViewTransformations

F F -1

Interaction

VisualPerception

Information Structure

• Visual mapping is starting point for visualization design

• Includes identifying underlying structure in data, and for display– Tabular structure– Spatial and temporal structure– Trees, networks, and graphs– Text and document collection structure– Combining multiple strategies

• Impacts how user thinks about problem - Mental model

RawInformation

VisualFormDataset Views

User - Task

DataTransformations

VisualMappings

ViewTransformations

F F -1

Interaction

VisualPerception

Information Vis Systems at UTPA

Information Vis Systems at UTPA

• Data mining, VAS - Visual Analysis System - Hubs and authorities

• Text visualization, ATV - Abstract Text Viewer - Tag clouds and such

• Clinician’s tool for personality, DID-TM– Dissociative Identity Disorder – Trait Mapper, Visualizing personality

• Reading: Knowledge domain citation and semantic structure– Knowledge worker’s tool

– Selectively varying density in graph visualization

– Perceiving organization

• Reports available on web site

Data mining, VAS - Visual Analysis System

Data mining, VAS - Visual Analysis System

• Data mining, VAS - Visual Analysis System– Hubs and authorities

• Emphasizes effort on data– Collection and transformation to form dataset

RawInformation

VisualFormDataset Views

User - Task

DataTransformations

VisualMappings

ViewTransformations

F F -1

Interaction

VisualPerception

Data Mining: Hubs and Authorities

• Attempt to overcome shortcomings of text indexed search engines

• Graph and cluster based approach

• Link structure of WWW – “latent human annotation”

• Link to page implicit “endorsement” of page

• Web as directed graph

• Based on link structure, characterizes pages as:1. “Authorities”

- best sources of information- high indegree (refined)

2. “Hubs”- provide collections of links to authorities- high outdegree (refined)

The System

• Goal: Allow user to rapidly and incrementally assess utility of web pages– Data mining (hubs and authorities)– Visualization– Filtering – User interaction tools

System Architecture

W W W Search Engine

User Interact

User QueryQuery Results

Fetch Pages

Hub Scores

Layout Pages

Filter for Display

• Goal: Allow users to systematically and incrementally access web pages

W W W Search Engine

User Interact

User QueryQuery Results

Fetch Pages

Hub Scores

Layout Pages

Filter for Display

• Goal: Allow users to systematically and incrementally access web pages

RawInformation

VisualFormDataset Views

User - Task

DataTransformations

VisualMappings

ViewTransformations

F F -1

Interaction

VisualPerception

Example Screen

• Only pages of highest hub and authority scores

• Red: Hubs• Blue:Authorities• User can select

pages

Example Screen - Detail

ATV - Abstract Text ViewerText Visualization

Tag cloud from infovis wiki

ATV - Abstract Text ViewerText Visualization

• Electronic presentation of text for a generation– Ubiquitous

• Manuals, web document/pages, books, …

– Surprisingly few tools for augmenting

• ATV:– Text reading tool for

electronic documents– Uses well-known and

novel techniques

ATV Electronic Presentation Techniques

• Overview + Detail– Facilitates orientation and navigation– Works for spatial data and text

• Abstract text’s content and use to organize– Enhance reader’s efficiency and effectiveness– Use existing elements: HTML tags– Use system derived elements: keywords, …

Paragraph View

• ATV is a browser

• Left for structure & content, “overview”

• Right for enhanced text, “detail”

HTML Structure View

• Headings reveal structure (outline)

• Entire document available

Link View

• All links (navigation elements) available

Word Frequency View

• Crawler reads domain

• Words above threshold in domain listed

• Overview of domain

Word Frequency View

• Words with frequencies > 2 thresholds displayed

Detail (Text) Window

• Darkness of text = f(relatedness) to entire document

• Similarity of paragraph to entire document

Detail (Text) Window

• Word search provided

ATV Conclusions

• Testbed for implementing and testing text abstraction and viewing techniques

• Currently provides tools targeting HTML documents

• Extension to non-marked documents

• Platform for usability testing

DID-TM

Clinician’s tool for personality, DID-TM

• Clinician’s tool for personality, DID-TM– Dissociative Identity Disorder – Trait Mapper– Visualizing personality

• Dissociative Identity Disorder – Trait Mapper

• Tool for clinician use– Manage complexity of case history– Show visually state and progress of client in integrating identities

• Well known visualization techniques– E.g., parallel coordinates

• Novel techniques– E.g., coding of communication and shift over time

DID-TM

• Personality profiles• Identity communication graph• Stored and indexed clinician’s notes

Visualizing Knowledge Domain Structure

• Knowledge worker’s (or anyone’s) tool– Yet again, managing large amounts of information

• -Tools for organizing knowledge domain– E.g., scientist (or student) learning about a new domain– Become acquainted with literature or find new relations

and information– Citeseer

• Exploring and retrieving information– Visual representation of citation network– Visual representations of semantic similarity of documents– Similar to Document Explorer

Network Visualization

Visualizing Knowledge Domain Structure

• Exploring and retrieving information

– Visual representation of citation network• Relationships among documents as shown by citations (references)

– Visual representations of document semantic similarity network

• Semantic document network• Again, relations, now based on content similarity

RawInformation

VisualFormDataset Views

User - Task

DataTransformations

VisualMappings

ViewTransformations

F F -1

Interaction

VisualPerception

Extracting and Organizing Content

• Networks: 1. Citations form graph 2. Document similarity

– Word cooccurrence– Similarity of Documents

• Compare all pairs of documents• Use distance matrix to derive

network

• Network density varies

RawInformation

VisualFormDataset Views

User - Task

DataTransformations

VisualMappings

ViewTransformations

F F -1

Interaction

VisualPerception

Displaying the Networks:Node Positioning using Spring Layout

• Physical spring analog– “Spring Embedder” algorithm

• Can vary spring length, strength, elastic properties– E.g., document similarity

• Example at right in 3D• Interaction by movement

RawInformation

VisualFormDataset Views

User - Task

DataTransformations

VisualMappings

ViewTransformations

F F -1

Interaction

VisualPerception

Network Visualization

• Visualizing Knowledge Domain Citation and Semantic Structure

• Citeseer Visualization

• 1,138 documents from Citeseer collection

• Citation network– Nodes are documents, links are citations (references)– Here, links are weighted by document similarity

Citeseer Visualization - Query• Query “information visualization”• Results used to form citation graph and visual representation

Citeseer Visualization – Results as MST

• Minimum cost spanning tree (graph) used to represent query results

Earlier Network Display & Interaction Tools

• Overviews– Nodes of highest degree– Landmarks: Visible, selectable

• Bookmarks– Set & return to viewpoint

• Fluid motion

• Network density selectable

• Anchors– User-defined selectable

• Signposts– Anchor labeled with overview

nodes– Global orientation at level of

local detail

• Expand and Collapse Nodes

• Color

Display & Interaction Tools, cont.

• Stereo Viewing

– LCD glasses

– Head tracked, “look around”

– Compromise immersion for text tasks

Selective Density in Network Viewing

Overview

• Reducing and managing network density for visualization– Varying structural density– Distorted space display techniques

• Deriving quantitative metrics from documents– From which document network created

• Pathfinder networks– Path length limited minimum cost networks

• A new hybrid representation to selectively vary density

• Internet alliances and partnershships, 2002

• www.orgnet.com/netindustry.html

• Trade relationships, 1992

• www.

Reducing and Managing Density

• Focus + context techniques “selectively” reduce density– User’s view affects display– Spatial distortion techniques

• Use same network, but change space upon which it is projected– Change network structure itself, depending on where focus is

• E.g., Furnas’ 1986 account of fevs, display as f(distance from focus)

• Threshold techniques – Display only links with weights > some value

• As part of structure derivation (network formation)– E.g., minimum cost spanning tree (MCST)

• Limiting case for connected graph– Pathfinder networks

Focus + Context – Hyperbolic View

• Again, selectively reduce complexity as f(user’s viewpt.)• Smooth change during interaction

Threshold

• Reduce complexity by eliminating links < some threshold– Not necessarily preserve connectivity

http://www.g3tvu.co.uk/Network,_Radio_Link_and_Route_Styles.htm

Varying and Reducing Density

• As shown, can vary display space and locally (selectively) reduce density– Distortion techniques

• Also, can reduce density globally (overall) – Link weight threshold, as shown

– Minimum spanning trees

– Here, Pathfinder networks

• Goal of work is to create representation that uses structural (vs. display) manipulation to provide global context and local detail

PfNets – Path Length Limited

• For some data set of distances– Here, data are provided by human subjects– Document network uses interdocument distances

• Construct network that is sufficiently dense that any node can be reached from any other node in q links

• q = n-1

Schvaneveldt et al., 1989

PfNets – Path Length Limited

• Smaller q– Denser graph

Schvaneveldt et al., 1989

Graph Display Considerations

• Graph display issues critical in visualization– And a field in itself

• Force directed layout– Widely used

• E.g., prefuse

• Works well for sparse graphs– Shows global relations well– Not so well for dense

PfNets forGlobal Context and Local Detail

• Combine sparse pfnet (inf, n-1) with more dense at point of interest

• Provide detail + context

Hybrid Pfnets

• Sparse– overv

iew.TIF

Hybrid Pfnets

• Dense

Hybrid Pfnets

• Dense zoomed in

Hybrid Pfnets

• Combined

Perception of Organization

Perception of Organization

• Self organizing systems

• Simple rules, complex behaviors

• Social insects– Ants, bees

• Flocks of birds– Fairly well modeled with few constraints– Coherence (cohesion) of flock– Distance from another individual– Direction

• What are the roles of the elements of organization used by people?

Perception of Organization

– Coherence (cohesion) of flock - Distance from another individual– Direction - Stereoscopy

Perception of Organization

– Coherence (cohesion) of flock - Distance from another individual– Direction - Stereoscopy

BONUS!

• Immersive interfaces, prescence, …

• New research effort

Introduction

• The “best” interfaces, and all systems, typically find their task utility through engagement (etc.) appropriate for the task

– This idea is at the core of arguments for the use of direct manipulation interfaces

• All of the following are interrelated:– Immersion, engagement, presence, virtual reality– 3D display and interaction devices

• In field of CS and HCI: “spatial interfaces”

• Will introduce the idea of presence

Immersion, “Virtual Reality”, and Virtual Environments

• Immersive interfaces– High sensory immersion – visual, auditory, haptic, proprioceptive

• “Virtual reality”, or, virtual environments– “Virtual reality is a technology that is used to generate a simulated environment

in digital form... Using the equipment, users are immersed in a totally virtual world.”

– Working definition – an immersive interactive system

• In context of “virtual reality”, immersion usually = spatial immersion

• Note: “Immersion” (and engagement and presence) is a continuum

– Text ... Visual and 3d .. Stereo ... HMD… “jacked in”– Cyberspace

• Term coined by Gibson in Neuromancer• … and in the 21st century, the Matrix

Immersion and Virtual Reality• “The mind has a strong desire to believe that the world it perceives

is real” – Jaron Lanier, among others

• For example, “illusion” (perception) of depth (for spatial immersion)• Stereo parallax• Head motion parallax• Object motion parallax• Texture scale

• Interaction: grab and move an object

• Proprioceptive cues: – when you reach out and see a hand where you believe your hand to be,

you accept the hand as your own

• Often you will accept what you see as “real” even if graphics poor

• Constellation of cues

Presence “The Aesthetic Impression of 3D Space”

• Sense of presence – Vividly 3d– Actually present in the world– Sense of being there– Holodeck …

• Presence has to do as much with engagement, as visual information– E.g., one can be “in the world”, when reading– Here, one sees, or visualizes, the world

• 3D depth cues are those elements that enhance feeling of 3 (vs. 2) dimensions in a display, – From occlusion to stereoscopic display

Presence “The Aesthetic Impression of 3D Space”

• Immersive interfaces– term used to describe interfaces/devices which lead toward immersion

(sense of presence, engagement) in the virtual environment presented on the display

• Virtual reality interfaces– term used similarly to immersive interfaces

• Degree of immersion– conventional desktop screen– fishtank virtual reality (semi-immersive workbench)– immersive virtual reality– augmented reality with video or optical blending– … number of cues …

Immersive and 3D Interfaces• Teleoperation

• Virtual and augmented reality

• Immersion and VR – contribution of components …

• Survey of 3D displays– Surround screen displays - CAVE– Input devices - Data glove– Data walls– Workbenches– Hemispherical display– Head-mounted displays– Arm-mounted displays– Virtual retinal display– Autostereoscopic displays

Sutherland’s 1960’s equipment• Ultimate display, 1965

• Sword of Damocles – 1st HMD– Actual camera-like metal shutters

Virtual and Augmented Reality

• Augmented reality shows real world with an overlay of additional overlay

• Knowlton (1975)

• Partially-silvered mirror over keyboard

• Programmable labels

• Tactile feedback

Augmented Reality, 2

• Enables users to see real world with an overlay of additional interaction– Situational awareness

• See through glasses

• Typically, add text+images to real world

• Very sensitive to head tracking, when used

Surround-screen displays

• Pro• less obtrusive headgear• multi-user?• better stereo• Con• occlusion problem• missing sides

Surround screen displays – CAVE, 1

• A room with walls and/or floor formed by rear projection screens– Head tracking– Stereo– Light scattering

problems

• Visual immersion– Field of view is

100% possible, ~200 degrees

Surround screen displays – CAVE, 2

• Typical size: 10’ x 10’ x 10’ room

• 2 or 3 walls are rear projection screens– Floor is projected from above

• User is – tracked – He/she also wears stereo shutter goggles…– Uses “wand” to manipulate

• Projects 3D scenes for viewer’s point of view on walls

– Walls vanish, user perceives full 3D scene– So, view is only correct for that viewer

• Cost is fairly high

UTPA Immersive Systems Lab~Summer, 2012

Proj.

Proj. Proj.

27’13’

Security area

21’

6’

TV

stor

age

~8’

CAVE

Ph

ysio

log

ica

lM

easu

rem

ent

E

qu

ipm

ent

Computers

Front Projection Screen

Proj.

Develo

pmen

t

Questions?

• .