developing visualization techniques for semantics-based information networks rich keller david hall...
TRANSCRIPT
Developing Visualization Techniques for Semantics-based Information Networks
Rich Keller David Hall NASA Ames QSS Group, Inc.
Information Sharing and Integration GroupComputational Sciences Division
NASA Ames Research CenterMoffett Field, CA
Virtual Iron Bird Workshop, Monterey, CA April 2, 2004
Goals of our Work
• Goal #2 (Engineering): Develop an effective visual interface for an existing NASA information / knowledge management tool
• Goal #1 (Scientific): Understand how semantic knowledge can be exploited to help visualize large network-structured information spaces (… like the Semantic Web)
Talk Outline
• SemanticOrganizer System
• Visualization Problem
• Proposed Semantic Approaches to Visualization Problem
Work in progress!
What is SemanticOrganizer?
• A semantics-based shared information space: designed to support distributed science and engineering project teams
• Facilitates information sharing, integration, correlation and dependency tracking
• Core is a digital information repository: users upload & download heterogeneous information (images, datasets, documents, and various types of scientific/engineering records)
• Features semantic cross-linkage: enables rapid intuitive access to interrelated information; permits linking facts and evidential information to scientific/engineering conclusions
• Serves as organizational memory: preserves details of investigative fieldwork, labwork, & associated data collection/ data analysis activities and processes
Operational Status
• First deployed in 2001
• Over 500 registered individual users from over 50 organizations within NASA
• Over 50 projects hosted
• Over 45,000 information nodes & 150,000 links in repository
• Over 14,000 electronic files stored (documents, image, datasets)
• Over 12,000 archived email messages
as of 4/1/04
Application Features
SemanticOrganizer Applications
SemanticOrganizer
ScienceOrganizer InvestigationOrganizer
NASA Astrobiology Institute
Mars Meteorite Research Team
NIH Malaria Study
Shuttle Columbia Investigation
Helios UAV Investigation
CONTOUR Spacecraft Investigation
MARTE Mars Analog Mission Moffett Airshow Investigation
MER Hypothesis Tracking
Mobile Agents Mars Exploration
ScienceOrganizer
• Real-time equipment control• Automated experimentation
Both• Collaborative image annotation
• Microsoft Office macros• Email lists & archive
InvestigationOrganizer
• Fault tree viewer• Event sequence editor
…
How is the Information Repository structured?
person
photo
measurement
siteinstrument
sample
document
• Links: defined relationships among resources
• Attached files: electronic products associated with resources (in almost any type of file format)
• Attributes: properties of resources (metadata)
• Nodes: key science or engineering resources (describing people, places, systems, hypotheses, evidence)
• date• size• format
Ontology:Specifies the types of nodes, attributes and links defined for each different application
(RDFS-type Representation)
Rules:Add/modify nodes, links & attributes in the network
DNA sequenceimage
field trip
culture
personsample
photographic image
SEM image
Scientific Investigation Ontology (partial)
other
experiment
Scientific Information Resouces
project
measurement
field site
equipment
camera
gas chromatograph
stub
O2 microsensor
N2 microsensorSEM
O2 concentration
N2 concentration
spectrometer
spectrograph
chromatogram
other
other
micrograph
cultivated-fromcultivated-by
has-genetic-sequence
pictured-in
researcher
lab tech
May-June 2001 Field Trip
BajaStudy Area
In Situ Diel6-4-01
Experiment
Measurements7
Images99
Documents3
EMERGProject
People15
Samples19
Brad’s Trip Planning
Document
ac
df
gh
i
j
a: has logisticsb: samples collected c: has objectivesd: conducted experimente: located atf: trip participantsg: destinationh: trip fori: measurements takenj: has photodocumentation
k: site forl: collected atm: experiment site forn: experiment staffo: has custodianp: pictured inq: has sequence infor: source ofs: has measurement
Links
b
Strawman for Focus group
Document
aa
t: employed in u: collected byv: led byw: authored byx: has subexperimentsy: has measurement passz: generated measurementsaa: associated documentsbb: has test point
Example Semantic Information Space
Pond 4 near 5Field Site
Projects2
Samples56
e k
l7
Experiments6
m
t x
ybb
Greenhouse Sulfate ManipulationExperiment
Test Point18
Experiments3
O2 Measurement
36
z
Thermal VarianceExperiment
Experiments2
x
SalinityExperiment
Diel CycleExperiment
MeasurementPass
2
bbs
s
People5
n
v
M13791-3Measurement
SC-8-11Culture
16S3 rRNADNA Sequence
Bebout, BradScientist
Carpenter, SteveLab Technician
o
r
qs
q
u
w
P4Mat-16Mat Sample
Images33
p
8
Instance space
Current SemanticOrganizer Interface
Links to Related
Records
create new records
modify recordicon identifies
record type
search for records
Right side displays metadata for the current repository record being inspected
Left side uses semantic links
to display all information
related to the repository
record shown on the right
semantic links
related records (click to
navigate)
current record
Interface Problems
• Graphical overview of information space needed for:
– Comprehension of information scope and context
– Non-local navigation
• Can’t display entire information space
– Over 45,000 nodes
– Over 150,000 links
• Can’t make sense of entire information space
Remedy: Filtering and Abstraction
• Filtering: Remove nodes/links
• Abstraction: Replace a set of nodes/links with a smaller number of nodes/links
Q: What is the basis for filtering or abstraction?
Sources of Knowledge for Filtering/Abstraction
• Graph-theoretic: based on topological properties of network (e.g. cut points)
• Content-based: using textual content stored in nodes (e.g., as in Web page clustering)
• Semantics-based:
– ontology (node-type, link-type, subsumption)
– auxiliary information:
• importance/intrinsicality of nodes/links
• usage context
Semantic Approaches to Simplifying Information Space Presentation
1. Contextual Filtering
2. Semantic Structure Abstraction
3. Semantic Navigation
1. Contextual Filtering
Observation: Not all nodes/links are relevant in a given context
Proposed Approach: Define explicit constraints that generate a meaningful subgraph of nodes in a specific context
Context examples:
• a specific scientific field trip
• a specific project
• a specific location (e.g., a scientific laboratory or field site)
Example: Using Constraints to define a Field Trip Context
FieldTripContext(f) = { {f} S M SA P R E FS I }
where: FieldTrip(f) // f is a node of type FieldTripS = {s | Sample(s) ∧ SamplesCollected(f, s)}M = {m | Measurement(m) ∧ MeasurementTaken(f, m)}SA = {sa | StudyArea(sa) ∧ Destination(f, sa)}P = {p | Person(p) ∧ TripParticipant(f, p)}R = {r | Project(r) ∧ TripFor(f, r)}E = {e | Experiment(e) ∧ ConductedExperiment(f, e)}FS = {fs | FieldSite(fs) ∧ CollectedAt(fs, s) ∧ sS}I = {i | Image(i) ∧ PicturedIn(s, i) ∧ sS}
a) subset of nodes linked directly to a field trip node+
b) images of samples gathered during that trip and field sites where those samples were collected
Field Trip Context =
a
b
Samples19May-June 2001
Field Trip
BajaStudy Area
In Situ Diel6-4-01
Experiment
Measurements7
EMERGProject
People15
P4Mat-16Mat Sample
Images33
M13791-3Measurement
Pond 4 near 5Field Site
a
df
gh
i
b
Results of Applying Field Trip Filter
p
s
e
Projects2
Experiments6
m
Samples56
aa
c
j
Images99
Documents3
Brad’s Trip Planning
Document
Strawman for Focus group
Document
8
u
SC-8-11Culture
16S3 rRNADNA Sequence
Bebout, BradScientist
Carpenter, SteveLab Technician
o
r
q
q
t x
ybb
Greenhouse Sulfate ManipulationExperiment
Test Point18
Experiments3
O2 Measurement
36
z
Thermal VarianceExperiment
Experiments2
x
SalinityExperiment
Diel CycleExperiment
MeasurementPass
2
bbs
People5
n
v
8
w
k
l7
2. Semantic Structure Abstraction
Proposed Approach:
• apply semantic patterns to identify these substructures
• represent them as abstract nodes
• display them using familiar representation
Observation: Graphs can obscure structure! Certain graph substructures are better depicted using more familiar visual representations
• Hierarchical structures trees
• List structures arrays
• Cross-correlated structures tables
• Time sequences PERT charts
Semantic Structure Abstraction: Approach
x
ybb
Greenhouse Sulfate ManipulationExperiment
Test Point5
Experiments3
O2 Measurement
10
Thermal VarianceExperiment
Experiments2
x
SalinityExperiment
Diel CycleExperiment
MeasurementPass
2
bbs
Gas FluxExperiment
Peak CycleExperiment
1. Recognize patterns
Greenhouse Sulfate ManipulationExperiment
Test Pointx Measurement
x Pass
2. Represent as abstract nodes
Greenhouse Sulfate Manipulation
SalinityThermal Variance
Diel Cycle
Gas Flux
Peak cycle
3. Display appropriately
B C D E
Pass 1 2
m8m1 m2 m3 m4 m5
Test Point A
m6 m7 m9 m10
O2 Measurement{
Experiment Hierarchy
2-dimensional measurementindexing structure
3. Semantic Navigation
Proposed Approach:
• Move from current detailed, fine-grained interface to more abstract navigation interface
• Abstract away the specific links and present only clusters of nodes radiating out from a focal node
• Use a semantics-based focus+context style display (e.g., fisheye, hyperbolic)
Observation: High-level semantic categories in an ontology can help users visualize and navigate the information space in a more effective, rapid, intuitive fashion
More Abstract Interface:Bull’s-Eye Navigator
Related 2 nd
order nodes
Related 1 st
order nodes
Focal Node
Artifacts Activities
People Social Places
Activities
Places
Social
Artifacts
People
fieldtrip
artifacts relatedto “field trip”(e.g., sample-X)people related
to “field trip” artifacts (e.g., labtech who analyzed sample-X)
scientistslab techs
traverseexpts …samples, msmts
…
projectsorgs …
labssites …
(1 link away)
(2 links away)
Focal Region
Context Region
Compactrepresentation ofinformation space surroundingfocal node
docs
Semantic categories:• People• Places• Activities• Artifacts• Social Structures
Summary
• Large information spaces are difficult to comprehend and navigate
• Visualization can help
• Semantic information provides leverage for visualization
• Three examples:
– Contextual Filtering
– Semantic Structure Abstraction
– Semantic Navigation