kmi hyper 2009
DESCRIPTION
Invited presentation to the Hypotheses, Evidence & Relationships Workshop, Elsevier Disruptive Technologies Lab, May 2009: http://hyp-er.wik.isTRANSCRIPT
1
The Hypermedia Discourse Project Tools for Annotating, Visualizing & Navigating Literature as Discourse Networks
Simon Buckingham Shum
Knowledge Media Institute The Open University Milton Keynes, UK
http://people.kmi.open.ac.uk/sbs http://projects.kmi.open.ac.uk/hyperdiscourse
HypER Workshop: Hypotheses, Evidence and Relationships11-12 May 2009, Elsevier, Amsterdam
2
Compendium Institute
http://projects.kmi.open.ac.uk/hyperdiscourse
3
Compendium Institute
http://projects.kmi.open.ac.uk/hyperdiscourse
questions
4
5
1665 throws a long shadow
Philosophical Transactions of the Royal Society of London March 1665
Le Journal des Sçavans January 1665
From: To…?
Chaomei Chen, 2006: Citation analysis
Buckingham Shum et al, 2003: lineage analysis
Buckingham Shum, S. (2007). Digital Research Discourse? Computational Thinking Seminar Series, School of Informatics, University of Edinburgh, 25 Apr. 2007. http://kmi.open.ac.uk/projects/hyperdiscourse/docs/Simon-Edin-CompThink.pdf
The question we used to ask in 2001 at the start of the ScholOnto project
In 2010, will we still be publishing scientific results
primarily as prose papers, or will a complementary
infrastructure emerge that exploits the power of
the social, semantic web to model the literature as
a network of claims and arguments?
6
The question we used to ask in 2001 at the start of the ScholOnto project
In 2010, will we still be publishing scientific results
primarily as prose papers, or will a complementary
infrastructure emerge that exploits the power of
the social, semantic web to model the literature as
a network of claims and arguments?
7
20xx?
8
Questions the next generation scientific infrastructure should help answer
• “What is the evidence for this claim?”
• “Was this prediction accurate?”
• “What are the conceptual foundations for this idea?”
• “Who’s built on this idea? How?”
• “Who’s challenged this idea? Why? How?”
• “Are there distinctive perspectives on this problem?”
• “Are there inconsistencies within this school of thought?”
assumptions
9
Researchers read meanings into texts that are not there, and with which the author might disagree so we will always require manual annotation tools we need ways to make connections to connections extremely complex connections may remain the province of human sensemaking
(e.g. is analogous to)
Good user interfaces will be needed to view, edit and navigate HypERnets, whether manually or automatically constructed
Scientific discourse is a social process we take huge care in our writing about how we position ourselves in relation to our
peers — will we trust unsupervised machines to extract and position our more complex claims?
10
modelling schemes: IBIS
11
12
Rittel’s IBIS: Issue-Based Information System
13
Compendium: customisable, collaborative, hypermedia IBIS mapping
Buckingham Shum, S., Selvin, A., Sierhuis, M., Conklin, J., Haley, C. and Nuseibeh, B. (2006). Hypermedia Support for Argumentation-Based Rationale: 15 Years on from gIBIS and QOC. In: Rationale Management in Software Engineering (Eds.) A.H. Dutoit, R. McCall, I. Mistrik, and B. Paech. Springer-Verlag: Berlin
14
IBIS mapping of Iraq debate
Buckingham Shum, S., and A. Okada. 2008. Knowledge cartography for controversies: The Iraq debate. In Knowledge cartography: Software tools and mapping techniques, ed. A. Okada, S. Buckingham Shum, and T. Sherborne, 249–66. London: Springer.
Mapping a nuclear power debate on a blog
Mapping a nuclear power debate on a blog
Mapping a nuclear power debate on a blog
modelling schemes: ScholOnto
18
19
ScholOnto schema Connecting freeform tags with naturalistic connections (“dialects”) grounded in a formal set of relations (from semiotics and coherence relations)
Mancini, C. and Buckingham Shum, S.J. (2006). Modelling Discourse in Contested Domains: A Semiotic and Cognitive Framework. International Journal of Human Computer Studies, 64, (11), pp.1154-1171. [PrePrint: http://oro.open.ac.uk/6441]
20
Paper: “The Scent of a Site: A System for Analyzing and Predicting Information Scent,
Usage, and Usability of a Web Site”
“Web User Flow by Information Scent (WUFIS)”
Paper: “Information foraging”
“Information foraging theory”
“Information scent models”
“People try to maximise their rate of gaining information”
?
applies
Scholarly discourse as CKS… Beyond document citations…
These annotations are freeform summaries of an
idea, as one would also find in researchers’ journals,
fieldnotes, lit. review notes or blog entries
Making formal connections between ideas creates a
semantic citation network —> novel literature navigation, querying and visualization
Method
Theory
Claim
topic maps and subject centric federation
21
Schematic: Documents, Subjects,and Relations
Document
Subjects in documents
Relations between subjects
Occurrence links
Topic Map of documents and their subjects
Federated Subjects
interaction design
24
25
Interaction design for literature visualization: pilot study: paper-based literature modelling
S. Buckingham Shum, V. Uren, G. Li, B. Sereno, and C. Mancini. Computational Modelling of Naturalistic Argumentation in Research Literatures: Representation and Interaction Design Issues. International Journal of Intelligent Systems, 22(1):17–47, 2006
26
Interaction design for lit. visualization From paper prototype to semiformal mapping tool The ClaiMapper tool
Evaluated in: V. Uren, S. Buckingham Shum, G. Li, and M. Bachler. Sensemaking Tools for Understanding Research Literatures: Design, Implementation and User Evaluation. International Journal of Human Computer Studies, 64(5):420–445, 2006
…to formal argument maps
Starting from paper-based modelling, move from literature sketches…
27
Interaction design for doc. annotation Pilot study: paper-based annotation
Pilot study reported in: B. Sereno, S. Buckingham Shum, and E. Motta. (2005). ClaimSpotter: an Environment to Support Sensemaking with Knowledge Triples. Proc. Int. Conf. Intelligent User Interfaces, pages 199–206, ACM
28
The ClaimSpotter annotation tool Web 2.0-style tagging with optional community/system tag
recommendations
Sereno, B., Buckingham Shum, S. and Motta, E. (2007). Formalization, User Strategy and Interaction Design: Users’ Behaviour with Discourse Tagging Semantics. Workshop on Social and Collaborative Construction of Structured Knowledge, 16th Int. World Wide Web Conference, Banff, Canada; 8-12 May 2007.
29
Lessons Learnt & Design Principles Untrained users can do it: in their first hour they created
coherent claims. UI design validated to this degree. —future work: longitudinal evaluation at scale
New users attend to what is highlighted for them (matching tags; primary doct.), and generally don’t click down a level —next version combines visualizations and document-centric features
Support incremental formalization —cf. use of is-about as a placeholder link; provide an Other… category and try to map automatically to the ontology
Users’ strategies vary — don’t assume a strong workflow a paper-based pilot study can provide insights into this
Web 2.0 UI simplicity: good design needed to provide high functionality, walk-up-and-use tools —we overwhelmed some users with overlaid suggestions for tags
Sereno, B., Buckingham Shum, S. and Motta, E. (2007). Formalization, User Strategy and Interaction Design: Users’ Behaviour with Discourse Tagging Semantics. Workshop on Social and Collaborative Construction of Structured Knowledge, 16th Int. World Wide Web Conference, Banff, Canada; 8-12 May 2007.
30
Cohere: from tag clouds to idea webs
Buckingham Shum, Simon (2008). Cohere: Towards Web 2.0 Argumentation. In: Proc. COMMA'08: 2nd International Conference on Computational Models of Argument, 28-30 May 2008, Toulouse, France. IOS Press [PrePrint: http://oro.open.ac.uk/10421]
31
Cohere: embedding an Idea or Map in another website (a blog post)
Buckingham Shum, Simon (2008). Cohere: Towards Web 2.0 Argumentation. In: Proc. COMMA'08: 2nd International Conference on Computational Models of Argument, 28-30 May 2008, Toulouse, France. IOS Press [PrePrint: http://oro.open.ac.uk/10421]
32
Cohere: a mashup visualization merging different connections around a common Idea
Buckingham Shum, Simon (2008). Cohere: Towards Web 2.0 Argumentation. In: Proc. COMMA'08: 2nd International Conference on Computational Models of Argument, 28-30 May 2008, Toulouse, France. IOS Press [PrePrint: http://oro.open.ac.uk/10421]
33
Cohere: semantically filtering a focal Idea by “contrasting” connections
Buckingham Shum, Simon (2008). Cohere: Towards Web 2.0 Argumentation. In: Proc. COMMA'08: 2nd International Conference on Computational Models of Argument, 28-30 May 2008, Toulouse, France. IOS Press [PrePrint: http://oro.open.ac.uk/10421]
34
Cohere: semantically filtering a focal Idea by “contrasting” connections
Buckingham Shum, Simon (2008). Cohere: Towards Web 2.0 Argumentation. In: Proc. COMMA'08: 2nd International Conference on Computational Models of Argument, 28-30 May 2008, Toulouse, France. IOS Press [PrePrint: http://oro.open.ac.uk/10421]
35
“What papers contrast with this paper?”
1. Extract concepts for this document 2. Trace concepts on which they build 3. Trace concepts challenging this set 4. Show root documents
Evaluated in: V. Uren, S. Buckingham Shum, G. Li, and M. Bachler. Sensemaking Tools for Understanding Research Literatures: Design, Implementation and User Evaluation. International Journal of Human Computer Studies, 64(5):420–445, 2006
“What is the lineage of this idea?”
36
Buckingham Shum, S.J., Uren, V., Li, G., Sereno, B. and Mancini, C. (2007).Modelling Naturalistic Argumentation in Research Literatures: Representation and Interaction Design Issues. International Journal of Intelligent Systems, (Special Issue on Computational Models of Natural Argument, Eds: C. Reed and F. Grasso, 22, (1), pp.17-47. [PrePrint: http://oro.open.ac.uk/6463]
Current projects: scientific collective intelligence through discourse
OLnet: Open Learning Network to connect the open educational resource movement’s discourse/evidence base: http://olnet.org
ESSENCE: e-Science/Sensemaking/Climate Change testing and integrating Web argumentation tools: http://events.kmi.open.ac.uk/essence
SocialLearn: Web 3.0 social learning/sensemaking platform with semantic discourse connections (launches end of year)
37
Marriage made in heaven?
Human annotation
38
Machine annotation
Workshop Qs:
Corpus you are working on; community, type of content (abstracts, full-text, book..)full text: scholarly/scientific, blogs, newspapers, real time
discussions (and video of it), mission doctrine/policy
Granularity of knowledge element you are identifying arbitrary: statements, single words, paragraphs
Relationships between knowledge elements you have identified IBIS: relational types + node types ScholOnto: relations + roles... Cohere 39
Workshop Qs:
Type of annotation: automatic, manual, combination manual annotation partial automatic highlighting of text based on
Simone Teufel's work on Argumentative Zoning Size of corpus you have annotated so far
40 pages of blog debate 12 hours of video distill 2 cm of policy docts into IBIS maps several books in a literature 10-30 papers in a sample literature 30 articles on Iraq 5 days workshop discussions 40
Workshop Qs:
Data standards, outline of architecture of system built (if relevant) Compendium: XML DTD; SQL Cohere API: RDF; XML; JSON TopicSpaces: XML Topic Map; RDF; OWL
Visualisations Compendium/ClaiMapper manual maps ClaiMaker/Cohere/TopicSpaces generated maps
41
Workshop Qs:
User studies: yes, focusing on interaction design and usage patterns in both field trials and lab studies IUI 2005: evaluation of ClaimSpotter IJHCS 2006: evaluation of ClaiMaker WWW'07 CKC: evaluation of ClaimSpotter IJRME 2008: evaluation of Compendium for mapping
climate change arguments Space Ex. Conf 2005: NASA Ames field trials DIAC 2008: evaluation of Compendium for mapping
planning discourse HCI (under review): evaluation of Compendium
mapping for hostage recovery 42