mik kersten thesis defense december 15, 2006 focusing knowledge work with task context
TRANSCRIPT
Mik Kersten Thesis defense
December 15, 2006
Focusing Knowledge Work with Task Context
Problems
Information overload• Many knowledge work tasks cut across system structure• Browsing and query tools overload users with irrelevant
information
Context loss• Tools burden users with finding artifacts relevant to the task• Context is lost whenever a task switch occurs• Users to waste time repeatedly recreating their context
Thesis
A model of task context that automatically weights the relevance of system information to a task by monitoring interaction can focus a programmer's work and improve productivity.
This task context model is robust to both structured and semi-structured information, and thus applies to other kinds of knowledge work.
Approach
Memory• Episodic memory: one-shot, only single exposure required• Semantic memory: multiple exposures required
Our approach• Leverage episodic memory, offload semantic memory
Tasks: episodes
Context: weighting of relevant semantics to task
Related Work
Memory• Episodic memory: one-shot, only single exposure required• Semantic memory: multiple exposures required
Our approach• Leverage episodic memory, offload semantic memory
Tasks: episodes• Explicit tasks (UMEA, TaskTracer): flat model, lack of
fine-grained structure
Context: weighting of relevant semantics• Slices (Weiser) and searches (MasterScope): structure only• Interaction-based (Wear-based filtering): no explicit tasks
Model & Operations
Interaction
Task context
Degree-of-interest (DOI) weighting• Frequency and recency of interaction with element• Both direct and indirect interaction
Model
interest
Task context graph• Edges added for relations between elements• Scaling factors determine shape, e.g. decay rate
Thresholds define interest levels
Topology
[l, ∞] Landmark
(0, ∞] Interesting
[-∞, 0] Uninteresting
Operations
Once task context is explicit• Can treat subsets relevant to the task as a unit• Can project this subset onto the UI• Perform operations on these subsets
Composition• See context of two tasks
simultaneously
Slicing • Unit test suite can be slow to run on large project• Find all interesting subtypes of TestCase
cd
b
c
ba
T
T
More operations
Propagation• Interacting with method propagates
to containing elements
Prediction• Structurally related elements of potential
interest automatically added to task context
Only interaction stored
Implementation: programming
Implementation: knowledge work
Validation
Validation
Questions• Does task context impact the productivity of programmers?• Does it generalize to other kinds of knowledge work?
Problems• Knowledge work environment hard to reproduce in the lab• No evidence that non-experts are a good approximation of
experts• Measure long-term effects to account for diversity of tasks
Approach• Longitudinal field studies• Voluntary use of prototypes• Monitoring framework for observation
Study 1: feasibility
Productivity metric• Approximate productivity with edit ratio (edits /selections) • Programmers are more productive when coding than when
browsing, searching, scrolling, and navigating
Subjects• Six professional programmers at IBM
Results• Promising edit ratio improvement
Study 2: programmers
Subjects• Advertised study at EclipseCon 2005 conference• 99 registered, 16 passed treatment threshold
Method and study framework• User study framework sent interaction histories to UBC
server• Baseline period of 1500 events (approx 2 weeks)• Treatment period of 3000 events, to address learning curve
Study 2: results
Statistically significant increase in edit ratio• Within-subjects paired t-test of edit ratio (p = 0.003)
Model accuracy• 84% of selections were of elements with a positive DOI• 5% predicted or propagated DOI• 2% negative DOI
Task activity• Most subjects switched tasks regularly
Surprises• Scaling factors roughly tuned for study, but still unchanged
Study 3: knowledge workers
Subjects• 8 total, ranged from CTO to administrative assistant
Method and study framework• Same framework as previous, monitor interaction with files
and web• No reliable measure of productivity, gathered detailed
usage data
Study 3: results
Task Activity• Users voluntarily activate tasks when provided with task
context• Activations/day ranged from 1 to 11, average is 5.8
Task context contents• Long paths are common• Density over system structure is low• Tagging did not provide a reliable mechanism for retrieval
Task context sizes• Non-trivial sizes, some large (hundreds of elements)• Many tasks had both file and web context
Model accuracy• Decay works, most elements get filtered
Summary
Contributions
Generic task context model• Weighted based on the frequency and recency of interaction • Supports structured and semi-structured data• Weighting is key to reducing information overload• Capturing per-task reduces loss of context when multi-tasking
Task context operations• Support growing and shrinking the model to tailor to activities• Integrate model with existing tools
Instantiation of the model • For Java, XML, generic file and web document structure• Can be extended to other kinds of domains and application
platforms
Monitoring and framework • Reusable for studying knowledge work
Conclusion
Tools’ point of view has not scaled• Complexity continues to grow, our ability to deal with it
doesn’t
Task context takes users’ point of view• Offloads semantic memory, leverages episodic memory
Impact on researchers• University of Victoria group extended it to ontology browsing
Users• “Mylar is the next killer feature for all IDEs” Willian Mitsuda
Industry• “…it’ll ultimately become as common as tools like JUnit, if
not more so.” Carey Schwaber, Forrester analyst