codetrail: connecting source code and web resources
TRANSCRIPT
ARTICLE IN PRESS
Contents lists available at ScienceDirect
Journal of Visual Languages and Computing
Journal of Visual Languages and Computing 20 (2009) 223–235
1045-92
doi:10.1
� Cor
E-m
(R.C. Mi
journal homepage: www.elsevier.com/locate/jvlc
Codetrail: Connecting source code and web resources
Max Goldman �, Robert C. Miller
MIT CSAIL, 32 Vassar Street, Cambridge, MA 02139, USA
a r t i c l e i n f o
Keywords:
Software development
Development environment
Web browser
Human–computer interaction
6X/$ - see front matter & 2009 Elsevier Ltd. A
016/j.jvlc.2009.04.003
responding author.
ail addresses: [email protected] (M. Goldman), r
ller).
a b s t r a c t
When faced with the need for documentation, examples, bug fixes, error descriptions,
code snippets, workarounds, templates, patterns, or advice, software developers
frequently turn to their web browser. Web resources both organized and authoritative
as well as informal and community-driven are heavily used by developers. The time and
attention devoted to finding (or re-finding) and navigating these sites is significant. We
present Codetrail, a system that demonstrates how the developer’s use of web resources
can be improved by connecting the Eclipse integrated development environment (IDE)
and the Firefox web browser. Codetrail uses a communication channel and shared data
model between these applications to implement a variety of integrative tools. By
combining information previously available only to the IDE or the web browser alone
(such as editing history, code contents, and recent browsing), Codetrail can automate
previously manual tasks and enable new interactions that exploit the marriage of data
and functionality from Firefox and Eclipse. Just as the IDE will change the contents of
peripheral views to focus on the particular code or task with which the developer is
engaged, so, too, the web browser can be focused on the developer’s current context and
task.
& 2009 Elsevier Ltd. All rights reserved.
1. Introduction
To suggest that the web is a source of information forsoftware developers is perhaps trite. The web connectsdevelopers to language and API documentation, codeexamples, bug reports and workarounds, tutorials, arti-cles, code ranging from one-liners to sketches to pagesof boilerplate, and innumerable sources of discussionand debate on programming systems popular and ob-scure. Since the domain of web resources ranges fromweblog posts and forum discussions to web-accessibleinterfaces for a development team’s own bug database orversion control, there is no one way to characterize webresources or shoehorn them into a developer’s existingworkflow.
ll rights reserved.
Once new tools such as the web prove their worth inthe developer’s toolset, they become integrated into thedevelopment environment. The Eclipse integrated devel-opment environment (IDE) provides ready evidence, withspecialized plug-ins for integrating with version controlsystems, bug databases, and many other tools. Butintegration with web resources remains lacking.
Software developers already use a powerful tool foraccessing the web: their browser. Web browsers arecustomized and configured according to the user’sinformation needs. The browser stores bookmarks,cookies, certificates, and browsing history. Custom stylesheets and font choices may be applied. In the case ofFirefox, an extension mechanism enables users to installcomponents that provide additional customizations, orthat allow programming-savvy users to construct newcustomizations of their own. Compare this rich interfaceagainst the built-in web browsing component that shipswith Eclipse. The latter provides none of the abovefeatures, and it is consequently the authors’ experiencethat this browser is rarely used. Instead of a simple HTML
ARTICLE IN PRESS
M. Goldman, R.C. Miller / Journal of Visual Languages and Computing 20 (2009) 223–235224
renderer with minimal UI, software developers need afull-featured web browser to be integrated into theirworking environment.
Our prototype system, Codetrail, integrates the webbrowser and the IDE via a shared communication channeland data pool, and provides new tools that help thedeveloper integrate web resources into the programmingworkflow and into the project itself. Of particular interestis the way Codetrail can enable the web browser to act asan IDE view, where the contents of the browser areautomatically focused on the code the developer iscurrently editing. Many other Eclipse tools behave in thisfashion.
Codetrail aims to eliminate the manual effort currentlyspent coordinating IDE and web browser in order to makeuse of web resources. By making previously manual tasksautomatic or previously laborious tasks straightforward,the developer remains focused on software development.Both developers and project teams gain the additionalbenefit of more comprehensive and organized records ofthe online resources used during the developmentprocess.
In the balance of this paper we report on the results ofan informal field study on developers’ web use, describe indetail a number of tools implemented on the Codetrailplatform, discuss the results of a user study in whichdevelopers worked with Codetrail, and then describegeneral requirements for a useful integration mechanismas well as specific elements of our implementation.Codetrail was first presented in Goldman and Miller [2],appearing here with elaboration on technical details andwith additional evaluation.
1 http://www.google.com/
2. Related work
Programmers’ working environments have long beenconstructed by combining a variety of tools [3], and thispractice has been examined by many researchers (e.g. thecase study of Wicks and Dewar [19]). In the classificationof IDE tool integration from Yang and Han [20], Codetrailfalls in the most powerful class, synchronizing theactivities of Eclipse and Firefox using a loosely coupledarchitecture.
Developers’ information needs have also receivedstudy. Much of the historical information about codecritical to developers’ work is implicit [11], and Codetrailis an attempt to make some of that knowledge explicit anddiscoverable. Given the results from Ko et al. [9], whichsuggest that programmers often turn to their colleaguesfor help, we contend that giving those programmers toolsto connect with web resources may be profitable. In anearlier examination of design requirements for mainte-nance-oriented IDEs [8], developers in a user study spent11% of the total task time either reading the Java API orswitching between applications such as their IDE and webbrowser, suggesting that even for relatively small tasks,documentation is important and there is overhead to beeliminated.
A variety of systems have been proposed that attemptto help developers use the web more effectively. Assieme
[5] augmented search results with code excerpts, andimproved search result accuracy by combining a webcrawler with a compiler that can identify implicitrelations between code. Mica [16] is an earlier systemwhich also augments search result display. Such toolscertainly do increase the usefulness or accessibility of webresources, but they do not integrate those resources intothe development environment.
Other projects have integrated new tools into the IDE.Hipikat [17,18] tracked project memory and recommendedresources to newcomers, including web resources. Jazz [1]integrated the presence and instant messaging to connectcoworkers. Mylyn (previously Mylar) [6] is based on thenotion of a task-focused user interface in the IDE. Itprovides integrated access to some of the same resourcesthat might be linked using Codetrail, but does so using itsown interface. Our focus is on what the web browser, as aflexible and familiar tool for accessing information on theweb, brings to the IDE.
3. Code and the web
In order to better understand the nature of developers’interaction with web resources, we conducted a smallinformal field study that examined the development-related browsing behavior of research group members.Four subjects participated for between one and threeweeks each by installing a tool that recorded informationabout files edited in Eclipse and web pages loaded inFirefox. All were actively involved in software develop-ment at the time. Before submitting their data, partici-pants had the opportunity to ‘‘scrub’’ the log of any website visits they considered sensitive and did not wish toreport, and no record of the number or nature of thesescrubbed sites was retained. In total, 6409 page loadswere reported.
The URLs and contents (where accessible) of the pageswere examined and each page was hand-labeled accord-ing to whether it was self-evidently development-related,e.g. because it contained code or described programmingtools. As a result, the count of development-related pagesmay be an underestimate, if participants visited siteswhose connection to software development was non-obvious or only by virtue of the problem domain. Overall,10% of reported page loads (646) were obviously devel-opment-related.
Since we are interested in the usage of the webbrowser and IDE together, we considered events intemporal proximity. For page loads that occurred withina 5-min window of saving some code file, 23% weredevelopment-related.
Development-related pages took a variety of forms,described below and enumerated in Table 1.
Searches: earches: Searches of all kinds were performedalmost exclusively using the Google web search engine.1
ARTICLE IN PRESS
Table 1Taxonomy of development-related web pages.
Page type Percent Count
Searches
Code elements 12 76
Code ideas 11 72
Tools 5 32
Concepts 3 20
Sites 3 18
34% 218
Documentation
Official 30 191
Unofficial 3 19
33% 210
Other resources
Content 24 157
Project 5 33
Navigation 4 28
34% 218
Percentages are of the total number of development-related pages, and
do not sum to 100 due to rounding error.
M. Goldman, R.C. Miller / Journal of Visual Languages and Computing 20 (2009) 223–235 225
Code elements: Example code element searches include‘‘java.lang.math,’’ ‘‘JSplitPane,’’ and ‘‘setBorder,’’ where thesearcher is using the name of some language element.
Code ideas: Example code idea searches include ‘‘phpxsl’’ and ‘‘read from input java,’’ where the searcher islooking for implementation-level information but has notidentified particular code elements.
Sites: Rather than discover information sources usingqueries for specific entities (e.g., by code element search),search queries were sometimes used to locate the sitesthemselves. Examples include ‘‘java 1.5 api’’ and ‘‘xulpla-net.’’ It is likely that these searches are a re-findingmechanism for developers who already know about thesites but choose not to use bookmarks or memorize URLs.
Tools: An example tool search is ‘‘subversion branch,’’where the subject of the search is a component of thedeveloper toolchain, not the program code.
Concepts: At a higher level than code elements or tools,searches were observed where the subject was concep-tual, related to general computational ideas. Examplesinclude ‘‘fibonacci heap’’ and ‘‘halloween problem.’’
Documentation: Documentation sites exhibit carefulstructure and are organized by documented element.
Official: Often results of code element searches ledusers to official documentation, such as the Java APIdocumentation provided by Sun.2 Other examples includeFacebook’s API documentation3 and Adobe’s documenta-tion on Flex.4
Unofficial: Not all documentation is maintained by theoriginator of a framework or system. Examples visited
2 http://java.sun.com/reference/api/3 http://wiki.developers.facebook.com/index.php/API4 http://livedocs.adobe.com/flex/3/langref/
include XUL Planet’s documentation on Mozilla APIs5 andW3Schools’ web technologies references.6
Other resources: Non-documentation resources.Project: Some visits were to project-related resources
maintained by developers on the project; a bug databaseand web frontend for a version control repository areexamples.
Navigation: Example navigation, or portal, pagesinclude the Eclipse and Sun home pages.
Content: Most web information sources are notorganized as documentation. Technical articles and tutor-ials, blogs and other personal web pages, wikis, forums,and mailing list archives were all visited by participants.
Note that the particular population of developers whoreported results heavily influences the numbers shown inTable 1. All did at least some development in Java,increasing the relative representation of official documen-tation, since Sun provides extensive API documentation onthe web. Participants were also working on smaller,loosely organized academic projects, decreasing therelative number of project resources used in comparisonto a large industry project with infrastructure to managefeatures and coordination.
This taxonomy motivates a general two-prongedapproach to connecting code and the web in Codetrail.First, site searches and code element searches are targetedfor automation, by creating stronger links betweenidentified sources of official and unofficial documentationand the code. With these links, the user no longer needs toturn to a search engine. Second, connections betweencode and other useful web resources are facilitated by ageneral-purpose bookmarking mechanism that is lessautomated, but significantly more straightforward thancurrently available.
4. Applications
Having revealed some aspects of developers’ webbehavior, we now describe several tools we have im-plemented in Codetrail, considering usage scenarios andrelevant design principles. The variety of tools serves todemonstrate that the system is a robust and flexibleintegration mechanism. We begin by describing howCodetrail automatically integrates structured documenta-tion sites, first in the context of automatically identifyingand browsing to Javadoc documentation (Sections 4.1 and4.2), and then more generally in Section 4.3. Section 4.4turns to integration with unstructured resources via user-created bookmarks, and in Section 4.5 we discussautomatically created bookmarks in the case of copiedcode.
All of these tools were designed using an iterativeprocess. Early versions of the system were deployed tostudents and faculty in our research group, who provided
5 http://www.xulplanet.com/references/6 http://www.w3schools.com/
ARTICLE IN PRESS
M. Goldman, R.C. Miller / Journal of Visual Languages and Computing 20 (2009) 223–235226
feedback on the usefulness and usability of Codetrail’sfeatures.
4.1. Automatic Javadoc attachment
Eclipse already includes a mechanism for ‘‘attaching’’Javadoc URLs to libraries, although the user interface forthis feature is somewhat obscure. Attached Javadocs areused to display relevant documentation for the codeelement (class name, method name, etc.) under the editorcursor in a separate view, and to add information to codecompletion suggestions (Fig. 1). While often useful, theamount of information presented and visible at any timeis limited. The separate view, in particular, is animpoverished window on the Javadocs, with no abilityto navigate to related methods or classes, or to see anycontext for the current artifact.
Codetrail bypasses the obscure Javadoc attachment UIby automatically identifying relevant Javadoc documenta-tion when the user browses to it in Firefox. Documenta-tion sites generated by Sun’s javadoc tool have a fixed
View
Tool-tip
Fig. 1. Built-in Eclipse features for accessing Javadocs: tool-tips and code comp
alongside it.
Fig. 2. Javadoc attachment prompt shown when the user browses to a Javadoc
form and include a machine-readable file enumerating thepackages that the site documents. Codetrail downloadsthis file and searches the active Eclipse projects forlibraries containing those packages. When a match isfound, the user is prompted as shown in Fig. 2. Since thematching process takes trivial time, the prompt appearsalmost immediately after the user loads the Javadoc site.
If the developer instructs Codetrail to attach theJavadocs, the existing Eclipse documentation attachmentmechanism is used. The benefits are immediate inimproved tool tip and auto-complete information, andthe existing Javadoc view will now become active. Furtherbenefits will be described in the sections to follow.
The user is prompted to confirm documentationattachment so that the system’s operation is visible: ourgoal is to have Codetrail improve workflow and add newfunctionality without making the development environ-ment more inscrutable. By presenting a prompt, weprovide the developer with a mental model of how thesystem operates: ‘‘when I visit a documentation page,Eclipse asks me about attaching it.’’ Without this model, auser might not make the connection between visiting
Code complete
letion suggestions presented within code, and a view of documentation
documentation source that may be linked to a project library in Eclipse.
ARTICLE IN PRESS
M. Goldman, R.C. Miller / Journal of Visual Languages and Computing 20 (2009) 223–235 227
documentation in the browser and subsequent links to thatdocumentation in Eclipse. Prompting does introduce a riskof annoyance, which we mitigate in several ways. Only oneprompt is offered per Javadoc site, and only when matchingpackages are found without an existing documentationattachment, so we expect users to find the dialogsinfrequent and timely. The prompt is also non-modal andthe user may choose to disregard it, in which case thedialog is dismissed when the next web page is loaded.
4.2. Automatic browsing to documentation
While automatic linking of Javadocs to library codeimproves the developer’s experience, we have not yetresolved the aforementioned issues with the existing waysto access that documentation in Eclipse. Rather than relyon an in-IDE view with limited screen space andfunctionality, Codetrail uses Firefox to automaticallybrowse to the full documentation for code elements underthe editor cursor. Automatically browsing to documenta-tion addresses the large number of queries for Java package,class, and method names seen in the initial field study. Italso reduces queries for information sources themselves, asdevelopers need only click on a code element associatedwith a given documentation source in order to reach thatsource and begin navigating within it.
Codetrail implements automatic browsing by creating,as needed, a special browser tab in Firefox (Fig. 3). Allautomatic browsing occurs in this tab, which consistentlydisplays an identifying label and Eclipse icon. The user isfree to browse within this tab, achieving the importantcapacity for exploration of other documentation related toa particular code element. While the current implementa-tion assumes that only one documentation item isrelevant for a given element (in line with Eclipse’s model),the user could be presented with a choice of multipledocumentation sources—or, multiple browser tabs couldbe used to put all sources at the ready.
Automatic browsing is our first example of treating theweb browser as an IDE view, a particular integration modewhere the browser becomes subordinate to the IDE editor.Operating as a view, the browser displays informationrelevant to the code artifacts on which the developer is
Label
Fig. 3. Automatic browsing tab, containing documentation automati
currently focused. This allows the developer to immedi-ately access full documentation for every class, method, orother keyword the cursor touches, just as views includedwith Eclipse give instant information about other proper-ties like location in a class hierarchy.
4.3. Generic documentation attachment
Javadoc documentation is only one form of the manykinds of official and unofficial documentation developersmay use. We would like to allow developers to associateany useful source of documentation with their project andreceive the same benefits of immediate access.
Many documentation sources exhibit significant struc-ture, even if that structure is unique to the particular source.XUL Planet is an excellent example, with an individual pagedocumenting each interface in the Mozilla API.
Consider the scenario in which a developer is workingon a project and visiting documentation sites as he works,looking up code elements related to the project. We wouldlike to automatically learn the association between key-words appearing in the developer’s code and pages on thedocumentation site, without having to ask the user todescribe that connection. Codetrail achieves this auto-mated documentation identification by observing bothcode contents and browsing history.
By storing a record of recent browsing history,Codetrail can identify common URL patterns that differin only a particular component. For example, suppose auser discovers the XUL Planet site via a Google query,visiting URLs:
www.google.com/search?q=nsifile
www.xulplanet.com/references/xpcomref/ifaces/nsIFile.html
www.xulplanet.com/references/xpcomref/
www.xulplanet.com/references/xpcomref/group_RDF.html
www.xulplanet.com/references/xpcomref/ifaces/nsIRDFNode.html
www.xulplanet.com/references/xpcomref/ifaces/nsIRDFLiteral.html
Codetrail will extract the pattern:
www.xulplanet.com/references/xpcomref/ifaces/okeyword4.html
The values of differing components are used as keywordsin a search through the user’s Eclipse projects—in this case,
Icon
cally retrieved by Codetrail, with an identifying icon and label.
ARTICLE IN PRESS
Fig. 4. Generic documentation prompt shown when the user browses within a site that may be linked to keywords in project files.
M. Goldman, R.C. Miller / Journal of Visual Languages and Computing 20 (2009) 223–235228
nsIFile, nsIRDFNode, and nsIRDFLiteral. Matching key-words in project files (in this case at least two of the threecandidates would have to be found) is strong evidencethat the site contains relevant documentation. If a matchis discovered, the user is shown a prompt similar to theone used for Javadocs, although here we identify thedocumentation link by keywords instead of library name,since we do not know anything about the semantics ofthese keywords (Fig. 4). While system visibility isimportant as before, here the prompt is additionallynecessary because we are using documentation identifi-cation heuristics that may generate some false positives.With the user’s approval, Codetrail uses the Yahoo! websearch API7 to retrieve pages matching the same URLpattern, additionally restricting matches to pages whosetitles contain words that were common to the originalpages used to identify keywords. Continuing our example,the query here would be: site:www.xulplanet.com in-url:references inurl:xpcomref inurl:ifaces
And the results returned would include:
www.xulplanet.com/references/xpcomref/ifaces/nsIUpdate.html
www.xulplanet.com/references/xpcomref/ifaces/nsIURI.html
... 900 more ...
From this set of URLs, the set of keywords to associatewith this documentation source is extracted and stored.The same automatic browsing technique is applied whenthe user moves their editing cursor over a knownkeyword. So now, if our example user clicks on ‘‘nsIURI’’in the Eclipse editor, XUL Planet documentation for thatinterface will come up in Firefox, without the programmerever having visited that particular page before.
As with many web structure extraction heuristics, analgorithm of this sort requires significant developmenteffort to work across many sites, each with its ownparticular quirks. The current Codetrail implementationhas been tested successfully on several sites, includingXUL Planet, the MySQL Reference Manual,8 and documen-tation sections of the Mozilla Developer Center.9
7 http://developer.yahoo.com/search/web/8 http://dev.mysql.com/doc/9 http://developer.mozilla.org/en/docs/
4.4. Bookmarks
Not all associations between the code and the weblink to organized resources that can be detected auto-matically. In order to handle arbitrary links, the systemprovides a flexible bookmarking mechanism that theuser can invoke to link any web location to any code file.Some developers may already create a weak versionof such links by including URLs in comments in sourcecode files. However, creating links for code files alone is,in our estimation, too restrictive. The tool searchesobserved in the initial field study might yield resourcesvaluable not in reference to a specific file or files, but tothe project as a whole, or to some part of it. Build scripts,configuration files, to-do lists, and binary libraries, forexample, all might have web resources relevant to them.Codetrail stores bookmarks as metadata, outside thetarget file, so that developers can create bookmarks infiles of different languages without requiring differentsyntaxes; in files whose language may not have acomment syntax, or have any particular syntax at all; onfolders; and on binary files and other unmodifiableresources.
Our approach is to create bookmark files in the Eclipseproject to hold the bookmark associations. Since thereis no popular cross-platform web link file type, we usea simple XML schema that also conforms to the specifica-tion for Apple web location (webloc) files. In bothFirefox and Eclipse, Codetrail presents interfaces foraccessing bookmarks, and for creating them by associatingweb locations with files or directories (Fig. 5). Equivalentfunctionality in both applications keeps bookmarkingclose at hand and eliminates application switchingto record an association. Codetrail enables these interfacesby giving Firefox access to the Eclipse file hierarchy,and Eclipse access to recent Firefox browsing. Bookmarksfor the currently edited file are displayed in an Eclipseview shown in Fig. 5c. This view shows bookmarksdirectly associated with the current file at the top, butkeeps in view all bookmarks attached to containers of thatfile up the resource hierarchy (in the figure, e.g., a link tothe Eclipse FAQ is visible from any file in the ‘‘plugin’’directory of the project).
ARTICLE IN PRESS
Fig. 5. (a) Eclipse and (b) Firefox bookmark creation interfaces and (c) Eclipse bookmark view, showing bookmarks attached to an artifact and its
containers.
M. Goldman, R.C. Miller / Journal of Visual Languages and Computing 20 (2009) 223–235 229
Storing bookmarks as metadata files additionallyallows the developer to organize them independently ofthe target code or other files or directories and provides astraightforward mechanism for sharing them with otherdevelopers. Codetrail allows bookmark files to be storedanywhere beneath a designated bookmarks directory inthe project, meaning that a developer or developmentteam can organize them according to their needs:tutorials in one folder, references in another; resourcesfor newcomers to the project who need to orientthemselves combined in one place; arranged by sourceweb site to track the sources of input to a project; and soon. Whether organized or not, storing bookmarks as filesalso allows them to be checked in to the project’s versioncontrol system, providing a clear path for sharing book-marks with other team members.
4.5. Code snippets
Although the design of the bookmark tool emphasizesflexibility, creating bookmarks to code-bearing webpages—tutorials, templates, examples, snippets of codeposted in forums—may still be a common case. In thisscenario, Codetrail can automate the process by identify-ing when code is copied from the web into a source file.Our approach is not limited to direct copy-and-pasteactions, since our observation suggests programmers willoften copy by hand in order to better understand the code,or to avoid introducing errors when undesired or incorrectpieces of their source are incorporated into the target.Instead, Codetrail examines all newly written code as it issaved to disk, and compares those pieces of source code topotential code blocks on recently browsed web sites.
Those potential blocks are currently identified using asimple heuristic that searches for HTML pre elements,corresponding to pre-formatted text, often used fordisplaying source code with a conventional fixed-widthfont. Correspondence between saved and browsed code isdetermined using the Smith–Waterman algorithm forlocal sequence alignment [15]. Code matches are classifiedas token sequences representing at least 25% of thechanged segment and of the potential web source withan alignment score of at least 5 (gap penalty of 1). Whensuch a code match is identified, a bookmark is auto-matically created.
Automatic creation of bookmarks to record the sourceof copied code serves to retain bread-crumbs that canlater allow developers to answer questions like ‘‘why did Ichoose to write the code this way,’’ or ‘‘that variable namelooks wrong, is it a bug?’’ Knowing the provenance of thecode can also help explain what it does and how it works,since the web page from which code was adapted mayoffer more context, discussion, and caveats.
5. Discussion
With the tools in the previous section, we havedemonstrated a few different automated means ofidentifying that project and web resources are related:by finding tokens common to code and URLs (for genericdocumentation attachment) or to code and web pagecontents (copied snippet identification), and by language-specific mechanisms (Javadoc attachment). Other techni-ques could be employed as well; for example, work onread wear [4] could be applied to identify relatedresources by their frequent use during development
ARTICLE IN PRESS
M. Goldman, R.C. Miller / Journal of Visual Languages and Computing 20 (2009) 223–235230
sessions. In deciding how to automate identification ofrelevant resources, a primary design constraint was thatthe technique must reveal not only the fact that a resourceis related, but also the nature of their relationship. Forgeneric documentation attachment in particular, thismeans that our approach enables Codetrail to linkkeywords in the user’s code to matching documentationweb pages even if those keywords never before appearedin the code, and the particular web pages have never beenvisited, as long as the use of other keywords previously ledto the discovery of the relevant web site.
For recording the relationship between code and webresources, we have already suggested that current practiceoften takes the form of URLs in source code comments,and outlined how this is often insufficient. Links as URLsin the source code also fail in the case of documentationattachment: since a site may document many keywordsappearing throughout many source code files, there is nosingle best location in the source to indicate the relation-ship. Thus, we prefer a strategy of recording relationshipswith an independent representation that can be storedand shared apart from source code.
6. User study
In order to evaluate the tools implemented in Code-trail, we conducted a user study with 11 participants, 4of whom are in our research group. Users downloadedand installed a version of Codetrail that was instrumentedto record a variety of programmatic and user interfaceevents in both Eclipse and Firefox. The system wasinstalled by participants on their own computers. Beforeusing Codetrail, users provided optional demographicinformation and were given the opportunity to readdescriptions or view short screencasts demonstratingsome of Codetrail’s features: documentation attachment,automatic browsing to documentation, and creatingbookmarks.
Usage logs collected from the study were periodicallyreported automatically as long as participants continuedto use the plug-ins. Participants were also sent a post-study survey via email, to which seven replied; this surveysolicited their feedback on Codetrail and probed forpossible improvements to the system.
All of the user study participants were male, most intheir 20s or 30s. All indicated that they do softwaredevelopment in Java; Python was the second most oftenmentioned language. Three identified themselves asprofessional software developers; the rest identified asstudents or researchers. We did not request specificdetails of participants’ current projects or tasks.
Participants used Codetrail for an average of 17.6 days(s ¼ 11:9), not counting days during which no eventswere recorded in either Eclipse or Firefox (e.g. forinstallations on a professional developer’s office compu-ter). We counted as active development time all timeintervals within 1 min of a user interface event in Eclipse(selecting windows, saving files, etc.). Participants aver-aged 24.6 h of active development during the study(s ¼ 27:9).
During that development time, each user browsed toan average of 74.5 URLs per hour (s ¼ 58:7). This numbermay have been influenced by the presence of Codetrail,but supports our assertion that the web is an importantresource for developers.
6.1. Automatic browsing to documentation
We tracked usage of the automatic browsing todocumentation feature (Section 4.2) by logging whenusers activated the browser tab created by Codetrail toshow documentation, and when they gave the Firefoxapplication focus with that tab foremost. Nine of theeleven participants used this feature; those who used itdid so an average of 7.9 times per development hour,viewing an average of 3.1 different documentation pagesper hour.
In survey feedback, participants who used this featurewere unanimously positive:
‘‘I used this feature often. I found it very useful’’(User 4).
‘‘I used (and still do) this feature all the time. This isone of the most useful features that I’ve encountered inEclipse ever since I started using the IDE’’ (6).
‘‘It’s much better than popup dialogs/hovers whichobscure the code’’ (11).
Some users suggested further automation of the brows-ing, for example: ‘‘I did use this feature, and I found ituseful. I feel it would have been nice if some hotkey wouldput the focus on Firefox, and bring the special tab to thefront’’ (7). And one user found it problematic that Codetrailwould always update the tab to the documentation of theidentifier under the Eclipse editing cursor, even if he wasstill using the previous page: ‘‘... it was too quick to browse*away* from documentation that I still needed’’ (10).
Additionally, participants’ use of the automatic brows-ing feature was almost exclusively with Javadoc docu-mentation sources that were already known to Eclipse.These sites included:
�
Java 5.0 API (http://java.sun.com/j2se/1.5.0/docs/api), � Java 6 API (http://java.sun.com/javase/6/docs/api), � JUnit 4.3 API (http://www.junit.org/junit/javadoc/4.3).Use of the features to automatically identify and attachsources of Javadoc and generic documentation (Sections4.1 and 4.3) was limited: four users elected to attachJavadocs at least once, and none used the genericattachment mechanism.
For Javadoc attachment, a total of nine attachmentprompts were shown to those four users, who instructedCodetrail to do the attachment every time. URLs attachedincluded:
�
New JSR 166 Java 7 APIs(http://gee.cs.oswego.edu/dl/jsr166/dist/jsr166ydocs), � JavaMail email & messaging API(http://java.sun.com/products/javamail/javadocs),
� JUnit API, current version(http://junit.sourceforge.net/javadoc),
ARTICLE IN PRESS
M. Goldman, R.C. Miller / Journal of Visual Languages and Computing 20 (2009) 223–235 231
�
ImageJ image processing & analysis API(http://rsbweb.nih.gov/ij/developer/api), � A proprietary API(file:/C:/build/dev[...]/[...]Framework/main/javadoc).
All but the last of these URLs represent the documenta-tion for Java libraries used by participants, precisely theuse case for which this feature was intended. The last URLrepresents an instance in which the user has attacheddocumentation for libraries developed internally to hisemployer. Regarding these attachments, one participantcalled the feature ‘‘very useful’’ (8); another, his ‘‘preferredway to attach online documentation to an existingproject’’ (4).
In contrast, while 86 generic attachment prompts wereshown to eight users, none were affirmed. This largenumber is due in part to a bug in the version of Codetrailinitially used in the study, which failed to remembernegative answers and would prompt users again onsubsequent visits to the same site.
User feedback was lackluster, and several usersindicated that documentation sources were not accuratelyidentified:
‘‘I noticed this feature, but I found it sometimesmisinterpreted my behavior’’ (2).
‘‘I think I used this [feature] once to attach a Javadoc,and I found that useful, but the system was never correctwhen it asked me to link any other sort of documenta-tion...’’ (7).
At least some users were also unsure about how thefeature would work, being familiar with Javadocs butunsure how other documentation sources would beintegrated with Eclipse: ‘‘At the end, I don’t believeI ended up attaching any of the pages. I think the mostlikely reason was that, even if I had attached any page,I wouldn’t know where to look for it’’ (6).
We conclude that the automatic browsing feature ofCodetrail, and the use of Firefox as an IDE view fordocumentation, will be appealing and useful to a largepopulation of developers. Automatically attaching doc-umentation is convenient, but more work is needed tomake it effective with documentation that does notprovide a machine-readable description of the codeelements it documents. The heuristics must be moreaccurate, taking into account more than URL keywords;and the consequences of attachment must be clear todevelopers, perhaps with a more descriptive attachmentprompt and immediate feedback in Eclipse.
6.2. Bookmarks and code snippets
The bookmarks feature (Section 4.4) was largelyunused. Five of the eleven user study participants createdone or two bookmarks; the rest did not use the feature.Nearly all of those bookmarks appear to be created inorder to test or explore the Codetrail interface, and nonemeet the subjective self-evidently development-relatedtest used in the initial exploratory study (Section 3).
Although users were asked to describe why the featurewas not useful or how to improve it, they offered limited
feedback. One user made reference to the commonpattern of re-finding as an alternative to bookmarking:‘‘I don’t use bookmarks much... I generally just searchagain’’ (User 7). Another participant wrote that he ‘‘didn’tuse this feature actively. I put some URLs in source codecomments... however, because that’s how I habituallyinclude links...’’ (10).
These comments suggest that web bookmarks asimplemented were not in line with our participants’development practices. Other survey feedback pointed toa lack of knowledge of this feature in Codetrail: ‘‘I wasn’treally sure how to create bookmarks or where to look forthem inside Eclipse’’ (6). This feature of the system wasthe only one in which participants had to explicitly enablean Eclipse view in order to make use of it. Since seven ofthe eleven participants did so, we conclude that while thebookmarks interface could certainly be improved, dis-coverability was not the primary barrier to uptake, andthat our model of how bookmarks link code artifacts toweb resources may require revision.
Codetrail also automatically identified 33 snippets ofcode duplicated by participants and created bookmarksfor them (Section 4.5). Six participants logged at least onesnippet. Based on informal discussions with three of thestudy participants accessible to us, some of these snippetsindeed represent code copied from Java API documenta-tion pages, including:
�
Exception handling(http://java.sun.com/javase/6/docs/api/java/lang/Throwable.html), � Cache flushing(http://java.sun.com/javase/6/docs/api/java/util/LinkedHashMap.html).
In other cases, the snippets represent instances wherethe user was viewing in Firefox part of the same sourcecode he was editing in Eclipse, either because the codewas being posted online, or because some code wasreproduced in the output of an automated build system.
Without anecdotes describing the use of this feature inits primary intended mode—copying source code fromexample or tutorial pages—and since URLs and matchingcode segments of participants’ suspected code cloneswere not collected for privacy reasons, it is difficult toevaluate this feature. Most users said they had not notedor used the feature. One was positive: ‘‘I saw thosebookmarks created in one case, and I was happy becausethey were the right ones’’ (10). Another opined that ‘‘...thecode copying stuff would be useful if an organization hada library of such snippets accessible via a web front-end—which is not the case here’’ (11), a reminder of howthis feature may need specialization for use in a closed-source setting. Overall, we conclude that these bookmark-related features of Codetrail need more refinement tobecome useful. In particular, bookmark storage in sourcecode, as opposed to metadata files, merits examination.
To close this user study discussion, we note that 10 ofthe 11 participants continued to have Codetrail installedand enabled for at least a week after the study ended,
ARTICLE IN PRESS
M. Goldman, R.C. Miller / Journal of Visual Languages and Computing 20 (2009) 223–235232
based on continued submission of usage logs. Of those,seven participants made use of the automatic documenta-tion browsing feature, including one who had not usedthat feature during the study period itself. We take this asstrong evidence that, for at least some users, the systemprovides substantively useful integration of the IDE andweb browser.
7. Infrastructure
Both Eclipse and Firefox provide plug-in mechanismsfor extending their functionality, and Codetrail uses thesefacilities. The design of Codetrail was motivated by thefollowing goals:
Shared data: Both Firefox and Eclipse are powerfulcomputation engines in their respective domains, butmost essential to Codetrail’s operation is the sharing ofdata between them. Much of Codetrail’s power is derivedfrom combining data produced by or (previously) knownto only the individual applications. At least session-longpersistent data are a requirement to enable, for example,heuristics that rely on the user’s recent web browsing orcode editing history.
Publish-subscribe: Components in both applicationsmust be able to listen for patterns of data produced byother components and take appropriate action in re-sponse.
User feedback: In order to facilitate a responsive userinterface, components must be able to present userfeedback at the source of an event. Immediate at-the-source feedback is important especially when the usershould be prompted about their current web browsinglocation, and we wish to present that prompt while theuser is still focused on that site.
Modularity: As a research system, the ability to addnew modules to the system, rapidly explore new ways ofcombining data and functionality, and keep dependenciesbetween modules at a minimum was necessary. Modulardesign is also important to users, so that individual piecesof functionality can be enabled or disabled by developerswho find them more or less useful.
Cross-platform: We require a cross-platform systemthat works for Eclipse and Firefox users on all three of themajor operating systems—Windows, Mac OS X, andLinux—where they are used.
The implementation of Codetrail uses Java in Eclipseand JavaScript in Firefox. A simple socket protocol is usedfor communication, and the pool of shared data isrepresented by an RDF graph (resource description frame-work, in which data are modeled as (subject, predicate,object) triples forming a directed labeled multigraph) [10].The graph is managed in the Eclipse JVM using the JenaRDF framework [12]. It is exposed in Firefox according tothe existing Mozilla interface for RDF datasources. The useof RDF in a tuple space is similar to other work onsemantic tuple spaces [7,13].
The functionality of Codetrail is divided into separatemodules. The system maintains a set of currentlyactivated modules, where each module implements someintegrative functionality, and is coupled to other modules
only by virtue of the data it produces or consumes and theschema of those data. The user has control over whichmodules are active in order to selectively enable function-ality useful to them.
7.1. Shared data
Codetrail modules communicate and coordinate byadding RDF triples to the shared graph, and subscribingthemselves as listeners for patterns of new triples. Forexample, one module we have implemented tracks theuser’s web browsing and adds to the graph structures withdata about each page load (such as the URL). Anothermodule is concerned with automatically identifyingJavadoc sites in order to attach them in Eclipse. Thismodule listens for updates from the browsing module,identifies Javadoc sites, and takes the appropriate actions.
Fig. 6 shows the evolution of the shared RDF graph overtime with these two modules in operation, based ontriples added by the browsing module. Initially (Fig. 6a),we suppose there are structures in the graph correspond-ing to two URLs, perhaps from previous browsing activity.One of these URLs is in an #ignore relationship with#mod-javadoc-attacher, indicating that while it is aJavadoc site, the user has previously elected not to attachit. If the user browses again to that site, the graph isextended as shown in Fig. 6b with a structure recordingthe page load. Browsing to the second, non-ignored site,would add the structure shown in Fig. 6c.
Adding to the graph is straightforward, but subscribingto notifications about those additions is more complex. Tosubscribe to updates, graph listeners specify both atemplate triple and an anchored query. The template tripletakes the form (subject, predicate, object), where each ofsubject, predicate, and object may be null. This template iscompared to new triples added to the shared graph, withnulls acting as wildcards.
When the template matches a new triple, the listener’sanchored query is executed. This query uses the SPARQLlanguage for querying RDF graphs [14]. It is anchored
because it may include a particular clause that assignsvariables in the query to elements of the newly addedtuple which matched the template and triggered queryexecution. If the results of executing the query—whichmay include additional filters—are nonempty, they areforwarded to the listener’s callback method. This anchoredquery mechanism enables listeners to extract relevantinformation from the updated graph without having tofilter out matches from old data, combining the expressivepower of a relational query with the efficiency of templatematching for examining new triples. Template triplesalone cannot match a significant amount of data, solisteners would inevitably have to run queries to select outmore. Subscription with queries alone, on the other hand,would require the system to re-run all queries after everychange to the graph, and to maintain a record of what datahad already been returned by a given query so as not toreport it again.
Returning to our example, the automatic Javadocidentification module of Codetrail is a listener with
ARTICLE IN PRESS
http://example.com/old-javadocs/allclasses-frame.html
#url
#mod-javadoc-attacher
#ignore
http://example.com/new-javadocs/allclasses-frame.html
#url
#event-1 #artifact
#source
#firefox http://example.com/old-javadocs/allclasses-frame.html
#url
#mod-javadoc-attacher
#ignore
#load
#action
http://example.com/new-javadocs/allclasses-frame.html
#url
#event-1 #artifact
#source
#firefox http://example.com/old-javadocs/allclasses-frame.html
#url
#mod-javadoc-attacher
#ignore
#load
#action
#event-2 #artifact
#source
http://example.com/new-javadocs/allclasses-frame.html
#url
#action
Fig. 6. The shared RDF graph (a) before browsing, (b) after browsing to an ignored Javadoc site, and (c) after browsing a new Javadoc site.
#load
#action
?s #artifact
#source
?source
?url.*/allclasses-frame.html
#url
?j
#ignore
?o
?p
?j?j?j
Fig. 7. Example (a) template triple and (b) anchored query used by the automatic Javadoc identification module to observe visits to potential sources of
Javadoc documentation.
M. Goldman, R.C. Miller / Journal of Visual Languages and Computing 20 (2009) 223–235 233
ARTICLE IN PRESS
M. Goldman, R.C. Miller / Journal of Visual Languages and Computing 20 (2009) 223–235234
template triple (null, #action, #load), which matches newtriples representing events whose action is a web pageload. This template triple is illustrated in Fig. 7a.
The Javadoc module also specifies an anchored query(Fig. 7b) that filters matching page load events to those forallclasses-frame.html, a file name characteristic of Javadocs,and which are not for URLs previously blacklisted by theuser. The query further selects out just the pertinentproperties of the page load event to return to the listener:the source of the page load (i.e., which instance of Firefox)and the URL. The SPARQL query is written as:
SELECT ?source ?url WHERE {
G
RAPH :match { ?s ?p ?o }{
?s :source ?source .?s :artifact [ :url ?url ] .
OPTIONAL { ?j :ignore ?url } .
FILTER regex(?url, ’’.*/allclasses-frame.html’’) .
FILTER ( ! bound(?j))
}
}
In the first page load of Fig . 6a and b, the templatetriple will match (#event-1, #action, #load), and theanchored query will be executed. Line 2 of the query willanchor the results to this matched triple, and the querywill return no results, because #mod-javadoc-attacher isbound to variable ?j in line 5. Thus, the blacklisted URL isignored. In the subsequent visit (6b and c), the templatetriple will anchor the query on (#event-2, #action, #load)and a result containing the source and url will beforwarded to the Javadoc module.
Finally, in addition to modifying and observing thegraph, some Codetrail modules record pieces of the sharedgraph as XML files in the user’s Eclipse projects: book-marks and generic documentation specifications arestored in this way. Changes to these files are synchronizedwith the in-memory representation so that the effects ofnew files (e.g., a bookmark just received via versioncontrol update) and power users’ manual edits areimmediately reflected in Codetrail’s behavior.
8. Future work
Codetrail is a working system that has been developedwith the feedback of users inside and outside our researchgroup, but future work remains. In the system itself, weplan to continue adding new modules and improvingexisting ones, with a special focus on scalability that willallow Codetrail to work efficiently with large Eclipseprojects. A number of components, including genericdocumentation identification and copied code tracking,use heuristics that can also be improved with furthertesting.
Although we have presented some preliminary dataon developers’ web usage habits, more remains to belearned. Our user study of Codetrail has shown that whilethe automatic browsing feature in particular was appre-ciated, there is more to be understood about how tosupport programmers’ use of web resources, particularlywith respect to bookmarks. And while the user studydemonstrated subjective satisfaction on the part of the
participants, we hypothesize that Codetrail will improvesoftware developers’ measurable effectiveness as well.Specifically, we predict that Codetrail can reduce theamount of time developers spend navigating to web pagesand the amount of time they spend re-finding previouslyvisited pages, as well as the cognitive load of theseactivities; and that these reductions will lead to increasedperformance in code understanding, authoring, anddebugging tasks. Empirical evaluation of these hypotheseswith a controlled laboratory or field experiment remainsto be undertaken.
The ultimate resources in the software developmentprocess are people, and another area for future work isCodetrail-supported collaboration and communicationbetween developers, further exploiting existing channelssuch as project version control, or creating new ones.
9. Conclusion
In this paper we have presented a taxonomy ofsoftware developers’ web usage as motivation for inte-grating the IDE and web browser, as well as anarchitecture for achieving that integration, tools that canbe built using it, an implementation of the entire system,and an evaluation of that implementation.
Web resources of a wide variety are important to theprogrammer’s development process, and Codetrail pro-vides both specific mechanisms to automate previouslymanual interactions with web resources, as well asgeneral mechanisms to integrate those resources intothe development environment where it was previouslytedious. The several tools described here, includingautomatic detection and connection of documentationand the creation of links, or bookmarks, from code torelevant web sites, demonstrate the advantages of bring-ing the user’s full-featured web browser into the IDE as aparticipant in the development process. Treating thebrowser as a view, by automatically loading contentrelevant to the source code currently being edited, is onenotable instantiation of this idea.
The documentation tools in Codetrail aim to satisfy theimportant goal of bringing needed information to thedeveloper as quickly and fully as possible. The moregeneral bookmarking function helps improve the collec-tive memory of a software project by keeping alongsidethe code a record of resources that helped shape that code.
Developers do not need a watered-down web browserin their IDE; they need their IDE to integrate with the webbrowser they already use and have customized to theirliking. Unlike many of the tools in an IDE, the web browseris an application platform in and of itself, and must betreated as a first-class citizen in any integration scenariobefore developers will be satisfied. Codetrail demonstrateshow this combination may be achieved with Eclipse andFirefox.
Acknowledgments
We thank members of the UID group for their valuablefeedback and all study participants for their time,
ARTICLE IN PRESS
M. Goldman, R.C. Miller / Journal of Visual Languages and Computing 20 (2009) 223–235 235
cooperation, and feedback. This work was supported inpart by the National Science Foundation under award IIS-0447800, by Quanta Computer, and by DARPA. Opinions,findings, conclusions, or recommendations expressed arethose of the authors and do not necessarily reflect theviews of the sponsors.
References
[1] L.-T. Cheng, S. Hupfer, S. Ross, J. Patterson, B. Clark, C. de Souza, Jazz:a collaborative application development environment, in: OOPSLA’03, 2003, pp. 102–103.
[2] M. Goldman, R. Miller, Codetrail: connecting source code and webresources, in: Proceedings of VL/HCC ’08, 2008, pp. 65–72.
[3] A. Habermann, D. Notkin, Gandalf: software development environ-ments, IEEE Transactions on Software Engineering, 1986.
[4] W.C. Hill, J.D. Hollan, D. Wroblewski, T. McCandless, Edit wear andread wear, in: CHI ’92, 1992, pp. 3–9.
[5] R. Hoffmann, J. Fogarty, D.S. Weld, Assieme: finding and leveragingimplicit references in a web search interface for programmers, in:Proceedings of UIST ’07, 2007, pp. 13–22.
[6] M. Kersten, G.C. Murphy, Using task context to improve programmerproductivity, in: Proceedings of SIGSOFT ’06/FSE-14, 2006, pp. 1–11.
[7] D. Khushraj, O. Lassila, T. Finin, sTuples: semantic tuple spaces,Mobile and Ubiquitous Systems: Networking and Services, 2004.
[8] A. Ko, H. Aung, B. Myers, Eliciting design requirements for main-tenance-oriented IDEs: a detailed study of corrective and perfectivemaintenance tasks, in: Proceedings of ICSE ’05, 2005, pp. 126–135.
[9] A. Ko, R. DeLine, G. Venolia, Information needs in collocatedsoftware development teams. in: Proceedings of ICSE ’07, 2007,pp. 344–353.
[10] O. Lassila, R.R. Swick, Resource description framework (RDF) modeland syntax specification, hhttp://www.w3.org/TR/1999/REC-rdf-syntax-19990222/i.
[11] T. LaToza, G. Venolia, R. DeLine, Maintaining mental models: a studyof developer work habits, in: Proceedings of ICSE ’06, 2006,pp. 492–501.
[12] B. McBride, Jena: a semantic web toolkit, Internet Computing, 2002.[13] L. Nixon, O. Antonechko, R. Tolksdorf, Towards semantic tuplespace
computing: the semantic web spaces system, in: Proceedings of SAC’07, 2007, pp. 360–365.
[14] E. Prud’hommeaux, A. Seaborne, SPARQL query language for RDF,hhttp://www.w3.org/TR/rdf-sparql-query/i.
[15] T. Smith, M. Waterman, Identification of common molecularsubsequences, Journal of Molecular Biology 147 (1981) 195–197.
[16] J. Stylos, B. Myers, Mica: A web-search tool for finding APIcomponents and examples, in: Proceedings of VL/HCC ’06, 2006,pp. 195–202.
[17] D. Cubranic, G. Murphy, Hipikat: recommending pertinent softwaredevelopment artifacts, in: Proceedings of ICSE ’03, 2003,pp. 408–418.
[18] D. Cubranic, G. Murphy, J. Singer, K. Booth, Hipikat: a projectmemory for software development, Software Engineering, 2005,pp. 446–465.
[19] M. Wicks, R. Dewar, Ad hoc tool integration; a case study, TheInternational Conference on Software Development, 2005.
[20] Y. Yang, J. Han, Classification of and experimentation on toolinterfacing in software development environments, in: Proceedingsof Asia-Pacific Software Engineering Conference, 1996.