INFO 4990: Information Technology Research Methods

Searching in the Research LiteratureLecture by A. Fekete

(based in part on materials by J. Davis and M. Maher)

“If I have seen further, it is only because I have stood

on the shoulders of giants”.

Attributed to Sir Isaac Newton

The literature

• Literature can help in finding a research problem– identify clear “next step” or “gap”

• It can also help you solve a problem– show how the field works (so you fit in)– provide evidence you can quote without repeating the work– provide the motivation to show importance

• eg our performance is better than that of [Cite]• eg [Cite] defined the following concept, about which we prove …• eg [Cite1, cite2, cite3] have all worked on systems like this.

• Critical (yet generous) reading.

Reading the literature

• Keep an annotated bibliography from the start– Complete bibliographical reference (including pages, dates)

• Detailed notes on each work– even if it seems irrelevant to your thesis– what is claim, what evidence, what argument, any doubts?

• Don’t rely on second hand summaries! Go to the original source always!– Get attributions right in your own writing

• (don’t just accept citations from other work, even with full reference!)

• Use comments and keywords to organise your thoughts.

Why literature review?

• Demonstrate that you know the field• Justifies your research, provides the rationale for the

research– how does your work differ from previous work– how does your work connect to previous work

• Allows you to establish the conceptual framework and methodological focus

Practicalities

• What is a citation• Getting the content• Finding citations

Citation

• Key pieces of information– That should be enough to uniquely identify the work and make access possible

• Authors• Title• Journal or other location

– Journal title (or standard abbreviation, eg JACM = Journal of the ACM)• With volume and number

– Proceedings of a conference (standard abbreviation eg SIGMOD)• With year

– Edited Collection • Date (Year, maybe also month)• Page numbers

Bibliographic files

• Keep data on all papers you read, and also those you get references to– Use standard formats eg BibTeX, EndNote

• Try to keep information precisely as in the published paper– When citing the work, you can/must adjust to the style of the place where you publish

• Capitalize titles or not• Use author initials, or fullnames, and maybe rearrange

• Keep title wording accurately as it appeared– Eg “Distributed Software Engineering with Versions” versus “Distributed SE with

Versions”– Don’t correct misprints!

• Keep author information accurately as it appeared– Eg “A. Fekete” versus “Alan D. Fekete”– Order of authors matters!

• ACM Digital Library provides data in standard formats for cut/paste into tools

Getting a paper

• Given a citation, you need to get the actual content• Librarians are expert at this

– Inter-library arrangements

• But try the simple things first by yourself– Look online

• ACM Digital Library• IEEE Digital Library• Publisher’s site• Google Scholar• Citeseer• Just use Google (with authors name and a few rare words from the title)

Most of the literature is now available in pdf,even very old papers have been scanned

ACM Digital Library

• http://portal.acm.org– Has contents of all the journals and conferences organised by ACM

and its subgroups– This is much of the key work in most areas of IT– Very useful search facilities– Also ACM has “The Guide” which is similar but has links to work from

other publishers too– University of Sydney subscription allows full access, if you go via

library home page (with library login)

• IEEE Xplore has similar facility for its conferences and journals: http://ieeexplore.ieee.org

DOIs

• Many important journals and conferences are published by commercial publishers– In IT, especially Springer Verlag see

http://www.springerlink.com• These now usually make the content accessible

permanently through the web as well as in bound paper-based format

• Digital Object Identifiers are special URLs that won’t change even if the publisher’s site moves domain etc

Web references

• It is now common to find work that exists only on the web– As distinct from finding (on the web) a pdf of work that has

a formal journal or other publication• Reference by giving the URL used, and the date on

which it was accessed– Because web information can change!

• Also, be very careful with getting a version of a published work from author’s web site, where it may be posted only in preliminary form

Finding relevant papers

• Your supervisor may point you to some– In many cases your “question” will be defined as

modifying/extending some existing work• You may find some by searching

– Based on terms of interest– Based on links from other work you find

• Links through references• Links through a community

Searching

• Search the whole web, or specific databases• Each search engine/database has its own format for

queries– Stemming or not?– Case sensitive or not?– Several words together mean AND, or exact phrase?– Don’t use the wrong one– Eg if you search for Data Replication, you may or may not

match a paper which includes “replicated database management”

Effective search

• We are all used to Google-style search– List words of interest

• Make sensible choices for your search– Prefer words that are not very common in general English

• Not much point searching for “design” or “information” or “relation”• But “congestion” or “heterogeneity” work well

– Often add a term for the field as a whole• Eg “database” or “network”

– Start with a few targeted terms• Add more if too much junk is returned

Extra features

• Most search tools offer a variety of features, especially important when words of interest are common– AND (implicit in Google)– Exact phrase match (Google: “words in exact order”)– Allow synonyms (Google: ~term)– Exclude docs with some term (Google: -term)

• This is useful when a term has several meanings in different domains– OR

• Also, search metadata not just content– Eg Google [intitle: term]

References as links

• Papers that are listed as references in an important paper are also worth examining– But this leads to older work

• Also, find (newer) papers that reference the important paper– Digital Libraries make this easy!

Communities of interest

• Your work should be targeted at a community– It must be explained in relation to work in that community

• Find who is in the community and where the work appears• Look at the authors of important papers

– Find their website (use Google!)– What else have they written– Who are their students– What current projects are they working on

• Look at the conferences/journals where the important papers appeared– What else is published there, especially recently

Filtering the lists

• It is easy to get hundreds of references, but its hard to read them all

• Use titles for a first filtering• Then look at abstracts

– If abstract seems relevant, then read the introduction– Don’t neglect papers that deal with different aspects of the domain,

because you will need to explain how your work differs from them• You should read some breadth as well as closely relevant

papers– Especially seminal work, or recent top conferences (so you can

motivate your work by showing how it connects to the current hot themes)

Organising the literature

• Isolate issues and highlight the findings and contributions that are central to your research

• Group together papers that deal with a common or related theme or issue

• Use diagrams, tables, concept maps to organise the materials• Try out different structures for organising; they should be most

relevant to the goals of your research• Chronological order is not particularly useful

– but citation chains are useful• Warning: papers often don’t use common terminology, or focus

on common issues, or explain relationships fairly– Clarifying these aspects is a key contribution you can make