1

SWM43 Research in Computing: Introduction to

Computing ResearchAnja Belz

Natural Language Technology GroupCMIS

A.S.Belz@brighton.ac.uk

2

Purpose of this lecture• To touch on all aspects of research at least

briefly, giving pointers to further reading (see list of references on last two slides)

• To look in more detail at the two areas that are of most practical relevance to you at the moment:– Planning research– Reviewing the literature

• To give practical guidance (including an exercise) on how to get started with research

3

OverviewPart I – Introduction

1. Four golden lessons2. What field are we in anyway?3. Examples of research projects in informatics

Part II – Preparing the ground4. Planning research5. Reviewing the literature6. Exercise: first steps in planning research

4

Part I – Introduction 1. Four golden lessons2. What field are we in anyway?3. Examples of research projects in

informatics

5

Four golden lessons (Weinberg, 2003)

1. No one knows everything, and you don't have to.

2. Go for the messes — that's where the action is.

3. Forgive yourself for wasting time. 4. Learn something about the history of

science, or at a minimum the history of your own branch of science.

6

What field are we in anyway? • Computing science, computer science,

information processing, information technology, software engineering … ?

• Now increasingly known as: Informatics • Old subdivisions:

– Computer science: study/build artificial systems– Artificial intelligence: emulate intelligence using

artificial systems– Cognitive science: study mind from

computational perspective… aren’t really separate anymore!

7

Informatics techniques (Bundy)

Types of techniques:• Theories• Architectures• Information representation• Algorithms• Software engineering processes

Research aims:1. Extend knowledge about properties of

techniques2. Improve existing techniques3. Create new techniques4. Combine techniques to create systems

8

Implications of technique type and research aim:– What broad phases your research will go through– What kind of evaluation is appropriate– What kind of use will be made of your research

Specific research field also important:– Fine-grained divisions: evolutionary robotics,

machine translation, computer vision, etc.– Standard tools and methodologies to apply– Terminology, knowledge you can take as given– Dissemination media and style

Informatics techniques (Bundy)

9

Examples of informatics research

• Academic: – Masters and PhD thesis research– Research internally funded by universities etc.; some

incidental, some more formally in projects– Publicly funded research projects (UK research

councils, European Commission, US National Science Foundation)

• Industry: R&D departments, research labs, dedicated research companies; limited dissemination

• Private: people working from their home computers, in their garages, attics, garden sheds– French girl who invented speech recognition

technology in her parents’ garage– Chinese farmer Wu’s robots– Participants in DARPA Autonomous Vehicle Challenge

10

Examples of informatics research

• Some Informatics MSc topics from University of Edinburgh (2003-07):– A P2P Network Visualiser – Automated Probability Assessment in Plausible

Crime Diagnosis – EvoTanks II: Co-evolutionary Development of Game

Playing Agents – Cooperative Multi Agent Systems in Automobiles – A Web Service Interface to Astronomical Databases – Clustering Tags of Social Bookmarking Sites

• Hardware Evolution: Automatic design of electronic circuits in reconfigurable hardware by artificial evolution (PhD, University of Sussex)

11

Examples of informatics research

• Probabilistic Deep Generation (3-year project, funded by EPSRC): “to develop, for the first time, a comprehensive, linguistically informed, probabilistic methodology for generating language that substantially improves development time, reusability and language variation in language generation systems, and thereby enhances their commercial viability”. Value (£):211,199, University of Brighton.

• Research Consortium in Speckled Computing (4-year project, funded by EPSRC): “a radically new concept in information technology […] realised by minute autonomous specks, each of which encapsulates sensing, programmable computation and wireless networking. Computing with minute specks will enable linkages between the material and digital worlds […] will be fundamental to truly ubiquitous computing”. Value (£):3,721,432, Edinburgh.

12

Part I – take-home points• Take on board Weinberg’s 4 golden lessons• There is a huge variety of research in

informatics– Be aware of the types of technique you’re

working on, and of what your overall research aim is

– It’s important to learn about the research methodologies, evaluation criteria and other conventions in your field of informatics research

• You can make an important contribution to research wherever you are

13

Part II – Preparing the ground4. Planning research

• First steps• Background reading• Methodology• Aims, outcomes, deliverables• Requirements• Subdividing and estimating effort: work

packages• Writing a research proposal

5. Reviewing the literature

14

Planning research – first steps

• Aim: Come up with a rough sketch of the research you want to do

• Identify an area of informatics that interests you• Do some superficial background reading:

– Wikipedia – but bear in mind it’s not 100% reliable, and don’t cite it!– Course web pages from leading universities– Websites of professional organisations– Research project websites– Individual researchers’ webpages

• Decide on a smaller area in which to locate your research (but still larger than your project), and identify several ways in which you could make an original contribution

15

Planning research – first steps

• Start identifying key characteristics of your chosen area:– Important conferences– Leading journals– Internationally leading researchers– Current research projects– Mailing lists– History: when did it begin? What are key

advances, when did they happen?– How good is current technology? – What are the issues research is currently

grappling with?

16

Planning research – background reading

• Aim: familiarise yourself with chosen field of research well enough to decide on your project and write a short outline of it

• Read survey papers and look up textbooks• Start reading (abstracts of) academic papers

in conference proceedings and journals• Start compiling a bibliography (with star

ratings)• Use tools like Google Scholar to check status

of publications• Continue to collect key characteristics of area• Write project outline (a few sentences)

17

Planning research – methodology

• Aim: to decide on the technical details of how you’re going to carry out your research

• Won’t be able to specify all of this in advance – some of it is necessarily part of doing the research

• E.g. if building a system: outline of architecture and functionality of modules; general approach (e.g. symbolic or statistical), even algorithms

• Good idea to include fall-back options (if A doesn’t work I’ll do B)

18

Planning research – aims, outcomes, deliverables

• Aim: clarify the purpose of your research to the point where you can write it down in detail

• Aims: overall goals you hope to achieve with your research

• Outcomes: specific results you plan to achieve with your research– Will knowledge be increased? How?– Will new resources be produced? Which ones?– Will new techniques be created? Existing ones improved?

• Deliverables: the specific documents, software and other resources you commit to producing, with deadlines– Technical reports, manuals, webpages, etc.– Software specifications (modules, interfaces), tools, systems, etc.– Data collections (database of images, corpora of texts, etc.)

19

Planning research – requirements

• Aim: to determine everything you will need to carry out your project, apart from your own time and effort

• Are you going to carry out experiments involving people? How many subjects? Will the university’s research ethics allow it?

• Programming environments, tools, your skills.• Equipment, data, licenses, etc.• Will any of it cost anything? Where will the

money come from?• If you’re not sure, find out now!

20

Planning research – work plan

• Aim: to create a detailed research plan which lists work packages, specifies the amount of time required for each and assigns a time slot to each

• Divide tasks into related groups (work packages, WPs); write short descriptive summary for each WP

• Estimate time/effort each WP will take (person days or weeks) – always add contingency!

• Establish partial order of WPs – which WP requires other WPs to have been completed?

• Create a calendar diagram where each WP is assigned a slot – don’t forget to allow for other commitments

21

Planning research – writing a research proposal

• Aim: to put the results of your planning work into prose that will convince people that your planned research is of quality and worthwhile

• Actually very time-consuming!• Sections:

– Synopsis – “executive summary”– Introduction – motivate your research, why is it needed?– Aims, outcomes, and objectives– Related research – compare and contrast with existing

work– Methodology – describe what you’re going to do, clarifying

what is new, and where you’re going to use existing resources and ideas

– Research in wider context – beneficiaries, dissemination, marketability, etc.

– Work plan– Bibliography

22

Planning research – writing a research proposal

• Give it to different people to read – for: – Grammar/style: does it read well?– Clarity: are your aims and plans clear?– Quality: is this a good idea?

• Look at general advice on academic writing

23

Writing a literature review• Literature review can build on, but goes

beyond, research planning• More in-depth reading/understanding

than you need for project proposal• Typically part of reports of completed

research (Masters and PhD theses, project reports etc.)

• Or publications in their own right: survey articles in journals or as book chapters

24

Writing a literature review• Aim: to thoroughly review a given area of

research, mentioning all important relevant research

• Two basic forms:– Survey/state-of-the-art: balanced overview of given

area of research; inclusion and space reflect importance of work in field; keep opinion to minimum

– Project-specific review: inclusion and space reflect relevance to project; lead up to justification and motivation for project; opinion is important part of review

• Important difference: for surveys, you don’t need to understand in detail how techniques work, but if it’s relevant to your project you do need to

25

Examples of survey-type literature reviews

• Emotional language generation: http://www.itri.brighton.ac.uk/~Anja.Belz/Publications/ITRI-03-21.pdf

• Speech technology (1994): http://doi.acm.org/10.1145/175247.175252

26

Writing a literature review – reading

• Identify relevant publications• Determine status/importance (e.g. use Google

Scholar to check on citations) – you can’t read all relevant publications

• Use star-ratings to reflect importance• Read at different depths

***: read article in depth**: skim article*: read abstract and conclusions

• Build up annotated bibliography (title, publication details, summary of contents, your comments)

27

Writing a literature review – writing

• Don’t just list names and contents of papers – that’s an annotated bibliography, not a literature review

• Turn it into a story – tell the story of the field• How does it all fit together?• What are the subfields, developments,

controversies?• What is the state of the art?• What are the hot topics at the moment (reflected

in special themes and special sessions at conferences, one-off workshops, and special issues of journals)

28

Writing a literature review – writing

• Give your review structure:– Introduce the field– Bring out commonalities and differences between

approaches– Can key results be summarised in a table or a graph?– Comment on more/less successful approaches– Conclusion:

• Survey: summarise the state of the art of the field• Project-specific: identify area(s) where more research is

needed– Bibliography– Appendix: research groups, data resources, web links

29

Part II – take-home points• Zoom in on your chosen area of research

gradually (don’t start with academic papers):– Superficial reading of online material– Background reading of survey articles, text-books,

etc.– Literature review of research area of appropriate size:

academic papers in conference proceedings and journals, book chapters, etc.

• Read at different depths: – Read most important papers carefully– Skim less important papers– Read abstracts and conclusions of least important

papers

30

Part II – take-home points• Plan every aspect of your research

thoroughly and in detail• For a literature review, do not just list

publications and contents – instead, tell a story!

31

Exercise: first steps in research planning

Aim: write a short review of the area of Machine Translation and prepare a brief presentation of it

Steps:1.Look at p. 14, do online research, and fill in as many of the categories on p. 15 as you can;2.Write your findings up as a 1-page report;3.Prepare a presentation of your findings, about 5 minutes in length;4.Deliver the presentation on Friday morning.

Work in lab (W622) this afternoon, tomorrow morning and on Thursday; finish report and presentation for Friday.

32

References• Stephen Weinberg’s Four Golden Lessons:

http://www.nature.com/nature/journal/v426/n6965/full/426389a.html

• Types of Research in Computing Science, Software Engineering and Artificial Intelligence by Aaron Sloman: http://www.cs.bham.ac.uk/research/projects/cogaff/misc/cs-research.html

• Alan Bundy’s Researcher’s Bible: http://homepages.inf.ed.ac.uk/bundy/how-tos/resbible.html

• CMU’s Advice on Research and Writing: http://www.cs.cmu.edu/afs/cs.cmu.edu/user/mleone/web/how-to.html

33

References• A Computer Scientist's Guide to Writing and

Publishing Technical Articles by Paul Martin: http://www.mcs.vuw.ac.nz/comp/Publications/archive/CS-TR-95/CS-TR-95-4.pdf

• Why you can’t cite Wikipedia in my class by Neil Waters (2007): http://doi.acm.org/10.1145/1284621.1284635

• Robert Dale’s tips for presentations: http://www.nltg.bton.ac.uk/teaching/SWM43/dale-presentations.pdf

• Robert Dale’s advice on time management: http://www.nltg.bton.ac.uk/teaching/SWM43/dale-time-management.pdf

34

References• Cooper, H. (1998). Synthesizing Research: A

Guide for Literature Reviews. Main points summarised here: http://library.ucsc.edu/ref/howto/literaturereview.html