literature review as a research method - aalto of this lecture • literature review as a research...

Literature review as a research method T-76.5050 Methods for Software Engineering and Business Research Varvana Myllärniemi [email protected]

5.10.2015 Varvana Myllärniemi, [email protected]

1

mailto:[email protected]

Overview of this lecture

• Literature review as a research methodology in software engineering

• Conducting the literature review

• To think about during this lecture – Why and when to do literature review(s) during your thesis work? – How to conduct the review for your thesis work?

5.10.2015

Varvana Myllärniemi, [email protected]

2

Literature review as a research methodology in software engineering

5.10.2015


3

A literature review…

Identifies, analyzes and synthesizes available relevant research

to a particular research question or topic (Kitchenham, 2004)

A study that uses other studies as its data

5.10.2015


4

Primary study

Secondary study Identifies

Analyzes Synthesizes

Motivation

Why to do a literature review?

5.10.2015


5

Primary study

Secondary study Identifies

Analyzes Synthesizes

Motivation

• Identify gaps in the current research

• Understand an established but fragmented topic

• Understand a new research topic

• Study how a theory or method is supported by empirical evidence

(Webster & Watson, 2002; Kitchenham, 2004)

5.10.2015


6

Literature review: standalone or supporting method?

5.10.2015


7

Bachelor’s theses Literature review

as the main contribution Seminar reports

Scientific publications

Master’s theses Literature review

to support the main contribution

PhD theses

How to support the main contribution with a literature review?

Consider the following Master’s thesis:

Research question: What are the challenges of applying

agile development methods in global software development? Research method: Case study

5.10.2015


8

5.10.2015


9

Background

Previous work Comparison with previous work

Research Problem

Research Questions

Research Method

Results Support in understanding and formulating

Are used to find and analyze

Is used to evaluate and argue novelty

Is used to do

Possible challenges

Similar challenges identified?

Using agile methods for global SW development

Agile methods Global SW

development

5.10.2015


10

Background

Previous work Comparison with previous work

Research Problem

Research Questions

Research Method

Results Support in understanding and formulating

Are used to find and analyze

Is used to evaluate and argue novelty

Is used to do

Conducting the literature review

5.10.2015


11

Conducting the literature review

5.10.2015


12

Analyzing

Writing

Selecting

Searching

Planning

Potentially relevant studies

Review protocol

Relevant studies

Manuscript

Analysis and synthesis

How systematic

and explicit?

Planning

• Identifying the need to the review • Specifying the research questions (standalone

review) • Identifying the topics to be covered from the

research questions (supporting review) • Developing and validating a review protocol

The level of planning often decides how systematic and explicit

your literature review will be

5.10.2015


13

Planning Review protocol

Searching

5.10.2015


14

Searching Potentially

relevant studies

Database searching Snowballing

How to deal with the huge number of potential studies?

Database searching

5.10.2015


15


relevant studies

Database searching

Construct search strings

Select databases Iterate

Database searching

5.10.2015


16


relevant studies

nelliportaali.fi

Don’t use Nelli directly to search

for studies

ScienceDirect Elsevier Wiley Interscience

Web of Science Scopus

PR

IOR

ITY

Goo

gle

Sch

olar

IEEE Xplore ACM Digital Library

SpringerLink

Select databases

Database searching

Database searching

5.10.2015


17


relevant studies

Database searching

Construct search strings

Research question: What are the challenges of applying

agile development methods in global software development?

(agile <or> lean) <and>

(global <or> offshor*)

Iterate

Add concepts to get

fewer hits

Add synonyms from relevant

studies

Leave concepts for

analysis (<or> outsourc*) (<and> “software

development”) (challenges)

Snowballing

5.10.2015


18


relevant studies

Snowballing

Backward snowballing

Which studies a relevant study

cites?

Forward snowballing

Which studies cite a

relevant study?

Recommended!

More effort, needs a reference DB (Web of Science, Scopus)

Selecting studies

5.10.2015


19

Selecting Relevant studies Potentially relevant studies

Q: How to select relevant studies? … and to do it systematically?

A: Apply explicit inclusion and exclusion criteria derived from the research questions.

Included if: Addresses any agile or lean SW development method

AND Addresses global SW development

Excluded if: Does not concentrate on SW development

OR Does not report any challenges OR Is not a peer-reviewed publication

Selecting studies

5.10.2015


20

(modified from Dybå et al., 2007)

Selecting Relevant studies Potentially relevant studies

Q: How to deal with a huge number of potentially relevant studies?

Include / exclude studies based on titles

Potentially relevant studies

Include / exclude studies based on abstracts

Include / exclude studies based on full paper content

Analyzing

5.10.2015


21

Primary studies = Data

Some analysis activities

Coding: mark all passages of text that are related to a certain study concept

Extract data about a certain study concept

to a concept matrix

Create a conceptual model

Analyzing Analysis and

synthesis Relevant studies

Analyzing

5.10.2015


22



Article Scrum Lean On-site customer

Smith et al, 2009 X Yes

Lee et al, 2013 x No

Steffen, 2014 x Not known

Woods et al, 2011 x Part-time

Temporal distance

Challenges

Geographical distance

Socio-cultural distance

Concept matrix

Conceptual model

Based on the results, it seems the practice of on-site customer is less frequently utilized in the outsourced development teams. This was…

On-site customer

Coding

Analyzing

5.10.2015


23

The most important phase in the literature review!

More than just a summary of the literature.

Focus on the concepts, not on the articles.



Writing

• Report the analysis and synthesis results

• Report the review method (all activities in all steps)

• Use present tense – ”Staples and Niazi (2007) report their

experiences on...”

– ”CMMI seems to be suitable for small organizations (Staples and Niazi, 2007).”

5.10.2015


24

Writing Manuscript

Writing

• Be careful in distinguishing findings in primary studies vs. your analysis – Also, keep own study setting / results out

• Do not cite a primary study without analyzing it well! – Citations based on abstract / introduction only – Would the original author agree with your

interpretation?

5.10.2015


25

Writing Manuscript

Conclusions

• Literature review – Identifies, analyzes and synthesizes research on a certain topic

• As part of a Master’s thesis – Gives background, identifies what is already known,

provides a yardstick for the results

• Conducting the literature review – Planning – Searching for primary studies – Selecting the primary studies – Analyzing – Writing

5.10.2015


26

References

Dybå, Tore; Dingsøyr, Torgeir; and Hanssen, Geir K. Applying systematic reviews to diverse study types: An experience report. Proc. of Symposium on Empirical Software Engineering and Measurement, 2007.

Kitchenham, Barbara. Procedures for performing systematic reviews. Technical report TR/SE0401, Keele University, 2004.

Staples, Mark and Niazi, Mahmood. Experiences using systematic review guidelines. Journal of Systems and Software, 80(9), 2007.

Tranfield, David; Denyer, David; and Smart, Palminder. Towards a methodology for developing evidence-informed management knowledge by means of systematic review. British Journal of Management, 14(3), 2003.

Webster, Jane and Watson, Richard T. Analyzing the past to prepare for the future: Writing a literature review. MIS Quarterly, 26(2), 2002.

5.10.2015


27

literature review as a research method - aalto of this lecture • literature review as a research...

Documents