ICT

Research Questions

ICT

Agenda

Why research questions? Relevance Rigor How to formulate one?

Design and creation Positivist Interpretive

ICT

Why research questions?

Discussion

(Write down your research question!)

ICT

The thinking man

You

Other researchers

Your researchPractice

Rigor

Relevance

ICT

Practical vs. Research Problems

PracticalProblem

ResearchQuestion

ResearchProblem

ResearchAnswer

motivates

definesfinds

helps to solve

AppliedResearch

Purpose

Aim

Objective

ICT

Conceptual Problems

ConceptualProblem

ResearchQuestion

ResearchProblem

ResearchAnswer

motivates

definesfinds

helps to solve

Pure/BasicResearch

ICT

Practical vs. Research Problems

Condition Cost

Practical Problem An undesirable situation in the real world

Unhappiness, pain, material cost, social cost, etc.

Research Problem Not knowing or not understanding something

Cost of not knowing (and thereby not being able to solve the practical problem).

ICT

Examples

Practical problem Research question

An aging population leads to more fall injuries. These injuries are costly in terms of mental, social and economical costs.

• What do we know about the causes of falls among seniors?

• What do we know about the role of IT in preventing falls among seniors?

• What type of IT systems are best in preventing falls among seniors?

ICT

The "So what?" Test

Topic: I am studying X Question: Because I want to answer Y

Significance: In order to help solve real world problem Z

From "So what?" So what if problem Z is not solved?

To "Oh no! What do we do about that?" Problem Z is important also for me! Please solve it!

Discussion: How relevant are your research questions?

ICT

Examples of the "So what?" test

Topic: I am studying falls among seniors. Question: Because I want to find out what causes these falls.

Significance: In order to create IT-based solutions that can reduce falls among seniors.

Topic: I am studying X Question: Because I want to answer Y

Significance: In order to help solve real world problem Z

ICT

Agenda

Why research questions? Relevance Rigor How to formulate one?

ICT

How does a rigorous research question look like?

Discussion

ICT

Non-rigorous research questions

Research questions that are not asked. Research questions that are not answered. Research questions that have Yes/No answers. Research questions whose answers don't produce any

new knowledge. Research questions that don't have "symmetry of

outcome". Research questions that are too broad. Research questions that are too narrow. Research questions that include presumptions.

ICT

Rigorous RQs should

Be focused, Identify the right paradigm,

Positivist, interpretive, critical.

Drive the right research design, Case study, design and creation, survey,….

Drive the management of the research project, Needed time, Needed resources, Inherent dependencies on others.

ICT

RQs should be focused

Research Area

Research Topic

Research Question

Problem space dimension

ICT

How to focus? Qualitative example.

Coherence Use of active verb:

Understanding, Exploring, Interpreting, Constructing, Explaining, etc.

Use of relevant nouns: Experiences, Feelings, Views,

Perspectives, Knowledge, etc. Indication of methodology:

Grounded theory, Action research, Exploratory study, etc.

Structure What will be studied? Who will be studied When will they be studied Where will they be studied What will be studied How it will be studied Why it will be studied

ICT

Examples of focusing

Area Topic Research Question

Aging and IT Falls among seniors and the use of IT

What do we know about the role of IT in preventing falls among seniors?

Empirical evidence of IT and falls among elderly

What types of systems have shown to actually reduce falls among seniors?

Exercise-based interventions for fall prevention and the role of IT

How can an IT system help seniors to comply with preventive exercise interventions?

What is the impact of using system X to promote exercise Y with N elderly living in nursing home A in Trondheim?

ICT

Research question should drive research design

PracticalProblem

ResearchQuestion

ResearchProblem

ResearchAnswer

motivates

definesfinds

helps to solve

ICT

The research design process

What is the impact of using system X to promote exercise Y with N elderly living in nursing home A in Trondheim?

ICT

Primary and secondary RQs

Primary Research Question

Primary Research Question

Secondary RQ1

Secondary RQ1

Secondary RQ1

Secondary RQ1

Secondary RQ1

Secondary RQ1

help answer help answer

Contributory research questions Ancillary research questions

Remember: There should always be a relation between primary and secondary RQ!

ICT

Focusing through designing: Lean research and iterative RQs

Pre-pilot Pilot Main study

Secondary Question 1

Secondary Question 2

Secondary Question 3

Main Question

ICT

Secondary questions and iterations

Sample n = 100e.g. questionnaire

Sub-sample n = 10e.g. interviews

Sub- sub-

sample n = 1

e.g. case study

Progress of the research

Third sample n = 100e.g. questionnaire

Second sample n = 10e.g. interviews

Initial sample n = 1

e.g. case study

ICT

Examples of primary and secondary RQs

Primary RQ Secondary RQs

What is the impact of using system X to promote exercise Y with N seniors living in nursing home A in Trondheim?

• What is system X?• What do seniors living in nursing

home A think system X is?• What do seniors in nursing home A

think about a paper prototype of system X?

• What is the effect of using system X for 4 weeks to promote exercise Y with N seniors…..?

• What is the effect… after one year?

ICT

Check list

Is my RQ something that I am curious about and that others might care about? Does it present an issue on which I can take a stand?

Does my RQ put a new spin on an old issue, or does it try to solve a problem?

Is my RQ too broad, too narrow, or OK? Is my RQ researchable…

…within the time frame of the assignment? …given the resources available at my location?

Is my RQ measurable? What type of information do I need? Can I find actual data to support or contradict a position?

What sources will have the type of information that I need to answer my RQ (journals, books, internet resources, government documents, interviews with people)?

ICT

Mr. Rigorosi and Mrs. Relevanté

Who wins the H2020 race?

ICT

Mr. Rigorosi and Mrs. Relevanté

Who wins the Nobel prize?

ICT

Agenda

Why research questions? Relevance Rigor How to formulate one?

Design and creation Positivist Interpretive

ICT

RQs should use the right wording

"What is the effect of ….?" Requires a design based on Randomized Controlled Trials.

"What is the impact/influence of ….?" Might require case study research.

"What are the attitudes of … towards …?" Might require case study or ethnography.

Etc.

ICT

RQ should use the right wording

Positivist: "What is…..?" "How big….?"

Interpretist: "How do….?" "Why do…..?"

Critical: "Why is it….?" "Why not….?"

Design and creation How good is….? How can we create…?

ICT

Design and creation RQs

First ask yourself: What is the role of the IT system in my research? Is it: The main focus of the research?

Research domain: Computer science. Research questions: Addressing the system as a phenomenon.

Is it: A vehicle for something else? Research domain: Information systems and applied CS. Research questions: Addressing the system in its context of use.

Is it: A tangible outcome of a process, where the process is my focus?

Research domain: Software engineering. Research questions: Addressing the process and tools used during

the development of the system.

ICT

Design and creation RQs: Areas

How might a persuasive system based on reward theories look like?

ICT

SE research questions

ICT

Quantitative RQs

Three Rules for Quantitative Research Questions They Begin with “How”, “What”, or “Why” and can NEVER be

answered by a simple Yes or No Specify the independent and dependent variables IF your questions deal with connections among multiple variables,

you will again - use relate or compare – just as you did in the purpose statement

General Descriptive Examples [using rules 1 and 2]: What factors positively impact employee turnover rates of hourly

workers? Why do Asian-American students outscore other student groups

on AP Calculus Exams?

ICT

RQs: Positivist or interpretive?

Research Question

Research Design

(Interpretive)

Hypothesis

Research Design

(Positivist)

Quantitative research starts with a hypothesis:• RQ: What do I want to know?• RH: What is my tentative

answer?

Avoid "Fishing expeditions"

Quantitative

Qualitative

Research Propositions

ICT

RQs in qualitative research

Generalizing Qs vs. particularizing Qs From "How do students use social media in their study?" To "How do 5. year CS students at NTNU use Facebook to coordinate

their project-based subjects?" Generalization important? Consider quantitative!

Instrumental Qs vs. realist Qs Do we need to reframe questions to collect objectively measurable data,

or do we collect what is there, even if subjective? Objective data mandatory? Consider quantitative!

Variance Qs vs. process Qs Variance Qs focus on differences and variations: "How much?", "To what

extent?", "Is there?" Consider quantitative! Process Qs focus on the meaning of events and the influence of the

physical and social context of use on these events.

ICT

RQs in qualitative research

Descriptive RQs Ask about what actually happened.

Interpretive RQs Ask about the meaning of what happened to those involved.

Theoretical RQ Ask about why those things happened.

Generalization RQ Normally not asked in QR (not big enough samples).

Evaluation RQ Normally not asked in QR (imposes values on people).

ICT

PICO (Quant.) and SPIDER (Qual.)

ICT

RQs in qualitative research

Will be revised during the research (very different than quantitative research!)

Beware of "smuggling unexamined assumptions"! Can be a statement rather than a question.

ICT

Qualitative RQs in context

Research Questions

PurposesConceptual context

Methods Validity

ICT

FINER (Clinical)

F:Feasible Adequate number of subjects Adequate technical expertise Affordable in time and money Manageable in scope

I: Interesting Getting the answer intrigues investigator, peers and community

N: Novel Confirms, refutes or extends previous findings

E: Ethical Amenable to a study that institutional review board will approve

R: Relevant To scientific knowledge To clinical and health policy To future research

ICT

Choosing the right questions

ICT

The wrong research question leads to wasted time and effort!

ICT

Credits

This presentation is based on a number of publications from others. The figure introduced in 5 is by (Booth, Colomb, and Williams 2003), so is the "So what?" test in slide 9. The terms used in "How to focus?" slide are by (Mantzoukas 2008). The figure in slide 18 and a lot of ideas in the slides are by (Oates 2005). The discussion of primary and secondary questions is adopted from (Andrews 2003). The research outputs/activities matrix is by (March and Smith 1995). The table showing software research questions is by (Shaw 2002). A lot of the material on qualitative data is adopted from (Maxwell 1996) and (Agee 2009).... Slides on PICO and SPIDER are from (Cooke, Smith, and Booth 2012). Some of the discussion on choosing a good research question and the graph are from (Alon 2009).

ICT

References

Agee, Jane. 2009. “Developing Qualitative Research Questions: A Reflective Process.” International Journal of Qualitative Studies in Education 22 (4): 431–47. doi:10.1080/09518390902736512.

Alon, Uri. 2009. “How to Choose a Good Scientific Problem.” Molecular Cell 35 (6): 726–28. doi:10.1016/j.molcel.2009.09.013.

Andrews, Richard. 2003. Research Questions. London: Continum. Booth, Wayne C., Gregory G. Colomb, and Joseph M. Williams. 2003. The Craft of Research. 2nd ed. Chicago,

London: The University of Chicago Press. http://www.press.uchicago.edu/ucp/books/book/chicago/C/bo5821939.html.

Cooke, Alison, Debbie Smith, and Andrew Booth. 2012. “Beyond PICO: The SPIDER Tool for Qualitative Evidence Synthesis.” Qualitative Health Research 22 (10): 1435–43. doi:10.1177/1049732312452938.

Mantzoukas, Stefanos. 2008. “Facilitating Research Students in Formulating Qualitative Research Questions.” Nurse Education Today 28 (3): 371–77. doi:10.1016/j.nedt.2007.06.012.

March, Salvatore T., and Gerald F. Smith. 1995. “Design and Natural Science Research on Information Technology.” Decision Support Systems 15 (4): 251–66. doi:10.1016/0167-9236(94)00041-2.

Maxwell, Joseph A. 1996. Qualitative Research Design- An Interactive Approach. Thousand Oaks: SAGE Publications.

Oates, Briony J. 2005. Researching Information Systems and Computing. 1st ed. SAGE Publications. Shaw, Mary. 2002. “What Makes Good Research in Software Engineering?” International Journal on Software Tools

for Technology Transfer 4 (1). Springer-Verlag: 1–7. doi:10.1007/s10009-002-0083-4.