IT3010 / TDT39Research Methodology

Week 6: Data analysis

Name, title of the presentation

Figure 3.1 in: B. J. Oates, Researching Information Systems and Computing. London: Sage Publications, 2006.

The research process

What is data analysis?

• Looking for (hidden) patterns in collected/generated data.

• Drawing conclusions based on patterns– Proposing new theories– Discovering evidence in support of existing theories or evidence for

fallacy of such.– Identifying the need for new research opportunities.– Etc.

Types of analysis

• Quantitative:– For number data.– Mostly statistical.

• Qualitative:– For text and other media.– Mostly based on researcher interpretation.

• Nota Bene!– Statistical does not mean objective!– Researcher interpretation does not mean subjective!

Quantitative Data Analysis

• Types of data:– Numeric summaries from questionnaires.– Numeric data/summaries from usage log files.

• Levels of analysis complexity:– Organize data into tables and charts.– Apply descriptive statistical techniques.– Apply complex statistical techniques.

• Cons and pros:– Reviewers like quantitative data.– Reviewers might doubt your understanding of methods if complex.

Types of quantitative data

• Nominal (categorical) data:– Categories, e.g. gender, occupation.

– Analysis type: Show frequency.

• Ordinal (ranked) data:– Numbers allocated to a quantitative scale, e.g. examination results.

– Can be used to code answers to questionnaires (e.g. 1="Strongly disagree").

– Ranking does not provide information about intervals (i.e. difference among ranks).

• Interval data:– Includes information about the interval between numbers, e.g. year number.

• Ratio data: – Interval data with a true zero point, e.g. people's age.

Visualizing quantitative data

Using statistics: examples

• Describing the central tendency– Mean (average), median (the "middle" value), mode (the most

frequent value).

• Describing the distribution– Range: The difference between highest and lowest values.– Fractiles: Dividing data into "pieces" e.g. ranges. (percentile = divide

into 100 pieces).– Standard deviation: The average "distance" of every data value from

the mean.

• Finding relationships in the data– Correlation coefficients: Shows how strongly two variables are co-

related (does NOT say anything about causal links, which needs regression analysis).

Interpreting the results

• Data analysis does not mean we are finished.• Need to look into the results and propose hypotheses,

theories, explanations.• Need to compare to other researchers' results

Qualitative Data Analysis

• Qualitative data includes all non-numeric data.• Generated commonly in case studies, action research

and ethnography.

• Not to be confused with "quantitative analysis of qualitative data".– E.g. number of times "love" is mentioned in an interview transcript.

• Qualitative data analysis is about abstracting, from qualitative data, the verbal, visual or aural themes and patterns.

Qualitative Data Analysis

• Criticized for:– Lack of rigor in the analysis process. "Conclusions appear by

magic".– The role of the researcher, in particular her bias and level of skills.

• Useful for:– Creating new theories or alternative explanations.– Specially in computer science and information systems where every

new system is unique in some way.

Analysing textual data

• Preparing the text:– Often from interview recordings or from unstructured field notes.– Transcribing, indexing, re-formatting…

• Analysing the text:– Initial categorization: Into irrelevant, descriptive (facts), and relevant

(related to research questions).– Refining categories: grouping, dividing, joining….– Looking for themes: Themes and patterns across the data (good to

have an analysis software!).– Making conclusions: Linking to existing or new theories.

• Where do categories come from?– Deductive: Based on existing theories.– Inductive: Based on the collected/generated data.

Grounded theory

• A method concerned with generating theories.• Theories that are grounded in data from the field.• Theories that are useful for practitioners and not only

for academics.• More than just inductive data analysis.• Specific methods for:

– Selecting people.– Data analysis.– Kind of theory generated

Grounded theory method

• Selection of people: – Start with one and add more based on analysis results.– Stop the cycle when no new insights.

• Data analysis: Three phases– Open coding: Based on words existing in the generated data.– Axial coding: Abstracting and finding relationships among data.

Finding important (axial) data.– Selective coding: Focus on important data that is vital for theory

generation.– Constant comparative: Re-coding based on new knowledge.

• Theories:– Should have practical value for people in the studied field.– Should make sense to those people.

Computer-based analysis

• Video of NVivo.

Next week

• Book chapter 18. Paradigm and evaluation of research.

• Feedback assignment 2.• Group 6 presents paper 6.