Advanced Quantitative Methods (ARM3)

An introduction and orientation to research using quantitative

data

Steve Strand

Aims of this session• Locate the research tradition / historical context for

quantitative research

• Be familiar with some key terms: ontology, epistemology, methodology, paradigm etc.

• Recognise paradigm assumptions, but take a pragmatic approach where the research question determines the choice of methods

• Recognised quantitative and qualitative are terms that refer to data, not to methods per se

• Appreciate some of the strengths of using quantitative data in Educational Research

Ontology

• from the Greek on (being) & logos (theory)

So literally theory of being.

• what is the nature of being? What is the nature of the reality we seek to explain? Does it exist out there waiting to be discovered, or only in the human mind?

• Nominalist – realist debate (if a tree falls in the forest……)

Epistemology

• From the Greek episteme (knowledge) and logos (theory) - so literally theory of knowledge

• What constitutes knowledge, how can it be acquired and communicated

• Realist view suggests observer role, determine laws, natural science methods (positivism)- Nominalist view suggests any number of different social worlds

Methodology

• From the Greek methodos (procedure for the attainment of a goal, in this case the acquisition of knowledge) and logos (theory) – so literally theory of the way in which knowledge is acquired

• How should knowledge be produced

• Ontology & epistemological views lead to distinct approaches to methodology – e.g., scientific method, collecting quantitative data, notions of control etc.

Data analysis andconclusions

Linked into paradigms

Ontological assumptions

Epistemological assumptions

Methodological considerations

Methods

Nature of Knowledge

• How do we come to know something?

Common sense doesn’t always hold (e.g. chance occurences & cause-effect relationships). Research is the systematic attempt to find answers.

• Ways of knowing

Intuition/introspection – but wider agreement?

Tenacity – frequent repetition->strong belief (confirmatory bias)

Authority – Bible, government, teacher

Science – objectivity, empirical data, control

Positivism There can be objective knowledge, existing

independently of the observer, that abides by universal laws and can be measured, quantified and predicted.

• Realism (real world independent of observer)

• Determinism (cause & effect relationships)

• Empiricism - strive for objective data observation, classification, quantification

• Through experimentation discern natural laws, and then make generalisations

• Replicability – public scrutiny

The ‘How’ of positivism• Experiment is considered the Gold Standard in many

disciplines (natural science, medicine, psychology)

• Classic features are: Clear testable hypothesis

Operationalise (measure) all concepts

Pre-test and post-test design

Control all extraneous variables (e.g. by random assignment of persons to treatments)

Give one group the treatment/Intervention, the other control group nothing or placebo

Repeat measure & compare groups

Determine causality (time order, controlled confounding variables)

Sufficient detail for replication by other researchers

Example: The US High/Scope Study

Measures taken during school

Achievement Tests, plus

Interviews with children

Social Profiles: questionnaires

Interviews with parents

Behaviour Ratings: teachers

Assignment to Special Education

IQ.

Jobs

Training

Parenthood

Welfare

Crime

Pre

-sch

ool g

roup

Con

trol

(no

sch

ool)

grou

p

3 5 AGE 18 27

Source: Schweinhart (2000)

Experimental research (cont.)

• Advantages:

Strong control over external factors

Best way of determining causality

• Disadvantages:

Often artificial situation

Generalisability to ‘real life’ setting?

Sometimes difficult to put into practice in school settings, ethical issues

People are not static entities (e.g., Hawthorn effects)

Quasi-experimental

• Quasi-experimental designs match individuals receiving different interventions rather than randomly assign

• e.g. effect of smoking on life expectancy, match smokers and non-smokers on key variables (gender, social class, occupation, age, lifestyle etc)

• Less control, may always been some relevant factor that has not been matched, so weaker inferences.

Survey research

• No experimental intervention, impossible or unethical, but detailed observation allows the use of statistical control

• Example is the Effective Pre-school and Primary Education (EPPE) Project

• Advantages

Study phenomenon in natural settings

Flexible approach, statistical control

• Disadvantages

Reduced control means we are never sure of causality, correlations may be spurious

25 nursery classes

590 children

34 playgroups

610 children

31 private day nurseries

520 children

20 nursery schools

520 children

7 integrated centres

190 children

24 local authority day care nurseries

430 children

home

310 children

Example: DfES EPPE Project 2005

Reception 6yrs 7yrs

Pre-school Provision (3+ yrs)

Key Stage 1

600 Schools

10yrs 11yrs

Key Stage 2

800 Schools

Interpretivism

• Social world is not the same as the physical world (no definable / quantifiable social facts or universal laws, socially defined)

• Rejection of natural science as a model (Researcher stands not in a subject-object relation to the social world but subject-subject relation, cannot be objective or unbiased)

• Concerned for a deep understanding of the particular, focus on subjective & experiential, stress multiple meanings

• Many different ‘schools’ of thought (Phenomenologists, ethno-methodologists, symbolic interactionists)

The ‘How’ of interpretivism• Wide range of methods and frameworks e.g.

ethnography

observation

Interview

detailed case study

documentary analysis etc.

• Preference for: words and images rather than numbers

meanings rather than behaviour

inductive, hypothesis-generating research

• Strengths

Portraying a slice of life

Reflexivity

Detailed description of complex phenomena

Triangulation

• Weaknesses

Subjectivity?

Reliability? e.g. adequately reported?

Validity? e.g. is it anecdotal rather than analytical?

Risk of generalising from particular data

Interpretive approaches

Two contrasting paradigms

Positivism InterpretivismRealism nominalism

Universal knowledge - generalisability

context specific, unique

Objectivity - research is unbiased Subjectivity – human element

Measurement & precision - quantities, amount, intensity, frequency

the qualities of entities, processes and meanings

‘Hard’ data - numeric ‘Soft’ data – language, texts

Control, cause & effect Understanding actions / meanings

Value free Value-laden

Discussion Task

• The ‘paradigm wars’ dominated social research most of last century (and still do in some ways)

• How do these issues relate to your planned research? Does you research fit neatly into one of these traditions?

• Is it useful to take an epistimological stand first and then construct your research questions, or should it be the other way round?

Pragmatism (1)• Both extremes are problematic, debate has

isolated ER and made it seem irrelevant “Education researchers write mainly for one another in their

countless academic journals, which are not to be found in a school staffroom (Hargreaves, 1996)”

• Social world is a social constructed, but there is a reality independent of our thinking (magic vs. science)

• We are biased by our cultural and socio-political contexts, but can strive for maximum objectivity

• Research is not a choice of paradigms, methods are determined by the research question

Pragmatism (2)

• Pragmatists argue not “is it true” but “does it work”?

• Quantitative data is good for: When we need quantitative answers (are ethnic minorities over-

represented among pupils excluded from school?)

Numerical change (are standards improving over time?)

Prediction (Do aspirations predict attainment?)

Hypothesis testing (can poverty explain ethnic under-achievement?)

• Qualitative data is good for: Looking at something in depth

Developing hypotheses and theories

Complex and broad issues

Meaning rather than causation

A research task

A parent at a primary school is concerned about the reading of his younger daughter, Ellie. She is age 7. He believes that his elder daughter, Gemima, who is now age 11, was a much more proficient reader at the same age. The parent suspects that something is going wrong in the school.

The governing body have agreed to investigate. They have asked you to undertake research to establish whether there is an issue with the teaching of reading in the school.

How would you go about doing the research?

Working on the task

• In small groups of about 4 people consider the question for 10 minutes;

• decide on an approach to the research and justify your choice of design and methods;

• Be prepared to offer brief feedback to the whole group.

Lastly, don’t confuse methods with data

Quantitative data

Method Qualitative data

Highly structured Interviews unstructured

Closed questions

Questionnaire Open-ended questions

Coding freq Observation Participant observation

Content analysis Documents Impressions and inferences

IQ Score Assessment Formative judgement

References

• For overview of research traditions and paradigms: Cohen et al (2007) Research Methods in Education.

Chapter 1 and Chapter 3. Muijs, D (2004). Quantitative research in education.

Chapter 1.

• For practical advice on methodology and selecting your methods: Bell, J. (2005) Doing your research project. Clough & Nutbrown (2007) A student’s guide to

methodology. Chapters 1 & 2.