Interrogating the Quantitative-Qualitative

Divide

Martyn Hammersley

The Open University

Revised version of a paper presented at the NCRM Research Methods Festival, St Catherine’s

College, Oxford, July 2012

Methodology, who needs it?

• All research, in fact all forms of action, involve method: knowledge of strategies to achieve some goal, their advantages and disadvantages, and ideas about the nature of the activity and of the world.

• Can there be too much concern with method? The German sociologist Max Weber complained of ‘methodological pestilence’ (Oakes 1975:13), and declared that ‘Method alone has never yet created anything’ (Bruun and Whimster 2012:xiv)

What is methodology?

• Methodology-as-technique: rules about what to do and when to do it. There can be important guidance, but research cannot be carried out by simply following rules, it requires thought-in-context.

• Methodology-as-philosophy: a substantial portion of the methodological literature today is preoccupied with philosophical ideas. These are important, but they do not solve research problems.

The Status of the Quantitative-Qualitative Divide

• It is widely used as a way of mapping methodological variation in social science.

• It has increasingly been questioned, albeit from a range of different positions: see, for example, Ercikan and Roth 2006, Bergman 2008, Vogt 2008, and Wood and Welch 2010.

How To Define the Quantitative-Qualitative Divide?

As:

• The difference between using qualitative or quantitative data, along with the methods typically employed for these?

Or as:

• The difference between the methodological ideas to be found amongst qualitative researchers versus those that are found amongst quantitative researchers?

Conflicting Paradigms• There are conflicting methodological

philosophies that are currently influential, such as positivism, interpretivism, and constructionism (see Hammersley 2007).

• However, while these reflect fundamental disagreements, these are to be found among qualitative researchers as much as between quantitative and qualitative researchers.

• Doing research well is not a matter of choosing a paradigm and then applying it in some field.

Mixed Methods as the Solution?

The rise of the mixed methods movement: see Tashakkori and Teddlie 2010. But ‘mixing’ methods is not new. Nor is it unproblematic:

• It effectively leaves the distinction between, and ideas about the character of, quantitative and qualitative methods intact: they are simply to be combined.

• There is no agreement on the methodological rationale that should underpin mixed methods research.

A Reconceptualisation of the DivideIdentifying the options available in relation to

each aspect of the research process, namely:

• Formulating research questions

• Planning the research

• Selecting cases for investigation

• Collecting data

• Analysing data

• Writing research reports

The multiple ways in which these options can be combined must also be recognised.

Within-case and Cross-case Analysis

• There can be investigation of associations and sequences within a particular case, and of participant ideas about what causes what. This evidence serves as a resource that, along with theoretical interpretation, allows explanatory ideas to be developed and perhaps also tested.

• Cross-case analysis: examination of similar and different cases, along the lines of J. S. Mill’s methods, in which theoretical interpretations are developed and tested against the evidence.

Within Case/Cross Case is not Isomorphic with

Qualitative/Quantitative

• Both these analytic strategies are used within both qualitative and quantitative research.

• However, their character varies between these two contexts: in terms of whether there is counting and measuring, whether statistical analysis is used, etc.

The ‘Qualitative Survey’ An example: explaining social-class variation

in applications to ‘high-status’ universities (Reay et al. 2005). These researchers relied on questionnaire data from 502 students, and interviews with 120 students, from 6 schools and colleges in the London area. They used qualitative analysis, identifying patterns of association between social class background and different styles of choosing, these tending to result in applications to different types of university.

Combination of, and Variation Within, these Strategies

• These two strategies can be combined, and very often this will be necessary in order to develop and test explanatory hypotheses effectively .

• There are also some alternative options within the two strategies. In the case of cross-case analysis, we can draw a distinction between single-factor and configurational approaches.

Single Factor versus Configurational Approaches

• Single factor: For example, ‘Do differences in social class background determine level of educational achievement over and above differences in ability?’

• Configurational: ‘What combinations of factors seem to determine differences in educational achievement?’

Differences in Assumption about Causation

• A deterministic approach to causation requires that the set of factors identified must always produce the specified type of outcome when certain specifiable conditions are met; and that this is the only set of factors that will do this.

• A contributional approach is concerned with identifying factors that significantly increase the likelihood or size of some outcome, without their being either necessary or sufficient conditions for it.

Two Currently Influential Approaches to Cross-Case Analysis• Correlational analysis of the kind used in much

quantitative (and some qualitative) work involves a single-factor approach, perhaps with other factors being controlled to estimate the candidate factor’s contribution to the likelihood or level of the outcome.

• Qualitative Comparative Analysis adopts a configurational strategy and a deterministic approach to causation, though this is moderated via notions of quasi-sufficiency and quasi-necessity.

Building on existing work and contributing to the field

• The fallacy of the one-shot study (Becker 1970)

• It is rarely if ever possible for a single study to answer a research question adequately. Also, all questions are embedded within wider assumptions that may need assessing. And any interesting practical or political implications drawn from research findings always depend upon assumptions that could need checking.

• Researchers’ work never starts from scratch, and it is never finished!

Conclusions• The quantitative-qualitative divide signals

important differences both in data/methods and in methodological ideas, but it obscures as much as it reveals

• Mixed methods research is no simple solution

• We need to think about specific options relating to each aspect of the research process

• Furthermore, selecting amongst these does often force us to face deep philosophical issues, for example about the nature of causation, that have been central to debates about methodological paradigms

Select BibliographyBecker. H. S. (1970) ‘The Life History and the Scientific Mosaic’ in Sociological Work: Method and

Substance, Chicago, Aldine.

Bergman, M. M. (2008) „The straw men of the qualitative-quantitative divide and their influence on mixed methods research‟, in M. M. Bergman (eds.), Advances in Mixed Methods Research: Theories and Applications. London: Sage.

Bryman, A. (1988) Quantity and Quality in Social Research, London, Allen and Unwin.

Cooper, B., Glaesser, J., Gomm, R., and Hammersley, M. (2012) Challenging the Qualitative-Quantitative Divide: Explorations in case-focused causal analysis, London, Continuum.

Ercikan, K. and Roth, W-M. (2006) ‘What good is polarizing research into qualitative and quantitative?’, Educational Researcher, 35, 5, pp14-23.

Hammersley, M. (1996) 'The relationship between quantitative and qualitative research', in J Richardson (ed.) Handbook of Qualitative Research Methods for Psychology and the Social Sciences , Leicester, British Psychological Society Books.

Hammersley, M. (2007) Methodological Paradigms in Educational Research. London: TLRP. Online at http://www.tlrp.org/capacity/rm/wt/hammersley (accessed 19 Jul 2012)

Reay, D., David, M., and Ball, S. (2005) Degrees of Choice: Social class, race and gender in higher education, Stoke on Trent, Trentham Books.

Tashakkori, A. and Teddlie, C. (eds.) (2010) Handbook of Mixed Methods in Social and Behavioral Research, Second edition, Thousand Oaks CA, Sage.

Vogt, W. (2008) ‘Quantitative versus qualitative is a distraction: variations on a theme by Brewer and Hunter (2006)’, Methodological Innovations Online, 3, 1, pp18-24.

Wood, M. and Welch, C. (2010) ‘Are “qualitative” and “quantitative” useful terms for describing research?’, Methodological Innovations Online 5, 1, pp56-71.