quality research

The Quality of Qualitative The Quality of Qualitative ResearchResearch

Uwe FlickUwe FlickAlice-Salomon-University Alice-Salomon-University

of Applied Sciences Berlinof Applied Sciences Berlin

[email protected]@asfh-berlin.de

18./19. 4. 200818./19. 4. 2008 Uwe Flick - BernUwe Flick - Bern 22

What is qualitative research and are we referring to?

Theoretical and methodological schools

Local approaches and schools

Discipline-specific developments

Different fields of application – Qualitative health research– Qualitative evaluation

Same answers to the quality question across the various areas and contexts?


QualityQuality of qualitative of qualitative researchresearch

Still an unanswered questionStill an unanswered question

Answers expected from outsideAnswers expected from outside

Answers needed for further establishing Answers needed for further establishing qualitative researchqualitative research

Answers needed for reassuring qualitative Answers needed for reassuring qualitative research in times of evidence and mixed research in times of evidence and mixed methodsmethods

One size fits all?One size fits all?


Four ways to answer the question of quality in qualitative research

To apply the traditional criteria

To reformulate traditional criteria like validity and reliability

To design new appropriate criteria

To use strategies of promoting quality


Answer I:Answer I:

TraditionalTraditional

CriteriaCriteria


Reliability, validity and objectivity

are strongly based on standardisation of methods and situationsare based on concepts like repeatabilityare based on a detachment of the person of the researcherare incompatible with the degrees of freedom, of flexibility, and of non-standardisation of qualitative research


Answer II:Answer II:

Reformulation of Reformulation of TraditionalTraditional

CriteriaCriteria


Validation in Qualitative Validation in Qualitative ResearchResearch

Communicative validation Communicative validation

Pragmatic validationPragmatic validation

To validate is to question To validate is to question

To validate is to checkTo validate is to check

To validate is to theorise (Kvale 2007)To validate is to theorise (Kvale 2007)


Answer III:Answer III:

NewNew

CriteriaCriteria


Credibility of Qualitative Credibility of Qualitative ResearchResearch

Prolonged engagement and persistent Prolonged engagement and persistent observation in the field and triangulation observation in the field and triangulation

Peer debriefingPeer debriefing

The analysis of negative cases The analysis of negative cases

Appropriateness of the terms of reference Appropriateness of the terms of reference of interpretations and their assessment;of interpretations and their assessment;

Member checks (Lincoln and Guba 1985)Member checks (Lincoln and Guba 1985)


Criteria for Grounded TheoryCriteria for Grounded Theory

CredibilityCredibility

OriginalityOriginality

Resonance

Usefulness (Charmaz 2006)


Credibility as CriterionCredibility as Criterion??

Benchmark problemBenchmark problem

How much credibility needed?How much credibility needed?

Member checks: all members‘ or some Member checks: all members‘ or some members‘consent?members‘consent?

Criterion or bona fides?Criterion or bona fides?

Criteria or strategies?Criteria or strategies?


Quality in Qualitative EvaluationQuality in Qualitative Evaluation

Research should be contributory in advancing wider knowledge defensible in design by providing a research strategy which can address the evaluation questions posedrigorous in conduct through the systematic and transparent collection, analysis and interpretation of qualitative datacredible in claim through offering well-founded and plausible arguments about the significance of the data generated (Spencer et al. 2003).


Answer IV:Answer IV:

StrategiesStrategies


TriangulationTriangulation Different methods also for collecting dataEqual weighting of methods or approachesSystematic use of different methodsIntegration/reflection of the theoretical backgrounds of the different methodsAddressing different levels, for example: : – subjective meaning and social structure – process and state, , – knowledge and activity– knowledge and discourseiscourse

Purposeful choice and use of methods


Quality Management in Qualitative Research

a definition of the goals to be reached and the standards of the project to be kept, which should be as clear as possible; all researchers and co-workers have to be integrated in this definition;a definition, how to reach these goals and standards and more generally: the quality to be obtained; a consensus about the way how to apply certain methods maybe through joint interview trainings and their analysis, are preconditions for quality in the research process;a clear definition of the responsibilities for obtaining quality in the research process andthe transparency of the judgement and the assessment and quality in the process.


Indication in Indication in Psychotherapy and MedicinePsychotherapy and Medicine

Which diseaseWhich disease

which symptoms which symptoms indicateindicate

which diagnosis which diagnosis

which populationwhich population

which which

treatment treatment

oror

therapy?therapy?


Indication in Indication in Qualitative ResearchQualitative Research

Which issueWhich issuewhich populationwhich populationwhich research questionwhich research question

indicateindicate

which knowledge which knowledge of issue and of issue and populationpopulation

which which

method(s)method(s)

oror

which which combination of combination of methods?methods?


Indication in Indication in Qualitative ResearchQualitative Research

When is/are which method(s) When is/are which method(s) appropriate and indicated?appropriate and indicated?

Are there criteria for a rational decision for Are there criteria for a rational decision for or against specific methods?or against specific methods?


Sound qualitative research ..

... is based on previous research and refers to state-of-the-art theoretical, methodological and topic-related literature. In this sense it stands „on the shoulder of giants“. – What about explanatory research, new fields and

grounded theory research in the strict sense of the word?

... is clearly linked to a social scientific theory like Symbolic Interactionism, Ethnomethodology, Cultural Representation Theory, Hermeneutics, Phenomenology, Discourse Theory or the like. – These are not theories in the strict sense of the word

(as far as concerning our context) but research perspectives, which are taken on an issue (or field …) under study.



... uses only data appropriate within the framework of the particular theory and applies corresponding methods for

analysis and interpretation. – This statement is too much theory-driven: Shouldn’t the

research perspective be appropriate to the issue under study and then only data be used that are appropriate to the issue under study?



Depending on the particular theory and methods used qualitative research presents an “emic” as well as an “etic” perspective. Including the subjective or “native” point of view does not discharge the researcher from the task of developing and defending his/her own interpretation which may or may not correspond to that of the research

subjects. – Nevertheless, it may be appropriate to seek member’s

consensus or disagreement for results or evaluations coming from qualitative research, depending on purpose and issue of the research



... is neither partisan nor indifferent toward social problems, but it aims at differentiating scientific analysis from personal and political stance on a given topic. Qualitative research reveals the complexity of the social world instead of lending itself to simplifications. – In the context of practical/political use (and relevance)

of qualitative research it may be necessary to elaborate qualitative results in a way that allows a political argumentation or decision and to condense them in a way that they are comprehensible and relevant for non-researchers if we want to become/remain relevant with our research.



.... is contextualised and the relevant information considering the concrete context is provided (without jeopardising the anonymity of informants, organisations etc.). Generalisations of results beyond this context must be discussed.

– Internal generalizability refers to the generalizability of a conclusion within the setting or group studied, while external generalizability refers to its generalizability beyond that setting or group (Maxwell 2005, p. 115).

Types, Goals and Interest groups ...


Who may be interested ..

Three answers given:– Anyone, who ....– A circular answer: .... qualitative research methods are qualitative research methods are

relevant for those, whose research question can be relevant for those, whose research question can be answered with the relevant qualitative approach and for answered with the relevant qualitative approach and for which appropriate data are available.which appropriate data are available.

– It depends ... As there are not enough common It depends ... As there are not enough common characteristics in qualitative researchcharacteristics in qualitative research


Who may be interested ..

Three possible answersThree possible answers– Those who are interested in fields Those who are interested in fields or topics, which are not well or topics, which are not well accessible for other forms of data accessible for other forms of data

– Those who are interested in having Those who are interested in having a comprehensive range of approaches a comprehensive range of approaches and tools for social researchand tools for social research

– Those, who are interested in Those, who are interested in exploring new fields, finding new exploring new fields, finding new insightsinsights


Types of qualitative methods

As the examples of interviews and textual As the examples of interviews and textual data show, we should distinguish between:data show, we should distinguish between:– Data collectionData collection

In this step the interview is a qualitative method In this step the interview is a qualitative method depending on how it is applied, but not on how it is depending on how it is applied, but not on how it is analysed in the endanalysed in the end

– Data analysisData analysisOther forms of data are mostly transferred into texts Other forms of data are mostly transferred into texts when it comes to analysis (oral data are transcribed, when it comes to analysis (oral data are transcribed, sooner or later, images are described and analysed by sooner or later, images are described and analysed by using words and textsusing words and texts

– Writing about data and findingsWriting about data and findingsAny presentations has to be reductionistic in some Any presentations has to be reductionistic in some way, if the reader shall grasp the message of the way, if the reader shall grasp the message of the report. It depends rather on how adequate the report. It depends rather on how adequate the selected form of reductionselected form of reductionism is to what is under ism is to what is under study - in collecting, analysing and presenting the study - in collecting, analysing and presenting the data.data.


Types of qualitative methods

Inductive and deductive approachesInductive and deductive approaches– Good qualitative research is always a Good qualitative research is always a combination of induction (in the combination of induction (in the beginning) and deduction (along the way)beginning) and deduction (along the way)

– Research always starts from some Research always starts from some assumptions about the field and the assumptions about the field and the issue, it depends rather on how far these issue, it depends rather on how far these assumptions are made explicit and subject assumptions are made explicit and subject of revision against the dataof revision against the data


Goals of qualitative methods

The major goal should be to use methods that are The major goal should be to use methods that are appropriate to issues under study, but also to the appropriate to issues under study, but also to the fields and participants where these issues are fields and participants where these issues are studiedstudied– Indication of methods starts from selecting the Indication of methods starts from selecting the

appropriate ones from the existing range of methods appropriate ones from the existing range of methods and approaches or from developing new ones if and approaches or from developing new ones if necessarynecessary

Do methods have goals?Do methods have goals?– What are the goals of researchers which are normally What are the goals of researchers which are normally

linked with using specific qualitative methods?linked with using specific qualitative methods?– Which methods allow (support, facilitate etc.) to pursue Which methods allow (support, facilitate etc.) to pursue

which goals more adequately?which goals more adequately?

Teaching Qualitative Teaching Qualitative MethodologyMethodology


Goals of TeachingGoals of Teaching

MA/PhD level:MA/PhD level:– Critical selection and reflection of a variety of Critical selection and reflection of a variety of

research methods for one‘s own research research methods for one‘s own research projectproject

– Independent use of (a variety of) methods for Independent use of (a variety of) methods for one‘s own researchone‘s own research

In general:In general:– Combination of an overview and of practical Combination of an overview and of practical

application of selected examplesapplication of selected examples


Goals of TeachingGoals of Teaching

Bachelor level in applied areas:Bachelor level in applied areas:– Ability to understand research examples and Ability to understand research examples and

to distinguish good from bad researchto distinguish good from bad research– Ability to apply a certain number of methods Ability to apply a certain number of methods

for answering a limited research questions in for answering a limited research questions in a thesisa thesis

MA/PhD level:MA/PhD level:– Critical selection and reflection of a variety of Critical selection and reflection of a variety of

research methods for one‘s own research research methods for one‘s own research projectproject

quality research

Technology