creswell, determining validity in qualitative inquiry

124

THEORY INTO PRACTICE / Summer 2000Getting Good Qualitative Data

WRITING ABOUT VALIDITY IN QUALITATIVE inquiry is challenging on many levels. Mul-

tiple perspectives about it flood the pages of books(e.g., Lincoln & Guba, 1985; Maxwell, 1996; Mer-riam, 1998; Schwandt, 1997) and articles and chap-ters (e.g., Altheide & Johnson, 1994; Lather, 1993;Maxwell, 1992). In these texts, readers are treated toa confusing array of terms for validity, including au-thenticity, goodness, verisimilitude, adequacy, trust-worthiness, plausibility, validity, validation, andcredibility. Various authors have constructed diversetypologies of validity (e.g., Maxwell’s five types,1992; Lather’s four frames, 1993; and Schwandt’sfour positions, 1997). It is little wonder that Don-moyer (1996), who wrote an editorial on validityin the Educational Researcher, commented on thediverse perspectives of validity by contrasting Milesand Huberman’s (1994) “traditional conception ofvalidity” with Lather’s (1993) “ironic validity” (p.21). Novice researchers, in particular, can becomeincreasingly perplexed in attempting to understandthe notion of validity in qualitative inquiry.

There is a general consensus, however, thatqualitative inquirers need to demonstrate that theirstudies are credible. To this end, several authors iden-tify common procedures for establishing validity in

qualitative projects (e.g., Lincoln & Guba, 1985;Maxwell, 1996; Merriam, 1998). Qualitative research-ers routinely employ member checking, triangulation,thick description, peer reviews, and external audits.Researchers engage in one or more of these proce-dures and report results in their investigations.

As helpful as they are, these discussions aboutvalidity procedures provide little guidance as towhy one procedure might be selected for use byresearchers over other procedures. In this article,we suggest that the choice of validity proceduresis governed by two perspectives: the lens research-ers choose to validate their studies and researchers’paradigm assumptions. We advance a two-dimen-sional framework that can help researchers identi-fy appropriate validity procedures for their studies.

The use of this framework can provide a ra-tionale for choice of a procedure beyond what thesetting and participants will bear and what col-leagues and faculty advisers recommend. Theframework helps researchers select proceduresbased on who assesses the credibility of a study andtheir own philosophical positions toward qualitativeinquiry. We begin by discussing the two perspectivesof the framework and then identify nine validity pro-cedures that fit the framework. We end by describinghow the lens and paradigm assumptions help guideour choice of validity procedures.

In this discussion we define validity as howaccurately the account represents participants’ re-alities of the social phenomena and is credible to

THEORY INTO PRACTICE, Volume 39, Number 3, Summer 2000Copyright © 2000 College of Education, The Ohio State University0040-5841/2000$1.50

John W. Creswell is professor of educational psychologyat the University of Nebraska-Lincoln; Dana L. Milleris assistant professor of research methods at DoaneCollege, Lincoln, Nebraska.

John W. CreswellDana L. Miller

Determining Validityin Qualitative Inquiry

Downloaded By: [Canadian Research Knowledge Network] At: 11:25 15 February 2009

125

Determining ValidityCreswell and Miller

them (Schwandt, 1997). Procedures for validityinclude those strategies used by researchers to es-tablish the credibility of their study. Throughoutthis discussion, we make the assumption that va-lidity refers not to the data but to the inferencesdrawn from them (Hammersley & Atkinson, 1983).

The Lens Used by the ResearcherWhen we refer to the lens, we mean that the

inquirer uses a viewpoint for establishing validityin a study. Qualitative inquirers bring to their stud-ies a different lens toward validity than that broughtto traditional, quantitative studies.

In quantitative research, investigators aremost concerned about the specific inferences madefrom test scores on psychometric instruments (i.e.,the construct, criterion, and content validity of inter-pretations of scores) (AERA, APA, & NCME, 1982)and the internal and external validity of experimentaland quasi-experimental designs (Campbell & Stan-ley, 1966). In contrast, qualitative researchers use alens not based on scores, instruments, or researchdesigns but a lens established using the views of peo-ple who conduct, participate in, or read and review astudy.

For example, one lens to determine the cred-ibility of a study is the particular lens of the re-searcher. Researchers determine how long to remainin the field, whether the data are saturated to es-tablish good themes or categories, and how theanalysis of the data evolves into a persuasive nar-rative. Patton (1980) describes this process as onewhere qualitative analysts return to their data “overand over again to see if the constructs, categories,explanations, and interpretations make sense” (p.339). Altheide and Johnson (1994) refer to it as“validity-as-reflexive-accounting” (p. 489) where re-searchers, the topic, and the sense-making processinteract.

Qualitative inquirers may use a second lensto establish the validity of their account: the par-ticipants in the study. The qualitative paradigmassumes that reality is socially constructed and itis what participants perceive it to be. This lenssuggests the importance of checking how accuratelyparticipants’ realities have been represented in thefinal account. Those who employ this lens seek toactively involve participants in assessing whether

the interpretations accurately represent them. Athird lens may be the credibility of an account byindividuals external to the study. Reviewers notaffiliated with the project may help establish va-lidity as well as various readers for whom the ac-count is written.

Paradigm AssumptionsThe lens researchers use—their own, study

participants, or individuals external to the project—is not the only perspective that governs the choiceof validity procedures. Researchers’ paradigm as-sumptions or worldviews (Guba & Lincoln, 1994)also shape their selection of procedures. As sug-gested by Ratcliffe (1983),

Quite different notions of what constitutes validityhave enjoyed the status of dominant paradigm at dif-ferent times, in different historical contexts, and un-der different prevailing modes of thought andepistemology. (p. 158)

Three paradigm assumptions, labeled by Gubaand Lincoln (1994) as postpostivist, constructivist,and critical influence researchers’ choice of valid-ity procedures. These assumptions have been asso-ciated with different historical moments in theevolution of qualitative inquiry (Denzin & Lincoln,1994). A brief overview of these paradigm assump-tions is advanced here.

The postpostivist researcher assumes that qual-itative research consists of rigorous methods and sys-tematic forms of inquiry. Identified by Denzin andLincoln as the “modernist” phase of qualitative in-quiry (1994, p. 8), this philosophical perspectiveemerged in social science research during the 1970sand continues today. Individuals embracing thepostpostivist position both recognize and supportvalidity, look for quantitative equivalence of it,and actively employ procedures for establishingvalidity using specific protocols. Maxwell (1996),in Qualitative Research Design: An InteractiveApproach, for example, exemplifies postpostivistassumptions toward qualitative validity.

The constructivist or interpretive positionemerged during the period of 1970 to 1987 (Denzin& Lincoln, 1994), and it is reflected in stances to-ward validity today. Constructivists believe in plu-ralistic, interpretive, open-ended, and contextualized(e.g., sensitive to place and situation) perspectives


126


toward reality. The validity procedures reflectedin this thinking present criteria with labels distinctfrom quantitative approaches, such as trustworthi-ness (i.e., credibility, transferability, dependability,and confirmability), and authenticity (i.e., fairness,enlarges personal constructions, leads to improvedunderstanding of constructions of others, stimulatesaction, and empowers action). The classical workby Lincoln and Guba, Naturalistic Inquiry (1985),provides extensive discussions about these formsof trustworthiness and authenticity.

A third paradigm assumption is the criticalperspective. This perspective emerged during the1980s as the “crisis in representation” (Denzin &Lincoln, 1994, p. 9). As a challenge and critiqueof the modern state, the critical perspective holdsthat researchers should uncover the hidden assump-tions about how narrative accounts are constructed,read, and interpreted. What governs our perspec-tive about narratives is our historical situatednessof inquiry, a situatedness based on social, politi-cal, cultural, economic, ethnic, and gender ante-cedents of the studied situations. The implicationfor validity of this perspective is that validity iscalled into question, its assumptions interrogatedand challenged, and the researchers need to be re-flexive and disclose what they bring to a narrative.

Richardson (1994) uses the metaphor of acrystal as an image for validity: “Crystals are prismsthat reflect externalities and refract within themselves.. . . What we see depends on our angle of repose” (p.

522). To this end, researchers engage in validity pro-cedures of self-disclosure and collaboration withparticipants in a study. These procedures help tominimize further the inequality that participantsoften feel. For example, Carspecken’s Critical Eth-nography in Educational Research (1996) reportsvalidity procedures for tracking bias and interviewswith oneself as ways for researchers to be situatedin a study.

Validity Within Lens and ParadigmsAs shown in Table 1, we use the lens and

paradigm assumptions to create a two-dimensionalframework for locating nine different types of va-lidity procedures. The discussion now turns to thesenine procedures with a brief definition of each,their location within a lens and paradigm perspec-tive, and approaches for implementing each proce-dure. This list is not exhaustive but includes thoseprocedures commonly used and cited in qualitativeliterature.

TriangulationTriangulation is a validity procedure where

researchers search for convergence among multi-ple and different sources of information to formthemes or categories in a study. The term comesfrom military navigation at sea where sailors trian-gulated among different distant points to determinetheir ship’s bearing (Jick, 1979). Denzin (1978)identified four types of triangulation: across data

Table 1Validity Procedures Within Qualitative Lens and Paradigm Assumptions

Paradigm assump- Postpositivist or Constructivist tion/Lens Systematic Paradigm Paradigm Critical Paradigm

Lens of the Triangulation Disconfirming Researcher Researcher evidence reflexivity

Lens of Study Member checking Prolonged engage- Collaboration Participants ment in the field

Lens of People Ex- The audit trail Thick, rich Peer debriefing ternal to the Study description (Reviewers, Readers)


127


sources (i.e., participants), theories, methods (i.e.,interview, observations, documents), and amongdifferent investigators.

As a validity procedure, triangulation is a steptaken by researchers employing only the research-er’s lens, and it is a systematic process of sortingthrough the data to find common themes or cate-gories by eliminating overlapping areas. A popularpractice is for qualitative inquirers to provide cor-roborating evidence collected through multiplemethods, such as observations, interviews, and doc-uments to locate major and minor themes. The nar-rative account is valid because researchers gothrough this process and rely on multiple forms ofevidence rather than a single incident or data pointin the study.

Disconfirming evidenceA procedure closely related to triangulation

is the search by researchers for disconfirming ornegative evidence (Miles & Huberman, 1994). Itis the process where investigators first establishthe preliminary themes or categories in a studyand then search through the data for evidence thatis consistent with or disconfirms these themes. Inthis process, researchers rely on their own lens,and this represents a constructivist approach in thatit is less systematic than other procedures and re-lies on examining all of the multiple perspectiveson a theme or category.

In practice, the search for disconfirming evi-dence is a difficult process because researchers havethe proclivity to find confirming rather than dis-confirming evidence. Further, the disconfirmingevidence should not outweigh the confirming evi-dence. As evidence for the validity of a narrativeaccount, however, this search for disconfirmingevidence provides further support of the account’scredibility because reality, according to construc-tivists, is multiple and complex.

Researcher reflexivityA third validity procedure is for researchers to

self-disclose their assumptions, beliefs, and biases.This is the process whereby researchers report onpersonal beliefs, values, and biases that may shapetheir inquiry. It is particularly important for re-searchers to acknowledge and describe their enter-

ing beliefs and biases early in the research processto allow readers to understand their positions, andthen to bracket or suspend those researcher biasesas the study proceeds. This validity procedure usesthe lens of the researcher but is clearly positionedwithin the critical paradigm where individuals re-flect on the social, cultural, and historical forcesthat shape their interpretation.

Researchers might use several options for in-corporating this reflexivity into a narrative account.They may create a separate section on the “role ofthe researcher,” provide an epilogue, use interpre-tive commentary throughout the discussion of thefindings, or bracket themselves out by describingpersonal experiences as used in phenomenologicalmethods (Moustakas, 1994).

Member checkingWith member checking, the validity proce-

dure shifts from the researchers to participants inthe study. Lincoln and Guba (1985) describe mem-ber checks as “the most crucial technique for es-tablishing credibility” (p. 314) in a study. It consistsof taking data and interpretations back to the par-ticipants in the study so that they can confirm thecredibility of the information and narrative account.With the lens focused on participants, the research-ers systematically check the data and the narrativeaccount.

Several procedures facilitate this process. Apopular strategy is to convene a focus group ofparticipants to review the findings. Alternatively,researchers may have participants view the raw data(e.g., transcriptions or observational field notes)and comment on their accuracy. Throughout thisprocess, the researchers ask participants if thethemes or categories make sense, whether they aredeveloped with sufficient evidence, and whetherthe overall account is realistic and accurate. In turn,researchers incorporate participants’ comments intothe final narrative. In this way, the participantsadd credibility to the qualitative study by having achance to react to both the data and the final nar-rative.

Prolonged engagement in the fieldAnother validity procedure is for researchers

to stay at the research site for a prolonged period


128


of time. Fetterman (1989) contends that “workingwith people day in and day out for long periods oftime is what gives ethnographic research its valid-ity and vitality” (p. 46). During repeated observa-tion, the researchers build trust with participants,find gatekeepers to allow access to people and sites,establish rapport so that participants are comfort-able disclosing information, and reciprocate by giv-ing back to people being studied. This lens isfocused on gaining a credible account by buildinga tight and holistic case.

Being in the field over time solidifies evi-dence because researchers can check out the dataand their hunches and compare interview data withobservational data. It is not a process that is sys-tematically established, but constructivists recog-nize that the longer they stay in the field, the morethe pluralistic perspectives will be heard from par-ticipants and the better the understanding of thecontext of participant views. In practice, prolongedengagement in the field has no set duration, butethnographers, for example, spend from 4 monthsto a year at a site.

CollaborationCredible data also come from close collabo-

ration with participants throughout the process ofresearch. Collaboration means that the participantsare involved in the study as co-researchers or inless formal arrangements. This validity lens is oneof building the participant’s view into the study. Itbelongs to a critical paradigm perspective becausethe intent of the process is to respect and supportparticipants in a study, not further marginalizethem.

In practice, collaboration may assume multi-ple forms. For example, participants may help formthe research questions, assist with data collectionand analysis, and be involved in writing the narra-tive account. Some qualitative researchers mayshare the profits, such as book royalties or co-au-thorship publication rights. By actively involvingparticipants in their studies, qualitative inquirersadd further credibility to their narrative accounts.

The audit trailNow the lens for establishing validity shifts

again. The credibility of a study is established by

turning to individuals external to the project, suchas auditors—formally brought into the study—orreaders who examine the narrative account and at-test to its credibility. In establishing an audit trail,researchers provide clear documentation of all re-search decisions and activities. They may provideevidence of the audit trail throughout the account orin the appendices. Researchers may also use an ex-ternal auditor to review their study. The goal of aformal audit is to examine both the process and prod-uct of the inquiry, and determine the trustworthi-ness of the findings.

Lincoln and Guba (1985) use the analogy ofa fiscal audit to describe this process. The audit isoften used in formal studies, such as in disserta-tions, particularly when committee members aretrained quantitatively and may be skeptical aboutqualitative studies. Certain audiences appreciate therigor of the audit process, and the lens for estab-lishing credibility becomes someone external to theproject. It is a systematic procedure in that thereviewer writes an analysis after carefully study-ing the documentation provided by the researcher.

An audit trail is established by researchersdocumenting the inquiry process through journal-ing and memoing, keeping a research log of allactivities, developing a data collection chronolo-gy, and recording data analysis procedures clearly.The external auditor examines this documentationwith the following questions in mind: Are the find-ings grounded in the data? Are inferences logical?Is the category structure appropriate? Can inquirydecisions and methodological shifts be justified?What is the degree of researcher bias? What strate-gies were used for increasing credibility? (Schwandt& Halpern, 1988). Through this process of docu-menting a study and a review of the documenta-tion by an external auditor, the narrative accountbecomes credible.

Thick, rich descriptionAnother procedure for establishing credibility

in a study is to describe the setting, the participants,and the themes of a qualitative study in rich detail.According to Denzin (1989), “thick descriptions aredeep, dense, detailed accounts. . . . Thin descriptions,by contrast, lack detail, and simply report facts” (p.83). The purpose of a thick description is that it


129


creates verisimilitude, statements that produce forthe readers the feeling that they have experienced,or could experience, the events being described ina study. Thus, credibility is established throughthe lens of readers who read a narrative accountand are transported into a setting or situation.

To use this procedure for establishing credi-bility, researchers employ a constructivist perspec-tive to contextualize the people or sites studied.The process of writing using thick description isto provide as much detail as possible. It may in-volve describing a small slice of interaction, expe-rience, or action; locating individuals in specificsituations; bringing a relationship or an interactionalive between two or more persons; or providing adetailed rendering of how people feel (Denzin,1989).

With this vivid detail, the researchers helpreaders understand that the account is credible. Richdescription also enables readers to make decisionsabout the applicability of the findings to other set-tings or similar contexts.

Peer debriefingA peer review or debriefing is the review of

the data and research process by someone who isfamiliar with the research or the phenomenon be-ing explored. A peer reviewer provides support,plays devil’s advocate, challenges the researchers’assumptions, pushes the researchers to the next stepmethodologically, and asks hard questions aboutmethods and interpretations (Lincoln & Guba,1985).

The lens for establishing credibility is some-one external to the study, and a critical paradigmis operating because of the close collaboration be-tween the external reviewer and the qualitative re-searcher. This procedure is best used over timeduring the process of an entire study. Peer debrief-ers can provide written feedback to researchers orsimply serve as a sounding board for ideas. Byseeking the assistance of peer debriefers, research-ers add credibility to a study.

Positioning OurselvesOur approach is to use several validity pro-

cedures in our studies. Certainly some strategiesare easier to use than others, particularly those in-

herent in the study design, such as triangulation ofmethods, prolonged observations in the field, andthe use of thick, rich descriptions. In deciding touse a formal audit or peer debriefer, researchersshould consider their audiences, the availability ofsuch individuals, and the expense of using them.Member checking is always important as well askeeping research logs to document the rigor of ourresearch processes. When faced with students orfaculty committees that seek rigor and a systemat-ic review of procedures, the process of establish-ing a clear audit trail is most important.

As we review the nine validity procedures,we acknowledge the importance of all three lensesand that their emphasis in a study will vary de-pending on the project, the audience for whom weare writing, and the people available to provide anassessment of our project. Our primary lens, how-ever, is always that of the participants in a study,and we have become more reflexive in our studies,acknowledging the inseparableness of the research-er and the process of inquiry.

As for our paradigm stances, we most close-ly align ourselves with the use of systematic pro-cedures, employing rigorous standards and clearlyidentified procedures (e.g., Creswell, 1998). How-ever, we also resonate with the critical perspectiveand engage in collaborative research practices thatare respectful of the individuals we study (e.g.,Miller, Creswell, & Olander, 1998). What is mostimportant is that the credibility of the account beconveyed in a qualitative study. We suggest thatthe use of validity procedures requires thinkingbeyond specific procedures—to acknowledge thelens being employed in a study and the paradigmassumptions of the researchers.

ReferencesAltheide, D.L., & Johnson, J.M. (1994). Criteria for

assessing interpretive validity in qualitative re-search. In N.K. Denzin & Y.S. Lincoln (Eds.),Handbook of qualitative research (pp. 485-499).Thousand Oaks, CA: Sage.

American Educational Research Association, AmericanPsychological Association, & National Council onMeasurement in Education. (1982). Standards for ed-ucational and psychological testing. Washington,DC: American Educational Research Association.

Campbell, D.T., & Stanley, J.C. (1966). Experimentaland quasi-experimental designs for research. In


130


N.L. Gage, Handbook of research on teaching (pp.1-76). Chicago, IL: Rand-McNally.

Carspecken, P.F. (1996). Critical ethnography in edu-cational research: A theoretical and practicalguide. New York: Routledge.

Creswell, J.W. (1998). Qualitative inquiry and researchdesign: Choosing among five traditions. ThousandOaks, CA: Sage.

Denzin, N.K. (1978). The research act: A theoreticalorientation to sociological methods (2nd ed.). NewYork: McGraw-Hill.

Denzin, N.K. (1989). Interpretive interactionism. New-bury Park, CA: Sage.

Denzin, N.K., & Lincoln, Y.S. (1994). Introduction: En-tering the field of qualitative research. In N.K. Den-zin & Y.S. Lincoln (Eds.), Handbook of qualitativeresearch (pp. 1-17). Thousand Oaks, CA: Sage.

Donmoyer, R. (1996). Educational research in an eraof paradigm proliferation: What’s a journal editorto do? Educational Researcher, 25(2), 19-25.

Fetterman, D.M. (1989). Ethnography: Step by step(Applied Social Research Methods Series, No. 17).Newbury Park, CA: Sage.

Guba, E.G., & Lincoln, Y.S. (1994). Competing para-digms in qualitative research. In N.K. Denzin &Y.S. Lincoln (Eds.), Handbook of qualitative re-search (pp. 105-117). Thousand Oaks, CA: Sage.

Hammersley, M., & Atkinson, P. (1983). Ethnogra-phy: Principles in practice (2nd ed.). New York:Routledge.

Jick, T.D. (1979). Mixing qualitative and quantitativemethods: Triangulation in action. Administrative Sci-ence Quarterly, 24, 602-611.

Lather, P. (1993). Fertile obsession: Validity after poststruc-turalism. The Sociological Quarterly, 34, 673-693.

Lincoln, Y.S., & Guba, E.G. (1985). Naturalistic inquiry.Newbury Park, CA: Sage.

Maxwell, J.A. (1992). Understanding and validity in qual-itative research. Harvard Educational Review, 62,279-300.

Maxwell, J.A. (1996). Qualitative research design: Aninteractive approach. Thousand Oaks, CA: Sage.

Merriam, S.B. (1998). Qualitative research and case studyapplications in education. San Francisco: Jossey-Bass.

Miles, M.B., & Huberman, A.M. (1994). Qualitative dataanalysis: An expanded sourcebook (2nd ed.). New-bury Park, CA: Sage.

Miller, D.L., Creswell, J. W., & Olander, L.S. (1998).Writing and retelling multiple ethnographic tales ofa soup kitchen for the homeless. Qualitative Inquiry,4, 469-491.

Moustakas, C. (1994). Phenomenological research meth-ods. Thousand Oaks, CA: Sage.

Patton, M.Q. (1980). Qualitative evaluation methods.Newbury Park, CA: Sage.

Ratcliffe, J.W. (1983). Notions of validity in qualitativeresearch methodology. Knowledge: Creation, Diffu-sion, Utilization, 5(2), 147-167.

Richardson, L. (1994). Writing: A method of inquiry. InN.K. Denzin & Y.S. Lincoln (Eds.), Handbook of qual-itative research (pp. 516-529). Thousand Oaks, CA:Sage.

Schwandt, T.A. (1997). Qualitative inquiry: A dictionaryof terms. Thousand Oaks, CA: Sage.

Schwandt, T.A., & Halpern, E.S. (1988). Linking audit-ing and metaevaluation: Enhancing quality in ap-plied research. Newbury Park, CA: Sage.


creswell, determining validity in qualitative inquiry

Documents