validity in qual research

7/29/2019 Validity in Qual Research

1/8

Determining Validity in Qualitative InquiryAuthor(s): John W. Creswell and Dana L. MillerSource: Theory into Practice, Vol. 39, No. 3, Getting Good Qualitative Data to ImproveEducational Practice (Summer, 2000), pp. 124-130Published by: Taylor & Francis, Ltd.Stable URL: http://www.jstor.org/stable/1477543 .

Accessed: 23/06/2011 06:15

Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless

you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you

may use content in the JSTOR archive only for your personal, non-commercial use.

Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at .http://www.jstor.org/action/showPublisher?publisherCode=taylorfrancis. .

Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printedpage of such transmission.

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of

content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms

of scholarship. For more information about JSTOR, please contact [email protected].
http://www.jstor.org/action/showPublisher?publisherCode=taylorfrancishttp://www.jstor.org/stable/1477543?origin=JSTOR-pdfhttp://www.jstor.org/page/info/about/policies/terms.jsphttp://www.jstor.org/action/showPublisher?publisherCode=taylorfrancishttp://www.jstor.org/action/showPublisher?publisherCode=taylorfrancishttp://www.jstor.org/page/info/about/policies/terms.jsphttp://www.jstor.org/stable/1477543?origin=JSTOR-pdfhttp://www.jstor.org/action/showPublisher?publisherCode=taylorfrancis


2/8

JohnW CreswellDana L. Miller

Determining alidityi n Qualitative Inquiry

W RITING ABOUT VALIDITY IN QUALITATIVE n-quiry is challenging on many levels. Mul-tiple perspectives about it flood the pages of books(e.g., Lincoln & Guba, 1985; Maxwell, 1996; Mer-riam, 1998; Schwandt, 1997) and articles and chap-ters (e.g., Altheide & Johnson, 1994; Lather, 1993;Maxwell, 1992). In these texts, readersare treatedto

a confusing arrayof termsfor validity, includingau-thenticity,goodness, verisimilitude,adequacy,trust-worthiness, plausibility, validity, validation, andcredibility.Various authorshave constructeddiversetypologies of validity (e.g., Maxwell's five types,1992; Lather's four frames, 1993; and Schwandt'sfour positions, 1997). It is little wonder that Don-moyer (1996), who wrote an editorial on validityin the Educational Researcher, commented on thediverse perspectivesof validity by contrastingMilesand Huberman's (1994) "traditionalconception ofvalidity" with Lather's (1993) "ironic validity" (p.21). Novice researchers, in particular,can become

qualitative projects (e.g., Lincoln & Guba, 1985;Maxwell, 1996; Merriam,1998).Qualitative esearch-ersroutinelyemploymemberchecking,triangulation,thick description,peer reviews, and external audits.Researchersengage in one or more of these proce-dures andreportresults in theirinvestigations.As helpful as they are, these discussions aboutvalidity procedures provide little guidance as towhy one procedure might be selected for use byresearchers over other procedures. In this article,we suggest that the choice of validity proceduresis governed by two perspectives: the lens research-ers choose to validate their studies and researchers'paradigm assumptions. We advance a two-dimen-sional framework that can help researchers identi-fy appropriatevalidity proceduresfor their studies.The use of this framework can provide a ra-tionale for choice of a procedure beyond what thesetting and participants will bear and what col-leagues and faculty advisers recommend. The


3/8

Creswell and MillerDetermining Validity

them (Schwandt, 1997). Procedures for validityinclude those strategies used by researchers to es-tablish the credibility of their study. Throughoutthis discussion, we make the assumption that va-lidity refers not to the data but to the inferencesdrawn from them (Hammersley& Atkinson, 1983).

The Lens Used by the ResearcherWhen we refer to the lens, we mean that theinquirer uses a viewpoint for establishing validityin a study. Qualitative inquirers bring to their stud-ies a different lens towardvalidity than thatbroughtto traditional, quantitative studies.In quantitative research, investigators aremost concerned about the specific inferences madefrom test scores on psychometric instruments(i.e.,the construct, criterion,and contentvalidity of inter-pretationsof scores) (AERA, APA, & NCME, 1982)and the internaland externalvalidityof experimentaland quasi-experimentaldesigns (Campbell & Stan-ley, 1966). In contrast,qualitativeresearchersuse alens not based on scores, instruments,or researchdesigns buta lens establishedusing the views of peo-ple who conduct,participate n, or read andreview astudy.For example, one lens to determine the cred-ibility of a study is the particular lens of the re-searcher. Researchersdeterminehow long to remainin the field, whether the data are saturated to es-tablish good themes or categories, and how theanalysis of the data evolves into a persuasive nar-rative. Patton (1980) describes this process as onewhere qualitative analysts returnto their data "overand over again to see if the constructs, categories,explanations, and interpretationsmake sense" (p.339). Altheide and Johnson (1994) refer to it as"validity-as-reflexive-accounting"p. 489) where re-

the interpretations accurately represent them. Athird lens may be the credibility of an account byindividuals external to the study. Reviewers notaffiliated with the project may help establish va-lidity as well as various readers for whom the ac-count is written.

Paradigm AssumptionsThe lens researchers use-their own, studyparticipants,or individuals external to the project-is not the only perspective that governs the choiceof validity procedures. Researchers' paradigm as-sumptions or worldviews (Guba & Lincoln, 1994)also shape their selection of procedures. As sug-gested by Ratcliffe (1983),Quite different notions of what constitutesvalidityhaveenjoyedthe statusof dominantparadigm tdif-ferenttimes, in differenthistoricalcontexts,and un-der different prevailing modes of thought andepistemology.(p. 158)

Threeparadigmassumptions,labeled by Gubaand Lincoln (1994) as postpostivist, constructivist,and critical influence researchers' choice of valid-ity procedures.These assumptions have been asso-ciated with different historical moments in theevolution of qualitative inquiry (Denzin & Lincoln,1994). A brief overview of these paradigm assump-tions is advanced here.The postpostivistresearcherassumes thatqual-itative researchconsists of rigorousmethods andsys-tematic forms of inquiry. Identifiedby Denzin andLincoln as the "modernist"phase of qualitativein-quiry (1994, p. 8), this philosophical perspectiveemerged in social science researchduringthe 1970sand continues today. Individuals embracing thepostpostivist position both recognize and supportvalidity, look for quantitative equivalence of it,


4/8

THEORY NTO PRACTICE Summer 2000Getting Good Qualitative Datatoward reality. The validity procedures reflectedin this thinking present criteria with labels distinctfrom quantitative approaches, such as trustworthi-ness (i.e., credibility, transferability,dependability,and confirmability),and authenticity(i.e., fairness,enlarges personal constructions,leads to improvedunderstandingof constructions of others, stimulatesaction, and empowers action). The classical workby Lincoln and Guba, Naturalistic Inquiry (1985),provides extensive discussions about these formsof trustworthinessand authenticity.A third paradigm assumption is the criticalperspective. This perspective emerged during the1980s as the "crisis in representation"(Denzin &Lincoln, 1994, p. 9). As a challenge and critiqueof the modern state, the critical perspective holdsthat researchers should uncover the hidden assump-tions about how narrativeaccounts are constructed,read, and interpreted. What governs our perspec-tive about narratives is our historical situatednessof inquiry, a situatedness based on social, politi-cal, cultural, economic, ethnic, and gender ante-cedents of the studied situations. The implicationfor validity of this perspective is that validity iscalled into question, its assumptions interrogatedand challenged, and the researchers need to be re-flexive and disclose what they bring to a narrative.Richardson (1994) uses the metaphor of acrystal as an image for validity: "Crystalsareprismsthatreflect externalitiesandrefractwithinthemselves.... What we see dependson ourangle of repose" (p.

522). To this end,researchers ngage in validity pro-cedures of self-disclosure and collaboration withparticipants in a study. These procedures help tominimize further the inequality that participantsoften feel. For example, Carspecken's Critical Eth-nography in Educational Research (1996) reportsvalidity proceduresfor trackingbias and interviewswith oneself as ways for researchersto be situatedin a study.

Validity Within Lens and ParadigmsAs shown in Table 1, we use the lens andparadigm assumptions to create a two-dimensionalframework for locating nine different types of va-lidity procedures.The discussion now turnsto thesenine procedures with a brief definition of each,their location within a lens and paradigm perspec-tive, and approachesfor implementing each proce-dure. This list is not exhaustive but includes thoseprocedures commonly used and cited in qualitativeliterature.TriangulationTriangulation is a validity procedure whereresearchers search for convergence among multi-ple and different sources of information to formthemes or categories in a study. The term comesfrom military navigation at sea where sailors trian-gulated among different distantpoints to determinetheir ship's bearing (Jick, 1979). Denzin (1978)identified four types of triangulation: across data

Table 1Validity Procedures Within Qualitative Lens and Paradigm Assumptions

Paradigm assump- Postpositivist or Constructivisttion/Lens Critical


5/8


sources (i.e., participants), theories, methods (i.e.,interview, observations, documents), and amongdifferent investigators.As a validity procedure, triangulation s a steptaken by researchers employing only the research-er's lens, and it is a systematic process of sortingthrough the data to find common themes or cate-gories by eliminating overlapping areas. A popularpractice is for qualitative inquirers to provide cor-roborating evidence collected through multiplemethods, such as observations,interviews, anddoc-uments to locate majorand minor themes. The nar-rative account is valid because researchers gothrough this process and rely on multiple forms ofevidence ratherthan a single incident or data pointin the study.Disconfirming evidenceA procedure closely related to triangulationis the search by researchers for disconfirming ornegative evidence (Miles & Huberman, 1994). Itis the process where investigators first establishthe preliminary themes or categories in a studyand then search through the data for evidence thatis consistent with or disconfirms these themes. Inthis process, researchers rely on their own lens,and this representsa constructivist approach n thatit is less systematic than other procedures and re-lies on examining all of the multiple perspectiveson a theme or category.In practice, the search for disconfirming evi-dence is a difficult process because researchershavethe proclivity to find confirming rather than dis-confirming evidence. Further, the disconfirmingevidence should not outweigh the confirming evi-dence. As evidence for the validity of a narrativeaccount, however, this search for disconfirming

ing beliefs and biases early in the researchprocessto allow readers to understand their positions, andthen to bracket or suspend those researcher biasesas the study proceeds. This validity procedureusesthe lens of the researcher but is clearly positionedwithin the critical paradigm where individuals re-flect on the social, cultural, and historical forcesthat shape their interpretation.Researchers might use several options for in-corporatingthis reflexivity into a narrativeaccount.They may create a separate section on the "role ofthe researcher,"provide an epilogue, use interpre-tive commentary throughout the discussion of thefindings, or bracket themselves out by describingpersonal experiences as used in phenomenologicalmethods (Moustakas, 1994).Member checkingWith member checking, the validity proce-dure shifts from the researchers to participants inthe study. Lincoln and Guba (1985) describe mem-ber checks as "the most crucial technique for es-tablishingcredibility" (p. 314) in a study. It consistsof taking data and interpretationsback to the par-ticipants in the study so that they can confirm thecredibilityof the information andnarrativeaccount.With the lens focused on participants,the research-ers systematically check the data and the narrativeaccount.Several procedures facilitate this process. Apopular strategy is to convene a focus group ofparticipants to review the findings. Alternatively,researchersmay have participantsview the raw data(e.g., transcriptions or observational field notes)and comment on their accuracy. Throughout thisprocess, the researchers ask participants if thethemes or categories make sense, whether they are


6/8

THEORY NTO PRACTICE Summer 2000Getting Good Qualitative Dataof time. Fetterman (1989) contends that "workingwith people day in and day out for long periods oftime is what gives ethnographic research its valid-ity and vitality" (p. 46). During repeated observa-tion, the researchers build trust with participants,find gatekeepersto allow access to people and sites,establish rapport so that participants are comfort-able disclosing information,andreciprocateby giv-ing back to people being studied. This lens isfocused on gaining a credible account by buildinga tight and holistic case.Being in the field over time solidifies evi-dence because researchers can check out the dataand their hunches and compare interview data withobservational data. It is not a process that is sys-tematically established, but constructivists recog-nize that the longer they stay in the field, the morethe pluralistic perspectives will be heard from par-ticipants and the better the understanding of thecontext of participantviews. In practice, prolongedengagement in the field has no set duration, butethnographers,for example, spend from 4 monthsto a year at a site.CollaborationCredible data also come from close collabo-ration with participants throughout the process ofresearch. Collaboration means that the participantsare involved in the study as co-researchers or inless formal arrangements.This validity lens is oneof building the participant'sview into the study. Itbelongs to a critical paradigmperspective becausethe intent of the process is to respect and supportparticipants in a study, not further marginalizethem.In practice, collaboration may assume multi-ple forms. For example, participantsmay help form

turning to individuals external to the project, suchas auditors-formally brought into the study-orreaders who examine the narrative account and at-test to its credibility. In establishing an audit trail,researchers provide clear documentation of all re-search decisions and activities. They may provideevidence of the audit trailthroughout he accountorin the appendices.Researchersmay also use an ex-ternal auditorto review their study. The goal of aformal audit s to examine both theprocessandprod-uct of the inquiry, and determine the trustworthi-ness of the findings.Lincoln and Guba (1985) use the analogy ofa fiscal audit to describe this process. The audit isoften used in formal studies, such as in disserta-tions, particularly when committee members aretrained quantitatively and may be skeptical aboutqualitativestudies. Certainaudiences appreciatetherigor of the audit process, and the lens for estab-lishing credibility becomes someone external to theproject. It is a systematic procedure in that thereviewer writes an analysis after carefully study-ing the documentationprovided by the researcher.An audit trail is established by researchersdocumenting the inquiry process throughjournal-ing and memoing, keeping a research log of allactivities, developing a data collection chronolo-gy, and recording data analysis proceduresclearly.The external auditor examines this documentationwith the following questions in mind: Are the find-ings grounded in the data? Are inferences logical?Is the category structureappropriate?Can inquirydecisions and methodological shifts be justified?What is the degree of researcher bias? What strate-gies were used for increasingcredibility?(Schwandt& Halpern, 1988). Through this process of docu-menting a study and a review of the documenta-


7/8


creates verisimilitude, statements that produce forthe readers the feeling that they have experienced,or could experience, the events being described ina study. Thus, credibility is established throughthe lens of readers who read a narrative accountand are transportedinto a setting or situation.To use this procedure for establishing credi-bility, researchersemploy a constructivist perspec-tive to contextualize the people or sites studied.The process of writing using thick description isto provide as much detail as possible. It may in-volve describing a small slice of interaction, expe-rience, or action; locating individuals in specificsituations; bringing a relationship or an interactionalive between two or more persons; or providing adetailed rendering of how people feel (Denzin,1989).With this vivid detail, the researchers helpreaders understand hatthe accountis credible.Richdescription also enables readers to make decisionsabout the applicability of the findings to other set-tings or similar contexts.Peer debriefing

A peer review or debriefing is the review ofthe data and research process by someone who isfamiliar with the research or the phenomenon be-ing explored. A peer reviewer provides support,plays devil's advocate, challenges the researchers'assumptions,pushes the researchersto the next stepmethodologically, and asks hard questions aboutmethods and interpretations (Lincoln & Guba,1985).The lens for establishing credibility is some-one external to the study, and a critical paradigmis operating because of the close collaboration be-tween the external reviewer and the qualitative re-

herent in the study design, such as triangulationofmethods, prolonged observations in the field, andthe use of thick, rich descriptions. In deciding touse a formal audit or peer debriefer, researchersshould consider their audiences, the availability ofsuch individuals, and the expense of using them.Member checking is always important as well askeeping research logs to document the rigor of ourresearch processes. When faced with students orfaculty committees that seek rigor and a systemat-ic review of procedures, the process of establish-ing a clear audit trail is most important.As we review the nine validity procedures,we acknowledge the importanceof all three lensesand that their emphasis in a study will vary de-pending on the project, the audience for whom weare writing, and the people available to provide anassessment of our project. Our primarylens, how-ever, is always that of the participantsin a study,and we have become more reflexive in our studies,acknowledging the inseparablenessof the research-er and the process of inquiry.As for our paradigmstances, we most close-ly align ourselves with the use of systematic pro-cedures, employing rigorous standards and clearlyidentified procedures (e.g., Creswell, 1998). How-ever, we also resonate with the critical perspectiveand engage in collaborative research practices thatare respectful of the individuals we study (e.g.,Miller, Creswell, & Olander, 1998). What is mostimportant is that the credibility of the account beconveyed in a qualitative study. We suggest thatthe use of validity procedures requires thinkingbeyond specific procedures-to acknowledge thelens being employed in a study and the paradigmassumptions of the researchers.


8/8

validity in qual research

Documents