r10s2w1

8/3/2019 R10S2W1

http://slidepdf.com/reader/full/r10s2w1 1/6

RESEARCH ON SOCIAL WORK PRACTICE Rubin / STANDARDS FOR RIGOR

Standards for Rigor in Qualitative Inquiry

Allen Rubin

President, Society for Social Work and Research

When the Society for Social Work and Research (SSWR) began to sponsor

this journal, its board asked that the journal make more explicit that in addi-

tionto rigorous quantitative studies, it encouragesthe submission of rigorous

studies that employ qualitative research methods. The journal’s editor, Bruce

Thyer, has met that request, and the manuscripts he has been receiving seem

to indicate that some social work researchers have heard the message. Brucehas appointed a number of qualitative researchers to the journal’s editorial

board, effective January 1, 2000. They include Deborah Padgett, James

Drisko, Jane Gilgun, ChristineLowery, andGardenia Harris. These appoint-

ments ought to enhance the fairness of appraisals of qualitative studies sub-

mitted, and they reflect Bruce’s commitment to make the journal more inclu-

sive of diverse research methods.

I am delighted with this development. Although most of my published

work has emphasized quantitative methods, I love rigorous qualitative stud-

ies that provide important findings. To wit, a lengthy chapter on qualitative

inquiry in the textbook I coauthored (Rubin & Babbie, 1997) identifies quite

a few qualitative studies that I admire. Also, I invited SSWR Board member

Deborah Padgett, author of the text Qualitative Methods in Social Work

(Padgett, 1998), to chair theAbstracts Review Committeefor thenext SSWRconference, which will convene January 2001 in Atlanta.

ButI wonder, based on theunevennessin thequalityof manuscripts I have

reviewedfor quitea few journals in recentyears, whether prospectiveauthors

have sufficient information on what is meant by rigor in qualitative inquiry.

Moreover, I doubtthat thereis sufficient agreementamongsocial scientists in

general about standards for rigor in qualitative inquiry. To be sure, I have

found as much unevenness in the quality of quantitative manuscripts I have

reviewed as well. Unevenness in the rigor of quantitative manuscripts canbe

moredishearteningthanunevenness in qualitativemanuscripts because there

has long been a fairly clear consensusabout standardsforquality in quantita-

tive inquiry. I find the unevenness in the quality of manuscripts reporting

qualitative studies to be more understandable because the use of qualitative

methods is growing in the absence of a consensus on standards.

173

Research on Social Work Practice, Vol. 10 No. 2, March 2000 173-178

© 2000 Sage Publications, Inc.

8/3/2019 R10S2W1


My hunch is that the unevenness in the quality of manuscripts reporting

qualitative studies may stem from two problems. One, which I already men-

tioned, was suggested to me in an e-mail discussion with Deborah Padgett,

whonoted“a lack ofconsensus(evenamong thoseon thesameepistemologi-

cal wavelength) about what standards for rigor are or should be.” The other

problem may pertain to epistemological differences. Some qualitatively ori-

ented investigators favor radical versions of relativistic epistemologies and

believe that a multiplicity of subjective truths implies the absence of objec-

tive truth.They maytherefore rejectany means,even qualitativeones, to pur-

sueobjectivity. They mayalso rejectany standards, even qualitativeones, for

evaluating the rigor of qualitative studies. In short, they may think that quali-

tative inquiry means anything goes. I will not repeat my argument with that

line of thinking here; readers canseemy January 2000 editorial for that. Suf-ficeit tosay that rigorous, atleast inthe context of thisjournal and inthe view

of thiswriter (andreviewer),meansthat qualitative studies should conform to

standards that have been developed for qualitative inquiry.

A chief strength of qualitative inquiry is thedepth of understanding that it

can provide. Manuscripts reporting qualitative studies should not be rushed

or superficial. They should report observations and findings in a thorough

manner that enables readers to gain an in-depth understanding of the phe-

nomenon being studied and that provides a compelling case for the author’s

interpretations. Readers shouldnot be left feeling that theauthors aremerely

saying, “Trust me.” Instead, reportage should conform to standards that pro-

vide qualitative analogues for dealing with concerns about the objectivity,

reliability, validity, and representativeness of observations. An excellent

introductory summary of those standards can be found in Deborah’s text.According to Deborah, thekey issues in qualitative research are trustwor-

thiness and credibility. Three important threats to trustworthiness and credi-

bilityinclude thedistorting effects of (a)reactivityconnected to theimpact of

the researcher’s presence in the field, (b) researcher biases in the way one

asks questions and conducts as well as filters observations, and (c) respon-

dent biases connected largely to social desirability issues.

Deborah proposes six strategies for addressing the above threats and thus

for enhancing the rigor of qualitative studies. One involves prolonged

engagement. This may involve conducting lengthy interviews rather than

relying on brief ones, conducting follow-up interviews (perhaps several per

respondent), and spending long periods of time in the field building trusting

relationships with respondents. Prolonged engagement is thought to lessen

the likelihood of reactivityandrespondentbias because theresearcher’s pres-

ence after a long period of engagement mayseem less obtrusive and because

respondents maybe less inclined or less able to deceive theresearcher during

174 RESEARCH ON SOCIAL WORK PRACTICE

8/3/2019 R10S2W1


a prolonged period with a trusting relationship. Lengthier interviews also

may improve the interviewer’s chances of overcoming potential social desir-

ability biases and getting at the truth.

A second strategy involves triangulation, in which one attempts to cor-

roborate observations via multiple observational strategies, multiple sources

of observations, multiple modes of analysis,andthe useof data analysts from

different disciplines and with different theoretical orientations. A third strat-

egy involves peer debriefing and support as a mechanism for guarding

against the researcher’s biases. Member checking is a fourth strategy. This

involves asking respondents if they agree with the codes and interpretations

of the researchers. A fifth strategy involves negative case analysis, in which

the researchers conduct a thorough search for cases that do not fit their inter-

pretations. A sixth strategy involves leaving an audit trail of raw data (fieldnotes, interview transcripts, and so on) that others can use to see if your find-

ings are reproducible and verified.

In discussing these strategies with me via e-mail, Deborah notes that they

are not presented so much as prescriptive “demands” but as guidelines.

According to Deborah, “It would be inappropriate for some studies to use

some of the strategies. For example, a researcher studying terminal cancer

patients mayfindmember checking to be unethicalandunfairlyburdensome.

Similarly, prolongedengagement maynotbe possiblewithsometargetpopu-

lations. Theidea is to apply as many as possible andas areappropriate.” I rec-

ommend that you read Deborah’s text for elaboration of these strategies and

for other useful guidance for conducting and reporting rigorous qualitative

studies.

When I review manuscripts reporting qualitative studies submitted forpublication to this journal as well as several other journals, I look to see if the

authors used mechanisms, such as the six strategies proposed by Deborah,

that attempt to alleviate threats to the trustworthinessandcredibility of quali-

tative inquiry. I also look to see if the way the study was conducted, and the

way the findings were reported, fit the in-depth nature of the aims of qualita-

tive inquiry. Alas,usually they donot(justas most of thequantitativereports I

review have equally serious flaws that need to be corrected—if possible—

before they merit publication).

Some of the qualitative studies that I have reviewed seem to suggest that

their authors think that calling their studies qualitative justifies an anything

goesmentality. Rather thanusingprolongedengagementand providingthick

descriptions that enable readers to walk in the shoes of their respondents and

thus discover new insights and deeper meanings, these studies tend to report

disparate brief (usually one line) quotes generated from open-ended ques-

tionnaireitemsor brief interviews with no follow-ups.Rarelyin these studies

Rubin / STANDARDS FOR RIGOR 175

8/3/2019 R10S2W1


do I see sufficient efforts to triangulate or search for negative cases. Report-

age of quantitative aspects of the findings is often vague. Rather than report

exactly how many respondents made statements supporting a particular

interpretation, the author will say “Many respondents said . . .” or “Often

respondents said . . .” When I read this sort of reportage, I cannot help but

wonder how much is many and how often is often. When authors are vague

about this and show no effort to delve thoroughly into cases that do not fit

their interpretations, readers might be skeptical about theextent to which the

interpretationsare influenced by theauthor’s biases (just as they maywonder

about uncontrolled biases in quantitative studies). On the other hand, the

ambiguity and brevity in some qualitative reports may have less to do with

author bias than with a lack of consensus about reportage standards or an

insufficient awareness of those standards among authors.Ambiguity also is common in the way sampling and measurement are

reported. Granted, using probability sampling procedures to attain represen-

tative samples is not expected in qualitative studies. But why not be specific

and thorough in reporting the nonprobability sampling procedures that are

employed? Some authors just say that theirs was a convenience sample and

leave it at that. This leaves important questions unanswered, the qualitative

nature of the study notwithstanding. For example, suppose two leading pro-

ponents of emphasizing single-case evaluation in research courses conduct

and report a qualitative study involving interviews of social work students

asking them their attitudes about their research courses. Suppose they just

reportthat it wasa conveniencesample of 50students.Theirfindings indicate

that the students found that learning about single-case evaluation was by far

the single most beneficial aspect of their entire social work education. Doesthefact that thestudywas qualitativeobviate theneed forus toknow whether,

for example, theconveniencesampleconsistedentirely of students currently

enrolled in the authors’ research courses (and hoping for good grades)? I

think not. Does the fact that the study was qualitative obviate the need for us

to know whether and exactly how many follow-up interviews were con-

ducted? Does it obviate the need to report how the interviews were con-

ducted, by whom, how questions were worded, and so on so as to protect

against researcher and respondent biases? Again, I think not. Does the fact

that the study was qualitative obviate the need for us to know whether other

researchers who do not share the authors’enthusiasm for single-case evalua-

tion conducted an independent analysis of the raw data and arrived at the

same conclusions. I do not think so.

Rigorous qualitative research is labor intensive. It is not a shortcut. It can

be harder and more time-consuming than quantitative research. Although

qualitative research uses flexible methods, it does not mean anything goes.


8/3/2019 R10S2W1


Faculty who are experienced and esteemed qualitative investigators try to

convey these caveats to those who take their courses. This maycompel some

doctoral students who prefer qualitative inquiry to opt for quantitative disser-

tations so thatthey cancomplete their doctoraleducationmore quickly. Other

doctoral students, because of the strength of their preferences for qualitative

inquiry, might letthesecaveatsgo in oneear andout theother. (Thosewho do

soandthenattempt todo qualitativedissertations oftengeta rude awakening!

Thesamegoes forstudentslooking forshortcuts aroundstandards forrigorin

quantitative dissertations, by the way.) Perhaps the pressure to publish

encountered when one joins a faculty is so great that these caveats become

forgotten. On the other hand, a more positive possible explanation for the

insufficient attention to methodological rigor in some manuscripts may be

that adequate strategies forrigorare beingused but arejust notbeingreportedin the methods sections of manuscripts. In this connection, Deborah men-

tioned in an e-mail discussion with me, “I am continuously dismayed to see

what mayhave been fairly rigorous qualitative studies with methods sections

that are completely nonspecific and nondescriptive. Granted, journals have

space requirements, but I see little excuse for leaving out a fairly detailed

description of the study’s methodology and for acknowledging in the

strengths and limitations section of the report what strategies for rigor were

and were not used (and perhaps why).”

Whatever the explanation, I urge readers contemplating doing qualitative

research to take the above mentioned caveats seriously. If you think that you

can get around issues such as bias just by calling your study qualitative, be

prepared to have your manuscript rejected (unless, of course, youcansubmit

it to a journal whose editors reject notions of objectivity and standards of rigor). On the other hand, if you have done a rigorous study, be sure to be

explicit in reportingitsmethods, andbe sure to thoroughly attend to issuesof

rigor in both the methods and limitations sections of your manuscript.

I hope readers will not misconstrue this editorialas reflecting a dislike for

qualitative research. What I dislike is research that lacks rigor. I dislike

shoddy studies regardless of the methods they employ—whether they are

quantitative or qualitative. As I noted at thebeginning of this editorial, I love

rigorous qualitative studies that provide important findings. So please do not

dismiss this editorialas theravings of an inchworm. Instead, pleasesee it as a

constructiveeffort to increase the submission andeventualpublication of rig-

orous qualitative studies.

—Allen RubinPresident, Society for Social Work

and Research

Rubin / STANDARDS FOR RIGOR 177

8/3/2019 R10S2W1


REFERENCES

Padgett, D. K. (1998). Qualitative methods in social work . Thousand Oaks, CA: Sage.

Rubin, A., & Babbie, E. (1997). Research methods for social work . Pacific Grove, CA:

Brooks/Cole.


r10s2w1

Documents