rntersecnons and utvergence

Sqndro Mqthison

lniroduction

Within the discipline and pmctice oI evaluarion there is confusion aboulholr prccircly l'csc.uch and evalundon lre differcnr. Add the ̂ qjccrive"lerninisf' to bodr and the confusion nray bc amplified. This cllaprcrdiscusses thc similarities ̂nd differenccs berlreen rese^rch ancl evi Lra-tion gener^lly, and conclucLcs by int.oduciD8 rllc core id€as of ferlinisrresearch and evaluat.ion; the full articularion offeminisr cvaiuarior\ is leftto other authors in this book.

Controsting Reseorch ond Evqluoiion

Olfcring a definition of et)aluation ^s the process and producr ofmakingjudgments nbout the value, merir, or worth of an evaluanal does little tomswer rhe percnnial questionr What is the difcrence between evalua-tion and research? This quesrion abour difterences between evaluariouand rcsearch is fuelcd by the fict that evaluation as a discipline clml\,s onother disciplines for irs foundarions, and especialty rhe sociat sciences forits methods- As evajuation has marured as a discipline andprcfession thisquesrion is somerimes poscd to clarily vhat is disrincr abolr evaluation.This delineation of a proiession of evatuation is atso ried to a discussionof who is and can be an evaluaror. What knowledge and skils does evatu-adon require, and how does this differ from the knowledge and skills of

ryi':.;'*X$tt{i: C t", & t}?€ R 2 rt:i*rxry]::;*Sf

qndEvoluotionReseorch

o@

I, o:saiM

@

(,)@

social scicrce rcscarchers? Scriven suggesN eraluarors must know ho\\, tosearch for uninknded and side effectsj how ro determine rahes withindifterert points ofvic1l'; how b deal wirh conrroversial issues and valucs;and ho* ro s)'nthcsizc facts and values (CoffnaD, 2003-2004).

Although there are argumenrs that evaluation and rcsearch, especially applied social science research, are no differenr, in general, evalu-ators do .taim there is a differcnce, bur rht the two are inrcrconnected.Because evaluation requires lhc inlesrigarion ofl\,har is, doing evaluationrcquires doing research. In orherwords, derennininE the valuc, mcrjr, orworth olan cvaluand requires some facrual, descriprile l<no ledge abourfte evaluand .rDd perhaps sinilar eraluands. But, of co\rrse, evaluarionrcquires more than "brute" facrs about evaluands (Coffman, 2003-2004).Evalu^tion also entails thc explicit synthesis of facrs and values in thedelerminadon ol mcrit, r{orth, or lalue. Rrsdarcr, on lhc other hand,invcstigates factrial knoNledge bul may Dor nccessarill invol,rc valuingand rhcrcfore need not (alLhough m^y)include cv^luation.

Attempting to provide a cle^r, unambiguous dcscrjption of rhc clif-ference between resezrrch and elaluntion can too casily rely on dualis-!ic arguments, when indeed most dualisms constrain nther ljhan enablcundcrstanding, Making rhis disrincdoD can bc fUrther obtuscaled byexplaining ev^llration-research differenccs by using oiher dualisms, suchas thc all inportanr lacr-value dichotomy. One long-sranding viel\' on thefact-value djsti!!don, inhclirecl from Flumean skepticism, is that valuescannot be deduced from lacts, thar is, l\'e cannot d€cide what oughr Lo bcthe casejust because wc know what is the case. In tlis version, researchis more about establishing thc facts ancl vatuiug is urorc abou! deciclingwhal ought to be thc case. ,{.nothcr perspectivc cscLews rhe notion Lhalval ejudgmenls are slarenenrs of personal prelcrcnce and rrcats bothfacts and values as intcrcoDnecrccl and consrilutive of orc anothcr, lhalis, facts dont exist scpamte froln values and vice versa (putnam, 2002).This view that facts and values are constitutive still does not necessarilyimply that knowing the valuc ofan er,alland is the same as knowing wbarto do about thc evaluand.In o&er words, loowing the valuc ofsomerhing(a program or product) docsnt seamlessly entail $,har we nced to do irthe futurc (dis/conrinnc funding rhe program or buy a produco, an.tso the knowlcdge abouL an claluand differs togically fron prescriprions.(See Taylor, 1961, for a thorough discrssion of this perspectir€.) Doingresearch requires values and mlue.judgmenb, brt it isn,t aiways the casethat the p.im,rry objecrive ofresearch is valuing, l{,hereas ftis is the sinequa non of evaluarion.

Thc similariries and dif|crences bcrween evaluation and researchrelatc often to lhe purpose of each (hat is, thc anticipatcd ourcome ofdoing research or e\'aluarion), rhe methods of inquiry used, the roles of

I n te tsections o nd D ivergence 43

Ii g

o

F E M I N I S T T H E O R Y , R E S E A R C H , A N D E V A L U A T ] O N

iDquiss, ard how one judges rhe qualiry of evaluarion and rcscarch.More often, evaluarion is focused on rhe pardculat whetber rhat is aprogram, policy, or int€ ention, and research is locused on the gcncrat,whether that is a tbcory, construct, or policy area. Borh evatuarior aldrcsearch may iead to decisions and aclion, alrhough evaluation ofren doesso ar a microlevel r4rile research more often bas an impacr ata macrolevct.Evahition bouows data collcctioD and rn!.lysis shategies from rnanydisciplines induding the social sciences, bur increasingty rhe cqluatorsare developing data collection srnicgies thar empbasize the centraliry ofstakeholdcrs' values and perspectives. Itcscarch, on lhc otber hand, isbound by the methodological tradiLions 1{irhin a particutar social scicncecliscipline (such as anthlopologt psychology, orsociology).In bodr evatu-ation and research, rhe inquirer\ role ranges fron objecrive oursidcr toknowledgeable insidcr and from tlle detachcd to the fulll cng^ged. TheinquircrJ role in both evaluation and rcsearch is dependenr on cpistemo"logical assumptions ancl Lhc particular skills mosr cenlral to a parricul^rparadigm. E1r^luation isjudgcd b)' its sefulness, pnrticularlyirs confribu-tion to improvcmen(, Iearning, ̂ nd change, and dccision making, whileresearch isjudgcd b)' its generation of theorctical knorelcdge.

These atu:ibrircs ot borh rescarch and cv^luarion ar.c i\rther elabo-rated in thc lbllowiDg secrions.

@@

Defining Sociol Science Reseqrch

Soci^l sciencc rcscarch thcory and pr;icricc is cc,mptcx and rcflccrs foun-dational differences in ontology and cpistemology, and so any characrer-ization of rescarch must ackno$'tedge this diversity. Recognizing the dif-lercnces in episterologyis a Soodplcce to s|arr. Research mcthodologicsare connected to rhrec primafy rheories ofknowtedge: objecrivisn, socjaiconstructivism, and subjectivism. Crolry (1998) argues rha! perspecrivcsand research methodologies are aligne.l lejrh rhesc theories of tnowt-edge (an objectivist theory of knoi{4cdge is associaied wirh neoposirivism,which favors experimcntal designs, and a social constructivist theory ofknoNledgc is associated r,ith inrerpretivisn. which favors merhodotogiesliLe symholic interactionisnr. The following secrions acknowteclge andillustrated thesc differences, albeit in a sometimcs oversinplifie.l !!,ay.

Purpose

Research can seNe a numbe. of purposes, but mosr fall into one oflhree categodes: expioration, descriprion, or explanarion. Exploratoryresearch may be done for a number ofreasons including sarisfying one,s

@Bsoa€ .Fem in i$Ev lhaFdsbh indb4 Ij / ] 3 r , 0 { 1 0 s F M i

@

o)

I ntercec I i a ns a nd Dive rg en ce

culiosiry, tcsting dre feasiLjlity oI a nore cxrensi\€ or inrensile srudy, oftesting and developing iechniqucs to be used in subsequenr rcsearchstud,ies. Xxploratory studies are not coDnned to any particular episremologi,cel pdadigm and exploration is uscful for ncopositirist and irtcrprcrivcresearch. Within a neopositivist framerrork, exploralory rcsearch oftenpaves the way for testing the feasibiliq' ofscaling up a srudy or dcFrmin-inga clcar research fo.rs.1'Virhin an inkrp.erilis! plrradigm, exploratoryrcsearch maywcllbe concomjranrwirh thc methodology; slrch is the casc,for cnmple, with grounded rhcory merhodotogy.

Much rcsearch is done ro describe rhe cunent state of some socialdomain. A good example ol descriptivc research is population ccnsussrudics, the purposc of hich is to accurarely descfibc characrcrisrics ofa popularion including l?cc, c|hnicity, ̂ ge, g;errde\ householcl size, andincome. Orhcr examples incl de actuarial studics, most rescarch cloneby demographels, prodrrci usc research, a'nd surveillance stuclies thatdescribe thc incjdence parlerns ofhuman beh^viors and condirions (likesn1oking, obesity, or homelessness). Some inrerprctjivc research methodo!ogies are primadlv descdprive, including, for example, e lnogmphy andsymbolic inteuctionist rescarch studies. Descriptive studies address thcquestions of$'ho, $'har, whcn, and rvherc.

Thc lhjrd purposc of resc^rch is lo provide explanations, the 1{hyquestion. Oltcn expl^narory r€scarch buikls on explora(ory and descrip-lilc rcscarch srlrdics, thus nraking scnsc of clara bv cramining relalion-ships ̂ ud building more absrrnct consLructs and theories to account folthc nature oi- thc social world. lbr cxample, crime starisrics are descrip-(ive, bul whe $'c begin ro explorc rvhy crc arc diffcl.cnccs (tike wtrycrime rates are higher in some cidcs rhan in orhcrs) lhen rhe purposeof the rescarch is explanatory. Again, all paradigms of research includeerplanaiion, ,lrlrongh thc forms of exptanntion differ neopositiljslresearchers look for causai relationships idile inrerpreti\dsl researclre$look for coherenr middle-mngc theoreLical frameworks.

While thcse are commonly understood purposes of resear.h, rhereare research perspcclives rhat focus on changc. The most plactical ofthese is action research, I'hich is meart to soL,e an immectiarc problcm.Action rescarch is a process wherei individuals share a fela need forchange and lr'ork collectively lo define and remedy rhe problem ihroughan acdon research spirai ofplanning (inctuding datn colection), acring.reflecting, and revising rhc ptan (CaII & Kemmis, 1986; Schdn, t983j.Acrion rescarch may look very much tike participatory evaluation, espc,cially }'her pardcipatory evatuarion has a transformative intent anilemphasizes the crcarion ofdcmocratic processes and empowering oppor-tunitics for everyone invoivcd in rhe prcgram or organizaiion (Cousins &Whihore, 1988)- Othe. crirical social science research approaches may

ErrriqFFm n5Le!ahaR€srh ndb 4s

@

@

F E M I N I S T T H E O R Y , R E S E A R C H , A N D E V A L U A T I O N

cnv ion change morc theoreticalll, suchas is rhe case wnh Maxisranaty-sis, dialcctics, and critical erhnogmphy.

,{ distinction bet1"'€en rheorerical and apptied research can also bcmade, a differen.e in cxpcctarion abour \lhcther tire rcsearch providcsimmediate, tangible, and usefui resulrs or more general knowtedge. Thegoal of dleoretical, or what is somerimes called ,,basic research,,: is roexplore the sociai o! physical Norld without any nccessarl cxpccrationthat a usefu], tangible resulr ill be fornd. For exampte, &eorericalresearch might ask quesdons such as "FIo does social ctass reproduceitseln" or "What is the biological h,sis of emotions?', Besic research isoften done to test theories and Dtake broad gen€ralizarions, and mal beseen as the foundation upon which applied research is built.

Appliecl rescmch ma,vbuild on basic. rheorerical rcsearchburirs pur_pose is to sohe pracric^l problems. Apptied research might ask quesrionslil€ "What is the bcst way ro leach childrcn to rc^d?', or,,I{ow can rhisdisease b€ cured?" The currently usecl rcrln .,eviclencc-based pmctice,'is an example of the connection bctween research and the developmentofknowledge for specific ̂ rc^s ofprncdce, sLtch ̂s hezrlth care, counse!ing, tcaching, and so on. Some applied rese^rch lool$ very much likeevalu^r.ion, especizllly if the rcs€arch is orienred !o improvcmcnt of rhchuman condirion, but differs because of the cxpcctation thar the resultsofapplied rescarch witl be useful in solving ̂ problenr across manl sitcs^ncl cont€xb. Action research is a kind ofapplied research rhar is spccifi-cally about seeking solutions within a parricular contex( ancl resenlblessomc lbrms of participatory evaluarion.

Methods

Research lrcrhods are fie ways in which researchers coltect and anatyzeeviclence, nnd lhese vary considerabli,. White urethods arc nor inhcr-cntly cornccred to parricular research paradigms, it is rhe case rharneopositivisr-oric1lftd research is norc likcly to use expc menrat, quasi-experimental, and survey designs, and to employ slatistical analysis. Inter-pretivist research, on rhc otherhand, is morc likety |o emptoy observarionand interviewiDg, and ro employ rhcmatic approaches to clnra aralysis.

TIle methods used in resea'.ch are often ried ro disciptinary tra.titions, and while rhcre is always vadarion Nirhin a specific discipline, parrcms .lo eherge. Ior examplc, one is likeil ro find tha! thosc cducatecl aspsychologists favor quasi-er.pedmenral and experimenrat methodotogiesand statisaical anaiysis ofdata, in contrastwirh those educated as anthro-pologists who favor field work, obseryation, open-etrde{:l intervicwing,and themadc analysis of dara.

o 0

ro

0o

I nte rsections o nd Divetg ence 47

Reseorrher Roles

When researchcrs adopt an objecrivist epislemology. tlleir role is one ofdistant, dlspassionatc, and neutral inquircr. A key ailn for the researcheris ro aloid innuencing the research contexr or parricipants, actions rharare prcsumed to inrroduce bias in answering research quesrions.

Researchcrs who adopt a sociai consuuctivisr ol' subjecrilisr episte-mology mal also adopt dispassionare nnd objective rotes, bur are morelikely to see themschrs as part ofthe research conrext. Depcncling on thcrelationship offte rescnr.che. ro the research conrexr, rhcy may bc seen asiNiders (i.e., nembers ofa social context rh:rt granrs privilegeci acccss toknowledge) or outsiders (i.c., outsi.Icrs ho arc gr.ante.l access ro knowl-edge as an orher) (Merton, 1972). Acucr and Adlcr (198?) pr.ovicle nxrredctail in dcscribing rhrcc interprclivjsr rcscarcher rolcst (I) periphemlmembet who obseNcs but does not pafticipatc in Lhe core activitics ofthc social context; (2) acrive rnember, ilho participares in acrivities ofdregrolrp lvitho!il comDiuing ro rhe valucs wirhin rhc so.ial courexrj aud (3)complete mcnrbet $,bo is alre^dy a member ofrho group or who becomesintcgrated into the social conrext during the coursc of the rcscarch (thisis also referred ln rs "going narile").

Addilion."tily, wirhin this research paradigm, the rcsear.cher is oftenseen as the "instlumenr," rhc prim^ry dara collec(ion stmregy, anclthercforc fiere is significant enphasis placecl on thc imporrarce of Urcresearchcfs rencxivily, a systematic approach to rcflection ll,ithin lhcresearch context. This reflection on personal cxperiencc and alrributesand their intersection \rith the researcb context is sccn as kcy ro clevclop-ing a warranted and clcar descdplion or explllnation ol a social conte;t.For example, Peshkin (1988) descrjbes his ,,rcsearch I,s,' \^,hile studyinga fundamentalist Christian school. $,hich arc (ey ro an ongolng proccssot undershnding how his subjecliviry is ret.rrcd to his jnterpfcrarions ofschool-com1nunily relarions.

Judging Quol i iy

Rescarch is gcnenlly expectecl b make a contdburion ro l(nowlectge, andar the corc rhis is rllc key c terion forjudgin8 its qualiq,. In ^ddirion, theappropdate use oflnethods and rhe rransparcncy of rhe research designcontibule to confidence in rhe contdbution rhe research nakes. Thcindicato.s ofqualiry differ dcpcnding on the rescarch paradignr jn neo-posili!isr research validit),, reliabiliry, replicabiliry, and generalizabilityare key, rehile in inrerprerivist researcti credibility, rransferabititl., depencl-ahilir)', :lnd .onfi.mabilry are t ey. rn gcnerat, r"..".cr,c,"... c"1,..iea t.acknor,ledge and contexrualize their wo* within bodies of retaticl theory

iiiseora ie @PM II Bd"d". FmhdE"d,"aFdchni; 4t @

o

F C I , , 1 , N S T - - F O q " . R E S E A R C F , A N D E V A I U c - I o N

ard rescarclr nakc convincing arguments (ith e clence, and contriburegeneralized knorledge.

While itis generally assumed fiat rescarch is about creadng generel_izable knowlcdge thar is clcarly an orcrsuremcnt and does not rE)resenrthe diversc contdbutions research llray make.Ior example, .an hisroricalanaltsis oI thc causes of fie lrench revolurion, an cftnography of theMinangkabau, or an ccologicat study of rhe Catapagos mal ;t bc con-cLucred in order ro generalize Lo all ,revolutions, all matdarchal culrures,or all selfcontaincd eco-sysrcms" (Nladrison, 2()O?, p. 190).

Defining Evoluotion

Evaluation, like rescarch, is ot a singular or coherent rhcory or praclice.Indccd some coDsidemble cffofi has been made in arrtempting to describerhc \sr ious 2ssurnpt ions Lnd(r lv i ' .g clr l i l r ion ipproichcs r ' rar ̂ ccounLfor di f terenccs in e\-r lurr ion pr ircLice ,se. Atkin. 2004j Shadi5h 60.1,& Lcviton, 1991). Alkin and Christic (2004) assert rhc roor ofall evalua"tion is accountabiliry and systcmatic social inquir.y, but dley distinguishamong cvaluation approaches aud ficories b^sed on rhe primacy givcnLo lhe use ofthe ev^luation proccss and findirrgs, the merhocls employecl,or thc valuing aspect. Each ofthesc three emphases becomes a branch onficir "evaluation rhcory rree."

Dilfercnccs in evaluado approaches also, ro a largc extenr, mirrorthe epistemological differcnces in reseafch desclibccl previously. Somenpproaches ro evaluarion adopr a ncoposirilist tlarncwork and see rhcdiscernnent of value, mcrit, lnd $,orth iD a discovery of $.hat $orks andhow change happens. A good exanple of this approach is realist evalua-tion (P^rdson & Tilley, 1907), which ..focuscs on dcvetoping exptauaLionsof the consequences ol social actions thal con(ribute ro a beLter under-standing of why, whcre and for ri,hom programs $,ork or. fail ro !vork,,(Henry. 2005, p. 359). Realisr evrhretion cloes rhis by i.lenril,jag an.{testing mechanisms rhar produce prograrnmatic ourcomes. These sir onAlkin and Christie's "medods" rree branch.

_ Othef approaches to evaluation adopt a social consrrl.tilist episte,mology, such as participatory evatualion, rrhich involves progran stahe-holders in decisjons about ptan ing, doing, and reporring an craluarion.Participalory evaluarion has been furrher divided into p;actic:J participatory evaluarion or ransformadve participarory evaluation (Cousins& Whitmore, lS98). The former vatues participation in the evatuarionlo increase owne^hip in and usefulness of evatuarion srudies. The tarrer more explicitly incorpolares thc idea that evaiuarion shoutd prc_mote sociai justice and change through the inclusion of and aftention

48

@@

@ ;iiaoa ,,oz,asru j

o

('

I ntercecti on s and D ive ry en ce

to opprcssed ald disclltlanchiscd stakcholden. Thcse approaches sii onIlkin and Christie's "valuing" ree branch.

Other approaches are morc aligned ith a pragmaiism, a focus ondoing enluarion $ar is wcful for decision tuaking, iDprovl:llcul, aldempo\{erment. Perhaps the mosll\,idely cired example is Michaet Palron'sutilization-focused evaluation and more reccnrly devetopmental evatua-rion (Patton, lSgZ 2010). $rhile liLront utilizarion fbcused evatuadonprorided a frams{orl< for identilying why and how evaluarion pl.acticecould be norc useful ar a programnaric lcrel, developmental evaluatbnextcnds those corccrns jn lighr of I'har lat|on perceives as inevirahtecomplexity in social problems, conrexts, and dre examinarion ofinnova.tive solurions that musr be incorporated in|o evaluation praclice. Theseapproaches sit on Alkin and Christiet "usc" tree branch.

Purpose

In part dre definition ofelaluadon defines its purposcr evd&ldtioTi is boththc process and product of derermining Lhc value, merit, or worrh ofanevaluand. While somc rese^rch may be evaluatir,e, ruos! research is not,bu[ lhcjudgmenr component is essenti^l to cvaluation. Dctermining d1evalric ofan evaluand c?rn be done for a numbcr ofreasons: to cletermineif goals arc mct; to determine ourcomcs, bolh anticipared anal unantici.palccl and inrurdcd atrd ulil!€nded; ro jmprove drc evaluand; to makcdecisions abouI an evaluancl (including de€isions about actoptinB, fund-ing, or dism..lnrling lhe evaiuand); |o inforrn public discourse and policyabout nn ev.luand; and b demonstmtc accounrabiiiry.

l'raluntion is usu^lll secn as relevdnr to llnderstanding andjuclging apnrticular evaluand, alrhough |har evaluand may be small and locat (like11 single program to supporr local food production) ro larsc an.l gtobaj(likc a globai policy on environmenr ly sustain^bte agriculture). Unlikeresearch, evaluation is always ab out looldng for soturions lo problems rhatare tangible and meaningful in the immedi,r. tirtlrre

Melhods

Like research, the methods for collecringand xnatyzing evidence in evalu-ation often reflecr the discjplinary tradirion 1{,ithin whjcll evaiuarors areeducatcd. For example, Lum and YanS (2005) inresrigarcd the merhodschoices of criminal jusrice evaluaroru and found that rhesc enluarors(whose primary academic disciptine \l,as generaly criminology) far,orednonej(pedmental merhods, such as forensics.

Evaluation has bI and large dmwn on the social sciences for irs meth_ods and so the ways evidence are collected and analyzed are similar ro

49

(>

o tridi; ia,tpt

@

FEMINIST THEORY, RESEARCH, AND EVALUATION

rhose used by researchers. In parl becausc many evaluato$ werc edlrcated vithin social s€ience rraditioDs, especialy psycholo$ and sociot-ogy, and lo amuchlesser cxrcnt anthropologv. ways ofestablishing empir-ical cvidcrcc trave becn informed by rhese rraditions_ Thus, evatuato$usc experiments, survey merhods, observarions, inrerviews, and so on.Bccause evaluation nccessarily addresses issu€s tike nccds, cosrs, cthicat-ily, fcasibitity, andjustjfiability, e1-aluators tuve also turne.t !o orher dis-ciplines, such as jurisprudence, j ournaiism, ar|s, phitosophy, accouming,and cthics, for ideas on merhods.

In addition, as the discipline of evaluation lralrures dicrc are Drertr-ods that hale been developed spocifically for e\,aluntion, for der€rmining\'alue, meri!, or wol.th. For example, Davies ̂nd Darr (2005) havc devel-oped the "most significant .hang€" melhod. This merhod

irlvohes fie colleciion of signifimn! change (SC) stories cmanaringf.om the field, and lhc sysremalic selecrion of rhc nrost significan! ofrhcsc stories by pancls oI desiguated sLakehotders or sraft Thc desig-nated staff and slakeholders are iniriatiy involvcd by .scarchjng' forprcjecr impact. Once chaDges havc bcen capftrred, various peoplc sirrogerhcr, rcad rhe srcries aloud and hare rcgutar and otien in-cl€pdrdiscussions abou! rhe value ofrhese rcported chang€s. When rhe tcch-nique is impl€menred successfull), wtrotc rcans of pcople begin ron)fl,s (hci arrendon on progfan impxct. (p. e)

Another cxample is Brinkerhoff's ,,success casc merhoal," r\,hich isa r.clatively simple two-s(ep sroryrelling-b.scd procedure lor idenrifyingporential and likely succcssfuljob perfol.mance (Brintrerhoff, q0O5). Thesuccess case method borrows various methods (survcys, key inforrnants,in'dcpth inteNie$'ing) from various disciplines (e.g., sociology, journal-isn) and packages them in a distinct lvay that infbrms thc key fcaturc ofevaluation, rhat is, to derermine the value, medt, or wofih of an evahr-

While collecring evidence in an evaluarion contexr oftcn looks quitesimilar to research, the contextinwhich the methods are used or rhe l{rysthe are packaged somerimes reveats rhe expiicitjndgment comporcnt ofevaluation by the search for success (such as in thc success case method),or the positive (such as in appreciative inqrLiry), or changc Guch as in rhemost significant change merhod).

Evoluolor Roles

Because evaluation theory and pmcrice is not unified into one perspeclive, there are many roles evaluators occupy (Mathison, 2005).

50

@ @

1,ar,3rPM Io

@

I nte5e.tions a nd Divery e nc e

Evaluarors vadously see themselves as objecrive inquirers, tcchnicians.values-conmitted inquirers, change agents, facilitators, collabomrors,and educators. These diffcrenr rolc conceptions place differing emphasison dre iDporlaocc oI crcarine a focus for rhc evatuarion, clara collectionand analtsis, reporring, and slakeholder engagement. Iting and Srevahn(2002) iclcntify three componenrs io all c\,atuator rotes: thc retarion-ships bet$een thc cvaluaror and cvalunrion parrjcipaDts, r1,c ftlarionshjpberween thc cvaluator and fie orgrnizarion, ard *re enluator's conflicrmanagemcnt and fesohrtion strategies,

Stakeholdcr eng,gcment as a componcnr of rhe eviluator's retationship to evaluarion participanrs is unique to evaluadon and jndeed cutsacross all perspectives on how evaluadon should be conducted. Overtime, evaluation pracLicc has come ro be defined by sensjlivjty to nnclengagement ofslakcholder pc$pecrivcs, albeit in differcnt l{ays. Realisrevalualo$, Jor exarnple, may especiallt piry attenlion to collccting valuepositions ofsrakeholders ro inrerprct rhe meaning of data, while partici,palory evatuators may parrnerwith evaluarion participants in concep!ual-izing, doing, ancTor reporting the elalualion. What is notable is th;r thisconrnitment to stakehoidcr engagcnrent is a cr.irical clistinciion berwccnresearch and ev^lualion,

Evaluators arc eirher intei.nal or external, rhar is, rhey $,ork wirhinthe organization or program drey arc evaluating or rhcy do evaluarion @for hire. Whilc there ^re cert^inly organizational rescarch units anddrerelore ilternal researchcrs, the commonplaceness of irrternal evrlu-ation pmcdcc makes this disrinction more salient in deternrining anclaluaror's r:olc (Marhiso[, ?011). The pmcrice of an inlemal or c;ter-nal evaluator may look similar, bur de different roles crcate uniquetcnsions and demands including fie need to juggle high sranctards toreviluation f irh organizationat coDrnirmefts nnd loyatties (Marhiso ,1991).

Judging Quolity

The quality of evaluarion builds on nolions sinrilar to the qulity ofrescarch and there is an expectarion rhar rhe descriptions and explanations about evaluands arc accuiare, but in addition evaluation isj;{lgcdby its utiliq! feasibiliry, and prop ery. Thcse dimensions forju<igilgitrcquality of evaluation are clear in thc Nletaevatuation Checklisr, \irhi.rh isbased on the Progran Evaluarion Srandards (Stuflebeam, 1999).

lvhile evaluadons can make contriburions ro more genemt knowl_ed8e, thc typical expecrltion is that evaluation js usefui wirtrin a cteartydefined conkxl I'irhin 1\'hich the cvaluarion occurred. This context naybe extensive (even narional or inkrnational in scope) or quite tocal.

5 l

@)

@ l'gr,oi1 ]:0,3,FM i

o

@

52 FEMIN S'I THEORT RESEARCH, AND EVATUAT]ON

Whot Does the "Feninist" Adiective Add to Corhos'icoReseorch ond Evoluoiion?

Defining Feminism

l-eminism is defined in many ways, bur common ro mosr definitions isthe idea of challcnging geDdcr inequalir). "Fer nism is: (a) a betief rhalwoncn univenally facc some lorn of oppression or .xptoiration; (b) acommitment to uncoler and undersrand 1\'har causes:rnd susrains oppres,sion, in all its forms and (c) a commirnenr to worL jndil,idually and col-lecdvely in cveryday life to cnd ,ll lorms of opprcssion" (Maguire, 1987,p.79). While feminismbegins $'irh rhe assumprion rharall human bcines,women andmen, arc ofequal worth, afeministperspecrive also adoprs rheassumption thatculrumlly menare rypically morc valued than women. Asa consequcDcc ofthis cultuml valuing ofmen, womcn face myriad formsof oppression lhat nrusrbe named to be overcome,

Putting womcn filsL somctines means a feminist pcrspective is usefulfor undcrstanding issucs ofgender more broadly, including masculinity,$'hich is a coDporrerrr ol feminine identity and experiencc. Ment libcra-!ion, $hether conceiled ofas a liberarjon from p^lriarchy or maldarchy,confionts gendcr inequaliLy by challenging nren's privileged institutional-ized and lived cxpeliences (Messner, 2000).

Fe|ninisl Reseor(h

If fe rinism is about Lrncovering and redrcssing opprcssion and unequaltrealment of wome , thcn |he plrrpose of frminisr. research can bcuuder-stood as the process by which fiis occuls. Like r€search in gen-eral, ltminist rescarch explores, describes, md cxplains the conciitionsof wonrcn\ lives. "By docuilenring women's lives, experiences ̂n.t con,cerns, illuminating gender-based stereotypes and biases, and uncartlringwomen's subjugated knowledge, feminisr r€scarch challcnges rhe basicstructures and idcologies that oppress women', (Brooks & I-Iesse-Biber,2007, p.4). Broolc andHesse-Bibcr (2007) continue by cmphasirngfemi-nist research's action orientaLion, a feature that distinguishcs feministr€sea-rchi "Fcminist reserch goals lbster empo$,erment and emancipa-tior ldr leomen and orher marginalized groups, and femirrisr researchersoften app\' their findings in rhe seryice ofprorroring social chrngc andsocialjustice for rvomen" (p.4). These goals rherefore emphasize both the''invisibiliry and disro.tion offemale expcrience" (Lathet 1988, p. 571).

F eni n i st Re seo rch Method ology

Feminist research generally sreps oursBodide the boundaries of thedebates about quantitarive and qualitative researcb and incleed hisrorica[y

o

r -t1e,o{ 1ltttpM L

@

I nt e Ee ct o n s a n d Diverye.ce

feminisl rcsearch has been ancl can be done wiihin any paradigm. Inother No.ds, any researcb merhodolog/ can be pressed into seNice todocumcnt gender inequaliries and to provide direcrion for emancipa-tioD fro!, tltose opprcssn'e inequalides. Three perspectives dominare thediscussions aboul feminisr research methodoiogtr feminist empiricism,standpoint feminism, and fel11inist posrmodcrnlsm.

!-emidst cmpiricish adoprs a rcalist orrulogy, an objecrivisr epij,remology, and emplo)'s rraditionrl social science rescarch methods. Unlikcempiricism, though, feuinist empiricism is criri€al of rhe pracrice ofsci-en.e, il not its foundntions, and looks !o both srudy $,oren's issucs andto obviare gcnder biirs in research techniques including biased instru-mentation, male-dominatcd sampling, and asking research quesrions rharemphasize $'omen's expcriences. Thcre is mr).h criricism ofthe conserva-tism offeminist eDpiricism including rhe constrainrs rhat empiricism hason ne\{ and ̂ lternnte I{ays ofrcasoning ilnd that scientific standards arethcrNelves a producr ofpatriarchy (HunaUeby. 2012).

Fcminisr sr^ndpoinr rescarch aligns closcly wirh rhc definitions offcminism ̂ nd women's political lnovemenls. Srandpoint approaches purr{omen's experiences at thc ccnter of the research and dcclare that.won1enarc bcst posirioned to undcrstand rhose expc ences. This merhoclologi-cal ̂ pproach favors asocial constructivist epistcmology, blurs the rolcs ofrescarchcr and rcsearched, and emphasizes thc importlnce ofresearcherrcllexivity. This appronch places greatest cmphasis on what women do,andtheconcrctcexperieDcesof rheir l ives; forcxample,Jaggar(1997)con^nects Nomen's cveryday nlrrturing activiries with their skiu in expressingand rcrding emotions. But ttminisr standpoinl rcsearch nlso elr)phasizesthe oppressiveness of vomen's cxperienccs and rhc uniquc perspecdvcthis gives 1\'omen in undcrstanding the social wortd, which, in rurn, pro-vides a lrcars lor rmderslanding dcsirable sociat changc. A challeng; forfeminist standpoint research is thar rherc zre mulriple and di\,el.se telni,ninc standpoints, srandpoints thar dedle frorn rhe intersection of gen,der with ra.c and//or class. for exaqrplc. Norcrhetess, fcminisr srandpoinircsearch merhodologics remain a dominant feminist resear.ctr approach.

Relalcd more to crirical theory a{d a rejection of foundarionatism,feDinist postmodernist resealch is focuscd more or contrasrs Grar univ€rsal undersiandings and critiques unitary notions ofvoman and gendcr (Fraser & Nicholson, 1SS0). In part, posrmodern feminisr researchaltempts to deal \a'irh the diffcrences auong reomen, rcjecting thc univc.-" a l i . r i d f r o r g r d n d r i , r d . i \ " o , \ r o - n . F x p c , i F n . e .

Alfiough tllere are clear and importanr differenccs among feministrescarch merhodologies, rhere are selerit .ommon etederts. Staht.yaDdWise (1990) include rhe relarionship betlvecn researcher and rcsearcned;lhe impo.tancc of the rcsearchcr's autobiography; the critical role of

53

@(D

@

FEM NIST THEORY, RFSEARCH, AND EVALUAT]ON

reflerdvity; crperience as rhe tocus of lhe research; and complex ques_tions ofpower. One would obviously add an atenrion ro gender as a cen,tral construct. Feminisl rcsearch canbc rhought ofas a normarir€ frame-

F em i ni sl Re sea rc h Melhods

Discussions olfeninisr research ofren eschew a clnim that rhcrc are femi-nisr research methods and claim thal any and all researcL mc&ods may beuscd in service ol:ihe gender-Iocused and problem-focused naruie of fenlrnisr research. And indeed, lcminisr rescar.chers adapt rescarch methocls,often through €ollabor^tion with research padicjpants, to inclucle g€nderissues, toacknowledge and cxplore subj cctive expcr'icnces, anclto empowerresearch p^rticipants. Inrervie$'ing, erhnographic fieid work, snrleys, andaction research, for example, are adapted ro meet rhc goals of ferninisrresearch (I-Iesse-Biber, 2012).Ior many lcminisr rcsearchers, tl)e adoptionofmethods rrhat have Lhe potenrial for cngagirlg and enrpor€].ing w;men^rc appealing. Onc example ofsuch mclhods arc visual- and imagc-baseclapproaches as exemplified, for example, in rhe Academy ANarclwinningdocumenkrry filmrorrl inta BraLhth: Cal&uta's Red lig,t, K,d.J and rhe visuaistorytelling o{ Photovoicc (W^ng, 1999; trVanij & Burris, 1994).

The feminist r€searchcr's adapration oI rcscarch methocls is siinilarto fte evaluator's adaprion ol research mcthods, molcling rhem ro a par_ticular intcrest in lived gendercd expericnces in lhc formcr inshnc; orm"king valucjudBmenrs in the lartcr instMcc.

Femini5l Evdludlion

Likc all evaluation approachcs, fer!inisrevatuario! is functamenratly ^boutascertaining (he value, rne r, or r{orlh of an cvaluand, bur wirh parricularattention "to gendcr issues, rhc needs of wornen. and the pronrorion of.hange" (Seig:r!. 2005, p. 155). !-emilist evrtuation is nor tl]. clalurtionof women's programs, but rarhcr, as Seigart (2005) exptains, ir is a per-spective d]at casrs a crirical and gender-focused eyc on a programs. ihisperspective reflecb |he foundational femif ist frameworl rhrt informs altfeminist evallrations. Wlile addressing and redrcssing women,s oppres_sion, some feminist evalllation attends also ro other social condirions ofoppression, sUchas race, class, ethniciry. serual orientation, anal2hleness_what lvlertens (2005) ca]ls "a transformative feminist approach.,,

Most ofren the core clemenrs of feminisr evaluation are rhose sum-narized in Sielbeck-Bowen, Brisolara, Seigart Tischler, and $ihirmore(2002, pp. 3 4) l

@@

€)

{D

o('

I n te Eections a n d Dive gen ce 55

* Feminist craluadon has as a central focus fte gencLel. inequidesdlar lead to social lqjustice.

E Disc mination or inequaliry based on gcnder is systemic and

d Evaluation is a polirical activityj the contexis in l{lich evaluarionoperaks are politicized; ancl |he personat expcrjences, pelspcc-tives, and.haracrerisri.s cvaluarors bring ro evatuarions (and ithwhich wc interact lead to a pafricrLlar potidcal stancc.

B Knolvledgeis apowcrful resource $arservcs an explicit orimplicilpurpose.

d (nowlcdge should be a resource of and for rhc pcople who crc-ate, hold, and sharc it. Conscquently, lhe elaluation or researchprocess can lead to significan! negarivc or posirive eflects on lrhcpeople involved in thc evalualion/research. Kno\'lcdge andvaluesare culturAlly, socially, and temporally continEenr. Knowleclge isalso fikcrcd through rhe knower.

t Thcre are muliplc ways ofknowing; s(nre ways are pri!'ilegcd over

Icminist evalltarion shares many of thc attributcs of participatory,empo$:erment, and democratic cvaluation approaches, tbosc cvalua"lion approaches thar rcst on rhe "valuing" and ,,use,'limbs ofAlkin andCh stie's (2004) evnlriadon theory trce. Indccct translbr-uurive ev^lua-tion(Mertens, 2009), empowermentevaluarion (ferierman, 2001), cdticaltheory cvaluation (Frceman, 2010), ff^nsformalivc participatory evalua-tion (Cor8ins & WhirDlorc, 1998), ard dctibcradve democmric e\,nluarion(House & Howe, 1999) are all approachcs rhar prolide a loundxrion forfeminis! evaluadon. Coupled wirh a feminisl perpcctive, including con-cerns about wo'nen's opprcssion and cnEtrciparion, any of rLcse elatua_tion approaches beco1ncs feminisr cvaluation.

The Procticol lmportonce of Dist nquishinq Evoluotionfrom Reseorch-

I have argued that rhcre is a distincrion berween evatuation and rcsearch,bul have also rded ro iilusrlare rhar dralving a tine bcrween the ${o isneithcr easy nor srnightforward. Bccause of fie diversity ofbod evatuation and research perspecrivcs, ir is casy ro find evatuation and researchthat are difficult to disdnguish, and ro iind evrtuation and research thatlook dramarically differen!- This is no less rhe case when one is ctescribingfeminist evaluadon or leminisr research.

@ ri,a*M I

@

€){0

56 F E M I N I S T T H E O R Y , R E S E A R C H , A N D E V A L U A T i O N

.\ kcy similariry is thar both eraluador and research use many oflhesame stEtegies for collecringand analyzing evidence, lhatis, they share ,rcommon ancestry r{th rcgard to me|hods. $rhile some methods are nowb€ing developed within rhe disciplinc of cvatuatioD, cspeciaUy Derlodsthat fbcus on perceived change or success, nonetheless l,har courts asevidence remains quire similar for elaluarion ancl research_

Akeydifferen.e is in tl]epripose or cxpected ourcome. Xvajuationis:boutvaluing, that is, definirion.rt. Rescarch, which js the pursuir olrheo-rctical desc ptions or explanations, necessarily involves values (becausefacts and values are inexrricable) but need not ancl otten does nor invokevaluing. Another key dilfererlcc is thni cvahradon is abour the parricuiar,whedler that is a program, policy, or intervention, and rescarch is abourthe geneml, {'herher rhat is a Lhcory, conslrnct, or policy area. Evallrxtionisjudged by its usefulness, parliclrlarly its contriburion ro improvemenr,learningand change, and decision making, while research isjudgeclbyirsgeneration o{ theorerical knowledge.

!-valuators and researchen allke 1\,ant rhcir worl( to matter, $,he!herwilhin a parricular conlext or in a more absrract theorctical way. Evalu_ators, ho$'cver, shouldcr a necessary burdcn of cnsuring that evaluadonas a pracdcc is relevani, usefulr and responsive to clients, p:trticulat con_lcxts, and social problems. It is thh burdcn that nlost esFecjally behoovcsus to be clear ̂ bou!when we are doing evnluation and when 1\rc are doing

RE FE RE NCES

Adlcr', P., & Adler, P. (1981). Mehbership roht in Jietd rareardl,. N€\\,blrry park, CAISagc.

Alkin, M. (20 04). -[,alratto n roa^: Tiacing theartL', ieus akd inflwk .e,..lha$MdOaks, CA: Sage.

Alkin, M., & Chrislie, C. A. (2004). Ar eyalualion iheory ree. In tv{. Alt<in (Ed.),Etaltation roats: Tiacing ths.ti|s' tiztut and irlrcnzs (pp. 12_ e6). Tl,ousardOaks, CA: Sage.

Brinkerhoff, R. O. (2005). The succe$ case method: A straregic evatua.ionappro.tch io inc.easing rhe ],alue and effect of traint.lg. Atuances in D:uetoping Hunan Rsources, 7(\, a6-rc1.

Brooks, A., & Hesse Bjber S. N. (200?). An inritarion to feminis[esearch. rn S.N. Hc$e,Biber & P Lealy (Eds.), Fehinin ftearch pracLiee: A priner..lhon_sand Oats, CA: Sage.

Carr, W., & IGmnis, S. (1986j. Beeoning oiticat: EdL tia . hnaukdge antt actianrrreafcl. Lewes. UK: Falmei

CoffrraD, J. (2003-2004, $!nrer). I,Iichaet S.rnFf on the differ€nces betweeneraluarion aDd social science research. ?r, Exatuatian Ea.hange, glt).

@ l

o

@

B;"aaFmr"HE"ahaF6rc'lldb' e

tntetsectians ond Diveryence 57

Rerrjerrd JlIy 30, 2a72, tton utu.gv.haruad.er1ultftf/cu.t/i su.2t/uptL

cousins,J. B., & Whilmore, E. (1998). Franing pafricipatory evaluarjon. In E.lvhnnore (Ed.), Underslandirg and prircriclng pariciparory e'aluaiion.Netu Dia.tioB fat Euluation, 19981.80),5,23.

Crotty, M. (1998). ?)l,rzzddtiow a.f sa.iat ftsearch. Newbury part, CA: Sage.Davies, R-, &Darr,J. (2005). The n.n tiCniJicant change technqre: A !1ti.te to its use_

RctricvcdJnl/ 30, ?012, fran uuu. dntt..tr.uty'tlud/MsCcuid.e.f dlI.ctterl11an, D. elAI). Founttatians o.l ehfal'entent clatuation. Thousnnd Oaks,

CA: Sagc.Fraser, N., & Nicholson, L. T. (1990). Sociat criricisn wirhour phitosophyr An

encouncr benfeen leminism and posrmodefrisn). In L.J. Nic|ohon ([c1.),I'eninis /po'tnademitn lpp. 19-3s). Ncw yorkr Rourtedge.

Freeman, M. (Ed.). (2010). Crjrical social rheory a c1 evaharion practice. Na-drDi.ttians .far Eratuation, 201A 027), r-98.

I-Ienry, G. (2005). Realisi cvaluarion, tn S. Madrison (Ed.), r.nrycroledia o.f elarua.Ito, (pp. 359-367). rhousnnd Oaks, CA:Snge.

Hesse-Biber, S. N. (Ed.). (201t)). Hnhnbaoh of?minkt aedtetL: Th.ary akd lrlxis.Thousand Oaks, CAr Sage.

House, L. R., & I-Iowe, K. (199s) . values in elutuation and s.ti,xt rcsearch. Thousa\.LelOaks, CA: Sage.

Hundleby, c. B. (2012). Feninist empiricjsm. In S. N. Hcsse-Bibcr ([d.), sar1,.baoh aJ.feinini't rcseak)h: 'l'hea|)

antt, pra\i' (,pp. 28-4s). Thousand Oaks, cA:sage. @

Jaggar, A. AlI (1997). Love and LDowlcdg., Emorion iD fcnrhisr cpisrcd,oto8y. rriS. Ikmp &J. Squircs (Ids.), F Dir$ns (pp. 188-l9S). oxford, Uri oxforclUniversiry hcss.

King,J. A., & SLevahn, L. (2002). Three framewort<s fo! conricicring evaluarorrolc.ID K. f. Ryan &T. A. Schlvandr (Eds.), ZqloltB el)ataxtar rate ann, iden.rr, (pp. 1-16). Charlorre, NC: Informarion Age pubtishers.

Larhcr P. (1983). [eminisr persp€crivc on empowcring resea.ch rn.ihodc,logjes.Workn" Stud,ies Inturnadanat.aorun, ll(O), b09 b6t.

Lum, C., & Yang, S. (2005). Why do evatuation researchers in crinc andjusricechoose non-experjmcnral mer\\ads? lbturnat a.f Lrpcrinentat crituinatag, 1,191-213.

Maguire, P (1987).,otrgfd i.ipatary ftsear.h: Afeninist opproach. 1\n11je6t Unrvcrsiry of Nlassachuseirs Pres.

Nlathhon, S. (1091). Role conflicrs fof inrernat evaluarors. rn S. N,larhison (Ed.),Authoriry in nnernal evalL\ rja[. haluatiak and progratu ptannine, U(2),173-179.

lvlalhison, S. (2005). Evaluarof ioles. In S. llathison (E(L), Lhryckpedia ol efutuatto, (pp. 146-147)- Thousand Oaks, C,\: Sagc.

It{athison, S. (2007). Whar is rhe diltercnce belwccn evaluarion D.l researctr?And why do wc cafe? In N. L. Srrirli & p. BraD.ton (E i,s.), Fukdanentdt isuesm ewtuation lpp. 183-196). New york: cu ford press.

Mathison, S. (:011). InrerDal evaluarion: Histo[.a]l), speat<ing. rn B. Votkov &

i/r"ror4 ]%-in ]

o

@

58 FEMIN]ST THEORY, RESEARCH, AND EVALUATION

M. DaroD (Eds.), L,|elMl elatua.ioD in rhe Zlsr ccnLnry. Ne1, Directians fottualuatiaL, 2A 11\132), lZ -24.

Mertens, D. M. (2005). Feminism.In S. Mathison (Ed.), Enqctohedid aJ exaruatian(p- 154). Thousand Oaks, CAr Sage.

Mertens, D. M. (2009). liansIamaltue resedrch akd e&/kton. Nerv yorkr cuitfor.l

Nlcr ton,R.K.( r9?2) . Ins idersandours ide.Af t , / tcanJatmata. fSqio lo l i l . ,?8\1) ,9-47.

Messn€r, N{. (2000). Paliti.s of na'.uLinitixs: Men in nalenents. Oxford, UIt:AltaMirrPress.

P^ttoD, M. Q, (19!7). Lttitizatianlauk.t stutuaLian: Thc n.u dntury t. (ztd c.\.).Thousand Oals, CAr Sagc.

Palton, M. q eln). Devetapnmtal eulratian: Aplt ikg canptexit, canupl, toenLanu inno&tion akd uy. New York: cuilford Prcss.

Pawson, R., & Illcy, N. (1997). Re(littic cualuation. London: Sagc.Peshkjn, A. (1988). ln scnrch of subjcctivityr Orc\ aw:n. lUutaLionat Researcher,

1711), 17-2r.Puha'n, H. (2002). Th. coudpv o.f ttu.fdat/uut:u' ttictetrn) utut ather essar,. C^\n-

bridgc, MAr Har!ard Unil€fsity Pfess.schdn, D. A. (1983). ?i, /rl eetil)e lmcLiti|ner Itau lrofessionak thinh i.n aetion. Ney

Yo*: BrLsic Books.Seigart, D. (2005). Fcminisr €valuarion. In S. Marhisob \lj.d.), nncrdal,edia o! loaht

arion (pp. 154*15?). Thousand Orks, c,4.:Sagc.Shadish, W R., Cook, T. D., & Leviron, L. C. (1901). Foundatia al lr\gtun euaht

axtakr Thearies aJ practxa. 't housand ol:!ks, CArSage.Sielbcck-Bow€D, Ii., BrisolarA, S., Seigar!, D.,1isclilcr, C., &Whi|trrore, E. (?002).

[xploring feminisr evaluarionr The ground from which $e rirc. Nea Dtrd.tiou lar L1t/ttt ation, 2n02 (96J, rI -B.

Slanloy, L., & Wise, S. (1990). Merhod, merhodology anct epislemology jn fcm!nistresearch processes.In L. Slanley (Ed.), ,rr4'rizd I'a,rr' (pp.20-60). Lon-don: Roulledgc.

SNnebea'n, D. (1999). Metaeutualion .tdAlsr. Rerr.ieved July 30, 2012, fromtuu tu. u tu i ch. e,Iu/ e M t. ty' drc hile _c h e.kti n s / p r a g ah _n e t atu at - I 0 l, t in t e.,V

Taylo! P. (1961). Nomdrnld dtr.orrr?. Englcvood Cliffs, NJ:prendcc Hall.Wog, C. C. (r999). Pho.ovoiccr A participarof), acdoD resea, cL suaregy applie.L

Io womcn s heahh.,rozlz t aJ \ranen\ LI.aIth, 8\2), t8r-t92.Wang, C., & Burris,lvl. A. (1094). Empowemlenr rhrough phoronoretla: porrreirs

ol participaiion. lJealr,tr ilz cariar and B.h atiar. 2j(2), t,t I-186.

o

. o ? { ; ;o