qualitative and quantitave evaluation...qualitative vs. quantitative evaluation -5 qualitative...

QUALITATIVEANDQUANTITAVEEVALUATION

UNMEvaluationLabSummerInstituteSoniaBettez

spbettez@unm.edu /evallab.unm.eduSummer2018

Qualitativevs.QuantitativeEvaluation-1

Qualitative Quantitative

Focuseson“why”and“how” Focuseson“what”and“howmany”

Qualitativevs.QuantitativeEvaluation-2

Qualitative Quantitative

Prioritizesunderstandingpeople’slivedexperience

Prioritizesquantities

Qualitativevs.QuantitativeEvaluation-3

Qualitative Quantitative

Tellsstorywithwords (narrative)todescribeincidents,actionsandprocesses

Tellsstorywithnumbers (numerical)

Qualitativevs.QuantitativeEvaluation-4

Qualitative Quantitative

Shouldbeaccurateandcredible Shouldbeaccurateandrepresentative

Qualitativevs.QuantitativeEvaluation-5

Qualitative Quantitative

Includesinterviews,focusgroups,observations,ethnographies,letterordiarywriting

Surveys,questionnaires,randomizedcontrolledstudies,semi-experimentaldesigns,useofsecondarydata.

OurCollaborativeApproach

• Evaluationteamandorganizationmeet,learnaboutprograms,clientele,resources,etc.• Asateam,decideprojectscopeandunderstandprogramtheoryandgoals(involvesdevelopingalogicmodelandrubricstomeasuresuccess)• Theteamdiscussesanddecideswhatoutcomestheywanttomeasure:whatquestionstheywanttoansweraboutaprogram(goalsoftheevaluation)• Thequestionschosendefinethemethodsandthewaystocollectdata• Theteamdecidesonprotocolfordatacollectionthataddressesthegoals• StaffparticipatesindataanalysiswithassistancefromtheLearningCommunity

QualitativeApproachSteps

IncollaborationtheUNMteamandtheorganization’steam:• Developevaluationgoals/purposes• Developevaluationquestions• Reviewacademicandquantitativedataavailable(nationalandlocal)• Discussandagreeon:bestdatacollectionmethod(s)(focusgroup,letter-writing,diaries,interviews,etc.• Developevaluationplan• Recruitevaluationsubjects• Implementevaluationactivities• Analyzedata• Writeevaluationreport,reviewitwithorganizationandpresentit

QuantitativeApproach

Possibleactivities:• Review:intakeanddischargeforms,dataalreadycollectedandorganized(inExcel,etc.)

• Analyzeandreviewofdataalreadycollected• Reviewuseofdatacollectionsystem(s)• Discussandagreeon:bestdatacollection:surveys,pre- posttests,etc.• Writeevaluationplan• Collectdata• Analyzedata,writeresults• Writeevaluationreport,reviewitwithorganizationandpresentit

MixedMethods

• Useofqualitativeandquantitativemethodsimprovescredibilityofresults(triangulation).• Oftentimesinterviewsandfocusgroupsleadtodevelopmentofsurveys.

Evaluation^2

• TheEvalLabschedulesanddedicates“MissionTime”throughouttheyear• TheEvalLabevaluatesitselfeveryyear(Sometimesinternallyandsometimesexternally,orboth)• Evaluationcoverstheclass,thelearningcommunity,theannualworkshop,theSummerInstituteandtheexperiencesoftheorganizations(interviewswithDirectors)• TheLabusesevaluationresultsforprogramplanningandimprovement• AnAdvisoryBoardofrepresentativesfromaffiliateorganizationsmeetstwice/year

THANKYOU!

• Questions?Comments?