qualitative and quantitave evaluation...qualitative vs. quantitative evaluation -5 qualitative...
TRANSCRIPT
QUALITATIVEANDQUANTITAVEEVALUATION
UNMEvaluationLabSummerInstituteSoniaBettez
[email protected] /evallab.unm.eduSummer2018
Qualitativevs.QuantitativeEvaluation-1
Qualitative Quantitative
Focuseson“why”and“how” Focuseson“what”and“howmany”
Qualitativevs.QuantitativeEvaluation-2
Qualitative Quantitative
Prioritizesunderstandingpeople’slivedexperience
Prioritizesquantities
Qualitativevs.QuantitativeEvaluation-3
Qualitative Quantitative
Tellsstorywithwords (narrative)todescribeincidents,actionsandprocesses
Tellsstorywithnumbers (numerical)
Qualitativevs.QuantitativeEvaluation-4
Qualitative Quantitative
Shouldbeaccurateandcredible Shouldbeaccurateandrepresentative
Qualitativevs.QuantitativeEvaluation-5
Qualitative Quantitative
Includesinterviews,focusgroups,observations,ethnographies,letterordiarywriting
Surveys,questionnaires,randomizedcontrolledstudies,semi-experimentaldesigns,useofsecondarydata.
OurCollaborativeApproach
• Evaluationteamandorganizationmeet,learnaboutprograms,clientele,resources,etc.• Asateam,decideprojectscopeandunderstandprogramtheoryandgoals(involvesdevelopingalogicmodelandrubricstomeasuresuccess)• Theteamdiscussesanddecideswhatoutcomestheywanttomeasure:whatquestionstheywanttoansweraboutaprogram(goalsoftheevaluation)• Thequestionschosendefinethemethodsandthewaystocollectdata• Theteamdecidesonprotocolfordatacollectionthataddressesthegoals• StaffparticipatesindataanalysiswithassistancefromtheLearningCommunity
QualitativeApproachSteps
IncollaborationtheUNMteamandtheorganization’steam:• Developevaluationgoals/purposes• Developevaluationquestions• Reviewacademicandquantitativedataavailable(nationalandlocal)• Discussandagreeon:bestdatacollectionmethod(s)(focusgroup,letter-writing,diaries,interviews,etc.• Developevaluationplan• Recruitevaluationsubjects• Implementevaluationactivities• Analyzedata• Writeevaluationreport,reviewitwithorganizationandpresentit
QuantitativeApproach
Possibleactivities:• Review:intakeanddischargeforms,dataalreadycollectedandorganized(inExcel,etc.)
• Analyzeandreviewofdataalreadycollected• Reviewuseofdatacollectionsystem(s)• Discussandagreeon:bestdatacollection:surveys,pre- posttests,etc.• Writeevaluationplan• Collectdata• Analyzedata,writeresults• Writeevaluationreport,reviewitwithorganizationandpresentit
MixedMethods
• Useofqualitativeandquantitativemethodsimprovescredibilityofresults(triangulation).• Oftentimesinterviewsandfocusgroupsleadtodevelopmentofsurveys.
Evaluation^2
• TheEvalLabschedulesanddedicates“MissionTime”throughouttheyear• TheEvalLabevaluatesitselfeveryyear(Sometimesinternallyandsometimesexternally,orboth)• Evaluationcoverstheclass,thelearningcommunity,theannualworkshop,theSummerInstituteandtheexperiencesoftheorganizations(interviewswithDirectors)• TheLabusesevaluationresultsforprogramplanningandimprovement• AnAdvisoryBoardofrepresentativesfromaffiliateorganizationsmeetstwice/year
THANKYOU!
• Questions?Comments?