evaluating qualitative research
DESCRIPTION
INFO 272. Qualitative Research Methods 16 April 2009. Evaluating Qualitative Research. Typical Reactions. is not generalizable / is “anecdotal” The sample is too small to say anything / is not a random sample / not representative What is the hypothesis you are testing? - PowerPoint PPT PresentationTRANSCRIPT
INFO 272. Qualitative Research Methods16 April 2009
Typical Reactions is not generalizable / is “anecdotal” The sample is too small to say anything / is
not a random sample / not representative What is the hypothesis you are testing? Great stories, but can you show me some
data that supports your claims? is subjective, the researcher’s presence in
the setting biases the data lacks rigor, procedure is unsystematic
Becker – epistemology of qual research
Quantitative Tradition Qualitative Tradition
Reliability – reproducing the findings through the same procedures, same findings from multiple observers
Accuracy – based on close observation not remote indicators
Validity – the degree to which one measured the phenomenon one claims to be dealing with
Precision – close to the thing discussed
Breadth – knowledge of a broad range of matters that touch on the topic
Criteria for Quant Research
Functional Equivalence Criteria for evaluating quantitative research is
not directly applicable to qualitative research Can we draw out some abstract, general
standards and then respecify for qualitative research
Kvale on epistemologyAbandoning a correspondence theory of truthDefensible (rather than absolute) knowledge
claims requiring argumentation
Functional EquivalenceQuantitative Tradition Qualitative TraditionReliability of measures (c)
Confidence (c)
Relevance (r)
Triangulation and reflexivity (c)
Internal validity (c) Transparency and procedural clarity (c)
Sample size (c) Corpus construction (c, r)
Representative sampling (r)
Thick description (c, r)
External validity (r) Local surprise (r)
Validity of measures (r) Communicative validation (r)
Triangulation and Reflexivity (c) In situ verification
process i.e. interviews about
Internet use supplemented by observation
Transparency (c)
Corpus Construction (c, r) Maximizing the diversity of unknown
representations and mapping those representations
Representativeness and ‘external validity’ is a matter of argumentation
Thick Description (c, r) ‘high-fidelity’ reportage: verbatim quotes –
demonstrating the provenance of a claim Footnotes and sources But also, do you get a whole picture of the
social world, its elements, and how they are interlinked? Especially the meaning of the social phenomenon.
Local Surprise (r) Surprise in relation to a common-sense view Surprise in relation to theoretical expectation Solely confirming evidence (just as totally
consistent evidence) should raise suspicion
Communicative Validation (r) Gaining feedback from research participants
(and others?) Remember interviewing technique of
‘interpreting’ on the fly to get confirmation from interviewees
The Future of Evaluation Websites and digital archives that make
qualitative data accessible to the public
Summary Make your methods visible Make your data (ideally) available Continual verification in situ (as part of your
iterative process) Closeness to the social phenomenon and
openness to surprises, the counter-intuitive
Re-read Becker on the “epistemology of qualitative research” for further suggestions