content analysis: measurement and validity
DESCRIPTION
Content Analysis: Measurement and Validity. Kimberly A. Neuendorf, Ph.D. Cleveland State University Fall 2011. Measurement. “Assignment of numerals to objects or events according to rules” (Stevens, 1951). Operationalization (Measurement). - PowerPoint PPT PresentationTRANSCRIPT
Content Analysis:Measurement and
ValidityKimberly A. Neuendorf, Ph.D.
Cleveland State UniversityFall 2011
Measurement “Assignment of numerals to objects or
events according to rules” (Stevens, 1951)
Operationalization (Measurement)
Categories that are exhaustive and mutually exclusive
High and appropriate level of measurement Ratio Interval Ordinal Nominal
Evaluating Measures:Comparing Reliability, Accuracy (lack of Bias), and Precision
Targets…also see p. 114 of book. See also:
Carmines, E. G., & Zeller, R. A. (1979). Reliability and validity assessment. Newbury Park, CA: Sage Publications.
High Reliability
High Accuracy
High Precision
High Reliability
Low Accuracy
High Precision
Low Reliability
Moderate Accuracy
High Precision
Reliability?
High Accuracy
Low Precision
Validity
Internal Validity Conceptual definition =matches= Operationalization The standard of having a “good” measure—”Are we hitting
the target we want?”
External Validity Generalizability
Sample Ecological validity Replicability
Validity
“Face” Validity Criterion Validity
Relate to established standard or behavior Concurrent Predictive
e.g., Oliver & Kalyanaraman (2002) study of sexual and violent content in trailers and films. . . Criterion is the MPAA rating
Validity
Content Validity
Construct = ???
Validity
Content Validity
Validity
Content Validity
Validity
Content Validity
High internal consistency reliability. . .
Validity
Construct Validity Related to other measures (constructs) in ways
consistent with theory Convergent—relationships as expected Discriminant—lack of relationships, as expected
Construct Validity Smith volume (1992) on “thematic content
analysis”—psychographic applications—has examples Trust-mistrust coding system for TAT stories
construct validated against: Other thematic coding systems (e.g., Affiliation; Intimacy) Demographics (e.g., Mothers’ feeding habits during infancy
(more or less principled), moderated by gender & age) Self-reported behaviors (e.g., Reported positive relationships
with others; Divorce, family fights in last year; Drug use) Personality measures (e.g., Impulsivity; Locus of Control)
Validity Assessment & CATA
Short et al. (2010) provides a model for validity and CATA, using tests of a new dictionary for “entrepreneurial orientation” Content Validity
Deductive Inductive
External Validity Reliability Dimensionality (a type of Construct Validity--Discriminant) Predictive Validity (seems to really be Construct Validity—
Convergent???)
Validity Assessment & CATA
A Slight Adaptation of Short et al. (2010), converging with Carmines and Zeller (1979): Face Validity Content Validity
Deductive Inductive
External Validity Construct Validity--Dimensionality (Discriminant Validity) Construct Validity--Predictive (Convergent Validity) Criterion Validity (Concurrent or Predictive) [Reliability—separate, but without it, validity is also
compromised]
Apply to the iCarly Case? (Disrespectful Behaviors) A Slight Adaptation of Short et al. (2010), converging with
Carmines and Zeller (1979): Face Validity Content Validity
Deductive Inductive
External Validity Construct Validity--Dimensionality (Discriminant Validity) Construct Validity--Predictive (Convergent Validity) Criterion Validity (Concurrent or Predictive) [Reliability—separate, but without it, validity is also
compromised]
pause