cmns 260: empirical communication research methods 4-measurement (part 1 of 2 slideshows) neuman...
Post on 23-Jan-2016
222 views
TRANSCRIPT
![Page 1: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/1.jpg)
CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows)
CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows)
Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8)
•systematic observation •can be replicated•requires:•construct (concept)•operational measure/instrument/tool for making empirical observations
![Page 2: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/2.jpg)
Today’s Lecture
• Review core notions: concepts, operational measures, empirical research, language of variables, hypothesis testing, errors in explanation, etc.
• Reliability & Validity: relationship, types• Levels of Measurement• Scales & Indices (if time– material for this in
another slideshow)
![Page 3: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/3.jpg)
Babbie (1995: 101)
Recall:The
Research Process
![Page 4: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/4.jpg)
““Dimensions” of ResearchDimensions” of Research
Neuman (2000: 37)
Purpose ofPurpose of
StudyStudy
Intended Use Intended Use of Studyof Study
Treatment of Time Treatment of Time in Studyin Study
Space Unit of Space Unit of
Analysis Analysis
ExploratoryExploratory
DescriptiveDescriptive
ExplanatoryExplanatory
BasicBasic
AppliedApplied
-Action-Action
-Impact-Impact
-Evaluation-Evaluation
Cross-sectionalCross-sectional
LongitudinalLongitudinal
-Panel-Panel
-Time series-Time series
-Cohort analysis -Cohort analysis
-Case Study-Case Study
--Trend studyTrend study
-dependent -individual-dependent -individual
-independent -family-independent -family
-household-household
-artifact-artifact
(media, (media,
technology)technology)
![Page 5: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/5.jpg)
The Research Wheel
ChooseTopic
FocusResearchQuestion
DesignStudy
CollectData
AnalyzeData
InterpretData
InformOthers
The“Research Wheel”
Source: Neuman (1995: 12)
Steps in theresearch process
![Page 6: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/6.jpg)
Developing research topics
![Page 7: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/7.jpg)
Concepts– Symbol (image, words, practices…)– definition
• must be shared to have social meaning• concepts with more than one possible value or attribute sometimes called variables
![Page 8: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/8.jpg)
Concept Clusters• Examples:– Peer Group– Role Model– Broadcast Media– Ethnic Identity– Cultural trauma– Collective memory– Political economy
![Page 9: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/9.jpg)
Measurement systematic observation can be replicated (by someone
else) Measures include:
Concepts (constructs), theories
measurement instrument/tools
Must recognize concept in observations (measures)
??(# of library holdings as a measure of quality of university?) MacLeans Magazine survey
results, 2000.
![Page 10: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/10.jpg)
From Concept to Measure
Neuman (2000: 162)
![Page 11: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/11.jpg)
Variables• Must have more than one possible “value” or
“attribute”• Types:– dependent variable (effect)– independent variable (cause)– intervening variable– control variable
![Page 12: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/12.jpg)
Causal Relationships
• proposed for testing (NOT like assumptions)• 5 characteristics of causal hypothesis – at least 2 variables– cause-effect relationship (cause must come before
effect)– can be expressed as prediction– logically linked to research question+ a theory– falsifiable
![Page 13: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/13.jpg)
Errors in Explanation
![Page 14: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/14.jpg)
Propositions
• logical statement about (causal) relationship between two variables
• i.e. “Increased television watching leads to more shared family time and better communication between children & their parents”
![Page 15: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/15.jpg)
Types of Hypotheses (note: plural form of Hypothesis)
• Null hypothesis– predicts there is no relationship
• Direct relationship (positive correlation)– more time spent studying leads to higher grades
• Indirect relationship (negative correlation)• More time spent playing video games leads to lower
grades
![Page 16: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/16.jpg)
Hypothesis Testing
![Page 17: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/17.jpg)
Possible outcomes in Testing Hypotheses (using empirical research)
• support (confirm) hypothesis• reject (not support) hypothesis• partially confirm or fail to
support• avoid use of PROVE
![Page 18: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/18.jpg)
Causal diagrams
X Y
X Y
Direct relationship (positive correlation)
Indirect relationship (negative correlation)
![Page 19: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/19.jpg)
Spurious Association example
![Page 20: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/20.jpg)
Causal Diagram
R= Racism against non-whitesD= Discrimination against non-whitesS=Intelligence Test Scores
SDR
![Page 21: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/21.jpg)
Good & Bad Research Questions
![Page 22: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/22.jpg)
Abstract to ConcreteConcept to Measure
![Page 23: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/23.jpg)
Reliability & Validity
Reliability dependability is the indicator consistent? same result every time?
Validity measurement validity - how well the conceptual and
operational definitions mesh with each other does measurement tool measure what we think ?
![Page 24: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/24.jpg)
Types of Validity
![Page 25: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/25.jpg)
Content Validity measure represents all the aspects of conceptual definition of
construct. how adequately a measure covers behavior representative of
the universe of behavior the test was designed to sample.
Love
![Page 26: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/26.jpg)
Face & Expert Panel Validity judgement by group or scientific community that indicator
measures the construct (conceptual def.) Examples:
Socio-economic status (education, income & ?) Digital Divide (differences in access to computers, internet, broadband?...)
Construct
MeasureScientific Community
![Page 27: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/27.jpg)
Criterion Validity The validity of an indicator is verified by comparing
it with another measure of the same construct in which a researcher has confidence.
Predictive : ex. Comparison of Aptitude test & performance measures
concurrent validity: ex. Comparison of new measure with established one
![Page 28: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/28.jpg)
Construct Validity A type of measurement validity that uses multiple
indicators– the construct is a combination of measures of the same variable
convergent : positive correlation with related measures
discriminate: negative correlation with measures of different variables
![Page 29: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/29.jpg)
Other Dimensions of Validity
Internal Validity no error of logic internal to research design
External Validity results can be generalized
Statistical validity correct statistical methodology chosen ? assumptions fully met
![Page 30: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/30.jpg)
Types of Reliability
stability over time
representative across different subgroups of a population
equivalence multiple indicators
intercoder type of equivalence reliability
![Page 31: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/31.jpg)
Improving Reliability
clearly conceptualize constructs increase level of measurement use pretests, pilot studies use multiple indicators :
DependentVariable Measure
IndependentVariable Measure
Empirical
Association?
a2 a3a1 b1 b2
A B
Specific IndicatorsSpecific Indicators
![Page 32: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/32.jpg)
Relationship between Measurement Reliability & Validity
reliability necessary for validity but does not guarantee it “necessary but not sufficient” measure can be reliable but invalid (ex. not measuring
what you think you are measuring)
![Page 33: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/33.jpg)
Quantitative & Qualitative“Trustworthiness”
![Page 34: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/34.jpg)
Creating Measures
Measures must have response categories that are: mutually exclusive
possible observations must only fit in one category
exhaustive categories must cover all possibilities
composite measures must also be: uni-dimensional
![Page 35: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/35.jpg)
Levels of Measurement
Levels of Measurement
•Categories (or attributes) must be exhaustive & mutually exclusive
Relations between levels --can collapse from higher into lower, not vice versa
![Page 36: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/36.jpg)
Nominal Measurement different categories (names, labels, images) not ranked• attributes are mutually exclusive and
exhaustive.
Babbie (1995: 137) Examples: What media do you use for finding out about news?
TelevisionNewspapers
RadioMagazinesInternetOther
![Page 37: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/37.jpg)
Ordinal Measurement different categories (mutually exclusive, exhaustive) rank-ordered• attributes indicate relatively more or less of that variable.• distance between the attributes of a variable is imprecise
Example: “How important are newspapers as your news source?”
![Page 38: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/38.jpg)
Interval Measurement
different categories ranked in order can also tell amount of difference between
categories
Babbie (1995: 137)
![Page 39: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/39.jpg)
Ratio Measurement different categories ranked in order amount of difference between categories also possible to state proportion (have a
true zero)Example: “What was your income in dollars last year?”
![Page 40: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/40.jpg)
Examples
![Page 41: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/41.jpg)
Continuous & Discrete Variables Continuous variables:
can have an infinite number of values interval and ratio levels of
measurement
Discrete variables: distinct categories nominal and ordinal levels of
measurement
![Page 42: CMNS 260: Empirical Communication Research Methods 4-Measurement (Part 1 of 2 slideshows) Neuman & Robson Chapters 5, 6, & (some of) 12 (p. 332-8) systematic](https://reader036.vdocuments.mx/reader036/viewer/2022062410/56649d495503460f94a25dcb/html5/thumbnails/42.jpg)
Composite Measures (continued in second slide series)
• Composite measures are instruments that use several questions to measure a given variable (construct).
• A composite measure can be either unidimensional or multidimensional.
• Ex. Indices (plural form of index) and scales