design into practice these lectures tie into terre blanche chapter 4 and 5 now you have a design –...
TRANSCRIPT
![Page 1: Design Into Practice These lectures tie into Terre Blanche chapter 4 and 5 Now you have a design – how do you run the study? Many practical issues involved](https://reader035.vdocuments.mx/reader035/viewer/2022062313/56649d0b5503460f949de95d/html5/thumbnails/1.jpg)
Design Into Practice
• These lectures tie into Terre Blanche chapter 4 and 5
• Now you have a design – how do you run the study?• Many practical issues involved in converting a
design into a study to run
![Page 2: Design Into Practice These lectures tie into Terre Blanche chapter 4 and 5 Now you have a design – how do you run the study? Many practical issues involved](https://reader035.vdocuments.mx/reader035/viewer/2022062313/56649d0b5503460f949de95d/html5/thumbnails/2.jpg)
Conceptualisation
• We want to speak of abstract things• “Intelligence”, “ability to cope”, “life
satisfaction”
• We cannot research these things until we know exactly what they are• Conceptualisation os the process of defining
terms before research
• Once a thing has been conceptualised it is a “Construct”
![Page 3: Design Into Practice These lectures tie into Terre Blanche chapter 4 and 5 Now you have a design – how do you run the study? Many practical issues involved](https://reader035.vdocuments.mx/reader035/viewer/2022062313/56649d0b5503460f949de95d/html5/thumbnails/3.jpg)
Making conceptual definitions
• Begin with the lay understanding of the definition• This will be understood by the subjects
• Then consult the experts (literature)• Can be confusing, contradictory
• Create a preliminary definition
• “Test it” hypothetically• Use thought experiments
![Page 4: Design Into Practice These lectures tie into Terre Blanche chapter 4 and 5 Now you have a design – how do you run the study? Many practical issues involved](https://reader035.vdocuments.mx/reader035/viewer/2022062313/56649d0b5503460f949de95d/html5/thumbnails/4.jpg)
The danger of reification
• You must not try to make constructs out of things that don’t exist – reification
• Careful grounding of the construct in extablished theory will prevent this
• Eg. Is Homophobia a construct? (is it not just prejudice)?
• Using reified constructs leads to empty, disconnected research
![Page 5: Design Into Practice These lectures tie into Terre Blanche chapter 4 and 5 Now you have a design – how do you run the study? Many practical issues involved](https://reader035.vdocuments.mx/reader035/viewer/2022062313/56649d0b5503460f949de95d/html5/thumbnails/5.jpg)
Operationalising variables
• Your design specifies the variables
• How do you measure the variables?• How do you put a number to “intelligence”?
• How do you put a number to “capacity to cope”?
• We need to convert abstract variables into things which we can measure in the real world - operationalisation
![Page 6: Design Into Practice These lectures tie into Terre Blanche chapter 4 and 5 Now you have a design – how do you run the study? Many practical issues involved](https://reader035.vdocuments.mx/reader035/viewer/2022062313/56649d0b5503460f949de95d/html5/thumbnails/6.jpg)
Operationalisation (2)• Turn your variable into a directly
measureable thing• Eg: How would you operationalise “success at
university?”
• Often there are developed scales available
• If you operationalise badly, you end up not studying what you want
• Eg. Operationalising “success in career” by looking at the paycheque only
![Page 7: Design Into Practice These lectures tie into Terre Blanche chapter 4 and 5 Now you have a design – how do you run the study? Many practical issues involved](https://reader035.vdocuments.mx/reader035/viewer/2022062313/56649d0b5503460f949de95d/html5/thumbnails/7.jpg)
Measuring variables• The operationalisation implies what to
measure variable – how do you do it?• If at all possible, use an established scale• If no scale exists, construct one
• Scales must be valid and reliable• The more of each of these properties, the better
the scale• Validity and reliability need to be sorted out
before you run your study
![Page 8: Design Into Practice These lectures tie into Terre Blanche chapter 4 and 5 Now you have a design – how do you run the study? Many practical issues involved](https://reader035.vdocuments.mx/reader035/viewer/2022062313/56649d0b5503460f949de95d/html5/thumbnails/8.jpg)
Reliability in scales
• Reliability: stability of a measure over time• If I measure you now and then in half an hour,
do I get the same reading?
• Max reliability depends on the construct• Some construct are unstable (eg. heart rate)
• Low reliability implies that other variables (“noise variables”) are being measured also• Speaks of the “accuracy” of the scale
![Page 9: Design Into Practice These lectures tie into Terre Blanche chapter 4 and 5 Now you have a design – how do you run the study? Many practical issues involved](https://reader035.vdocuments.mx/reader035/viewer/2022062313/56649d0b5503460f949de95d/html5/thumbnails/9.jpg)
Ensuring reliability
• Reliability suffers when subjects have to interpret• Everyone’s interpretation is slightly different
• Objective scales are always more reliable• Allow little interpretation
• Using a fixed response format helps• Eg. Multiple choice, Likert type• Researcher does not have to interpret what the
subject meant
![Page 10: Design Into Practice These lectures tie into Terre Blanche chapter 4 and 5 Now you have a design – how do you run the study? Many practical issues involved](https://reader035.vdocuments.mx/reader035/viewer/2022062313/56649d0b5503460f949de95d/html5/thumbnails/10.jpg)
Examples of response types
• Open ended item:Briefly describe your most frightening
experience
• MCQ:The most frightening for me isA) DogsB) SnakesC) SpidersD) None of the above
![Page 11: Design Into Practice These lectures tie into Terre Blanche chapter 4 and 5 Now you have a design – how do you run the study? Many practical issues involved](https://reader035.vdocuments.mx/reader035/viewer/2022062313/56649d0b5503460f949de95d/html5/thumbnails/11.jpg)
More examples
• Likert type:
Circle the one which best describes your experience.
I find dogs to be1 2 3 4 5 6 7
Not frightening at all
Terrifyingly frightening
![Page 12: Design Into Practice These lectures tie into Terre Blanche chapter 4 and 5 Now you have a design – how do you run the study? Many practical issues involved](https://reader035.vdocuments.mx/reader035/viewer/2022062313/56649d0b5503460f949de95d/html5/thumbnails/12.jpg)
Validity in scales
• Validity: the degree to which a scales measures what it is supposed to
• Validity is subdivided into many types• We will look at 2 most important
• Criterion Realted Validity
• Construct Validity
![Page 13: Design Into Practice These lectures tie into Terre Blanche chapter 4 and 5 Now you have a design – how do you run the study? Many practical issues involved](https://reader035.vdocuments.mx/reader035/viewer/2022062313/56649d0b5503460f949de95d/html5/thumbnails/13.jpg)
Criterion Related Validity
• The degree to which this scales matches other established scales
• By comparing to a scale known to be valid, you can be sure yours is valid
• Why make a new scale if one already exists?• Maybe yours is quicker to do
• Maybe the established is not for group testing
![Page 14: Design Into Practice These lectures tie into Terre Blanche chapter 4 and 5 Now you have a design – how do you run the study? Many practical issues involved](https://reader035.vdocuments.mx/reader035/viewer/2022062313/56649d0b5503460f949de95d/html5/thumbnails/14.jpg)
How to check for criterion related validity
• This is done through a set of studies• Run a sub-study in which you give the subjects
your scale and the established one
• Run a correlation between the two scales
• If the correlation is statistically significant, your scale compares well to the established one.
• It is better to run several of these validity studies rather than just one.
![Page 15: Design Into Practice These lectures tie into Terre Blanche chapter 4 and 5 Now you have a design – how do you run the study? Many practical issues involved](https://reader035.vdocuments.mx/reader035/viewer/2022062313/56649d0b5503460f949de95d/html5/thumbnails/15.jpg)
Example: intelligence test• An accepted test is the WAIS-R
• Very long to run (3 hours)
• You need something quicker (20 minutes), create the QIQ
• Create a test, select a group of subjects• Make them take the WAIS-R and then the QIQ
• Compare the results (correlation)
• If they correlate well, your test is measureing intelligence
![Page 16: Design Into Practice These lectures tie into Terre Blanche chapter 4 and 5 Now you have a design – how do you run the study? Many practical issues involved](https://reader035.vdocuments.mx/reader035/viewer/2022062313/56649d0b5503460f949de95d/html5/thumbnails/16.jpg)
Construct Validity
• Construct validity: Does the scale actually measure the construct?• Eg: measuring cranial circumference to
measure intelligence
• Closely tied into the theory of the construct
• Most difficult to achieve, most important
• Measures lacking in construct validity are almost useless
![Page 17: Design Into Practice These lectures tie into Terre Blanche chapter 4 and 5 Now you have a design – how do you run the study? Many practical issues involved](https://reader035.vdocuments.mx/reader035/viewer/2022062313/56649d0b5503460f949de95d/html5/thumbnails/17.jpg)
How to check for construct validity
Think abou it for a minute:How can you show that a scale truly measures what it claims to?
How would you show that your depression scale has construct validity?
Hint: Compare it not to scales of the same thing, but to similar and dissimilar things
![Page 18: Design Into Practice These lectures tie into Terre Blanche chapter 4 and 5 Now you have a design – how do you run the study? Many practical issues involved](https://reader035.vdocuments.mx/reader035/viewer/2022062313/56649d0b5503460f949de95d/html5/thumbnails/18.jpg)
The strategy
• Similar procedure to criterion related validity:• Before your actual study, run a set of sub-
studies to check your measure
• You will need 2 sets of studies• Concurrent construct Validity
• Discriminant construct validity
![Page 19: Design Into Practice These lectures tie into Terre Blanche chapter 4 and 5 Now you have a design – how do you run the study? Many practical issues involved](https://reader035.vdocuments.mx/reader035/viewer/2022062313/56649d0b5503460f949de95d/html5/thumbnails/19.jpg)
Quick aside: direction of correlations
• Correlation: the degree of relationship between two variables, A and B
• Positive correlation: when A has a high value, B has a high value. When A has a low value, B has a low value
• Negative correlation: when A has a high value, B has a low value. When A has a low value, B has a high value
![Page 20: Design Into Practice These lectures tie into Terre Blanche chapter 4 and 5 Now you have a design – how do you run the study? Many practical issues involved](https://reader035.vdocuments.mx/reader035/viewer/2022062313/56649d0b5503460f949de95d/html5/thumbnails/20.jpg)
Correlations example
• Positive correlation: the relations ship between amount smoked and probability of heart disease
• Negative correlation: the relationship between amount of daily exercise and probability of heart disease
• No correlation: the relationship between whether you drink tea or coffee and the probability of heart disease
![Page 21: Design Into Practice These lectures tie into Terre Blanche chapter 4 and 5 Now you have a design – how do you run the study? Many practical issues involved](https://reader035.vdocuments.mx/reader035/viewer/2022062313/56649d0b5503460f949de95d/html5/thumbnails/21.jpg)
Concurrent validity
• Show that your scale relates positively to related concepts• People who do are depressed will have many
sad thoughts (mood conguency effect)
• Establish concurrent validity against several other constructs
![Page 22: Design Into Practice These lectures tie into Terre Blanche chapter 4 and 5 Now you have a design – how do you run the study? Many practical issues involved](https://reader035.vdocuments.mx/reader035/viewer/2022062313/56649d0b5503460f949de95d/html5/thumbnails/22.jpg)
Discriminant validity
• Show that your scale relates negatively to opposite concepts• People who are depressed will have very low
energy
• Establish discriminant validity against several other constructs
![Page 23: Design Into Practice These lectures tie into Terre Blanche chapter 4 and 5 Now you have a design – how do you run the study? Many practical issues involved](https://reader035.vdocuments.mx/reader035/viewer/2022062313/56649d0b5503460f949de95d/html5/thumbnails/23.jpg)
Ensuring construct validity
• Best way: be an expert on that construct• Theory should tell you what things to include
• BUT: only if the theory is well-established!
• Second way: consult the experts/literature closely
• Stay with the uncontroversial aspects of that construct
![Page 24: Design Into Practice These lectures tie into Terre Blanche chapter 4 and 5 Now you have a design – how do you run the study? Many practical issues involved](https://reader035.vdocuments.mx/reader035/viewer/2022062313/56649d0b5503460f949de95d/html5/thumbnails/24.jpg)
Validity & reliability summary
• Aim: make sure that your variables are correctly operationalised
• Reliability: scale is stable over time/place
• Validity: scale is truly measuring the construct not something else
![Page 25: Design Into Practice These lectures tie into Terre Blanche chapter 4 and 5 Now you have a design – how do you run the study? Many practical issues involved](https://reader035.vdocuments.mx/reader035/viewer/2022062313/56649d0b5503460f949de95d/html5/thumbnails/25.jpg)
Validity & Reliability summary (2)
• Ensuring reliability: require verly little interpretation / increase objectivity
• Ensuring validity: base the measure closely on current understanding of construct
• Measuring validity: positive correlations with related scales, negative correlations with opposite scales