understanding the variability of your data: dependent variable
TRANSCRIPT
Understanding the Variability of Your Data:
Dependent Variable
Understanding the Variability of Your Data:
Dependent Variable
• Two "Sources" of Variability
Understanding the Variability of Your Data:
Dependent Variable
• Two "Sources" of Variability
– Independent (Predictor/Explanatory) Variable(s)
Understanding the Variability of Your Data:
Dependent Variable
• Two "Sources" of Variability
– Independent (Predictor/Explanatory) Variable(s)
– Extraneous Variables
Understanding the Variability of Your Data:
Dependent Variable
• Two Types of Variability
Understanding the Variability of Your Data:
Dependent Variable
• Two Types of Variability
– Unsystematic
Understanding the Variability of Your Data:
Dependent Variable
• Two Types of Variability
– Unsystematic
– Systematic
Understanding the Variability of Your Data:
Dependent Variable
• Three "labels" for the variability
Understanding the Variability of Your Data:
Dependent Variable
• Three "labels" for the variability
– Error Variability - unsystematic due to extraneous variables
Understanding the Variability of Your Data:
Dependent Variable
• Three "labels" for the variability
– Error Variability - unsystematic due to extraneous variables
• Within conditions variability
Understanding the Variability of Your Data:
Dependent Variable
• Three "labels" for the variability
– Error Variability - unsystematic due to extraneous variables
• Within conditions variability• Individuals in same condition affected differently
Understanding the Variability of Your Data:
Dependent Variable
• Three "labels" for the variability
– Error Variability - unsystematic due to extraneous variables
• Within conditions variability• Individuals in same condition affected differently• Affects standard deviation, not mean, in long term
Understanding the Variability of Your Data:
Dependent Variable
• Three "labels" for the variability
– Error Variability - unsystematic due to extraneous variables
Common sources
individual differences
procedural variations
measurement error
Understanding the Variability of Your Data:
Dependent Variable
• Three "labels" for the variability
– Primary Variability – systematic due to independent variable
Understanding the Variability of Your Data:
Dependent Variable
• Three "labels" for the variability
– Primary Variability – systematic due to independent variable
• Between conditions variability
Understanding the Variability of Your Data:
Dependent Variable
• Three "labels" for the variability
– Primary Variability – systematic due to independent variable
• Between conditions variability• Individuals in same condition affected similarly
Understanding the Variability of Your Data:
Dependent Variable
• Three "labels" for the variability
– Primary Variability – systematic due to independent variable
• Between conditions variability• Individuals in same condition affected similarly• Individuals in different conditions affected differently
Understanding the Variability of Your Data:
Dependent Variable
• Three "labels" for the variability
– Primary Variability – systematic due to independent variable
• Between conditions variability• Individuals in same condition affected similarly• Individuals in different conditions affected differently• Affects mean, not standard deviation, in long term
Understanding the Variability of Your Data:
Dependent Variable
• Three "labels" for the variability
– Secondary Variability – systematic due to extraneous variable
Understanding the Variability of Your Data:
Dependent Variable
• Three "labels" for the variability
– Secondary Variability – systematic due to extraneous variable
• Between conditions variability
Understanding the Variability of Your Data:
Dependent Variable
• Three "labels" for the variability
– Secondary Variability – systematic due to extraneous variable
• Between conditions variability• Individuals in same condition affected similarly
Understanding the Variability of Your Data:
Dependent Variable
• Three "labels" for the variability
– Secondary Variability – systematic due to extraneous variable
• Between conditions variability• Individuals in same condition affected similarly• Individuals in different conditions affected differently
Understanding the Variability of Your Data:
Dependent Variable
• Three "labels" for the variability
– Secondary Variability – systematic due to extraneous variable
• Between conditions variability• Individuals in same condition affected similarly• Individuals in different conditions affected differently• Affects mean, not standard deviation, in long term
Understanding the Variability of Your Data:
Dependent Variable
• Roles played in the Research Situation
– Error Variability• A nuisance – the ‘noise’ in the research situation
Understanding the Variability of Your Data:
Dependent Variable
• Three "labels" for the variability
– Error Variability• A nuisance – the ‘noise’ in the research situation
– Primary Variability• The focus – the potentially meaningful effect
Understanding the Variability of Your Data:
Dependent Variable
• Three "labels" for the variability
– Error Variability• A nuisance – the ‘noise’ in the research situation
– Primary Variability• The focus – the potentially meaningful effect
– Secondary Variability• The ‘evil’ – confounds the results
Example
• Two sections of the same course
Example
• Two sections of the same course
• Individual’s score as combination of ‘sources’
Statistical decision-making
• The logic behind inferential statistics
• Deciding if there is ‘systematic variability’– primary vs. secondary
• What do the data tell us?
• What decisions should we make?
Statistical decision-making
• A Research Example
– Research Hypothesis
– IF students chant the “Statistician’s Mantra” before taking their Methods exam THEN they will earn higher scores on the exam.
Statistical decision-making
• A Research Example
Your Class (M = 80, SD = 15, n = 25) compared to a known population Mean (M = 70) for a
standardized exam
Statistical decision-making
• A Research Example
Can estimate the Sampling Distribution
See if Population mean ‘fits’
Cause effect relationship not clear
Statistical decision-making
• A Research Example using experimental approach
– Research Hypothesis– IF students chant the “Statistician’s Mantra”
(vs. not chanting) before taking their Methods exam THEN they will earn higher scores on the exam.
Statistical decision-making
• Procedure
– Randomly divide class into two groups
• Chanters – are taught the “Statistician’s Chant” and chant together for 5 minutes before the exam
Statistical decision-making
• Procedure
– Randomly divide class into two groups
• Chanters – are taught the “Statistician’s Chant” and chant together for 5 minutes before the exam
• Non-chanters – sing Kumbaya together for 5 minutes before the exam
Statistical decision-making
• Results
– Compute exam scores for all students and organize by ‘condition’ (levels of IV).
Show ‘changing’ distribution
Statistical decision-making
• Results
– Compute exam scores for all students and organize by ‘condition’ (levels of IV).
– Compare Mean Exam Scores for two Conditions
Statistical decision-making
• Results
– Compute exam scores for all students and organize by ‘condition’ (levels of IV).
– Compare Means Exam Scores for two Conditions
– What will you find?
Statistical decision-making
• Research Hypotheses generally imprecise
– Predictions are not specific
– So “testing” the Research Hypothesis, using the available data, not reasonable
Statistical decision-making
• Null Hypothesis – a precise alternative
– Identifies outcome expected when NO systematic variability is present
Statistical decision-making
• Null Hypothesis – a precise alternative
– Identifies outcome expected when NO systematic variability is present
– But still must decide how close to the predicted outcome you must be to ‘believe’ in the Null Hypothesis
Statistical decision-making
• The Null Hypothesis Sampling Distribution
Statistical decision-making
• The Null Hypothesis Sampling Distribution
– All possible outcomes when the Null Hypothesis is true
• (when there is no ‘systematic’ variability present in the data)
Statistical decision-making
• The Null Hypothesis Sampling Distribution
– All possible outcomes when the Null Hypothesis is true
– Finding all the possible outcomes?
Statistical decision-making
• The Null Hypothesis Sampling Distribution
– All possible outcomes when the Null Hypothesis is true
– Finding all the possible outcomes?
– Seeing where your results fit into the Null Hypothesis Sampling Distribution
Statistical decision-making
• Deciding what to conclude based on the ‘fit’
Statistical decision-making
• Deciding what to conclude based on the ‘fit’
“True” State of the World
• Ho True Ho False• Reject Ho Error Correct Rejection
Decision• Not Reject Ho Correct Error• Nonreject
Statistical decision-making
• Deciding what to conclude based on the ‘fit’
“True” State of the World
• Ho True Ho False• Reject Ho Type 1 (p) Correct Rejection
Decision (power = 1 – Type 2)• Not Reject Ho Correct Type 2• Nonrejection• Deciding what confidence you want to have that you
have not made any errors
Statistical decision-making
• Trade-offs between Types of Errors– I believe I can fly?
Statistical decision-making
• Trade-offs between Types of Errors
• Factors affecting Type 2 Errors (Power)– “Real” systematic variability (size of effect)– Choice of Type 1 probability– Precision of estimates (sample size)
Statistical decision-making
• Trade-offs between Types of Errors
• Factors affecting Type 2 Errors (Power)– “Real” systematic variability (size of effect)
• Assume .5 * SD, a moderate size effect is good
– Choice of Type 1 probability• Use traditional .05
– Precision of estimates (sample size)• Sample of 50 (2 groups of 25)
Statistical decision-making
• Factors affecting Type 2 Errors (Power)
– Type 2 error probability = .59– Power = .41
Statistical decision-making
• Each ‘Decision” has an associated ‘error’
• Can only make Type 1 if “Reject”
• Can only make Type 2 if “Not Reject”
Statistical decision-making
Interpreting “Significant” Statistical Results
• Having decided to “reject” the Null Hypothesis you can:– State probability of Type 1 error– State confidence interval for population value– State percent of variability in DV ‘accounted for’
Statistical decision-making
Interpreting “Significant” Statistical Results
• For Chant vs. No Chant example– State probability of Type 1 error
• .05
– State confidence interval for population value• 95% CI is approximately +2 * SE• Point estimate of 10 + 8 (Real difference between 2
and 18)
– State percent of variability in DV ‘accounted for’• Eta2 = .20, or 20%
Statistical decision-making
Interpreting “Significant” Statistical Results
• Statistical Significance vs. Practical Significance
• How unlikely is the event in these circumstance– versus
• How much of an effect was there
Statistical decision-making
Interpreting “Non-significant” Statistical Results
Having decide you cannot reject the Ho
State the estimated ‘power’ of your design
