using modern missing data analyses for effective inference about hunters’ satisfaction towards ofw...
TRANSCRIPT
Using Modern Missing Data Analyses for effective inference about Hunters’ satisfaction towards OFW Program
Muhammad Imran Khan
Motivation of Study• Hunting & fishing are part of Nebraska's
heritage
• NGPC is interested in improving hunter/angler recruitment & retention (NGPC,2008)
• Data collected in 2013 to know about hunters’ motivations & satisfactions towards OFW lands
• Purpose of this study is to compare estimates using appropriate imputation methods
2
Missing Data• Missingness in Surveys (Groves et al., 2004)– Noncoverage– Unit Nonresponse– Item Nonresponse– Partial Nonresponse (Brick & Kalton,1996)– Data Entry Error (Anne & Andrea,2014)
• Missing data Mechanism(Buuren, 2012)– Missing Completely At Random (MCAR)– Missing At Random (MAR)– Missing Not At Random (MNAR)
3
How much missing data is “problematic”• Researchers assign some limits:– > 5% (Schafer,1999)– >10% (Benntt,2001)– >20% (Peng et al., 2006)– (Widaman,2006) specified the following scaleo 1%-2% (Negligible)o 5%-10% ( Minor)o 10%-25% (Moderate)o 25%-50% (High)o >50% (Excessive)
• Important problems of missingness (Bell & Fairclough,2013)– decrease in precision– Increase bias in parameter estimation
4
NGPC & UNL conducted survey• Sampling frame: hunters who purchased hunting
license for hunting in 2012 in NE– The survey contained three parts:o Where, & what hunt; Environment Impacto Motivations(Relatedness, Competence, Autonomy)o Socio-demographic factors
• About collected data– Total questions = 42 (used 19 Qus. for analysis)– Sample size = 8181– Completely filled =1555 (19%)– Unit nonresponse = 627 (8%)– Item nonresponse = 5999 (73%)o Varies from 1 to 8 missingness per respondent in all 19 Qus.
5
81%
Determining Type of Missing Data• Test for MCAR (Little, 1988)– Little’s Test of MCAR (Omnibus test of all specified
variables) o If test is not significant, then data can be assumed MCARo If test is significant, then Then, data may be MAR or MNAR
• For given data test is sig. So data are MAR– 3256.783 with .
• Table shows number & percent missing
6
M. Satisf. Rel_1 Rel_2 Comp. Auto. H_Days “Harvest” Educ. Income Age
Ns. 5171 332 332 345 397 5096 0 1088 1465 1263
% 0.685 0.04 0.04 0.046 0.053 0.675 0 0.144 0.194 0.167
Data used for analysis• 13 Questions for motivation based on SDT
5 Questions on relatedness transformed to 2 factors
7
Data used for analysis• 13 Questions for motivation based on SDT
4 Qus. on competence & autonomy transformed each to 1 factor
8
Satisfaction=Rel_1+Rel_2+Comp+Auto+ Educ+Age+Income+H_Days+Harvest
Model used for the analysis9
Variable Description of the variable [measured on 7 point Likert scale]
Satisfaction How satisfied were you with your experience on private lands enrolled in the Open Fields and Waters (OFW)?
Releatedness_1 I enjoy mentoring other huntersReleatedness_2 I go hunting primarily to spend time with others & people I care aboutCompetence Overall, Hunting makes me feel competent in other areas of my lifeAutonomy Hunting helps me to feel independent; self-sufficient and more control
in lifeEducation Highest level of education that you have complete (<HS;HS;S.C;C;≥G)
Age Age (Approximately in years)
Income Total annual income for your household before taxes (8 diff. levels)
Hunting_Days Visiting OFW sites allowed me to increase total days I spent hunting
“Harvest” If you hunted in 2012 on a OFW site, did you harvest? (Yes/No)
• Deletion or non-imputing methods:o List-wise Deletion (Pigott, 2001)o Pair-wise Deletion (Bennett, 2001)
• Nonstochastic or ad-hoc methods:o Mean Imputation (Graham,2003)o Regression Imputation (Qin et.al., 2007)
• Stochastic or Established methods:o Stochastic Regression (Todd et al., 2013)o Multiple Imputation(MI) (John, et al., 2007)o Full Information Maximum Likelihood(FIML)o Expectation Maximization (EM)(Yiran & Chao-Ying, 2013)
Methods for Handling Missing Data 10
Mean Imputation 11
Comparing Results 12
Fitted Model
List-wise Deletion Mean Imputation
p-value p-valueIntercept 0.415 0.205 0.043 0.381 0.062 0.000Releatedness_1 -0.023 0.040 0.565 -0.005 0.010 0.614Releatedness_2 0.038 0.045 0.401 0.017 0.011 0.120Competence 0.147 0.079 0.062 0.023 0.019 0.227Autonomy
0.049 0.075 0.5140.009 0.018 0.619
Education-0.045 0.039 0.241
-0.011 0.010 0.296
Age -0.001 0.003 0.682 0.000 0.001 0.563Income 0.003 0.022 0.903 0.002 0.006 0.754Hunting_Days 0.135 0.017 0.000 0.162 0.007 0.000“Harvest” 0.569 0.077 0.000 0.364 0.028 0.000
5999 cases or rows are Deleted m=1, maxit=1
Multiple Imputation 13
Comparing Results 14
Fitted Model
List-wise Deletion Mean Imputation Multiple Imputation
p-value p-value p-valueIntercept 0.415 0.205 0.043 0.381 0.062 0.000 0.316 0.183 0.093Releatedness_1 -0.023 0.040 0.565 -0.005 0.010 0.614 -0.019 0.037 0.605Releatedness_2 0.038 0.045 0.401 0.017 0.011 0.120 0.048 0.037 0.205Competence 0.147 0.079 0.062 0.023 0.019 0.227 0.097 0.077 0.219Autonomy 0.049 0.075 0.514 0.009 0.018 0.619 0.017 0.061 0.787Education -0.045 0.039 0.241 -0.011 0.010 0.296 -0.032 0.027 0.245Age -0.001 0.003 0.682 0.000 0.001 0.563 -0.001 0.002 0.731Income 0.003 0.022 0.903 0.002 0.006 0.754 0.007 0.022 0.761Hunting_Days 0.135 0.017 0.000 0.162 0.007 0.000 0.152 0.013 0.000“Harvest” 0.569 0.077 0.000 0.364 0.028 0.000 0.575 0.060 0.000
5999 cases or rows are Deleted m=1, maxit=1 m=20, maxit=10
Comparing Results 15
Fitted Model
List-wise Deletion
Full Information Maximum Likelihood
(FIML) Imputation
Expectation Maximization
(EM) Imputation
p-value p-value p-valueIntercept 0.415 0.205 0.043 0.309 0.185 0.096 0.301 0.155 0.053Releatedness_1 -0.023 0.040 0.565 -0.012 0.032 0.713 -0.010 0.034 0.781Releatedness_2 0.038 0.045 0.401 0.061 0.036 0.089 0.061 0.034 0.076Competence 0.147 0.079 0.062 0.102 0.065 0.116 0.106 0.065 0.106Autonomy 0.049 0.075 0.514 0.016 0.062 0.798 0.013 0.062 0.839Education -0.045 0.039 0.241 -0.034 0.034 0.319 -0.030 0.033 0.359Age -0.001 0.003 0.682 -0.001 0.002 0.779 0.005 0.020 0.803Income 0.003 0.022 0.903 0.006 0.020 0.766 -0.001 0.002 0.752Hunting_Days 0.135 0.017 0.000 0.148 0.014 0.000 0.148 0.015 0.000“Harvest” 0.569 0.077 0.000 0.599 0.062 0.000 0.598 0.060 0.000
5999 cases or rows are Deleted EM algorithm (MLE) converges in 37 iterations
• EM only shows that Releadness_2 is significant• EM estimates smallest standard error for Income• Comparison of Imputation Methods
Summary16
% of smaller estimations than List-wise Deletion out of 10 variablesApproaches Estimates Std. Err. P-value SuggestionsList-wise Deletion Base Base Base Avoid to useMean Imputation 60% 100% 40% Careful useMultiple Imputation 30% 100% 20% BetterFull Information Maximum Likelihood
30% 100% 20% Better
Expectation Maximization
40% 90% 20% Preferred if converged
Thanks for your kind attention
Special Thanks to: Dr. Andrew Tyre, Uni. Of Nebraska, LincolnDr. Lisa Pennisi, Uni. Of Nebraska, Lincoln Dr. Allan McCutcheon, Uni. Of Nebraska, Lincoln
Nebraska Game & Parks Commission
Anne-Kathrin,F. & Andrea B. (2014). The economic performance of Swiss drinking water utilities. Journal of Prod. Analysis.41:383-397. doi 10.1007/s11123-013-0344-0
Bell, M. L.,& Fairclough,D.L. (2013). Practical and statistical issues in missing data for longitudinal patient reported outcomes.Statistical Methods in Medical Research, 0(0), 1-20. doi: 10.1177/0962280213476378Bennett, D.A. (2001). How can I deal with missing data in my study? Australian and New Zealand Journal of Public Health, 25,
464-469.Brick, J., & Kalton, J. (1996). Handling missing data in survey research. Statistical Methods in Medical Research, 5, 215–238. doi:10.1177/096228029600500302Buuren, S.V.(2012). Flexible imputation of missing data. Taylor & Francis, FL: CRC Press.John, W. G. & Allison E. O. & Tamika D. G.(2007). How many imputations are really needed? some practical clarifications of multiple imputation theory, Springer,8:206- 213.Graham, J. W. (2003). Adding missing-data-relevant variables to FIML based structuralequation models. Structural Equation Modeling, 10,80–100.Groves, R., Fowler, F., Couper, M., Lepkowski, J., Singer, E., & Tourangeau, R. (2004). Survey methodology. Hoboken, NJ: John Wiley.Little, R.J.A. (1988). A test of missing completely at random for multivariate data with missing values. Journal of the American Statistical Association , 83, 1198-1202. NGPC (2008). Nebraska 20 year hunter/angler recruitment, development and retention plan. Lincoln, NE.Pigott, T. D. (2001). A Review of Methods for Missing Data. Educational Research and Evaluation, 7(4), 353-383.Peng, C.Y., Harwell, M., Liou, S.M., & Ehman, L.H. (2006). Advances in missing data methods and implications for educational
research. In S Sawilowsky (Ed.), Real data analysis (pp.31-78), Greenwich, CT: Information Age.Qin,Y.,Zhang,S.,Zhu,X.,Zang,J.,& Zhang,C. (2007). Semi-parametric optimization for missing data imputation. Appl Intell 27,79-88.
DOI 10.1007/s10489-006-0032-0Schafer, J.L. (1999). Multiple imputation: A primer. Statistical Methods in Medical Research. 8: 3-15. Todd D. L., Terrence D. J., Kyle M. L., & Whitney M. (2013). On the joys of missing data. Journal of Pediatric Psychology, 1-12. doi:10.1093/jpepsy/jst048 Yiran D. & Chao-Ying J.P.(2013). Principled missing data methods for researchers. Springer, 2:222.
References18
