justin mezetin and meredith massey nchs questionnaire design research laboratory fedcasic 2014:...
TRANSCRIPT
![Page 1: Justin Mezetin and Meredith Massey NCHS Questionnaire Design Research Laboratory FedCASIC 2014: Survey Evaluation Methods March 19, 2014](https://reader035.vdocuments.mx/reader035/viewer/2022062714/56649d085503460f949da255/html5/thumbnails/1.jpg)
THE Q-SUITE: Q-BANK AND Q-NOTES
Justin Mezetin and Meredith Massey
NCHS Questionnaire Design Research Laboratory
FedCASIC 2014: Survey Evaluation Methods
March 19, 2014
![Page 2: Justin Mezetin and Meredith Massey NCHS Questionnaire Design Research Laboratory FedCASIC 2014: Survey Evaluation Methods March 19, 2014](https://reader035.vdocuments.mx/reader035/viewer/2022062714/56649d085503460f949da255/html5/thumbnails/2.jpg)
Outline
Introduction Analytical Framework Q-Notes: Qualitative Analysis software Q-Bank: Question Evaluation library and
resource Next Steps
![Page 3: Justin Mezetin and Meredith Massey NCHS Questionnaire Design Research Laboratory FedCASIC 2014: Survey Evaluation Methods March 19, 2014](https://reader035.vdocuments.mx/reader035/viewer/2022062714/56649d085503460f949da255/html5/thumbnails/3.jpg)
Introduction
What is cognitive interviewing/question evaluation?
Benefits of Cognitive Interviewing
Validity studies showing all phenomena the question is capturing
![Page 4: Justin Mezetin and Meredith Massey NCHS Questionnaire Design Research Laboratory FedCASIC 2014: Survey Evaluation Methods March 19, 2014](https://reader035.vdocuments.mx/reader035/viewer/2022062714/56649d085503460f949da255/html5/thumbnails/4.jpg)
Analytical Framework
![Page 5: Justin Mezetin and Meredith Massey NCHS Questionnaire Design Research Laboratory FedCASIC 2014: Survey Evaluation Methods March 19, 2014](https://reader035.vdocuments.mx/reader035/viewer/2022062714/56649d085503460f949da255/html5/thumbnails/5.jpg)
Background Challenges
Time consuming to conduct a thorough analysis
Larger sample sizes (N > 20)Multiple projects with over 100-150 interviews
CollaborationMultiple Interview LocationsResearchers with varying levels of
experienceTesting in Multiple Languages
![Page 6: Justin Mezetin and Meredith Massey NCHS Questionnaire Design Research Laboratory FedCASIC 2014: Survey Evaluation Methods March 19, 2014](https://reader035.vdocuments.mx/reader035/viewer/2022062714/56649d085503460f949da255/html5/thumbnails/6.jpg)
Q-NOTES: ANALYSIS
SOFTWARE
![Page 7: Justin Mezetin and Meredith Massey NCHS Questionnaire Design Research Laboratory FedCASIC 2014: Survey Evaluation Methods March 19, 2014](https://reader035.vdocuments.mx/reader035/viewer/2022062714/56649d085503460f949da255/html5/thumbnails/7.jpg)
Level One: Conducting Interviews Interview guide Consistency Pre-selected
Themes Multiple Languages Raw data storage
![Page 8: Justin Mezetin and Meredith Massey NCHS Questionnaire Design Research Laboratory FedCASIC 2014: Survey Evaluation Methods March 19, 2014](https://reader035.vdocuments.mx/reader035/viewer/2022062714/56649d085503460f949da255/html5/thumbnails/8.jpg)
Interview Guide
![Page 9: Justin Mezetin and Meredith Massey NCHS Questionnaire Design Research Laboratory FedCASIC 2014: Survey Evaluation Methods March 19, 2014](https://reader035.vdocuments.mx/reader035/viewer/2022062714/56649d085503460f949da255/html5/thumbnails/9.jpg)
Question Translation
![Page 10: Justin Mezetin and Meredith Massey NCHS Questionnaire Design Research Laboratory FedCASIC 2014: Survey Evaluation Methods March 19, 2014](https://reader035.vdocuments.mx/reader035/viewer/2022062714/56649d085503460f949da255/html5/thumbnails/10.jpg)
Level Two: Summarizing Interview Notes Synthesizing raw
data Tagged videos Researcher
collaboration Viewing data for a
single respondent
![Page 11: Justin Mezetin and Meredith Massey NCHS Questionnaire Design Research Laboratory FedCASIC 2014: Survey Evaluation Methods March 19, 2014](https://reader035.vdocuments.mx/reader035/viewer/2022062714/56649d085503460f949da255/html5/thumbnails/11.jpg)
Summarizing Interview Notes
![Page 12: Justin Mezetin and Meredith Massey NCHS Questionnaire Design Research Laboratory FedCASIC 2014: Survey Evaluation Methods March 19, 2014](https://reader035.vdocuments.mx/reader035/viewer/2022062714/56649d085503460f949da255/html5/thumbnails/12.jpg)
Level Three: Comparing Across Respondents Developing thematic
schema Project level themes Question level
themes Flexibility Consistency across
coders Cross interview
comparison
![Page 13: Justin Mezetin and Meredith Massey NCHS Questionnaire Design Research Laboratory FedCASIC 2014: Survey Evaluation Methods March 19, 2014](https://reader035.vdocuments.mx/reader035/viewer/2022062714/56649d085503460f949da255/html5/thumbnails/13.jpg)
Viewing one question across all respondents
![Page 14: Justin Mezetin and Meredith Massey NCHS Questionnaire Design Research Laboratory FedCASIC 2014: Survey Evaluation Methods March 19, 2014](https://reader035.vdocuments.mx/reader035/viewer/2022062714/56649d085503460f949da255/html5/thumbnails/14.jpg)
Level Four: Comparing Across Groups Development of
Advanced Schema Sub-group analysis Comparative
analysis
![Page 15: Justin Mezetin and Meredith Massey NCHS Questionnaire Design Research Laboratory FedCASIC 2014: Survey Evaluation Methods March 19, 2014](https://reader035.vdocuments.mx/reader035/viewer/2022062714/56649d085503460f949da255/html5/thumbnails/15.jpg)
Subgroup Analysis
![Page 16: Justin Mezetin and Meredith Massey NCHS Questionnaire Design Research Laboratory FedCASIC 2014: Survey Evaluation Methods March 19, 2014](https://reader035.vdocuments.mx/reader035/viewer/2022062714/56649d085503460f949da255/html5/thumbnails/16.jpg)
Subgroup Analysis Results
![Page 17: Justin Mezetin and Meredith Massey NCHS Questionnaire Design Research Laboratory FedCASIC 2014: Survey Evaluation Methods March 19, 2014](https://reader035.vdocuments.mx/reader035/viewer/2022062714/56649d085503460f949da255/html5/thumbnails/17.jpg)
Comparative Analysis
![Page 18: Justin Mezetin and Meredith Massey NCHS Questionnaire Design Research Laboratory FedCASIC 2014: Survey Evaluation Methods March 19, 2014](https://reader035.vdocuments.mx/reader035/viewer/2022062714/56649d085503460f949da255/html5/thumbnails/18.jpg)
Level Five: Drawing Conclusions
Report template Summary tables Thematic schema
![Page 19: Justin Mezetin and Meredith Massey NCHS Questionnaire Design Research Laboratory FedCASIC 2014: Survey Evaluation Methods March 19, 2014](https://reader035.vdocuments.mx/reader035/viewer/2022062714/56649d085503460f949da255/html5/thumbnails/19.jpg)
Report Template
![Page 20: Justin Mezetin and Meredith Massey NCHS Questionnaire Design Research Laboratory FedCASIC 2014: Survey Evaluation Methods March 19, 2014](https://reader035.vdocuments.mx/reader035/viewer/2022062714/56649d085503460f949da255/html5/thumbnails/20.jpg)
Q-BANK: DISSEMINATING
FINDINGS
![Page 21: Justin Mezetin and Meredith Massey NCHS Questionnaire Design Research Laboratory FedCASIC 2014: Survey Evaluation Methods March 19, 2014](https://reader035.vdocuments.mx/reader035/viewer/2022062714/56649d085503460f949da255/html5/thumbnails/21.jpg)
Q-Bank Users
Data Analysts Questionnaire Designers Survey Methodologists Survey Managers
![Page 22: Justin Mezetin and Meredith Massey NCHS Questionnaire Design Research Laboratory FedCASIC 2014: Survey Evaluation Methods March 19, 2014](https://reader035.vdocuments.mx/reader035/viewer/2022062714/56649d085503460f949da255/html5/thumbnails/22.jpg)
Q-Bank Benefits
Deeper understanding of what survey variables are capturing
Share/view findings from studies conducted at other organizations
Resource to learn more and dialog about Question Evaluation
Mechanism to compare different methods and findings
![Page 23: Justin Mezetin and Meredith Massey NCHS Questionnaire Design Research Laboratory FedCASIC 2014: Survey Evaluation Methods March 19, 2014](https://reader035.vdocuments.mx/reader035/viewer/2022062714/56649d085503460f949da255/html5/thumbnails/23.jpg)
Searching Q-Bank
![Page 24: Justin Mezetin and Meredith Massey NCHS Questionnaire Design Research Laboratory FedCASIC 2014: Survey Evaluation Methods March 19, 2014](https://reader035.vdocuments.mx/reader035/viewer/2022062714/56649d085503460f949da255/html5/thumbnails/24.jpg)
Q-Bank Search Results
![Page 25: Justin Mezetin and Meredith Massey NCHS Questionnaire Design Research Laboratory FedCASIC 2014: Survey Evaluation Methods March 19, 2014](https://reader035.vdocuments.mx/reader035/viewer/2022062714/56649d085503460f949da255/html5/thumbnails/25.jpg)
Question Selection
![Page 26: Justin Mezetin and Meredith Massey NCHS Questionnaire Design Research Laboratory FedCASIC 2014: Survey Evaluation Methods March 19, 2014](https://reader035.vdocuments.mx/reader035/viewer/2022062714/56649d085503460f949da255/html5/thumbnails/26.jpg)
Question Details
![Page 27: Justin Mezetin and Meredith Massey NCHS Questionnaire Design Research Laboratory FedCASIC 2014: Survey Evaluation Methods March 19, 2014](https://reader035.vdocuments.mx/reader035/viewer/2022062714/56649d085503460f949da255/html5/thumbnails/27.jpg)
Q-Suite Features
Collaboration Consistency Transparency
Q-Notes: Qualitative Analysis Software Q-Bank:
Library of Evaluation studiesResource on Question Evaluation
![Page 28: Justin Mezetin and Meredith Massey NCHS Questionnaire Design Research Laboratory FedCASIC 2014: Survey Evaluation Methods March 19, 2014](https://reader035.vdocuments.mx/reader035/viewer/2022062714/56649d085503460f949da255/html5/thumbnails/28.jpg)
Q-Suite Next Steps
Q-NotesRun a public beta test Summer 2014*Export feature to submit into Q-Bank
Q-BankContinue increasing content Inclusion of all methods in Q-BankPilot test with NHIS to link key variables to
Q-Bank testing reports