aaron w. bernard, md jonathan fisher, md qualitative methods in educational research
TRANSCRIPT
Objectives
Define qualitative researchContrast with quantitative researchDescribe how to conduct qualitative research
by using examples in the literature
Quantitative Research
Test well-specified hypothesis concerning some predetermined variables
Answer questions like; Whether…….. How much….
Quantitative Educational Research
Attempts to study links between teaching factors and learning outcomes
Improved Medical Student Satisfaction and Test Performance with a Simulation-based Emergency Medicine Curriculum: A Randomized Controlled Trial.
Ten Eyck. Ann Emerg Med. 2009;54:684-691
Qualitative Research
Qualitative research questions tend not to ask whether or how much but rather; What Why How
Qualitative research offers insight into social, emotional, and experiential phenomena.
Qualitative reports do not typically generate answers but rather generate narrative accounts, explanations, conceptual frameworks.
Qualitative Research
Qualitative research is inquiry-guided, inductive, and descriptive.
Similar to an anthropologic or sociologic descriptive study.
Data Collection Methods
Field observations
Interviews Acad Med. 2009.
Document analysis Teach Learn Med. 2004.
What Do Emergency Medicine Learners Want from Their Teachers? A Multicenter Focus Group Analysis. Acad Emerg
Med 2005.
Background Effective teaching practices in the ambulatory setting
have been documented The ED is a different type of learning environment A qualitative study to determine what EM teachers
consider important has been done1
What do EM learners consider important to effective EM teaching? Quantitative methods vs.. qualitative methods
Bandiera . Ann Emerg Med 2005; 45:253-261
What Do Emergency Medicine Learners Want from Their Teachers? A Multicenter Focus Group Analysis. Acad Emerg
Med 2005.
Methods
Focus groups at Five EM programs in Ontario Multiple sites adds validity
Medical student, EM Resident, Off-service resident, Junior FRCP, Senior FRCP Purposeful sampling 28 participants total
What Do Emergency Medicine Learners Want from Their Teachers? A Multicenter Focus Group Analysis. Acad Emerg
Med 2005.
Methods
Semi-structure interviews by a trained moderator
Tape-recorded and transcribed by a third party
Participants used terms like “junior” to identify themselves to allow the data to be matched to type of participant
What Do Emergency Medicine Learners Want from Their Teachers? A Multicenter Focus Group Analysis. Acad Emerg
Med 2005.
Methods/Data Analysis Transcripts were reviewed by two independent
assessors and a coding framework was derived using grounded theory methods
Novel ideas were assigned a new teaching code or a new qualifier as they emerged
When a new code was added the entire transcript was reviewed
The two codes that emerged were compared and consensus was reached
Agreement on 41 coded items and disagreement on 9 5/9 were subthemes
What Do Emergency Medicine Learners Want from Their Teachers? A Multicenter Focus Group Analysis. Acad Emerg
Med 2005.
What Do Emergency Medicine Learners Want from Their Teachers? A Multicenter Focus Group Analysis. Acad Emerg
Med 2005.
Data analysis The transcripts were then reviewed against the final
code by one investigator to check for frequency of codes
A second investigator reviewed 20 percent of the transcript to allow for an estimation of interrater reliability Computer software exists to help with this!
What Do Emergency Medicine Learners Want from Their Teachers? A Multicenter Focus Group Analysis. Acad Emerg
Med 2005.
Summary Qualitative methods were most appropriate for
this investigation Focus group interviews are a common method to
obtain data Purposeful sampling (low numbers, key players) Grounded theory to create code Frequency of themes to assess importance of
themes to subject
Medical student professionalism narratives: a window on the informal and hidden curriculum. Acad Med 2010
Background The informal and hidden curriculum drives
professionalism development There is a lack of evidence regarding what
students experience.
Objective To understand students experiences as they relate
to an understanding of professionalism
Medical student professionalism narratives: a window on the informal and hidden curriculum. Acad Med 2010
Methods Students on IM required to post 2 narratives over 2
months Retrospective review of narratives June – November
2007 272 narratives from 135 students
Medical student professionalism narratives: a window on the informal and hidden curriculum. Acad Med 2010
Methods/Data Analysis Thematic content analysis Immersion/crystallization method Reading and rereading narratives until themes
emerge Code was created with the first 50 narratives A narrative could be coded more then once
Medical student professionalism narratives: a window on the informal and hidden curriculum. Acad Med 2010
Reliability1. One author created the code based on the first 50
narratives then two others independently reviewed the work.
2. Created focus groups of students to review themes
Medical student professionalism narratives: a window on the informal and hidden curriculum. Acad Med 2010
Two major domains14 themes63.4% positive, 29.1%
negative, 7.5 % hybrid
Medical student professionalism narratives: a window on the informal and hidden curriculum. Acad Med 2010
Medical student professionalism narratives: a window on the informal and hidden curriculum. Acad Med 2010
SummaryQualitative methods were most
appropriate for this investigationDocument analysis is a common
method to obtain dataMethodological steps to improve
validity is importantThe use of direct representative
quotes
Medical student participation in a disaster seminar and drill: brief description of activity and report of student experiences. Teach Learn
Med 2010
Background Disaster training is relevant to medical students who
as physicians may be asked for guidance during training or real events.
Medical school offer little formal curriculum
Objective Provide 4th year medical students with a didactic and
clinically experience in disaster medicine Evaluate the effectiveness through lecture evaluations
and a qualitative analysis of focus groups
Medical student participation in a disaster seminar and drill: brief description of activity and report of student experiences. Teach Learn
Med 2010
Methods 43 students 4 hour didactic session the evening before the drill Training in the use of a standardized assessment tool
for disaster medicine drills
Evaluation and Focus Group 5 point likert scale surveys regarding the didactics 2 Focus groups after the drill
Semi-structured interviews
Medical student participation in a disaster seminar and drill: brief description of activity and report of student experiences. Teach Learn
Med 2010
Methods/Data Analysis Median, means, SD for the survey data Notes were taken in real time One researcher performed the qualitative analysis and
created the thematic domains of the code
Medical student participation in a disaster seminar and drill: brief description of activity and report of student experiences. Teach Learn
Med 2010
28 of the 42 students participated in both the didactic and the drill
17 agreed to participate in the focus group
Themes Changes in self-perceived
attitude toward DM Changes in student’s
ability to apply this knowledge in a simulated setting.
Medical student participation in a disaster seminar and drill: brief description of activity and report of student experiences. Teach Learn
Med 2010
Effective use of qualitative research to create a descriptive study of an educational curriculum that is publishable.
Combination of qualitative and quantitative methods
Final Conclusions
Qualitative research is a tool that can be used for educational investigations.
Qualitative research is inquiry-guided, inductive, and descriptive.
Data usually comes from field observations, interviews, or document analysis.
Grounded theory is used to review the data to create themes and subthemes. This is known as coding.
Frequency of themes is often used as a marker of importance of themes to subjects studied
Various techniques can be used to increase reliability of the analysis
References
Tehaerani A, O’Brien B, Masters DE, et al. Burden, responsibility, and reward: preceptor experiences with the continuity of teaching in a longitudinal integrated clerkship. Acad Med. 2009;84(10 Supp):s50-s53.
Alford CL, Currie DM. Introducing first-year medical students to clinical practice by having them “shadow” third-year clerks. Teach Learn Med. 2004;16(3):260-263.
Thurgur L, Bandiera G, Lee S, et al. What do emergency medicine learners want from their teachers? A multicenter focus group analysis. Acad Emerg Med 2005; 12:856-861.
Karnieli-Miller O, Vu R, Holtman M, et al. Medical students’ professionalism narratives: a window on the informal and hidden curriculum. Acad Med. 2010;85:124-133.
Giacomini MK, Cook DJ, et al. Users’ guides to the medical literature XXIII. Qualitative research in health care A. Are the results of the study valid? JAMA. 2000;284:357-362.
Smith AK, Fisher J, Schonberg MA, et al. Am I doing the right thing? Provider perspectives on improving palliative care in the Emergency Department. An Emerg Med. 2009;54:86-93.
Kaji AH, Coates WC, Fung C-C. Medical student participation in a disaster seminar and drill: brief description of activity and report of student experiences. Teach Learn Med. 2010;22(1):28-32