advanced curriculum development › conferences › healthcare... · advanced curriculum...
TRANSCRIPT
Advanced Curriculum Development:Developing a Mastery Learning Curriculum
David H. Salzman, MD, MEdAssociate Professor - Departments of Medical Education & Emergency MedicineJeffrey H. Barsuk, MD, MSProfessor – Departments of Medical Education & Medicine
Disclosures
David H. Salzman, MD, MEd Nothing to disclose
Jeffrey H. Barsuk, MD, MS Has received research support from Simulab, Co.
Objectives
By the end of the session participants will be able to:1. Identify the seven essential elements of Mastery Learning2. Analyze how integration of mastery learning impacts the six steps of
curriculum development 3. Apply the concepts of curriculum design and mastery learning to modify
an existing curriculum.
Previous Experiences with Mastery Learning
Curriculum Development 1. Problem Identification and General Needs AssessmentWhat is the healthcare problem that needs to be solved?
What are the current vs. ideal approaches?
2. Needs Assessment of Targeted Learners
What do your students need?What does your school or local environment need?
3. Goals and ObjectivesWhat are the overarching
goals and specific measurable objectives?
4. Educational StrategiesWhat teaching and assessment methods
are best suited to your objectives?
5. ImplementationWhat resources are necessary to
deliver this curriculum?
6. Evaluation and FeedbackDid it work?
Individual learners and Program
Traditional Training
Why does this happen in the traditional model of curriculum design for medical education?
Mastery Learning Approach
• Assumes normal distribution of performance is artificial• Assumes all participants can master performance goal
When Traditional Approaches Fail To Meet Expectations
Simulation-Based Mastery Learning:A Better Approach
Pretest
Lecture, video, demonstration and deliberate practice
Posttest
Additional Practice PASS
McGaghie, William C.; Barsuk, Jeffrey H.; Wayne, Diane B. Mastery Learning With Deliberate Practice in Medical Education Academic Medicine. 90(11):1575, November 2015.
SBML: A Better Approach
SBML: A Better Approach
McGaghie, William C.; Barsuk, Jeffrey H.; Wayne, Diane B. Mastery Learning With Deliberate Practice in Medical Education Academic Medicine. 90(11):1575, November 2015.
Components of Mastery Learning
1. Baseline, or diagnostic testing2. Clear learning objectives, sequenced as units in increasing difficulty3. Engagement in educational activities (eg, skills practice, data interpretation,
reading, focused on reaching objectives)4. A set minimum passing standard (eg, test score) for each educational unit5. Formative and/or summative testing to gauge unit completion at a present
minimum passing standard for mastery6. Advancement to the next educational unit given measured achievement at or
above the mastery standard.7. Continued practice or study on an educational unit until the mastery standard is
reached.
Mastery Learning1: Baseline, or diagnostic testing
Mastery Learning2: Clear learning objectives, sequenced as units in increasing difficulty
Mastery Learning3: Engagement in educational activities
Mastery Learning4: A set minimum passing standard for each educational unit
Actual cumulative % of students
Final Passing Score (%) 54%
Minimum and Maximum FAIL RATE
Minimum and Maximum PASS SCORE
Mastery Learning5: Formative and/or summative testing to gauge unit completion at a present minimum passing standard for mastery
Mastery Learning6: Advancement to the next unit given measured achievement at or above the mastery standard.
Mastery Learning
If at first you don’t succeed…..
………..Try, try again
Until meeting or exceeding the standard
7: Continued practice or study until the mastery standard is reached
Components of Mastery Learning
1. Baseline, or diagnostic testing2. Clear learning objectives, sequenced as units in increasing difficulty3. Engagement in educational activities (eg, skills practice, data interpretation,
reading, focused on reaching objectives)4. A set minimum passing standard (eg, test score) for each educational unit5. Formative and/or summative testing to gauge unit completion at a present
minimum passing standard for mastery6. Advancement to the next educational unit given measured achievement at or
above the mastery standard.7. Continued practice or study on an educational unit until the mastery standard is
reached.
Combining SBML with 6 Steps of Curriculum Design
Combining SBML with 6 Steps of Curriculum Design
Pretest
Lecture, video, demonstration and deliberate practice
Posttest
Additional Practice PASS
1. Problem Identification and General Needs Assessment
What is the healthcare problem that needs to be solved?
What are the current vs. ideal approaches?2. Needs Assessment of
Targeted Learners
What do your students need?
What does your school or local environment need?
3. Goals and Objectives
What are the overarching goals and specific measurable
objectives?
4. Educational Strategies
What teaching and assessment methods are best suited to your objectives?
5. Implementation
What resources are necessary to deliver this curriculum?
6. Evaluation and Feedback
Did it work?
Individual learners and Program
Convergence of ML and 6 Steps1. Problem Identification and General Needs AssessmentWhat is the healthcare problem that needs to be solved?
What are the current vs. ideal approaches?
2. Needs Assessment of Targeted LearnersWhat do your students need?
What does your school or local environment need?
3. Goals and ObjectivesWhat are the overarching goals and specific measurable objectives?
4. Educational StrategiesWhat teaching and assessment methods are best suited to your objectives?
5. ImplementationWhat resources are necessary to deliver this curriculum?
6. Evaluation and FeedbackDid it work?
Individual learners and Program
Similar Approach
Mastery Learning Model impacts these steps
Convergence of ML and 6 Steps1. Problem Identification and General Needs AssessmentWhat is the healthcare problem that needs to be solved?
What are the current vs. ideal approaches?
2. Needs Assessment of Targeted LearnersWhat do your students need?
What does your school or local environment need?
3. Goals and ObjectivesWhat are the overarching goals and specific measurable objectives?
4. Educational StrategiesWhat teaching and assessment methods are best suited to your objectives?
5. ImplementationWhat resources are necessary to deliver this curriculum?
6. Evaluation and FeedbackDid it work?
Individual learners and Program
Similar Approach
Curriculum Development
• Who does the problem affect?
• What does it affect?
Step 1: Problem Identification
Convergence of ML and 6 Steps1. Problem Identification and General Needs AssessmentWhat is the healthcare problem that needs to be solved?
What are the current vs. ideal approaches?
2. Needs Assessment of Targeted LearnersWhat do your students need?
What does your school or local environment need?
3. Goals and ObjectivesWhat are the overarching goals and specific measurable objectives?
4. Educational StrategiesWhat teaching and assessment methods are best suited to your objectives?
5. ImplementationWhat resources are necessary to deliver this curriculum?
6. Evaluation and FeedbackDid it work?
Individual learners and Program
Similar Approach
Curriculum Development
• Determination of the learners• What is known about the learners and their needs
Step 2: Needs Assessment of Targeted Learners
Design a Simulation-Based-Mastery Learning Course
Design a Simulation-Based-Mastery Learning Course
Convergence of ML and 6 Steps1. Problem Identification and General Needs AssessmentWhat is the healthcare problem that needs to be solved?
What are the current vs. ideal approaches?
2. Needs Assessment of Targeted LearnersWhat do your students need?
What does your school or local environment need?
3. Goals and ObjectivesWhat are the overarching goals and specific measurable objectives?
4. Educational StrategiesWhat teaching and assessment methods are best suited to your objectives?
5. ImplementationWhat resources are necessary to deliver this curriculum?
6. Evaluation and FeedbackDid it work?
Individual learners and Program
2: Clear learning objectives, sequenced as units in increasing difficulty
Objectives
Objectives:
What should your students be able to do upon completion of the session that they were unable to do prior
Who will do how much (how well) of what, by when?
This is influenced by the Minimum Passing Score
Objectives
• Levels of objectives -
ObjectivesWrite Learning Objectives
Create and Submit Objectives
http://bit.ly/SBML19LO
http://bit.ly/SBML19LO
CAPS IMPORTANT
Create and Submit Objectives
http://bit.ly/SBML19LO
Sharing Objectives
Convergence of ML and 6 Steps1. Problem Identification and General Needs AssessmentWhat is the healthcare problem that needs to be solved?
What are the current vs. ideal approaches?
2. Needs Assessment of Targeted LearnersWhat do your students need?
What does your school or local environment need?
3. Goals and ObjectivesWhat are the overarching goals and specific measurable objectives?
4. Educational StrategiesWhat teaching and assessment methods are best suited to your objectives?
5. ImplementationWhat resources are necessary to deliver this curriculum?
6. Evaluation and FeedbackDid it work?
Individual learners and Program
Engagement in educational activities
Baseline, or diagnostic testing
Advancement to the next unit given measured achievement at or above the mastery standard.
Curriculum Development
Many different types of educational strategiesStep 4: Educational Strategies
Curriculum Development
• Key to ensure that the methods of instruction are congruent with the goals and objectives
• Use multiple educational methods• Choose educational methods that are feasible in terms of resources
Step 4: Educational Strategies
How these components interact with 6 steps
• How is material going to be delivered?• How will students practice? / how to create engagement in educational activities• Assessment strategies
Step 4: Educational Strategies
How these components interact with 6 steps
• Modalities to be considered: lectures, videos, simulations, debriefing, flipped classrooms
• Ensure engagement in educational activities: deliberate practice, data interpretation, reading, focused on the objectives
Step 4: Educational Strategies
Design curriculum and set standards
(passing score)
Training: mastery learning with
deliberate practice
passPre-
intervention assessment on simulator fail
Post-intervention assessment on simulator
Figure 2. Standard SBML curriculum.
additional
practice
How these components interact with 6 steps
Deliberate Practice
Step 4: Educational Strategies - Deliberate Practice
A. Ericsson 2007
Deliberate Practice (DP)
• Identify well-defined task• Focused, repetitive practice• Informative feedback• Opportunity to correct errors• Goal: skill improvement
Ericsson, Academic Medicine, 2004
Design Educational Strategies
Describe and Submit Educational Strategies
http://bit.ly/SBML19ES
Sharing Educational Strategies
Convergence of ML and 6 Steps1. Problem Identification and General Needs AssessmentWhat is the healthcare problem that needs to be solved?
What are the current vs. ideal approaches?
2. Needs Assessment of Targeted LearnersWhat do your students need?
What does your school or local environment need?
3. Goals and ObjectivesWhat are the overarching goals and specific measurable objectives?
4. Educational StrategiesWhat teaching and assessment methods are best suited to your objectives?
5. ImplementationWhat resources are necessary to deliver this curriculum?
6. Evaluation and FeedbackDid it work?
Individual learners and Program
Curriculum Development
• Resources• Support• Administrative mechanisms
to support• Barriers• Plan for introduction
Step 5: Implementation
How these components interact with 6 steps
• How the amount of time impacts… (not everyone will need the same amount of time)
Step 5: Implementation
Implementation Science DomainsConsolidated Framework for Implementation Research (CFIR)
1. Intervention
2. Outer setting (e.g., the medical education enterprise)
3. Inner Setting (e.g., culture of a hospital)
4. Individuals involved
5. Process of Implementation (planning, engaging, executing, and evaluating)
Damschroder et al. Implementation science 2009;4:50McGaghie et al. Acad Med 2015;90(11)1487-94
Implementation Planning
Implementation Planning
http://bit.ly/SBML19IP
Sharing Implementation
Convergence of ML and 6 Steps1. Problem Identification and General Needs AssessmentWhat is the healthcare problem that needs to be solved?
What are the current vs. ideal approaches?
2. Needs Assessment of Targeted LearnersWhat do your students need?
What does your school or local environment need?
3. Goals and ObjectivesWhat are the overarching goals and specific measurable objectives?
4. Educational StrategiesWhat teaching and assessment methods are best suited to your objectives?
5. ImplementationWhat resources are necessary to deliver this curriculum?
6. Evaluation and FeedbackDid it work?
Individual learners and Program
Actual cumulative % of students
Final Passing Score (%) 54%
Minimum and Maximum FAIL RATE
Minimum and Maximum PASS SCORE
A set minimum passing standard for each educational unit
Diagnostic testing
Curriculum Development
• Individual level, determination if a learner has met the objectives – Assessment• Determine of success of the program – Evaluation
Step 6: Evaluation and Feedback
Curriculum Development
• Additional concepts in evaluation and feedback that won’t be covered here:- Checklists
• Creation of Checklists• Reliability and validity of checklists
- Data collection- Data analysis & statistical methods- Ethical considerations with delivery of content
Step 6: Evaluation and Feedback
How these components interact with 6 steps
• Need to establish MPS• Cannot advance to next unit unless meets MPS• Curriculum effectiveness – Translational Outcomes T1-T4
Step 6: Assessment and Feedback
Learner Assessment & Program Evaluation
http://bit.ly/SBML19AE
Sharing Learner Assessment
How these components interact with 6 steps
1. Highly motivated learners with good concentration;2. Engagement with a well-defined learning objective or task; at an3. Appropriate level of difficulty; with 4. Focused, repetitive practice; that leads to5. Rigorous, precise measurements; that yield6. Informative feedback from educational sources (e.g., simulators,
teachers); and where7. Trainees also monitor their learning experiences and correct
strategies, errors, and levels of understanding, engage in more DP; and continue with
8. Evaluation to reach a mastery standard; and then9. Advance to another task or unit10. Goal: constant improvement
Step 4: Educational Strategies - Deliberate Practice
Simulation-Based Mastery Learning:A Better Approach - Questions
Pretest
Lecture, video, demonstration and deliberate practice
Posttest
Additional Practice PASS
McGaghie, William C.; Barsuk, Jeffrey H.; Wayne, Diane B. Mastery Learning With Deliberate Practice in Medical Education Academic Medicine. 90(11):1575, November 2015.
Components of Mastery Learning
1. Baseline, or diagnostic testing2. Clear learning objectives, sequenced as units in increasing difficulty3. Engagement in educational activities (eg, skills practice, data interpretation, reading,
focused on reaching objectives)4. A set minimum passing standard (eg, test score) for each educational unit5. Formative testing to gauge unit completion at a present minimum passing standard for
mastery6. Advancement to the next educational unit given measured achievement at or above the
mastery standard.7. Continued practice or study on an educational unit until the mastery standard is reached.
Apply to Your Own Curriculum1. Problem Identification and General Needs AssessmentWhat is the healthcare problem that needs to be solved?
What are the current vs. ideal approaches?
2. Needs Assessment of Targeted LearnersWhat do your students need?
What does your school or local environment need?
3. Goals and ObjectivesWhat are the overarching goals and specific measurable objectives?
4. Educational StrategiesWhat teaching and assessment methods are best suited to your objectives?
5. ImplementationWhat resources are necessary to deliver this curriculum?
6. Evaluation and FeedbackDid it work?
Individual learners and Program
Similar Approach
Mastery Learning Model impacts these steps