milestones - southern illinois university school of medicine...yreportable milestones will be set by...
TRANSCRIPT
![Page 1: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/1.jpg)
MilestonesReady or not….Here they come!SIU SOMMarch 6, 2013
![Page 2: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/2.jpg)
A Word of ThanksDonald BradySusan GuralnickJustin HeldEric HolmboeTom NascaJohn MellingerJulie Rhodes
Jennifer RodgersHilary SanfeyDebi SantiniAndy VarneyEric WarmEvery RRC member, program director, fellow DIO and GME leader who let us shamelessly borrow their great ideas
![Page 3: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/3.jpg)
Objectives Participants will be able to……
Define Milestone, Entrustable Professional Activity and Competency Based Assessment
Regurgitate current (today) and short-term (18 months) ACGME Milestone-related expectations
Share collective Angst and Wisdom
Determine next steps for faculty development
![Page 4: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/4.jpg)
My Favorite Color is….
1 2 3 4
100%
0%0%0%
1. Blue2. Red3. Puce4. I Can’t Work This
Thing
![Page 5: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/5.jpg)
My program is well-prepared for ACGME Milestone Reporting
1 2 3 4 5
5%
33%
5%
19%
38%
1. What’s a Milestone?2. A Little (25%)3. Half –Way There
(50%)4. Pretty Much (75%)5. Absolutely!! (100%)
![Page 6: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/6.jpg)
Definitions and Background….
Competence: ◦ An observable ability of a Health Professional,
integrating multiple components, such as knowledge, skills, values. (International CBME Collaborators)
◦ The ability to do something successfully◦ A quality◦ An adjective◦ Can have multiple domains
![Page 7: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/7.jpg)
Entrustable Professional Activities
An activity is something you can observe – a noun
A unit of work that should only be entrusted upon a competent enough professional (ten Cate, 2007)
May be awarded…qualification at the moment when supervisors confirm that the trainee is ready to assume responsibility for such activities. Entrustment can occur formally or informally.
An event/outcome that marks a turning point or stage in training that is often manifest through a trainee being granted increased level of autonomy, responsibility or decision-making capacity. (Varney)
![Page 8: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/8.jpg)
![Page 9: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/9.jpg)
EPA: Capable in the ICU Setting
Run Code
Use order sets
Effective in Transitions of Care
Lead an interdisciplinary team
Run a family meeting
![Page 10: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/10.jpg)
MilestonesACGME NAS FAQs December 2012
Observable developmental steps that describe a trajectory of progress on the competencies from novice (entering resident) to proficient (graduating resident) and, ultimately, to expert.
They articulate shared understanding of expectations, set aspirational goals of excellence, provide a framework for discussions across the continuum and track educational outcomes
![Page 11: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/11.jpg)
Eric Holmboe, 2013 ACGME Meeting
![Page 12: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/12.jpg)
Milestones are a measure of Development
Think Denver or Bailey
Consistency, habits and skills are demonstrated in multiple contexts and settings over time
![Page 13: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/13.jpg)
EPAs Competencies Milestones
1. Titrate insulin2. Manage ventilator3. Choose antibiotics4. Treat pain5. Diagnose VTE6. Manage a team7. Share decision making8. Hand‐off properly
1. Patient Care2. Medical Knowledge3. Professionalism4. Communication Skills5. Systems Based Practice6. Practice Based Learning
The Good Resident
We see these things in the
clinics and on the wards
Yet we measure learners with these.
Can DO these well.
Courtesy of Eric Warm, MDUniversity of Cincinnati
![Page 14: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/14.jpg)
EPAs Competencies Milestones
1. Titrate insulin2. Manage ventilator3. Choose antibiotics4. Treat pain5. Diagnose VTE6. Manage a team7. Share decision making8. Hand‐off properly
1. Patient Care2. Medical Knowledge3. Professionalism4. Communication Skills5. Systems Based Practice6. Practice Based Learning
The Good ResidentCan DO these well.
This is what we should be measuring
These are what we should be able to sort by
Courtesy of Eric Warm, MDUniversity of Cincinnati
![Page 15: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/15.jpg)
EPAs Competencies Milestones
1. Titrate insulin2. Manage ventilator3. Choose antibiotics4. Treat pain5. Diagnose VTE6. Manage a team7. Share decision making8. Hand‐off properly
1. Patient Care2. Medical Knowledge3. Professionalism4. Communication Skills5. Systems Based Practice6. Practice Based Learning
The Struggling Resident
This is what we see them struggle with These might point towards the reasons (and
solutions) for the struggles
CANNOT DO these things well.
Courtesy of Eric Warm, MDUniversity of Cincinnati
![Page 16: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/16.jpg)
Held and Warm, 2013 ACGME Meeting
![Page 17: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/17.jpg)
Held and Warm, 2013 ACGME Meeting
![Page 18: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/18.jpg)
Milestones a la NAS
Ingredients Steps
Learning ActivitiesClinical ExperiencesKnowledge TestsRotation EvalsEPA AttainmentCurricular MilestonesPortfolio Products360 degree evalsPatient Satisfaction dataCase logsMini CEXsOSCEs Peer EvalsSimulationsChart Stimulated Recalls
1. Mix all assessment ingredients together
2. Add a generous amount of direct observations
3. Season with the shared wisdom and judgement of a Clinical Competence Committee
4. Transfer one half to promotions decisions and one half to an ungreased WebADs Milestones reporting form
![Page 19: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/19.jpg)
Better Ingredients…
Better Milestones!
![Page 20: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/20.jpg)
ACGME Milestones…
![Page 21: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/21.jpg)
Eric Holmboe, 2013 ACGME Meeting
![Page 22: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/22.jpg)
Eric Holmboe, 2013 ACGME Meeting
![Page 23: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/23.jpg)
7 Early Adopter RRCs
Diagnostic RadiologyEmergency MedicineInternal MedicineNeurologic SurgeryOrthopedic SurgeryPediatricsUrology
![Page 24: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/24.jpg)
Timetable and Logistics….Phase 1 programs will submit initial milestone data in Dec
2013 and June 2014
All other core programs will begin submitting December 2014
Subspecialty milestone development will begin July 2013
Validity of milestones has yet to be established – work in progress!
Reportable milestones will be set by RRCs◦ Programs may have additional curricular milestones
Milestone data will be submitted via web ADs de-identified -RRC will see only aggregate data for program
![Page 25: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/25.jpg)
![Page 26: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/26.jpg)
![Page 27: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/27.jpg)
![Page 28: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/28.jpg)
Exhibits integrity and ethical behavior in professional conduct
Critical Deficiencies Ready for unsupervised practice Aspirational
Dishonest in clinicalinteractions,documentation,research, or scholarly activity
Refuses to beaccountable forpersonal actions
Does not adhere to basic ethicalprinciples
Blatantly disregardsformal policies orprocedures.
Honest in clinicalinteractions,documentation, research,and scholarly activity.Requires oversight forprofessional actions
Has a basic understandingof ethical principles, formalpolicies and procedures,and does not intentionally disregard them
Honest and forthright inclinical interactionsdocumentations, research, and scholarly activity
Demonstrates accountabilityfor the care of patients
Adheres to ethical principlesfor documentation, followsformal policies andprocedures, acknowledgesand limits conflict of interest, and upholds ethicalexpectations of research and scholarly activity
Demonstrates integrity,Honesty, and accountability topatients, society and the profession
Actively manages challengingethical dilemmas and conflicts of interest
Identifies and respondsappropriately to lapses of professional conduct among peer group
Assists others in adhering to ethical principles andbehaviors including integrity, honesty, and professionalresponsibility
Role models integrity,honesty, accountabilityand professional conduct in all aspects of professional life
Regularly reflects on personal professional Conduct
Comments:
![Page 29: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/29.jpg)
Wide Variability Among RRCsRadiology – 12 Milestones
Patient Care :
1. Consultant 2. Competence in
procedures
Internal Medicine -◦ 22 Reporting Milestones◦ 142 curricular Milestones
Patient Care:1. Gathers and synthesizes essential and
accurate information to define each patient’s clinical problem(s).
2. Develops and achieves comprehensive management plan for each patient.
3. Manages patients with progressive responsibility and independence.
4. Skill in performing procedures.
5. Requests and provides consultative care.
![Page 30: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/30.jpg)
IPS and C – 7 Early Adopter RRCsDonald Brady, Vanderbilt
![Page 31: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/31.jpg)
Professionalism – 7 Early Adoptor RRCsDonald Brady, Vanderbilt
![Page 32: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/32.jpg)
Clinical Competence Committee
Each program is expected to have a CCC by June 2013CCC members should include core faculty who can observe and evaluate residents. May include other membersCommon model: 1 or 2 faculty review all evals for a resident before committee discussion
![Page 33: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/33.jpg)
Eric Holmboe, 2013 ACGME Meeting
![Page 34: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/34.jpg)
Eric Holmboe, 2013 ACGME Meeting
![Page 35: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/35.jpg)
Random Important Points
A Milestone must be an OBSERVABLE behavior or set of
behaviors
Milestone ratings DO NOT correspond to PGY level
ACGME Milestones are not the same as Curricular Milestones
![Page 36: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/36.jpg)
Please rate the extent to which the following statements describe your program
![Page 37: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/37.jpg)
The evaluations of residents in my program are organized by the 6 ACGME competencies
1 2 3 4 5
0%7%
71%
18%
4%
1. Not at all (0%)2. A Little (25%)3. About Half (50%)4. Pretty Much (75%)5. Absolutely (100%)
![Page 38: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/38.jpg)
The evaluations of residents in my program have a defined progression of competence or competencies
.
1 2 3 4 5
0%
15%
35%
42%
8%
1. Not at all (0%)2. A Little (25%)3. About Half (50%)4. Pretty Much (75%)5. Absolutely (100%)
![Page 39: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/39.jpg)
Teaching/learning opportunities, assessments and desired outcomes are aligned
.
1 2 3 4 5
0%
11%
4%
57%
29%
1. Not at all (0%)2. A Little (25%)3. About Half (50%)4. Pretty Much (75%)5. Absolutely (100%)
![Page 40: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/40.jpg)
Learning experiences are deliberate(not circumstantial).
1 2 3 4 5
0%
4%
30%
37%
30%1. Not at all (0%)2. A Little (25%)3. About Half (50%)4. Pretty Much (75%)5. Absolutely (100%)
![Page 41: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/41.jpg)
Desired outcomes drive teaching-learning and assessment decisions
1 2 3 4 5
0%4%
27%
58%
12%
1. Not at all (0%)2. A Little (25%)3. About Half (50%)4. Pretty Much (75%)5. Absolutely (100%)
![Page 42: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/42.jpg)
Assessments are criterion referenced
1 2 3 4 5
0%4%
38%42%
15%
1. Not at all (0%)2. A Little (25%)3. About Half (50%)4. Pretty Much (75%)5. Absolutely (100%)
![Page 43: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/43.jpg)
All faculty utilize a common frame of reference for resident assessment
1 2 3 4 5
0%
21%
7%
43%
29%
1. Not at all (0%)2. A Little (25%)3. About Half (50%)4. Pretty Much (75%)5. Absolutely (100%)
![Page 44: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/44.jpg)
Group assessments are employed in the rotation evaluation of residents
1 2 3 4 5
8%
29%
13%
21%
29%
1. Not at all (0%)2. A Little (25%)3. About Half (50%)4. Pretty Much (75%)5. Absolutely (100%)
![Page 45: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/45.jpg)
My program has a fully functional Clinical Competence Committee
1 2 3 4 5
18%
7%
43%
11%
21%
1. Not at all (0%)2. A Little (25%)3. About Half (50%)4. Pretty Much (75%)5. Absolutely (100%)
![Page 46: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/46.jpg)
My program’s Clinical Competence Committee is prepared for the NAS
1 2 3 4 5
12%
44%
8%
24%
12%
1. Not at all (0%)2. A Little (25%)3. About Half (50%)4. Pretty Much (75%)5. Absolutely (100%)
![Page 47: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/47.jpg)
All of my graduating residents are ready for independent practice
1 2 3 4 5
0%
12%
35%
42%
12%
1. Not at all (0%)2. A Little (25%)3. About Half (50%)4. Pretty Much (75%)5. Absolutely (100%)
![Page 48: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/48.jpg)
I know what kind/areas of faculty development my program needs to be ready for Milestone assessment and reporting
1 2 3 4 5
8%
23%
12%
42%
15%
1. Not at all (0%)2. A Little (25%)3. About Half (50%)4. Pretty Much (75%)5. Absolutely (100%)
![Page 49: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/49.jpg)
How do you translate competency- based info from rotational evals, 360 degree
evals, CEXs, etc., into Milestone assessment?
![Page 50: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/50.jpg)
Should specific Milestones be embedded into rotation
evaluations?
![Page 51: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/51.jpg)
Every milestone should be evaluated in multiple settings
Need to determine which milestones best go in which settings
Not everybody has to evaluate everything
Rotation evals may end up with comb of EPAs, milestone items, milestone sub-items and non-milestone items.
Guralnick, 2013 AGME Meeting
![Page 52: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/52.jpg)
How can you ensure that all faculty have a common frame of
reference for resident assessment?
![Page 53: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/53.jpg)
Competency Based AssessmentEffect of Construct Alignment
RESULTS AND OUTCOMES – CoBRA - Impact on grade inflation% of time an intern received an 8
23.0%
19.0%21.2%
25.0%
33.6%
17.1%
23.1%
9.7%11.1% 10.0% 10.3%
12.0%
9.9%10.5%
6.0%
3.9%3.0%
6.9% 6.9%
3.8%5.1%
4.0%
7.0%5.1%
10.8%
8.2%
5.7%6.8%
2.2%3.2%
1.6%
3.8%4.9%
1.6%2.9%
0.0%
5.0%
10.0%
15.0%
20.0%
25.0%
30.0%
35.0%
40.0%
Patient Care Medical Knowledge Practice Based Learning &
Improvement
Interpersonal & Communication Skills
Professionalism Systems‐Based Practice
OVERALL
ABIM 7/1/07 ‐ 2/1/09 2/1/09 ‐ 10/31/10 11/1/10 ‐ 7/1/11 7/1/11 ‐ 6/30/12
![Page 54: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/54.jpg)
Result and Outcomes – CoBRA – Impact on use of left side of scale% of time a PGY1 resident received a 3
Competency Based AssessmentEffect of Construct Alignment
0.0% 0.0% 0.0%
0.7%
0.0%
3.0%
0.2%0.6%
1.1%
2.5%
1.5%
0.6%
1.5% 1.3%
2.3% 2.2%
3.5%
3.0%
1.7%
3.0%2.6%
3.3%
1.3%
3.2%
1.3%1.9%
10.8%
3.6%
4.3%
5.4%
3.3%
6.5%
2.2%
10.9%
5.4%
0.0%
2.0%
4.0%
6.0%
8.0%
10.0%
12.0%
Patient Care Medical Knowledge Practice Based Learning &
Improvement
Interpersonal & Communication Skills
Professionalism Systems‐Based Practice
OVERALL
ABIM 7/1/07 ‐ 2/1/09 2/1/09 ‐ 10/31/10 11/1/10 ‐ 7/1/11 7/1/11 ‐ 6/30/12
![Page 55: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/55.jpg)
DIFFERENCE BETWEEN PGY 1 AND 3 RESIDENT ACROSS ACGME Competencies – Increased Assessor discrimination
Competency Based AssessmentEffect of Construct Alignment
0.74 0.75
0.62 0.63
0.46
0.73
1.051.10
1.04 1.00 0.99
1.15
1.38
1.24
1.03
0.81
1.09
1.46
1.261.18
1.04 1.03 1.03
1.30
0.00
0.20
0.40
0.60
0.80
1.00
1.20
1.40
1.60
1.80
2.00
Patient Care Medical Knowledge Practice Based Learning & Improvement
Interpersonal & Communication Skills
Professionalism Systems‐Based Practice
6/1/04 ‐ 6/30/07 (ABIM) 7/1/07 ‐ 10/31/10 (CoBRA) 11/1/10 ‐ 7/1/11 (CoBRA) 7/1/11 ‐ 6/30/12 (CoBRA)
![Page 56: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/56.jpg)
How do you know if your assessment strategies/tools really measure meaningful
outcomes?
![Page 57: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/57.jpg)
How do you know that your graduating residents are ready
for independent practice?
![Page 58: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/58.jpg)
There are a lot of Milestones and EPAs! How do you prioritize
them for your program?
![Page 59: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/59.jpg)
What should we (Community of SIU GME
Leaders) do next?
![Page 60: Milestones - Southern Illinois University School of Medicine...yReportable milestones will be set by RRCs Programs may have additional curricular milestones yMilestone data will be](https://reader033.vdocuments.mx/reader033/viewer/2022050120/5f500e7f7affc25ee42184ad/html5/thumbnails/60.jpg)
WHEW!!!!
THANK YOU