implementation and experience with a standardized
TRANSCRIPT
![Page 1: Implementation and Experience with a Standardized](https://reader033.vdocuments.mx/reader033/viewer/2022060505/6295b26bbcd5453b0c098bd6/html5/thumbnails/1.jpg)
Implementation and Experience with a Standardized Assessment of a
Clinical Encounter Report (STACER) in a Surgical Residency Program
Morgan Langille MD, Erin Wright MDCM, MEd
Division of Otolaryngology-Head & Neck Surgery University of Alberta
![Page 2: Implementation and Experience with a Standardized](https://reader033.vdocuments.mx/reader033/viewer/2022060505/6295b26bbcd5453b0c098bd6/html5/thumbnails/2.jpg)
Disclosure
• EDW • Speaker – Schering/Merck, GlaxoSmithKline • Travel Grant – Medtronic • Consultant – J & J Medical Products
![Page 3: Implementation and Experience with a Standardized](https://reader033.vdocuments.mx/reader033/viewer/2022060505/6295b26bbcd5453b0c098bd6/html5/thumbnails/3.jpg)
Introduction
• In – Training Evaluation Reports (ITER)
– End of rotation evaluation – Based on overall impression of resident’s progress
on a given rotation • Generally a summative assessment
![Page 4: Implementation and Experience with a Standardized](https://reader033.vdocuments.mx/reader033/viewer/2022060505/6295b26bbcd5453b0c098bd6/html5/thumbnails/4.jpg)
Introduction
• In – Training Evaluation Reports (ITER) – Ideally based on objective (and preferentially
DOCUMENTED) assessments • Comprehensive • Document direct observational evaluations
– Clinic – Surgical setting
![Page 5: Implementation and Experience with a Standardized](https://reader033.vdocuments.mx/reader033/viewer/2022060505/6295b26bbcd5453b0c098bd6/html5/thumbnails/5.jpg)
Evaluation – General Concepts
• Summative evaluation: – Can the resident perform the surgery successfully? – Typically quantitative
• Formative evaluation: – Evaluations intended as the basis for improvement – Offers residents feedback
![Page 6: Implementation and Experience with a Standardized](https://reader033.vdocuments.mx/reader033/viewer/2022060505/6295b26bbcd5453b0c098bd6/html5/thumbnails/6.jpg)
Evaluation: Formative vs. Summative
"When the cook tastes the soup, that’s
formative; when the guests taste the soup, that’s summative."
- Robert Stakes
![Page 7: Implementation and Experience with a Standardized](https://reader033.vdocuments.mx/reader033/viewer/2022060505/6295b26bbcd5453b0c098bd6/html5/thumbnails/7.jpg)
STACER
• Standardized Assessment of a Clinical Encounter Report – Initial version designed by Specialty Committee
(Otolaryngology-Head & Neck Surgery)
• Intent: – Enhance formative aspect of evaluation – Robust assessment of performance
• Direct and objective assessment • Immediate feedback
![Page 8: Implementation and Experience with a Standardized](https://reader033.vdocuments.mx/reader033/viewer/2022060505/6295b26bbcd5453b0c098bd6/html5/thumbnails/8.jpg)
![Page 9: Implementation and Experience with a Standardized](https://reader033.vdocuments.mx/reader033/viewer/2022060505/6295b26bbcd5453b0c098bd6/html5/thumbnails/9.jpg)
Surgical STACER items
1. Pre-operative plan 2. Knowledge of indications & contraindications 3. Obtains an informed surgical consent 4. Communication, plan of surgery, including
safe surgery checklist 5. Pre-incision planning, including equipment
preparation, imaging, patient positioning
![Page 10: Implementation and Experience with a Standardized](https://reader033.vdocuments.mx/reader033/viewer/2022060505/6295b26bbcd5453b0c098bd6/html5/thumbnails/10.jpg)
Surgical STACER items
6. Incision, flap elevation, exposure and ID of landmarks
7. Dexterity, hemostasis, surgical technique 8. Decision making related to inter-operative or
unexpected findings 9. Wound closure and dressing 10.Post-operative orders and communication of
care plan 11.Documentation of peri-operative encounter
![Page 11: Implementation and Experience with a Standardized](https://reader033.vdocuments.mx/reader033/viewer/2022060505/6295b26bbcd5453b0c098bd6/html5/thumbnails/11.jpg)
![Page 12: Implementation and Experience with a Standardized](https://reader033.vdocuments.mx/reader033/viewer/2022060505/6295b26bbcd5453b0c098bd6/html5/thumbnails/12.jpg)
Clinical STACER
1. History of presenting complaint 2. Co-morbid issues related to present
complaint 3. Past medical history, medication history 4. Social history 5. Organization of interview
![Page 13: Implementation and Experience with a Standardized](https://reader033.vdocuments.mx/reader033/viewer/2022060505/6295b26bbcd5453b0c098bd6/html5/thumbnails/13.jpg)
Clinical STACER
6. Physical examination 7. Case Presentation, including organization,
succinctness 8. Investigations, including interpretation of
past investigations 9. Diagnosis/Differential Diagnosis 10.Care Plan and Patient Counseling/Education 11.Surgical Consent (if applicable)
![Page 14: Implementation and Experience with a Standardized](https://reader033.vdocuments.mx/reader033/viewer/2022060505/6295b26bbcd5453b0c098bd6/html5/thumbnails/14.jpg)
![Page 15: Implementation and Experience with a Standardized](https://reader033.vdocuments.mx/reader033/viewer/2022060505/6295b26bbcd5453b0c098bd6/html5/thumbnails/15.jpg)
Implementation
• Staff and Residents oriented to STACER during
divisional grand rounds
• Residents asked to complete 2 STACERs per rotation: – 1 Clinical – 1 Surgical
![Page 16: Implementation and Experience with a Standardized](https://reader033.vdocuments.mx/reader033/viewer/2022060505/6295b26bbcd5453b0c098bd6/html5/thumbnails/16.jpg)
Implementation
• STACERs collected over 1 year
• Added to residents’ ongoing portfolio
![Page 17: Implementation and Experience with a Standardized](https://reader033.vdocuments.mx/reader033/viewer/2022060505/6295b26bbcd5453b0c098bd6/html5/thumbnails/17.jpg)
Feedback on STACER Program
• A Semi – Structured Questionnaire was
designed for STACER feedback
• Responses received from 100% residents and 60% of staff
![Page 18: Implementation and Experience with a Standardized](https://reader033.vdocuments.mx/reader033/viewer/2022060505/6295b26bbcd5453b0c098bd6/html5/thumbnails/18.jpg)
Semi – Structured Questionnaire
• What was your initial opinion/expectation when the STACER evaluation tool was implemented? – Skeptical – More paperwork – Time consuming – “Not another evaluation!” – Good idea
![Page 19: Implementation and Experience with a Standardized](https://reader033.vdocuments.mx/reader033/viewer/2022060505/6295b26bbcd5453b0c098bd6/html5/thumbnails/19.jpg)
Semi – Structured Questionnaire
• What were your impressions with the implementation process? – Easy – Multiple reminders helpful
• Program Director • Designated Resident
– Circulated in advance – Hard copies available in multiple locations – Several aspects were “not applicable”
• Variation in case selection
![Page 20: Implementation and Experience with a Standardized](https://reader033.vdocuments.mx/reader033/viewer/2022060505/6295b26bbcd5453b0c098bd6/html5/thumbnails/20.jpg)
Semi – Structured Questionnaire
• What was your initial experience with the STACER tool? – Somewhat unfamiliar at first – Confusion around logistics short lived – Many experienced no problems – Good “Snapshot” of performance
![Page 21: Implementation and Experience with a Standardized](https://reader033.vdocuments.mx/reader033/viewer/2022060505/6295b26bbcd5453b0c098bd6/html5/thumbnails/21.jpg)
Semi – Structured Questionnaire
• What is your current perspective of the STACER tool? – Provides useful feedback – Surgical STACER particularly relevant – Turns a typical case into a learning experience – Encourages honest assessment
![Page 22: Implementation and Experience with a Standardized](https://reader033.vdocuments.mx/reader033/viewer/2022060505/6295b26bbcd5453b0c098bd6/html5/thumbnails/22.jpg)
Semi – Structured Questionnaire
• What are the benefits/value of STACERs? (Has it made evaluation better and how?) – Learn tips and tricks that would otherwise be
missed – Residents use the STACER to “claim” a case – Facilitates discussion – Practical
![Page 23: Implementation and Experience with a Standardized](https://reader033.vdocuments.mx/reader033/viewer/2022060505/6295b26bbcd5453b0c098bd6/html5/thumbnails/23.jpg)
Semi – Structured Questionnaire
• What are the perceived downsides to STACERs? Challenges to their use? – Remembering to complete prior to end of rotation – Awkward having Staff watch clinical STACER – Only provide a single snapshot
![Page 24: Implementation and Experience with a Standardized](https://reader033.vdocuments.mx/reader033/viewer/2022060505/6295b26bbcd5453b0c098bd6/html5/thumbnails/24.jpg)
Semi – Structured Questionnaire
• What CanMEDS roles do you find best assessed by the STACER tool? – Manager – Health Advocate – Professional – Communicator – Collaborator
![Page 25: Implementation and Experience with a Standardized](https://reader033.vdocuments.mx/reader033/viewer/2022060505/6295b26bbcd5453b0c098bd6/html5/thumbnails/25.jpg)
STACER - Conclusions
• Benefits: – Enhances the validity of our ITERS
• Specific documentation and evidence
– Royal College reviewer liked our STACERs – Applicable to any surgery / Clinic – Standardized – Well received by our division
![Page 26: Implementation and Experience with a Standardized](https://reader033.vdocuments.mx/reader033/viewer/2022060505/6295b26bbcd5453b0c098bd6/html5/thumbnails/26.jpg)
STACER - Conclusions
• Drawbacks – Less objective outcome measures * – Lack of validation for specific procedures (but is
applicable to almost any procedure)