University of ManitobaPre-Survey Meeting withProgram Administrators

Date: July 3, 2013Time: 2:00 to 4:00 p.m.Room: Pharmacy Apotex, Theatre # 264

Objectives of the Meeting

•Provide an overview of• Accreditation Process• Questionnaire and Appendices• Schedule• Standards

•Opportunity to network with colleagues

• Who has been involved in a survey process?• Last regular survey in 2008• External review• Internal review

• Is a process to:• Improve quality of postgraduate medical education

• Provides a means for assessment of residency programs

• Assists program director in reviewing conduct of program

• Based on Standards

Accreditation

Internal Reviews

1

2

34

5

6

Monitoring

Six Year Survey Cycle

Pre-Survey Process

Royal CollegeComments

Questionnaires

University

Specialty Committee

Questionnaires

Questionnaires &Comments

Program Director

Comments

Surveyor

•First impression of the program by the surveyor is the questionnaire and appendices• All sections of the questionnaire are

complete• Error-free vs typos• Neat vs sloppy • Clear descriptions vs confusing information• Spell out abbreviations

Questionnaire and Appendices

When submitting your questionnaire and appendices please format and name them as follows:

Questionnaire • Each individual section (i.e. each standard) is

submitted as a separate PDF file• The files are named as:

• General_Information.pdf, Standard_B1.pdf, Standard_B2.pdf, etc.

Questionnaire and Appendices

Appendices• For each appendix, the name of the appendix (e.g.

Appendix A) is clearly indicated on the first page.• Each individual appendix is submitted as a separate PDF

file.• The files are named as: Appendix_A.pdf, Appendix_B.pdf,

etc.• Any additional attachments are labelled/formatted as above

with the name corresponding to the appendix name as listed in the General Information section of the PSQ (e.g. Appendix_K.pdf, Appendix_L.pdf, Appendix_M.pdf, etc.)

Questionnaire and Appendices

• Review – Length & Day• Based on• Number of residents in program• Traveling to different sites• Last schedule in 2008

• Coordinated by the Royal College and PGME office• Inform PGME office if you foresee a scheduling

conflict• PGME office will notify the program of the day of the

review

Surgical Foundations will be reviewed on Monday morning.

The Survey Schedule

• Transportation• Arranged by program• Include details • Name & contact info for person responsible

• Morning• Pick-up at 07:45 a.m.

• At the Fairmont Winnipeg• To and from different sites• When required during review

• Afternoon• Survey must end no later than 4:00 p.m.

• At the Fairmont Winnipeg

The Survey Schedule

• Sequence to follow• Document review (30 min)• Program director (75 min)• Department head * (30 min)• Residents * (per group of 20 - 60 min) • Teaching faculty * (60 min)• Residency Program Committee (60 min)• Exit Meeting (15 min)

• Morning after the review concluded• 07:30-07:45 a.m. at the Fairmont Winnipeg

* Scheduled anytime after program director but before RPC

The Survey Schedule

Document Review (30 min)• Scheduled at beginning of schedule• Available in meeting room

• Residency Program Committee minutes - last 6 years• Resident assessment files

• If automated, arrange access to a computer, make sure that it is easy to navigate and retrieve

• Provide assistance/demo• Remember surveyors have limited time to review

the documents on-site

The Survey Schedule

• Program director (75 min)• Scheduled after document review

• To discuss• Overall view of program• Evaluation of Standards• Strengths, challenges and weaknesses• Specialty Committee Comments

− Sent by email to program director week prior to on-site visit

The Survey Schedule

• Department head (30 min)• Scheduled anytime after program director but before RPC

• To discuss• Support for program director and program• Resources available to program

The Survey Schedule

• Residents (60 min)• Scheduled anytime after program director but before RPC• Groups of 20 residents

− Can be organized in various groups By postgrad year By Junior & Senior years

• Encourage residents to:• Complete the ‘confidential’ CAIR questionnaire• Sent by CAIR

• Meet as a group to discuss strengths & challenges prior to visit

• Send reminders – date, time, locations

The Survey Schedule

• ALL residents invited to attend/participate in their resident meeting(s)? • Residents who cannot attend• Arrange link by video- or tele- conferencing

• Are visa trainees or (clinical) fellows invited to attend?• Different funding / title across Canada

• For accreditation purposes only, a resident is a person who is following the same academic program/training as a resident eligible to write College exams – funding doesn’t matter !

The Survey Schedule

• Teaching faculty (60 min)• Scheduled anytime after program director but before RPC

• To discuss• Involvement with residents• Communication with program director• Teaching faculty who sit on RPC need only attend RPC

meeting

The Survey Schedule

• Tours (15 min)• Optional

• Tours of wards/clinics not necessary• Is there something special to showcase?

• MUST be scheduled before RPC

The Survey Schedule

• Residency Program Committee (60 min)• Program director attends first half of meeting• Resident representative(s) attend the meeting• Smaller programs• Often faculty all sit on RPC• Can combine faculty & RPC together into one meeting

• MUST be scheduled as LAST meeting of review

The Survey Schedule

• Survey Team Discussion• Your program will be discussed the evening of the day that

your review ends• Identify the strengths & weaknesses• Provide a recommendation

The Survey Schedule

• Chair - Dr. Sarkis Meterissian– Responsible for general conduct of survey

• Deputy chair – Dr. Maureen Topps– Visits teaching sites / hospitals

• Surveyors • Resident representatives – CAIR• Regulatory authorities representative – FMRAC•Teaching hospital representative – ACAHO

The Survey Team

New terminology – June 2012• Approved by the Royal College, CFPC and CMQ.

• Recommendation• Accredited program• Follow-up:

− Next regular survey− Progress report (Accreditation Committee)− Internal review− External review

• Accredited program on notice of intent to withdraw accreditation• Follow-up:

− External review

The Survey Schedule

• Exit Meeting (15 min)• Morning after review • 07:30 – 07:45 a.m.• At the Fairmont Winnipeg

• Program director will be informed • Survey team recommendation• Strengths & weaknesses of program

The Survey Schedule

BE PROACTIVE

• Start early• Share the information with your program

director•Make a plan and set timelines• Create and use your own checklist• Consult with the PGME office• In doubt: ask questions

The Survey Schedule

• Be available to assist the surveyors as needed.• ‘Do Not Disturb – Meeting in Progress’ sign on door.• Ensure that the surveyors are comfortable (i.e.:

private room, water, a layout of the area, etc.)• Arrange for beverages, healthy snacks for breaks

and a lunch, when appropriate.•Organize all the required documents in the room

and ensure that there is a computer with internet access available to the reviewers.

• Retrieve all confidential documents from the room(s) at the end of the day.

Day of the Survey

• Final exit with University• Friday, February 28, 2014 between 9:00 and 10:00 a.m.• CFPC/Royal College Chairs present a summary of survey week

• Survey reports sent to PGME• 6 to 8 weeks post-survey

• Program response sent to College• Report any errors of fact only

• Accreditation Committee Decisions• May/June 2014• Dean & postgraduate dean attend• Appeal process is available

After the Survey

“A” Standards• Apply to University, specifically the PGME office

“B” Standards• Apply to EACH residency program• Updated January 2011

“C” Standards• Apply to Areas of Focused Competence (AFC)

programs

General Standards of Accreditation

•Applicable to the University•Reviewed by chair’s team (Team A)• Schedule organized by PGME office

•Looking at• Structure in place• Training sites• Liaison with hospitals

“A” Standards

•Applicable to ALL programs•Reviewed by surveyors•Looking at

B1 Administrative StructureB2 Goals & ObjectivesB3 Structure and OrganizationB4 ResourcesB5 Clinical, Academic & Scholarly ContentB6 Assessment of Resident

http://www.royalcollege.ca/portal/page/portal/rc/common/documents/accreditation/accreditation_blue_book_b_standards_e.pdf

“B” Standards

There must be an appropriate administrative structurefor each residency program.

• Program director• Time & support

• Residency Program Committee• Operation of the program• Program & resident evaluations• Appeal process• Selection & promotions of residents• Process for teaching & assessment of competencies• Research• Regular review of program• Faculty assessments

B1 – Administrative Structure

•Program director autocratic•Residency Program Committee dysfunctional

– Unclear Terms of Reference (membership, tasks and responsibilities)• Agenda and minutes poorly structured• Poor attendance

– Department head unduly influential– RPC is conducted as part of a Dept/Div meeting

•No resident voice

B1 – Administrative Structure“Pitfalls”

There must be a clearly worded statement outlining the Goals & Objectives of the residency program.

• Rotation-specific• Address all CanMEDS Roles• Functional / used in:• Planning• Resident assessment

• Distributed to residents & faculty• Reviewed regularly• At least every 2 years

B2 – Goals and Objectives

B2 – Goals & Objectives“Pitfalls”

•Missing CanMEDS roles in overall structure– Okay to have rotations in which all CanMEDS

roles may not apply (research, certain electives)•Goals and objectives not used by

faculty/residents•Goals and objectives dysfunctional – does

not inform assessment•Goals and objectives not reviewed regularly

There must be an organized program of rotations and other educational experiences to cover the educational

requirements of the specialty.

• Increasing professional responsibility• Senior residency• Service / education balance• Resident supervision• Clearly defined role of each site /

rotation• Educational environment

B3 – Structure & Organization

•Graded responsibility absent•Service/education imbalance

– Service provision by residents should have a defined educational component including evaluation

•Educational environment poor

B3 – Structure & Organization “Pitfalls”

There must be sufficient resources –Specialty-specific components as identified by

the Specialty Committee.

• Number of teaching faculty• Number of variety of patients,

specimens and procedures• Technical resources• Ambulatory/ emergency /community

resources/experiences• Educational

B4 - Resources

• Insufficient faculty for teaching/ supervision• Insufficient clinical/technical resources• Infrastructure inadequate

B4 – Resources “Pitfalls”

The clinical, academic and scholarly content of the program must prepare residents to fulfill all the Roles of the

specialist.

• Educational program• Organized curriculum

- Content specific areas defined by Specialty Committee• CanMEDS Roles• Teaching of the individual competencies

B5 – Clinical, Academic & Scholarly Content of Program

•Organized academic curriculum lacking or entirely resident driven

– Poor attendance by residents and faculty•Teaching of essential CanMEDS roles missing•Role modelling is the only teaching modality

B5 – Clinical, Academic & Scholarly Content of Program “Pitfalls”

Appendix ‘H’Formal Academic Curriculum

•The surveyor will be verifying that the content of the academic curriculum covers all the of the CanMEDS Roles.

B5 – Clinical, Academic & Scholarly Content of Program

There must be mechanisms in place to ensure the systematic collection and interpretation of assessment

data on each resident.

•Based on objectives•Include multiple assessment techniques•Regular, timely, formal•Face-to-face

B6 – Assessment of Resident Performance

• Mechanism to monitor, promote, remediate residents lacking

• Formative feedback not provided and/or documented

• Assessments not timely, not face to face• Summative evaluation (ITER) inconsistent

with formative feedback, unclearly documents concerns/ challenges

B6 – Assessment of Resident Performance “Pitfalls”

44

University of ManitobaRegular on-site Survey

February 23 to 28, 2014

accred@royalcollege.ca613-730-6202

Office of Education

Sarah TaberAssistant Director

Education Strategy & Accreditation

Educational Standards Unit

Sylvie LavoieSurvey Coordinator

Contact Information at theRoyal College