scholarly activity evaluation to adhere to the acgme outcome

Dr. Suporn Sukpraprut-Braaten, MSc, MA, PhD

Dr. Dewey McAfee DO, FACOFP, PharmD

Dr. David Covey, MD, FACP

Graduate Medical Education

Unity Health

Searcy, Arkansas

Scholarly Activity Evaluation to Adhere to the

ACGME Outcome-based Milestones

Unity Health is the largest employer in an eight surrounding county area in with more than 2,000

associates located in Searcy, Newport, and Bradford, Arkansas.

The facility has a combined total of 438 licensed beds and a medical staff of more than 150

physicians that specialize in various areas of healthcare.

INTRODUCTION

Unity Health – White County Medical CenterLocation of White County, Arkansas Unity Health – Harris Medical Center2

UNITY HEALTH – WHITE COUNTY MEDICAL CENTER

3

54 miles to Little Rock

104 miles to Memphis

ARCOM

NYIT

INTRODUCTION

Dr. Suporn Sukpraprut-Braaten

Assistant Professor for Research, Unity Health

Vice Chair on the Scholarly Activity & Faculty Development Committee of Kansas City

University of Medicine and Biosciences (KCUMB) - GME Consortium

Adjunct Clinical Assistant Professor of Preventive Medicine, Department of Specialty

Medicine, KCUMB

Adjunct Clinical Assistant Professor in the Department of Academic Affairs, Arkansas

College of Osteopathic Medicine (ARCOM), Fort Smith, AR

Former Director of Preventive Medicine Research Center and Assistant Professor of

Preventive Medicine and Public Health at Edward Via College of Osteopathic Medicine

(VCOM) – Virginia Campus

4

Unity Health’s Osteopathic Residency Program

Dr. Dewey McAfee

Director of Medical Education

Family Medicine Program Director

Dr. David Covey

Internal Medicine Program Director

Arkansas’s White County Health Officer

Dr. James S. Stanley

Psychiatry Program Director

KEY PERSONNEL

5

Dr. John Henderson

Chief Medical Officer

Dr. Delaney Kinchen

Institutional Education Officer

Dennis Yecke, MBA

Operations Director

Unity Health’s Osteopathic Residency Program:

Unity Health’s Residency Program is affiliated with Kansas City University of

Medicine and Biosciences – Graduate Medical Education Consortium (KCU-GME

Consortium).

The program is the first osteopathic residency program in the state of Arkansas.

The program is the second residency program in the state after a residency program at

University of Arkansas Medical Center (UAMS).

RESIDENCY PROGRAM

6

Unity Health’s Osteopathic Residency Program:

The program was started in July, 2015 and approved for three specialty programs

Family Medicine – 3 PGY1 and 1 PGY2

Internal Medicine – 5 PGY1 and 2 PGY2

Psychiatry – 3 PGY1 and 1 PGY2

RESIDENCY PROGRAM

7

Common ProgramAOA Requirements ACGME Requirements

AOA Basic Documents for Postdoctoral Training – Research &

Scholarly Activity:

IX.D Research and Scholarly Activity

9.1 Each OPTI shall require each member institution to establish policies

and guidelines that govern scientific research activities in accordance

with local, state and federal guidelines.

9.2 Each OPTI shall facilitate and provide research education, assistance

and resources directly to trainees and institutions to encourage

research and to meeting the Specialty College requirements.

9.3 Each OPTI shall provide in collaboration with its member COM(S),

hospitals and other teaching institutions access to basic science

and/or clinical research mentorship.

9.4 The OPTI shall support and provide a mechanism to recognize

trainees who conduct research activities.

9.5 The OPTI shall provide budgeted funding for OPTI-wide or program

specific research for its trainees.

9.6 The OPTI shall demonstrate its support of trainee scholarly activity.

ACGME Common Program Requirements – Residents’

Scholarly Activities:

IV.B Residents’ Scholarly Activities

IV.B.1 The curriculum must advance residents’ knowledge of

the basic principles of research, including how research

is conducted, evaluated, explained to patients, and

applied to patient care.

IV.B.2 Residents should participated in scholarly activity.

IV.B.3 The sponsoring institution and program shall allocate

adequate educational resources to facilitate resident

involvement in scholarly activities.

8

Family Medicine Program

AOA Requirements ACGME Requirements

Family Medicine:

VII.B.7.1.e) Attend a minimum of one of the following: National

American College of Osteopathic Family Physicians (ACOFP)

annual convention or the AOA Osteopathic Medical Conference &

Exposition (OMED), registered as an ACOFP Family Physician.

VII.C Research and Scholarly Activity Requirements

VII.C.7.1 The participation of each resident in a scholarly activity

is required. This requirement can be met by participating in or

completion of any of the following:

a. Resident scholarly project within the department of family

medicine

b. Institutional or regional research programs in which family

physicians are actively involve

c. Authoring an original paper on a health-care related topic

d. Presentation at a state, regional, or national meeting

e. Authoring a grant

Family Medicine:


IV.B.1 The curriculum must advance residents’ knowledge of the

basic principles of research, including how research is

conducted, evaluated, explained to patients, and applied to

patient care.*

IV.B.2 Residents should participate in scholarly activity.*

IV.B.2.a) Residents should complete two scholarly activities,

at least one of which should be a quality improvement

project.

IV.B.3 The sponsoring institution and program should allocate

adequate educational resources to facilitate resident

involvement in scholarly activities. *

*Core requirement9

Internal Medicine Program

AOA Requirements ACGME Requirements

Internal Medicine:

VII. Resident Requirements

VII.7.7 The residents must attend the American College of

Osteopathic Internists (ACOI) Annual Convention

and Scientific Sessions or another ACOI

continuing education program once during the

training program.

VII.7.8 The resident must complete a scholarly project that

is approved by the program director and submitted

for publication or presented at a scientific meeting,

or participate in two critiqued evidenced-based

presentations.

Internal Medicine:


IV.B.1 The curriculum must advance residents’

knowledge of the basic principles of research,

including how research is conducted, evaluated,

explained to patients, and applied to patient care.*


IV.B.3 The sponsoring institution and program should

allocate adequate educational resources to

facilitate resident involvement in scholarly

activities.*

*Core requirement

10

Psychiatry ProgramAOA Requirements ACGME Requirements

Psychiatry:

VII. Resident Requirements

VII.B.4 Write one (1) professional research project

pertinent to psychiatry which is suitable for

publication.

The paper shall be submitted the American

College of Osteopathic Neurologists and

Psychiatrists (ACONP) during the last year of

training prior to completion of the residency

training.

Psychiatry:


IV.B.1 The curriculum must advance residents’ knowledge of the basic

principles of research, including how research is conducted, evaluated,

explained to patients, and applied to patient care.*


IV.B.2.a) The program must provide residents with the opportunities for research

and development of research skills for residents interested in

conducting research in psychiatry or related fields.

IV.B.2.b) The program must provide interested residents access to and the

opportunity to participate actively in ongoing research under a mentor.

IV.B.2.c) All residents must be educated in research literacy and in the concepts

and process of evidence-based clinical practice to develop skills in

question formulation, information searching, critical appraisal, and

medical decision-making.

IV.B.3 The sponsoring institution and program should allocate adequate

educational resources to facilitate resident involvement in scholarly

activities.*

*Core requirement 11

SUMMARY

Common Program Requirements:

I. Establish policies and guidelines that govern scientific research activities

II. Provide research education and establish a curriculum to advance residents’ knowledge

of the basic principles of research

III. Provide access to basic science and/or clinical research mentorship

IV. Allocate adequate educational resources to facilitate resident involvement in scholarly

activities and provide budgeted funding for the residents

V. Have the residents participate in scholarly activity

12

I. Establish policies and guidelines that govern scientific research activities

13

II. Establish a curriculum to advance residents’ knowledge of the basic principles of research

Unity Health’s GME has created a course titled “Principles of Evidence Based Medicine and Research”.

Unity Health’s GME has created:

A course titled “Principles of Evidence Based Medicine and Research”.

The curriculum is designed to advance the residents’ knowledge of basic principles of evidence

based medicine and research.

Monthly Meeting with Research Mentors

14

II. Establish a curriculum to advance residents’ knowledge of the basic principles of research

Course Structure:

The course is started in January and end June during the noon didactic once a month.

At the end of each lecture, there is a workshop section to go through some practice questions and

calculations related to the materials in the lecture.

Lecture Description

1 The Foundation of Evidence Based Clinical Practice and

Hierarchy of Research Evidence and Study Designs

2 Methods and Requirements for using of human subjects for Institutional Review Board (IRB)

3 Validity of Diagnostic Test and Random Error

4 Types of Data, Test Hypothesis, and Statistical Analyses

5 Understand How to Interpret the Results and Translate the Evidence to Patient Care

6 Research Biases: Sources of Bias and Strategies for Minimizing Biases15

III. Provide access to basic science and/or clinical research mentorship

Unity Health’s GME allocates two types of research mentors to provide assistance regarding

research methodology and clinical guidance to facilitate the residents’ involvement in

scholarly activities.

Clinical Research Mentor is an expert in the field of the resident’s research area of interest.

The clinical research mentor will help the resident by:

Approving the research topic and study design

Determining an appropriate conference and journal in order to submit the research poster

and/or manuscript

Approving the final abstract, poster, and/or manuscript

16

Study Research Mentor is an expert in study design and research methodology.

The study research mentor will help the resident by:

Teaching the residents to understand principles of research and become competent in

research skills

Advising the residents in designing a research study, including providing an assistance in

choosing an appropriate data set, statistical analysis, interpretation of results, conducting a

research poster, and/or writing a manuscript

17

IV. Allocate adequate educational resources to facilitate resident involvement in

scholarly activities and provide budgeted funding for the residents

Unity Health’s GME allocates the following educational resources:

For internal medicine and family medicine residents:

1. Harrison’s Principles of Internal Medicine Textbook

2. American College of Physicians (ACP) MKSAP 17 Part A and B books

For psychiatry residents:

1. Kaufman’s Clinical Neurology for Psychiatrists Textbook

2. Stahl’s Essential Psychopharmacology Textbook

3. The Human Brain Book18

Unity Health’s GME allocates the following educational resources to all of the residents:

1. Microsoft Surface Pro to allow the residents to access patient’s information and search for

research articles

2. Educational stipend:

$1,000 for the PGY1 and PGY2

$1,500 for the PGY3 and PGY4

3. Scholarship fund to help the residents to pay for conference attendance and presentation

4. Monthly meeting with the research mentors

5. High quality noon didactic lectures with the CME credit available for faculty and free lunch

6. Access to the KCUMB library

7. Discount rate for an access to the COMBANK to help the residents to prepare for the

COMLEX-3

8. Provide a dedicated time for the residents to prepare for their special board exams19

V. Have the residents participate in scholarly activity

A. Scholarly activity productivity

B. Expectation for graduation

C. Research timeline

D. Scholarly activity evaluation:

1. Research Responsibility

2. Research Project or Case Report Progression

3. Scholarly Activity Productivity

4. ACGME – Milestones

5. Overall Competency

20

V.A: Scholarly Activity Productivity

Required Scholarly Activity:

Level 1: Literature Review

The residents are required to conduct a literature review for the following activities:

Research project or case report

Noon didactic

Journal club

Patients support group

Level 2: Attend a regional or national medical conference

21


Level 2: Attend a regional or national medical conference (required)

Family Medicine Residents:

Attend the American College of Osteopathic Family Physicians (ACOFP) or AOA OMED

Conference (AOA VII.B.7.1.e)

Internal Medicine Residents:

Attend the American College of Osteopathic Internists (ACOI) Annual Convention and

Scientific Sessions (AOA VII.7.7)

Psychiatry Residents:

Attend the American College of Osteopathic Neurologists and Psychiatrists (ACONP)

(AOA VII.7.7)22


Optional Scholarly Activity:

†Level 4 is designed as a graduation target.

The residents are not required to be listed as the primary investigator or the first author of the research poster or research article.

Level 3: Conduct a case report and a poster presentation at a regional or a national conference

Level 4.1†: Conduct an original research project and a poster presentation at a regional or a

national conference

Level 4.2†: Publish an original research article in a peer review scientific journal

Level 4.3†: Submit a grant proposal as a principal investigator or a co-investigator

Level 5.1: Write a book chapter in a textbook related to medicine

Level 5.2: Receive a grant award as a as a principal investigator or a co-investigator

23

V.A: Expectation for Graduation

1. Every resident is required to provide noon didactics, conduct literature review for his/her case report

or research project, present/attend journal clubs and patients support group, and attend a medical

conference

2. Every resident is required to accomplish one of the following four options of scholarly activity

productivities in order to graduate from Unity Health’s residency program

Option 1: Two case reports for poster presentations (Level 3)

Option 2: An original research project for a poster presentation or publication (Level 4†)

Option 3: A grant application (Level 4†) or grant award (Level 5)

Option 4: Accomplish a submission of a text book chapter (Level 5)

†Level 4 is designed as a graduate target. 24

V.B Research Project Timeline

25

V.B Case Report Timeline

26

V.C: Scholarly Activity Evaluation

Clinical Competency Committees (CCC) of each program evaluate and monitor the residents’

scholarly activity and progression.

A CCC is the ACGME required body comprising three or more members of the active

teaching faculty who is advisory to the program director and reviews the progress of all

residents in the program1.

The evaluation is conducted semi-annually.

The evaluation results are reported to the GME Committee.

1Source: Andolsek K, Padmore J, Hauer KE, and Holmboe E. “A Guidebook for Programs: Clinical Competency Committees.” Accreditation Council

for Graduate Medical Education, Jan. 2015. Web. 1 April 2016.

27

V.C: Scholarly Activity Evaluation

The residents are evaluated according to these sections:





5. Overall Competency

28


29


PGY 1

30


PGY 2

31


PGY 3

32


33


The ACGME Review Committee will examine milestone performance data for each program’s residents

as one element in the Next Accreditation System (NAS) to determine whether residents overall are

progressing according to the following areas:

34

PBLI – Practice-Based Learning and Improvement

PROF – Professionalism

C - Communication

PC – Patient Care

MK – Medical Knowledge

SBP – System-Based Practice


Family Medicine ACGME – Milestones

PBLI-1: Locates, appraises, and assimilates evidence from scientific studies related to the

patients’ health problems

PBLI = Practice-Based Learning and Improvement †Level 4 is designed as the graduation target

PBLI-1: Locates, appraises, and assimilates evidence from scientific studies related to the

patients’ health problems

Milestones Achievement

Month

Level Competency Component 6 12 18 24 30 36

Level 1 Able to describe basic concepts in clinical epidemiology, biostatistics, and clinical

reasoning ☐ ☐ ☐ ☐ ☐ ☐

Level 1 Know different categories of research study design ☐ ☐ ☐ ☐ ☐ ☐

Level 2 Able to identify pros and cons of various study designs, associated types of bias, and

patient-centered outcomes ☐ ☐ ☐ ☐ ☐ ☐

Level 2 Able to formulates a searchable question from a clinical question ☐ ☐ ☐ ☐ ☐ ☐

Level 2 Able to evaluates evidence-based point-of-care resources ☐ ☐ ☐ ☐ ☐ ☐

Level 3 Able to apply a set of critical appraisal criteria to different types of research, including

synopses of original research findings, systematic reviews and meta-analyses, and

clinical practice guidelines

☐ ☐ ☐ ☐ ☐ ☐

Level 3 Able to critically evaluate information from others, including colleagues, experts, and

pharmaceutical representatives, as well as patient-delivered information ☐ ☐ ☐ ☐ ☐ ☐

Level 4† Able to incorporates principles of evidence-based care and information mastery into

clinical practice ☐ ☐ ☐ ☐ ☐ ☐

Level 5 Able to independently teach and assess evidence-based medicine and information

mastery techniques ☐ ☐ ☐ ☐ ☐ ☐

35

Family Medicine ACGME – Milestones

PBLI-2: Demonstrates Self-Directed Learning

PBLI = Practice-Based Learning and Improvement †Level 4 is designed as the graduation target

36

Internal Medicine ACGME – Milestones

PROF-4: Exhibits Integrity and Ethical Behavior in Professional Conduct

i. Critical Deficiencies: Dishonest in clinical interactions, documentation, research, or scholarly activity

ii. Professionalism Early Concern Note will be written if the resident is suspected of dishonesty in

reporting his/her scholarly activities or plagiarism on the research abstract, poster, or manuscript

PROF = Professionalism

PBLI = Practice-Based Learning and Improvement †Ready for Unsupervised Practice: This level is designed as the graduation target

37


PBLI-2: Learns and Improves via Performance Audit


38


PBLI-4: Learns and Improves at the Point of Care


39

Psychiatry ACGME – Milestones

PBLI-1: Development and execution of lifelong learning through constant self-evaluation, of

research and clinical evidence

A: Self-Assessment and self-improvement

B: Evidence in the clinical workflow

PBLI = Practice-Based Learning and Improvement

40


PBLI-1: (Continue)

PBLI = Practice-Based Learning and Improvement †Level 4: This level is designed as the graduation target

41


PBLI-3: Teaching

A: Development as a teacher

B: Observable teaching skills

PBLI = Practice-Based Learning and Improvement

42

V.E: Overall Competency in Scholarly Activity

43

Research Project Team Structure

44

Family Medicine Residents

45

Internal Medicine Residents

46

Psychiatry Residents

47

Medical Students (OMS III)

48

Ray Montgomery, FACHE – Unity Health’s President and CEO

John Henderson, M.D., FACC – Chief Medical Officer

Dewey McAfee, D.O., FACOFP, PharmD – Director of Medical Education/Family Program Director

James S. Stanley, M.D., FANP – Psychiatry Program Director

Delaney Kinchen, D.O., FAEP – Institutional Education Officer

Dennis Yecke, MBA – Operations Director, Graduate Medical Education Program

Rachael Sowers, MBA – Program Coordinator , Graduate Medical Education Program

Leslie Provence, MBA – Program Coordinator , Graduate Medical Education Program

Kelly Bopp – Administrative Assistant, Graduate Medical Education Program

49

THANK YOU

50