Find Your Place

Move to three different areas of the room:• Front: Sites seeking initial accreditation• Middle: Accredited sites with a self-study due in

6 months- 1.5 years• Back: Accredited sites with a self-study due in 1.5

years or more

S.O.S! Assistance Writing a Self-Study Under the Standards of

AccreditationTerri DeWalt, Ph.D., Matt Reiser, Ph.D., Carina Sudarsky-Gleiser, Ph.D., & Kenli Urruty, Ph.D.

September 25th 2018

Introductions

• Terri DeWalt- University of Wisconsin- Whitewater: • Seeking initial accreditation- program began in 2014, self-study submitted in March 2018, site

visit scheduled for Dec. 2018.• Matt Reiser - Southern Utah University

• APA Accredited- Sought initial accreditation, self study completed Dec 2016, site visit July 2017, 10 years accredited October 2017.

• Carina Sudarsky-Gleiser - William & Mary• APA Accredited- Sought initial accreditation, has completed several self-studies

• Kenli Urruty - University of Colorado Boulder• APA Accredited- Sought reaccreditation in 2017 with site visit in July 2017; program first

accredited in 2003

Agenda

• Brief, broad overview of Standards of Accreditation (SoA)

• Similarities and differences in G&P and SoA• Overview of SoA self-study

• Discussion of elements, activities, and outcomes

• Overview of required tables and other attachments

• Site Visit preparation

Learning Objectives

1. Participants will be able to identify key similarities and differences in G&P and SoA.2. Participants will be able outline the key steps in completing a self-study and site visit

under SoA.3. Participants will be able to develop a clear timeline for completion of the self-study.4. Participants will be able to describe the required components of the self-study,

including the required tables.5. Participants will be able to identify the differences between elements, activities, and

outcomes and how each is used in demonstrating compliance with SoA and intern competencies.

Take 5

• At your table discuss what you are hoping to get out of this presentation.

• Do you have specific questions about:• The differences between the G & P and SoA?• The process of writing a self-study?• The site visit process?• What are you most worried about in doing your self-study?• Other questions??

History and Overview of the SoA1995-1996 Guidelines and Principles for Accreditation in Professional Psychology (G&P) -adopted in 1995/ implemented in

1996

July, 2012 – February, 2015

APA moved through:1. A repeated process of questions and time for public comments, to a2. Roadmap presented for public feedback in council meetings and virtual town hall meetings, to the3. Release of a draft of the SoA for each level of accreditation at the CoA Open Forum during the 2013 APA

Convention, to 4. An updated draft for further public comment, to the 5. introduction of the SoA to APA Governance.

February, 2015

The Standards of Accreditation (SoA) are approved by the APA Council of Representatives replacing the G&P.

January, 2017 ● APA begins accrediting programs in health service psychology under the SoA. ● Programs submitting a self-study on/after 9/1/16 required to use the 2017 self-study instructions.● Revised version of the Accreditation Procedures and IRs got into effect on this date.

(https://www.apa.org/ed/accreditation/section-c-soa.pdf )

G&P and SoA - Similarities Setting & Resources:

● Training is an integral mission of the institution.● Budget/resources enable the program to achieve its goals.● Population is sufficient in number and variability.

Public Disclosure● Policies/Procedures/…..available to all interested parties● Accreditation status & contact for CoA - in public materials

.Diversity

● Efforts are made to recruit/retain diverse interns/faculty/staff● Didactic and experiential MC training is provided.● Engagement in periodic self-assessment re: training climate

Supervision● Min 4hrs/wk; at least 2hrs wk=individual.

Record Keeping● Periodical evaluations of interns ● Proximal & Distal data used for self-evaluation ● Outcome data demonstrates that graduates attained MLAs

Interns● Identifiable/qualified/sufficient in # body of interns● Training status/Title such as “Psychology Intern.”

Administrative Structure ● Training director is appropriately trained/credentialed. ● Sufficient number of trained/credentialed supervisors (minimum 2)

Communication ● Timely inform CoA of any substantial changes.● Communication between doctoral program and internship.

G&P and SoADifferences

G&P SoA

Scope of Accreditation Professional Psychology Health Service Psychology (HSP)*

Training Model Need to demonstrate how training aligns with a given training model.

No need to specify training model

Goals, Objectives, Competenciesvs.Aims and Competencies

Focus on program’s goals, objectives, and competencies.

Focus on aims and competencies.

G&P and SoADifferences

G&P SoA

Organization Domain A: Eligibility

Domain B: Program Philosophy, Objectives, and Training Plan

Domain C: Program Resources

Domain D: Cultural and Individual Differences

Domain E: Intern–Staff Relations

Domain F: Program Self-Assessment and Quality Enhancement

Domain G: Public Disclosure

Domain H: Relationship With Accrediting Body

Standards:

I. Institutional and Program ContextII. Program Aims, Learning Structure, and OutcomesIII. Students/Interns/ResidentsIV. Faculty/Supervisors/StaffV. Communication

Reorganization intended to better link training to outcomes, reflect the nature of training, and reduce redundancies. (Remained redundancies were intentional).

G&P and SoA - Differences (cont’d) G&P SoA

Competencies Programs selects competencies they intend to train interns on.

Nine (9) Profession-Wide Competencies:

1. Research2. Ethical and legal standards3. Individual and cultural diversity4. Professional values, attitudes, and behaviors5. Communication and interpersonal skills6. Assessment7. Intervention8. Supervision9. Consultation/interprof./interdisciplinary skillsOptional: program specific competencies related to program’s aims.

G&P and SoA - Differences (cont’d)

G&P SoA

Diversity Diversity = Domain D Diversity is a consistent theme throughout the SoA.

Direct Observation Not required. Evaluation is to be based in part on direct observation.

Communication between Doctoral and Internship Programs

Responsibility of communication between doctoral and internship programs placed on both entities. (Standard V.A.3 clarifies the nature of and the timeframe in which communication should occur.)

How the SoA Applies to the Self- Study and to Accreditation Decisions

Five principles that guide accreditation decisions; programs whose policies and procedures violate them, would not be accredited:1. Commitment to cultural and individual differences and diversity. 2. Broad and general preparation for practice at the entry level.3. Advanced preparation for practice at the postdoctoral level in a focus area and/or

recognized speciality. 4. Science and Practice: Promotion of the understanding of the value of science for

the practice of psychology and the value of practice for the science of psychology. 5. Program Aims and Trainee Competencies: “A program … [is} evaluated in light

of its educational aims…, the demonstrated competencies of its trainees, and the career paths of its graduates.

The Heart of the Matter

• Overview of SoA self-study (Standards I-V)

• Elements, activities, and outcomes

• Overview of required tables and other attachments

Standards

The self-study document is broadly organized by standards:

I. Institutional and Program ContextII. Aims, Training, Competencies, and Outcomes

III. InternsIV. Supervisors/Faculty/Staff/LeadershipV. Communication Practices

The Portalhttps://coaportal.apa.org/login

AimsEach program must provide information on the specific aims of the training program. The aim/s should align with the activities and intended outcomes.

• Colorado University Boulder: • The aim of the training program at CU Boulder is to train generalist health service psychologists in a

college mental health setting.• Southern Utah University:

• The overarching aim of SUU’s doctoral psychology internship is to prepare interns for competent, generalist entry-level psychological practice.

• University of Wisconsin- Whitewater:• To promote the development of clinical skills and professional identity of a generalist psychologist that

includes the provision of individual and group counseling, crisis intervention, and supervision within a framework of evidence based practice and professional ethics.

• William & Mary: • The program’s aim is to prepare entry level providers in health service psychology, skilled in

profession-wide competencies, to serve a diverse public.

Activities and Elements

• Activities: The training program is responsible for developing a clear and coherent plan for educational activities that support interns’ achievement of both Profession Wide Competencies (PWCs) and any program specific competencies.

• Learning elements: these are the building blocks for the activities and must meet the following criteria• Experiential• Include observation by psychologists• Sequential, cumulative, and graded in complexity

Outcomes

• By the end of internship, each intern must demonstrate achievement of the PWCs as well as any program-specific competencies.

• For each competency, you must:• Specify how you will evaluate the intern’s performance• Identify Minimum Level of Achievement (MLAs)• Provide data showing that interns met the MLAs• Base the evaluation at least in part on direct observation

Outcomes, cont’d

• Program alumni: provide data on how well the program prepared alumni to meet the PWCs and any specific competencies

• Self evaluation:• Involves stakeholders (i.e., staff, interns, graduates, others involved with program)• Has procedures for using proximal and distal data to monitor and make changes• Provides opportunities to enhance training quality through professional

development• Evaluates appropriateness of aims, activities, P&Ps in light of local, regional, and

national needs, standards, and evidence base for the profession

Tables for Standard I and IIStandard 1: Institutional and Program ContextTable 1: Program Policies and ProceduresConsortium form (if applicable)

Standard II: Aims, Training, Competencies & OutcomesTable 2: Profession-wide CompetenciesTable 3: Program-specific Competencies (if applicable)Table 4: Program Interns Post Internship Experience

Provide information below to illustrate how the program ensures that ALL interns can acquire and demonstrate substantial understanding of and competence in:

Competency: (i) Research

Elements associated with this competency from IR C-8 I

· Demonstrates the substantially independent ability to critically evaluate and disseminate research or other scholarly activities (e.g., case conference, presentation, publications) at the local (including the host institution), regional, or national level.

Program-defined elements associated with this competency

· NA

Required training/experiential activities to meet each element.

· Each professional issues seminar disseminates reading materials which interns are expected to critically evaluate.· Each intern presents a case conference at the end of the internship year and is expected to incorporate research evidence during the presentation.· Interns are encouraged to continually evaluate research evidence for therapeutic practices and discuss the empirical basis for their treatment approaches with primary and secondary supervisors.

How outcomes are measured for each training/experiential activity listed above. List where in the self-study all associated evaluation tools are located.

How outcomes are measured:·Items referencing “5” in the Competency column on each tab of the Intern Evaluation are directly assessing this competency

Evaluation tool and self-study location:· Intern Evaluation (Appendix II.D.1.a.1.5)

Minimum levels of achievement (MLAs) for each outcome measure/evaluation tool listed above.

MLA = 6.0 or higher on the Competency Summary at mid-year evaluation, and 7.0 or higher on the Competency Summary at the year-end evaluation

Table 2

Tables for Standard III: Interns

Table 5: Intern StatisticsTable 6: Intern DemographicsTable 7: Intern Professional ActivitiesTable 8: Program Intern Pre-Internship Education/ Experience

Tables for Standard IV: Supervisor/ Faculty/ Staff Leadership

Table 9.a: Current Program Training Supervisors Table 9.b: Current Other Agency/ Institutions SupervisorsTable 9.c: Current Other ContributorsTable 10: Current Training Supervisor DemographicsTable 11: Professional Activities

Steps and Timeline1. Become familiar with the requirements of SoA to ensure that program meets the

standards of the SoA.

2. Review the latest Self- Study Instructions: http://www.apa.org/ed/accreditation/about/policies/self-study-worksheets.aspx?tab=2

3. Advocate for uninterrupted time to work on the self-study

Steps and Timeline (con’td)

4. Find a process that works for you… Some examples include:

• Complete the 10 required policies and 11 Tables (really 14 tables…)i. Take your time on the Policies and the Profession/ Program Wide Competency Tables

• Begin with the shortest and easiest parts first

• Begin with the narrative and then completing the tables

Steps (Cont’d)

5. Begin the Self-Study - Recommendations from presenters based on their experiences:

a. Begin writing the self-study within the self-study worksheets (in a Word document) outside of the portal.

b. Working directly on the self-study within the CoA portal is not recommended as several TDs have lost their work due to the CoA portal malfunctioning.

c. Explore your director’s preference regarding review of information. If your director doesn’t have a strong preference, do you?

d. Transfer and save work from worksheets into the CoA portal on a regular basis. Required information can change as time goes on; usually we will hear through listserv.

Steps (Cont’d)

5. 3 months- 1 year (or more) later… complete self-study6. Possibly prepare an executive summary for administrators7. Send to Co-Signer (Vice Chancellor, etc.)8. Co-Signer returns to you9. Submit Self- Study- Celebrate!

Timeline

Starting point

Read Self-Study Template

Complete Policies(if necessary)

Complete Tables

Begin writing self-study

Age several years, gain grey hair, pull out hair...

Finish the Self-Study(you think)

Submit to Co-Signer

Co-Signer sends it back to you

Take a deep breathe and

hit the “Submit” button

Have 2 interns, financial and administrative support and energy drinks/ snacks ready

CELEBRATE!

What the Timeline Feels Like

Tips and Pointers

• Protect time to work on self-study/advocate with director for decrease in clinical load while working on self-study/having some uninterrupted time is key.

• Look at examples of self-studies on the ACCTA website• Form self-study email groups with other TDs who are working on their

self-study to ask questions, receive/offer support, etc. • Delegate tasks as appropriate• Get support from training committee

Differing Experiences

• We recognize that there are differing experiences for accredited vs non-accredited programs

• Programs already accredited:• Have a foundation to work from• History of interns• Specific date the Self-Study is due

Challenges Facing Programs Seeking Initial Accreditation

• Needing to start from scratch• Need 2 interns on site or committed to coming in order to submit self-study• Development of policies/ procedures, Training Manual, organizational buy in and structure• Financial resources

• Timeline• Trying to anticipate timeframe of steps needed to get accredited before the next Match Process• Submitting self-study before the wave of self studies on certain due dates (Jan. 1st, etc.)

• Going through the Match as an unaccredited site• How do we market ourselves• There are now more positions than there are interns

2018 APPIC Match Statistics

• Applicants: 3,779• Unmatched after Phase I & II: 143

• Internship Positions: 3,940• Accredited : 3,383• Non-Accredited: 523

• Total Unfilled Positions in Phase II: 380 • Accredited positions unfilled in Phase II: 162 • Non-accredited positions unfilled in Phase II: 218

https://appic.org/Internships/Match/Match-Statistics/Match-Statistics-2018-Combined

Resources for Sites Seeking Initial Accreditation• Accreditation Readiness Project (ARP) through APPIC

• https://www.appic.org/About-APPIC/Accreditation-Readiness-Project-ARP • Clover will begin 10/1/18 to recruit up to 12 programs to receive monthly consultation and support in the development of a self-study for

accreditation over the period of January-December, 2019.

• Selected programs will participate in Clover-administered “Learning Communities”, which will be comprised of small groups of internship programs. Clover will provide instructional webinars related to the accreditation process, monthly group Q&A sessions, tailored consultation to individual programs, and feedback on self-study documents.

• Consultants (Clover Educational Consulting Group, ACCTA members)

• State Training Director collaborations• APA Board of Educational Affairs Grants (ended in July 2017)

• APF/ National Register Internship Partnership fund (3 grants to increase # of internship slots, up to $15k, due 11/1/18) http://www.apa.org/apf/funding/internship-fund.aspx

Tips for Sites Seeking Initial Accreditation

• Be aware of when the CoA meets to make decisions about program accreditation and how this impacts your status for the upcoming match.

• Accreditation is awarded on the date of the CoA decision meeting• These meetings appear to occur in April, July and October • Site visit report needs to be submitted the month prior to the decision

meeting (i.e. in order for a decision to be made in October the site visit has to happen by mid- September.)

• http://www.apa.org/ed/accreditation/calendar.aspx

Take 5.2

At your table, discuss:

• The strengths and challenges you are bringing into this process. • What do you need to succeed in the process?

Now you’re on to the site visit!

• What/how to prepare

• Ways to prep staff

• Commonly asked questions

Site Visit Timeline

1. CoA provides authorization of site visit2. Preparation for site visit (once approved)3. Finding site visitors and selecting a date4. Actual site visit5. Report from site visit 6. CoA decision process of accreditation

CoA Decision Process of Site Visit

1. First come first serve, though the CoA only meets 3 times per year2. Decisions can be:

a. approved for site visitb. need additional informationc. denied site visit

3. Once approved you can schedule site visitors and a site visit date

Site Visitor Selection and Scheduling

1. The CoA will assign the chair of the site visit.2. You may select from 5 potential site visitors from a list generated from the

CoA (and you may need additional lists of 5 for more options). 3. The date may be collaborated with the site visit chair and your staff. Then

schedule with second site visitor. 4. Once a two day site visit has been scheduled, you can start to prepare.

Preparation for Site Visit

1. Preparing staff and organizing site visit participants2. Mock site visits with consultants3. Contact past interns and offices on campus that may participate4. Jeopardy FUN5. Example of site visit schedule

Preparing staff and partners

• Keep your staff up to date on the process of re-accreditation and start talking to them early about the function and purpose of the site visit

• Prepare written materials for constituents (staff, campus partners, administrators, etc.) to “study” prior to participating in the site visit

• Prepare current and past trainees for the questions they may get from site visitors

• Make preparation fun! Jeopardy is a great way to prepare everyone immediately before the site visit

Site Visit Agenda: Example Day 1

7:30 Pick up at hotel8:00-8:55 Welcome and tour of center9:00-9:55 Meeting with department leadership team10:00-10:55 Meeting with training team11:00-11:55 Meet with site supervisors12:00-12:55 Lunch1:00-1:55 Meet with current interns2:00-2:55 Meet with clinical team3:00-3:55 Meet with CAPS staff and campus partners4:00-4:55 Review intern files

Site Visit Agenda: Example Day 2

8:00-8:30 Meet with postdocs/practicum students8:30-9:00 Meet with administrative staff9:00-9:55 Meet with VC of SA and AVC of SA10:00-10:55 Meet with former interns 11:00-1:50 Work time/lunch2:00-3:30 Feedback to TD, leadership team, and all staff3:30 Site visitors depart for airport5:00

Frequently Asked Questions• Program strengths and areas for growth.• How is diversity woven into the department and program?• How do the department and training program fit within the mission of the university

and Division of Student Affairs?• How do staff provide modeling for interns?• Lots of questions about how evaluations are completed for each competency…• How do trainee cohorts (intern, postdoc, practicum, etc.) interact?• How do we get feedback from campus partners on consultation skills?• How do we collect feedback? Do we review it with interns? What do we do with

feedback?• What is the role of the training team? How does the team oversee training in the

department?

Actual Site Visit

1. Site visitors are the “eyes and ears” of the CoA2. They are ensuring your training program is doing what you say you are

doing in accordance to the SoA3. They make their own travel arrangements, but it is helpful to give them

helpful information about traveling to your site, as well as information on restaurants and things to do

4. You do not pay for meals for them during the site visit5. End of the 2nd day they give feedback on the program; then they write up

the site visit report

Site Visit Report and Follow Up

1. Site visitors have one month to turn in their report to the CoA. 2. The CoA will take some time to review it and to get it back to the program.3. Review the report and follow up on what additional information may be

needed. 4. Turn in what may be needed for CoA decision.

CoA Decision of Accreditation

1. Decisions are made in the April, July & October meetings2. Could be approved for 7-10 years3. Could ask for additional information4. Could be denied Accreditation5. If accredited it is now from the date of when the CoA met and gave the

decision; not when the site visit happened.

Take 5.3: Your Goals

• Take 3-5 minutes to write down goals to help you prepare for your next self-study.

• Discuss your goals with your small group.

Your Thoughts

• What did you learn from the presentation?• What insights do you have to share?• What questions or comments do you have?