Clues for Solvingthe Mystery of the

Fifth-Year Interim Report

2013 SACSCOC Annual MeetingDecember 7-10, 2013

Atlanta, Georgia

UCF Team

Tace Crouse –coordinator of UCF’s Fifth-Year Report and peer reviewer

Diane Chase – current SACSCOC liaison for UCF and peer reviewer

Heidi Watt – member of UCF Fifth-Year Report leadership team and manages on-going SACSCOC compliance activities

Denise Young – former SACSCOC liaison for UCF and peer reviewer (including Fifth-Year reports)

2

University of Central Florida

3

60,000 students200+ programs

Multiple campuses Research I UniversitySACSCOC Level VI

• second largest university in U.S.• one of twelve Florida state universities

4

Workshop Outcomes

To prepare for their next report, participants will analyze the Fifth-Year Interim Report process including the:

previous performance of institutions new and most challenging standards litmus test questions technology tools evaluator’s perspective organizational matrix

Workshop Agenda

background check

providing clues for preparing accreditation reports with a focus on the SACSCOC

Fifth-Year Interim Report input from peers

Evaluator perspectives and input into an Organizational Matrix are embedded throughout to provide you with clues for how your work might be reviewed and a jump start on your next report!

5

Background Check of Fifth-Year Report

Why a fifth-year report requirement? response to federal oversight (Higher Education

Opportunity Act) ten-year span for major review deemed too long

What is the SACSCOC Fifth-year Report?

four or five-part report describing your institution, how it complies with designated standards, impact of its QEP, and action letter follow-up reports as required

may include a visit to off-campus sites initiated since previous reaffirmation

revised in 20126

Background Check of Fifth-Year Report…continued

includes many of the most challenging standards

Quality Enhancement Plan (QEP) Impact Report

rigor required equal to that of decennial report

evaluators scrutinize 10+ reports at one time

quick deadline if referral report needed

referral reports evaluated by the Compliance and Reports Committee

7

8

SACSCOC Fifth-Year Report Standards (2012)

1. CR 2.8 Number of full‐time faculty 2. CR 2.10 Student support services 3. CS 3.2.8 Qualified administrative and academic officers 4. CS 3.3.1.1 Institutional effectiveness: educational programs 5. CS 3.4.3 Admissions policies 6. CS 3.4.11 Qualified academic program coordinators 7. CS 3.11.3 Physical facilities 8. FR 4.1 Student achievement 9. FR 4.2 Program curriculum 10. FR 4.3 Publication of policies 11. FR 4.4 Program length 12. FR 4.5 Student complaints 13. FR 4.6 Recruitment materials 14. FR 4.7/CS 3.10.2 Title IV program responsibilities/financial aid audits 15. FR 4.8 Distance and correspondence education 16. FR 4.9 Definition of credit hours 17. CS 3.13 Policy compliance

What is “new” about the SACSCOC Fifth-Year Report?

9

language changes in CR 2.8, 2.10, CS 3.2.8, FR 4.1, 4.2, and 4.7

new standards: FR 4.8.1, 4.8.2, 4.8.3, and 4.9

policy compliance: CS 3.13.1, 3.13.3, and 3.13.4

more on these later…

The Fifth-Year Report – a Challenging Mystery, Indeed

76 percent of

2012 and 2013 institutions

were required to submit

at least one referral report

Reported in Busting Myths about SACSCOC by

Michael Johnson, SACSCOC Senior Vice President

10

2013 Institutions – Major puzzlers % Required to Complete Referral Reports

Standard Track A Track B

CR 2.8 Sufficiency of Full-Time Faculty 24% 38%

CS 3.3.1.1 Institutional Effectiveness 33% 48%

CS 3.4.11 Qualified Academic Coordinators 39% 18%

CS 3.13.1 Accrediting Decisions 27% 8%

CS 3.13.3 Complaint Record 12% 10%

FR 4.1 Student Achievement 33% 5%

FR 4.4 Program Length 0% 8%

FR 4.5 Student Complaints 6% 13%CS 3.10/FR 4.7

Financial Aid Audits/Title IV Responsibilities 15% 20%

FR 4.8.2 Protecting Student Privacy 0% 0%

FR 4.9 Credit Hours 6% 3%

QEP Impact Report 3% 15%

11

12

Form your Accreditation Scene Investigator (ASI) Teams

8-10 members meet and greet color-coded for inclusive responding

designate a reporter

13

Referral Trends - ASI Teamwork (five-minute drill)

Do you believe the results for 2013 were unusual?

What do you think were the top five referral

producers for Fifth-Year Reports over the last four

years and why have they been

problem areas?

14

Top Five Referral Report Producers 2010-2013 (Combined Tracks A and B)

5 - FR 4.5 Student Complaints - 17%

4 - CS 3.10.2/FR 4.7 Financial Aid/Title IV - 21%

3 - CS 3.4.11 Qualified Academic Coordinators – 32%

2 - CR 2.8 Number of Full-time Faculty – 42%

1 - CS 3.3.1.1 Institutional Effectiveness: Educational Programs, to include Student Learning Outcomes – 49%

15

Referral reasons

Failure to communicate compliance clearly

Failure to provide sufficient or appropriate evidence for compliance

Make a case for compliance with the wrong criteria due to a misinterpretation of the standard

Rarely, but sometimes, we do not comply.

Detecting clues

for preparing all

SACSCOC accreditation reports

16

Clues for Accreditation Leadership

It takes a village…

but it also takes a president who is strongly committed to accreditation

through peer review

17

18

Clue for leadership team

All Fifth-Year Report leadership team members should attend

SACSCOC Summer Institute and/or Annual Meeting at least

once prior to their submission year.

effective delegators and facilitators

able to “cut to the chase” in addressing each requirement

pragmatic: able to move the process along; intervene when needed

respectful of the process and those who implement it

extensive knowledge of institution

• well-known and respected on campus19

Building your team - optimal characteristics of report leaders

20

Applying Optimal Characteristics -ASI Individual work (five-minute drill)

Using your Organizational Matrix, identify individuals in your institution who exhibit these characteristics and would make good leaders for guiding the process for each standards.

Guiding questions: Do you choose to have a few or many people

overseeing the report? Do you choose to have one coordinator with

multiple information providers? Do you choose to have information providers also

write? Do you have a succession plan for people who

leave?

Share your challenges and your results with your team.

21

Managing the calendar

SACSCOC Fifth-Year Report schedule

notification of report requirements sent from SACSCOC 11 months prior to due date

report due March 25th for Track B and Sept 15th for Track A

Fifth-Year Report Review Committee reviews occur in May/June for Track B and in December for Track A

referral reports due July-ish or January-ish and reviewed by Compliance and Reports Committee in December or June

Schedule clues

begin early

begin early

begin early

pace the data collection, writing, and internal review

processes to finish well ahead of the due date

monitor the progress regularly

remember Murphy’s Law

22

23

More schedule clues

An institution’s own report preparation schedule is dependent upon the complexity of the institution and the preparers having:

a thorough understanding of the institution timely access to needed data time to focus on the report previous report-writing experience access to reports on related issues access to internal/external reviewers who can respond

quickly

Again, assume Murphy’s Law will apply!

24

Sample scheduleTrack A schools – work backward from September 15 deadlineTrack B schools – work backward from March 25 deadline

16-18 months to submission orient/organize – president or designee assignment of reporting requirements and determine review

structure/editor initiate development of technology tools: Web shell,

report platform

13-16 months to submission organizational meetings: vice presidents, deans more complex institutions may begin readiness audit/gap

analysis (especially for CR 2.8 and CS 3.3.1.1)

25

Sample schedule…continued

8-12 months to submission

complete readiness audit and first draft of QEP Impact report

receive letter of notification from SACSCOC (11 months out)

review of readiness audit and act on areas of immediate need

review CR 2.8 rationales for FT faculty numbers for all programs

26

Sample schedule…continued

clarify necessary terms, documentation timespans

identify a representative sample of CS 3.3.1.1 IE plans and reports

review credentials and rationales for CS 3.2.8 administrators and officers and CS 3.4.11 academic coordinators

ensure all publications have consistent information (Web sites, catalogs, brochures, presentations)

update boards and institutional community on progress regularly

27

Sample schedule…continued

6-7 months to submission first draft of narratives (Parts II, III, and QEP Impact) completed

and reviewed

4-6 months to submission second draft of narratives reviewed narratives deemed “final” are uploaded to submission platform

1-3 months to submission final narratives reviewed pdf’s of all evidence documents created prep Final Reports; load on platform and test DVDs/drives test all DVDs/drives using PC’s and Apple computers

28

Sample schedule…continued

0-1 month to submission

introduction/submission letter from president loose ends finalized submit

Partly borrowed from Hillsborough Community College Web site

**Review UCF’s actual CS 3.3.1.1 schedule

29

Your schedule- ASI Individual and Teamwork (five-minute drill)

Organization Matrix time!

When is your Fifth-Year Interim Report due?

Count back from that 16 or 18 months to determine when to begin your process and then input the dates you wish to use in as many places on the matrix as you can.

Share your challenges and your results with your team.

30

DOs and DON’Tsfor accreditation

report preparation

31

analyze each standard carefully highlight each of its components clarify terms and timespans to be reported

(Resource Manual is great help)

DO

32

Analyze the Standard - ASI Teamwork (two-minute drill)

What are the major areas that must be addressed/clarified for CS 3.3.1.1?

The institution identifies expected outcomes, assesses the extent to which it achieves these outcomes, and provides evidence of improvement based on analysis of the results in the following area:

3.3.1.1 educational programs, to include student learning outcomes

33

Analyzing CS 3.3.1.1Sample answers

The institution

identifies expected outcomes in [all] educational programs, to include student learning outcomes,

assesses the extent to which it achieves these outcomes

provides evidence of improvement based on analysis of the results

34

Analyzing CS 3.3.1.1…continued

clarifies terms “student learning outcomes”

measurable, define targets provides “evidence of improvement based on analysis of

results” What’s the +?

clarifies timespans for data

For example: Year 1: data analysis indicates a change is needed and

faculty determine what change is needed Year 2: change implemented; initial results [could be]

collected on change impact Year 3: results collected on change impact; [could]

determine if change was an improvement

35

use the Resource Manual for SACSCOC reports (more

on this later)

conduct a gap analysis/readiness audit of the status of

how your institution has addressed each standard to

enable time for correction, if needed (16-18 months

before deadline)

DO

36

explore reaffirmation reports from your sister institutions

– remembering that you are unique

http://www.mcneese.edu/sacs

DO

37

assemble all the evidence:

identify and collect appropriate data for the necessary time

periods to support the case for compliance

Clue: much of the evidence can be used for multiple standards

Ex: The institution’s mission statement is used many times. Certain components can be extracted for specific standards.

DO

use technical writing techniques, not creative writing techniques that embellish the argument or cover up inadequacies

use only the evidence that applies: be precise; address the specific standard; don’t say too much; don’t say too little

analyze and assemble the information so it clearly communicates compliance

address every standard and every part of every standard

38

DO

39

Where did these go wrong?

FR 4.1 Student achievement: Forty percent of the 2013 Best College nursing students passed the NCLEX exam on their first attempt. This was up from 2012 when thirty percent passed. Two nursing instructors are set to retire in 2015 and we should be able to hire more qualified faculty to help the students do better after that.

CR 2.8 Number of full-time faculty: All our faculty members have appropriate credentials for teaching in their programs as attested by the Faculty Roster included in the addenda.

40

Let’s Investigate an example -ASI Teamwork (five + minute drill)

How would you improve each of the following?

41

CS 3.3.1.1 The institution identifies expected outcomes, assesses the extent to which it achieves these outcomes, including student learning outcomes

1. Data from assessment measures were collected from students located on the main campus. The online and branch campuses were not included due to their low enrollments.

2. The assessment process for academic programs involves creating outcomes, implementing measures, and collecting results. One academic year of data provided us with a wealth of data to verify the process is being followed.

3. Listed as an IE process improvement: Implemented a new mathematics learning platform faculty members saw demonstrated at the MAA conference. Student performance increased by 25 percent on mid-term and final exam scores.

42

use multiple reviewers (internal and external) to judge clarity

and completeness

Reminder: Reviewers add time to the process!

DO

43

ensure ease of navigation throughout the materials

(both for reading and for using the technology)

DO

44

Technology is great when it works

Test every flash drive or dvd on multiple types of computers

Label each narrative clearly

Ensure every link to evidence works

Ensure you can close one standard’s narrative and go directly to another without having to go out of the whole document and start over

…more on technology clues later

45

ensure consistency and accuracy with the use of primary

sources of evidence

Post resources used in multiple sections in a centralized site

for all writers to use.

DO

46

contact your SACSCOC Vice President

to discuss unclear requirements or uncertainties

on how you plan to address standards

DO

47

rely on one person to prepare the entire report

assume writing a portion of an accreditation report is an intuitive process and everyone’s top priority

submit the report without reviewing for correctness and consistency

DON’T

Several accreditation standards reference the institution’s mission statement which is to be specific to and appropriate for the institution and must be easy to access.

DON’T provide outdated or inconsistent copies of the mission

refer the reader to the whole student catalog or any other entire document as evidence of publication of the mission

fail to rectify inconsistencies between the mission and institutional activities

48

49

Break Time

50

Uncovering clues for

remaining general areas,

specific standards and the QEP Impact

Report.

51

General clue regarding policies

Implicit in every [Core Requirement/Comprehensive Standard/Federal Requirement] mandating a policy or procedure is the expectation that the policy or procedure is in writing and has been approved through appropriate institutional processes, published in appropriate institutional documents, accessible to those affected by the policy or procedure, and implemented and enforced by the institution.

PrinciplesDO

refer to and provide copies and verify posting of policies and procedures for all standards for which they exist

(not just for CS 3.13.A-C)

Ex: FR 4.5 Student complaints - Provide institutional policy document, procedures, and a pdf of where published

52

Organizational Matrix

deadlines for report drafts and reviews

responsible persons

organizes requirements, resources, assignments, and deadlines in one document

identifies SACSCOC-delineated documentation

identifies institutional documentation

assists with survey development

53

Let’s Investigate the MatrixIndividual Activity (5 minute drill)_

focus on standards CS 3.4.11 Qualified Academic Coordinators and CS 3.11.3 Physical Facilities

review the matrix information

pencil in your institution’s information for the remaining last five columns

What challenges do you predict?

54

Clues for CR 2.8 Number of Full-time Faculty Language Change

The number of full-time faculty members is adequate to

support the mission of the institution and to ensure the

quality and integrity of each of its academic programs.

Discussion: What’s a “program?”

55

CR 2.8 CluesDO

disaggregate full-time faculty by program/discipline, mode, and location

provide a rationale for why the number is adequate for each program, mode, and location

provide information regarding faculty loads, including committee work, service, advising, curriculum development, research

provide information that students and programs have been successful with current numbers (external recognitions, awards, and rankings; licensure exams)

56

CR 2.8 Clues

DON’T incorporate the faculty roster and faculty

qualifications as data for this standard

advocate for more faculty than is considered “adequate” just to get more faculty

confuse reviewers with too much information

57

CR 2.8 Template

 1

 2

 3

 4

 5

 6 

 Location of Instruction

 

 Number & Percent

of Student UndergraduateCredit Hours

Taught by Full-time Faculty for each

Major or Program content area

 Number & Percent

of Student UndergraduateCredit Hours

Taught by Part-time or Adjuncts or Graduate Teaching

Assistants for each Major or

Program content area

 Number & Percent

of Student Undergraduate

Credit Hours Taught by Part-time Faculty

or Adjuncts or Graduate Teaching

Assistants in General Education

Courses

 Number & Percent

of Student Graduate Credit Hours Taught by Full-time Faculty

in each Degree Program

 Number & Percent

of Student Graduate

Credit Hours Taught by Part-time Faculty or adjuncts

in each Degree Program.

   

  

       

    

         

58

CS 2.10 Student Services

The institution provides student support programs, services, and

activities consistent with its mission that are intended to promote

student learning and enhance the development of its students.

59

CR 2.10

DO provide a comprehensive list of student support

services

conduct a thorough review of all student support Web sites to ensure the services are included in the report

ensure equivalent services are provided for all students, everywhere, and that these are included in the report

60

CS 3.13.1 Accrediting Decisions of Other Agencies

DO

create a table listing:

federally-recognized agencies that currently accredit the institution and its programs

dates of most recent reviews any negative actions taken by the agencies and the

reasons for each copies of statements used in describing the institution to

each agency dates and reasons for voluntary or involuntary

termination of accreditation

61

CS 3.13.3 Complaint Procedures Against the Commission or its Accredited InstitutionsDocumentation of written student complaints:

What is the policy and where is it published?

What offices maintain records and what is/are their location(s)?

What is included in the complaint record?

What is the review process for complaints?

Note: Complaint records will be reviewed during the next decennial evaluation. Be ready!

62

CS 3.13.4 Reaffirmation of Accreditation and Subsequent Reports

An institution includes a review of its distance learning programs in the Compliance Certification.

DOprovide evidence of assessment of standards that apply to distance and correspondence education programs and courses(Be sure distance learners are included in all appropriate assessment populations and use examples for CS 3.3.1.1 that highlight them.)

63

QEP Impact Report Clues

DO provide succinct list of initial goals and outcomes

report on what actually transpired, not on the plan merits

provide clear rationales for incomplete or changed activities

provide complete and clear assessment results

64

QEP Impact Report Clues…continued

DON’T fail to report reasons for changes

fail to report results and their analysis

fail to report the impact(s) on student learning and/or the learning environment

65

Technology

SACSCOC and other agency requirements

management platform

word and page counts for some sections

report presentation one printed copy other copies usually electronic

must not be tied to internet media must be labeled clearly media must be user-friendly media must work - test on various computers;

test all drives

66

Technology Tools

Internal document sharing:

Dropbox

SharePoint

Google docs

Atlassian Confluence

What others have you used?

67

Technology Tools…continued

Final preparation and presentation platforms

Compliance Assist

Xitracs

LiveText

Digital Measures

TaskStream

Home grown

What others have you used?

68

Technology Tools…continued

Open communication – to share or not to share

http://louisville.edu/institutionalresearch/accreditation/5year.html

http://www.mcneese.edu/sacs

http://www.virginia.edu/sacs/fifthyear.html

http://www.tstc.edu/docs/6291.pdf

69

Litmus Test Questions

Is each narrative focused only on the requirements of the standard?

Does each narrative state just enough, and not too much, in making a case for compliance for each requirement?

Were only past and present tense verbs used to verify compliance?

Were adjectives and adverbs limited to necessary descriptors?

Has relevant, sufficient, reliable, verifiable, representative evidence been provided to support a claim for compliance for each component of each standard?

70

Litmus Test Questions…continued

Are evidence documents highlighted or pertinent sections extracted to focus the reader on the specific information needed to verify compliance?

Were names and identifying information redacted when necessary?

Is each narrative clear to all readers (no jargon, acronyms, assumptions)?

Were pertinent quotes from the mission statement provided for most of the standards as needed?

Were the pertinent policies or procedures, including verification of their being approved by appropriate bodies and published, provided and highlighted where needed (which is just about everywhere)?

And, of course, was everything in the report accurate?

71

Input from our peers

What did you find most challenging in completing the SACSCOC Fifth-Year Interim Report?

ambiguity of terms/expectations and what that might mean to the reviewers

time management; managing internal deadlines

starting the report. Narrative content, quality, length, etc.; gathering appropriate evidence documentation and uploading electronically; and the QEP Impact

72

Input from our peersWhat was the biggest surprise you found during the SACSCOC Fifth-Year Interim Report process?

The approach taken with the [decennial] report five years prior was insufficient at the time of the Fifth-Year report even when there had been no change in the standard.

The abbreviated [Fifth-Year] review was very labor intensive, the logistics of a 5 vs. 10 year- committee organization & constituent review- not much different.

inclusion of new standards (several responses)

73

Input from our peersWhat advice would you give to those preparing a 2014 SACSCOC Fifth-Year Interim Report?

start early- particularly on CR 2.8 and CS 3.3.1.1

use the Resource Manual

develop a consistent method of labeling evidence files

employ an editor

perform gap analysis and policy inventory on each standard at least a year before report is due and discuss progress monthly with administrators

74

Advice from peers cont.There is no off-site/on-site double review to

encourage consistency with meeting standards.

It is not [just] an interim report but a reaffirmation report.

Give serious consideration to a management system (s.a. Compliance Assist or Xitracs) to streamline the process and allow you to focus on content.

When completing any standard that could be relevant to distance education offerings, ensure that you’ve mentioned how distance ed students are served/supported. This makes CS 3.13.C easier.

75

Advice from peers cont.

Build in way more time than you think you will need; anticipate that standards will be added or significantly revised.

Stay current with SACSCOC website and ask questions; communicate regularly with your VP.

Fashion your narrative for a reviewer who does not know your institution at all.

Have IT nearby at all times. [Make sure it works!]

Allow as much time as possible for editing. Have independent readers critique the semi-final drafts of the report.

What opportunities arise from the Fifth-Year Report preparation?

collaboration of a team from across the institution with a common goal

communication and education of institutional strengths to internal and external audiences

validation of quality by peers

clarification of definitions, policies, and procedures

76

identification of areas where improvements can be made even when currently in compliance

gap analysis/preparation for the next decennial report

thorough review of all Web pages and publications to ensure they are consistent and correct

What are other opportunities arising from the report?

77

What opportunities arise from the Fifth-Year Report preparation? …continued

78

Resources

The Fifth-Year Interim Review Process & Included Standards - Crystal Baird’s 2013 Summer Institute Presentation

http://www.sacscoc.org/cbaird.asp

Busting Myths About SACSCOC - Michael Johnson’s 2013 Summer Institute Plenary Presentationhttp://www.sacscoc.org/institute/2013/Institute13/Plenaries/johnson1.PDF

SACSCOC Templates for the Compliance Certificationhttp://www.sacscoc.org/cctemplates.asp

79

Resources…continued

Handbook for Review Committees http://www.sacscoc.org/pdf/handbooks/Exhibit%2018.HandbookForReviewCommittees.pdf

Evaluator Training Modules: Institutional Effectiveness http://www.sacscoc.org/trngmods/IEModules.pdf

Evaluator Training Module: Student Services http://www.sacscoc.org/trngmods/SSModules.pdf

Analyzing a Case for Compliance rubrichttp://www.sacscoc.org/pdf/ANALYZING%20A%20CASE%20FOR%20COMPLIANCE_SEPT2010%20_2_.pdf

80

Resources…continued

SACSCOC Listserv

To subscribe send an email LISTSERV@LISTSERV.UHD.EDU

Leave the subject line blank The body of the email should contain only the command:

“Subscribe ACCSHE”

Subscribers post to: ACCSHE@LISTSERV.UHD.EDU

…and the reason we do all this

81

University of Central FloridaDr. Tace CrouseDirector, Special ProjectsFifth-Year Report CoordinatorAcademic Affairstace.crouse@ucf.edu

Dr. Diane ChaseExecutive Vice ProvostSACSCOC LiaisonAcademic Affairsdiane.chase@ucf.edu

Ms. Heidi WattDirector, Accreditation and Quality AssuranceAcademic Affairsheidi.watt@ucf.edu

Dr. Denise YoungAssociate Vice PresidentFormer SACSCOC LiaisonRegional Campusesdenise.young@ucf.edu

82http://afia.ucf.edu/accreditation