Colorado State University’s Plan for Researching Improvement and Supporting Mission

PRISMBeginning in June 1998, CSU’s governing board placed the institution on a “learner centered” paradigm. This new commitment involved developing a formal, comprehensive, and effective institutional assessment program. The University’s strategic planning for FY03 identified periodic assessment with subsequent programmatic modification as an essential component of enhancement for both academic and service/support programs at CSU (Annual Update of the USP, FY03). In early October 2002, CSU’s Academic Programs Assessment and Improvement Committee (APAIC) endorsed a University-wide Plan for Researching Improvement and Supporting Mission (PRISM). The following concept model communicates this assessment planning process to the University community.

PRISM is a comprehensive, systematic process for continuously improving academic programs in three areas: 1) student learning (including undergraduate, and graduate education), 2) faculty research/scholarship, and 3) faculty service/outreach. The process also includes improvement planning for student affairs and academic support areas. It helps programs coordinate multiple improvement reporting requirements—regional accreditation, specialized accreditation, and CSU program review—while informing CSU strategic planning. Its central theme is that faculty/staff learn about themselves and act on what they learn. It is an organizational learning infrastructure.

Use of direct assessment methods, faculty evaluation of data to identify program strengths and weaknesses, and formative improvements based on systematic performance research are characteristics of the process. Online access to program assessment plans, to planning best practices, and to units’ evaluation instruments, enables faculty and staff to improve program performance through the use of these shared resources. [Concept Model Follows]

Applied Human Sci.

Business

Engineering

Liberal Arts

Natural Resources

Library

Natural Sciences

Veterinary Sciences

HLC 10

Yrs. Program Review

5-7 Yrs

Special Accredit.

4-8 Yrs.

CSUannual

operationalplanningS

umm

ative Data

Form

ative Data

INPUT

Intended Quality Improvements

1. CSU Strategic Plan Goals

2. Faculty Expertise

3. External & Internal Constituencies’

Needs

4. Available Resources

MISSION

CSU Strategic PlanContinuous

ImprovementObjectives

Formative Assessment

Student LearningFaculty ResearchFaculty Outreach

Support ProgramsSpecial Accreditation

Program Review

HLC Accreditation

Phase 2 Refining

Phas

e 3

Shar

ingPhase 1 C

ollecting

PRISM

O

UT

PUT

1

. Pr

ogra

m Im

prov

emen

ts

2

. Res

earc

h M

etho

dolo

gies

3

. Str

ateg

ies t

o A

chie

ve O

utco

mes

4

. Pla

nnin

g Pr

oces

s Bes

t Pra

ctic

es

O

UT

PUT

1

. Pr

ogra

m Im

prov

emen

ts

2

. Res

earc

h M

etho

dolo

gies

3

. Str

ateg

ies t

o A

chie

ve O

utco

mes

4

. Pla

nnin

g Pr

oces

s Bes

t Pra

ctic

es

AgricultureC

ol lege Improvem

ent Goa ls

Student &

Adm

inis trative Support

Divisional Im

provement G

oals

Plan for ResearchingImprovement &SupportingMission

Planning Process

InstitutionalR

esearch

Phase 2: Refining Planning & Research Effectiveness (January-May)

5) Integration of Annual Assessment & Evaluative Six-Year Program Review6) Multiple Peer-Review Levels (short- & long-term planning) 7) Documentation of Planning Changes and Improvement Dialogues

Phase 1 Collecting

Phase 2 Refining

Phas

e 3

Shar

ing

Phase 1: Collecting & Configuring Data (September-December)

1) University Plan Concept Model 2) Annual On-Line Plan Building Platform3) Descriptive Time Lines Monitor Participation & Organize Activity4) Data Management Tools Configure Evidence To Satisfy Multiple Accrediting Criteria

Phase 3: Sharing the System’s Planning Output(June-August)

8) Resources for Improving Planning & Evaluation 9) Transparency Website for External Constituents a. regional and special accreditation teams b. state accountability staff or legislators c. constituencies, students, parents, alumni, others d. faculty access to plans and their strategies10) Reports Generated on Planning Process Output and Quality of Performance Research Using a Classification Process and Planning Taxonomy

PRISM’S 10 OPERATING COMPONENTS & TIME LINE

Planning Process

Theme 2: Planning Context and Integration

7) Annual assessment plans are formally linked to six-year program review self-studies in a database permitting University review of annual planning effectiveness over time.

8) Six-Year action plan goals use annual assessment planning to support long term objectives 9) Action plan goals of program review link in a database to university strategic plan goals

Sustainability

Context &Integration

Org

aniz

atio

nal

Lea

rnin

g

Theme 1: Sustainability & Visibility

1) Concept model expands recognition2) Annual Plan Platform develops behavior3) Peer-Review sustains quality self-evaluation4) Visible on-line time lines track program assessment progress and participation5) Embedding assessment into on-line program review self-studies gives learning visibility6) Expanding faculty dialogues adds buy-in

Theme 3: Organizational Learning Environment

10) Best practice pool of items instruct better process 11) On-Line viewing of all assessment instruments informs better ways to measure performance12) Multiple peer-review groups learn from their on-line interactive dialogues about quality13) On-line viewing of all departmental goals and

strategies—annual and six-year—informs means14) Transparency site expands learning to constituents

PRISM’S 3 THREE OPERATING THEMES

System Themes

PRISM’S 10-PART PROCESS & INFORMATION FLOW

ConceptModel

Process Steps& Time Lines

Monitored

Selecting & Configuring

Evidence

Aligning Assessment Plans with ProgramReview Self-Studies

Documentation ofProgram Changes,

Improvement Dialogues,& New Goals

Research Tools& Best Practices

Shared

External EngagementWeb Site For

Multiple Audiences

Reports Generated on Planning Process

Performance

Strategic Plan Informs Units from Top Down at

Steps 2 & 5

Multiple Review Levelsto Ensure Compliance with Quality Standards

Annual On-LineAssessment PlanBuilding Platform

System FEEDBACK (7, 8, 9, 10) Informs Faculty on Program

Improvement & Self-Evaluation (steps 2 & 5)

System FEEDBACK (7, 8, 9, 10) InformsLong-Term Planning

FACULTY EXPERTISEOPTIMIZED BY DEPARTMENTS

at steps 2 & 5 with other steps done by institution

PRISM’S Federalism Dynamics of Centralization and Decentralization

Source: Joel Cutcher-Gershenfeld and J. Kevin Ford, Valuable Disconnects in Organizational Learning Systems : Integrating Bold Visions and Harsh Realities, New York: Oxford University Press, 2005 (p. 86).

CONTEXT OF THE GROWING DEMANDFOR CONTINUOUS, SYSTEMATIC DEPARTMENTAL PLANNING & EVALUATION

1) Faculty as professionals should study themselves and act on what they learn.2) The Association of Governing Boards encourages boards to “ . . . determine that systematic

and rigorous assessments of the quality of all educational programs are conducted periodically, and board members should receive the results of such assessments.”

3) The University Strategic Directions (USD) goals and metrics require measurement of PRISM’s impact on continuous improvement.

4) The USD requires academic programs to present “quality assessments of learning” as part of program reviews.

5) The USD metrics require increased participation in program assessment plans to demonstrate advances in active and experiential learning.

6) The USD metrics request programs to present evidence on international recognition in research and scholarship.

7) University Guidelines for Program Review require evidence of continuous planning and evaluation.

8) Quality monitoring by special and regional accreditation bodies expects continuous planning and evaluation with assessment results informing improvements.

9) The state of Colorado annually reviews CSU students’ knowledge of content taught in AUCC courses.

10) Market forces respond favorably to department-generated evidence of impact and quality.11) Professionally, the fluid & globally competitive higher education environment requires

current data for daily decision making at the department level.

PEER REVIEW COMMITTEE

The Academic Planning & Evaluation

Council (APEC)

Agricultural Sciences

Nancy Irlbeck

Applied Human Sciences

Dale DeVoe

Business

Suzanne Lowensohn

Engineering

Tom Siller

Libraries

Jeff Bullington

Liberal Arts

Irene Vernon

Natural Resources

Peter Newman

Natural Sciences

Simon Tavener

Veterinary Medicine &

Biomedical Sciences

Lori Kogan

Assessment Director

Kim Bender

V.P. for Undergraduate Studies

Alan Lamborn

Applied Human Sci.

Business

Engineering

Liberal Arts

Natural Resources

Library

Natural Sciences

Veterinary Sciences

Phase 1 Collecting

HLC10Yrs.Program

Review 5-7Yrs

SpecialAccredit.

3-7Yrs.

CSU1-yearCyclesS

um

mative D

ataF

orm

ative Data

Plan for Researching

Improvement &

Supporting Mission

Budgets &Institutional

Analysis

MISSION

Strategic PlanContinuous

ImprovementObjectives

Formative Assessment

Student LearningFaculty ResearchFaculty Outreach

Support ProgramsSpecial Accreditation

Program ReviewHLC Accreditation

Phase 2 Monitoring

Agriculture

Co ll ege Im

provement G

oals

St udent &

Adm

ini s tr at ive Suppor t

Divi s ional I m

pr ovement G

oal s

APEC Colorado State University

UNIVERSITY CHARGE:

The Council operates as a learning group with members experiencing professional development that strengthens departmental planning, self-evaluation and improvement, which is intended to expand unit visibility and impact. At the same time, the Council serves the University by strengthening its planning and evaluation processes and by applying the intent of its strategic plan.

DSA PEER REVIEW COMMITTEE

Sub-Committee of

The Assessment and Research

Steering Committee (ARSC)

Academic Support

Ann Malen

Ann Wilcox

Mike Brake

Advocacy Services

Rose Kreston

Campus Life

Jeannie Ortega

Housing Services

Dave McKelfresh

Lory Student Center

Robert Peters

Wellness Programs/Services

Loretta Capra

Chuck Davidshofer

Pam Zimdahl

Applied Human Sci.

Business

Engineering

Liberal Arts

Natural Resources

Library

Natural Sciences

Veterinary Sciences

Phase 1 Collecting

HLC10Yrs.Program

Review 5-7Yrs

SpecialAccredit.

3-7Yrs.

CSU1-yearCyclesS

um

mative D

ataF

orm

ative Data

Plan for Researching

Improvement &

Supporting Mission

Budgets &Institutional

Analysis

MISSION

Strategic PlanContinuous

ImprovementObjectives

Formative Assessment

Student LearningFaculty ResearchFaculty Outreach

Support ProgramsSpecial Accreditation

Program ReviewHLC Accreditation

Phase 2 Monitoring

Agriculture

Co ll ege Im

provement G

oals

St udent &

Adm

ini s tr at ive Suppor t

Divi s ional I m

pr ovement G

oal s

ARSC Colorado State University

DIVISION OF STUDENT AFFAIRS CHARGE:

1) Conducting an audit of current assessment and research activities in the Division of Student Affairs at Colorado State University.

2) Assisting the Division of Student Affairs in advancing our commitment to assessment, continuing to develop this as an area of professional competence for our staff, and using the results to continuously improve the quality of programs, services, and facilities for students and the campus community.

3) Assisting departments within the division with program reviews that departments have incorporated into their work plans and suggest ways to facilitate the program review process.

4) Identifying assessment and evaluation needs in order to develop and conduct workshops that will increase staff members' knowledge, professional competence, and confidence in using assessment approaches, and integrating assessment and research practice into new programs and services.

5) Continuing to identify significant studies, publications, or Student Affairs' organizations at our peer institutions pertinent to our work in benchmarking, assessment, and research.

References Supporting The Use of Systematic Processes for Improving Program Performance

Berkes, F., J. Colding and C. Folke (2003). Navigating social-ecological systems: Building resilience for complexity and change. Cambridge: Cambridge University Press.

Gunderson, L. and C. Holling. 2002. Panarchy: Understanding transformations in human and natural systems. Washington: Island Press.

Kezar, A. (2005). What campuses need to know about organizational learning and the learning organization. New Directions for Higher Education, 131, 7-22.

Milam, J. (2006). “Ontologies in higher education.” In A. Metcalfe, Knowledge management and higher education: A critical analysis. (34-62). Hershey, PA: Information Science Pub.

Senge, P. M. (2006). The fifth discipline: The art & practice of the learning organization (2nd ed.). New York: Currency/Doubleday.

Tierney, William G. (1999). Building the Responsive Campus: Creating High Performance Colleges and Universities. London: Sage Publications.

Wergin, Jon F. (2003). Departments That Work: Building and Sustaining Cultures of Excellence in Academic Programs. Bolton, MA: Anker Publishing Company, Inc.

END OF PLANNING DESCRIPTION