designing and implementing a sustainable assessment process

11/18/18

1

Designing And Implementing a Sustainable Assessment Process

Practical Strategies for Balancing Accountability and Improvement

Felix WaoAssessment Institute Pre-Conference Workshop, Indianapolis, IN

September, 2018

Workshop Outcomes

Upon completion of this workshop, participants should be able to the following at their institutions:v Design and implement effective assessment initiatives.v Develop strategies for balancing accountability and

improvement.

11/18/18

2

Activity 1__________________

What does assessment mean to you… in one word?

Assessment is…“…is a systematic process ofunderstanding and improving

student learning.”

Angelo, 1995.

Assessment is…

11/18/18

3

Levels of Assessment

Activity 2__________________

Working in groups of two or three, review Characteristics of Effective Assessment

in the handout provided.To what extent are they currently reflected at

your institutions?

11/18/18

4

Elements of Effective Assessment Planning

Key Elements in Planning Effective Assessment

Effective Assessment

Identify & Engage Stakeholders

Determine Appropriate Timing

Use Evidence to Cultivate Assessment

Culture

Design an Assessment Plan

Align Assessment with Strategic

Planning

11/18/18

5

Identify & Engage your stakeholders…

Staff

Stakeholders

students

Academic Administrators

Faculty

Student Affairs Professionals

Design a Plan…

“…written guidance on who

does what when.”

Suskie, 2004

11/18/18

6

Needs Assessment

No assessment process in place

Developed a simple 4-step process: 1=plan, 2=measure3=analyze4=adjust

Developed guidelines detailing each of the four components of the assessment process.

Significant (structural) variation among assessment reports. No guidelines.

Focus primarily on Compliance, not student learning

Refocused attention to student learning (template divided into two parts).

Resulting Actions…

11/18/18

7

Some programs excluded from the process.

Developed an outreach campaign to all department and provided examples of assessment reports.

Developed a detailed and informative color-coded review rubric.

No feedback to departments

No opportunities for dialogue among Liaisons

Created the “OU Assessment Conference”, facilitated workshops and frequent outreach to departments.


No Institutional Assessment Office

Provost Office Created Office of Academic Assessment.

Formed Provost Advisory Council for Learning Outcomes Assessment (PACLOA).

No Assessment Committee


11/18/18

8

Sample Assessment Process…

Programs have the greatest difficulty in the assessment

cycle of “closing the loop”

Banta & Blaich, 2011

Step 1

Hint: Use of Bloom’s Taxonomy of active verbs.

Key Question…

11/18/18

9

Creating

Evaluating

Analyzing

Applying

Understanding

RememberingLower order learning

Higher order learning

Bloom’s Taxonomy

Examples of Weasel Words.[Can be used to define a Goal, but NOT a Learning Outcome]

imagine, realize, acquainted with, familiar with, understand, know,

believe, appreciate,conscious of, interested in,

awareness of, see.

11/18/18

10

STUDENTLEARNING

OUTCOMES

CORE COURSES

A D E F

1

2

3

WHERE?

Step 2

HOW?

Hint: v Undergraduate: Focus on core courses and other educational experiences required of all students.v Graduate: Center on culminating experiences and processes that prepare students for culminating

experiences.

Key Questionsq What do the data show about your students’ level of

mastery of intended learning outcomes?

q Are there areas where students are outstanding? Are they consistently weak on certain skills?

q Are there areas where performance is good, but not outstanding, and you’d like to see better/higher performance?

Step 3

Hint: Focus on (aggregate) student performance.

11/18/18

11

Key Questionsq What will you do to improve student learning? q Which program elements should be:

Ø Reinforced?

Ø Maintained?

Ø Strengthened?

Ø Modified?

Step 4

Tip: Focus on program level action plans based directly on evidence of student achievement.

Align Assessment Process with Strategic Plan…

Assessment Institutional Strategic Plan

11/18/18

12

Align Assessment Process with Strategic Plan… (Pace University)

Where are we? …And how are we

doing? retention

admission

funding

research

Graduation rates

Teaching & learning

Program review

accreditation assessm

ent

Timing is crucial…

11/18/18

13

Timing is crucial… (University of Houston)

Institutional Assessment Plan

Academic Programs

Administrative Units

Cultivate a Culture Based on EVIDENCE…

EVIDENCE

Care about student learning

Trust and Integrity

Curriculum Design & Pedagogy

Delivery and

assessment

Formative Summative

TeachingFaculty

Research

11/18/18

14

Activity 2 __________________

Working in groups of two or three, use the template provided to rate the level of

importance of each element in implementing assessment at your current institution.

11/18/18

15

Techniques for Successful Implementation of Assessment Plans

Leadership Matters…

11/18/18

16

Encourage Faculty to Actively Participate in Committees…

Provide Adequate Resources…

11/18/18

17

Provide Professional Development Opportunities

Faculty Learning

CommunitiesPeer Coaching

Needs Assessment Faculty

Mentoring Programs

Professional Development

Plan

Assess Both Processes and Outcomes…Assess Both Processes and Outcomes…

Processes Outcomes

improves

informs

11/18/18

18

Disseminate Results…

Faculty

Students

Staff

Commu

nity

Deans

Chairs

Suskie (2009)

Use Findings Fairly, Ethically and Responsibly…

11/18/18

19

Strategies for Sustaining Effective Assessment

Disseminate Credible Results to Multiple Stakeholders…

Faculty

Students

Academic Advisors

Community

Deans

Retention

Student Funding

Admission Standards

Course Revisions

Course Sequence

Faculty Hiring

Licensure Pass Rates

Accredito

rs

Student Affairs Professionals

Assessment Process

Admission Officers

11/18/18

20

MISSING INFORMATION

NEEDS REVISION

MEETS OR EXEEDS

EXPECTATIONS

Assessing Assessment…

Activity 3 __________________

Reviewing Assessment Reports

11/18/18

21

Resistance… how do you handle it?

Key to Balancing Accountability and Improvement…

ImprovementAccountability

Planning*, implementing and sustaining effective

assessment initiatives focused on continuous

improvement.

*Embed external requirements during the planning phase.

11/18/18

22

• What is the most valuable lesson that you learned from this session?• What is one question that you still have?• What is one thing you will implement on your campus

as a result of this presentation?

One-minute Eval…

?

11/18/18

23

Felix Wao [email protected]

University of OklahomaOffice of Academic Assessment

www.ou.edu/assessment

References

Banta, T. W., & Blaich, C. (2011). Closing the assessment loop. Change: The Magazine for Higher Learning, 43(1), 22–27.

Suskie, L (2004). Assessing Student Learning: A Common Sense Guide. Bolton, MA: Anker.

Suskie, L (2009). Assessing Student Learning: A Common Sense Guide (2nd Ed.). San Francisco, CA: Jossey-Bass.

mailto:[email protected]

http://www.ou.edu/assessment

Course 1 Course 2 Course 3 Course 4 Course 5 Course 6

BASIC Curriculum Mapping Template

Part AProgram student learning outcomes (SLOs).

SLO #1

SLO #5

SLO #3

SLO #4

SLO #2

Part BRequired Courses Part C

Indirect Measures

Part C

Map indirect measures.

Insert "X" under each course to align with the SLO it addresses. Insert program student learning outcomes (SLOs).

INSTRUCTIONS

Part B

Indirect measures ask students to reflect upon and report their perceptions of their gains in knowledge, skills, etc.

Examples of indirect measures include graduating surveys, exit surveys, alumni surveys, and focus groups.

If the program conducts surveys or focus groups with students, alumni, etc., place an "x" in the cell(s) in which a part or all of the survey's (or focus group's) findings may align with a program-level SLOs.

1. Insert program outcomes in the rows labeled "Program level student learning outcomes" . Insert additional rows if needed.

2. Insert an abbreviated course name, number, and units/sections (if applicable) in labeled columns. Insert additional columns if needed.

It is helpful to insert course information in the sequence in which they are typically taught.

1. Request that faculty teaching each course listed in the columns to determine which, if any, of the program student learning outcomes (SLOs) are addressed and/or assessed in their course.

2. Ask those same faculty members to identify alignment of each course with one or more SLOs. Select a course in which the SLO is addressed.

It's perefectly ok to have more than one course addressing a single SLO and/or more than one SLO addressed by a single course.

Part A

Course 1 Course 2 Course 3 Course 4 Course 5 Course 6

MAPPING KEY: I=Introduced , A=Advanced , M=Mastery

ENHANCED Curriculum Mapping Template

A. Insert units/sections (if applicable)

Program student learning outcomes (SLOs).

A. Insert course names & #'s in columns and program (SLOs) in

rows D. Indirect Measures

Required Courses

B. Insert "I," "A," or "M"

D. Insert potential assessments



C. Insert instructional activities


SLO #1

SLO #2

SLO #3

SLO #4

SLO #5










EMap indirect

measures.

INSTRUCTIONS

Insert course and program student learning outcomes (SLOs).

Insert expected knowledge or skill level of each course.

BA

1. Insert program outcomes in the rows labeled "Program level student learning outcomes" . Insert additional rows if needed.

2. Insert an abbreviated course name, number, and units/sections (if applicable) in labeled columns. Insert additional columns if needed.

It is helpful to insert course information in the sequence in which they are typically taught.

1. Request that faculty teaching each course listed in the columns to determine which, if any, of the program student learning outcomes (SLOs) are addressed and/or assessed in their course.

2. Ask those same faculty members to identify whether students--after taking the course--are expected to demonstrate:- introductory knowledge or skill (e.g., recall or explain facts, concepts), - advanced knowledge or skill (e.g., apply a procedure or analyze how parts relate to or contrast from one another), or - mastery (e.g., evaluate or make judgments based on criteria; create a novel approach, product, or artifact).

In order to represent each faculty member's determination, use the following key:

"I" for Introductory , "A" for Advanced , "M" for Mastery.

Indirect measures ask students to reflect upon and report their perceptions of their gains in knowledge, skills, etc.

Examples of indirect measures include graduating surveys, exit surveys, alumni surveys, and focus groups.

If the program conducts surveys or focus groups with students, alumni, etc., place an "x" in the cell(s) in which a part or all of the survey's (or focus group's) findings may align with a program-level SLOs.

1. Instructional activities occurring in and out of class reinforce learning objectives and prepare students for assessments. Request that faculty teaching each course list the instructional activities for each course.

Frequently used instructional activities are listed below. (This list is not all-inclusive; faculty should insert any activities not found in this list.)

--case study--client project--debate--discussions (online and face-to-face)--exhibition of work--group work--lab reports--lecture (e.g., by faculty, guests)--oral presentation--performance--public review--recitation--service learning--studio--video clips--Book review and discussions

Insert instructional activities used in courses

1. Request that faculty teaching each course state how student knowledge or skill is assessed in their course, as aligned with the program-level SLOs.

2. Insert potential assessment methods into the appropriate cells.

Frequently used assessments are listed below. (This list is not all-inclusive; faculty should insert any activities not found in this list.)

--case study--client project--debate--exhibition of work--exam*--group work/project--lab reports--oral presentation--performance--problem sets--service learning--studio--written work--internship/practicum--reflection papers--journal article critiques

*Besides multiple-choice tests/exams, the quality of student work in the rest of the assessments require use of rubrics .

C DInsert how the outcome is assessed in the

course.

Perspectivesin Western Art

Perspectives in Ancient & World

Art

CriticalTheories in Art

FoundationStudio I

AdvancedStudio I

Future Media and Advanced

Techniques

Senior Studio, Exhibition, &

PortfolioGraduates of the AA in Art should be able

to: 2 3 5 4 4 5 8SLO 1: Appropriately conduct and incorporate research findings into their work

x x x x x x x

SLO 2: Evaluate art movements from various cultures and time periods x x x x x x x

SLO 3: Articulate a philosophical and aesthetic approach to their art and its place in the larger cultural and historical context

x x x x

SLO 4: Design and execute projects effectively x x x x

SLO 5: Use new tools and methods with facility x x x xSLO 6: Create a distinctive body of work that embodies their personal approach and their creative and technical mastery

x x x x

BFA in Art Example of BASIC Undergraduate Curriculum Map

Program Student Learning Outcomes (SLOs)

Required Courses

Indirect Measures



Perspectives in Ancient & World

Art


FoundationStudio I

AdvancedStudio I


Techniques

Senior Studio, Exhibition, &

PortfolioGraduates of the AA in Art should be able

to: 2 3 5 4 4 5 8SLO 1: Appropriately conduct and incorporate research findings into their work

I I I I A A M

SLO 2: Evaluate art movements from various cultures and time periods I I I I A A M

SLO 3: Articulate a philosophical and aesthetic approach to their art and its place in the larger cultural and historical context

I A A M

SLO 4: Design and execute projects effectively I I A M

SLO 5: Use new tools and methods with facility I I I A

SLO 6: Create a distinctive body of work that embodies their personal approach and their creative and technical mastery

I I A A

BFA in Art Example of BASIC Undergraduate Curriculum Map (with Levels)


Program Student Learning Outcomes (SLOs)

Required Courses

Indirect Measures


Perspectives in Ancient & World Art


FoundationStudio I

AdvancedStudio I


Techniques

Senior Studio, Exhibition, & Portfolio

Graduates of the AA in Art should be able to: A. Course units/sections 2 3 5 4 4 5 8

B. Insert "I," "A," or "M," I I I I A A M


Virtual and in-person resource orientations,

lectures, group discussions, debates

Virtual and in-person resource orientations,

lectures, group discussions, debates

Lectures, group discussions, artist

seminars

Artistic work process, iterative critique and

dialogue w/peers& faculty





Exhibition & portfolio process, iterative critique

and dialogue w/peers& faculty

D. Insert potential assessments Written critiques Written critiquesMid-term andfinal papers

Artistic work product, oral presentation



Exhibition & portfolio presentation

B. Insert "I," "A," or "M," I I I I A A M


Lectures, group discussions, debatesvirtual and in-person

museum tours, PowerPoint reviews

Lectures, group discussions, debatesvirtual and in-person

museum tours, PowerPoint reviews

Lectures, group discussions,

artists seminars

Artistic work process, iterative critique and dialogue w/peers &

faculty


faculty


faculty



D. Insert potential assessments Written critiques Written critiques Mid-term andfinal papers

Artistic work product, oral presentation, written

critiques of peer work





Exhibition & portfolio presentation, written


B. Insert "I," "A," or "M," I A A M



faculty


faculty


faculty



D. Insert potential assessments Oral presentation of artistic work product

Oral presentation of artistic work product

Oral presentation of artistic work product

Exhibition & portfolio presentation


Perspectives in Ancient & World Art


FoundationStudio I

AdvancedStudio I


Techniques

Senior Studio, Exhibition, & Portfolio

I=Introduced , A=Advanced , M=Mastery

Program Student Learning Outcomes

(SLOs)

A. Insert course names & #'s in columns and program SLOs

in rows

Required Courses

D. Indirect Measures

Example of ENHANCED Undergraduate Curriculum Map BFA in Art

Program Student Learning Outcomes

(SLOs)

A. Insert course names & #'s in columns and program SLOs

in rows

Required Courses


MAPPING KEY:

Appropriately conduct and incorporate research findings into their work.

Focus Groups and

Alumni and Graduating

Surveys

Evaluate art movements from various culturesand time periods.

Focus Groups and


Surveys

Articulate a philosophical and aesthetic approach to their art and its place in the larger cultural and historical context

Focus Groups and


Surveys

Graduates of the BFA in Art should be able to: A. Insert course units 2 3 5 4 4 5 8

B. Insert "I," "A," or "M," I I A M



faculty


faculty


faculty





faculty


faculty


faculty



B. Insert "I," "A," or "M," I I I A



faculty


faculty


faculty

Exhibition and portfolio process, iterative critique


D. Insert potential assessments Artistic work product Artistic work product Artistic work product Exhibition, portfolio presentation

B. Insert "I," "A," or "M," I I A A



faculty


faculty


faculty



D. Insert potential assessmentsArtistic work product,

oral presentationArtistic work product,

oral presentationArtistic work product,

oral presentationExhibition & portfolio

presentation


Use new tools andmethods with facility

Focus Groups and


Surveys

Create a distinctive body of work that embodies their personal approach and their creative and technical mastery

Focus Groups and


Surveys


Design and executeprojects effectively

Focus Groups and


Surveys

ãFaculty Needs Survey, Office of Academic Assessment, University of Oklahoma

1

UNIVERSITY OF OKLAHOMA OFFICE OF ACADEMIC ASSESSMENT

NEEDS ASSESSMENT SURVEY FOR FACULTY

Part 1: YOUR VIEWS ABOUT ASSESSMENT

a) Please indicate your level of agreement with the following statements. My department...

Agree Strongly Agree Disagree Disagree

Strongly I Don't Know

Coordinates its student assessment activities annually in conjunction with campus administrators o o o o o Is recognized by faculty in other colleges/departments for its approach to program assessment. o o o o o Has an effective plan for monitoring student outcomes. o o o o o Demonstrates a great deal of consensus on its approach to student learning o o o o o Collects information about employer needs for specific skills and knowledge among our graduates o o o o o Is perceived as a campus leader on issues of student assessment. o o o o o From an educational standpoint, it is necessary for us to monitor what students learn. o o o o o Has influence on assessment techniques I use in my course(s) o o o o o


2

b) Please indicate your level of agreement with the following views about assessment.

Agree Strongly Agree Disagree Disagree

Strongly I Don't Know

The results of student evaluations of my teaching influence my approach to assessing student work. o o o o o State or federally mandated assessment requirements improve the quality of undergraduate education. o o o o o Student assessment reduces the quality of education. o o o o o Student assessment limits the amount of time I have to devote to other academic activities. o o o o o Student assessment is more effective when determined by the faculty member rather than by the institution. o o o o o Student assessment has improved the quality of education at OU. o o o o o From an educational standpoint, it is necessary for us to monitor what students learn. o o o o o The effectiveness of teaching is enhanced when faculty regularly assess students. o o o o o What I learn by assessing student learning has immediate relevance to what takes place in the classroom. o o o o o Regular assessment of students accurately captures what they are learning in my classroom o o o o o Monitoring student assessment is a distraction and competes with essential academic work o o o o o Faculty have a professional obligation to regularly assess what students are learning in courses o o o o o


3

Part 2: PROFESSIONAL DEVELOPMENT TOPICS a) The following section reflects important aspects of assessment at both course and program levels. Please

rate your perception of each aspect and indicate if you would be interested in attending workshops/sessions designed to address them.

Your perception of the importance of this aspect

Would you be interested in attending a

workshop designed to address this aspect?

Very

Important Moderately Important

Of little Importance

N/A

Definitely Yes

Probably Yes

Probably Not

Unsure

• Defining course level student learning objectives O O O O O O O O

• Defining program level student learning outcomes O O O O O O O O

• Aligning course objectives with program level student learning outcomes O O O O O O O O

• Determining appropriate assessment strategies for face-to-face courses O O O O O O O O

• Determining appropriate assessment strategies for blended or fully on-line courses (including Open Courses).

O O O O O O O O • Developing strategies for assessing

students' dispositions, virtues and attitudes

O O O O O O O O • Providing feedback to learners and

using feedback to enhance instruction O O O O O O O O • Organizing content in the best

sequence for learning O O O O O O O O • Using digital media to support learning

(wikis, blogs, social media) O O O O O O O O • Designing and assessing team-based

learning sessions O O O O O O O O • Making lectures engaging and

interactive O O O O O O O O • Using clickers to elicit student

participation and promote engagement O O O O O O O O • Developing service learning initiatives to

achieve learning goals O O O O O O O O • Designing competency-based

teaching/learning strategy O O O O O O O O • Designing problem-based learning

strategies O O O O O O O O • Promoting case-based instruction O O O O O O O O • Assessing large classes O O O O O O O O • Integrating and assessing academic

service learning into your course O O O O O O O O • Developing rubrics for assessing

assignments and projects O O O O O O O O • Writing effective multiple-choice tests

(that target higher order thinking skills) O O O O O O O O • Classroom Assessment Techniques

(CATs)/Informal, formative assessments

O O O O O O O O • Use and evaluation of portfolios O O O O O O O O


4

b) What other topics/areas would you like to see presented?

________________________________________________________________

________________________________________________________________

________________________________________________________________ c) Would you be willing to join the staff of the Center for Teaching Excellence as a guest speaker or a facilitator for any of

the topics/areas mentioned above? Is yes, please write your name and e-mail address in the space below as well as and the topic(s) you'd be interested in facilitating.

________________________________________________________________

________________________________________________________________

________________________________________________________________ d) Can you suggest possible guest speakers or facilitators (on/off campus) for our workshops? Please provide names,

institutions and their areas of expertise.

________________________________________________________________

________________________________________________________________

Part 3: PREFERENCES FOR WORKSHOP DELIVERY METHODS AND FORMATS

Preferences for workshop formats often depend on the topic. However, it would be helpful if you could provide us with your general preferences for workshop formats, time, and length. a) Preferred formats (please check all that apply)

O Formal face-to-face presentations followed by discussion

O Combination of presentation, group/interactive work, and discussion

O Self-paced/self-directed materials (e.g., Web-based resources, video tapes/DVD's, handouts, etc.)

O Informal face-to-face events (e.g., presentations, brown bag meetings, etc.)

O Informal on-line sessions (e.g., web-based presentations, chat sessions, etc.)

O Other (please specify): ________________________________________________

b) Preferred day/time (please check 3 preferred starting times for each day of your choice) Day Time

8am 9am 10am 11am Noon 1pm 2pm 3pm 4pm Monday O O O O O O O O O Tuesday O O O O O O O O O

Wednesday O O O O O O O O O Thursday O O O O O O O O O

Friday O O O O O O O O O


5

c) Preferred length (please check all that apply)

▢ up to 60 minutes

▢ up to 90 minutes

▢ up to 2 hours

▢ up to 2 1/2 hours

▢ up to 3 hours

▢ 1/2 day

▢ Other (please specify): _____________________

d) Preferred way to receive information about upcoming

workshops/sessions, programs, and program materials (please check all that apply)

▢ Office of Academic Assessment website

▢ E-mail

▢ Flyer in campus mail

▢ Listserv

▢ Posters

▢ Other (please specify) ______________________ e) Please provide any additional comments or suggestions below.

_________________________________________________________________________________________________

_________________________________________________________________________________________________

Part 6: DEMOGRAPHIC INFORMATION This section is optional. However, it will help us to identify needs of specific groups.

a) Your primary appointment O Department O College

b) Position

O Professor

O Associate Professor

O Assistant Professor

O Instructor

O Ranked Renewable Term

O Adjunct Professor/Visiting Lecturer

O Graduate Assistants Teaching Stand-alone Courses

O Other (Please Specify) ________________________________________________

c) What is your tenure status?

O Tenured (please enter year you received tenure)

O On Tenure Track

O Not on Tenure Track

d) Please indicate the type and format of courses you primarily teach.

Face-to-Face Blended or Hybrid 100% Online

• Undergraduate credit courses O O O

• Graduate courses O O O • Non-credit courses O O O


6

e) Gender

o Female

o Male

o Do not wish to respond

f) Time at OU

o Less than 1 year

o 2-5 years

o 6-9 years

o More than 10 years

g) What is your race/ethnicity? (Select all that apply)

▢ American Indian or Alaskan Native

▢ Asian

▢ Black or African American

▢ Hispanic or Latino/Latina

▢ Native Hawaiian or Other Pacific Islander

▢ White

▢ Mixed Race

▢ Other (please specify) ________________________________________________

1 a) a)

a) a)

b) b)

a) a)

b) b)

a) a)

b) b)

©OFFICE OF ACADEMIC ASSESSMENT - UNIVERSITY OF OKLAHOMA

Report includes information on dissemination of results to faculty and an action plan for continous improvement.

One or more of the descriptions does not address gaps in student performance and is not aligned with the SLOs, direct measures (and, where applicbale, indirect measures), performance targets and/or assessment results.

Descriptions for ALL SLOs address gaps in student learning and are directly aligned with the SLOs, direct measures (and, where applicbale, indirect measures), performance targets and assessment results.

Report shows critical evaluation of past and current assessment, including strengths and possible opportunities for continous improvement.

Descriptions for one or more SLOs are: (1) based exlusively on end-of-course grades or indirect measures, (2) devoid of specific examples of planned and/or implemented use of assessment results for program improvement, (3) consistently indicating that no changes are needed without further explanation.

Descriptions for ALL SLOs reflect specific examples of planned and/or implemented use of assessment results primarily from direct and, where applicable, indirect assessments for program improvement. Explanations are provided where descriptions indicate that no changes are needed.

3 ASSESSMENT RESULTS

Briefly describe students' actual performance on each SLO. Report data on aggregate .

4 USE OF ASSESSMENT RESULTS

Describe in detail how faculty have used or plan to use results of student achievement and, where possible, augment outcomes of direct measures with students' perceptions about their learning, to make curricular changes . Include an action plan on how data on student achievement are or will be used to support program improvements tied directly to student learning.

Use of assessment results information is missing.

STUDENT LEARNING OUTCOMES (SLOs)

Articulate 3-5 statements that reflect specific knowledge, skills or abilities graduates of your degree program should be able to

SLOs are missing.

Trends or patterns over time are discussed to indicate progress of student performance. Assessment results indicate areas where students excelled, met standards, and fell short.Results for one or more SLOs does not indicate the number of

students assessed.Results for ALL SLOs indicate the number of students assessed.

REVIEW RUBRIC FOR PROGRAM ASSESSMENT REPORTSThisrubricwasdevelopedbytheOfficeofAcademicAssessmenttoprovidefeedbackregardingtheannualProgramAssessmentReports.Therubricreflectsbriefdescriptionsofthe4-stepOUProgramAssessmentProcessandColorCodesindicatingtheoutcomeofthereviewsofcontentsforeachstep.Forinstance,ifaStudentLearningOutcome(SLO)isnotstatedin"Measurable"terms,thenitem"a"under"NEEDSREVISION"willbehighlightedinYellowtosignifythattheSLOneedstoberevised.Inshort,onlythehighlightedtext/area(inGreen,YelloworRed)appliestoyourreport.

COLLEGE______________________

Assessment results or findings are missing.

Results for one or more SLOs: (1) rely exclusively on end-of-course grades or indirect measures, (2) are not clearly described and not reported on aggregate, and (3) are not aligned with methods or performance targets in SLOs.

Results for ALL SLOs: (1) are presented using appropriate direct assessments, (2) are clearly described and reported on aggregate and (3) are directly aligned with methods and performance targets in each SLO.

2 ASSESSMENT METHODS

Describe how each SLO is assessed using at least one direct measure (e.g., exams, Capstone projects, etc.) and, where possible, an indirect measure (e.g., exit surveys).

Assessment methods are missing.

At least one of the methods does not reflect description of data collection process and use of appropriate direct measure. Performance targets are not reported and no explanation is provided.

ALL methods clearly describe the data collection process and include at least one appropriate direct measure for each SLO. Performance targets are reported -- if none is reported, an explanation is provided.

Report includes attachment of sample rubric(s).

One or more methods reflect exclusive use of end-of-course grades and/or indirect methods (e.g., surveys).

ALL methods reflect evaluation of the quality of students' work using a rubric (where appropriate). Indirect measures (e.g., surveys), if mentioned, are used only to supplement direct measures.

Methods include information on psychometric properties of assessment instruments or tools (e.g., inter-rater reliability of rubrics).

Degree Program:________________________________

One or more SLOs is unclear and not stated in measurable and/or observable terms. SLOs statements are too broad and/or stated in terms of program effectiveness goals, not student learning outcomes.

ALL SLOs are clearly stated in measurable and/or observable terms. SLOs reflect specific knowledge, abilities or skills graduates of the degree program are expected to demonstrate.

SLOs include initiatives that resulted from previous assessments.

OFFICE OF ACADEMIC ASSESSMENT

STEPS OF PROGRAM ASSESSMENT PROCESS

REVIEW COLOR CODES FOR EACH STEP OF THE ASSESSMENT PROCESS

RECOMMENDATIONSMISSING INFORMATION NEEDS REVISION MEETS EXPECTATIONS EXCEEDS EXPECTATIONS

Information for ONE OR MORE of the Assessment Steps is missing OR the assessment Report was NOT submitted.

Assessment Report reflects ONE OR MORE of the following IN EACH STEP of the assessment process:

Assessment Report reflects ONE OR MORE of the following IN EACH STEP of the assessment process:

All of the "Meets Expectations" indicators are met and AT LEAST ONE of the following additional indicators is met (IN EACH STEP).

designing and implementing a sustainable assessment process

Documents