1

Student Learning Outcomes Workshop University of Tennessee Chattanooga – September 24, 2012

Jillian Kinzie, Indiana University, Center for Postsecondary Research

Assessment is an ongoing process aimed at understanding and improving student learning. It involves: Assessment

• Making program objectives and student learning outcomes explicit and public • Setting appropriate criteria and high expectations • Systematically gathering, analyzing, and interpreting evidence to

determine how well programs and services match those expectations and standards

• Using the resulting information to document, explain, and improve • Demonstrating student learning gains and contributions to student

success • Showcasing department/program strengths • Identifying areas where the department/program can improve

Assessment needs to be authentic and meaningful to those orchestrating it. There should be furthered through a feeling of “ownership” about assessment activities. Assessment should ideally work to provide a deep understanding of the connection between institutional, departmental/major programs and course level learning and the effects on students. Assessment should additionally and simultaneously help improve the student experience based on such tangible factors as performance, retention, and graduation, and such less-tangible factors as student health, citizenship, and overall lifelong learning and well-being. At its best, assessment reflects the ethic of inquiry that informs academic and faculty work – it brings scholarly habits of research and inquiry into the work of teaching and learning. Assessment generally occurs at 3 levels: Institutional, Program or Department, and the course/classroom level.

Institutional Level = identified set of learning outcomes expected of all students; assessment provides broad picture of students program on outcomes; additionally, campuses employ assessments of overall quality of the student experience; activities usually directed at central level, but results can inform department/program and class level and including in and out of classroom learning environment. Program/Department Level = articulation of program goals and learning outcomes at the department or major level – What should our majors know and be able to do? What kind of experiences do our students have and what do we know about what makes a difference to post-college outcomes? Locally developed approaches to assessment, including portfolios, presentations, senior projects, and even Major Field Tests, can be used; Assessment activities can be powerful educational experiences for students, and provide information to departments for determining needed improvements. Course/Classroom Level = assessment of course level student learning and quality of course experiences – always integral to classroom teaching! Approaches can be simple, and formative – asking students about the “muddiest point” – or complex involving pre-post tests of central course concepts, or using rubrics across multiple sections of a course.

2

Program Goals & Outcomes:

Program goals are broad statements that describe the overarching long-range intended goals of the academic unit/program. Describes the primary function of academic unit/program; key activities involved; and student competencies upon graduation from the academic unit/program. Goals may align with respective professional organization and program-specific accrediting bodies.

Program Goals & Outcomes should describe: 1. What should a graduate of your program know, do, and value? 2. What program experiences contribute to producing and supporting students in your program?

Do your goals describe desired performance? Checklist to determine if the goal statements are appropriate.

Are your goals consistent with your institutional and program mission?

Are your goals aligned with program values and key experiences?

Student Learning Outcomes (SLOs): Student Learning Outcomes (SLOs) are specific, measurable statements that use action verbs to articulate the knowledge, skills, and perceptions students should gain or improve through engagement in the academic program or learning experience during a specific time frame. Distinguishing Course (SLOs) from Program (SLOs)? • Program student learning outcomes (SLOs) identify what students should learn as a result of their entire

educational experience. For example “Upon completion of the XXXX program, students will be able to discuss the processes of urbanization and modernization in the developing world.”

• Course student learning outcomes (SLOs) identify what students should learn as a result of taking a specific course. For example “Upon completion of COUR XXX, students will be able to analyze cities as products of modernization, as expressions of various processes, such as investment and employment.”

• Program and course student learning outcomes (SLOs) are connected via disciplinary and institutional outcomes e.g., content knowledge/major, general education competencies, accreditation standards.

• Although every course student learning outcome (SLO) does not have to reflect every program, institutional and disciplinary student learning outcome, course student learning outcomes (SLOs) should contribute in some combination to those outcomes. Mapping of course student learning outcomes (SLOs) to program, institutional and disciplinary student learning outcomes (SLOs) helps assure there are integrated and layered opportunities for students to develop the desired outcomes.

Adapted from: Maki, P.L. (2004). Assessing for learning: Building a sustainable commitment across the institution. Sterling, VA: Stylus. Purpose and Use of SLOs: • SLOs identify skills that our students are learning. Are these the skills that we want them to learn? Are

these the skills that we are teaching them? • SLOs enable students to demonstrate what they are learning or have learned (both in- and outside of

the classroom). • SLOs help faculty, chairs/department heads, deans, administrators, and staff understand how to better

facilitate student learning through continuous quality improvement.

3

To consider when developing student learning outcomes (SLOs): 1. Focus on the end result – student learning.

“Upon completion of __________, the student will be able to _______” 2. Determine a limited number of student learning outcomes that are most applicable to either the

course or program. For program student learning outcomes, a general guideline would be 3-5. 3. Use concrete action verbs that result in observable behavior that can be measured (see table

below). Avoid verbs that call for students’ behavior which cannot be easily measured (e.g. exposed to, familiar with, learn, understand or know).

Adapted from: a. Palo Alto Colleges (http://www.alamo.edu/pac/assessment/development/AssessmentResources.aspx) b. Bloom, B.S. (Ed.) (1956). Taxonomy of educational objectives: The classification of educational goals: Handbook I, cognitive domain. New York: Longmans, Green. c. University of Louisville, Office of Academic Planning & Accountability, Institutional Effectivenes (Gilchrist, Goldstein & Bays)

Bloom’s Taxonomy of Cognitive Skills Action Verb List

4. Ensure that the student learning outcome (SLO) can be measured and that learning is being demonstrated.

Cognitive Learning Examples of Action Verbs: Remembering: can the student recall or remember the information?

define, duplicate, list, memorize, recall, repeat, reproduce state

Understanding: can the student explain ideas or concepts?

classify, describe, discuss, explain, identify, locate, recognize, report, select, translate, paraphrase

Applying: can the student use the information in a new way?

choose, demonstrate, dramatize, employ, illustrate, interpret, operate, schedule, sketch, solve, use, write.

Analyzing: can the student distinguish between the different parts?

appraise, compare, contrast, criticize, differentiate, discriminate, distinguish, examine, experiment, question, test.

Creating: can the student create new product or point of view?

assemble, construct, create, design, develop, formulate, write.

Affective Learning

appreciate, accept, attempt, challenge, defend, dispute, join, judge, praise, question, share, support

Psychomotor Learning

write, manipulate, react

4

Examples of Student Learning Outcomes (SLO) Student learning outcomes that are general and hard to measure1… … will value exercise as a stress reduction tool. … will be able to develop and apply effective problem solving skills that would enable one to adequately navigate through the proper resources within the university. … will demonstrate ability to resolve personal conflicts and assist others in resolving conflicts. … will demonstrate effective communication, such as facilitating conversations about social issues.

Student learning outcomes that are specific and relatively easy to measure1 … … will be able to explain how exercise affects stress. … will be able to identify the most appropriate resource that is pertinent to their university concern. … will be able to assist roommates in resolving conflicts by helping them negotiate agreements. … will demonstrate the ability to analyze and respond to arguments about racial discrimination.

1 Adapted from University of Rhode Island, Office of Learning & Outcomes Assessment (assess@uri.edu)

Checklist to ensure that your learning outcomes are adequately stated Aligned to the mission (and goal) statements

: Do/Are the SLO statements:

Indicate the level and type of competence that is required of graduates of a program Distinctive and specific to the program Framed in terms of the program rather than individual courses or individual students Simply stated so outcomes requiring different assessment methods are not bundled into 1

statement Describe intended outcomes not actual outcomes Focus on the learning results, not the learning process Stated so that more than one measurement method could be used Measurable and there are available resources for measurement Useful to identify areas to improve Include concrete action verbs rather than passive verbs

METHODS: Traditionally, assessment methods have been categorized as either direct or indirect. These two classifications are based on the distinction between assessing SLOs and student experience. Direct assessors evaluate the competence of students in the program. Indirect assessors are concerned with students’ experiences, opinions, or perceptions, rather than knowledge and skills (national, local and program surveys).

A Resource for methods: “Measuring Quality Inventory” sponsored by AIR, ACE, and NILOA http://applications.airweb.org/surveys/

5

DIRECT MEASURES Performance-Based

• Capstone course assignments • Capstone projects • Case studies, hypothetical situation

responses • Minute papers • Course-embedded questions • Portfolio assignments (standard across

program) • Assessment of papers, projects with

standard rubric • Research papers • Performance appraisal of in class exercises • Expert evaluation

Other

• Observations by evaluator (not the teacher) • Peer evaluation of practical skills using rubric • Clinical practice or internship skill

assessment

Examinations and Tests • Standardized Exams & Tests

• National Test • State Test

• Local Examinations and Tests • Local Tests • Pre-post test • Test-embedded questions (across

several course sections) • Licensure Exams

(Adapted from the University of Central Florida Academic Program Assessment Handbook; Suskie, 2009; Maki, 2004)

Blending Direct & Indirect Evidence: Use a matrix to categorize each assessment tool as being direct or indirect depending whether instruments are used to evaluate a student’s abilities & knowledge of academic support services (direct) or reflect perception of academic activities (indirect).

Example 1: Matrix of Direct & Indirect measures

Targets and Timeframe: When defining the outcomes, it is important to describe where you would like to be within a specified time period (e.g., 10% improvement in exam scores within one year, 90% satisfaction rating for next year, 10% improvement in student communication performance within two years). Also, determine what standards are expected from students in your program. For some learning outcomes, you may want 100% of graduates to achieve them. This may be unrealistic for other outcomes. You may want to determine what proportion of your students achieve a specific level (e.g., 80% of graduates pass the written portion of the standardized test). If you have previously measured an outcome, it is helpful to use this as the baseline for setting a target for next year.

To use SLO assessment data to facilitate student learning through quality improvement: • Involve your faculty in planning how to best use results to improve teaching and learning. • Use results to demonstrate accountability by sharing results internal & externally. • Address disappointing assessment in positive and developmental manner.