Evidence of Student Learning

Fall Faculty SeminarOffice of Institutional Research and Assessment

August 15, 2012

Agenda • Overview of Assessment

• What assessment is• What it isn’t • Why we do it

• Assessment Levels • Classroom Assessment• Course Assessment• Program Assessment• General Education Assessment

• Institutional Commitment to Student Learning• Assessment of General Education• Data Analysis and Results

Assessment of Student Learning is . . .

… the collection of evidence about student learning, used for improvement of teaching and learning.

Owolabi, 2009

Defining Assessment of Student Learning

• Action research• Gathers indicators that will be useful for decision making• Standardized tests or “objective” measures• Assessment means basing decisions about curriculum,

pedagogy, staffing, advising, and student support upon the best possible data about student learning and the factors that affect it Walvoord, 2004, p. 2.

Assessment of Student Learning is not…

Is not program review

… about evaluating individual faculty members

… about using course grades to assess student performance

… an infringement on faculty academic freedom

… an assignment or test used by a single instructor to determine students’ understanding in a specific classroom

Benefits of Assessing Learning

• Focus on student learning• Articulate learning outcomes• Improve courses, programs, curricula, and pedagogy• Promote conversations and dialogue about

teaching and learning• Align curriculum with learning outcomes• Redesign our curriculum

Classroom Assessment

“ The teacher of the senior capstone course evaluates her students’ final projects, assigns grades, and uses the information for her own improvement next semester.”

Walvoord, 2004, p. 2

An instructor asks students to indicate the muddiest point in the concept he/she has just taught. The instructor uses the data to improve instruction

Course Assessment

“Assessment at the course level means not just assigning individual grades but also reflecting on how well students as a whole are achieving the course’s learning goals. This is done by aggregating assessment results…”

Suskie, 2009, p. 6

Course Assessment • What evidence of student

learning do we collect?• Are we systematic about

collecting these evidence?• What actions have we

taken to improve pedagogy and curriculum based on the results of assessment?

Program Assessment

“…helps determine whether students can integrate learning from individual courses into a coherent whole. It is interested in the cumulative effects of the educational process.”

Palomba and Banta (1999) (p.5-6).

“The faculty teaching the senior capstone report annually to the department, outlining the strengths and weaknesses of the students’ work in relation to departmental learning goals. The department uses these and other data, such as student and alumni surveys, to inform decisions about curriculum, pedagogy, and other factors tat affect student learning.” Walvoord, 2004, p. 3

Commitment to Student Learning

• General Education –Maturing Outcomes

• Diversity of Learners and Learning

• Co-curricular Learning and Learning Environments

Accreditations• Higher Learning Commission of North Central

Association of Colleges and Schools

• National Council for Accreditation of Teacher Education (NCATE)

• Council for Accreditation of Counseling and Related Educational Programs (CACREP)

• National Association of Schools of Music

AQIP Pathway1

4

7

Systems Portfolio

*

Action Projects

StrategyForum

Systems Appraisal

Quality Checkup

Reaffirmation of Accreditation

Higher Learning Commission of North Central Association of Colleges and Schools

Current Action Projects

• Mentoring New Faculty and Staff• Assessment Using a Data Driven Decision

Making Model• Resources for Undergraduate and Graduate

Students

Serra, 2011 “Academic Quality Improvement Program AQIP.” PowerPoint presentation . New Faculty Orientation.

General Education Outcomes

• Engage different viewpoints in thoughtful and deliberative ways by making critical judgments and defending them reasonably using appropriate quantitative and qualitative sources of information and methods

 • Read texts critically with understanding • Listen actively and communicate ideas effectively through

speaking and writing  • Examine the human, natural, and aesthetic worlds and

articulate their place in them 

General Education Outcomes

• Conduct research, engage in the ways of knowing and participate in discourse across the arts, sciences, and humanities

 • Reflect theologically and philosophically on significant

questions about the value and meaning of life and living  • Live and serve as wise, humane, responsible, and ethical

citizens of our diverse nation and world

•  Articulate a Christian world view and its implications for living

Assessment of General Education• ETS Proficiency Profile Test• Critical thinking • Reading• Writing• Mathematics• Humanities• Social sciences • Natural sciences

ETS Proficiency Profile Test• Reviewed mean scores and identified

trends

• Total mean score on the ETS Proficiency Profile test has a possible range of 400-500

• Sub scores have a possible range of 100-130

Demographic Variables

• Age• Best Language• Credit Hours• Curriculum

Completed• Enrollment Status• Ethnicity

• Gender• GPA• Hours Working• Major• Program Enrolled• Transfer Status.

ETS Proficiency Profile: Sub Scores

ETS – Reading Scores

ETS – Social Sciences

ETS Proficiency ProfileDemographic Analysis

AgeAge 2010-2011 2009-2010 2008-2009 2007-2008 2006-2007

<20 436.47 436.36 439.53 437.90 439.09

20-29 445.66 443.77 445.70 444.03 449.94

Difference 9.19 7.41 6.17 6.13 10.85

Performance by Cohort and Curriculum Completed

Curriculum Completed

2007/2008 2006/2007

Freshmen (None) 437.60 438.25

Seniors (100%) 450.41 447.46

Value Added 12.81 9.21

Performance by Cohort and Credit Hours

Credit Hours Completed

2007/2008 2006/2007

Freshmen (None) 437.20 438.28

Seniors (More than 90 Semester Hours or more than 145 quarter hours)

448.39 447.69

Value Added 11.19 9.41

Recommendations

• Eliminate one of the testing windows since the data is limited in usage and no item analysis can be done

• Develop additional method of assessing general education outcomes even if the ETS tests are deemed worth repeating as we currently do

• Develop signature assignments and rubrics to assess the newly defined general education outcomes.

Recommendations

• Embed the signature assignments in courses within the general education program

• Maximize the use of Black Board (BB) to manage our assessment data 

 • Restructure the assessment committee to

include representatives from colleges quality assurance committees

Assessment requires…

• Beginning where it makes sense

• Collecting data and information that is meaningful and useful

• Persevering through the process

Questions or comments