© 2008 Brigham Young University–Idaho 1

Course Evaluations at BYU-Idaho

© 2008 Brigham Young University–Idaho 2

1. Assess course quality and teaching performance.

2. Assist in teacher improvement.

3. Give students an opportunity to provide input

4. Satisfy accreditation requirements

Primary Objectives of Course Evaluation

© 2008 Brigham Young University–Idaho 3

The Numbers

F09 W10 S10

Classes Evaluated 1,514 1,406 1,264

Instructors Evaluated 494 509 485

Evaluations Administered 44,794 43,035 39,996

© 2008 Brigham Young University–Idaho 4

Completion Rates

© 2008 Brigham Young University–Idaho 5

Completion Rates

College F09 W10 S10

Agricultural & Life Sciences 65% 64% 62%

Business & Communication 69% 69% 67%

Education & Human Development 72% 68% 69%

Language & Letters 64% 66% 66%

Performing & Visual Arts 53% 53% 52%

Physical Sci & Engineeering 76% 78% 76%

Grand Total 66% 68% 68%

© 2008 Brigham Young University–Idaho 6

The Overall Instructor Rating

© 2008 Brigham Young University–Idaho 7

• We aggregate and track . . .• Overall instructor rating• Overall course rating• Perceived learning• Hours of preparation• Satisfaction• Learning model scales

• We use the overall instructor rating to . . .• Support CFS decisions• Compare classifications (online, adjunct, veteran, 1-year)• Watch trend• Red flag instructors at or below 10%tile

How Does the Administration Use The Data?

© 2008 Brigham Young University–Idaho 8

Time of Day

Student traits• Age• Academic aptitude• GPA• Class level• Personality

Instructor Traits• Age• Years of teaching

experience

What the Research Shows – Non-factors

© 2008 Brigham Young University–Idaho 9

Class SizeSmaller classes tend to receive higher ratings.

Reason for Taking Course

Elective courses receive higher ratings than required courses.

Expected GradePositive but low.

DisciplineThe highest ratings go in the following order to:

1. Arts & Humanities

2. Biological & Social Sciences

3. Business & Computer Science

4. Math, Engineering, & Physical Sciences

What the Research Shows - Factors

© 2008 Brigham Young University–Idaho 10

Difficulty LevelCourses that are more difficult or have greater workloads received higher ratings.

Course LevelUpper division receive higher ratings than lower division.

MotivationPrior interest in subject matter or class leads to higher ratings.

Student MajorMajors are a bit more positive.

What the Research Shows - Factors

© 2008 Brigham Young University–Idaho 11

Instructor GenderSame-gender instructors receive slightly higher ratings.

Instructor Personality

Knowledgeable, warm, outgoing, and enthusiastic teachers receive higher ratings.

ScholarshipTeachers with more publications receive slightly higher ratings.

TimingAdministration during final exams are lower.

PRFraming re promotion & tenure gets slightly higher ratings.

What the Research Shows - Factors

© 2008 Brigham Young University–Idaho 12

DO THESE FINDINGS HOLD FOR BYU-IDAHO?

© 2008 Brigham Young University–Idaho 13

Findings From 2009 – 2010 Data

© 2008 Brigham Young University–Idaho 14

Findings From 2009 – 2010 Data

© 2008 Brigham Young University–Idaho 15

Findings From 2009 – 2010 Data

© 2008 Brigham Young University–Idaho 16

Findings From 2009 – 2010 Data

© 2008 Brigham Young University–Idaho 17

Findings From 2009 – 2010 Data

PS & Eng

Ag & LS Bus & Comm

Ed & HD

© 2008 Brigham Young University–Idaho 18

Findings From 2009 – 2010 Data

Gender of Student

Gender of Instructor Female Male

Female 5.85 5.69

Male 5.82 5.89

Overall 5.83 5.86

© 2008 Brigham Young University–Idaho 19

Findings From 2009 – 2010 Data

© 2008 Brigham Young University–Idaho 20

Findings From 2009 – 2010 Data

© 2008 Brigham Young University–Idaho 21

• Supplement with direct measures of learning• Adjust the composite measures for

• Discipline• Gender• Reason for taking class• Grade Expected

• Rework the Learning Model scores• Better reports• Shorten the instrument• Develop some information for students

Dreams for the Future

© 2008 Brigham Young University–Idaho 22

• Are we asking the right questions?• Are we asking too many questions?• Should we evaluate every course every semester?• Is the timing of the evaluation optimal?• How do you use course evaluation data (numbers

and comments)?• Is the information valuable?• Are the reports adequate?• Should some of the course evaluation data be

made available to students to counter biased information on the web?

Discussion Items

© 2008 Brigham Young University–Idaho© 2008 Brigham Young University–Idaho

© 2008 Brigham Young University–Idaho 24

Raw vs Weighted Averages

• 85% see little or no difference (-.1, 0, .1) = 85%• 76 get a better rating; 21 get a worse rating• 2 get a significantly worse rating