© 2008 brigham young universityâ€“idaho course evaluations at byu-idaho 1
Post on 04-Jan-2016
Embed Size (px)
Course Evaluations at BYU-Idaho1 2008 Brigham Young UniversityIdahoCourse Evaluations was a deliberate choice of terminology1Assess course quality and teaching performance.Assist in teacher improvement.Give students an opportunity to provide inputSatisfy accreditation requirementsPrimary Objectives of Course Evaluation2 2008 Brigham Young UniversityIdahoThe Numbers3F09W10S10Classes Evaluated1,5141,4061,264Instructors Evaluated494509485Evaluations Administered44,79443,03539,996 2008 Brigham Young UniversityIdahoWith 44 data items a semester, data items exceed 1.2M!!!3Completion Rates4
2008 Brigham Young UniversityIdahoCompletion Rates5CollegeF09W10S10Agricultural & Life Sciences65%64%62%Business & Communication69%69%67%Education & Human Development72%68%69%Language & Letters64%66%66%Performing & Visual Arts53%53%52%Physical Sci & Engineeering76%78%76%Grand Total66%68%68% 2008 Brigham Young UniversityIdahoThe Overall Instructor Rating6
2008 Brigham Young UniversityIdahoThe trough represents the transitional period: new courses, new programs, busy faculty, etc.6We aggregate and track . . .Overall instructor ratingOverall course ratingPerceived learningHours of preparationSatisfactionLearning model scales
We use the overall instructor rating to . . .Support CFS decisionsCompare classifications (online, adjunct, veteran, 1-year)Watch trendRed flag instructors at or below 10%tile
How Does the Administration Use The Data?7 2008 Brigham Young UniversityIdahoAdministration = Academic Office and President; we track Big 6 for colleges and departments7Time of Day
Student traitsAgeAcademic aptitudeGPAClass levelPersonalityInstructor TraitsAgeYears of teaching experienceWhat the Research Shows Non-factors8 2008 Brigham Young UniversityIdahoClass SizeSmaller classes tend to receive higher ratings.Reason for Taking CourseElective courses receive higher ratings than required courses.Expected GradePositive but low.
DisciplineThe highest ratings go in the following order to:Arts & HumanitiesBiological & Social SciencesBusiness & Computer ScienceMath, Engineering, & Physical Sciences
What the Research Shows - Factors9 2008 Brigham Young UniversityIdahoDifficulty LevelCourses that are more difficult or have greater workloads received higher ratings.
Course LevelUpper division receive higher ratings than lower division.
MotivationPrior interest in subject matter or class leads to higher ratings.
Student MajorMajors are a bit more positive.
What the Research Shows - Factors10 2008 Brigham Young UniversityIdahoInstructor GenderSame-gender instructors receive slightly higher ratings.Instructor PersonalityKnowledgeable, warm, outgoing, and enthusiastic teachers receive higher ratings.ScholarshipTeachers with more publications receive slightly higher ratings.TimingAdministration during final exams are lower.PRFraming re promotion & tenure gets slightly higher ratings.What the Research Shows - Factors11 2008 Brigham Young UniversityIdahoDo these findings hold for byu-idaho?12 2008 Brigham Young UniversityIdahoFindings From 2009 2010 Data13
2008 Brigham Young UniversityIdahoSubstantiates the research. Pattern did not hold for L&L (major/minor > elective by far) and ED (Foundations > major) and PVA (major/minor > elective by a little)13Findings From 2009 2010 Data14
2008 Brigham Young UniversityIdahoMeasure of difficulty, probably more investment in course. Pattern holds for all colleges except VPA. 14Findings From 2009 2010 Data15
2008 Brigham Young UniversityIdahoSubstantiates the research. Pattern holds almost identically for each college.15Findings From 2009 2010 Data16
2008 Brigham Young UniversityIdahoSubstantiates the research. No relationship at the university level. However . . . 16Findings From 2009 2010 Data17
PS & EngAg & LSBus & CommEd & HD 2008 Brigham Young UniversityIdahoMeans disguise these interesting patterns.17Findings From 2009 2010 Data18Gender of StudentGender of InstructorFemaleMaleFemale5.855.69Male5.825.89Overall5.835.86 2008 Brigham Young UniversityIdahoNot good.18Findings From 2009 2010 Data19
2008 Brigham Young UniversityIdahoSubstantiates the research. Discipline difference.19Findings From 2009 2010 Data20
2008 Brigham Young UniversityIdahoExactly what research shows. Uncanny!!20Supplement with direct measures of learningAdjust the composite measures forDisciplineGenderReason for taking classGrade ExpectedRework the Learning Model scoresBetter reportsShorten the instrumentDevelop some information for studentsDreams for the Future21 2008 Brigham Young UniversityIdahoAdjust for factors which penalize but are outside of teachers control.21Are we asking the right questions?Are we asking too many questions?Should we evaluate every course every semester?Is the timing of the evaluation optimal?How do you use course evaluation data (numbers and comments)?Is the information valuable?Are the reports adequate?Should some of the course evaluation data be made available to students to counter biased information on the web?
Discussion Items22 2008 Brigham Young UniversityIdaho
2008 Brigham Young UniversityIdaho 2008 Brigham Young UniversityIdaho23Raw vs Weighted Averages24
85% see little or no difference (-.1, 0, .1) = 85% 76 get a better rating; 21 get a worse rating 2 get a significantly worse rating 2008 Brigham Young UniversityIdaho