putting the results into practice
TRANSCRIPT
![Page 1: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/1.jpg)
Center for Excellence in Learning and Technology
Putting the Results from the Student Evaluation of Teaching
into PracticeJodi Sterle, Ph.D.
Harman Endowed Professor In Teaching & Learning and
Teaching Section Leader
Animal [email protected]
http://www.ans.iastate.edu/
Laura Bestler, Ph.D.Program [email protected] 294-5357 http://www.celt.iastate.edu/
![Page 2: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/2.jpg)
Agenda
•About Online Course Evaluations•The Process•Types of Reports•Who has access? Who views the reports?•Big Picture•How Animal Science Uses Course Evaluations•Moving Forward•Student Participation•Other Uses for Class Climate
![Page 3: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/3.jpg)
About Online Course Evaluations
![Page 4: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/4.jpg)
In 2002, a national study found that 98% of the nation's "most-wired" colleges, still used a paper-based for the student evaluation of teaching (Hoffman, 2003).
![Page 5: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/5.jpg)
Paper Course Evaluations
![Page 6: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/6.jpg)
Paper Course EvaluationsPros Cons
Done in-class - good response rates Very time consuming (preparation before and after administration)
Students feel anonymous May take 4-12 weeks for faculty to get results
Resource intensive to type up answers to open ended questions
Lots of room for error (scanning errors, student errors-using pen, etc., illegible comments, handling errors: students can tamper with data or forget to return, people often put forms in a packet for the wrong class, etc)
Students may be apathetic and just fill in anything
Costly (cost of forms, pencils, bins, envelopes, work hours, scanner)
Bad for the environment (uses lots of paper)
Students may not take time on open ended questions because of time
![Page 7: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/7.jpg)
Online Course Evaluations
![Page 8: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/8.jpg)
Online Course Evaluations
Pros Cons
Immediate results Lower response rate?
Far less room for error (no lost forms, scanning issues)
Has to be done on the students’time (unless technology allows for in-class)
Far less time consuming Anonymity concerns
Far less costly (no scanner, paper forms, much less work hours,etc.)
Students who have not attended class are now able to give opinions
Green- no need for paper
More student comments
Flexibility for questions/scales
Students who take the time to do them have an opinion
Students are able to fill them out 24 hours a day
Less time intensive for administrators
![Page 9: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/9.jpg)
Iowa State UniversityISU started using Scantron's Class Climate online course evaluation system on a pilot basis in Spring 2011.
S2011 F2011 S2012 F2012
DEPTS 56 92 103 115
COURSES/SECTIONS 2071 4680 4576 5574
INSTRUCTORS 717 1540 1667 1901
EVAL INVITES 55,359 142,568 159,420 165,950
![Page 10: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/10.jpg)
Response Rates
F2012
S2012
F2011
S2011
OVERALL AVERAGE 59.74 62.30 63.50 74.40
WEIGHTED AVERAGE 60.17 59.65 63.18 71.57
Weighted Average: is taking the difference of the number of students into accountFormula: =sumproduct(enrolled students in the course,response rate/sum(enrolled students in the course)
![Page 11: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/11.jpg)
The Process
![Page 12: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/12.jpg)
The Process: Departments
Each department/college has local control of:• evaluation questions• email texts• when email invitations are sent out• number/frequency of email reminders• when evaluations starts and ends• when instructors receive course reports
![Page 13: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/13.jpg)
The Process: Email Invitation
SUBJECT: ISU Online Course Evaluation for [COURSENAME] [COURSETYPE]
Iowa State Student [PARTICIPANT_EMAIL],
This email asks you to respond to an anonymous online course evaluation survey for[COURSENAME] [COURSETYPE].Please follow the link to open the course evaluation survey, fill-out the questionnaire, and click “submit” when completed.
[DIRECT_ONLINE_LINK]
Please complete this evaluation by 12:00 p.m. (Noon) on Friday, Month XX, 2012. This survey will not be reopened.You will receive this reminder email periodically until the survey is complete. Please complete all online surveys you receive. Links to your course surveys will also appear in your Blackboard Learn page under “My Course Evaluations.”
Thank you for your participation,- [SUBUNIT] Course Evaluation Administrator
![Page 14: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/14.jpg)
The Process: Email ReminderEMAIL HEADER: Course Evaluation Reminder for [COURSENAME] [COURSETYPE]
Iowa State Student [PARTICIPANT_EMAIL],
This email is to remind you that you have not yet completed the survey for:[COURSENAME] [COURSETYPE].
Please follow the link to open the course evaluation survey, fill-out the questionnaire, and click “submit” when completed.
[DIRECT_ONLINE_LINK]
Please complete this evaluation by 12:00 p.m. (noon) on Friday, Month XX, 2012. This survey will not be reopened.
You will receive this reminder email periodically until the survey is complete. Please complete all online surveys you receive. Links to your course surveys will also appear in your Blackboard Learn page under “My Course Evaluations.”Thank you for your participation,- [SUBUNIT] Course Evaluation Administrator
![Page 15: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/15.jpg)
The Process: Students fill out the evaluation
![Page 16: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/16.jpg)
The Process: Departments/College run and send Instructor/Course Reports
![Page 17: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/17.jpg)
Types of Reports
![Page 18: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/18.jpg)
Example Course Report
![Page 19: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/19.jpg)
Instructor Profile:a combined report of all courses taught during a term.
Example Instructor Profile
![Page 20: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/20.jpg)
Report Creator: Any Compilation (Tree structure)
Example of Department Overall report
![Page 21: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/21.jpg)
Example Course Report: Open Ended Questions
![Page 22: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/22.jpg)
Example of Profile Line
![Page 23: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/23.jpg)
Report for Comparison on Questionnaire
The “Hook” Profile line into standard Report
![Page 24: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/24.jpg)
Who has access?Who views the reports?
![Page 25: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/25.jpg)
Who has access to Class Climate?
• Departments/Colleges:• Designated administrator(s) (Merit, P&S, or Faculty
member)• Only their respective department(s)
• System-wide:• CELT administrator for Class Climate
(2 FT P&S staff members)• IT administrator for Class Climate
(1 FT P&S staff member)
![Page 26: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/26.jpg)
Who sees instructor reports?
• Departmental designated administrator(s) (Merit, P&S, or Faculty member)
http://bit.ly/ccadmins
• Instructor (may share with whomever they wish)
• Department Chair and/or College Administrator
• Teaching Coordinator or other departmental designee
![Page 27: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/27.jpg)
Big Picture
![Page 28: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/28.jpg)
Big Picture: What is your Position Responsibility Statement (PRS)?
• Research, Teaching, Extension and Outreach
• Each area needs to be documented and then evaluated
VanDerZanden, A.M.. (2012, January 30). You have course evaluations: Now what? Department of Animal Science [Presentation]. Ames, IA: Iowa State University.
![Page 29: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/29.jpg)
Big Picture: Teaching
Evaluation of
Teaching
Peer review of teaching
Portfolio of teaching
Teaching philosophy
Student evaluation of teaching
VanDerZanden, A.M.. (2012, January 30). You have course evaluations: Now what? Department of Animal Science [Presentation]. Ames, IA: Iowa State University.
![Page 30: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/30.jpg)
Promotion & Tenure
http://www.provost.iastate.edu/help/promotion-and-tenure
![Page 31: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/31.jpg)
How Animal Science Uses Online Course Evaluations
![Page 32: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/32.jpg)
• 969 undergraduates
• ~200 grad students
• >80 courses/sections per semester
• Every course and section evaluated using ClassClimate since Spring ‘11
• Senior exit surveys
• Advising surveys
Animal Science
![Page 33: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/33.jpg)
• Evaluation results and departmental averages sent to instructors
• Teaching coordinator and Department Head review
• Teaching coordinator assists in interpretation of results and development of improvement plan
• Department Head addresses results in annual review
Using the Results
![Page 34: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/34.jpg)
• Teaching coordinator and Department Head also pass along outstanding teaching faculty names to awards committee
• Use results (with benchmarks, over time) to show improvement
• P&T packets
• Used with Peer Evaluation of Teaching evaluations
Using the Results
![Page 35: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/35.jpg)
• Lots of initial anxiety over move to online (and mandatory) course evals
• Love getting results early
• Open and transparent
• Departmental trends
• Focus on improvement
What I’ve Seen
![Page 36: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/36.jpg)
Moving Forward
![Page 37: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/37.jpg)
•Snap shot in time
•Don’t take everything at face value
•Concentrate on the 80%; not the 20%
•Discuss with a trusted mentor
• What can you do with the constructive feedback?
Moving Forward: Reflecting on Fall 2012
VanDerZanden, A.M.. (2012, January 30). You have course evaluations: Now what? Department of Animal Science [Presentation]. Ames, IA: Iowa State University.
![Page 38: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/38.jpg)
Moving Forward: Strategies for Analyzing Feedback
• Control your defense mechanisms.
• Analyze the source of your students’ reactions in a way that sheds light on any issues and problems that have been identified.
• Work hard not to under-react or over-react to information that you receive via evaluation feedback.
• Divide the issues raised by students into actionable and non-actionable categories.Moore, S., & Kuol, N. (2005). A punitive tool or a valuable resource? Using student evaluations to enhance your teaching. In G. O’Neill, S.
Moore, & B. McMulline (Eds)., Emerging issues in the practice of university learning and teaching (pp. 141-148). Dublin: All Ireland Society for Higher Education.
![Page 39: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/39.jpg)
• Communicate with students before and after their provision of feedback.
• Do not make the simplistic assumption that all positive responses are related to good teaching and all negative responses are related to bad teaching.
• Remember that small changes can have big effects.
• Develop a teaching enhancement strategy that takes into account the evaluation feedback (145-6).
Moore, S., & Kuol, N. (2005). A punitive tool or a valuable resource? Using student evaluations to enhance your teaching. In G. O’Neill, S. Moore, & B. McMulline (Eds)., Emerging issues in the practice of university learning and teaching (pp. 141-148). Dublin: All Ireland Society for Higher Education.
Moving Forward: Strategies for Analyzing Feedback
![Page 40: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/40.jpg)
Comments Analysis Worksheet
![Page 41: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/41.jpg)
Comments Analysis Worksheet
• Often multiple comments are related to the same category; for example:10 students may all make comments about the assignments being unclear. This is not really 10 different comments but rather one comment 10 times.
• The multiple mentions give it weight, but it is only one area that needs to be addressed for improvement.
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 Canada License. Winer, L., Di Genova, L., Vungoc, P.-A., & Talsma, S. (2012). Comments analysis worksheet. Excerpted from Interpreting end-of-course evaluation results. Montreal: Teaching and Learning Services, McGill University available at: http://www.mcgill.ca/tls/teaching/course-evaluations/interpretation.
![Page 42: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/42.jpg)
Comments Analysis Worksheet
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 Canada License. Winer, L., Di Genova, L., Vungoc, P.-A., & Talsma, S. (2012). Comments analysis worksheet. Excerpted from Interpreting end-of-course evaluation results. Montreal: Teaching and Learning Services, McGill University available at: http://www.mcgill.ca/tls/teaching/course-evaluations/interpretation.
![Page 43: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/43.jpg)
Comments Analysis WorksheetThe worksheet has been organized alphabetically in sections according to most frequently commented categories.• Note any student comments that will help in
interpretation. • Indicate positive and negative comments.• Record the frequency of comments surrounding each
theme to help identify the areas where students felt most strongly.
• Add any personal notes that will help in the process of building on the feedback received.
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 2.5 Canada License. Winer, L., Di Genova, L., Vungoc, P.-A., & Talsma, S. (2012). Comments analysis worksheet. Excerpted from Interpreting end-of-course evaluation results. Montreal: Teaching and Learning Services, McGill University available at: http://www.mcgill.ca/tls/teaching/course-evaluations/interpretation.
![Page 44: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/44.jpg)
How Can You Help with Student Participation?
![Page 45: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/45.jpg)
Increasing Student Participation
• Give students examples on how the results enhance your course and teaching.
• Include the course evaluation start/end date and/or information in your course syllabus.
• Add the “My Course Evaluation” module into your BbLearn course.
• Request that your departmental administrator send you the response rate notification prior to the end of the evaluation period. You will then receive an automatic email that your course has a low response rate (set at the local level).
Contact your departmental administrator:http://bit.ly/ccadmins
![Page 46: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/46.jpg)
Should there be incentives for completing online course evaluations?
24% No
70% Yes
6% Don’t Care
2012 Comets survey 5000 random ISU students 763 students responded15% response rate
![Page 47: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/47.jpg)
What would be considered good incentives?
![Page 48: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/48.jpg)
Additional Uses for Class Climate
![Page 49: Putting the Results into Practice](https://reader035.vdocuments.mx/reader035/viewer/2022062419/5583f4f9d8b42afa438b4dd6/html5/thumbnails/49.jpg)
What else can Class Climate be used for?
• Mid-semester course/instructor evaluations
• senior exit surveys• advising surveys• program evaluations• research projects (STEM, etc.)
Contact your departmental administrator:http://bit.ly/ccadmins