+
Measuring Teaching Quality in the Online ClassroomAnn H. TaylorDirector, Dutton e-Education InstituteCollege of Earth and Mineral Sciences
+How do we review teaching effectiveness at Penn State?
Per Policy HR-23 and the “Statement of Practices for the Evaluation of Teaching Effectiveness for Promotion and Tenure”: Student reviews via the SRTE Peer review
Procedures, including frequency, are left to academic units
“Your mileage may vary”!
+How can we review teaching effectiveness online?
Formal reviews
Student reviews
Informal reviews
+Are there differences in what we are looking for?
Seven Principles of Good Practice
Good practice in undergraduate education:
1.encourages contact between students and faculty,
2.develops reciprocity and cooperation among students,
3.encourages active learning,
4.gives prompt feedback,
5.emphasizes time on task,
6.communicates high expectations, and
7.respects diverse talents and ways of learning.
+Good teaching is good teaching.But the evidence may look quite different!
+
+What are we looking for?
Interactions with students (including response rates and availability)
Use of technology (including usability of instructor-created content)
Pedagogical strategies (including facilitation of discussions)
Overall management of the administrative aspects of the course
+
+What should we do with the results?
Celebrate and reward good teaching
Provide professional development opportunities Mentoring Learning design support Resources! (weblearning.psu.edu)
+Student Reviews
+What are students reviewing?
Course content
Interactions with instructors
Use of technology
Appropriateness of the technology
Pedagogical strategies
+What are some concerns?
What can, and cannot, students evaluate?
How can you get meaningful feedback?
Are students qualified for the task?
Will they complete online evaluations?
How do I get them to fill them out?
+What should faculty do with student feedback?
Be prepared to make a few changes
Take the small steps approach
Read results with a thick skin
Look for trends
Overlook outliers
Seek explanations, but don’t explain away
(see http://www.lehigh.edu/~infdli/FD-evaluations.htm)
+Tool 1: SEEQ
Students’ Evaluation of Educational Quality
Mid-semester feedback instrument
Developed by educational psychologist Herbert Marsh
Normed to determine the kinds of questions students really could answer!
32 standardized questions grouped into nine dimensions of teaching
+Tool 2: SALG
Student Assessment of their Learning Gains
Sample instrument that can be customized
Statements about the degree of "gain" (on a five-point scale) which students perceive they've made in specific aspects of the class
Includes open-ended questions
Delivered to students online
Sponsors provide a statistical report of the results
+Tool 3: SRTE
Student Rating of Teaching Effectiveness
177 Question Choices
4 mandatory items1. Are you taking this course as an elective? (If uncertain,
omit.)2. What grade do you expect to earn in this course?3. Rate the overall quality of this course.4. Rate the overall quality of the instructor.
2 open-ended mandatory items 1. What helped you learn in this course?2. What changes would improve your learning?
Up to 15 department items and up to 5 instructor items
+SRTE for online courses in EMSSRTE Question Bank Items:
Rate the instructor's availability for individual help and consultation. (#42)
Rate the effectiveness of the instructor's responses to student questions. (#48)
Rate the instructor's skill in keeping the course on schedule. (#70)
Rate the promptness with which graded exams, reports, and other materials were returned. (#101)
Additional questions:
Rate your agreement with the following statement: “The instructor initiated communication with students on a regular basis.”
Rate your agreement with the following statement: “The instructor responded to my inquiries in a timely manner.”
Rate your agreement with the following statement: “The instructor provided meaningful feedback on my class assignments.”
Rate your agreement with the following statement: “I would recommend this course to others.”
+Tool 4: DIY
Simple web-based form
Ask anything!
Lots of examples available
Lots of tools available Google Template SurveyMonkey Survey Gold Zoomerang
+Another alternative:Gather informal feedback
Minute papers, muddiest points
Self-check using formal instruments Peer Review of Online Teaching Faculty Competencies for Online
Teaching
Online Mentoring Program (pilot)
+How do we get started?
Start with faculty How do they define good teaching? What criteria do they think should be
measured? How do we involve part-time faculty?
Form a steering committee of key stakeholders
Take small steps and build on successes
Measure impact throughout the process
Strive to continuously improve
+What can administrators do?
Create an institutional environment where good teaching is highly valued & professional growth is expected
Provide adequate and accessible faculty development tools & resources
Tell faculty, staff, and students how this will benefit them
Measure impact!
Recognize and reward good teaching at all levels – celebrate!
Encourage trial and error
Act!
+What can faculty do?
Utilize a wide array of strategies to collect feedback
Document and share evidence of good teaching with superiors
Request faculty development programming
Work with faculty senate and other faculty bodies to strengthen support
Tell your students how this will benefit them
+So what are YOU going to do?