evaluating courses & teaching gregory light, phd denise drane, phd palestinian polytechnic...
TRANSCRIPT
![Page 1: Evaluating Courses & Teaching Gregory Light, PhD Denise Drane, PhD Palestinian Polytechnic University Hebron March 17 th, 2014](https://reader036.vdocuments.mx/reader036/viewer/2022062515/56649c745503460f94926eeb/html5/thumbnails/1.jpg)
Evaluating Courses & Teaching
Gregory Light, PhDDenise Drane, PhD
Palestinian Polytechnic UniversityHebron
March 17th, 2014
![Page 2: Evaluating Courses & Teaching Gregory Light, PhD Denise Drane, PhD Palestinian Polytechnic University Hebron March 17 th, 2014](https://reader036.vdocuments.mx/reader036/viewer/2022062515/56649c745503460f94926eeb/html5/thumbnails/2.jpg)
Course EvaluationWorkshop Overview
• Situating Evaluation• Why Evaluate (discussion)• Student ratings (summative &
formative) (Interactive discussion)
• Formative Evaluations (Activity)
![Page 3: Evaluating Courses & Teaching Gregory Light, PhD Denise Drane, PhD Palestinian Polytechnic University Hebron March 17 th, 2014](https://reader036.vdocuments.mx/reader036/viewer/2022062515/56649c745503460f94926eeb/html5/thumbnails/3.jpg)
1. What learning outcomes do you want your students to achieve, (intellectual, practical, interpersonal, and personal) as a result of taking your course?
2. How will your course help your students achieve these learning outcomes?
3. How will you know if the students on your course have achieved these learning outcomes?
4. How will you know if and how your teaching has contributed to your students’ learning outcomes?
Situating Evaluation in the Teaching for Learning
Quality Cycle
Light G., Cox R. & Calkins S. (2009
![Page 4: Evaluating Courses & Teaching Gregory Light, PhD Denise Drane, PhD Palestinian Polytechnic University Hebron March 17 th, 2014](https://reader036.vdocuments.mx/reader036/viewer/2022062515/56649c745503460f94926eeb/html5/thumbnails/4.jpg)
1. What learning outcomes do you want your students to achieve, (intellectual, practical, interpersonal, and personal) as a result of taking your course?
2. How will your course help your students achieve these learning outcomes?
3. How will you know if the students on your course have achieved these learning outcomes?
4. How will you know if and how your teaching has contributed to your students’ learning outcomes?
Situating Evaluation in the Teaching for Learning
Quality Cycle
Light G., Cox R. & Calkins S. (2009
![Page 5: Evaluating Courses & Teaching Gregory Light, PhD Denise Drane, PhD Palestinian Polytechnic University Hebron March 17 th, 2014](https://reader036.vdocuments.mx/reader036/viewer/2022062515/56649c745503460f94926eeb/html5/thumbnails/5.jpg)
Why evaluate our courses
and our teaching?
![Page 6: Evaluating Courses & Teaching Gregory Light, PhD Denise Drane, PhD Palestinian Polytechnic University Hebron March 17 th, 2014](https://reader036.vdocuments.mx/reader036/viewer/2022062515/56649c745503460f94926eeb/html5/thumbnails/6.jpg)
Purposes of Evaluation• For departmental purposes (program
review)• For institutional purposes (appraisal,
P&T, etc.)• For external purposes (NSSE, school
ranking, accreditation, etc.)• To improve student learning in the
course• To identify and correct problems• To better understand how students
approach learning in the course
• To Improve a course• To better align the course with your learning
goals• To improve quality of activities,
assignments, assessments• To find out if you achieved your goals
• To develop as a teacher
![Page 7: Evaluating Courses & Teaching Gregory Light, PhD Denise Drane, PhD Palestinian Polytechnic University Hebron March 17 th, 2014](https://reader036.vdocuments.mx/reader036/viewer/2022062515/56649c745503460f94926eeb/html5/thumbnails/7.jpg)
Purposes of Evaluation• For departmental purposes (program
review)• For institutional purposes (appraisal,
P&T, etc.)• For external purposes (NSSE, school
ranking, accreditation, etc.)• To improve student learning in the
course• To identify and correct problems• To better understand how students
approach learning in the course
• To Improve a course• To better align the course with your learning
goals• To improve quality of activities,
assignments, assessments• To find out if you achieved your goals
• To develop as a teacher
![Page 8: Evaluating Courses & Teaching Gregory Light, PhD Denise Drane, PhD Palestinian Polytechnic University Hebron March 17 th, 2014](https://reader036.vdocuments.mx/reader036/viewer/2022062515/56649c745503460f94926eeb/html5/thumbnails/8.jpg)
How do you evaluate Your
courses and your teaching?
![Page 9: Evaluating Courses & Teaching Gregory Light, PhD Denise Drane, PhD Palestinian Polytechnic University Hebron March 17 th, 2014](https://reader036.vdocuments.mx/reader036/viewer/2022062515/56649c745503460f94926eeb/html5/thumbnails/9.jpg)
Types of EvaluationStudentSurveys
andQuestionnaires
Review of teaching
materials/course designObservations
Focus groups
Student AchievementStudent retention
Student work placements
Peer-reviewTeaching Scholarship
Specialized Instruments
(ATI)
![Page 10: Evaluating Courses & Teaching Gregory Light, PhD Denise Drane, PhD Palestinian Polytechnic University Hebron March 17 th, 2014](https://reader036.vdocuments.mx/reader036/viewer/2022062515/56649c745503460f94926eeb/html5/thumbnails/10.jpg)
"Have you ever fallen asleep in class and awoke in another? That's the way I felt all term.“
"This class was a religious experience for me ... I had to take it all on faith.“
"The recitation instructor would make a good parking lot attendant. Tries to tell you where to go, but you can never understand him."
MIT Student Evaluation Comments
What are your
experiences with student
evaluations?
Student Evaluations (ratings)
![Page 11: Evaluating Courses & Teaching Gregory Light, PhD Denise Drane, PhD Palestinian Polytechnic University Hebron March 17 th, 2014](https://reader036.vdocuments.mx/reader036/viewer/2022062515/56649c745503460f94926eeb/html5/thumbnails/11.jpg)
Student Evaluations: Caution
• It is true that student evaluationss can be influenced by – Student and/or instructor gender– Instructor age & experience– Instructor’s personal traits (personality,
language, etc.)– Discipline and course type– Student achievement level– Student approaches to learning
• BUT: When used well, evaluations can be an effective tool for improving teaching & learning
Aigner & Thum, 1986; Bosshardt & Watts, 2001; Entwistle & Tait, 1990; Frick et al., 2007; Mehdizadeh, 1990; Millea & Grimes, 2002, Shevlin, Banyard, Davies, & Griffiths, 2000; Sprague & Massoni, 2005; Wolfer & Johnson, 2003; Worthington, 2002
![Page 12: Evaluating Courses & Teaching Gregory Light, PhD Denise Drane, PhD Palestinian Polytechnic University Hebron March 17 th, 2014](https://reader036.vdocuments.mx/reader036/viewer/2022062515/56649c745503460f94926eeb/html5/thumbnails/12.jpg)
Student Evaluations: Unintended & Intended Consequences
Intended
• Faculty use ratings to make improvements in teaching.
• Faculty are rewarded for high ratings.
• Faculty with very low ratings are encouraged & offered resources to improve.
• Students see ratings as a catalyst for change and use ratings to suggest improvement.
Unintended
• Faculty “teach to the ratings.”
• The institution looks only at numerical ratings as indicator of teaching quality.
• Faculty avoid engaging in the process.
• The rating process becomes a useless exercise.
Ory & Ryan, 2001
![Page 13: Evaluating Courses & Teaching Gregory Light, PhD Denise Drane, PhD Palestinian Polytechnic University Hebron March 17 th, 2014](https://reader036.vdocuments.mx/reader036/viewer/2022062515/56649c745503460f94926eeb/html5/thumbnails/13.jpg)
Asking Good Questions• Some questions linked to learning…
Frick et al., 2007
How much time and effort did students put into
key areas of the
course?
To what degree did students see relevance or
authenticity in the learning tasks?
Do students feel they clearly
understood learning goals?
Do students feel they received
enough coaching?
Do students feel they successfully integrated
material, both new and old?
Is the course aligned?
![Page 14: Evaluating Courses & Teaching Gregory Light, PhD Denise Drane, PhD Palestinian Polytechnic University Hebron March 17 th, 2014](https://reader036.vdocuments.mx/reader036/viewer/2022062515/56649c745503460f94926eeb/html5/thumbnails/14.jpg)
Using Student Ratings to Inform Teaching
1. Read between the lines
2. Consider how student approach to learning can impact comments – and what you can do about it
3. Ask questions to elicit information that can help you improve learning
![Page 15: Evaluating Courses & Teaching Gregory Light, PhD Denise Drane, PhD Palestinian Polytechnic University Hebron March 17 th, 2014](https://reader036.vdocuments.mx/reader036/viewer/2022062515/56649c745503460f94926eeb/html5/thumbnails/15.jpg)
Reading Between the LinesComment Possible Reason What you can do
“This class was really unfair. I never understood what the instructor wanted, and my grades suffered as a result.”
“There was too much busywork. I really didn’t see the point of most of the assignments.”
“I didn’t learn anything.”“This class was really boring.”
“The instructor didn’t really care about the students.”
“There was no focus – the teacher was disorganized.”
![Page 16: Evaluating Courses & Teaching Gregory Light, PhD Denise Drane, PhD Palestinian Polytechnic University Hebron March 17 th, 2014](https://reader036.vdocuments.mx/reader036/viewer/2022062515/56649c745503460f94926eeb/html5/thumbnails/16.jpg)
Reading Between the LinesComment Possible Reason What you can do
“This class was really unfair. I never understood what the instructor wanted, and my grades suffered as a result.”
Expectations unclear to students; goals did not match assessments
Clearly communicate assessment criteria; be sure course is well aligned
“There was too much busywork. I really didn’t see the point of most of the assignments.”
Objectives for assignments not clear to students
Be transparent: Explain your rationale and learning objectives for each activity
“I didn’t learn anything.”“This class was really boring.”
Students not adequately engaged; learning is at a surface level; students not aware of learning objectives
Provide opportunity for students to engage at deeper levels; increase relevance; communicate objectives
“The instructor didn’t really care about the students.”
Inadequate faculty–student interaction
Increase communication & develop rapport: office time, email, conversation, feedback
“There was no focus – the teacher was disorganized.”
Students lacked organizing structure for course concepts
Provide “road map” of course; opportunities for integration of ideas
![Page 17: Evaluating Courses & Teaching Gregory Light, PhD Denise Drane, PhD Palestinian Polytechnic University Hebron March 17 th, 2014](https://reader036.vdocuments.mx/reader036/viewer/2022062515/56649c745503460f94926eeb/html5/thumbnails/17.jpg)
Course Evaluation and Students’ Approaches to
Learning
![Page 18: Evaluating Courses & Teaching Gregory Light, PhD Denise Drane, PhD Palestinian Polytechnic University Hebron March 17 th, 2014](https://reader036.vdocuments.mx/reader036/viewer/2022062515/56649c745503460f94926eeb/html5/thumbnails/18.jpg)
Student 1
•“This course was unfair – the prof did not make it clear what we were supposed to do in the assignments.”
•“We wasted time on activities when the prof should have been doing her job, presenting the information.”
•“I didn’t learn a lot in class. It would have been better to give us the lecture notes so that we’d know exactly what the prof was getting at.”
Student 2
•“At first I wasn’t clear about what to study, but later I really enjoyed the freedom to explore ideas.”
•“Although at times I was a little unsure of my footing, I felt engaged in this class and enjoyed the assignments.”
•“Great class – definitely an opportunity to think independently.”
How would you reconcile these divergent comments?
![Page 19: Evaluating Courses & Teaching Gregory Light, PhD Denise Drane, PhD Palestinian Polytechnic University Hebron March 17 th, 2014](https://reader036.vdocuments.mx/reader036/viewer/2022062515/56649c745503460f94926eeb/html5/thumbnails/19.jpg)
Student Orientations to Learning
Reproduction Meaning
Surface approachExtrinsic motivationFear of failureTeacher has
responsibility for teaching and, by extension, learning
Deep Approach Intrinsic motivationIntention to
understandTake responsibility for
own learning
Biggs, 1999; Entwistle, 1997; Marton, 1983; Prosser & Trigwell, 1999
![Page 20: Evaluating Courses & Teaching Gregory Light, PhD Denise Drane, PhD Palestinian Polytechnic University Hebron March 17 th, 2014](https://reader036.vdocuments.mx/reader036/viewer/2022062515/56649c745503460f94926eeb/html5/thumbnails/20.jpg)
Learning Orientations & Evaluation
REPRODUCTION ORIENTATION
MEANING ORIENTATION
I: Focus on efficiency III: Focus on communication
• Basic lecturing skills• Provision of clear goals &
standards• Systematic organization of
course• Workload and level of
difficulty
• Quality of explanations provided
• Use of real-life illustrations• Use of humor and
enthusiasm • Empathy for students
II: Focus on organization IV: Focus on independent learning
• Interesting and relevant content
• Level at which material is pitched
• Pace at which topics are covered
• Clear structure within lectures
• Assignmts providing choice, resources
• Full explanations in feedback • Assessment related to
course aims• Advice on study skills and
strategies
Adapted from Entwistle & Tait, 1990
![Page 21: Evaluating Courses & Teaching Gregory Light, PhD Denise Drane, PhD Palestinian Polytechnic University Hebron March 17 th, 2014](https://reader036.vdocuments.mx/reader036/viewer/2022062515/56649c745503460f94926eeb/html5/thumbnails/21.jpg)
What influences approach to learning?
• Engagement– Choice over subjects and method of study – Opportunity to pursue subjects in depth– Perceived emphasis on understanding vs. rote learning
• Confidence– Clarity of goals, expectations– Perceived workload– Level of perceived threat/anxiety
• Support– Level of communication between teacher and students– Depth & quality of feedback– Perceived degree of teacher empathy for students
Dart et al., 1999; Gibbs, 1992; Trigwell, 1991
![Page 22: Evaluating Courses & Teaching Gregory Light, PhD Denise Drane, PhD Palestinian Polytechnic University Hebron March 17 th, 2014](https://reader036.vdocuments.mx/reader036/viewer/2022062515/56649c745503460f94926eeb/html5/thumbnails/22.jpg)
Responding to Student Comments:
• Resist temptation to ignore or invalidate negative comments
• Consider the meaning behind the comment, not the way it was phrased
• Categorize• by teaching area (e.g., lecture style,
communication) • by numerical rating – to get a sense of why
students feel the way they do (You can view individual raters’ comments)
• Probe as necessary (follow-up survey, discussions, etc.) Lewis, 2001
![Page 23: Evaluating Courses & Teaching Gregory Light, PhD Denise Drane, PhD Palestinian Polytechnic University Hebron March 17 th, 2014](https://reader036.vdocuments.mx/reader036/viewer/2022062515/56649c745503460f94926eeb/html5/thumbnails/23.jpg)
Image from http://ctl.byu.edu/ctls-mid-course-evaluation-tool
What are your experiences
with formative evaluation?
![Page 24: Evaluating Courses & Teaching Gregory Light, PhD Denise Drane, PhD Palestinian Polytechnic University Hebron March 17 th, 2014](https://reader036.vdocuments.mx/reader036/viewer/2022062515/56649c745503460f94926eeb/html5/thumbnails/24.jpg)
Benefits of Formative Evaluation
• Allows for mid-course correction• Can help students feel more
engaged in the course• Can help improve student–
instructor communication• Helps provide longitudinal
documentation of course change/improvement
• Shown to improve end-of-year ratings*
Cohen, 1980; *Marsh & Roche, 1993; Murray et al., 1996
![Page 25: Evaluating Courses & Teaching Gregory Light, PhD Denise Drane, PhD Palestinian Polytechnic University Hebron March 17 th, 2014](https://reader036.vdocuments.mx/reader036/viewer/2022062515/56649c745503460f94926eeb/html5/thumbnails/25.jpg)
Types of Formative Evaluation
Instructor-created student
questionnaires (individual and
group)
Peer review of teaching materials/course
design
Peer observation
Small-group analysis (SGA)
Classroom assessment techniques
– Minute paper– Learning logs
– Memory matrix– Muddiest point
– Assignment assessments– PRS
![Page 26: Evaluating Courses & Teaching Gregory Light, PhD Denise Drane, PhD Palestinian Polytechnic University Hebron March 17 th, 2014](https://reader036.vdocuments.mx/reader036/viewer/2022062515/56649c745503460f94926eeb/html5/thumbnails/26.jpg)
Key Points• Evaluation as summative and
formative• Students’ approaches to learning
influence their responses to the course
• Often necessary to read between the lines
• Formative evaluation can be integrated into your course & regular classroom activities
• Students do want opportunities to give meaningful feedback that can impact teaching (Spencer, 2002; Chen, 2003)
![Page 27: Evaluating Courses & Teaching Gregory Light, PhD Denise Drane, PhD Palestinian Polytechnic University Hebron March 17 th, 2014](https://reader036.vdocuments.mx/reader036/viewer/2022062515/56649c745503460f94926eeb/html5/thumbnails/27.jpg)
References• Entwistle, N. & Tait, H. (1990) ‘Approaches to learning,
evaluations of teaching and preferences for contrasting academic environments’, Higher Education, 19 (2): 169–94.
• Peter Ewell (2008) Assessment & Accountability in America Today: Background and Context in New Directions for Institutional Research: Assessment Supplement. Wiley InterScience.
• Knight, J. (2011) University of Colorado. NAS/HHMI Summer Institute on Undergraduate Biology Education, Madison, WI.
• Light, G, & Micari, M. (In press) Making Scientists: Six Principles for Effective College Teaching, Harvard University Press.
• Beauchamp, McConaghy, Parsons & Sanford. (1996) Teaching From the Outside In. Duval: 1996, 37.