rethinking student feedback
DESCRIPTION
Student feedback is a hot topic in higher education, with students demanding more of it, quicker. This session discusses a project that attempted to define the concept of feedback from both a student and faculty perspective and then develop workflows and possible extensions to Blackboard to improve the creation, delivery and learning from feedback.TRANSCRIPT
Dr Malcolm Murray
RethinkingStudent Feedback
Dr Malcolm Murray
Durham University, UK
13th
Jul
y 20
11 3
:00
pm V
enet
ian
Con
gres
s C
ente
r -
Titi
an 2
206
Dr Malcolm Murray
Acknowledgement
Much of this presentation draws on conversations with and research by my colleagues Janet Lavery and Judith Jurowska
[email protected]@durham.ac.uk
Dr Malcolm Murray
Slides freely available
http://www.slideshare.net/malcolmmurray
Dr Malcolm Murray
Recorded Delivery
http://www.audioh.com/projects/recorded_delivery.html
Dr Malcolm Murray
Session Plan
Perceptions of Students
Why give feedback?
What we’ve tried
What staff want
What students want
What next?
Dr Malcolm Murray
http:
//w
ww
.flic
kr.c
om/p
hoto
s/rig
htre
adin
g/12
4323
9706
/size
s/m
/in/
phot
ostr
eam
/
Perceptions of Students
Dr Malcolm Murray
Student as Consumer
Ham KhanTimes Higher Educational Supplement14th December 2007
Dr Malcolm Murray
Education by Numbers
http://www.bbc.co.uk/news/education-13874483
Dr Malcolm Murray
Student as Participant
in anAcademic
Communityof Practice
http://knowledgemanagement-review.blogspot.com/2011/02/knowledge-management-and-community-of.html
Dr Malcolm Murray
Implications
Consumers• Focus on the end result• Never look back
Participants• Learning the rules• Want to gain acceptance
Dr Malcolm Murray
Why Feedback?
http:
//w
ww
.flic
kr.c
om/p
hoto
s/fin
5bjh
/444
2706
561/
size
s/l/
in/p
hoto
stre
am/
Dr Malcolm Murray
Five Functions of Feedback
Learners can use feedback to:
confirm
add to
overwrite
tune
restructure
information in their memory
Butler & Winne (1995)
Dr Malcolm Murray
Feedback is a Dialogue
Feedback on assessment whether formative or summative is a personal dialogue between a lecturer and a student about the student’s assessment and aspirations.
It is not necessarily a face-to-face discussion, but it is a dialogue.
Janet Lavery
Dr Malcolm Murray
Feedback is about improvement
“Feedback that does not tell you how to improve is pointless” said one of the students in a focus group to the roar of approval from the other students.
For improvement to occur feedback has to be personal, i.e. in context of the specific assessment, identify strengths and weakness in the assessment, and provide insights into how the student can improve in time for the next assessment.
Janet Lavery
Dr Malcolm Murray
Laurillard’s Conversational Framework
http://www2.smumn.edu/deptpages/~instructTech/lol/laurillard/index.htm
Dr Malcolm Murray
Effect of Modularization
of students responded that feedback was given too late to be helpful, as they got it after the end of the module
59%Hartley & Chesworth (2000)
Dr Malcolm Murray
Speaking Different Languages
“students who do not yet share a similar understanding of academic discourse as the tutor would… …have difficulty in understanding and using the feedback”
Melanie Weaver (2006)
Dr Malcolm Murray
Feedback by Larson
Dr Malcolm MurrayWhat we have tried
Dr Malcolm Murray
Tablet Annotation
Dr Malcolm Murray
Tablets: Good Points
• Novelty makes the process more fun• Don't have to fiddle with envelopes• Admin processing time is significantly
reduced• You can type comments on the work
as well so you don’t have to write the same thing out again and again
• Rubbing out your writing is easy• The output looks quite professional
Dr Grant IngramSchool of
Engineering & Computer Sciences
Dr Malcolm Murray
Tablets: Down Side
• Size of tablet - small movements are required and writing in the margins is tricky
• Zooming needed to get good quality handwriting - makes getting an overview of the work harder
• The quality of your handwriting is much poorer than with an ordinary pen
• A great deal of electronic fiddling: concatenating the marking matrix, naming the PDF files correctly, converting from Word format for students who don't follow instructions and so on.
• The marking time is comparable to doing it with an ink pen
Dr Malcolm Murray
Inline Annotation: Grademark
Dr Malcolm Murray
Rubric Deployed
Dr Malcolm Murray
Dr Steve LyonAnthropology
Rationale
“…when we started doing this [on paper] we had students coming and saying, “I got lots of comments and so and so only got two little lines.”
I wanted somehow to, not impose, but encourage a more consistent amount of feedback for everyone and ensure similar things were being flagged up.”
Dr Malcolm Murray
Staff Feedback
Fatigue is less of an issue… The twentieth bad essay no longer invokes lots of exclamation marks, ‘What is this!’
The ‘marking rubric’… helps automate the process of allocating marks against set criteria. This was particularly helpful to the teaching assistants.
Lyon, Steve. “Making the grade: Helping postgraduate teaching assistants with their marking and feedback..” QED (Durham University) 2008.
Dr Malcolm Murray
Student Feedback
• The student gets at least six comments even if the postgraduate marker [is] less confident…
• Students really like it especially as I give them the rubric before the assignment so they know what they are working to.
• Students like being able to get the feedback online.
• They get more feedback … the essays will be riddled with these little comments that are specific to a paragraph
Dr Malcolm Murray
Audio Feedback
Prof Ian Greener
School of Applied Social
Sciences
Quicker than typing
More effective
Students pay attention
Tone of voice / inflection
Personal - Allows you to talk direct to student work
Upload process is complicated
Dr Malcolm Murray
Student Voice
http://www.flickr.com/photos/jiscimages/436585751/sizes/m/in/photostream/
“it’s not face to face but it’s certainly one to one”
I think this is a much better system than paper feedback. I know I got a lot more from it than if you had just had to tick boxes and given me a comment in that little box on the piece of paper.
Dr Malcolm Murray
Katherine Griffiths
Alia Moser
German Department
School of Modern
Languages
1:1 Feedback using Blogs
• Language students given weekly homework tasks using a blog
• Staff edit the posts, marking mistakes in red and any corrections in green.
• Students are invited to correct their mistakes in blue (crossing out the original mistakes, but not deleting them) and to look at the rest of the corrections
• Thanks to the colour code, staff can then quickly check whether the students have accurately corrected their mistakes
Dr Malcolm Murray
Sample Feedback: Blogs
Dr Malcolm Murray
Staff Rationale
“the previous year I was disappointed because students were doing their translations uncritically, without thinking or looking at the feedback.”
Alia Moser
“With the blog you have the work and feedback online and we can look back and say, “Look, we have mentioned adjective endings before, so why is it still going wrong?” Students can look back at areas where they did well or were weak. This helps build confidence. It was important to make them feel we had a record they could be proud of.”
Katherine Griffiths
Dr Malcolm Murray
Peer Feedback: Blogs
It ensures corrections are completed andare easily re-marked – cuts out time, andif this had been hand written +marked, I would probably put it straight in myfile without bothering with corrections!
Dr Malcolm Murray
Student Feedback: Blogs
I prefer posting my written work on the blog to handing in handwritten work because of its positive effect on my learning.
The opportunity to be able to go back for amendments and revision is important for me.
I can learn from reading my peers’ work and the tutor’s feedback on their work.
I am happy for my peers to comment on my work.
80%of students
agreed
Dr Malcolm Murray
Understanding how Students work
Dr Malcolm MurraySummary of findings
Dr Malcolm Murray
Feedback – staff
Find simpler means to development and deliver feedback online
The current mechanisms for developing and delivering digital feedback (text, audio, etc…) are according to many staff: “inefficient”, “require too many steps”, and generally too difficult.
Staff want access to sophisticated feedback generation tools that are easy to use and provide simple delivery mechanisms.
Dr Malcolm Murray
Feedback - students
Praise for feedback limited to that ‘like school’ or ‘what we are used to’.
More unhappiness then happiness with the actual content of feedback.
Enthusiastic about receiving feedback developed using new technologies such as audio recordings or digital mark-up systems.
Students very happy with their current feedback were noticeably less enthusiastic about the possibility of new technologies – prefer handwritten comments on a copy of the essays and face-to-face discussions.
Dr Malcolm Murray
Next Steps
http://www.flickr.com/photos/techsavi/3946238357/sizes/o/in/photostream/
Dr Malcolm Murray
Potential Feedback Tool
Dr Malcolm Murray
Complications – Naming Policy
Dr Malcolm Murray
Choice: Feedback Format
Dr Malcolm Murray
Simplifying Download
Dr Malcolm Murray
Simplifying Mark Entry
Dr Malcolm Murray
Simplifying Upload
Dr Malcolm Murray
Mobile Marking
Dr Malcolm Murray
My Grades
2/10 Blackboard-Must try harder…
Dr Malcolm Murray
Feedback Dashboard
Dr Malcolm Murray
Simple, but Consistent
Look at the processes for submission & feedback
Push for standardization• in the lecturer’s experience• in the student’s experience
Dr Malcolm MurrayConclusion
Dr Malcolm Murray
Rubric cubism (sorry)
feedback [is] an essentially problematic form of communication involving particular social relationships…
…external conditions interplay, mediate (and are mediated by) patterns of power, authority, emotion and identity
Higgins, Hartley & Skelton (2001)
http://www.forevergeek.com/2009/09/rubiks_cube_fancy_dress_costume/
Dr Malcolm Murray
Findings so far…
Creative ways exist for staff & peers to provide feedback
Most have ugly workflows, discouraging wider adoption
No one-size fits all – (5 purposes of feedback)
Need to make things better without breaking those that already work
Electronic delivery can make feedback more visible, thus more likely to result in learning
Must provide/promote timely feedback
We need to learn how our students could use feedback to feed forward…
Dr Malcolm Murray
Please provide feedback for this session by [email protected].
The title of this session is:
Rethinking Student Feedback
Dr Malcolm Murray
Get in touch:
@learntechdurham
@malcolmmurray
http://www.dur.ac.uk/lt.team/blog/
Questions?