rethinking student feedback

Dr Malcolm Murray

RethinkingStudent Feedback

Dr Malcolm Murray

Durham University, UK

13th

Jul

y 20

11 3

:00

pm V

enet

ian

Con

gres

s C

ente

r -

Titi

an 2

206

Dr Malcolm Murray

Acknowledgement

Much of this presentation draws on conversations with and research by my colleagues Janet Lavery and Judith Jurowska

janet.lavery@durham.ac.ukj.e.jurowska@durham.ac.uk

Dr Malcolm Murray

Slides freely available

http://www.slideshare.net/malcolmmurray

Dr Malcolm Murray

Recorded Delivery

http://www.audioh.com/projects/recorded_delivery.html

Dr Malcolm Murray

Session Plan

Perceptions of Students

Why give feedback?

What we’ve tried

What staff want

What students want

What next?

Dr Malcolm Murray

http:

//w

ww

.flic

kr.c

om/p

hoto

s/rig

htre

adin

g/12

4323

9706

/size

s/m

/in/

phot

ostr

eam

/

Perceptions of Students

Dr Malcolm Murray

Student as Consumer

Ham KhanTimes Higher Educational Supplement14th December 2007

Dr Malcolm Murray

Education by Numbers

http://www.bbc.co.uk/news/education-13874483

Dr Malcolm Murray

Student as Participant

in anAcademic

Communityof Practice

http://knowledgemanagement-review.blogspot.com/2011/02/knowledge-management-and-community-of.html

Dr Malcolm Murray

Implications

Consumers• Focus on the end result• Never look back

Participants• Learning the rules• Want to gain acceptance

Dr Malcolm Murray

Why Feedback?

http:

//w

ww

.flic

kr.c

om/p

hoto

s/fin

5bjh

/444

2706

561/

size

s/l/

in/p

hoto

stre

am/

Dr Malcolm Murray

Five Functions of Feedback

Learners can use feedback to:

confirm

add to

overwrite

tune

restructure

information in their memory

Butler & Winne (1995)

Dr Malcolm Murray

Feedback is a Dialogue

Feedback on assessment whether formative or summative is a personal dialogue between a lecturer and a student about the student’s assessment and aspirations.

It is not necessarily a face-to-face discussion, but it is a dialogue.

Janet Lavery

Dr Malcolm Murray

Feedback is about improvement

“Feedback that does not tell you how to improve is pointless” said one of the students in a focus group to the roar of approval from the other students.

For improvement to occur feedback has to be personal, i.e. in context of the specific assessment, identify strengths and weakness in the assessment, and provide insights into how the student can improve in time for the next assessment.

Janet Lavery

Dr Malcolm Murray

Laurillard’s Conversational Framework

http://www2.smumn.edu/deptpages/~instructTech/lol/laurillard/index.htm

Dr Malcolm Murray

Effect of Modularization

of students responded that feedback was given too late to be helpful, as they got it after the end of the module

59%Hartley & Chesworth (2000)

Dr Malcolm Murray

Speaking Different Languages

“students who do not yet share a similar understanding of academic discourse as the tutor would… …have difficulty in understanding and using the feedback”

Melanie Weaver (2006)

Dr Malcolm Murray

Feedback by Larson

Dr Malcolm MurrayWhat we have tried

Dr Malcolm Murray

Tablet Annotation

Dr Malcolm Murray

Tablets: Good Points

• Novelty makes the process more fun• Don't have to fiddle with envelopes• Admin processing time is significantly

reduced• You can type comments on the work

as well so you don’t have to write the same thing out again and again

• Rubbing out your writing is easy• The output looks quite professional

Dr Grant IngramSchool of

Engineering & Computer Sciences

Dr Malcolm Murray

Tablets: Down Side

• Size of tablet - small movements are required and writing in the margins is tricky

• Zooming needed to get good quality handwriting - makes getting an overview of the work harder

• The quality of your handwriting is much poorer than with an ordinary pen

• A great deal of electronic fiddling: concatenating the marking matrix, naming the PDF files correctly, converting from Word format for students who don't follow instructions and so on.

• The marking time is comparable to doing it with an ink pen

Dr Malcolm Murray

Inline Annotation: Grademark

Dr Malcolm Murray

Rubric Deployed

Dr Malcolm Murray

Dr Steve LyonAnthropology

Rationale

“…when we started doing this [on paper] we had students coming and saying, “I got lots of comments and so and so only got two little lines.”

I wanted somehow to, not impose, but encourage a more consistent amount of feedback for everyone and ensure similar things were being flagged up.”

Dr Malcolm Murray

Staff Feedback

Fatigue is less of an issue… The twentieth bad essay no longer invokes lots of exclamation marks, ‘What is this!’

The ‘marking rubric’… helps automate the process of allocating marks against set criteria. This was particularly helpful to the teaching assistants.

Lyon, Steve. “Making the grade: Helping postgraduate teaching assistants with their marking and feedback..” QED (Durham University) 2008.

Dr Malcolm Murray

Student Feedback

• The student gets at least six comments even if the postgraduate marker [is] less confident…

• Students really like it especially as I give them the rubric before the assignment so they know what they are working to.

• Students like being able to get the feedback online.

• They get more feedback … the essays will be riddled with these little comments that are specific to a paragraph

Dr Malcolm Murray

Audio Feedback

Prof Ian Greener

School of Applied Social

Sciences

Quicker than typing

More effective

Students pay attention

Tone of voice / inflection

Personal - Allows you to talk direct to student work

Upload process is complicated

Dr Malcolm Murray

Student Voice

http://www.flickr.com/photos/jiscimages/436585751/sizes/m/in/photostream/

“it’s not face to face but it’s certainly one to one”

I think this is a much better system than paper feedback. I know I got a lot more from it than if you had just had to tick boxes and given me a comment in that little box on the piece of paper.

Dr Malcolm Murray

Katherine Griffiths

Alia Moser

German Department

School of Modern

Languages

1:1 Feedback using Blogs

• Language students given weekly homework tasks using a blog

• Staff edit the posts, marking mistakes in red and any corrections in green.

• Students are invited to correct their mistakes in blue (crossing out the original mistakes, but not deleting them) and to look at the rest of the corrections

• Thanks to the colour code, staff can then quickly check whether the students have accurately corrected their mistakes

Dr Malcolm Murray

Sample Feedback: Blogs

Dr Malcolm Murray

Staff Rationale

“the previous year I was disappointed because students were doing their translations uncritically, without thinking or looking at the feedback.”

Alia Moser

“With the blog you have the work and feedback online and we can look back and say, “Look, we have mentioned adjective endings before, so why is it still going wrong?” Students can look back at areas where they did well or were weak. This helps build confidence. It was important to make them feel we had a record they could be proud of.”

Katherine Griffiths

Dr Malcolm Murray

Peer Feedback: Blogs

It ensures corrections are completed andare easily re-marked – cuts out time, andif this had been hand written +marked, I would probably put it straight in myfile without bothering with corrections!

Dr Malcolm Murray

Student Feedback: Blogs

I prefer posting my written work on the blog to handing in handwritten work because of its positive effect on my learning.

The opportunity to be able to go back for amendments and revision is important for me.

I can learn from reading my peers’ work and the tutor’s feedback on their work.

I am happy for my peers to comment on my work.

80%of students

agreed

Dr Malcolm Murray

Understanding how Students work

Dr Malcolm MurraySummary of findings

Dr Malcolm Murray

Feedback – staff

Find simpler means to development and deliver feedback online

The current mechanisms for developing and delivering digital feedback (text, audio, etc…) are according to many staff: “inefficient”, “require too many steps”, and generally too difficult.

Staff want access to sophisticated feedback generation tools that are easy to use and provide simple delivery mechanisms.

Dr Malcolm Murray

Feedback - students

Praise for feedback limited to that ‘like school’ or ‘what we are used to’.

More unhappiness then happiness with the actual content of feedback.

Enthusiastic about receiving feedback developed using new technologies such as audio recordings or digital mark-up systems.

Students very happy with their current feedback were noticeably less enthusiastic about the possibility of new technologies – prefer handwritten comments on a copy of the essays and face-to-face discussions.

Dr Malcolm Murray

Next Steps

http://www.flickr.com/photos/techsavi/3946238357/sizes/o/in/photostream/

Dr Malcolm Murray

Potential Feedback Tool

Dr Malcolm Murray

Complications – Naming Policy

Dr Malcolm Murray

Choice: Feedback Format

Dr Malcolm Murray

Simplifying Download

Dr Malcolm Murray

Simplifying Mark Entry

Dr Malcolm Murray

Simplifying Upload

Dr Malcolm Murray

Mobile Marking

Dr Malcolm Murray

My Grades

2/10 Blackboard-Must try harder…

Dr Malcolm Murray

Feedback Dashboard

Dr Malcolm Murray

Simple, but Consistent

Look at the processes for submission & feedback

Push for standardization• in the lecturer’s experience• in the student’s experience

Dr Malcolm MurrayConclusion

Dr Malcolm Murray

Rubric cubism (sorry)

feedback [is] an essentially problematic form of communication involving particular social relationships…

…external conditions interplay, mediate (and are mediated by) patterns of power, authority, emotion and identity

Higgins, Hartley & Skelton (2001)

http://www.forevergeek.com/2009/09/rubiks_cube_fancy_dress_costume/

Dr Malcolm Murray

Findings so far…

Creative ways exist for staff & peers to provide feedback

Most have ugly workflows, discouraging wider adoption

No one-size fits all – (5 purposes of feedback)

Need to make things better without breaking those that already work

Electronic delivery can make feedback more visible, thus more likely to result in learning

Must provide/promote timely feedback

We need to learn how our students could use feedback to feed forward…

Dr Malcolm Murray

Please provide feedback for this session by emailingBbWorldFeedback@blackboard.com.

The title of this session is:

Rethinking Student Feedback

Dr Malcolm Murray

Get in touch:

malcolm.murray@durham.ac.uk

@learntechdurham

@malcolmmurray

http://www.dur.ac.uk/lt.team/blog/

Questions?