exploring and utilising students' perspectives on feedback: a mixed method, longitudinal...
TRANSCRIPT
Exploring and utilising students' perspectives on feedback: a mixed
method, longitudinal approach
Kara Peterson, Simon Croker, Dr. Peter Hills, and Dr. Rachel
Manning
WelcomeWebinar Programme
Welcome and Introduction Peter Hills & Rachel Manning
‘In Our Ideal World…’ Kara Peterson
Q & A and Discussion
Differences in perceptions of feedback based on academic achievement and student engagement
Kara Peterson & Simon Croker
Q & A and Discussion
The Next Step Simon Croker
Q & A and Discussion
Final Q & A and Closing Comments Peter Hills & Rachel Manning
Project BackgroundThis research aimed to understand students’ perspectives of
feedback, as well as develop a feedback system that enhanced student learning and encouraged active
engagement with feedback.
Who We Are:
Dr Peter Hills Dr Rachel ManningSenior Lecturer Principal [email protected] [email protected]
Kara Peterson Simon CrokerStudent Research Assistant Student Research [email protected] [email protected]
Feedback on my work has helped me to clarify things I did not understand.
I have received detailed comments on my work.
Feedback on my work has been prompt.
Overall, I am satisfied with the quality of the course.
0% 25% 50% 75% 100%
71%
75%
71%
92%
2013 NSS results for Anglia Ruskin University psychology students
Departmental Context
• Electronic submission & feedback– Feedback Assistant– Grademark
• Coded Annotation– Speed of marking with detailed comments– Standardisation of feedback
• Rapid Department Growth
Departmental Context
Departmental Context
Project TimelineJanuary 2013 Beginning of research
February 2013 Conduct pilot focus
groups
March – May 2013 Phase 1 focus
groups and online survey
June – October 2013 Feeding forward on
perceptions of feedback
November 2013 – February 2014 Phase 2 focus groups and online
survey
March – April 2014 Evaluation of Project
Methodology• Peer-led semi-
structured focus groups
• Anglia Ruskin University psychology undergraduates
• 40 phase 1 participants
• 19 phase 2 participants
• Participants from all three undergraduate years and a range of academic profiles
Methodology• Online survey with open
and closed response format
• Anglia Ruskin University psychology students
• 98 phase 1 participants
• 78 phase 2 participants
• Participants from all three undergraduate years and a range of academic profiles
‘In Our Ideal World…’
Kara Peterson
Survey Findings
Figure 1. Satisfaction levels with different aspects of feedback
Overall Time to Get Feedback
Amount of Feedback
Codes Talking with Lecturers
Helping for Future
Usefulness of Feedback
Positive Feedback
Negative Feedback
0
1
2
3
4
5
6
7
Aspects of Feedback
Satis
facti
on L
evel
Survey Findings
Figure 1. Satisfaction levels with different aspects of feedback
Overall Time to Get Feedback
Amount of Feedback
Codes Talking with Lecturers
Helping for Future
Usefulness of Feedback
Positive Feedback
Negative Feedback
0
1
2
3
4
5
6
7
Aspects of Feedback
Satis
facti
on L
evel
Survey Findings
Figure 1. Satisfaction levels with different aspects of feedback
Overall Time to Get Feedback
Amount of Feedback
Codes Talking with Lecturers
Helping for Future
Usefulness of Feedback
Positive Feedback
Negative Feedback
0
1
2
3
4
5
6
7
Aspects of Feedback
Satis
facti
on L
evel
Survey Findings
Figure 1. Satisfaction levels with different aspects of feedback
Overall Time to Get Feedback
Amount of Feedback
Codes Talking with Lecturers
Helping for Future
Usefulness of Feedback
Positive Feedback
Negative Feedback
0
1
2
3
4
5
6
7
Aspects of Feedback
Satis
facti
on L
evel
Survey Findings
Figure 1. Satisfaction levels with different aspects of feedback
Overall Time to Get Feedback
Amount of Feedback
Codes Talking with Lecturers
Helping for Future
Usefulness of Feedback
Positive Feedback
Negative Feedback
0
1
2
3
4
5
6
7
Aspects of Feedback
Satis
facti
on L
evel
Survey Findings
Figure 1. Satisfaction levels with different aspects of feedback
Overall Time to Get Feedback
Amount of Feedback
Codes Talking with Lecturers
Helping for Future
Usefulness of Feedback
Positive Feedback
Negative Feedback
0
1
2
3
4
5
6
7
Aspects of Feedback
Satis
facti
on L
evel
Survey Findings
Figure 1. Satisfaction levels with different aspects of feedback
Overall Time to Get Feedback
Amount of Feedback
Codes Talking with Lecturers
Helping for Future
Usefulness of Feedback
Positive Feedback
Negative Feedback
0
1
2
3
4
5
6
7
Aspects of Feedback
Satis
facti
on L
evel
Survey Findings
Figure 1. Satisfaction levels with different aspects of feedback
Overall Time to Get Feedback
Amount of Feedback
Codes Talking with Lecturers
Helping for Future
Usefulness of Feedback
Positive Feedback
Negative Feedback
0
1
2
3
4
5
6
7
Aspects of Feedback
Satis
facti
on L
evel
Survey Findings
Figure 1. Satisfaction levels with different aspects of feedback
Overall Time to Get Feedback
Amount of Feedback
Codes Talking with Lecturers
Helping for Future
Usefulness of Feedback
Positive Feedback
Negative Feedback
0
1
2
3
4
5
6
7
Aspects of Feedback
Satis
facti
on L
evel
Key Themes
Convenience
Continuing Dialogue
Consistency
Convenience
‘I just read the e-mail. I haven’t collected any of my feedback. I just use the emails and go from there. I think it should be easier to actually collect your essay.’
‘Yeah, I mean, it’s a bit difficult when you don’t know when to collect your work.’
‘I never bother going to pick up my [feedback].... when you go there you’re waiting in a queue to pick it up in a room’
Convenience
Figure 2. Percentage of students that collect feedback and how they use feedback.
Convenience
‘You have to keep going back to check the different codes which takes ages.’
‘It’s really just long going through all the codes just to refer back to what they mean. I just leave it.
’Yeah it’s like you have to read it and then you have to go back to your other piece of paper that says what the code means, it’s just long and you just feel like you can’t be bothered to do it.
Continuing Dialogue
‘Yeah I think [feedback] should be a lot more one on one…’
‘I think that [feedback] is helpful in a way but at the same time it just
tells you what you’ve done wrong not where you can improve and
that is a really big aspect that you have to kind of do better in the future rather then making the same mistakes
over and over again’
Continuing DialogueIt’s more of a one way thing at the moment, whereas you just get the
feedback and then your expected to
do it yourself.’
‘If someone’s marked your work that they marked a while back and they’ve noticed that there is an improvement in something or, you know, they’ve seen that your
marks are different … it would be nice to know that you have
improved it this time rather then just be over looked
‘I don’t know whether it’s my slackness or whatever, but I don’t
know who I’m, not allowed to speak to, but I would never realised I could go to my personal tutor because they haven’t marked it.’
Consistency
‘[Feedback] is quite mixed, like you either get three pages full of
things that you could’ve done better or you get a little blank box and something and sort of a few little hand annotations on the thing..’
‘[Feedback] should be standardised but obviously because
of the topic it might vary what comments we’ll get written but I think if there was like a standard procedure
for each topic then it might be a bit clearer…’
‘I think personally I’d want like bullet points at the start and then I’d want
like details throughout’
Communicatio
n
Consistency Convenience
Continuing Dialogue
Q & A and Discussion
Feel free to ask any questions or share any of your our experiences of
students’ perceptions of feedback
Differences in perceptions of feedback based on
academic achievement and student engagement
Kara Peterson and Simon Croker
Compared to students with higher grades, students with lower grades:
Were less satisfied with feedback, r(91) = .19, p = .066.
”It’s not actually been very good because I, I still kept getting the same thing and I was just like ‘are they really reading over it
or is it, is that just like a generic thing?'"
Were less satisfied with positive comments, r(91) = .18, p = .081.
"...when you get a really low mark it would have been nice to maybe have a little bit of an idea of maybe something you have done right, because I just felt like everything was just rubbish..."
Felt feedback was less helpful for future work, r(90) = .21, p = .046.
"Not at all, cause I have no idea like what I’ve done wrong really [INT: um hm] cause obviously if you do something wrong you
don’t know [INT: Yeah] and you can’t improve it ..."
Compared to students with higher grades, students with lower grades:
Were less satisfied with feedback, r(91) = .19, p = .066.
”It’s not actually been very good because I, I still kept getting the same thing and I was just like ‘are they really reading over it
or is it, is that just like a generic thing?'"
Were less satisfied with positive comments, r(91) = .18, p = .081.
"...when you get a really low mark it would have been nice to maybe have a little bit of an idea of maybe something you have done right, because I just felt like everything was just rubbish..."
Felt feedback was less helpful for future work, r(90) = .21, p = .046.
"Not at all, cause I have no idea like what I’ve done wrong really [INT: um hm] cause obviously if you do something wrong you
don’t know [INT: Yeah] and you can’t improve it ..."
Compared to students with higher grades, students with lower grades:
Were less satisfied with feedback, r(91) = .19, p = .066.
”It’s not actually been very good because I, I still kept getting the same thing and I was just like ‘are they really reading over it
or is it, is that just like a generic thing?'"
Were less satisfied with positive comments, r(91) = .18, p = .081.
Felt feedback was less helpful for future work, r(90) = .21, p = .046.
"Not at all, cause I have no idea like what I’ve done wrong really [INT: um hm] cause obviously if you do something wrong you
don’t know [INT: Yeah] and you can’t improve it ..."
Compared to students with higher grades, students with lower grades:
Were less satisfied with feedback, r(91) = .19, p = .066.
”It’s not actually been very good because I, I still kept getting the same thing and I was just like ‘are they really reading over it
or is it, is that just like a generic thing?'"
Were less satisfied with positive comments, r(91) = .18, p = .081.
"...when you get a really low mark it would have been nice to maybe have a little bit of an idea of maybe something you have done right, because I just felt like everything was just rubbish..."
Felt feedback was less helpful for future work, r(90) = .21, p = .046.
"Not at all, cause I have no idea like what I’ve done wrong really [INT: um hm] cause obviously if you do something wrong you
don’t know [INT: Yeah] and you can’t improve it ..."
Compared to students with higher grades, students with lower grades:
Were less satisfied with feedback, r(91) = .19, p = .066.
”It’s not actually been very good because I, I still kept getting the same thing and I was just like ‘are they really reading over it
or is it, is that just like a generic thing?'"
Were less satisfied with positive comments, r(91) = .18, p = .081.
"...when you get a really low mark it would have been nice to maybe have a little bit of an idea of maybe something you have done right, because I just felt like everything was just rubbish..."
Felt feedback was less helpful for future work, r(90) = .21, p = .046.
Compared to students with higher grades, students with lower grades:
Were less satisfied with feedback, r(91) = .19, p = .066.
”It’s not actually been very good because I, I still kept getting the same thing and I was just like ‘are they really reading over it
or is it, is that just like a generic thing?'"
Were less satisfied with positive comments, r(91) = .18, p = .081.
"...when you get a really low mark it would have been nice to maybe have a little bit of an idea of maybe something you have done right, because I just felt like everything was just rubbish..."
Felt feedback was less helpful for future work, r(90) = .21, p = .046.
"Not at all, cause I have no idea like what I’ve done wrong really [INT: um hm] cause obviously if you do something wrong you
don’t know [INT: Yeah] and you can’t improve it ..."
Compared to students with higher grades, students with lower grades:
Were less likely to collect feedback,
r(88) = .25, p = .019.
Were less likely to speak to lecturers, r(87) = .22, p = .035.
"I find it really difficult. I haven’t seen any lecturers about any of my work because I’m so limited on time ...'
Compared to students with higher grades, students with lower grades:
Were less likely to collect feedback,
r(88) = .25, p = .019. "There is not that much like more detail in my hard piece anyway
so I just don’t really bother to collect my feedback."
Were less likely to speak to lecturers, r(87) = .22, p = .035.
"I find it really difficult. I haven’t seen any lecturers about any of my work because I’m so limited on time ...'
Compared to students with higher grades, students with lower grades:
Were less likely to collect feedback,
r(88) = .25, p = .019. "There is not that much like more detail in my hard piece anyway
so I just don’t really bother to collect my feedback."
Were less likely to speak to lecturers, r(87) = .22, p = .035.
Compared to students with higher grades, students with lower grades:
Were less likely to collect feedback,
r(88) = .25, p = .019. "There is not that much like more detail in my hard piece anyway
so I just don’t really bother to collect my feedback."
Were less likely to speak to lecturers, r(87) = .22, p = .035.
"I find it really difficult. I haven’t seen any lecturers about any of my work because I’m so limited on time ...'
Compared to students with higher grades,
students with lower grades:
Were less satisfied with feedback, r(91) = .19, p = .066.
Were less satisfied with positive comments,
r(91) = .18, p = .081.
Felt feedback was less helpful for future work,
r(90) = .21, p = .046.
Were less likely to collect feedback, r(88) = .25, p = .019.
Were less likely to speak to lecturers, r(87) = .22, p = .035.
Felt that feedback mattered less to them, r(89) = .20, p = .058.
Compared to students with lower grades,
students with higher grades:
Felt that handwritten comments over typed comments showed that lecturers
were more interested in their work, β = .01, p = .011
Preferred annotations over codes, β = .01, p = .001
Preferred to speak to lecturers about feedback rather than using codes,
β = .02, p < .001
Preferred to speak to lecturers about feedback rather than
reading paragraphs, β = .01, p = .007
Wanted all the comments on
their work highlighted
compared to students with lower grades
whom wanted only a selected
number of comments
highlighted, β = .01, p = .004
Other Differences
• ‘It’s been good to a certain level, however, some of the feedback they give, erm the lecturers, if they sit and criticise we should have done they don’t say how we could have improved it. [INT: Yeah] They just mention what we did wrong but they don’t say what they would have preferred for us to do, so.’
• ‘I just think they should provide model answers for like if they do criticise then they should say, or they should write what they think would’ve been a better answer to what, where I went wrong or something.’
• I’m satisfied with any feedback because if you don’t get any feedback you don’t know you’re improving, if you’re not being pointed erm where you’ve gone wrong, how can you correct it? I don’t always know, I mean some of it’s really clear and I know exactly where to go and what to do to improve what they’ve said.’
Other Differences
• 1/3 of students do not want all their mistakes highlighted.
• Students wanted M=5 comments highlighted (range 2-10).
• Students with higher grades tended to want all mistakes highlighted, β = .01, p = .004
Q & A and Discussion
Feel free to ask any questions or share any of your our experiences
about student engagement and different practices that encourage
student engagement
The Next Step…
Communicatio
n
Consistency
Convenience
Continuing
Dialogue
Feedback: What you told us
Issues Raised by Students
• Worst rated area of feedback was the use of codes• Codes considered generic and a hassle to look up • Second worst rated area of feedback was not
providing students with enough positive feedback• Useful to know what areas have been done well
along with areas that need improvement
Issues Raised by Students
• Feedback too generalised• Specific and personalised feedback is more useful• Lack of specificity in how and what to improve• Doesn’t recognise progress or improvements that
have been made • Confusion about who to talk to with questions
about feedback
Feedback: What you told us
What’s new
Coursework Feedback and Reflection Sheet
• New reflection sheet for you to fill out and paste to the end of your coursework (on VLE – ‘Coursework Guidance’)
• Gives you a role in deciding what type of feedback you receive• Enables you to highlight how you have used previous feedback
on your work, what areas you have found difficult, and grade targets
• Allows markers to give personalised feedback based on your coursework goals and how you have used previous feedback.
What’s new
Coding System and Positive Feedback
• No more A1, B2, C3….• Simpler coding system
– EX: CIT – Provide more citations, SENT – Use proper sentences
• Positive codes added– EX: Introduction - Good introduction, Evaluation - You evaluate the evidence for a theory/position
• Information about resources to aid improvement will now be provided
– EX: If you get feedback about improving references, information about where to find the APA referencing guide will be provided on your electronic feedback sheet
What you can do
Engage in the Feedback Process
• Online survey revealed a significant positive correlation between grades and engagement with feedback
• Get the most out of feedback by utilising the resources available to you:– Read through the comments on your coursework together with your feedback– Speak to lecturers – If you have any questions or concerns about your feedback go
and see the lecturer that marked your work– Personal tutor – For ongoing feedback book an appointment with your personal
tutor and bring in your past coursework feedback
Changes Implemented
If you would like to know more about the feedback process or discuss any feedback you have received previously, please contact your personal tutor whose details can be found on your E-Vision page.
Interactive Coursework Feedback and Reflection Sheet
Changes ImplementedThese are the things you did well:
• • •
You could do the following things to improve your work in future:
• • •
Structural Changes
GradeMark – “I like that we don’t have to look through like this and find out what QM or whatever these codes mean. You can just get over it and it just says. I prefer that. It’s a lot better.”
Reflection Sheet – “I also like that we can write when we want specific feedback on, which is helpful.”
Positive Comments – “I found my first year was more negative. I haven’t felt that this time and that was important for me.”
Continuing Dialogue – “I’ll have to be honest and say I’ve only had one feedback where I felt confident and knew what I needed to do to improve.”
Communication – “I think some people don’t know where you can go so I think they need to reiterate that you can go see them other times. They just need to remind people.”
Consistency – “I found that this year there’s quite a lot of inconsistencies. For one of my modules last semester there was a lot less feedback given to me on that when the other modules.”
Recommendations
Faculty
Department
Advisory Group Student Reps
Student Body
Recommendations: Student-focused
Anglia Ruskin Psychology Student Feedback Handbook
• What is feedback?• How can I use my feedback?• Where can I can more information on feedback?• Will contain information on advisory group, cover sheets and
reflection sheets.
Recommendations: Staff-focused
Anglia Ruskin Psychology Staff Feedback Handbook
• Staff handbook was produced and given to all staff.
• Ensuring that it is used and kept up to date with any new changes.
Communicatio
n
Consistency Convenience
Continuing Dialogue
The Next Step…
• Further promote the issues surrounding feedback to both staff and students.
• Keep student input at the core of feedback research.
Q & A and Discussion
Feel free to ask any questions or share any different ways to improve
feedback
Final Q & A
Feel free to ask any unanswered questions
Communicatio
n
Consistency
Convenience
Continuing
Dialogue