Assessment Standards Knowledge exchange

Engaging students with assessment feedback

Prof. Margaret Price, Director ASKe Centre for Excellence

FDTL Engaging Students with assessment feedbackhttps://mw.brookes.ac.uk/display/eswaf/Home

ASKe Assessment Standards Knowledge exchangeCentre for Excellence in Teaching and Learning

http://www.brookes.ac.uk/aske/!aske@brookes.ac.uk

Purpose of Workshop

• Problems and responses

• Engagement with feedback

• Where to start

• Resources and effectiveness

We have a problem!

• Surveys and audits

• Research literature

Feedback problems

• Unhelpful feedback (Maclellan, 2001)• Too vague (Higgins, 2002)• Subject to interpretation (Ridsdale, 2003)• Not understood (e.g. Lea and Street, 1998)• Don’t read it (Hounsell, 1987. Gibbs & Simpson

2002)• Damage self-efficacy (Wotjas, 1998)• Has no effect (Fritz et al, 2000)• Seen to be too subjective (Holmes & Smith,

2003)

Some responses to the feedback ‘crisis’:

• Provide more of the same • Simplistic rules about timing• Standardisation• Label feedback

• Setting expectations• Introducing new methods

• a complex problem so no simple solution

Exploring feedback (activity)

• What is its purpose?

• What counts as feedback?

• What can it achieve?

• How do you know it is working?

Student engagement with feedback

Assessment feedback

CollectionImmediate attention

Cognitive response

Immediate or latent action

Specific, immediate action, such as applying feedback insights to next

assignment

Developmental changes which generate the latent potential for future action

Self-efficacy

ConfidenceResources; skills

Opportunities for action

... or, after reflection, rejects, or

‘mis’understands, etc.

… or does not collect

... or cannot understand, so

does not proceed

.. or cannot read, so ignores

OU

TC

OM

E

Student outcome influences later

engagement with feedback

Price et al (submitted)

Activity

In 3’s, discuss:

• How do you currently prepare students to understand and engage with feedback?

Where to start

Preparation and setting expectations early in the programme

Identifying ‘feedback moments’ and application opportunities within the programme

Emphasize the relational dimension of feedback

Building in space for dialogue

What can we do? (1)

• Aligning expectations (of staff & students, & between teams of markers)

• Identify what is feasible in a given assessment context - written feedback can often do little more than ‘diagnose’ development issues and then direct students to other resources for help and support

• Identifying all feedback available

• Ensure it is timely - ‘quick and dirty’ generic feedback, feedback on a draft, MCQs & quizzes, etc. (using technology may help)

• Model and encourage the application of feedback

What can we do? (2)

• Require and provide feedback on self-assessment

• Improve the linkage of assessment strategies across programmes and between modules/units

• Consider the role of marks - they obscure feedback

• Reduce over-emphasis on written feedback - oral can be more effective (McCune, 2004). Face to face feedback with 140 students (FDTL Case study: https://mw.brookes.ac.uk/display/eswaf/Home.

• Review resource allocations

What can we do (3)

• Support the relational dimension of feedback Students say that relationships in which staff are supportive and approachable help them to engage Avoid anonymous marking

Ensure associate (and permanent) staff have sufficient time and/or space

Provide some continuity of staff contact (personal tutors)

Provide opportunity for dialogue (e.g. discuss feedback in class, peer review, peer assisted learning)

Peer marking using model answers (Forbes & Spence, 1991)

Scenario:• Engineering students had weekly maths problem sheets

marked and problem classes• Increased student numbers meant marking impossible

and problem classes big enough to hide in• Students stopped doing problems• Exam marks declined (Average 55%>45%)Solution:• Course requirement to complete 50 problem sheets• Peer assessed at six lecture sessions but marks do not

count• Exams and teaching unchangedOutcome: Exam marks increased (Av. 45%>80%)

Peer feedback - Geography (Rust, 2001)

Scenario• Geography students did two essays but no apparent improvement

from one to the other despite lots of tutor time writing feedback• Increased student numbers made tutor workload impossible

Solution:• Only one essay but first draft required part way through course• Students read and give each other feedback on their draft essays• Students rewrite the essay in the light of the feedback• In addition to the final draft, students also submit a summary of how

the 2nd draft has been altered from the1st in the light of the feedback

Outcome: Much better essays

Peer feedback - Computing (Zeller, 2000*)

The Praktomat system allows students to read, review, and assess each other’s programs in order to improve quality and style. After a successful submission, the student can retrieve and review a program of some fellow student selected by Praktomat. After the review is complete, the student may obtain reviews and re-submit improved versions of his program. The reviewing process is independent of grading; the risk of plagiarism is narrowed by personalized assignments and automatic testing of submitted programs.

In a survey, more than two thirds of the students affirmed that reading each other’s programs improved their program quality; this is also confirmed by statistical data. An evaluation shows that program readability improved significantly for students that had written or received reviews.

[*Available at: http://www.infosun.fim.unipassau.de/st/papers/iticse2000/iticse2000.pdf]

Figure 1: Peer-review as a method of encouraging students to discuss and compare their understanding of assessment criteria

2. Students bring draft individual

assignments for peer review

MODULE TIMELINE

4. Students rewrite and

submit individual

assignments

Week1 Week12

1. Tutor leads discussion on assessment criteria and

process of peer review

3. In-class discussions

between student groups as they

review each other’s work, monitored by

tutor.

5. Tutor marks

assignments and prepares

feedback

6. Tutor hands back assignments

and leads discussion on

feedback

Assignmentpoint

OU

T O

F C

LA

SS

A

CT

IVIT

YIN

-CLA

SS

A

CT

IVIT

Y

Figure 1

Figure 2: the use of 'exemplars' as amechanism for encouraging dialogue about assessment criteria

1. Tutor leads discussion of previously-

marked exemplars

annotated with feedback

OU

T O

F C

LAS

S

AC

TIV

ITY

IN-C

LAS

S

AC

TIV

ITY

2. Students write and

submit individual

assignments

3. Tutor marks

assignments and prepares

feedback

4. Tutor hands back assignments

and leads discussion on

feedback

MODULE TIMELINE

Assignmentpoint

Week1 Week12

Figure 2

Figure 3: Generic feedback and self critique

1. Students draft and submit

individual assignments

2. Tutor marks sample of

assignments and prepares

generic feedback

5. Tutor grades

assignments

4. Students rewrite and submit individual assignments with

reflective commentary on how they have incorporated the generic feedbackO

UT

OF

CLA

SS

A

CT

IVIT

YIN

-CLA

SS

A

CT

IVIT

Y

MODULE TIMELINE

Assignmentpoint

Week1 Week12

3. In-class discussion of generic cohort

feedback based on

coursework sample

6. Tutor hands back assignments with grade

only

Figure 3

Activity

Individually:

Choose one or more specific ideas to improve feedback that you think you could use. In as much detail as possible, identify how you would put the idea/s into practice.

In pairs:

Take it in turns to explain your plans to your partner. The job for the listener is to be a friendly and constructive critic

Feedback moments

• Where there is a clear opportunity to apply feedback

• Pre assessment

• Reflection points

Identify them within each programme

Figure 4:Taking an overview

Student preparation – HE orientation

PR

OG

RA

MM

E D

UR

AT

ION

End Yr1

End Yr2

End Yr3

Regular review meetings with personal tutor to

discuss feedback

Sem 1: Introduction to self-assessment and peer review.

Discussion of criteria and use of exemplars

Sem 2: Introduction to group work, continued development of peer and self-assessment. Continued

use of exemplars

Year 2: Support in the transition to Stage 2 modules, with more

formative feedback

Year 3: Expectation that students will engage in more self-

assessment, and will demonstrate ability to critique own work

Student involvement in peer-assisted learning

(voluntary)

Student involvement in mentoring others (voluntary)

PROGRAMME LEVEL MODULAR LEVEL

Refs

• Forbes, D., & Spence, J. (1991). An experiment in assessment for a large class. In R.Smith (Ed.), Innovations in engineering education. London: Ellis Horwood.

• Fritz, C.O., Morris, P.E., Bjork, R.A., Gelman, R. & Wickens, T.D. (2000) When further learning fails: Stability and change following repeated presentation of text, British Journal of Psychology, 91, pp. 493-511

• Gibbs, G. & Simpson, C. (2002) Does your assessment support your students’ learning available at: http://www.brookes.ac.uk/services/ocsd/1_ocsld/lunchtime_gibbs.html (accessed November 2002)

• Higgins, R., Hartley, P. & Skelton, A. (2002) The conscientious consumer: reconsidering the role of assessment feedback in student learning. Studies in Higher Education, 27 (1) pp. 53-64

• Hounsell, D. 1987. Essay writing and the quality of feedback. In J.T.E. Richardson, M.W. Eysenck & D. Warren-Piper, eds. Student Learning: Research in Education and Cognitive Psychology, 42, no.2: 239-54.

• Holmes, L. E., & Smith, L. J. (2003). Student evaluations of faculty grading methods. Journal of Education for Business, Vol. 78 No. 6, 318.

• Lea, M. & Street, B. (1998) Student Writing in Higher Education: an academic literacies approach. Studies in Higher Education, 23 (2), pp. 157-172

• McCune, V., (2004) Development of first –year students’ conceptions of essay writing. Higher Education, 47, pp. 257-282.

• Maclellan, E. 2001. Assessment for learning, the different perceptions of tutors and students. Assessment and Evaluation in Higher Education. 26, no.4: 307-318

• Ridsdale, M.L.“I’ve read his comments but I don’t know how to do”:International postgraduate student perceptions of written supervisor feedback. In ‘Sources of confusion: refereed proceedings of the national language and academic skills conference held at La Trobe University, November 27-28,2000’ edited by \k \charnock, pp272-282.

• Rust, C. (2001) A briefing on assessment of large groups, LTSN Generic Centre Assessment Series, No. 12, York, LTSN

• Wotjas, O. 1998. Feedback? No, just give us the answers. Times Higher Education Supplement. September 25