seminar at mary immaculate college, university of limerick, jan 2016

Consistency and Conformity in Assessment in Higher Education: managing the apparently competing demands of systems and academic decision-making

Rachel ForsythCentre for Excellence in Learning and

Teaching, Manchester Metropolitan University, UK

@rmforsythr.m.forsyth@mmu.ac.uk

1. About Manchester Metropolitan University 2. About our assessment project 3. Next steps

Image credit: © Copyright Neil Theasby and licensed for reuse

Barrier to implementing assessment for learning: pink cardExample of good practice in changing assessment: green card

8 faculties

Around 1,000 courses

36, 000 students

2700 on Combined Honours

Links with over 70 professional bodies

1,500 teaching staff. plus 1,000 associates

Over 500,000 assignment submissions annually

33.23.43.63.8

MMU Assessment and Feedback outcomes in NSS 2008-2011

2008200920102011

NSS Questions

NSS

Sati

sfac

tion

2008 2009 2010 20113.4

3.5

3.6

3.7

3.8

3.9

4

Overall NSS satisfaction outcomes for assessment and feedback

UK averageMMU

NSS

satis

facti

on sc

ore

for

Asse

ssm

ent a

nd F

eedb

ack

Transforming Assessment and Feedback for Institutional Change (TRAFFIC)

Trying to get assessment data which…• Is accessible and helpful to personal

tutors; • Shows alignment with UK Quality

Code;• Can be used to produce a Higher

Education Achievement Report (HEAR);• Identifies areas for institutional

development

Examinati

on Ess

ay

Report

Portfolio

Presen

tation

Projec

t

Case St

udy

Dissert

ation

Critical

Analysis

Group Assessm

...0%

5%

10%

15%

20%

25%

MMU Most common assignment types 2013/14

level 4 - total number 1077, 65 typeslevel 5 - total number 1333, 71 typeslevel 6 - total number 1606, 74 types

Freq

uenc

y of

ass

ignm

ent t

ype

Principal findings of TRAFFIC baseline report

• Data was inconsistently recorded with respect to assignment tasks, submission, feedback and moderation, use of grade descriptors and assessment criteria• Need to address assignment

clustering

Actual Rules• Assignments had to be

aligned to learning outcomes • Maximum of two

assignments per module (15 ECTS credits)• Grades given in

percentages • All marking to be

internally and externally moderated

Examination

Essay

Report

Portfolio

Presentation

Project

Case Study

Disserta

tion

Critical A

nalysis

Group Assessm

...0%

5%

10%

15%

20%

25%

MMU Most common assignment types 2013/14

level 4 - total number 1077, 65 typeslevel 5 - total number 1333, 71 typeslevel 6 - total number 1606, 74 types

Freq

uenc

y of

ass

ignm

ent t

ype

Blog PosterWorkbookAutobiography Computer Based Logbook Override Portfolio report Programming Assignment Programming Exercises Website Bibliography Documentation Learning Agreement Oral-practical Placement assessmentAnalysis Class Activity Data Exercise Interview Moot SeminarSource Analysis Critical Evaluation Essay/Report/Case StudyExhibition Film Independent Study Negotiated Personal Dossier Presentation Essay Report/PresentationArticle Booklet CGI scene/documentation Close Reading Critical Commentary Design and Product Diary Ensemble projectEthical Practice Report Film/Documentary Group & Solo projectGroup Essay Group presentation Group Web Page H&S Report IT Exercises Journal Laboratory report Literature Rev & Search Magazine Media Project Observation Oral Examination Plan Practical Portfolio Practical Report Précis Exercise Process BlogResearch File Research Project Review Scenario Exercise ScriptSeen Essay Skills Portfolio Skills TrainingTeaching Report Team Portfolio Text

Will I look forward to marking it?

Will it let students demonstrate achievement of learning

outcomes?

Will students understand what to do?

Can I link it clearly into what I’m going to teach?

Will it be straightforward to mark and moderate?

Student ID (+ Unit code)

Deadlines / extensions / feedback return dates / provisional marks

Personalised submission sheet

Coursework submission

Principal Outcomes• Rewrote

institutional code of practice on assessment and associated procedures for assignment briefs, marking, moderation and feedback to students. • Embedded principle of

separation of procedures from academic decision-making about assessment

Previous grade descriptors

• Excellent

• Very Good

• Good

• Pass

• Marginal Fail

• Fail

Grade band Indicative language 86%-100%

Creative, insightful, illuminating, inspiring, exciting, authoritative

70%-85%

Persuasive, sophisticated, original, reflective, ambitious, meticulous, critical, convincing, unexpected

60%-69% Fluent, thorough, analytical, precise, rigorous 50%-59%

Clear, confident, consistent, thoughtful, accurate, careful, congruent, coherent

20%-40% Erroneous/wrong, missing, extremely limited, inappropriate, insufficient, incoherent, unstructured

0%-19% Absent/none, lacking, formless, detrimental

You made me fail my students

33.23.43.63.8

44.2

MMU Assessment and Feedback outcomes in NSS 2010-2015

201020112012201320142015

NSS Questions

NSS

Sati

sfac

tion

2008 2009 2010 2011 2012 2013 2014 20153.4

3.5

3.6

3.7

3.8

3.9

4

Overall NSS satisfaction outcomes for assessment and feedback

UK averageMMU

NSS

satis

facti

on sc

ore

for A

sses

smen

t and

Fe

edba

ck

Image credit: © Copyright Neil Theasby and licensed for reuse

Try to add a solution to the reverse of the pink cardSuggest how you might implement good practice on reverse of green card

https://assessmentinhe.wordpress.com/@assessmentinhe

http://www.slideshare.net/rmforsyth

@rmforsythr.m.forsyth@mmu.ac.uk

Picture credits Images from MMU, except

photo of Allsorts by clrcmck, CC licensed on Flickr Detail from "Les Lavandières de la nuit" ("The Washerwomen of the Night"), by Jean-Édouard Dargent c.1861Anger by Saurabh Vyas, CC licensed on FlickrWashing Image credit: © Copyright Neil Theasby and licensed for reuseOverloaded car by Stringer_bell, CC licensed on Flickr