oxford centre for staff and learning development developing a variety of assessment methods chris...

Oxford Centre for Staff and Learning Development

Developing a variety of assessment methods

Chris Rust

Oxford Centre for Staff

and Learning Development

Oxford Brookes University


Constructive alignment - issues of validity

“We continue to assess student learning - and to graduate and certify students much as we did in 1986, 1966, or 1946, without meaningful reference to what students should demonstrably know and be able to do”(Angelo, 1996)

“Assessment systems dominate what students are oriented towards in their learning. Even when lecturers say that they want students to be creative and thoughtful, students often recognise that what is really necessary, or at least what is sufficient, is to memorise”(Gibbs, 1992)


Constructive alignment - what is it?

“The fundamental principle of constructive alignment is that a good teaching system aligns teaching method and assessment to the learning activities stated in the objectives so that all aspects of this system are in accord in supporting appropriate student learning”

(Biggs, 1999)


Constructive alignment: 3-stage course design• What are “desired” outcomes?• What teaching methods require students to behave

in ways that are likely to achieve those outcomes?• What assessment tasks will tell us if the actual

outcomes match those that are intended or desired?

NB Importance of consistent use of the same appropriate verb in all three stages

This is the essence of ‘constructive alignment’(Biggs, 1999)


Why variety?

• Constructive alignment and validity• Traditional assessment only assesses a narrow

range of skills• Fairness• Logic of outcomes-based course design• Choice and variety can interest & motivate

(& aid flexible learning) • Be strategic

• Economical & efficient • Student learning & skills development

(& retention)


Student learning and strategic assessment

“Assessment defines what students regard as important, how they spend their time and how they come to see themselves as students and then as graduates.........If you want to change student learning then change the methods of assessment”

(Brown, G et al, 1997)


Maximising student potential 1 - Clarify expectations/keep them busy

“Setting students assignments as soon as soon as they arrive at university could help cut dropout rates………integrate students into university life as quickly as possible. This involves making them aware of the quality and quantity of work expected from them…..The freedom…is too much for some and they probably need more structure in the first year.”

Higher, 24/1/03

Reporting Student Transition and Retention (STAR) project

(in 5 universities)


Maximising student potential 2 - Pace learning

Multiple-staged assignments

• Self-assessment

• Peer-assessment

CAA


Maximising student potential 3 - Skills development

• information retrieval• ICT skills • academic writing skills• time & project management• self-assessment• marking exercises/peer assess• reflection• group skills• etc……


Maximising student potential 4 - Positively reinforce/allow for early failure (a)

Low self-belief can adversely affect achievement

(leading to drop-out):• Believe failure due to lack of intelligence• Leads to ‘learned helplessness’• Difficult tasks prompt giving up• Overly concerned with ‘saving face’

Mantz Yorke, based on study of six institutions. ILT symposium on widening participation and promoting student retention, 27th September, 2001


Maximising student potential 4 - Positively reinforce/allow for early failure (b)

Possible assessment solutions:• Adjust the curriculum to foster development• Set sub-goals for longer assignments• Allow for ‘slow learning’ in the programme’s

assessment strategy• Make first semester assessments primarily

formative

Mantz Yorke, based on study of six institutions. ILT symposium on widening participation and promoting student retention, 27th September, 2001


Maximising student potential 5 - Feedback

11 conditions under which assessment supports learning (Gibbs and Simpson, 2002)

5. Sufficient feedback is provided, both often enough and in enough detail

6. The feedback focuses on students’ performance, on their learning and on actions under the students’ control, rather than on the students themselves and on their characteristics

7. The feedback is timely in that it is received by students while it still matters to them and in time for them to pay attention to further learning or receive further assistance

8. Feedback is appropriate to the purpose of the assignment and to its criteria for success.

9. Feedback is appropriate, in relation to students’ understanding of what they are supposed to be doing.

10.Feedback is received and attended to.11.Feedback is acted upon by the student


5 Strategies for diversifying assessment• Change the criteria• Change the task• Mechanise assessment• Assess groups• Involve the studentsSee:

Brown, S., Rust, C. & Gibbs, G (1994)) Strategies for diversifying assessment in higher education, Oxford, Oxford Centre for Staff Development

Rust, C. (2001)A briefing on assessment of large groups: LTSN Generic Centre Assessment Series No. 12, York, LTSN, available at: http://www.ltsn.ac.uk/application.asp?app=resources.asp&process=

full_record&section=generic&id=12


Involve the students - 1 marking exercise

Immediate results participants av. mk non participants av. mk.

Cohort 1 (99/00) 59.78 54.12Cohort 2 (00/01) 59.86 52.86Cohort 3 (01/02 55.7 49.7

Results 1 year later

Cohort 1 57.91 51.3Cohort 2 56.4 51.7

Rust, C., Price, M & O’Donovan, B.(2003) "Improving students’ learning bydeveloping their understanding of assessment criteria and processes”Assessment and Evaluation in Higher Education, Vol. 28, No. 2


Involve the students - 2 self assessment

Strengths of this piece of work

Weaknesses in this piece of work

How this work could be improved

The grade it deserves is…..

What I would like your comments on


Involve the students - 3 peer marking using model answersScenario:• Engineering students had weekly maths problem sheets

marked and problem classes• Increased student numbers meant marking impossible

and problem classes big enough to hide in• Students stopped doing problems• Exam marks declined (Average 55%>45%)Solution:• Course requirement to complete 50 problem sheets• Peer assessed at six lecture sessions but marks do not

count• .Exams and teaching unchangedOutcome: Exam marks increased (Av. 45%>80%)


Involve the students - 4 peer feedbackScenario• Geography students did two essays but no apparent

improvement from one to the other despite lots of tutor time writing feedback

• Increased student numbers made tutor workload impossibleSolution:• Only one essay but first draft required part way through

course• Students read and give each other feedback on their draft

essays• Students rewrite the essay in the light of the feedback• In addition to the final draft, students also submit a summary

of how the 2nd draft has been altered from the1st in the light of the feedback

Outcome: Much better essays


So why don’t we?

• Not convinced (personally)

• Institution/colleagues/students not convinced

• Tradition/inertia/no incentive to change

• Time and/or other costs?

• Too difficult (need help/training)

• Not allowed (regulations/procedures)

• Other

oxford centre for staff and learning development developing a variety of assessment methods chris...

Documents