Assessing large groups: engaging first year

students

Professor Sally Brown

The current context Increasing pressure on units of resource in

universities; Increased cohort sizes even if raw numbers

don’t increase; International trends to take student

satisfaction ever more seriously (the Hunt report calls for a National Student Survey for Ireland);

Increasing competition for international students.

Why is assessment such a big issue? Good feedback and assessment practices

are essential to student learning; Student satisfaction surveys around the

world frequently highlight significant dissatisfaction around these issues;

In tough times, staff often find the pressure of achieving fast and formative feedback a heavy chore;

The Hunt Report emphasises the importance of good assessment.

Why is assessing first year students well so important?

Too little, too much or the wrong kind of assessment can impact negatively on students at risk of failure, withdrawal or underperformance;

Well-integrated authentic assessment in the first year can energise and motivate students;

Stress and mental health problems can be triggered for first year students by the assessment experience.

What are the purposes of assessing students in the 1st year? To help:

Us to know which students are likely to need supportive interventions;

Students to gauge the demands of the work and to understand what is the standard of work required;

Students to get a measure of their own achievement;

International students with diverse assessment experiences to become familiar with the Irish context.

Drop out and assessment

“Roughly two-thirds of premature departures take place in, or at the end of, the first year of full-time study in the UK. Anecdotal evidence from a number of institutions indicates that early poor performance can be a powerful disincentive to continuation, with students feeling that perhaps they were not cut out for higher education after all – although the main problems are acculturation and acclimatisation to studying”. (Yorke p.37)

Implications: assessment in the first semester is critical: it should be formative, informative, developmental and remediable.

We need to fully engage first year students

The first half of the first semester of the first year is our best opportunity meaningfully to engage with students;

The patterns of learning they establish in the early parts of the first year are crucial;

First year assignments which are for not just of learning can foster information literacy and other essential skills.

Strategies for ensuring assessment is for rather than of learning

It needs to be built-in rather than bolt-on; Assignments need to be authentic, that is,

assessing learning that is identified in the learning outcomes;

Learning outcomes need to be designed to be specific, measurable, achievable, realistic and time-constrained (SMART);

The assessment strategy should make sure that assignments are fit-for-purpose.

Assessment for Learning: see http://www.northumbria.ac.uk/sd/central/ar/academy/cetl_afl/

Emphasises authenticity and complexity in the content and methods of assessment rather than reproduction of knowledge and reductive measurement.

Uses high-stakes summative assessment rigorously but sparingly rather than as the main driver for learning.

Offers students extensive opportunities to engage in the kinds of tasks that develop and demonstrate their learning, thus building their confidence and capabilities before they are summatively assessed.

Is rich in feedback derived from formal mechanisms e.g. tutor comments on assignments, student self-review logs.

Is rich in informal feedback e.g. peer review of draft writing, collaborative project work, which provides students with a continuous flow of feedback on ‘how they are doing’.

Develops students’ abilities to direct their own learning, evaluate their own progress and attainments and support the learning of others.

Good feedback practice (after Nicol et al):

1. Helps clarify what good performance is (goals, criteria, expected standards);

2. Facilitates the development of self-assessment (reflection) in learning;

3. Delivers high quality information to students about their learning;

4. Encourages teacher and peer dialogue around learning;

5. Encourages positive motivational beliefs and self-esteem;

6. Provides opportunities to close the gap between current and desired performance;

7. Provides information to teachers that can be used to help shape the teaching.

What can we do to engage first year students through assessment?

Set plenty of small early assessed tasks (formative or summative) and turn them round fast in the crucial first semester;

Monitor student attendance / engagement and take action when students disappear and particularly when work is not handed in;

Do what we can to personalise the learning experience;

Streamline assessment, to maximise the number of opportunities students can engage in the process in the first year and gain feedback on their work.

Why would we wish to streamline assessment?

Huge pressure on resources in higher education;

Larger numbers of students in cohorts; Ever-increasing demands on staff time; Staff indicate they spend a disproportionate

time on assessment drudgery; The means exist nowadays to undertake

some aspects of assessment more effectively and efficiently.

Looking at the alternatives

Each of the following methods aims to make giving feedback to students more effective and efficient.

Any single method used exclusively is unlikely to be acceptable to students;

It’s best to ring the changes so that your means of assessment provides a variety of different kinds of feedback.

To give feedback more effectively & efficiently, we can:

Use model answers; Use assignment return sheets; Write an assignment report; Feedback in class to groups of students; Use statement banks; Use computer-assisted assessment; Involve students in their own assessment.

Using model answers: why? They give students a good idea of what

can be expected of them; It is sometimes easier to show students

than tell them what we are after; They can be time efficient; They show how solutions have been

reached; They demonstrate good practice; The commentary can indicate why an

answer is good.

Using model answers: how? Staff preparing an assignment can draft a

model answer; Student work (or extracts from several

student’s answers) can be anonymised and (with permission) used as a model;

Text can be placed on page with explanatory comments appended (‘exploded text’);

However, caution should be exercised in order to lead students to think only one approach is acceptable.

Assignment return sheets: why?

Proformas save assessors writing the same thing repeatedly;

Helps to keep assessors’ comments on track; Shows how criteria match up to performance

and how marks are derived; Helps students to see what is valued; Provides a useful written record.

Assignment return sheets: how?

Criteria presented in assignment brief can be utilised in a proforma;

Variations in weighting can be clearly identified;

A Likert scale or boxes can be used to speed tutor’s responses;

Space can be provided for individual comments.

Written assignment reports: why?

Provides feedback to a group as a whole; Allows students to know how they are doing

by comparison with the rest of the course; Offers a chance to illustrate good practice; Minimal comments can be put on scripts.

Assignment reports: how? Staff mark assignments with minimal in-

text comment and provide grades/marks as normal;

Notes are made of similar points from several students’ work;

A report is compiled which identifies examples of good practice, areas where a number of students made similar errors and additional reading suggestions.

Feeding back orally to groups of students: why?

Face-to-face feedback uses tone of voice, emphasis, body language;

Students learn from feedback to each others’ work;

Students can ask questions; Makes feedback a shared experience.

Feeding back orally to groups of students: how?

Staff mark assignments with minimal in-text comment and provide grades/marks as normal;

At the start of a lecture or seminar, the tutor provides an overview of class performance and orally remediates errors ,clarifies misunderstandings, and praises good practice;

Students have a chance to ask and answer questions.

Statement banks: why?

Harnesses a resource of comments you already use;

Avoids writing same comments repeatedly; Allows you to give individual comments

additionally to the students who really need them;

Can be automated with use of technology.

Statement banks: how?

Tutor identifies a range of regularly used comments written on students’ work;

These are collated and numbered; Tutor marks work and writes numbers on text

of assignment where specific comments apply, or provides a written (or emailed) detailed commentary which pulls together the appropriate items into continuous prose.

Computer-assisted assessment: why? Enables feedback to be given regularly and

incrementally; Saves tutor time for large cohorts and

repeated classes; Can allow instant (or rapid) on screen

feedback to e.g. MCQ options; Saves drudgery, (but not a quick fix); Can track the performance of test items.

Computer-assisted assessment: how?

This should not be a cottage industry! Training and support both in designing

questions and applying the relevant technology are essential;

Testing and piloting of CAA items is also imperative;

Make use of existing test packages (e.g. from publishers), colleagues with expertise and advice from software companies (e.g. QuestionMark).

Use CAA for rather than of learning We can explore employing computer-assisted

formative assessment with responses to student work automatically generated by email;

Students seem to really like having the chance to find out how they are doing, and attempt tests several times in an environment where no one else is watching how they do;

We can monitor what is going on across a cohort, so we can concentrate our energies either on students who are repeatedly doing badly or those who are not engaging at all in the activity.

Involving students in their own assessment: why? Available research indicates that involving students in

their own assessment makes them better learners (deep not surface learning);

S &PA have the potential to save some time for staff (but effort is front loaded);

Students learning how to give feedback take the feedback they receive more seriously;

Students can get inside the criteria and start to work out what they really mean;

They are valuable for developing lifelong learning capabilities. (“How do I know how I’m doing?”)

However:

Criteria need to be explicit and clear to all concerned from the outset;

Assessment must use evidence matched against the criteria;

Students and staff need training and rehearsal before it is implemented ‘for real’.

Involving students in their own assessment

Get students to peer each other’s work in class (drafts, posters);

Ask them to critique an assignment they hand in, using the same assignment return form as you;

Get students to peer assess each other’s presentations

Get students to rate their contributions to a group activity.

Implementing self and peer assessment

There are no quick fixes in assessment; Effective implementation needs careful

briefing of all parties , rehearsal and unpacking;

Self and peer assessment rely on the provision of appropriate evidence against clear explicit and readily-available criteria;

You need to decide who (self, intra-peer, inter-peer) and how (formatively or summatively) you will implement it.

Students giving feedback to peers

Can be hugely beneficial if managed effectively (but there are no quick fixes!);

Students will need training or refreshing in purposes and practices of peer feedback;

Work on language use is crucial; Building students’ expertise in giving peer

feedback helps them get more from the feedback they receive.

To design an assessment strategy to engage first year students, we need to:

Explore ways in which assessment can be made integral to learning;

Constructively align (Biggs 2003) assignments with planned learning outcomes and the curriculum taught;

Provide realistic tasks: students are likely to put more energy into and play fairer with assignments they see as authentic and worth bothering with;

Maximise the uses students make of feedback.

Encouraging students to take assessment more seriously

All assessment needs to be seen to be fair, consistent, reliable, valid and manageable;

Many assessment systems fail to clarify for students the purposes of different kinds of assessment activity;

Low-stakes early formative assessment helps students, especially those from disadvantaged backgrounds, understand the rules of the game.

Making assessment work well Intra-tutor and Inter-tutor reliability need to be

assured; Practices and processes need to be

transparently fair to all students; Cheats and plagiarisers need to be

deterred/punished; Assessment needs to be manageable for both

staff and students; Assignments should assess what has been

taught/learned not what it is easy to assess.

Students benefit if we can make feedback timely Aim to get feedback on work back to students very

quickly, while they still care and while there is till time for them to do something with it.

The longer students have to wait to get work back, especially if they have moved into another semester by the time they receive their returned scripts, the less likely it is that they will do something constructive with lecturer’s hard-written comments.

I argue for a maximum three-week turnaround: some argue for much less.

Helping students make the transition into tertiary education

For some, higher education will feel remote, alienating and frightening: maximising engagement is crucial;

Designing assessment activities that ensure students work in groups which valorise supportive collegiate behaviour can make a big difference;

It helps to build confidence by starting with a great deal of support and move progressively to higher levels of independence and autonomy.

Using formative assessment to scaffold independence and learning

Investigate how learning can be advanced in small steps using a ‘scaffolding’ approach;

Provide plenty of ‘low stakes’ assignments in the early stages when students may not understand the ‘rules of the game’;

This can then be progressively removed as students become more confident in their own abilities.

Play fair with students by avoiding using ‘final language’ (Boud)

Avoid destructive criticism of the person rather than the work being assessed.

Try not to use language that is judgmental to the point of leaving students nowhere to go.

Words like “appalling”, “disastrous” and “incompetent” give students no room to manoeuvre.

However, words like ”incomparable” and “unimprovable” don’t help outstanding students to develop ipsatively either.

Play fair by giving feedback to students with diverse abilities

Students at the top end of the ability range sometimes feel short changed by minimal feedback;

Students with many weaknesses easily become dispirited if there is too much negative feedback;

Consider giving an assessment sandwich. Start with something positive, go into the detailed critique and find something nice to say at the end (to motivate them to keep reading!);

Explore ways to incentivise reading of feedback; Consider which medium to use for students with

disabilities (e.g. don’t use bad handwriting for those with visual impairments or dyslexia!).

Conclusions Assessment impacts highly on student

learning and resilience in the first year so we need to rethink how we can assess most effectively, taking account of new contexts, new technologies and new opportunities;

Efficient and effective feedback is just about the most important thing we do to enhance student learning, progression and success.

Useful references: 1

Biggs, J. (2003) Teaching for Quality Learning at University (Maidenhead: SRHE & Open University Press).

Bowl, M. (2003) Non-traditional entrants to higher education ‘they talk about people like me’ Stoke on Trent, UK: Trentham Books

Brown, S. Rust, C. & Gibbs, G. (1994) Strategies for Diversifying Assessment Oxford Centre for Staff Development.

Boud, D. (1995) Enhancing learning through self-assessment London: Routledge.

Brown, G. with Bull, J. and Pendlebury, M. (1997) Assessing Student Learning in Higher Education London: Routledge.

Brown, S. and Glasner, A. (ed.) (1999) Assessment Matters in Higher Education, Choosing and Using Diverse Approaches, Maidenhead: Open University Press.

Brown, S. and Knight, P. (1994) Assessing Learners in Higher Education, London: Kogan Page.

Useful references 2 Brown, S., Race, P. and Bull, J. (eds.) (1999) Computer Assisted

Assessment in Higher Education London: Routledge. Carroll, J. and Ryan, J. (2005) Teaching International students:

improving learning for all London: Routledge SEDA series. Falchikov, N. (2004) Improving Assessment through Student

Involvement: Practical Solutions for Aiding Learning in Higher and Further Education, London: Routledge.

Gibbs, G. (1999) Using assessment strategically to change the way students learn, In Brown S. & Glasner, A. (eds.), Assessment Matters in Higher Education: Choosing and Using Diverse Approaches Maidenhead: SRHE/Open University Press.

Kneale, P. E. (1997) The rise of the "strategic student": how can we adapt to cope? in Armstrong, S., Thompson, G. and Brown, S. (eds) Facing up to Radical Changes in Universities and Colleges, 119-139 London: Kogan Page.

Useful references 3 Knight, P. and Yorke, M. (2003) Assessment, learning and employability

Maidenhead, UK: SRHE/Open University Press. McDowell, E. & Brown, S. (1998) Assessing students: cheating and

plagiarism, Red Guide 10/11 University of Northumbria, Newcastle Mentkowski, M. and associates (2000) Learning that lasts: integrating

learning development and performance in college and beyond p.82, San Francisco: Jossey-Bass.

Nicol, D. J. and Macfarlane-Dick, D. (2006) Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, Vol 31(2), 199-218

Peelo, M. and Wareham, T. (eds.) (2002) Failing Students in higher education Buckingham, UK, SRHE/Open University Press.

Sadler, D. R. (1989) Formative assessment and the design of instructional systems Instructional Science 18, 119-144.

Sadler, D. R. (1998) Formative assessment: revisiting the territory Assessment in Education: Principles, Policy and Practice 5, 77-84.

Pickford, R. and Brown, S. (2006) Assessing skills and practice London: Routledge.

Useful references 4 Race, P. (2001) A Briefing on Self, Peer & Group Assessment in

LTSN Generic Centre Assessment Series No 9 LTSN York. Race P. (2006) The lecturer’s toolkit (3rd edition) London:

Routledge. Race P and Pickford r (2007) Making Teaching work: Teaching

smarter in post-compulsory education, London: Sage. Rust, C., Price, M. and O’Donovan, B. (2003). Improving students’

learning by developing their understanding of assessment criteria and processes. Assessment and Evaluation in Higher Education. 28 (2), 147-164.

Ryan, J. (2000) A Guide to Teaching International Students Oxford Centre for Staff and Learning Development.

Stefani, L. and Carroll, J. (2001) A Briefing on Plagiarism http://www.ltsn.ac.uk/application.asp?app=resources.asp&process=full_record&section=generic&id=10

Yorke, M. (1999) Leaving Early: Undergraduate Non-completion in Higher Education, London: Routledge.