Improving Assessment and Feedback Practices in a

Technology-Enhanced Teaching and Learning Environment:

An Overview of the Literature

Dr Ann OomsFaculty of Health and Social Care SciencesKingston University and St. George’s University of London

Hendrik van der SluisAcademic Development CentreKingston University

Content

•Institutional context•Definitions•Value of formative assessment and

feedback•Effective feedback•Use of technology in formative

assessment and feedback

Institutional context• Decreased resources since 1980’s (Gibbs et al, 1996; Gibbs, 2006; Hounsell,

2007)▫ Increased student numbers ▫ Increased large class sessions▫ Increased student-staff ratio▫ Decreased contact hours, increased self-study time▫ Less coursework▫ Less time per student for guidance and support

• Modular & semester (Gibbs, 2006; Hounsell, 2007)▫ Increased modular and semester structure▫ Small modules, more (small) ‘end-load’ summative assessment▫ Shorter length of modules leaves less time to practice, more focus on content,

less on progression • Student characteristics (Gibbs, 2006; Hounsell, 2007)

▫ More part-time students (employed)▫ Concerns about coping with workload▫ Conservative in assessment approach▫ Try to identify the hidden curriculum▫ Diminished belief that feedback ‘makes a difference’

Three strands in literature

•Value of formative assessment and feedback

• ‘Good’ formative assessment and feedback•Use of technology in formative assessment

and feedback

•Literature review▫Post 1998▫Mainly post-secondary education ▫90 publications

Definitions: Assessment

Assessment is all activities that educators and students undertake to get information that can be used diagnostically to alter teaching and learning (Black and Wiliam, 1998)

Formative Assessment for learning (Black and Wiliam, 1998; Chappuis, 2009) Learning-orientated assessment (Knight, 2006) Formal and informal processes teachers and students use to gather

evidence for the purpose of improving learning (Chappuis, 2009) ‘Low stakes’ assessment judgement, “fuzzy and exploratory and

conversational in character” (Knight, 2002: 277)Summative

Assessment of learning Assessments that provide evidence of student achievement for the

purpose of making a judgement about student competence or program effectiveness (Chappuis, 2009)

‘High-stakes’, “the judgements need to be highly reliable” (Knight, 2006: 439)

Definitions: feedback

Formative A dialogue between to provide information

about students’ understanding Informs both educators and students to

measure effectiveness of learning activity and to adapt teaching if necessary

Summative Measure to what extent learning outcomes

have been achieved‘Low-value’ - ‘high-value’ (Hounsell, 2007)

Potential impact of the feedback on learning

Value of formative assessment and feedback for students• Identify gaps between desired and current

knowledge, understanding, skills (Ramaprasad, 1983; Sadler, 1989)

• Provides information about the effectiveness of their learning strategies (Yorke, 2003; Sadler, 1998)

• Significant learning gains (Black and Wiliam, 1998)

• Particularly beneficial for lower achieving students (Black and Wiliam, 1998)

• Impacts student progression (Hodgson and Bermingham, 2004)

• Improves conceptual development (Bell, 1995)

Value of formative assessment and feedback for students

•Enhances motivation (Orsmond et al, 2005, Nicol and MacFarlane-Dick, 2004)

•Encourages reflection (Orsmond et al, 2005) •Empowers students as self-regulated

learners (Black et al, 2003; Nicol, 2008) •Improves self-esteem (Nicol and

MacFarlane-Dick, 2004) •Impact general academic experience

(Hodgson and Bermingham, 2004)•Impacts students employability (Yorke,

2005)

Value of formative assessment and feedback for educators

• Information (Black and Wiliam, 1998; Wiliam et al, 2004) ▫Data collected through formative assessment

provides information to educators about the effectiveness of teaching practices and learning activities

▫Educators can use this data diagnostically to alter teaching practices and learning activities when needed

• Formative assessment is essential for good pedagogy and teaching (Black and Wiliam, 1998; Garrison and Ehringhaus, 2004)

Feedback

Students identify misunderstandings and weaknesses

diminish misunderstandings

tailor future study efforts

insight in own learning strategies

Teachers identify students’ misconceptions, challenges, adapt teaching practices

Components of Effective Feedback • Rapid/timely (Tilson et al 1998, Yorke 2007, Yorke

2005, Scheeler et al 2006, Di Battista et al 2004, Gibbs & Simpson 2004)

• Corrective (Scheeler et al, 2006)• Constructive

▫ Specific comments (Bangert-Drowns, Kullick and Morgan, 1991; Elawar and Corno, 1985)

▫ Specific areas / suggestions for improvement (Bangert-Drowns, Kullick and Morgan, 1991; Elawar and Corno, 1985)

• Encourages students to focus their attention on their learning and understanding rather than on getting the right answer (Bangert-Drowns, Kullick and Morgan, 1991; Elawar and Corno, 1985)

7 Principles for Good Feedback (Nicol and Macfarlane-Dick, 2004)

1. Clarify what good performance is (goals, criteria, standards)

2. Facilitate the development of reflection and self-assessment in learning

3. Deliver high quality feedback to students4. Encourage educator and peer dialogue around

learning5. Encourage positive motivational beliefs & self

esteem6. Provide opportunities to close the gap between

current and desired performance7. Provides information to educators that can be

used to help shape future teaching

Use of Technology in Formative Assessment and Feedback• Technologies used in online and face-to-face

learning▫Virtual Learning Environments

▫ Discussion board & Test Manager ▫Electronic Voting System▫Podcasts▫Personal Digital Assistants▫Text messaging▫Tablet PCs

• limited research-based literature

Discussion board: Examples

• Keppell et al, 2006; Gaytan & McEwen, 2007▫ Potential for peer feedback, sharing and responding

• Wickstrom, 2003▫ Self-assessment, reflection & sharing

• Prins et al, 2005; Gaytan & McEwen, 2007 ; Nicol, 2010▫ General feedback, question – answer

• Handley & Williams, 2009▫ Exemplar feedback (general feedback)

• Heinrich, et al, 2010 ▫ Clarify assessment expectation and requirements▫ “Defining and clarifying of assignment tasks and avoid

duplication of students’ questions”

Computer-Adaptive Testing

• Lilley and Barker (2007) ▫ Correct errors in understanding▫ Enable student to identify what they need to achieve in

order to increase understanding▫ Increased proficiency in summative assessment

• Buchanan, (2000), Dibattista et al (2004)▫ Received instant feedback▫ Enhance performance

• Peat & Franklin (2003)▫ No differential effect on performance

• Miller (2009)▫ Moderately effective student learning▫ Student use of feedback lower as expected

Electronic Voting Systems

• Freeman, 1997; Cue, 1998; Wit, 2003; Boyle and Nicol, 2003; Draper and Brown, 2004; Ashton et al, 2004; Masikunas et al, 2008; Hudson & Bristow, 2006). ▫Useful for providing rapid feedback to students▫Enables students to compare their answers

with their peers. Self-directed learning▫Educators can use it as a diagnostic tool▫Feedback students receive helps them to learn▫Knowledge reinforcement

Podcasts

• Ribchester et al, 2008, Middleton et al, 2009; Rodway-Dyer et al, 2009; (Rotheram, 2007) “In the right circumstances, podcasting can enhance assessment

feedback and, therefore, augment the student learning experience” (Ribchester et al, 2008)

▫ Oral feedback on summative assessment▫ Oral feedback on formative assessment▫ Greater depth and detail of feedback▫ Highly personalized▫ Students felt more engaged with oral feedback than with

feedback in writing▫ Focused more on feedback than on mark ▫ Challenge: feedback is separated from assignment▫ Feed forward

Personal Digital Assistant

•Perry, 2003▫Palmtops have potential to act as a study

aid via interactive quizzes and exercises for which rapid feedback can be provided

Tablet PCs

•Cicchino and Mirliss, 2006▫Used Tablet PCs for student presentations

and peer feedback▫Peers provided feedback in writing on

Tablet PC, which was valued

References and contact info

• References see notes

• Dr. Ann Ooms▫ A.Ooms@sgul.kingston.ac.uk

• Hendrik van der Sluis▫ H.vanderSluis@kingston.ac.uk