ahe exemplars

Managing dialogue around exemplars

David Carless, Masterclass

AHE Manchester,

June 28, 2017

http://davidcarless.edu.hku.hk/

Twitter: @CarlessDavid

The University of Hong Kong

http://davidcarless.edu.hku.hk/

Overview

1. Exemplars: rationale and benefits

2. Challenges and how they might be tackled

3. Managing dialogue

4. Implementation suggestions


Teaching Enhancement Project

Enhancing dialogic use of exemplars amongst 10 teachers in a Faculty of Education


What are exemplars?

Samples used to illustrate dimensions of quality

Usually assignments from a previous cohort


Why used?

Exemplars convey messages that nothing else can (Sadler, 2002)


Promotes self-evaluation

Illustrate what good work looks like

Benchmark for comparison with own performance


Benefits for students

Gain experience in making judgments

Apply insights to own work & improve learning outcomes


Hendry et al., (2012)

Students positive about exemplars

Balanced teacher-led discussion of exemplars as crucial


Student views

Rubrics: vague, unclear and do teachers really use them?

Exemplars: useful & concrete; students want more of them

(Carless, 2015, 2017)


‘Model answer syndrome’

May reduce student creativity

May lead to copying

(Handley & Williams,

2011)


Scaffolded use of exemplars

Students work on assessment task prior to exposure to exemplars


Scaffolding steps

Instructional sequences combining:

- Student work in progress;

- Interaction with peers and teachers;

- Analysis of exemplars.

(Carless et al., 2018)


Time issues

• Collecting them

• Obtaining consent

• Time taken from instruction

• How many exemplars?


Content vs process

Too much time spent teaching content, insufficient attention to learning processes


It’s challenging!

Difficulties for students in evaluating exemplars accurately


Have I addressed challenges satisfactorily?

Any other thoughts or queries?


DIALOGIC USE OF EXEMPLARS


Key assumption

The quality of dialogue about exemplars is crucial


Research question

What are the main features of a specific exemplars dialogue and how is it orchestrated?


Context of the case

Participants:

• Trainee Science teachers

• BEd/BSc double degree

Teacher-researcher:

Dr Kennedy Chan


Data collection

• Classroom observation

• Open-ended student survey

• 2 focus group interviews

• Interview with teacher-researcher

• Teacher journal

• Student artefacts: ‘exit slips’, assignments etc.


Coding scheme

Main teacher moves:

- Eliciting student views

- Summarizing student views

- Elaborating student views


Exemplars provided

2 high quality exemplars

– Same format (i.e. reflective essays) but different

content focus

– Design aimed at reducing ‘copying’


Management of dialogue

1. Prior to class, students analyzed exemplars;

2. Students discussed exemplars in pairs;

3. Students elicited views from peers and made mini-presentations;

4. Teacher-orchestrated dialogue;

5. Students submitted exit slip of reflections.


Dilemmas

Students

constructing

notion of quality

VS

Telling students

about quality

Time for students

to talk with

peers

VS

Time for

developing shared

understandings

Exemplars as

guide

VS

Exemplars as

model

Using students’

voices

VS

Making

teacher’s voice

explicit


Good Exemplars dialogue

• Airs multiple & divergent viewpoints

• Shows linkages between peer talk & whole-class discussion

• Evidences development of student views

• Makes explicit some key qualities of exemplars

(Carless & Chan, 2016)


Implications


Implementation blueprint

Assessment task

Students devise or engage with criteria

for good task response

Two samples read before class

Peer discussionTeacher-led

dialogueStudent ownership

of insights


Exemplars & feedback

By developing capacities to make judgments,students are learning to decode feedback(Sadler, 2010)

Analysis of exemplars facilitates student uptakeof feedback (Carless et al., 2018; To & Carless,2016)


Engaging with quality

Student need to engage with what quality looks like and develop capacities in making judgments


COMMENTS,

QUESTIONS


References

Carless, D. (2015). Excellence in University Assessment: learning from award-winning teachers. London: Routledge.

Carless, D. (2017). Students’ experiences of Assessment for Learning. In D. Carless, S. Bridges, C.K.W. Chan & R. Glofcheski (Eds.), Scaling up Assessment for learning in Higher Education. Singapore: Springer.

Carless, D. & K.K.H. Chan (2016). Managing dialogic use of exemplars. Assessment and Evaluation in Higher Education,http://dx.doi.org/10.1080/02602938.2016.1211246

Carless, D., Chan, K.K.H., To, J., Lo, M. & E. Barrett (2018). Developing students’ capacities for evaluative judgement through analysing exemplars. In D. Boud, R. Ajjawi, P. Dawson & J. Tai (Eds), Developing Evaluative Judgement in Higher Education: Assessment for knowing and producing quality work. London: Routledge.

Handley, K. & Williams, L. (2011). From copying to learning: Using exemplars to engage students with assessment criteria and feedback. Assessment and Evaluation in Higher Education, 36(1), 95-108.


http://dx.doi.org/10.1080/02602938.2016.1211246

References

Hendry, G., Armstrong, S. & Bromberger, N. (2012). Implementing standards‐based assessment effectively: Incorporating discussion of exemplars into classroom teaching. Assessment and Evaluation in Higher Education, 37(2), 149-161.

Sadler, D. R. (2002). Ah! … So that’s ‘quality’. In P. Schwartz & G. Webb (Eds.), Assessment: Case Studies, Experience and Practice from Higher Education (p.130-136). London: Kogan Page.

Sadler, D. R. (2010). Beyond feedback: Developing student capability in complex appraisal. Assessment and Evaluation in Higher Education, 35(5), 535-550.

To, J. & Carless, D. (2016). Making productive use of exemplars: Peer discussion and teacher guidance for positive transfer of strategies. Journal of Further and Higher Education, 40(6), 746-764.



Productive assessment task design

Appreciating the nature of quality work

Student engagement with feedback

Learning-oriented assessment framework