scaling up assessment for learning
TRANSCRIPT
Scaling up Assessment for Learning
CITERS 2017, June 9
Professor David Carless
University of Hong Kong
The University of Hong Kong
Overview
• Assessment for learning (AfL)
• AfL & MOOCs
• Peer feedback & technology
• Leadership & communities of practice
The University of Hong Kong
Chapter 1: Scaling up AfL: Progress & prospects (Carless)
Chapter 14: How does Technology enable Scaling up AfL? (Dawson & Henderson)
The University of Hong Kong
AfL definition
Assessment for which the
first priority is promoting students’
learning (Black et al., 2004)
Cf. Formative assessment
Learning-oriented assessment
(Carless, 2007, 2015a)
The University of Hong Kong
Interlinked AfL strategies
1. Productive assessment design
2. Students making judgments
3. Effective feedback processes
4. Developing student appreciation of quality
(Carless, 2017)
The University of Hong Kong
Scaling up (Coburn, 2003)
Spread
Depth
Sustainability
Shifts in ownership
The University of Hong Kong
Why scale-up AfL?
Research evidence
(Black & Wiliam, Hattie)
Dissatisfaction with current assessment & feedback practices
The University of Hong Kong
Black & Wiliam (1998)
Formative assessment & learning gains
Effect sizes: 0.4 - 0.7
… But implementation is generally weak
The University of Hong Kong
MOOC Assessment design
Alignment of goals, activities & low-stakes assessment
MC quizzes + automated
feedback
Higher order thinking?
The University of Hong Kong
Peer assessment
Peer reviewed assignments + detailed rubrics as key feature of MOOCs
(Admiraal et al, 2015; Huisman et al., 2016)
The University of Hong Kong
Reliability of PA
Moderate reliability of peer assessment (Admiraal et al., 2015)
Use of multiple peer assessments to mitigate variance in judgments (Hew, 2016)
The University of Hong Kong
Peer feedback for learning
Learners gain more from composing than receiving peer feedback (Nicol et al., 2014)
The University of Hong Kong
Peer & Self-evaluation
Students did 5 peer reviews then self-evaluated own work
(Hew, 2016)
The University of Hong Kong
Audio & Video feedback
Rapport
Nuance
Personalisation
Monologue or Dialogue?
Time saver?
The University of Hong Kong
Peer video feedback
Peer-to-peer video feedback
delivered via Facebook
Hung (2016)
The University of Hong Kong
Personalised feedback at scale
Using learning analytics to
scale the provision of
personalised feedback
https://www.ontasklearning.org/
The University of Hong Kong
Sustainable feedback
Students generating & using feedback from peers or self as part of self-regulated learning (Carless et al., 2011)
The University of Hong Kong
MOOC limitation?
MOOCs do not systematically develop student understanding of quality
(Dawson & Henderson, 2017)
The University of Hong Kong
Using exemplars The University of Hong Kong
Exemplars convey messages that nothing else can
(Sadler, 2002)
Appreciating quality
Students need to debate nature of quality & develop capacities in making judgments
The University of Hong Kong
Enhancing design
Adding technology to existing design & expecting improvements is flawed
Invest in improved assessment & feedback designs to leverage gains from technology
(Dawson & Henderson, 2017)
The University of Hong Kong
TELI
Technology-Enhanced Learning Initiative
The University of Hong Kong
Associate Vice-President (Teaching and Learning) Ricky Kwok
Communities of practice (CoPs)
Surfacing and sharing AfL practices (Hounsell & Zou, 2017)
The University of Hong Kong
References
Admiraal, W., Huisman, B., & Pilli, O. (2015). Assessment in massive open online courses. The Electronic Journal of e-Learning, 13(4), 207-216.
Black, P., Harrison, C., Lee, C., Marshall, B., & Wiliam, D. (2004). Working inside the black box: Assessment for learning in the classroom. Phi Delta Kappan, 86(1), 8-21.
Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education, 5(1), 7-74.
Carless, D. (2007). Learning-oriented assessment: Conceptual basis and practical implications. Innovations in Education and Teaching International, 44(1), 57-66.
Carless, D. (2015a). Exploring learning-oriented assessment processes. Higher Education, 69(6), 963-976.
Carless, D. (2015b). Excellence in University Assessment: learning from award-winning teachers. London: Routledge.
Carless, D. & K.K.H. Chan (2016). Managing dialogic use of exemplars. Assessment and Evaluation in Higher Education, http://dx.doi.org/10.1080/02602938.2016.1211246
Carless, D., Salter, D., Yang, M., & Lam, J. (2011). Developing sustainable feedback practices. Studies in Higher Education, 36(4), 395-407.
Coburn, C. (2003). Rethinking scale: Moving beyond numbers to deep and lasting change. Educational Researcher, 32(6), 3-12.
The University of Hong Kong
References
Hattie, J. (2009). Visible learning. London: Routledge.Hew, K. F. (2016). Promoting engagement in online courses: What strategies can we
learn from three highly rated MOOCS. British Journal of Educational Technology, 47(2), 320-341.
Hounsell, D. & Zou, T. (2017). Surfacing and Sharing Advances in Assessment: A Communities-of-practice approach. In D. Carless, S. Bridges, C.K.W. Chan & R. Glofcheski (Eds.), Scaling up Assessment for learning in Higher Education. Singapore: Springer.
Huisman, B., Admiraal, W., Pilli, L., van de Ven, M., & Saab, N. (2016). Peer assessment in MOOCs: The relationship between peer reviewers’ ability and authors’ essay performance. British Journal of Educational Technology. Doi: 10.1111/bjet.12520.
Hung, S.-T. A. (2016). Enhancing feedback provision through multimodal video technology. Computers & Education, 98, 90-101.
Nicol, D., Thomson, A., & Breslin, C. (2014). Rethinking feedback practices in higher education: a peer review perspective. Assessment & Evaluation in Higher Education, 39(1), 102–122.
Sadler, D. R. (2002). Ah! … So that’s ‘quality’. In P. Schwartz & G. Webb (Eds.), Assessment: Case Studies, Experience and Practice from Higher Education (p.130-136). London: Kogan Page.
The University of Hong Kong