appendix 4 quality of assessment practices presentation
DESCRIPTION
From the NQC websiteTRANSCRIPT
Quality of assessment practices
Rob StowellLearning Australia
Background
In late 2008 the NQC commissioned Learning Australia to undertake a scoping study to: • review what constitutes and contributes to quality
assessment and how perceptions or understandings of this may have changed in the last five years from the point of view of key stakeholders, including industry.
• identify the key issues impacting on the integrity and quality of assessment outcomes.
• analyse the breadth of existing materials available to support the VET workforce in determining quality assessment outcomes.
• determine what the critical components of quality assessment are.
Structure of report
Two stage project that built on Investigations into industry expectations of vocational education and training assessment - approved by NQC in mid 2008.
Stage 1 report comprises two volumes:• Volume one describes the outcomes of the scoping study
on the quality of assessment practices. • Volume two provides a listing of assessment support
materials Stage 2 report provides:
• review of assessment materials identified in stage 1.• a strategy for addressing the gaps, and establishes a
plan for the dissemination and implementation of any new, updated or existing materials for supporting quality assessment.
Copies of the reports
http://www.nqc.tvetaustralia.com.au/nqc_publications
Quality assessment
Clear and well understood benchmarks for assessment.
Principles of validity, reliability, fairness and flexibility.
Assessment decisions based on evidence which comply with the rules of evidence.
Quality assured - the AQTF2007 Essential Standards for Registration.
An integral part of learning and training process and the outcomes of assessment are actually used to inform the development of programs, modify training delivery to meet the needs of learners and improve learning. In quality assessment practices, evidence and feedback are used to identify where learners are in their learning, what they need to do and how best to achieve this.
Quality judgements.
Key recommendations in scoping
study Clear benchmarks for assessment Status and capability of assessors Improve assessment quality assurance by supporting the
effective implementation of the current AQTF standards High quality, industry endorsed model assessment support
materials Build assessment expertise within the VET sector Improve capability of assessors to make quality assessment
judgements Ensure that assessment is an integral part of the learning
and training process.
Changing emphasis in assessment
NowPast Future
Outcomes focused
Learner centered
Producing learners
Competence
Authenticity
Developing capacity to
make judgements
Learning and assessment skills beyond
formal training
Knowledge focused
Teacher centered
Testing students
Educational measurement
Activity - Complete the cartoon
Assessment and learning
Assessment may be seen as a summative process which has little or no connection to the processes of training and teaching.
Assessment evidence used for certification purposes and may not be used to identify learners’ achievements, diagnose their strengths and weaknesses and inform the design of learning plans that assist learners to achieve competency.
Summative or ‘assessment of learning’ is important but there is increasing interest in ‘assessment for learning’ or and its role in promoting learner achievement.
Assessment for learning
Assessment for learning focuses on the gap between where learners are in their learning, and where they need to be – the desired goal ie: competent performance. It is based on the idea that learners will improve most if they understand:
the competency to be achieved, where they are in relation to the required level of
performance how they can achieve the required level of
performance.
Refining formative assessment
Paul Black and Dylan Wiliam, Inside the Black Box - 1998 Refined the term ‘formative assessment’ by emphasising
that assessment is only formative when: it is an integral part of the learning and teaching
process; and assessment evidence is actually used to modify
teaching to meet the needs of learners and improve learning.
The Black Box: findings
Black and Wiliam’s research indicates that improving learning through assessment depends on five deceptively simple factors:
1. Providing effective feedback to students.
2. Learners’ active involvement in their own learning.
3. Adjusting teaching to take account of the results of assessment.
4. Recognising the profound influence of assessment on learners’ motivation and self-esteem - both crucial influences on learning.
5. Ensuring learners assess themselves and understand how to improve.
Performance range in control
groups
Range of typical performance enhancements in ‘assessment for learning’ groups
Performance range in Assessment for Learning
groups
Source: Black and Wiliam (1998)
Average effect size: 25% shift in performance compared to control groups after 2.5 years, and a reduced ‘spread’ in the performance range.
Low
Low
High
High
Evidence – Black and Wiliam
Research on assessment for learning in VET
Lack of a research base Policy and practice in formative assessment driven by
concerns about learner motivation, engagement and inclusion.
Connection between formative assessment and learner performance not adequately explored in VET.
Some recent work • Davies and Ecclestone 2008• Boud UTS – www. assessment futures.com.
Assessment for learning principles
Assessment for learning is part of effective planning for VET delivery focuses on how learners’ learn is central to training delivery involves high quality interactions between VET trainers and
learners is a key professional skill of Vet practitioners takes account of the importance of learner motivation assumes that learners understand the criteria on which they are
assessed assumes that learners receive constructive guidance about how
to improve develops learners’ capacity for self assessment so that they
become reflective and self managing recognises the full range of achievements of all learners
Five key strategies …
Sharing LearningExpectations
Questioning
Clarifying and sharing learning intentions and criteria for success
Engineering effective discussion, questions, and learning tasks
Feedback Moving learners forward with feedback
Self AssessmentActivating learners as the owners of their own learning
Peer AssessmentActivating learners as instructional resources for one another
…and one big idea.
Use evidence about learning to adapt instruction to meet learner needs.
Find x.
3 c
ms
4 cms
x
So, how do you do it?
The procedure is actually quite simple. First you arrange things into different groups. Of course, one pile may be sufficient, depending on how much there is to do.If you have to go somewhere else due to lack of facilities, that is the next step.Otherwise you are pretty well set. It is important not to overdo things – that is, it is better to do too few things at once than too many.In the short run, this might not seem important, but complications can easily arise. A mistake can be expensive as well.After the procedure is completed, one arranges the materials into different groups again. Then they can be put into their appropriate places.Eventually they’ll be used once more and the whole cycle will have to berepeated. However, that is a part of life.So, how do you do it?
Washing clothes
What do these words in the text refer to?
Different groups Somewhere else Facilities Complications Mistake Appropriate places
Key question
What do Pythagoras and clothes washing have to tell us?
Skills of VET practitioners
Provide learners with constructive guidance about how to improve. This may involve:
Ensuring that learners know what competent performance is;
Pinpointing learners’ strengths and advising on how to develop them;
Being clear and constructive about any weaknesses and how they might be addressed; and
Providing opportunities for learners to improve upon their work.
Develop learners' capacity for self-assessment so that become reflective and self-managing learners.
Obtain information about learner progress and use it in planning at individual, group and program levels.
Design and implement learning activities that promote high quality interactions with learners that feature thoughtful questions, careful listening and reflective responses.
Plan assessment, observe learning, analyse and interpret evidence of learning, give feedback to learners and support learners in self-assessment.
Negotiate and manage flexible assessment options including self assessment, peer assessment and collaborative assessment.
Ensure that learners understand the criteria on which they are assessed so that they know what it is they are trying to achieve.
Activity – Rate VET Practitioner skills
Powerful in promoting quality learning and assessment
Difficult to implement
Easy to implement
Not so powerful in promoting quality learning and assessment
1 2
3 4
Implementing assessment for learning VET practitioners ‘know’ most of this already So the problem is not a lack of knowledge The problem is how to do assessment for learning Telling VET practitioners what to do doesn’t work - there
is a need for action research to develop practitioners skills to implement assessment for learning
Experience alone is not enough - if it were, then the most experienced teachers would be the best teachers - we know that’s not true (Hanushek, 2005)
People need to reflect on their experiences in systematic ways that build their accessible knowledge base, learn from mistakes, etc. (Bransford, Brown & Cocking, 1999)
Further information
Publications:Inside the Black Box, Raising Standards Through Classroom Assessment (Paul Black and Dylan Wiliam, 1998)
Straitjacket or springboard for sustainable learning? The implications of formative assessment practices in vocational learning cultures in The Curriculum Journal, Volume 19, No. 2 June, 2008
Quality Assurance Agency for Higher Education, Integrative Assessment : Balancing assessment of and assessment for learning, QAA 2007
Formative Assessment in Action: Weaving the Elements Together (Shirley Clarke, 2005)
Websites:www.aaia.org.uk
www.assessmentfutures.com
www.qca.org.uk
www.slamnet.org.uk
http://arg.educ.cam.ac.uk
http://cms.curriculum.edu.au/assessment/
http://www.assessment-reform-group.org/
http://www.tki.org.nz/r/assessment/atol_online/tutorials_e.php