glancing back, looking forwards: learning from an institutional approach to change in assessment...
TRANSCRIPT
Glancing back, looking forwards:Learning from an institutional approach to change in assessment practice and culture
Helen Parkin and Graham HoldenQuality Enhancement and Student SuccessStudent and Learning Services
Session Outline
• Overview of the Assessment for Learning Initiative at Sheffield Hallam University
• Discuss the Change Model adopted by the Initiative
• Outline deliberate steps taken by the Initiative
• Describe the approach taken to evaluate the initiative
• Share the initial headlines from the evaluation
HEA 201022nd June2010
The Assessment for Learning Initiative• TALI was established in 2006 focusing on the deliberate
actions required to support the development of::
• Assessment practices that are learner focused and promote student engagement and attainment.
• Regulations that are clear, consistent and student centred.
• Assessment processes that are efficient and effective and which enable the delivery of a high quality experience for staff and students.
HEA 201022nd June2010
A Joined Up Approach
Regulations and
Frameworks
Academic
Practice
Process and Systems
HEA 201022nd June2010
Key features of the approach to change• Acknowledge that changing assessment practice is complex and time
consuming. • Responsibility for changing the assessment culture and practices lies with
different groups of stakeholders • Advantages in promoting institutional change through key personnel at
ground level, (Newton, 2003). • Importance of emergent change and the need for connectivity across the
institution (Seel, 2005). • Model for an integrated approach to assessment (adapted from Joughin
and Macdonald, 2009) providing a clear framework within which the complex nature of assessment could be understood enabling the formulation of a quality enhancement strategy to move the University towards assessment for learning.
HEA 201022nd June2010
Deliberate StepsStudent: Where assessment matters: student experience of assessment, student involvement in assessment processes …For example: Feedback for learning campaign in collaboration with students union and student representatives,
Module Level : Where assessment happens: module design, teachers’ experience of assessment….For example: Targeted resources (blue folder) and development opportunities (e.g. Blackboard Grade Centre) Course/programme Level : Supporting good practice: departmental culture, programme design, innovation and improvement, staff development….For example: staff development opportunities supported by Faculty Teaching Fellows acting as 'local authentic voices'
HEA 201022nd June2010
Deliberate StepsFaculty Level : the Faculty context: Faculty culture, strategies and action plans, resources, support, funding For example: Staff development opportunities and faculty funded projects and initiatives that align to University priorities.
Institutional Level: the institutional context for good practice: principles, policies and regulations, resources/funding, recognition and reward (for good teaching, including assessment) For example: Assessment and Feedback Policy and guidelines for the module and course design, Changes to module template and requirements re course approval
Wider Level: the external context: reflecting political, economic and social forces impacting on HE, as well as initiatives from within the HE sector itself.For example: HEA funded research project - Technology, Feedback Action!
HEA 201022nd June2010
Key Changes to Regulations
• Calculation of Degree ClassificationIntroduction of an element of compensation at final level (taking best 100 at level 5 and best 100 at level 6 / or just best 100 at level 6)
• Passing a ModuleStudents must achieve an overall mark of 40 or above to pass a module
• Limited Re-registrations
A student can re-register on a module on only one occasion
HEA 201022nd June2010
Key Changes to Regulations
• Credit LoadingStudents are permitted to study mixed level to retrieve failure. Student progression will be determined by the maximum student loading regulations
• Retention and Attainment
Where in-module retrieval is available, a student may choose to take the assessment again if they achieve below 40% in the assessment task
• Acknowledging SuccessIntroduction of Pass, Merit, Distinction for Foundation Degrees and Postgraduate Courses and Distinction for Ordinary Degrees
HEA 201022nd June2010
Evaluating an institutional approach to change in assessment practice and culture
To evaluate the staff and student perceptions of the impact of the University's Assessment for Learning Initiative
Analyse the impact on key performance indicators such as:Good degree statisticsFirst time pass ratesStudent retention
HEA 201022nd June2010
Key Features of Implementation
• Academic Practice– Teaching Fellows
• Regulatory Change– Faculty Implementation Teams
• Systems and Processes– Cross disciplinary working
All initiated through two-day residentialfocussed on approach to change
HEA 201022nd June2010
Methodology
• An Appreciative Inquiry Approach
• Data mining exercise to explore existing data sets
• Workshop attended by 30 members of academic, administrative and managerial staff
• 40 semi-structured interviews
HEA 201022nd June2010
Academic Practice• Raised awareness of assessment for learning and feedback
across the University• Encouraged staff to 'rethink' assessment strategies and
provided an opportunity for module redesign• Opportunities to share best practice well received• Evidence that assessment now features in 'peer review' across
the University• Inconsistencies in practice remain• Implications for workload (or shift) particularly where in-
module retrieval has been adopted
HEA 201022nd June2010
Teaching Fellows
What worked well• 'Faculty face' rather than
'central body'• Seen as expert in
assessment matters• Engagement at subject
group level• Commitment and passion
for assessment and improving the student experience
Lessons Learned• Tensions between faculty
need and central need• Need for clearly defined
roles with expectations and accountability
• Need more time• Who makes the decisions?
HEA 201022nd June2010
Regulatory Change
• New regulations 'celebrate student success'– removal of 'hurdles'
• Acted as stimulus to encourage thought around assessment strategies
• Professional Body requirements added complexity• Implementation was inconsistent at faculty level with possible
consequences for the student experience of assessment• Greatest impact where implementation is consistent and
widespread
HEA 201022nd June2010
Faculty Implementation Teams
What worked well• Consisted of appropriate
people to make decisions for the faculty
• Every module though revalidation or minor modification process
• Maintained a dialogue around assessment
• Ongoing commitment to assessment
Lessons Learned• Variations between
faculties in how teams worked
• Need to achieve consistency at course level
• Tracking of data is different for each faculty and often difficult to retrieve if outside the faculty
HEA 201022nd June2010
Systems and Processes• Highly flexible regulations and practice require sophisticated
and complex bespoke developments of corporate systems• A number of 'work-arounds' have been developed at Faculty
level– Double entry of grades has implications for workload and the
accuracy of data– The use of Blackboard Grade Centre as an 'administrative tool'
detracts from its pedagogic benefits• System is perceived as dictating practice• Work ongoing through 'Improving Systems and Processes'
Project
HEA 201022nd June2010
Key Performance IndicatorsGood Degree Statistics
Overall remained fairly consistent but students awarded a First has risen from 8.2% (2006/07) to 12% (2008/09) and those awarded a 2:1 has risen from 48% to 49.1%
First-time pass ratesDifficulties with obtaining accurate data but the number of
modules 'passed' has been sustained despite growth and limited re-registrations
Evidence of improved pass rate in modules that adopted 'Model A'
Significant improvements in the faculty where adoption was widespread and consistent
Retention and ProgressionWithdrawals down from 9.1% to 8.5%
HEA 201022nd June2010
Future PrioritiesPractice• Authentic assessment - preparing our students for an uncertain future (tie
in with graduate attributes)• Technology enhanced learning - e-PDPPolicy / Regulations• Student entitlement (feedback on all assessed work within 4 weeks;
feedback on exams)• Refresh regulations and guidance / update ancillary regulations
(extenuation etc)Process and Systems• Move towards all modules to providing access to grades online• University wide system for the management of module marks
HEA 201022nd June2010
ReferencesHEFCE (2009) 'National Student Survey'. http://www.unistats.com/
[last accessed 18 January 2010].
Macdonald, R. & Joughin, G. (2009) 'Changing Assessment in Higher Education: A model in support of institution wide improvement' In Joughin, G. (ed.) Assessment, Learning and Judgement in Higher Education. Springer. pp.193-214.
Newton, J. (2003) 'Implementing an institution-wide learning and teaching strategy: lessons in managing change' Studies in Higher Education. 28 (4) pp.427–441.
Seel, R. (2000) 'Culture and Complexity: New insights on Organisational Change' Culture and Complexity - Organisation and People. 7 (2) pp.2-9.
HEA 201022nd June2010