overview of helfelectronic management of assessment surveyresults 2013
DESCRIPTION
Overview of HeLFElectronic Management of Assessment SurveyResults 2013TRANSCRIPT
Overview of HeLFElectronic Management of
Assessment SurveyResults 2013
Dr Barbara Newland, Brighton
Lindsey Martin, Edge Hill
Alice Bird, Liverpool John Moores
Electronic Management of Assessment (EMA) includes a wide range of activities so the following definitions were used:
Definitions
eSubmission electronic submission of an assignment
eMarking electronic marking (including offline marking eg in Word)
eFeedback electronic feedback (ie text, audio, video but not hard copy)
eReturn electronic return of marks
To identify current practice with regard to Electronic Management of Assessment (EMA) in UK HE
To gain a snapshot of the strategic overview identifying key issues relating to strategic change, policies and practices
To reflect on longitudinal developments from findings of 2011 and 2012 surveys.
Aim
A network of senior staff in institutions engaged in promoting, supporting and developing technology enhanced learning
Over 138 nominated Heads from UK Higher Education institutions
A regular programme of well attended events
Represents the interests of its members to various national bodies and agencies including the Higher Education Academy and JISCwww.helf.ac.uk
Heads of eLearning Forum (HeLF)
The survey was available to HeLF members who were asked to respond with regard to their knowledge of their own institution.
The survey was available in March/April 2013 and took about 10 minutes to complete
The questions were a mixture of closed multiple-choice and multiple selection as well as open response type
Participants were assured that all data collected in the survey would be held anonymously and securely
No personal data was asked for or retained unless the participant indicated a willingness to participate in the follow-up activity
The results are being analysed using quantitative and qualitative methods
52 responses from HeLF members – 38% response rate
Methodology
More positive attitudes towards EMA and its normalising within their institutions
Challenges in relation to buy-in, take up and roll-out processes, functionality, service disruption and standardisation and whether the latter is desirable and achievable.
Main findings
Institution-wide policy
eSubmission eMarking eFeedback0
5
10
15
20
25
30
35
40
45
50
YesNo
Closer to having an institution-wider policy than a year ago?
eSubmission eMarking eFeedback0
5
10
15
20
25
30
YesNoDon't know
Is the move to eFeedback leading to more standardisation in terms of the feedback provided?
University-wide
Department-wide
Neither
Which software does your institution recommend for eFeedback in text format?
Turnitin (stand-alone)Turnitin (integrated into VLE)VLEHome grownOther
Current practice
05
10152025303540
University-wideSome department-wideIndividual academics
Is there evidence of increased use of mobile devices?
eSubmission eMarking eFeedback0
2
4
6
8
10
12
14
16
Academic staffStudents
Have you experienced critical issues with the eSubmission, eMarking or eFeedback software over the past year?
YesNoDon't know
Technical issues◦ service interruptions and outages◦ limited functionality and bugs with integration tools
Institutional circumstances◦ lack of university standardisation◦ lack of institutional policy◦ lack of consistency across Schools which meant that
the existing technology could not meet all the requirements of Schools
Skills - staff skills gap
Critical issues
eSubmission is a high stakes activity
Concerns about the service being hosted by a third party and also with quality standards
Other challenges:◦ unsatisfactory problem solving◦ unannounced upgrades and maintenance downtimes
during assignment submission periods◦ service not being able to cope with large numbers of
students◦ lost assignments although these were eventually
recovered
Technical concerns
Where Turnitin unavailability coincided with submission deadlines, one institutional representative reported that they had to issue advice to academics to extend deadlines.
For another institution individual staff made their own contingencies, for example, paper submission and paper feedback.
“Caused a lot of stress!”
Solutions?
Does your institution have a fully integrated EMA approach?
YesNoUnder consideration
Does your EMA system cope with the following marking processes?
Anon
ymou
s m
arking
Double
mar
king
Exte
rnal e
xam
iner
s0
5
10
15
20
25
30
YesNoDon't know
How would you rate attitudes to eSubmission?
Positive Negative Neutral Don't know0
5
10
15
20
25
30
35
40
45
50
Academic staffAdministrative staffStudents
How would you rate attitudes to eFeedback?
Positive Negative Neutral Don't know0
5
10
15
20
25
30
35
40
45
Academic staffAdministrative staffStudents
How would you rate academic staff attitudes to eMarking?
PositiveNegativeNeutralDon't know
‘There has been a change in attitude towards eFeedback, with a number of members of staff recognising that they already do this in some form.’
It is regarded as an urgent sector-wide agenda and a common student expectation, and for many institutions it is increasingly becoming embedded in departmental practice
Changing attitudes
Students
Staff
Senior management
National agendas
‘Student use. Getting positive stories from staff and student users.’
‘Positive student feedback through module evaluations and staff, student consultative committees.
Drivers
Consultation with all stakeholders
Leadership by senior management - buy-in and support
Effective communication strategy - visible campaigns,
online awareness-raising and support resources
Departmental champions
Digital literacy - the provision of training
Support from central/departmental learning technologists
Technical – robust, easy to use systems integrated with
other central systems.
Critical success factors
While mandatory policy has been a key driver in some institutions, a number of respondents suggest that inclusive policy and flexible processes work well
‘It depends very much on the culture of your organisation. Here a partnership model between central services, and School / faculty admin and academics tends to work. Top down imposition of systems is less successful. In other institutional cultures this may be different.’
Institutional differences
‘e-Submission is quite straight forward. However, we could have managed the process of e-marking and feedback better. Academic staff need a lot of time to come round to the idea if they are changing years of established practice.’
‘Never underestimate the effort involved with winning hearts and minds of colleagues.’
Challenges and concerns
Assessment is mission critical – it determines whether students achieve, progress and gain awards (and at which level)
All stakeholders in the assessment process are wary of making errors and need to be convinced of the merits in changing long-established practices
Change in practice supported by technology is only worthwhile if it effectively enhances stakeholders’ experiences and/or improves the overall process workflow
Change management - EMA
‘There is a general acceptance of e-submission policy, with fewer complaints, and anecdotal evidence points to greater academic participation in emarking and efeedback’.
Usage has moved from individual early adopters to more widespread, formalised use. There is an increased appetite for standardisation across faculties. However, a very small proportion of respondents report no, little or slow moving change in attitudes.
Summary
Dr Barbara Newland [email protected]
Lindsey [email protected]
Alice [email protected]
Acknowledgement: Dr Rachel Masika, University of Brighton
Contact details